A bug in PBR levels less than 0.5.10 causes the
/usr/lib/python2.X/site-packages/cinder/db/sqlalchemy/migrate_repo/migrate.cfg
file to not be properly installed. This causes 'cinder-manage db sync'
to fail due to the missing file.
This change sets the required PBR level to >=0.5.10,<0.6 to avoid the problem.
In the case that user already has a 0.5.X level of PBR installed that is less
than 0.5.10 it should be updated rather than encountering issues as our
test system did.
Change-Id: I532158277a4acbb2f8cb75d03c91097e6d7e0312
fixes: bug 1182203
The update of log was mostly to get coloring to work again
in devstack, and jsonutils was picked up as well.
The update to jsonutils also requires the six dependency
which is added to pip-requires.
Change-Id: I95d23aafe19b12d4a427149aa5f7a66394d178c6
lintstack is a pylint-based static analysis tool to find bugs
in Python code.
To run the test via tox: $ tox -e pylint
To run the test directly: $ tools/lintstack.sh
Change-Id: If155f47ed84eb820ac505ee741bdd18edd41b1eb
Tests in oslo-incubator now need the horrendous hack to workaround an
issue with eventlet on RHEL6. We've moved the patch_tox_venv tool
and redhat-eventlet.patch into oslo-incubator, so add them to
openstack-common.conf.
Change-Id: I1acfec282b64d7582144b155119bedb1741db2f7
Remove setuptools-git as a run time dependency since
it is only used to generate a tarball.
Change-Id: I6c48147408e97ab88ede660938e783f6b660746a
Signed-off-by: Chuck Short <chuck.short@canonical.com>
This has been deprecated in favour of git-review for quite a while now
and was removed from Nova 10 months ago.
Change-Id: Ic4bfa1aee3a4b6a7588be3678ac033a1b13e2b18
This patch updates all the files from oslo under
stable/grizzly but policy.py. policy.py is a high risk
change at this date and it has lots of changes since the
last time we updated it.
Fixes bug #1157126
Change-Id: I399a1cd8474f718ed5196def90d2fea546fb01f5
This patch is an update of pip-requires and test-requires
using the openstack/requirements script.
One exception was made to pep8, which is being fixed in the
upstream script.
Change-Id: I4bd61e0ec6f1995f9f36a724eb8dbe15345caca0
Fixes bug #1128256
oslo.config has now been released to PyPI in time for Grizzly RC1 so
we can switch to using it directly.
Change-Id: I655f831718ae5f4e25e941ee206fe195214a9a91
In order to support running unit tests on RHEL 6.x we need to patch
eventlet with contrib/redhat-eventlet.patch. We already
have support for this in the tools/install_venv_common.py but we
need to make a couple changes to allow tox to consume this:
1) Sync in the latest intall_venv_common.py from oslo. This changes
patch to use the -N option (ignore already applied patches) and
makes it safe to call the patching function more than once.
2) Add a new patch_tox_venv.py script in tools.
3) Update tox.ini to call patch_tox_venv.py before it runs tests and
coverage.
NOTE: This will hopefully go away eventually once this patch lands:
https://bitbucket.org/eventlet/eventlet/issue/89/add-a-timeout-argument-to-subprocesspopen#comment-3342969
Change-Id: I3e2b07c3f718e4aede5c5f231ff0cdb7721ec885
old install_venv_common used oslo-config, which makes for bad
recursive dependencies, and we can't build a venv from a fresh
tree.
Change-Id: I09336d760842d4eb57f2b70fdfe19afaf6520a48
Fixes bug 1130706
Related to blueprint xenapinfs-glance-integration
This patch enables users to use any images recognised by qemu-img to
create volumes on XenAPINFS.
Change-Id: I9dd8ec237309da82ec7f73db1acb48e5b7411c9e
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.
Add the 2013.1b3 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.
Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.
Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.
Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).
Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
It doesnt make sense to put rtslib in tools/test-requires
since it is not directly using the python module while running
the tests, since the tests use cinder-rtstool.
Change-Id: Ib39b91e25f22b4943ef17eee50967828f7460b25
Signed-off-by: Chuck Short <chuck.short@canonical.com>
This patch enables LIO as an iSCSI backend for Cinder, using
python-rtslib.
To enable, set "iscsi_helper = lioadm" in cinder.conf.
This requires python-rtslib 2.1.fb27, which is available from pip.
Implements blueprint lio-iscsi-support
DocImpact
Change-Id: Ifb23de65f26a40997afd6148a1d0be39bcc8d196
sync _reset_databases from nova, which is needed for working
postgresql. Add opportunistic testing for postgresql, which will
run in CI.
Change-Id: I8166c2f723cabc5c5c07c7babaec8ffed5cee650
(Copy latest update from oslo)
Not doing this raised exceptions when it tried importing modules
that didn't exist in the path.
Fixes LP# 1112484
Change-Id: I2be7bde0bf873a22318617b987a76b209caef22a
New cfg module abandons previous disable_interspersed_args() and
recommand to use add_subparsers from argparser module instead.
This patch pull cfg module from Oslo and update the affected
cinder-manage utils.
Change-Id: I8913fafb8fdb19b3fe0a695a70c8b1e8f59c1027
This syncs install_venv_common.py from oslo and reworks the
tools/install_venv.py script to use the new library.
Change-Id: Ia271de047cc80e462fa0850cd8d626d2da777dff
Based on recent changes in Nova, this provides a cleaner/simpler looking config file:
- Move help message before each option, rather than after.
- Print types like "string value" instead of "(StrOpt)"
- Use fewer pound symbols
Switch to a new format that looks much more like a normal config file.
Change-Id: I69f9b7b84215a2a69bdb0c4af9d5e1cb1e105a45
The current 3rd-party module requirements for cinder are still from the
nova git import. Usually, having an upper bound for requirements makes
only sense where API-incompatible changes are expected (like WebOb). For
the others it is better the only require a minimum version because
usually distributions differ on the shipped versions of these Python
modules. For instance, openSUSE has newer versions for almost all of
those and running the testsuite there works flawlessly.
Change-Id: I4614a2ea87084f927a557b9fb85dcd10ae9dfeff
In order to do more sophisticated scheduling (e.g. schedule based on volume
type), filter scheduler is introduced. Several changes are made to make this
possible, some of them are similar to the counterpart in Nova:
- add HostState class to host_manager in order to store volume capabilities
- implement get_volume_stats() method of iSCSIDriver as an example to
demonstrate how volume backend driver reports capabilities as well as status
- add scheduler_options.py and 'scheduler_json_config_location' flag to be
allow loading json configuration file for scheduler at run time
- port common filters/weights from oslo
- add capacity weigher (scheduler/weights/capacity.py) for picking up
target volume backend by weighing free capacity of the host. The default
behavior is to spread volumes across hosts; by changing the
'capacity_weight_multiplier' to negative number, volume placing behavior will
become stacking.
- add capacity filter which filters those hosts have insufficient storage space
to serve the request.
- add 'reserved_percentage' config option to indicate how much space is
reserved. Capacity reservation is needed when volume resize is enabled.
- add 'publish_service_capabilities()' method to volume RPC API to allow
scheduler to have volume service report capabilities. This bumps volume RPC
API to version 1.2
- remove 'volume_force_update_capabilities' config option, volume status will be
report to scheduler in every checking.
The implication of this change to storage/backend driver developer:
- implementation of get_volume_stats() of the driver is now a *MUST*, filter
scheduler heavily relies on the status/capabilities reported by backend driver
to makeplacement decision. To ensure Cinder works seamlessly on the storage
system, driver should at least report following capabilities/status:
----------------------+---------------------------+---------------------------
Capability/Status | Description | Example
----------------------+---------------------------+---------------------------
'volume_backend_name'| back-end name, string | 'Example_Storage_Backend'
----------------------+---------------------------+---------------------------
'vendor_name' | vendor name, string | 'OpenStackCinder'
----------------------+---------------------------+---------------------------
'driver_version' | version, string | '1.0a'
----------------------+---------------------------+---------------------------
'storage_protocol' | supported protocols, | 'iSCSI', 'RBD', 'FC', 'NFS'
| string or list of strings | ['iSCSI', 'NFS', 'FC']
----------------------+---------------------------+---------------------------
'total_capacity_gb' | capacity in GB, integer | 102400
----------------------+---------------------------+---------------------------
'free_capacity_gb' | available capacity in GB, | 1000
| integer |
----------------------+---------------------------+---------------------------
'reserved_percentage' | reserved space in | 0, 10
| percentage, integer |
----------------------+---------------------------+---------------------------
The implication of this change to Cinder administrator:
- the default setting for filter scheduler should work well with the benefits
of:
* being able to fully utilize capacity of backends (driver now has to report
actul total space and space utilization and scheduler uses these info) not
limited by the 'max_gigabytes' config option any more;
* being able to choose placement policy between spreading & stacking for
volume creation (by modifying the 'capacity_weight_multiplier' in
CapacityWeigher)
- with filter scheduler, Cinder is now able to utilize the advanced features/
capabilities provided by different storage back-ends via: defining different
volume types with proper extra_specs. Volume types can be considered as sets
of back-end capabilities requirement.
For example, a volume type which has 'storage_protocol':'FC' key/value pair
definition in its extra_spec can only be served by those back-ends who report
they support FiberChannel protocol. Another example is volume type has 'QoS'
requirement can only be served by back-ends support QoS.
Note/TODO:
* Currently scheduler makes its decision based on the status and capabilities
information reported by volume nodes, and these information is stored in memory
of scheduler process. More sophisticated way may be add on table in DB to
record status/capabilities of all volume nodes, like Nova does for compute nodes.
implement bp volume-type-scheduler
DocImpact
Change-Id: I296b3727db8de0d4cf085fac602d122a7b474842
implements blueprint hp3par-volume-driver
We have the driver broken into 2 files:
hp_3par_common.py and
hp_3par_iscsi.py
The reason we do this is because we have a fibre channel driver
that will be submitted shortly after this is committed. The
fibre channel driver and the iscsi driver share a lot of the same
code that talks to the 3PAR array for provisioning. So,
it made sense not to have duplicate code. The fibre channel driver
will be dependent on the fibre channel support I am actively working
on for nova/cinder grizzly release.
The driver uses a 2 mechanisms to talk to the 3PAR array:
1) a python REST client (hp3parclient) that lives in the pypi
repository here:
http://pypi.python.org/pypi/hp3parclient
2) SSH. We had to pull in some of the ssh code from the base san
driver to help fix an issue with executing commands on the 3PAR
array. The 3PAR has the ability to turn on CSV output for command
results, which makes this easier to parse. Unfortunately, there
is no way to turn CSV mode on permanently for all ssh requests.
So, we have to turn on the CSV output for every single ssh command
issued. Since we use ssh as well, we require the san_* options
to be set.
We use a dual mechianism because the REST API that ships with the 3.1.2
firmware doesn't support all of the capabilities a cinder driver needs
to export volumes.
When a newer version of the firmware comes out that supports host
management on the 3PAR array, then we will get rid of the SSH code.
Change-Id: I9826ba1a36e27a9be05457ee9236a491dbfd0713
In order to do more sophisticated scheduling (e.g. schedule based on volume
type), filter scheduler is introduced. Several changes are made to make this
possible, some of them are similar to the counterpart in Nova:
- add HostState class to host_manager in order to store volume capabilities
- implement get_volume_stats() method of iSCSIDriver as an example to
demonstrate how volume backend driver reports capabilities as well as status
- add scheduler_options.py and 'scheduler_json_config_location' flag to be
allow loading json configuration file for scheduler at run time
- port common filters/weights from oslo
- add capacity weigher (scheduler/weights/capacity.py) for picking up
target volume backend by weighing free capacity of the host. The default
behavior is to spread volumes across hosts; by changing the
'capacity_weight_multiplier' to negative number, volume placing behavior will
become stacking.
- add capacity filter which filters those hosts have insufficient storage space
to serve the request.
- add 'reserved_percentage' config option to indicate how much space is
reserved. Capacity reservation is needed when volume resize is enabled.
- change default scheduler to 'filter.FilterScheduler'.
- add 'publish_service_capabilities()' method to volume RPC API to allow
scheduler to have volume service report capabilities. This bumps volume RPC
API to version 1.2
- remove 'volume_force_update_capabilities' config option, volume status will be
report to scheduler in every checking.
The implication of this change to storage/backend driver developer:
- implementation of get_volume_stats() of the driver is now a *MUST*, filter
scheduler heavily relies on the status/capabilities reported by backend driver
to makeplacement decision. To ensure Cinder works seamlessly on the storage
system, driver should at least report following capabilities/status:
----------------------+---------------------------+---------------------------
Capability/Status | Description | Example
----------------------+---------------------------+---------------------------
'volume_backend_name'| back-end name, string | 'Example_Storage_Backend'
----------------------+---------------------------+---------------------------
'vendor_name' | vendor name, string | 'OpenStackCinder'
----------------------+---------------------------+---------------------------
'driver_version' | version, string | '1.0a'
----------------------+---------------------------+---------------------------
'storage_protocol' | supported protocols, | 'iSCSI', 'RBD', 'FC', 'NFS'
| string or list of strings | ['iSCSI', 'NFS', 'FC']
----------------------+---------------------------+---------------------------
'total_capacity_gb' | capacity in GB, integer | 102400
----------------------+---------------------------+---------------------------
'free_capacity_gb' | available capacity in GB, | 1000
| integer |
----------------------+---------------------------+---------------------------
'reserved_percentage' | reserved space in | 0, 10
| percentage, integer |
----------------------+---------------------------+---------------------------
The implication of this change to Cinder administrator:
- the default setting for filter scheduler should work well with the benefits
of:
* being able to fully utilize capacity of backends (driver now has to report
actul total space and space utilization and scheduler uses these info) not
limited by the 'max_gigabytes' config option any more;
* being able to choose placement policy between spreading & stacking for
volume creation (by modifying the 'capacity_weight_multiplier' in
CapacityWeigher)
- with filter scheduler, Cinder is now able to utilize the advanced features/
capabilities provided by different storage back-ends via: defining different
volume types with proper extra_specs. Volume types can be considered as sets
of back-end capabilities requirement.
For example, a volume type which has 'storage_protocol':'FC' key/value pair
definition in its extra_spec can only be served by those back-ends who report
they support FiberChannel protocol. Another example is volume type has 'QoS'
requirement can only be served by back-ends support QoS.
Note/TODO:
* Currently scheduler makes its decision based on the status and capabilities
information reported by volume nodes, and these information is stored in memory
of scheduler process. More sophisticated way may be add on table in DB to
record status/capabilities of all volume nodes, like Nova does for compute nodes.
implement bp volume-type-scheduler
DocImpact
Change-Id: I8b5305bce3b24bcc85365037356a8f36d0107133
The version of WebOb being used in OpenStack was more than
1 year old. This change updates to the latest stable release.
Upgrading WebOb resolves a version conflict between OpenStack
and Pecan, the web framework used by the Ceilometer team for
version 2 of the ceilometer API.
Refer to http://docs.webob.org/en/latest/news.html
for the list of changes between 1.0.8 and 1.2.3.
bug 1092227
Change-Id: I44fdd11acbf1d5245e0c9c5a6199fd53200928ed
Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>
Change the WebOb version to >=1.0.8 as an temporary
measure to allow the actual version update to 1.2.3
to roll out across the projects one at a time without
breaking the integration tests.
Change-Id: I9969b02a6896f7b952301e40e96f0ef6df8f34c0
Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>
auth_token middleware has been move from keystone to python-keystonecient,
so add python-keystoneclient dependence instead of keystone.Also adds
missing signing_dir option.
Change-Id: I24a4f422c35b721338c7519212c21ed2fbc255b3
Along with moving to pep8 1.3.3, we also want to standardize
on what we ignore. This patch get's us most of the way there
by setting the ignore list to:
N4,E125, E126, E711,E712.
Almost all changes made here are white-space/indentation changes.
The removal of Hacking N4 errors from the ignore list will
be handled in a seperate patch.
Change-Id: If45f156600485d23769449018590f60b4f69b0c5
Ubuntu is using a much newer version of lxml so
we shouldn't lock the versions in our infrasctructure.
Change-Id: I73589af70e4156968f14d2494b3af36fa8f4794e
Signed-off-by: Chuck Short <chuck.short@canonical.com>
