Python 3.5 was the target runtime for the Rocky release. The current
target py3 runtime for Stein is Python 3.6, so there is no reason to
keep testing against the older version.
https://governance.openstack.org/tc/reference/runtimes/stein.html#python-runtime-for-stein
Change-Id: If810570859e4c4a942f44097717170d5a0affbd0
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This cleans up the cases where we had D001 violations so we can stop
skipping that check in doc8 runs.
Change-Id: Ie52f6ecac1a645fcbcc643b9ca63e033b622d830
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
We use *.inc files extensively, especially in the api-ref, but these
files are not covered by default with our doc8 validation.
This adds support for those files and fixes a few issues that were
identified after enabling coverage.
Change-Id: I807823f3cb44274c3940db55e901d434304a09f5
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Determines how many commits to check based
on the $FAST8_NUM_COMMITS env variable.
If set to "smart", it uses git to try to run
against all unsubmitted commits. This allows
fast8 to be more useful when actively developing
a series of patches.
Change-Id: I0adfba626ba2ea3479faf0f6ade712f14d3080d3
This patch adds the following functionality:
1) Base framework for test cases
2) Volume type tests
Change-Id: I0e51bcc8c634923ca6fcff99a9f40b786933caa5
Partially-Implements: blueprint generating-sample-files
The actual target of openstack-tox-lower-constraints is using the lower
version package to run unit test.
But in current Cinder CI,there was a hard code to specify the
upper-constraints.txt, that means we install the package in
upper-constraints.txt, but the lower-constraints.txt doesn't work well.
This patch try to fix this error, move the hard code to 'deps'.
We also refresh the lower-constraints.txt and requirements.txt to make
CI works well:
1. Add libxml2-dev, libxslt-devel, libxslt1-dev to bindep.txt to make
lxml can be built successfully, like what we do in other project. [0]
2. Upgrade oslo.messaging to 6.4.0 to avoid "ACCESS_REFUSED
- Login was refused using authentication mechanism AMQPLAIN." error.
3. Upgrade oslo.utils to 3.34.0 to add a new method escape_ipv6 [1] to
oslo_utils.netutils, which is needed by [2].
4. Upgrade keystoneauth1 to 3.7.0 to fix "no such option split_loggers
in group [service_user]" [3]
[0] https://github.com/openstack/nova/blob/bb3ded3/bindep.txt#L17-L19
[1] https://review.openstack.org/#/c/495166/
[2] https://github.com/openstack/cinder/blob/fe421fe/cinder/volume/drivers/netapp/utils.py#L172
[3] https://review.openstack.org/#/c/578008/
Change-Id: Ide84d49c283c0ad179a5cfdb0392d34ee73d97ea
Closes-bug: #1801239
Using "python setup.py check -r -s" method of checking the package has
been deprecated with the new recommendation to build the sdist and
wheel, then running "twine check" against the output.
Luckily, there is already a job that covers this that only runs when the
README, setup.py, or setup.cfg files change, making running this in the
pep8 job redundant. This covered by the test-release-openstack-python3
that is defined in the publish-to-pypi-python3 template.
More details can be found in this mailing list post:
http://lists.openstack.org/pipermail/openstack-dev/2018-October/136136.html
Change-Id: Ia981b27852b9061c8a406e68388c60595aa528f8
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Quotes around {posargs} cause the entire string to be combined into one
arg that gets passed to stestr. This prevents passing multiple args
(e.g. '--concurrency=16 some-regex')
Change-Id: I8ce3c403904ea1c6990f88d52c1c1a068aaed75a
This commit does several things:
- Setup and run pylint directly rather than running
through a script. This allows the user to see what is happening
while the user is running through pylint.
- Allow the user to either run pylint on a particular changeset,
or the entire cinder tree.
- Allow the user to run on a particular changeset. Using like
HEAD~1, etc.
- Since the pylint gate check I disabled the tests that were
reported by pylint. The thought here would be go through
the failures and correct them.
- Update pylint to 2.1.1.
- Removed old pylintrc.
Change-Id: I708c93843d991e8c0ac114b0b2d6ede373b96d0a
Signed-off-by: Chuck Short <chucks@redhat.com>
This adds a tox setting for skipping default targets if the
specified version is not available. This allows tests to run
using python 3.6 if available, but conversely also allows
skipping python 3.5 or even 2.7 if those runtimes are not
present.
This only affects local default tox runs and does not change
what is tested in the gate.
Change-Id: Ia34fd32ab8d4672a2725468ee3d37697f51d5b3a
We want to default all jobs to use python 3 except for
the ones that use explicit versions. Unfortunately, the
current tox behavior for setting basepython to python3
is not intuitive.
If the system default is python 3.5, then when basepython
is set to python3, both the py35 and py36 targets will
end up running with python 3.5.
To get around this we need to not set a default base
version and explicitly set each testenv to python3 that
we want to run using 3 instead of 2 but do not really
care which minor version is used.
Change-Id: I3971eeafbd1e732a584b0c52d2eb6c2175b709aa
This switches the pylint target to run under python3. In
order to work right, it also raises the version of pylint
used to a newer version.
Related-bug: #1761800
Change-Id: I0bc68a5aee24a53b9d3a08c20549b89d08989678
The long term goal for OpenStack is to support python3 for
all things except explicit python2 testing by the T release.
Part of preparing for that is ensuring all ancillary things
like docs jobs and other tooling work with python3.
This switches the default for our docs jobs to python3 so
anything that does not specify another version will end up
using it.
Further work is necessary for the pylint job due to changes
between the runtimes. That will be done in a follow up patch.
There was also a difference in behavior with the genopts job
where it ends up trying to recreate the venv under which it
is currently running under, resulting in a corrupted venv
and a failure. This cleans up that script and changes it so
rather than a tox job calling a tox job it just runs the
script.
Change-Id: Ie79528b33070e6c60f9f5e1f2219084a2f0cb039
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Change-Id: I329e6364d537f2acac796e389f15f3c7162a9244
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
There are slight differences with pep8 when run under python 3.
It is a little more strict on certain things, plus we want to
get to just supporting python 3 eventually, so this switches
over the pep8 and fast8 targets to use python 3.
Change-Id: I3212bfb2dd173e79dae6d548827a803f68923da9
Now that sphinxcontrib.apidoc is available, we should get rid of
the custom code we have for generating this information ourselves.
This also cleans up some really stale docs that are no longer
needed.
Change-Id: Iaa9fecc2478326b45b67a2cfd98de5f93f537efe
This fixes some formatting issues in docstrings to allow us
to enable the warnings-as-errors flag for our docs build.
Co-authored by: Jay S. Bryant <jungleboyj@electronicjungle.net>
Change-Id: I9567e232686d8775ff67418e8333c164f8acd8f3
If you launch py35 and py27 envs from the same
directory simultaneously, the find -delete commands
here will race against each other and cause a
failure.
Set the find option to ignore readdir races.
Change-Id: I4977ad687150b5a1d6f09a961fca4db5d768ef71
For "tox -e docs", run doc8 first.
doc8 runs much faster than sphinx builds and
will provide quick feedback on syntax issues.
Change-Id: Id4f77020a70acd8eb9df513d43f86d9fe78f7e22
With the switch to stestr we lost the automatic output of the slowest
tests. We can still get that output, but it is a separate command to
run after the tests. Add that to our tox.ini command for test execution.
Change-Id: I37937c1ebe4a10b896f1deb77f64b520ceba2830
Clear doc/build directory before running docs job may better guarantee
each building output the newest docs.
Change-Id: I17537742a4c1421f2bd0561d2c48c5ae56d89026
This patch adds documentation and sample
file for default policy in code feature.
Change-Id: I597971a29ec61a1bf8c991b2715ec7644b2e2692
Partial-Implements: blueprint policy-in-code
This change just updates the tox.ini file to call stestr directly
instead of going through ostestr.
For more background you can read the ML posting:
https://goo.gl/TFvcxy
So now you won't need `--n` to run single tests, and you can also
supply regex directly without even the `--`. For example:
`tox -epy27 $regex`
Change-Id: I75d4ca19f87fe1930cf54b554b7e232632a4a21c
os-testr has moved over to use stestr instead of testr. While this
is usually compaible with existing settings, there is a warning
that is emitted when .stestr.conf is not present. It is usually
able to fall back to parsing the .testr.conf file, but to be more
correct and to prevent future problems we should update the config.
Change-Id: I11e43cff87cad20b9c880e276e20a04123f830fb
This is the basic patch which consits of the framework
code for default policy in code feature as well as
support in attachment resource.
Change-Id: Ie3ff068e61ea8e0e8fff78deb732e183e036a10c
Partial-Implements: blueprint policy-in-code
Leftover data in the autogenerated docs can result in build
failure. Remove these files before every build to be safe.
Change-Id: Iec3ff7334b7af80cb34019590544fab0b1c95240
Closes-bug: #1716478
Doc builds can fail if you have .egg-info directories
in your repo. This change makes sure that those
directories are ignored in the doc8 check.
Change-Id: I8e713651919f43feb80fca59b74c103b3255dd75
Some of the available checks are disabled by default in flake8, like:
[H106] Don’t put vim configuration in source files
[H203] Use assertIs(Not)None to check for None
This patch is to enable the H106 and H203 checks in Cinder project.
The C312 hacking rule will be removed when turn on H203.
Change-Id: I2e883c301b64d5977bbb907b63c9c144bc6f959d
This avoids falling back to the global bindep-fallback.txt which
installs a pretty big environment. Without this change, a lot
of additional but unnecessary packages are installed on the nodepool
workers.
Change-Id: I8c581225eada929bc947b13f851bb228000f32fe
The v1 API is still in the tree, even if it is off by
default and deprecated. We need to keep documentation on
how it works at least until we remove the code. This was
published to [1] which was the primary way to find this
information, and it is now missing.
[1] https://developer.openstack.org/api-ref/block-storage/v1/
This marks v1 as deprecated in the toctree.
This reverts commit 8139f7f108.
Change-Id: Ic9d0fcce5ca0f8455c212292664b8f9694edfce3
Make use of doc8 to verify all rst files which are not
autogenerated for errors and fail if there are any issues
found. The doc8 checks are now part of the tox 'docs'
environment and ran automatically. Checks can also be called
direcly via 'tox -e docs'.
Fix all issues found by doc8.
Closes-Bug: #1566765
Change-Id: I2b25247030a1aadcfe029c9e071ef17f2f72046b
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
The gating on python 3.4 is restricted to <= Mitaka. This is due to
the change from Ubuntu Trusty to Xenial, where only python3.5 is
available. There is no need to continue to keep these settings.
Change-Id: Ief6f4b8b37113f08dd0ae767f3dd2b5b2177e3ce
Many deprecations are triggered early (on imports, for example)
before the warnings are enabled by the WarningsFixture in the
base test class.
To make sure all DeprecationWarning messages are emitted we enable
them via the PYTHONWARNINGS environment variable.
Note: this copy from 1be35ff039d67c50866f2556ef847b048cbf3578
Change-Id: I7c6106149832d048d4d3bd2701c2c0db81bd87cb
We get a warning "WARNING:test command found but not installed in testenv"
when building docs. This patch fix it.
Change-Id: Ib9798e59c55099d5a4bf8989401b63487b0c6338
Per our conversation in the Cinder weekly meeting [1] we will
now switch to just running local tests against python 3.5 as
that is the more recent and common 3.x version available on new
distros. Other versions can still be run explicitly, but this
changes the default tox run to only use 3.5.
[1] http://eavesdrop.openstack.org/meetings/cinder/2016/cinder.2016-11-30-16.00.log.html#l-54
Change-Id: If2226cb229f016e36c4dfab6224d5ae353b926bb
Looks like we should now be able to add oslo.versionedobjects[fixtures]
to test-requirements.txt and remove it from tox.ini, so this commit does
that.
Change-Id: I420ea16d8ad6d883e7c2c8de80f912ea35500367
Remove useage of old os-api-ref version and replace with current version.
Follow other repositories and merge v1 and v2 in a single document.
Note that we need to have a single document so that preview of the
document works properly in our CI.
Change-Id: I82c8b9ca317298b3dc1f7133ce8c0e9b3503730d
Two years ago the translation files have been split into several
files, separating the log messages of different log levels from each
other, like X.pot, X-log-warning.pot, X-log-info.pot, and so on.
However, the setup.py command `compile_catalogs`, that comes from the
babel package and compiles the corresponding .po files into .mo
files, only supported one file per python package. This means that
during packaging `compile_catalogs` never compiled the X-log-*.po
files, so the corresponding translations were always missing.
Since babel 2.3 the domain can be set to a space separated list of
domains. This change adds the the additional log level files to the
domain list.
The obsolete check that .po and .pot files are valid is removed from
tox.ini.
Change-Id: I149c2254cb04297e598cfd3ca73b24efd0c8ef18
Closes-Bug: #1536226
Change [1] updated tox.ini to use constraints rather than
specific, non-constrained install commands but neglected
to update the api-ref environment. In addition to fixing
that omission we also remove a bit of ugly trailing
whitespace.
Co-authored-By: Gorka Eguileor <geguileo@redhat.com>
[1] I879f34129c0d958be0a73bfdc7f641e178d4efe9
Change-Id: I58ec756cd3de2e501c77a4d947e680d0959d22ec
Infra is now ready to use constraints everywhere [1], so adapt
releasenotes, coverage, and venv to use the default install command
instead of a specific non-constrained install command.
With this change, all tox environments now use constraints.
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html
Change-Id: I879f34129c0d958be0a73bfdc7f641e178d4efe9
Currently Sphinx will fail to generate the documentation if we are using
decorators from any OVO in cinder.objects, because the OVOs are only
added to the cinder.objects namespace when the CLI programs are run.
So we need to run `register_all` method during documentation generation
to avoid failures like:
AttributeError: 'module' object has no attribute 'Volume'
This patch modifies autogenerated doc/source/conf.py file and since now
it is no longer completely autogenerated it is added to PEP8 checks,
which required some minor changes.
Change-Id: Ifeeef61a778f9e3f3daceba8ed05cd2036219499
tox -e pip-missing-reqs was failing due to a
bug in pip_missing_reqs. The fixed package
has also been renamed to pip_check_reqs as of
version 2.0.
Change-Id: If91b979fedac8de5c611a87b900a2bcdb80d58b3
We have tox -e gendriverlist that outputs an RST-ish report of all drivers
in the tree. This output can be used in the docs build to automatically
publish the list of drivers to make it easier to find officially supported
drivers.
This effectively removes the existing drivers.html that was generated prior
that did not actually contain any useful information.
Change-Id: I8de78723af76aabcc976733ac4b248db0b8ca16f
The commands used by constraints need at least tox 2.0. Update to
reflect reality, which should help with local running of constraints
targets.
Note: this is a clone of mriedem's manila change [1].
[1] I27348462a21daab479d76b24b48e3a4e017504cc
Change-Id: I185c8f80285f2e7b4f3d3d45a2e6a7dd5741bc1d
When jumping between different review branches .pyc files of newly
introduced .py files are getting left behind. When these are related to
DB migrations all tests start to fail with: "AssertionError: There is no
script for XY version". This is pretty annoying, so why not remove all
*.pyc files before running any tests? This commit implements that.
Change-Id: Iac223fb156af3729b51fc2b11323a7cc30f9c07d
gate-cinder-tox-compliance is broken because run-tox.sh script fails
with ".tox/compliance/bin/pip: No such file or directory". This is
because we're reusing pep8 venv (envdir tox directive), so
.tox/compliance directory never gets created.
To fix hat we can simply stop reusing that venv and let compliance tox
job create it's own venv. It can be done by removing envdir directive
from tox.ini.
Change-Id: Ib809872f5fb1581e1b098775c2ebdc4e724c70e2
Closes-Bug: 1596508
This is the start of an effort to both validate that drivers fully
implement the expected minimum requirements as well as to create a clear
place for driver developers to learn what needs to be implemented and get
documentation explaining what is expected for each method.
This also enables us to create tooling for documenting the available
drivers and their capabilities, to some degree. A follow up patch will
show some of what I'm thinking there, but it will make it possible to write
scripts for different needs.
This is somewhat a cleanup attempt to the ABC work that was started a
while back. This does not aim to replace that effort, but give a
mechanism for some of the things expected out of that effort that ended
up not being possible with how it evolved.
In most cases we do not really care if a driver is inherited from a
certain base class, just that it conforms to the given interface.
The interface/inheritance work really centers around two separate
things:
* Ensuring drivers conform to an expected interface
* Allowing code reuse and common implementation
This is really for the first item. Additional work is needed to complete
the ABC work we've done, but that really focuses on the second item, and
is out of scope for the intent of this patch.
Change-Id: I4168225126fe88c31712d94f0a130e9e7ede3446
The oslo sphinxconfiggen module was added to the oslo.config
2.3.0 release. This enables config file generation as part of
the sphinx doc generation.
The generated config file will pick up the current config
options from the code base. And as an added bonus, it will
now be published to the docs.openstack.org site for easy
reference or download.
This also puts us inline with what other projects like Nova
are doing for sample config files and is the recommended
method from the Oslo team.
Change-Id: I912a97eb2686d3dc56e50d8641d7bd930179bc18
Running the api-ref environment currently warns that 'rm' is not
installed in testenv. This patch removes the warning by whitelisting
the rm command.
Change-Id: I4072491713a38b11d9f55b26a8ba0a3cf49445f0
Closes-Bug: #1581778
This patch is for converting API Reference to RST and host it
in the Cinder tree.
This patch contains all the RST for cinder to bring over to their
repos to begin building API reference information from within.
This contains .inc files which have all the contents of the .rst files
but are grouped together for easier editing
This is the results of the RST conversion from WADL. It creates a
single index plus a bunch of included files which represent sections
of the API.
Cleaning task will be done once this part is merged.
As we have removed XML API support in Newton release, we need to
remove XML from API reference as well.
But we still have installations which are using XML.
So implementation plan is to first merge this patch in Newton release
keeping XML for now, then I will backport same to stable branches.
After that I will remove same from Newton release to match API
reference with actual API implementation.
Implements: bp api-reference-to-rst
Change-Id: I865ac922538bfa5bd45c24eb4bc49f5e966dc811
The most recent version of bandit no longer requires a configuration
file (bandit.yaml) to run the default test suite. Removing the
configuration file and updating the bandit arguments will run all of
the currently available bandit tests in the default test suite.
Change-Id: If144c4017ce9bb58c3e83b62a5e31325353200b5
Now that all cinder unit tests pass on python 3.4 [1], we can run py34
tests by default alongside py27 and pep8.
This commit also addresses the annoyance of py34 tox tests failing with
'db type could not be determined' if py27 tests were initially run in
the workspace.
[1] https://haypo.github.io/openstack_mitaka_python3.html
Change-Id: If60e5d0d3185e78f38fa2bfc7b6bb4840f09d840
Closes-bug: #1489059
This flips the tox.ini to always use constraints installation for
all targets. It drops the extra -constraints targets in the process.
This makes it so there is no developer change required to operate in our
new world order.
Change-Id: I2b56835edb2d75b7c871aae030e909578dbb958d
Depends-On: Iecd35d6bc3b1e9e0ed31b48a34e4149d34d55ac2
* Remove tests-py3.txt: all unit tests now pass on Python 3
* tox.ini: remove specific command for py34 test environment, it now
inherits testenv commands (same command than Python 2.7)
Partial-Implements: blueprint cinder-python3
Change-Id: I52c68a63740b3ae6097fad419b59a9c29422b525
Functional unit tests should not be a part of unit
tests job. This patch adds new tox environment to run
functional tests:
$ tox -e functional
It also removes logging from functional tests. Some of log calls were
replaced with asserts.
Related Implements: blueprint cinder-integrated-tests
Change-Id: I0ebfef2fe05f502cd5fb08fbc8af09008949e457
Even though testenvs pep8 and pep8-constraints included cinder/common
directory to be processed by flake8, global configuration in flake8
secion was excluding common glob preventing the directory from being
checked.
This patch removes glob from flake8 exclude section, unnecessary
cinder/common inclusion when running flake8, and fixes
cinder/common/config.py.
If we had a common directory in our code root and wanted to exclude it,
glob should be ./common instead of just common like we had now.
Change-Id: Iefc64c02335ed9296bda323fdcb5688125fb4e77
This allows discovery of dependencies that are
not listed in [test-]requirements.txt. Taken from
oslo.db.
Change-Id: I66000bb61e433f8d75ed15d600f2e87ef45140b8
Following the upgrade to hacking version 0.10.0 the H105 issues were
ignored. There was a note left in tox.ini to clean up the H105 issues
and to re-enable the check.
Updated tox.ini to re-enable the check of H105 and fixed corresponding
pep8 issues. This way Cinder aligns rows with Nova, Neutron, Glance etc
that have H105 enabled.
Closes-Bug: 1531104
Change-Id: I2da70cbeb838f16ec8479284dda9d7c62566dace
Currently, "tox -e pep8" builds two tox envs:
a "pep8" env, and then a "venv" env for genopts.
This is rather wasteful -- reuse the pep8 env
instead.
Change-Id: I8ba3d4e541f279e5bc62dba2b1caaac447fee7e3
2f9e416 Downstream Fix for Genconfig
collided with
fb17ed8 Add -constraints for CI jobs
while merging and this only got added to
pep8-constraints. It should be in "pep8" as well.
opts.py is updated here since it is out of sync,
due to this test not running.
Change-Id: I668df6be4301a052a4b47549c702a81b43abd8fc
This patch adds the opts.py file to the tree as a fix for deployers
that package cinder. The opts.py file is no longer being deleted right
away by generate_sample.sh after the cinder.sample.conf is being
generated.
This patch also introduces a pep8 check to make sure that the opts.py
file is up to date, so that it will catch when new opts get added to
Cinder without the opts.py being updated.
To support the ability to keep and check the opts file a number
of changes were needed in the check_uptodate.sh script as well as
the generate_sample.sh script:
- check_uptodate now takes --checkopts instead of --checkonly
When checkopts is used the opts.py file is generated using the
current code and the generated file is compared to the existing
file. The check fails if there are differences.
- generate_sample now has the --nosamplefile option.
When this option is used, only the opts.py file is generated.
The oslo-config-generator code is skipped so no sample file
is created.
- generate_sample also has some coding style consistency changes.
- Added the 'genopts' option to tox so users can generate
a fresh opts.py without a sample file when necessary.
Closes-Bug: 1501820
Co-Author: Jay Bryant <jsbryant@us.ibm.com>
Change-Id: I1f5494ebb19d5f4e8c651cbeef0acad07ad96829
Use the ObjectVersionChecker fixture from oslo.versionedobjects to check
when Cinder's registered objects have a change (new remotable method,
new field, etc) that require a version bump.
Fixes some registration problems along the way:
1. CinderObject doesn't need to be registered.
2. TestObject should be registered where it's used, not globally.
Removes deps=requirements.txt from tox.ini since it's not necessary and
prevents us from pip installing oslo.versionedobjects[fixtures].
Closes-Bug: #1514926
Change-Id: Id57c56a75ea11411e9e54104165bd44b577c1485
Other projects have updated to 1.8 or later for the minimum
tox version (nova, neutron), so we should probably do the
same.
Change-Id: Iddb9ab286519d555671ec39c0ce75c9351025730
This adds the constraints factor to the base section and duplicates
the sections required for CI with -constraints as tox does not
currently support factors in sections. Work towards enabling factor
support in sections is currently stalled, as such we will need to
duplicate sections adding -constraints to facilitate running sections
with the constrained install_command.
Implements Blueprint: Requirements-Management
Change-Id: I7afd52517750b82dcac94b8aae68dc0e94d00e54
This reuses the .tox/pep8 environment rather than
building a separate .tox/fast8 environment for
fast8.
This reduces duplicated install time and disk space.
Change-Id: I230465b28ba3d0f2c3a2ba38011ff9f7d472d719
"tox -e py34" now uses ostestr instead of testtools.run to run tests.
Tests are now executed in parallel.
Move the tests whitelist from tox.ini to a new file tests-py3.txt.
Fix NaElement.__str__() of netapp driver: return Unicode on Python 3,
not bytes.
Partial-Implements: blueprint cinder-python3
Change-Id: I4cda85fbe76ac853beaec3a9587726166daa063d
Modify assertRaisesAndMessageMatches() to tolerate subclasses. On
Python 3, a permission error now raises an exception PermissionError
which is a subclass of OSError, whereas the test expected exactly the
OSError type.
Partial-Implements: blueprint cinder-python3
Change-Id: Ia232c27de6a67eafad30b25301531e292fe349dc
Fix query_cluster_vols_for_ssc(): cast string (records) to int. On
Python 3, comparison between str and int now raise a TypeError.
tox.ini: add cinder.tests.unit.test_netapp_ssc to Python 3.4.
Partial-Implements: blueprint cinder-python3
Change-Id: I05127340029a5bb6c8444f51341be8008aab8dc8
* Add close() method to FakeHttplibSocket,
FakeDirectCmodeHTTPConnection, FakeDirect7modeHTTPConnection
* Replace StringIO() with BytesIO() for socket buffers
* makefile(): ignore optional arguments after mode
* Fix bytes versus Unicode: use bytes for HTTP body. Encode HTTP body
to UTF-8 and HTTP headers to Latin1.
* tox.ini: add cinder.tests.unit.test_netapp to Python 3.4
Partial-Implements: blueprint cinder-python3
Change-Id: I3dc8c36a84ffbb47be3e85eed34518c6adad48ec
* Rewrite ReadOnlyDict using collections.Mapping: collections.Mapping
has no method to modify the dictionary and is available on Python 2
and Python 3. Add also a __repr__() method to ease debug. Remove
ReadOnlyDict.update() method: a read-only dictionary must not be
modifiable.
* test_host_manager.py: sort dictionaries using a key function to
have a reliable order for the list of dictionaries. On Python 3,
the hash function is now randomized.
* tox.ini. add cinder/tests/unit/scheduler/ tests to Python 3.4
Blueprint cinder-python3
Change-Id: I34d98d6a75fef907251719f0d46b025d8e8b2b65
BlkioCgroup._set_limits(): sort devices before iterating on them to
have a reliable behaviour.
The devs variable is a dictionary. On Python 3, the hash function is
now randomized, so iterating on a dictionary gives items in a random
order. Use sorted() to iterate on the list of sorted devices instead.
tox.ini: add cinder.tests.unit.test_volume_throttling to Python 3.
Blueprint cinder-python3
Partial-Bug: #1348818
Change-Id: Icf7141f772397c7ac08f0f1e21ad74cb86a06351
* don't use hasttr() to check if a lazy SQLAlchemy is loaded or not: use
the obj_attr_is_set() method instead. On Python 3, hasattr() pass
through SQLAlchemy exceptions which is unexpected.
* Replace a/b with a//b to use integer division, not float division.
* Replace filter() with list-comprehension using an if.
* Replace file() with open() and open /dev/null in binary mode (not text
mode, so Unicode on Python 3).
* test_volume require "import cinder.volume.targets.tgt" on Python 3
* Use a key function to sort a list of dictionaries. Dictionaries are no
more comparable on Python 3.
* tox.ini: add the following tests to Python 3.4.
- cinder.tests.unit.keymgr.test_mock_key_mgr
- cinder.tests.unit.test_volume
Note: test_list_availability_zones_enabled_service() of test_volume
was broken, the test checked that the result of .sort() is None...
Blueprint cinder-python3
Change-Id: If1a26acc0138db9bda7fde1cb1f40093d9b3c494
* Replace func.func_name with func.__name__: the func_name attribute
was removed in Python 3, whereas the __name__ attribute exists on
Python 2 and Python 3
* Create INITIATOR_TARGET_MAP using list(set()) to get the same order
than the tested code. The exact order is not reliable, it depends
on the hash function which is now randomized by default on
Python 3. The hash function is also different between Python 2.7
and 3.4.
* tox.ini: add test_pure to Python 3.4
Blueprint cinder-python3
Change-Id: I78601278259f1d34ad6ac3458d2dd0a3aca9d77a
This replicates the run_tests.sh -8 behavior, running
flake8 only on changes made in the last commit and
working tree.
Change-Id: Iebd5746c78e840225f0860843c77f4c638877c26
As it stands, the opts.py file that is passed into
oslo-config-generator isn't being generated dynamically
and the old way of generating the cinder.conf.sample is
dependent on oslo-incubator which Cinder is trying to
move away from. oslo-config-generator works differently
than oslo-incubator so a number of changes had to be made
in order to make this switch.
This patch adds the config directory to Cinder and in it
are two files:
-generate_cinder_opts.py that will take the
results of a grep command to create the opts.py
file to be passed into oslo-config-generator.
-cinder.conf which is the new configuration for
oslo-config-generator. The file is inside the config
directory to be consistent with other projects.
Some changes were made to the generate_sample.sh file in
order to give the base directories and target directories
to the generate_cinder_opts.py program.
tox.ini was edited to remove the checkonly option because
all that needs to happen in check_uptodate.sh is a check to
ensure that the cinder.conf.sample is actually being
generated with no issues.
All options were removed from the check_uptodate.sh
because they were unnecessary given the new, more simple
way of generating the cinder.conf.sample.
setup.cfg was also edited in order to add information
oslo-config-generator needs to run.
Co-Authored By: Jay Bryant <jsbryant@us.ibm.com>
Co-Authored By: Jacob Gregor <jgregor@us.ibm.com>
Change-Id: I643dbe5675ae9280e204f691781e617266f570d5
Closes-Bug: 1473768
Closes-Bug: 1437904
Closes-Bug: 1381563
Port cinder.image_utils.check_qemu_img_version() to Python 3.
Add the following tests to Python 3.4 in tox.ini:
* cinder.tests.unit.test_image_utils
* cinder.tests.unit.test_migrations
* cinder.tests.unit.test_misc
* cinder.tests.unit.test_service
Partially implements: blueprint cinder-python3
Change-Id: I69feda94c90206adc8771fbd5ddc6aae5bb6b64b
Encode Unicode to UTF-8 for salt and authentication key when computing
the crypt hash.
Partially implements: blueprint cinder-python3
Change-Id: I19d337cf0d40d91378e1c42061bc51b6009970a2
testtools.ExpectedException(exc_class, regex) expects the exact
exc_class class, wheras the
test_setup_should_throw_exception_if_mount_nfs_command_fails() test
raises a PermissionError on Python 3.3+ instead of OSError. Replace
testtools.ExpectedException() with self.assertRaisesRegex() to port the
test to Python 3, this method accepts subclasses of exc_class.
Replace also testtools.ExpectedException() with self.assertRaisesRegex()
in other tests.
Blueprint https://blueprints.launchpad.net/cinder/+spec/cinder-python3
Change-Id: If5a74529e5ac68a2118afa8ecfd86bf24307b0b7
Adds --checkonly to tools/config/check_uptodate.sh.
This allows verifying that a configuration file was
generated.
We have had a number of issues introduced which
have caused config file generation to fail since we
removed the sample config file check in the gate.
This does not add back checks to ensure that the file
has been updated, it only checks to ensure that it is
still possible to update the sample file.
Change-Id: I2e0e376fbdec5a1bb584c6739231649e1d0f5d8e
This reverts commit e67adc385c.
Adding optional dependency tracking would be nice,
but Cinder is not yet in a place to do this.
- Cinder supports tox 1.6 (minversion in tox.ini)
- This functionality does not work on 1.6, requires 1.7. Tox
fails to build a test environment if using tox 1.6.
- We have decided we can't move to tox 1.7 for now.
( https://review.openstack.org/#/c/211614/ )
This reverts commit 478e8e970e.
- Need to remove oslo.vmware from test-requirements.txt now, as
well.
Closes-Bug: #1484035
Change-Id: Icff52508ff1cd67e8c17964c840d3a5cdcfbbce3
Driver-specific requirements are not hard requirements, since the
choice of drivers is up to the operator. The oslo.vmware module is
herein moved out of requirements.txt and instead listed as an optional
dependency using the extras functionality in setup.cfg.
A check is added to the vmdk driver to gracefully handle import errors
if the vmdk driver is used and oslo.vmware is not found.
Change-Id: I9b00edc38f0700304a1a164f0679a734f8701ebe
Closes-Bug: #1475739
According to the PEP8(H405), multi-line docstring summaries
should be separated by an empty line.
Change-Id: I5cd8a9064dcefc504e85946ecdf1f56f10145d35
Closes-bug: #1407162
This change adds manage existing api task in flow. The task
is used in the volume api to provide full value task flow for
manage existing process. All errors occurred during manage
flow set volume to 'error' state.
Entry creating moved from volume api to EntryCreateTask. Also
added ManageCastTask to provide manage process to scheduler.
Related-Bug: #1364550
Change-Id: I12a4311953c1c86d584b5bf2fe2888e5b5127d43
This change adds a basic bandit config for Cinder. It can be invoked
by running the tox environment for bandit;
tox -e bandit
This is intended as a starting point for using bandit with Cinder
and it should be revisited to improve the testing as more is learned
about the specific needs of the Cinder code base.
Tox is configured to only show results for high and medium severity
results.
https://wiki.openstack.org/wiki/Security/Projects/Bandit
Change-Id: I0247e0ccaed6faacacb2b8d2f8b141a8edc704af
* Replace __builtin__ with six.moves.builtins.
* Replace tempfile.mkstemp() with tempfile.NamedTemporaryFile() to get a
text file instead of a binary file.
* tox.ini: add the following tests to Python 3.4
- cinder.tests.unit.test_hitachi_hnas_backend
- cinder.tests.unit.test_hitachi_hnas_iscsi
- cinder.tests.unit.test_hitachi_hnas_nfs
Blueprint cinder-python3
Change-Id: I9a26b4e67033a443271e8f13bcaea5e122ec865a
* When parsing share configuration: replace "\040" with a space
explicitly instead of using the Python unicode_escape encoding:
see the bug #1180984.
* Encode Unicode to UTF-8 before hash it with MD5.
* tox.ini: add cinder.tests.unit.test_glusterfs to Python 3.4
Blueprint cinder-python3
Change-Id: Iea943a1f2dfb050fe43ff41411e258adb3f3b6d0
It is useful to have a maintained (in source control) list
of drivers that exist in Cinder. It could be used in docs
and unit tests to check method impl on backend drivers.
This change add tool for generate list of drivers based on
existing BaseVD class in class hierarchy of volume drivers.
Output example:
Drivers: ['cinder.volume.drivers.lvm.LVMVolumeDriver',
'cinder.volume.drivers.rbd.RBDDriver',
...]
Implements: blueprint drivers-list-generator
Change-Id: I0e10906873e659e09a6e34531a0c932495d7c399
This patch attempts to refactor Huawei volume driver in liberty.
We add a base driver to implement the basic functions.
The sub-class will inherit from the base driver according to different
storages.
The following changes were made in this refactor:
1. Abstract a base class named HuaweiBaseDriver to make Huawei driver more
universal. You can find it in the huawei_driver.py.
2. Put all static variables into the constants.py.
3. Rename rest_common.py to rest_client.py. rest_client.py stores the
relevant methods implemented for Huawei driver.
4. Migrate some public methods from rest_client.py to huawei_utils.py,
such as parse_xml_file(), _get_volume_type() and so on.
5. This refactor only involves structural adjustment and does not involve
functional changes.
Change-Id: I768889e2577a4d975397218eb31e89b42e08e04f
Implements: blueprint refactor-huawei-volume-driver
It has been over a month since the CI maintainer was communicated via
third party mailing list, and the CI is still not reporting.
Change-Id: I25e125d76beb27da10a8ac617c70db357fcef57c
UpgradeImpact: OpenvStorage driver removed