The commands used by constraints need at least tox 2.0. Update to
reflect reality, which should help with local running of constraints
targets.
Change-Id: Ifc2c2ad88d77fb2590b5bb598458bdfd4a423a9e
Functional tests are executed by trovestack script, which uses python
in default OS version which is 2.7.
This change wraps python in tox and executes trovestack int_tests in
virtual enviroment, starting them in python3.
Any future python version change could be managed by tox framework.
Change-Id: I3a849978241d3d0669ef9e1c802ff504ac3c32cb
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I2b04e865fb72296f672ab206fcda15e30450db97
The job is currently broken, it runs with testr while
we switched to stestr. Use the usual setup for coverage as done by
other projects using stestr in cover.
Previously, we run with concurrency=1 - continue to run in serial mode.
Change-Id: I6e389db5aed8db25b49906b2479af081237fc909
This is a mechanically generated patch to switch the documentation
jobs to use the new PTI versions of the jobs as part of the
python3-first goal.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Also, add docs tox environment so that the job works.
Change-Id: I8dd911d46623c0185d238b293e0218cfa435ed46
Story: #2002586
Task: #24342
Having the pylint environment run redundant commands just makes the
job take longer.
Change-Id: Ibb86b40e589d1b6f9c3af5901549d7aa5170e8c8
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I9ee34642c700d1e6ba9c2f3891b7fa1f7f7e1e1d
Depends-On: I8989fd4798e80eae27408017e1543819a68b4ab1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
This patchset changes version of python-troveclient package version
used in tests.
So far, version from global constrain file was used [1].
In order to achive this, a local constrain file for tests requirements
has been added. This made it possible to override global constrain
versions.
[1] openstack/requirements/upper-constraints.txt
Change-Id: Ic4383f0249e5bd367461f63f66217af959aefba4
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
This patch migrates the legacy gate jobs to Zuul v3 native ones(a new
trove-devstack role is created for all those legecy jobs, and the
previous way of running gate jobs is still kept as we still cannot
migrate the genade job to a Zuul v3 native one), following the
documents below:
https://docs.openstack.org/infra/manual/zuulv3.htmlhttps://docs.openstack.org/devstack/latest/zuul_ci_jobs_migration.html
Change-Id: I10a1adae3a79b7322118fce180119f0ee5ff2e1b
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
We have migrated to stestr for a while, however the related requirements
is not updated, this patch does the cleanup, also updates requirements
for stestr.
Change-Id: I9c8cf2f90ced2261293f91faefc3099740540fe7
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
The low-constraints job should run unittests by the original desgin, we
didn't put the commands about running unittests inside the default
testenv settings, so in fact the low-constraints job did nothing except
creating a virtualenv.
This patch fixes and also move the unittests commands to the default
testenv settings like the other OpenStack projects.
Change-Id: Ia344a14a9baf7574b043fc08332869083da6dc1e
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Change-Id: I5352398603e5755d46c9c033b05af637e379eea0
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
* Tests for _write_file_as_root failed because tempfile.NamedTemporaryFile
use binary mode by default, changing to text mode fixes.
* Migrate to olso_serialization.base64 for Base64Codec. Also change the
return value from bytearray to bytes for Base64Codec.deserialize.
Base64Codec supports reverse encoding(i.e. binary data will be written
to the dest file), in this situation, the dest file should be opened
with binary mode(and when reading from the file, binary mode should
also be used).
* stream_codecs.StringConverter converts iterable objects with map
function. The behavior of map function is different under Python 2.x
and 3.x. However csv.writerows(before Python 3.5) and unpack_singleton in
trove.common.utils both need a list object. Converting the return
value of map function to a list explicitly before passing to these
functions fixes.
guestagent.test_operating_system is the last blacklist regex pattern for
py3 unittests. With the above problems fixed, blacklist-py3.txt is not
needed any more, and tox.ini is also updateted.
Migrating from ostestr to stestr is also done while updating tox.ini.
Here is the ML post for more information about os-testr and stestr:
http://lists.openstack.org/pipermail/openstack-dev/2017-September/122135.html
Partially implements: blueprint trove-python3
Change-Id: I31f1f97901d6ebff8a91c1b70a343e724ab806eb
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
A new entrypoint in setup.cfg and a config file are added for
using olso.policy helper script to generate the sample file.
A new tox target also is added to simplify the environment
setting up. Now policy sample file can be generated
automatically, so the in-repo sample file is no longer needed.
Co-Authored-By: Andrew Laski <andrew@lascii.com>
Partial-Implements: blueprint policy-in-code
Change-Id: Ic336fa154ccc05b5e9db3a8e751a484b1cc5aa9c
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
As bandit-baseline check just call tox to run tests, adding it to
in-tree checks instead of gate jobs.
Closes-Bug: #1618549
Change-Id: If6848c69e09117789d566ae47977034d5503fa09
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
This patch adds a doc8 check of .rst files to the current pep8 check.
It includes fixes to the .rst files that didn't pass the check.
Change-Id: I866ee72cfdd6067ae219a751aa7c2a0c391886f7
Replace basestring with six.string_types and add a hacking rule
to check it. This modification refers to neutron and merge
translation_checks into checks .Follow-up will modify the other
on the python3 common
Change-Id: I66d58ecaf664705660bbeb85058ad3cd092ef82f
reference:https://github.com/openstack/neutron/tree/master/neutron/hacking
Issue 1:
The os-testr 1.0.0 release had a couple of required config changes due
to it's internal usage of stestr.
This change to stestr changed the way tests were discovered by
os-testr and as a result the unit test run was picking up tempest
tests.
A regex is added to the py3base environment call of ostestr because
the use of --serial and --blacklist-file together is broken in stestr
and adding the regex parameter allows the blacklist-file to be
processed. The stestr issue is documented here [1].
Issue 2:
Cache dirs for PKI tokens have been removed for all services in
devstack under I5680376e70e74882e9fdb87ee1b95d5f40570ad7.
We must also remove the use here to pass the right parameters to
configure_auth_token_middleware.
Issue 3:
Keystone V2 APIs have been removed. When creating Nova and Glance
clients, the test code was either hard coding v2 Keystone or not
providing enough information for the V3 auth.
Issue 4:
Oslo context has deprecated parameters such as 'tenant', has removed
them from its constructor and is using a rename decorator to handle
them. As such, the code and test case to check for unrecognized
parameters to TroveContext and Context is erroneously removing the
tenant parameter. Oslo context has also changed the from_dict method
since the original code to remove parameters was introduced into
Trove. The new method signature and code should already provide most
or all of the protections against incompatibility the original code
was attempting to provide. The fix for this issue is to change
TroveContext's from_dict method to use the kwargs to handle its own
__init__ parameters and be more in line with what Nova is doing in
its RequestContext subclass.
Issue 5:
Jobs run as jenkins on Zuul v2 but run as user zuul on Zuul v3.
Issue 6:
Ignore one case of pylint E1101 in the Ceilometer notification
code base.
[1] https://github.com/mtreinish/stestr/issues/103
Change-Id: Ic55187b0d73d4c572d7f8332882b4f455a6177c8
The infra scripts run msgfmt since some time, no need to run it here as
well.
This fixes the pep8 failure in https://review.openstack.org/#/c/493290/
since that change removes the remaining po files and now the find fails
since there is no po file anymore.
Change-Id: I6a8b99a9d63e421f4a6eae6f0aab6d0955f7916b
dib is now going to have to be installed explicitly for us to actually
run the gate. the old method of just cloning the repo won't work with
dib v2.
There are other ways to accomplish this but I'm choosing this approach
knowing that it isn't ideal. First, what's the downside? This means
that dib gets installed wherever trove control plane is installed. Is
that necessary, NO. Is it the end of the world, NO.
What are the options?
- do what Octavia did and have a second requirements.txt in a folder
called diskimage-create. Would this work, yes but we don't have such
a mechanism right now and nothing would keep this requirements.txt
file in sync with g-r that I know of.
- hardcode the pip install command somewhere in the devstack
plugin. yes, this would work but this would also have no ability to
sync with g-r.
So, while the solution proposed here isn't ideal, it strikes me as not
bad, and much less risk than the other two options.
We also need to figure out where the elements are and fix the path for
disk-image-create.
Some pep8 failures appears to have crept in as well, those are
addressed.
Change-Id: Ic64d91e082102057417995a0f6851d03b9e0ca74
This is the first patch to enable the translation checks.
These translation checks were taken from neutron_lib/hacking
Translation checks include:
* validate_log_translation: Translate log messages
* no_translate_debug_logs: Don't transate debug level logs
* check_raised_localization_exception: Translate exception message
The pep8 tests will fail until all files meet the standards enforced
by these translaiton checks. So this patch is depenedent on the
following:
Depends-On: Ieb7e006de497c974756941608aea91f63e860dc5
Depends-On: I1d0649f240365b7f62b13af6edb1e8dbb5d06597
Depends-On: I48f95b3c090e9b6f91185c14113daae5bc621d0b
Depends-On: I5feafe8945960e34a8d0e038ad4617f39502f19b
Depends-On: If29f7b5df7f6958cec4b26f72babc47ca70f3706
Depends-On: Iddd738c2d9a7c9a57fcd445650d087123dbccfc4
Depends-On: Ifdf426684473fb05cae63877c334e6cf65aa5234
Depends-On: I94a341ba1fb4178e0c358c37ea31623d48938ec6
Depends-On: Ic8664bf105d601909c6efa5cf73db6c27adaf42e
Depends-On: I80f2ea6a20944cfbfdc79a8a71ad744f23aeaac1
Depends-On: If68b38ef5bdeba2fdc28f94d964ef8ce6d15c352
Depends-On: Ia54014fa8e44aacc7935bd73c1ee8d3139a19735
Depends-On: Iea58f4df337c79785dc7afe0e31bcfa7e231f374
Depends-On: I2b0313021fd9599bdaeb375358e8cf834581d493
Depends-On: I56972d16634654b8b71853d6aba10299cc9e2418
Depends-On: If632727fd29ec8d36c6890ebd156a46be70ba783
Depends-On: I1df8d4cba6a3a2ec23e8a1b2aa4ffc03c97b149e
Depends-On: I4070117b6335f4d85f35cc6473653c17d7c14bac
Depends-On: I83f2ccd623588af3fc5f94334b753cebcc3b6c18
Depends-On: I44ac13e9431018e18980e8ff7ff442c7fea68a13
Depends-On: I88284f61b5f37d78ad050a97c679f6bde23d832d
Depends-On: I6fb2bdcc4b83457e08b24599fb4a297ef6ec6c14
Change-Id: Ia2844799a2af8e020470d4c513ad55a51ec36ce1
Many deprecations are triggered early (on imports, for example)
before the warnings are enabled by the WarningsFixture in the
base test class.
To make sure all DeprecationWarning messages are emitted we
enable them via the PYTHONWARNINGS environment variable.
This copy from 1be35ff039d67c50866f2556ef847b048cbf3578
Change-Id: Ibe67b6b57494d731b49e76c590ac2681ae786c33
<long rant deleted>
This change enables the hacking extensions ability in trove. That
includes bumping the version of hacking that we use, and also adding
the enable-extensions like in the [flake8] section. I've enabled two
of the hacking extensions here.
There are some limitations to the hacking rules that I've enabled so
I'm also going to enter two bugs that others can fix (if they so
desire).
Change-Id: I6d8fc4bc03f948b84ad823e5248d78a4caa5c042
Related-Bug: 1642554
Related-Bug: 1642552
This commit will merge into trove, the trove-integration tree as of
commit 9f92ca853f8aa2f72921e54682c918941a8f0919. This is in
preparation for making trove-integration go away.
In addition, it supresses any consideration of the integration
directory in the trove tox tests as it is understandably a small pile
of pooh and in need of much cleanup.
Change-Id: Ib7f2655c4c5ed86b5454708c04371ee55e37ec2d
Partially-Implements-Blueprint: eliminate-trove-integration-and-redstack
As there is no module that violates this rule, we shoud
get rid of H237 ignore. H237 is defined at [1].
[1]: http://docs.openstack.org/developer/hacking/
Change-Id: I739db63508604d2963b2d7f6aedaf2d80871448a
[H238] Old style classes are deprecated and no longer available in Python 3
(they are converted to new style classes). In order to avoid any unwanted side
effects all classes should be declared using new style. See the new-style class
documentation for reference on the differences.
http://docs.openstack.org/developer/hacking/#python-3-x-compatibility
Change-Id: I79e2f0d7d53d398fcf5ba18cefc8318ee6e07426
Since the config is not in a deterministic order makes it hard to
compare two config's and see what changed. Personally, I'm not
positive I understand this use-case; i.e. you have an existing config
file, you save it, and then rebuild and then diff the two files. I'd
have thought you'd just run check and the output of the tool was the
diff.
I however do see the value in sorting the file so that when someone
submits a change that includes a change to the config, reviewers can
see more easily what the change is doing.
Similarly, the output from pylint (errors) are generated one file at a
time and os.walk makes no guarantee of deterministic order. So we
should collect all errors (across all files) and then print an ordered
list for human consumption.
The intent is also to make pylint voting soon (in master). the changes
to contributing.rst and tox.ini are to make that easier. The config
file has also been sorted in place.
This change was motivated by an email exchange with Peter so I am
marking him as a co-conspirator.
The line numbers were removed from the tools/trove-pylint.config file
as these would change whenever the line numbers in the file changed
(since they are currently not being used in the comparison; they can
be re-added if deemed necessary at the cost of having every 'rebuild'
run create a different file).
The tools/trove-pylint.config was regenerated as well, since the
remaining two errors seem to be innocuous:
ERROR: trove/taskmanager/manager.py 392: E1101 no-member,
Manager.upgrade: Instance of 'BuiltInstance' has no 'upgrade' member
(new method introduced by instance upgrade; other BuiltInstance
member errors are already ignored.)
and
ERROR: trove/guestagent/datastore/experimental/postgresql/service/
access.py 80: E1101 no-member, PgSqlAccess.list_access: Instance ofi
'PgSqlAccess' has no '_find_user' member
(this is due to the fact that PostgreSQL is spread over multiple
files and pylint should cease to complain once
https://review.openstack.org/#/c/346082/ lands.)
Change-Id: I910c738d3845b7749e57910f76523150ec5a5bff
Closes-Bug: #1625158
Closes-Bug: #1625245
Co-Authored-By: Peter Stachowski <peter@tesora.com>
this is a pylint wrapper for Trove's tox tests. This commit includes a
basic infrastructure for running pylint through tox.
It also fixes some very obvious import errors that are flagged by the
tool. One is to handle missing imports for _ and _LE.
There is one instance where an exception is being raised but
trove.common.exception isn't imported, and another where an exception
is being incorrectly thrown.
A short readme is also provided.
Change-Id: I0a38f5efde3cb491f1f6c27f6c6500ab29987968
Partial-Bug: #1621636
This commit cleans up a couple of issues with Trove's tox.ini.
1. something that was reported to me by Matthew Treinish who
observed that the tox jobs are oddly bundled together.
This is important because infra's reporting is unable to generate some
useful metrics and tracking of trove's jobs because of the way they
are tied together.
2. with the cleanup of docs, apidocs is no longer a directory and can
be cleaned up, similarly api-ref is a thing and should be handled in
.gitignore.
3. that py27 used to use testr, switched it to using ostestr. I've
retained it for coverage.
4. the sqlite db needs to be deleted between runs because we otherwise
fail (on a developer configuration) because some tests intentionally
appear to leave stuff around.
5. I've set all ostestr jobs to run with --serial because there are
clearly cases where multiple tests can attempt to setup the DB at the
same time and get into trouble. Till we can fix those (if we feel it
is worthwhile) it is more reliable to run ostestr in --serial mode. It
only takes 2.5 minutes. This includes the cover jobs which runs with
--no-parallel.
An infra change will follow that will enable testing jobs.
Change-Id: I0a971c6ef6c1a4669490f2aa0e67c795fe86e190
Closes-Bug: 1618098
Closes-Bug: 1618149
Added an entry for the py35 tests. Put the default
values into a py3base entry and referred to them
from both py34 and py35.
Change-Id: Ibd9e77c24f56e7886459fce00a4ed33b3e9e1011
Since this repo does not publish DocBook XML content anymore, we can
remove openstack-doc-tools and the tox.ini targets for it.
Also, remove the config file doc-test.conf, it's not used anymore.
Depends-On: Ie90eb7fa6f1b1d7b99869083099201d2bfe10018
Change-Id: Ia92429f120fd18553e2785f63f6a12ad0fc9f12f
With this email[0], you must migrate API reference docs into RST. The
conf.py and the tox environment are also cribbed from nova.
Still need to retain the install_command in tox.ini, otherwise the
api-ref job fails.
[0] http://lists.openstack.org/pipermail/openstack-dev/2016-May/093765.html
Co-Authored-By: Anne Gentle <agentle@cisco.com>
Co-Authored-By: Amrith Kumar <amrith@tesora.com>
Change-Id: I3315261aa18729fa7a6aa79d4a1d6c24de1e2c6b