remove pymod2pkg as well as it was only used in renderspec
Change-Id: If00a0efed80f8ec821397805f59ee4da483361e4
Signed-off-by: Matthew Thode <mthode@mthode.org>
This was merged into horizon under the new name "openstack_auth"
see https://review.openstack.org/#/c/482561/
Depends-On: I93f6365ccf661568b37ea9c737f8ce6115873739
Change-Id: Iec8a64b25aa4ab188a940cf919545275d6967391
pep8 was renamed to pycodestyle, so allow that one in order to
be able to transition to it.
The depends-on remove pyflake mentions as most projects adopted
flake8/hacking, we don't need to allow explicit dependencies
on the tools integrated by flake8 (mccabe, pycodestyle/pep8, pyflakes).
Cleanup those from the tests/ files to avoid grep'ing finding them.
Depends-On: I12b5e114c5c6783e9da5cca2818ac9266a00fc87
Depends-On: Ib145e2afa97f441c07fada9c30f0f0e2410870ae
Depends-On: Id44c90ddd4a8a2204fd7ce600097bafbc9469993
Change-Id: If9f30bf39143d5e6fefca794385cafc4456d2197
This is merged with cliff and shouldn't be used separately anymore
Depends-On: I8221542e666502986626b9fa53956721da1a3220
Depends-On: I24d0c348b040f601f9f36b08972c27d0289540de
Change-Id: I0607e905e4f2bb33d5a329c50d379b59250394a9
We want to ensure that every entry in global-requirements has
at least a minimum version specified, and if there is an exclude,
the exclude is not outside the lower bounds.
Change-Id: Ie12d99d3317bd4c81de9a47a43fa99345b2b9664
1) Update doc links according to OpenStack document migration
2) Use https instead of http for docs links
Change-Id: I2d558e68b060ae893bafccf51333d1fd5d061322
Historically, to write Mistral actions you would need to depend on
Mistral itself. However, with the introduction of mistral-lib this is no
longer required. The only consumer of mistral-as-a-library within
OpenStack was tripleo-common. It has been fully migrated to mistral-lib
(see depends-on) and the requirement can thus be removed.
Change-Id: I0a99c19d3cbcfbb396b09c008aff7a379a273d4c
Depends-On: I7ae2e0e4dbca6c02ed8ceedc066ca35eeb13c322
The regular expression to parse requirement line is also used for
constrains files. However, it will not match those lines which
point to VCS, such as git+http or git+https which pypi accept.
This commit fixes the issue.
Despite that we only need to change the RE to
^(?P<url>\s*(?:-e\s)?\s*(?:(?:git\+)?(?:https|http)|file)://[^#]*)#
egg=(?P<name>[-\.\w]+)
to fix the issue for git+http and git+https, pypi accepts more VCS
URI schemes other than git. This patch also looses the match for
these cases to be accepted.
Change-Id: I202078f5ca4ba6da4ab0ed52d97d41997674c1b4
The check_constraints utility wraps a new check_exists cmd.
This command looks at a project's requirements and checks:
- That the requirement name alone exists in both g-r and u-c
- If it exists in both, then check that the version in u-c
is covered.
- Added an extra g-r check as a flag, but not quite happy with it.
It needs more work.. but would be a _little_ more correct
if/when g-r stops defining the minimum.
This patch is currently WIP as tests are still required. But is
the starting point for the fuzzy checking for projects managing
their own lower-constraints.
Change-Id: I36690fddcfc24d49aab0d28d7d46dcd8d9128b2a
We need to generate constraints for all python versions we use. This is
python 2.7 (trusty and xenial), 3.4 (trusty) and 3.5(xenial). There
isn't a supportable way to install all 3 versions on a single node.
Add a version-map that allows us to clone the output of a freeze between
python versions. This assumes that the python version in question are
compatible during the transition window.
It's possible to run with:
... --version-map 3.4:3.5 --version-map 3.5:3.4 ...
This will mean that we generate constraints consistently between versions
regardless of which version /usr/bin/python3 is.
Change-Id: I55a3597ea2f2bf48e30dc564a4bf4b2ef7f178b3
Actually unreferenced in any project subject to g-r
Co-Authored-By: Swapnil Kulkarni <me@coolsvap.net>
Change-Id: Idb76cd4f450cf09853982b905c41473b20c51735
If you mistakenly pass a path that isn't a directory, or doens't exist,
you get a traceback that ends with:
---
File "/Users/tony8129/projects/openstack/openstack/requirements/openstack_requirements/cmds/update.py", line 229, in _copy_requires
dest_path, project.merge_setup_cfg(proj['setup.cfg'], output_extras)))
KeyError: 'setup.cfg'
ERROR: InvocationError: '.../openstack/requirements/.tox/update/bin/update-requirements -H -s ./nova'
---
This is a little confusing. This patch checks that the arg is a
directory so a more explicit error can be generated.
Change-Id: I9f3f43d5c9b90746bbf2d651ee4cb37afade1528
A project needs to be able to specify that it depends on a project
including extras for that project and still have it's version maintained
by global-requirements.
Currently if the requirements are seen to mismatch then the string from
global requirements is dropped in place of the project string, but this
drops any extras specified by project.
In the case that extras are different and the version needs updating add
a new combined requirement to the project with the original extras and
the new version.
Closes-Bug: #1567809
Change-Id: Ife48b7963a5e6706289f1b9a47cb95fae7f0bc22
When using generated version output strings these produce stable but
unusually ordered outputs. Fudge the tests to create the expected
version strings to ease the update process.
Change-Id: I021f2befe2422525f528e27b041f60ec684e1eae
This reverts commit 0cf3c06ad98fa86378ffd561b65c0154989b179c.
setuptools 20.3 was released today which fixes the
vendorized exception problem.
Change-Id: I24edaf6cb2f0b354e2ebb33451f1f15bfd888183
New setuptools 20.2.2 seems to be throwing a new exception
as it uses a vendorized version of packaging now. Let's make
sure our tests works with both the older and newer versions.
Change-Id: Iab9247e2026896bb7d25b125788ec3072f819dad
In change id I60adf0dca4aa32f4ef6bca61250b375c8a3703c6, we added
flake8_docstrings to the black list. However as seen in
I8bfff1b2e65541959e7f2ae0fa6ca6c17889eb54, the generate-constraints
job ends up proposing adding "flake8-docstrings===0.2.1.post1" to
the upper-constraints.txt. This is because of the '-' vs '_'
difference. So just like in Ibaa22657a2cf2c0ad96dbd0b9bc43cdafe6a1d56,
we need to convert the items in black list to the correct safe name
before trying to check if the item is in the black list.
Change-Id: Ieb1cd44e908b80d58a20bf0d0e1e6b10f17ae95e
Starting with oslo.db+Nova, we would like to specify say
oslo.db[fixtures] in Nova's test-requirements. If you see
the proposed review I4131e534d4cb12e4888d398fa4fb7c922e369210
the gate-nova-requirements fails because code in
project-config/jenkins/scripts/project-requirements-change.py
does not understand extras at all and treats both oslo.db
and oslo.db[fixtures] the same. Since that module
uses requirement.py, we should expose the information here
first and then fix project-config after that.
Change-Id: I146baa3ef94cc8bbf29af371786f3ea95a42cb9f
Problem showed up with django_openstack_auth job that installs from
source. The change to devstack to verify LIBS_FROM_GIT kicked in
(Iffef2007f99a0e932b68c4c897ebbfb748cac2b4) and starting failing the
job. The problem was that the edit_constraint was not fixing the
django-openstack-auth line in upperconstraints.txt as the setup.cfg
had name = django_openstack_auth. Note that when things finally get
installed setup_tools' safe_name kicks in to standardize names and for
example glance_store will show up as glance-store in pip freeze. So we
should be able to treat this situation better and allow constraints to
have the safe names (using dashes).
Co-Authored-By: Robert Collins <rbtcollins@hp.com>
Change-Id: Ibaa22657a2cf2c0ad96dbd0b9bc43cdafe6a1d56
The current url_re regexp ignores dots in egg section, so it proceeds
url with "#egg=oslo.messaging" as "#egg=oslo". This patch fixes it.
Change-Id: Ifcea3e825218578f799c0a951baac8f8e8ef5de5
Having the data validation happening inside the unit tests makes
understanding requirement update failures confusing because the output
is hard to read and does not explain what the error actually is. Move
the checks to their own command and set up a tox env to run it. A
separate patch in project-config will add a new job for the repository.
Address the dedent comment from
https://review.openstack.org/#/c/204181/3/openstack_requirements/tests/test_requirement.py,cm
Add a test to ensure that all items in global-requirements.txt are also
listed in either upper-constraints.txt or blacklist.txt. Ignore a few
items that we don't know how to constrain yet.
Add a test to ensure that items in blacklist.txt are not in
upper-constraints.txt.
Change-Id: Icb717b0f36afb6ea29f50bc6935917dddf47fd4c
The test to validate the constraints file against the global requirements
list was looking for any spec from any package to match. Fix it to only
look at the specs for the package from the constraint being checked.
Add some more unit tests to demonstrate the old conditions that were
failing, and to verify the new logic passes.
Add stdout capture to the tests so we can provide more helpful error
messages to folks contributing patches to the different requirement
lists about why the integration test might be failing.
Add a test to enforce the format of upper-constraints.txt using 3 = to
indicate an absolute constraint. Fix 2 constraints which were previously
specified with only 2 =.
Change-Id: I46fd11031ba8f775fe62406380bf6417684f1f06
I have a project that includes a requirement line this:
-e git://git.openstack.org/openstack/neutron.git#egg=neutron
Every time update.py runs against the file, it adds an extra newline
after this. For example, if I run update.py a few times, I'll have a
few new lines in the file. This patch prevents the extra newlines
from getting added.
Change-Id: I92d02a45c437025b16df067d7b58a3727d42c0e0
Signed-off-by: Russell Bryant <rbryant@redhat.com>
constraints files, unlike global_requirements, need to be able to
contain urls (specifically file:/// paths) so that we can constrain
libraries to be run from their git trees, rather than having them flap
all over the place as different servers specify different bounds. The
first edit-constraints implementation failed to parse such lines,
throwing errors instead.
This patch makes the support for urls optional, preventing changing
the semantics of requirements update.py logic.
Change-Id: I33bd2d9dff9cb7dc1a50177db7286b7317966784
Depends-On: I0f07858e96ea3baf46f8a453e253b9ed29c7f7e2
This will provide space to store the result of parsing requirements
like -e file:///opt/stack/new/oslo.THING, which we need to do for
edit-constraint to work in devstack when more than one library is
being installed from git. This is a separate patch to make reviewing
the actual new functionality easier - it is entirely mechanical,
adding the field and dealing with test fallout.
Change-Id: Ied4e92717b0dab871dac42ae639d1f2a6c236414
This allows shell editing of a constraints file, which we need to do
in devstack when installing a constrained library from git.
Change-Id: Ibdbfc786b5d2f56d2e41d329c8766bef7b2e6854
We shouldn't be able to bump global requirement versions if that
would be incompatible with the current test set of versions. To assess
'works' we need to run them in tests which means that we need to
update upper-constraints.txt at the same time. We don't test all
versions however, so just checking that the constraints version is
compatible with the updated global requirements is sufficient.
This adds a dependency on the pypa packaging library. The library is
maintained by the authors of setuptools and pip, is Apache V2, in good
condition and Python 3 ready. Its a test-only dependency so I don't
think the distro packaging status matters. This is *the* reference
library for packaging facilities on Python.
Change-Id: I032dd9a384453dcfa4911ea905c8f89b0c974211
This regressed when I half-split it into actions and IO (exception
raising). Moving it entirely into actions lets us see all the errors
very easily.
Change-Id: I12e6d910bf983841b4ad81aa41a95bf341456fe4
This script when run creates a constraints file with exact pins for
the transitive dependencies of a requirements file.
Change-Id: I1282f8e4010c0ec46c473495bacddf044d06c0af
This required some care. There doesn't seem to be a sane Python3 ready
comment-preserving ini parser around, so I wrote a
minimal-for-our-case one in Parsley. Parsley is already in use in
infra in bindep, but I need to add it to global-requirements as this
is the first use in a managed project of it.
Change-Id: I48de3a2f36e945f75b534f689e3af802bbdc5be9
Depends-On: I7d7e91694c9145fac0ddab8a9de5f789d723c641
Depends-On: I16e967356d5c56f1474ee661b954b3db11a608cb
setup.cfg uses foo:MARKER rather than foo;MARKER as requirements.txt
files do. To be able to process setup.cfg sections we need to support
':' as a marker separator.
Change-Id: I3d78b3df8ea9fad1fd7b3c634eb0fb0f7e7e9124
This means we can use _sync_requirements for setup.cfg stanzas, which
have a different format to requirements.txt files.
Change-Id: Ifdb612017677fa2cf3e91de09e4b152dd11ca8ed
We're going to be parsing different sorts of files soon, and
sync_requirements shouldn't need to figure that out.
Change-Id: I98677e083cdf55a8626334bee03786aa021045e3
This is needed when a single specifier set is insufficient to handle
both Python2.7 and 3.x. We're not currently dealing with this, but I
have observed it in other projects and we can be forearmed quite
easily.
Change-Id: If1493625d0d2ebf750159ccb8ed0fd22699766a1