This enforces consistent spacing in global-requirements.txt
and sorts the specifiers to be always in a specific order
(namely minimum bound first, then exceptions, then upper bound).
Add test coverage for this normalisation.
Change-Id: If41732bfe4476e422bc6b9f9f896eb978db8be84
We want to ensure that every entry in global-requirements has
at least a minimum version specified, and if there is an exclude,
the exclude is not outside the lower bounds.
Change-Id: Ie12d99d3317bd4c81de9a47a43fa99345b2b9664
This is already fixed in master:
https://github.com/mitmproxy/mitmproxy/pull/2435
So it'll hopefully in the next release.
Because the pyOpenSSL package name and requirement name do not match we
also add code to lower the name for comparison.
Also the check-uc only complains about a single issue but there are 3.
Change-Id: I6657802908ac4052eeb6c73fb03098b0c3557eb2
The check_constraints utility wraps a new check_exists cmd.
This command looks at a project's requirements and checks:
- That the requirement name alone exists in both g-r and u-c
- If it exists in both, then check that the version in u-c
is covered.
- Added an extra g-r check as a flag, but not quite happy with it.
It needs more work.. but would be a _little_ more correct
if/when g-r stops defining the minimum.
This patch is currently WIP as tests are still required. But is
the starting point for the fuzzy checking for projects managing
their own lower-constraints.
Change-Id: I36690fddcfc24d49aab0d28d7d46dcd8d9128b2a
It's somewhat strange to say you can build a package with pbr >= 1.8 but
to install it you need 2.0.0. Synchronise these values.
Change-Id: I47640a0226be2e82f2fbbf866d2c235f5e7ed415
The validation command is not able to handle that and I'm not sure
if it is really worth fixing that.
Change-Id: I8abc66143333522b77ab4ff6b2fef66bc707a36a
We want to avoid packages with common names sorting close together in
the constraints list, because when we batch-release from projects like
Oslo we have a lot of merge conflicts trying to land those individual
patches. This changes the constraints list to be sorted based on the
sha1 hash of the dependency name (ignoring version information so we
have some consistency in the ordering). This gives us the appearance of
randomness while maintaining a stable sort.
Change-Id: I761967f8281c4d5df02e1dc88895b76d9ac89e6e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
We need to generate constraints for all python versions we use. This is
python 2.7 (trusty and xenial), 3.4 (trusty) and 3.5(xenial). There
isn't a supportable way to install all 3 versions on a single node.
Add a version-map that allows us to clone the output of a freeze between
python versions. This assumes that the python version in question are
compatible during the transition window.
It's possible to run with:
... --version-map 3.4:3.5 --version-map 3.5:3.4 ...
This will mean that we generate constraints consistently between versions
regardless of which version /usr/bin/python3 is.
Change-Id: I55a3597ea2f2bf48e30dc564a4bf4b2ef7f178b3
Implement XFAIL handling for ignoring known problems so that
we can make the job voting until the remaining issues are sorted
out.
Change-Id: Ia2cf2c440a72e6365a9ff08086704dc535bbc072
We need the integration gate to work with both trusty (python3.4) and
xenial (python3.5). Rather than hard code the full python binary just
use python 3 and let the OS mange this for us. This is less than ideal
but it's the best we can do right now to unblock the gate.
We also address a short term issue with out upper-constraints.txt files
and manually copy the constraints from 3.4 to 3.5
Related-Bug: 1620436
Change-Id: I6acabf86933b7a7bce7baf44e6b512e7d0d2f6db
Rather than have this as an external tool that open codes some of the logic
we have in the existing codebase we should integrate it.
Change-Id: I979dcefeec4ae28705088d4fac1e41b4d33e40e0
Use pkg_resources.require to report conflicts to ensure
co-installability.
Co-Authored-By: Igor Yozhikov <iyozhikov@mirantis.com>
Change-Id: I7443496b6a06d02bb9cc9457b729dea71451f44a
If you mistakenly pass a path that isn't a directory, or doens't exist,
you get a traceback that ends with:
---
File "/Users/tony8129/projects/openstack/openstack/requirements/openstack_requirements/cmds/update.py", line 229, in _copy_requires
dest_path, project.merge_setup_cfg(proj['setup.cfg'], output_extras)))
KeyError: 'setup.cfg'
ERROR: InvocationError: '.../openstack/requirements/.tox/update/bin/update-requirements -H -s ./nova'
---
This is a little confusing. This patch checks that the arg is a
directory so a more explicit error can be generated.
Change-Id: I9f3f43d5c9b90746bbf2d651ee4cb37afade1528
Add a command and tox environment to let us define a job to ensure that
all repos in projects.txt have the check-requirements job defined. Two
projects are missing the job now, so this patch depends on the patch to
add those jobs.
Change-Id: I61658a7265a77e72401978dda558f41b28a81d92
Depends-On: I81e1c70257e28583e776171580d75341e620bf8d
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
A project needs to be able to specify that it depends on a project
including extras for that project and still have it's version maintained
by global-requirements.
Currently if the requirements are seen to mismatch then the string from
global requirements is dropped in place of the project string, but this
drops any extras specified by project.
In the case that extras are different and the version needs updating add
a new combined requirement to the project with the original extras and
the new version.
Closes-Bug: #1567809
Change-Id: Ife48b7963a5e6706289f1b9a47cb95fae7f0bc22
In python3 izip functions got renamed to zip in itertools module.
This was breaking support for python3.
Change-Id: I6fda87b78e40be134d7c94698cca5d85a51eb989
This change is necessary in order to use the project's
update-requirements.py script when preparing the environment on the
Hyper-V compute nodes used in the Hyper-V CI. Using the mentioned script
will set the latest requirements in nova, neutron, manila, etc.,
reducing the CI's amount of failures caused by outdated requirements.
Including Microsoft Windows as a supported operating system on this
project. Also, we update the edit-constraints.py and project.py
files to support os.rename() of Windows.
On Windows, os.rename() will create a file with the source
name, without deleting the initial file first.
Example of a traceback:
Traceback (most recent call last):
File "C:\Python27\Scripts\edit-constraints-script.py", line 9, in
<module>
load_entry_point('openstack.requirements==0.0.1.dev2497',
'console_scripts', 'edit-constraints')()
File
"C:\Python27\lib\site-packages\openstack_requirements\cmds\edit_constraint.py",
line 74, in main
os.rename(args[0] + '.tmp', args[0])
WindowsError: [Error 183] Cannot create a file when that file already exists
This could be easily solved by removing the file before the rename.
Proposed fix:
Add before "os.rename(args[0] + '.tmp', args[0])" the line "os.remove(args[0])"
in edit_constraint.py
Add before "os.rename(tmpname, fullname)" the line "os.remove(fullname)"
in project.py
Add "Operating System :: Microsoft :: Windows" in setup.cfg
DocImpact: Added Windows as a supported OS in setup.cfg
Change-Id: If123a65fd8d49d5c67a1db16827a9617ce520dba
Signed-off-by: Costin Galan <cgalan@cloudbasesolutions.com>
In change id I60adf0dca4aa32f4ef6bca61250b375c8a3703c6, we added
flake8_docstrings to the black list. However as seen in
I8bfff1b2e65541959e7f2ae0fa6ca6c17889eb54, the generate-constraints
job ends up proposing adding "flake8-docstrings===0.2.1.post1" to
the upper-constraints.txt. This is because of the '-' vs '_'
difference. So just like in Ibaa22657a2cf2c0ad96dbd0b9bc43cdafe6a1d56,
we need to convert the items in black list to the correct safe name
before trying to check if the item is in the black list.
Change-Id: Ieb1cd44e908b80d58a20bf0d0e1e6b10f17ae95e
*) We're removing our upper caps - they are just time bombs. Make pbr
be consistent here too.
*) pbr 1.8 handles comments in marker lines properly. We don't need to
update the run-time minimum, but we do need a build-time minimum of
1.8 to ensure OpenStack packages can build properly.
Change-Id: I9fe0c62e774c00c2e521e1e69c5f2f5d87865ec1
Problem showed up with django_openstack_auth job that installs from
source. The change to devstack to verify LIBS_FROM_GIT kicked in
(Iffef2007f99a0e932b68c4c897ebbfb748cac2b4) and starting failing the
job. The problem was that the edit_constraint was not fixing the
django-openstack-auth line in upperconstraints.txt as the setup.cfg
had name = django_openstack_auth. Note that when things finally get
installed setup_tools' safe_name kicks in to standardize names and for
example glance_store will show up as glance-store in pip freeze. So we
should be able to treat this situation better and allow constraints to
have the safe names (using dashes).
Co-Authored-By: Robert Collins <rbtcollins@hp.com>
Change-Id: Ibaa22657a2cf2c0ad96dbd0b9bc43cdafe6a1d56
Having the data validation happening inside the unit tests makes
understanding requirement update failures confusing because the output
is hard to read and does not explain what the error actually is. Move
the checks to their own command and set up a tox env to run it. A
separate patch in project-config will add a new job for the repository.
Address the dedent comment from
https://review.openstack.org/#/c/204181/3/openstack_requirements/tests/test_requirement.py,cm
Add a test to ensure that all items in global-requirements.txt are also
listed in either upper-constraints.txt or blacklist.txt. Ignore a few
items that we don't know how to constrain yet.
Add a test to ensure that items in blacklist.txt are not in
upper-constraints.txt.
Change-Id: Icb717b0f36afb6ea29f50bc6935917dddf47fd4c
Without this version, markers will fail to install in a growing number
of projects.
Change global-requirements to prevent
pip -r requirements.txt; pip install .
failing with an already installed and too old for setup_requires pbr.
Change update.py to signal to setuptools the minimum version required,
giving clearer error messages in the case of downgrades during
devstack.
Change-Id: I074274b470a7a4f2c95211677b135038ab2c942c
I have a project that includes a requirement line this:
-e git://git.openstack.org/openstack/neutron.git#egg=neutron
Every time update.py runs against the file, it adds an extra newline
after this. For example, if I run update.py a few times, I'll have a
few new lines in the file. This patch prevents the extra newlines
from getting added.
Change-Id: I92d02a45c437025b16df067d7b58a3727d42c0e0
Signed-off-by: Russell Bryant <rbryant@redhat.com>
constraints files, unlike global_requirements, need to be able to
contain urls (specifically file:/// paths) so that we can constrain
libraries to be run from their git trees, rather than having them flap
all over the place as different servers specify different bounds. The
first edit-constraints implementation failed to parse such lines,
throwing errors instead.
This patch makes the support for urls optional, preventing changing
the semantics of requirements update.py logic.
Change-Id: I33bd2d9dff9cb7dc1a50177db7286b7317966784
Depends-On: I0f07858e96ea3baf46f8a453e253b9ed29c7f7e2
This will provide space to store the result of parsing requirements
like -e file:///opt/stack/new/oslo.THING, which we need to do for
edit-constraint to work in devstack when more than one library is
being installed from git. This is a separate patch to make reviewing
the actual new functionality easier - it is entirely mechanical,
adding the field and dealing with test fallout.
Change-Id: Ied4e92717b0dab871dac42ae639d1f2a6c236414
This allows shell editing of a constraints file, which we need to do
in devstack when installing a constrained library from git.
Change-Id: Ibdbfc786b5d2f56d2e41d329c8766bef7b2e6854
