A symlink is kept to update.py to keep this working for infra until we
update the propose scripts to try the new location and fall back.
Until that is done and the symlink removed, update.py cannot refer to
other modules (though they can refer to it).
Change-Id: If878bc2bb426c0e5478ed031f1144e26f05e85a2
This is thoroughly documented in the cross project spec 'requirements
management'. In short though, we manage the exact versions we test
with as a change controlled thing and use async processes to push
convergence and keep up to date.
Change-Id: I5fc747646ae9371f0cc085b1e15af45951d20152
This moves things that are pretty much copy-paste into one place,
letting us focus on the actual differences amongst the tests.
I've also made the local test fixtures explicit, so we can see what
each test really needs, which has also eliminated redundant test
overhead, the slowest test is now 0.05s vs 0.1 before. This is more
than paid for by the reduced duplication in the basic setup, making
the patch a net reduction in LOC.
It also fixes the tempdir leak that was present in two of the test
scripts.
Change-Id: I63eee7681657b2b5eb7959738cc89c57c117fa3b
Looks like a long time ago we used this to list development versions
of OpenStack libraries which have not yet been released to PyPI. We
don't really need this now. Do we?
Change-Id: I5f8fce7068ee8dba5817afbdd658650b6f2b74dd
This should let us stop using pip internals, which breaks us all the
time.
Conflicts:
update.py
Change-Id: I10ada6860768017d4eb3c729f2870f2c7f4a46d7
(cherry picked from commit 166a9f1d50a4df2ed1305de431563a2494d673d2)
The latest release of pip seems to have changed code that we rely on.
e8e2566279
As such, we can't run update.py on any project, this patch changes
the attribute from url to Link.url.
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: Icf60f64f4e84f0c506263202821c8374c914e121
Closes-Bug: 1440984
Add a flag to allow the update script to update the hacking dependency
like any other, to make it easier to bulk-update projects with multiple
repositories.
Change-Id: I1edc795da86064360aaecf04856a9a9b1c97a2b1
setup.cfg is used by pbr instead of setup.py.
Projects not using pbr while not have setup.cfg at all
or will have it with completely different contents.
There's already a check in update.py whether or not
the target project is using pbr, but it relies on
presence of setup.cfg, which may not be the case.
After this patch update.py will try to read setup.cfg
only after it detected that pbr is used.
Change-Id: I21fbbe159007b25db546fa6d36721e018d1dca24
In an attempt to make the proposal bot changes for requirements be
more meaningful make the update.py script return something that's more
user friendly.
Now by default the script will return the following to stdout on success:
Version change for: mox, mox3, testrepository, testtools
Updated project/test-requirements.txt:
mox==0.5.3 -> mox>=0.5.3
mox3==0.7.3 -> mox3>=0.7.0
testrepository>=0.0.13 -> testrepository>=0.0.17
testtools>=0.9.27 -> testtools>=0.9.32
Add a verbose flag that will provide much of the previous debug info.
Eliminate the giant dict dump of all the requirements files, as that
was largely impossible to make sense of, and rarely provided any
insight. During a devstack run you'll have git hashes for all the
trees, so can track down exact file contents if needed.
By making this the default stdout we can then capture and use this for
the git commit messages.
In order to make the test output matching human readable ignore the 80
column limit for the expected blocks via #noqa.
Change-Id: I85604db7bffafcd20bf3cc546fe3e5d7bda72193
Previously, if any change was made to a line, it would update the line. This
includes whitespace changes around comments on the line. See
https://review.openstack.org/#/c/144782/ for an example of a unhelpful update.
Change-Id: I0d43c604c10e18c72810d42046d447fbf1deed10
This provides a new flag to update.py which allows for a "soft
update", which updates all the requirements found in g-r to the g-r
specified versions, but lets any unknown lines pass through as is,
without warning. This makes the tool usable for ecosystem projects
that want to test with g-r versions of the g-r specified dependencies,
but also want to be able to add their own dependencies above and beyond.
Change-Id: I1f195ef9ff1509659848e14ec9936ff6f66a6496
PBR supports files of the style requirements-pyN where N is the
bundling python version, which means we need to sync the same number
of supported possible PBR files that can exist.
This adds in the missing py2 that was not being synced even though
it is a viable requirements file for PBR.
Fixes bug 1371936
Change-Id: I7956d77c275694cf09971766d16af054d2692261
During sync, add a header to the requirements files warning anyone
against changing the order of the contents, especially against sorting.
Addresses-Bug: #1365061
Change-Id: I64ae9191863564e278a35d42ec9cd743a233028e
Now that projects don't directly use pep8 anymore but rather hacking,
update the ignore pep8 clause to say hacking.
Related patch to ignore hacking in the requirements sync check:
I114b05fb1ec99ec1e0e0162988edb5fa5517df46
Change-Id: Ib21646db18220b40db505856f4796c3d51228abc
In python < 2.7.4, a lazy loading of package `pbr` will break
setuptools if some other modules registered functions in `atexit`.
solution from: http://bugs.python.org/issue15881#msg170215
This problem originally appeared in stevedore
before the library was adopted by Oslo
(74a719b117)
This change places the fix from Icd412ca8435dca4b0f3b4075eca39b903dde6d27
in the right location to be synced into the other repositories.
Change-Id: I7560e474e68bf361f3af61bdfc8e2dbc9e2413f1
there are non integrated projects in openstack that use devstack
framework for running jobs. They need a lever in their jobs to
tell the infrastructure that we won't enforce global-requirements
strict compliance on them. We'll still generate a warning so that
they know they are using requirements outside of OpenStack's
accepted list, but it will be non fatal.
Change-Id: Id90547e2fb0a5736385c9f31ba386e7c2e0616df
Don't silently remove dependencies not in global reqs, instead let them
fail installation.
Note: this assumes that devstack will terminate if this fails.
Change-Id: I23be4c2084e07a6b60b64347af9b4c5338b06ada
We don't need vim modelines in each source file, it can be set in
user's vimrc.
Change-Id: Ib2b5642b7d9051d4586419a6dd4785060cd59825
Closes-Bug: #1229324
Due to the fact that we'd like to keep pbr's requirements in sync with
global requirements we should disable syncing of setup.py for pbr.
* existing tests fixed by adding standard setup.cfg file;
* new test case added for syncing pbr project (with pbr like setup.cfg file).
Change-Id: Ibf5fcdad118fffb4fd5dd50c5d1decfa6f9a6479
This patch adds the ability to handle requirements-py3.txt and
test-requirements-py3.txt as usual.
Change-Id: Ibff0bbd182cf1027f8a3ef2278d1d226538f8222
If projects.txt is going to be the basis for select-mirror in the
gate, we should also list projects in projects.txt that need to be
constrained by the mirror for their testing. In order to do that, we
need to not try to unconditionally update setup.py, since devstack and
devstack-gate do not have it.
openstack/requirements itself also does not use the mirror, so remove
it from the list.
Change-Id: Iea33e8f4d5453bf413f3d479b85eaed99938b302
Correct the copyright headers to be consistent with the policies
outlined by the OpenStack Foundation at
http://www.openstack.org/brand/openstack-trademark-policy/
Remove references to OpenStack LLC, replace with OpenStack Foundation
Change-Id: Iec8f554a5c279c39580c00abd50e3a9b2e6bf219
Fixes-Bug: #1214176
setuptools can never upgrade things based on setup_requires versions.
What this means is that pbr can NEVER EVER introduce a feature or a
change that would require specifying a newer version of pbr in the
setup.py file, because if will break. This is a non-flexible
requirement.
The requirements.txt file can catch the actual version we're interested
in having around, if it's an install time requirement.
Change-Id: Ic491f295156ea0ee3f6296360ca8b31d5134ae07
When we run update.py automatically in the devstack gate, we
have the issue that we are programatically changing files which
are actually git versioned. This makes lots of people sad pandas.
As this is all being done automatically we could actually update
the files with a suffix, then have devstack use that instead.
This will make developers happy, as they won't have to keep
resetting their devstack trees if they do development straight in
them.
Change-Id: I3d264f15ab7123f258f198b375b9667895c4104c
This allows us to pass through an exact line from
dev-requirements.txt as valid, if it's not exactly valid we'll
revert to the base requirement. This is needed so that projects
can keep development requirements in their tree.
Redo an if statement to make E125 happy....
Change-Id: I50a1292c321b7092ca1c7aeee5068f0828ece7b0
Removes the need for setuptools_git. Also, change the upper bound,
as the next planned release will be 1.0 and we'd like to coordinate
that move.
Change-Id: I9a9ff8c62f800da3bf02d4afb3bc24ddfd0d24d5
If a project isn't on pbr, then force-upgrading their setup.py will
be a categorical disaster.
Once glance and swift have finished moving, we can remove this 'feature'
but for now, it's a blocker.
Change-Id: I6ab41be97a2ff83c395861a95244d090c543d1fb
One of the points of pbr was ensuring that we had a consistent build
environment, but we still have boilerplate setup.py laying around.
Install the right boilerplate at update time.
Change-Id: If4b2e16b469171204270ace8144792e607cfddd7
... and in the darkness bind them.
This moves the contents of requirements.txt and
test-requirements.txt to a single file, global-requirements.txt.
This means that regardless of whether a requirement is in
either file it will get checked. This massively simplifies the
checking and upgrading of req and test-req files.
This also adds in a unit testing framework for the update.py
script to ensure it does what we think it does. This is
accomplished by making a fake tree with a set of req and test-req
files, running update.py, and ensuring the updates were made
that we expected. It includes testing for the oslo url case,
as well as test-requires.
Change-Id: Ib9b86ade4cb8317509e218aec31f32e5d08f4035
In our gating logic (project-requirements-change.py), we ignore
tarballs, editable reqs and --find-links lines. Similarly, in update.py
we just want to pass these through on unmodified.
Add a _pass_through() function which encapsulates this logic and use it
to ignore lines when parsing the source requirements.txt and also to
pass-through matching lines from the destination requirements.txt.
Change-Id: I7c34dd65ecd866e923c117686d10e86177bb320e
This removes reordering of the requirements
when updatin the files, as preserving the
order makes reviewing significantly easier.
Also preserve comments around the requirements.
In addition, search for missing test-requirements
also in the tools/pip-requires, to avoid accidentally
dropping them.
Change-Id: I3c51783a42db9907da9d9a98b42b52b5f639bcc8
pep8 requirements were always written twice to the
updated requirements. Simply preserve the original pep8
line, like the source code comment mentioned.
Change-Id: I6b23f756594feeea0a5dbe14c58be981501bc5a4
Within Havana we generally rename tools/pip-requires to
requirements.txt and tools/test-requires to test-requirements.txt.
Handle both locations gracefully.
Fix flake8 warnings.
Change-Id: I5ad03a1b7efa76e31522f5b6f0da1bb3dd8cf6bc
