Global requirements for OpenStack
Go to file
Tony Breeds 587460bac1 Support cloning one freeze to multiple
The idea behind the version-map code is to work around a lack of a node
that has multiple python3 versions by taking a freeze from one version
(say python-3.5) and cloning that to another (say python-3,4).  This was
written to facilitate the transition from trusty (3.4) to xenial (3.5).

With the transition from xenial (3.5) to bionic (3.6) we need to do
something similar.  To aid the transition it'd be nice to be able to
duplicate a 3.5 freeze to both 3.4 *and* 3.6.  The current version_map
doesn't allow for that.

Enhance the version_map code to accommodate this situation.  The idea
would be to backport this as far as Ocata[1]

[1] We need to look again at separating the requirements code from the
data

Change-Id: I8784509bc162eb6f2e80261bc2d81dbe63ce7989
2018-07-07 00:50:16 +00:00
.zuul.d trigger requirements-check if the lower-constraints.txt file changes 2018-03-29 12:58:57 -04:00
babel-test Babel Integration tests 2016-04-24 17:21:18 -05:00
doc/source Fix bug URL for report a bug link 2017-08-27 10:42:10 +02:00
openstack_requirements Support cloning one freeze to multiple 2018-07-07 00:50:16 +00:00
playbooks skip virtualenv setup when there is already a virtualenv 2018-04-11 20:09:56 +00:00
tools Fix grep-all after we removed minimums from global-requirements 2018-06-29 09:25:37 +10:00
.gitignore add stestr directory to gitignore 2018-05-29 16:50:18 -05:00
.gitreview Added .gitreview 2012-08-06 17:05:39 -05:00
.stestr.conf Move requirements to the PTI 2018-05-21 14:58:48 -07:00
bindep.txt Update bindep.txt for ubuntu-bionic 2018-03-06 22:04:37 -05:00
blacklist.txt Allow Pygments in openstack/requirements 2018-05-17 14:00:54 +07:00
detail.py Trivial: Update pypi url to new url 2018-04-21 02:32:36 +00:00
global-requirements.txt Merge "Proposing distro for global-requirements" 2018-07-03 22:03:23 +00:00
lower-constraints-xfails.txt remove mitmproxy 2018-03-22 13:50:56 -05:00
lower-constraints.txt Merge "Proposing distro for global-requirements" 2018-07-03 22:03:23 +00:00
MANIFEST.in Make openstack.requirements installable. 2013-07-05 00:31:50 -04:00
projects.txt Merge "Add metalsmith to projects.txt" 2018-06-06 04:16:47 +00:00
README.rst add build-lower-constraints command 2018-07-02 11:29:34 -04:00
requirements.txt Updated from global requirements 2018-03-04 10:29:16 +00:00
setup.cfg add build-lower-constraints command 2018-07-02 11:29:34 -04:00
setup.py Updated from global requirements 2017-03-02 11:55:44 +00:00
test-requirements.txt Move requirements to the PTI 2018-05-21 14:58:48 -07:00
tox.ini change tox default to python3 2018-06-28 13:28:36 -04:00
update.py Move commands into cmds subdir. 2015-06-29 15:27:25 +12:00
upper-constraints-xfails.txt remove mitmproxy 2018-03-22 13:50:56 -05:00
upper-constraints.txt Merge "Proposing distro for global-requirements" 2018-07-03 22:03:23 +00:00

Global Requirements for OpenStack Projects

image

Why Global Requirements?

Refer to the Dependency Management section of the Project Team Guide for information about the history of the project and the files involved.

Tools

All the tools require openstack_requirements to be installed (e.g. in a Python virtualenv). They all have help, which is the authoritative documentation.

update-requirements

This will update the requirements in a project from the global requirements file found in .. Alternatively, pass --source to use a different global requirements file:

update-requirements --source /opt/stack/requirements /opt/stack/nova

Entries in all requirements files will have their versions updated to match the entries listed in the global requirements. Excess entries will cause errors in hard mode (the default) or be ignored in soft mode.

generate-constraints

Compile a constraints file showing the versions resulting from installing all of global-requirements.txt:

generate-constraints -p /usr/bin/python2.7 -p /usr/bin/python3 \
  -b blacklist.txt -r global-requirements.txt > new-constraints.txt

edit-constraints

Replace all references to a package in a constraints file with a new specification. Used by DevStack to enable git installations of libraries that are normally constrained:

edit-constraints oslo.db "-e file://opt/stack/oslo.db#egg=oslo.db"

build-lower-constraints

Combine multiple lower-constraints.txt files to produce a list of the highest version of each package mentioned in the files. This can be used to produce the "highest minimum" for a global lower constraints list (a.k.a., the "TJ Maxx").

To use the script, run:

$ tox -e venv -- build-lower-constraints input1.txt input2.txt

Where the input files are lower-constraints.txt or requirements.txt files from one or more projects.

If the inputs are requirements files, a lower constraints list for the requirements is produced. If the inputs are lower-constraints.txt, the output includes the highest version of each package referenced in the files.

Proposing changes

Look at the Review Guidelines and make sure your change meets them.

All changes to global-requirements.txt may dramatically alter the contents of upper-constraints.txt due to adding or removing transitive dependencies. As such you should always generate a diff against the current merged constraints, otherwise your change may fail if it is incompatible with the current tested constraints.

A change to the minimum specified vesion of a library in global-requirements.txt currenty requires adjusting the lower-constraints.txt file alongside with the new constrainted coinstallable version of minimums.

Regenerating involves five steps.

  1. Install the dependencies needed to compile various Python packages:

    sudo apt-get install $(bindep -b)
  2. Create a reference file (do this without your patch applied):

    generate-constraints -p /usr/bin/python2.7 -p /usr/bin/python3 \
      -b blacklist.txt -r global-requirements.txt > baseline
  3. Apply your patch and generate a new reference file:

    generate-constraints -p /usr/bin/python2.7 -p /usr/bin/python3 \
      -b blacklist.txt -r global-requirements.txt > updated
  4. Diff them:

    diff -p baseline updated
  5. Apply the patch to upper-constraints.txt. This may require some fiddling. edit-constraint can do this for you when the change does not involve multiple lines for one package.

Resources