openstack/ansible-role-python_venv_build
Code Issues Proposed changes
Ansible role to manage the building of python venvs
167 Commits 10 Branches 9 Tags 2.7 MiB
Python 73.4%
Shell 26.6%
880ed4b185
Go to file
Jesse Pretorius (odyssey4me) 880ed4b185 Only create local facts folder when necessary 
If we do not plan to set any local facts, we should
not be creating this folder. If we do need to set the
facts, then the tasks will need to be run via sudo to
ensure that they has the appropriate rights to create
the folder/files under /etc.

If this change is not implemented, and the playbook
running this role is not executed as root, then the
task fails because it does not have rights to create
the folder - even when there are no facts to set.

Change-Id: Idcb093775507728a7c68a877a05b8895d4c8c39f
2020-07-29 08:31:08 +00:00
defaults Use python3 version by default 2020-02-24 13:24:04 +01:00
doc Merge "use newer openstackdocstheme and reno versions" 2020-06-05 10:58:42 +00:00
handlers Revise the role to properly cater to use-cases 2018-05-04 18:03:59 +01:00
meta Add integrated coverage 2019-05-21 16:48:29 +02:00
releasenotes use newer openstackdocstheme and reno versions 2020-06-03 18:56:42 +02:00
tasks Only create local facts folder when necessary 2020-07-29 08:31:08 +00:00
tests remove ansible_python_interpreter 2020-06-11 15:58:50 -04:00
vars Ensure g++ is present on debian/ubuntu for wheel builds 2020-03-11 14:35:01 +00:00
zuul.d Switch to CentOS-8 based tripleo Job 2020-03-27 14:37:38 +05:30
.gitignore Updated from OpenStack Ansible Tests 2019-08-20 03:07:42 +00:00
.gitreview OpenDev Migration Patch 2019-04-19 19:30:04 +00:00
bindep.txt Updated from OpenStack Ansible Tests 2020-05-22 07:12:14 +00:00
CONTRIBUTING.rst [ussuri][goal] Update contributor documentation 2020-05-13 00:10:31 +03:00
LICENSE Initial commit 2018-03-09 17:50:21 +00:00
README.rst Update python_venv_build role docs 2019-03-21 15:17:51 +00:00
run_tests.sh Updated from OpenStack Ansible Tests 2019-07-18 22:21:24 +00:00
setup.cfg Cleanup py27 support 2020-04-25 15:17:06 +02:00
setup.py Cleanup py27 support 2020-04-25 15:17:06 +02:00
tox.ini Cleanup py27 support 2020-04-25 15:17:06 +02:00
Vagrantfile Updated from OpenStack Ansible Tests 2019-07-20 09:42:29 +00:00

README.rst

Team and repository tags

image

OpenStack-Ansible python_venv_build

This Ansible role prepares a python venv for use within the OpenStack-Ansible project, but it may be used for other projects as well.

The role requires the following to be present prior to execution:

  • virtualenv >= 1.10 (to support using the never-download option)
  • pip >= 7.1 (to support using the constraints option) in the virtualenv once it has been created.

Use-cases

This role is built for the following use-cases:

  1. Using a build host (a.k.a. repo server):
    • Build python wheels on a repo server with a given list of python packages.
    • Prepare a requirements.txt and constraints.txt file on the repo server, and use them to ensure that the build and installation processes are both consistent and idempotent.
    • On the build host, install the distribution packages required at build time.
    • On any number of target hosts, create a virtualenv and install these built wheels into it using the pip --find-links option.
    • On any number of target hosts, install the distribution packages required at run time.
    • Re-use previously built wheels to speed up any subsequent builds..
  2. Not using a build host:
    • On any number of target hosts, create a virtualenv, then locally install the distribution packages required at build and run time, then locally compile and install the given list of python packages.
    • This negates the need for a repo server, but takes longer due to the increased number of dependencies to install and the compilation happening on every target host.
    • The only situation where a build host provides no benefit is where there is only a single target host (with no containers) and none of the packages installed into the venv will be used again for any other venvs built by this role on the same host.

It may be useful to review the Python Build/Install Process Simplification specification to understand the background that led to the creation of this role.

Process

  1. Pre-requisites are checked.
  2. If wheel building is enabled, and there is a repo server in the environment, then the following happens on the repo server:
    1. The distribution packages required to execute the python wheel compile are installed.
    2. A set of requirements and source-constraints for the venv are compiled for pip to use when building the wheels. These are also used to determine whether there are changes to either for the purpose of idempotence.
    3. The python wheels are compiled, and an install-time constraints file is created. The install-time constraints file has the list of python packages with their versions - this differs from the source-constraints which may contain git SHA's.
  3. The installation of the python packages then commences on the target hosts:
    1. If the wheel build was enabled:
      1. Only the distribution packages required at runtime by the python packages are installed.
      2. A python venv is created at venv_install_destination_path.
      3. The requirements and constraints files for the venv are prepared in the venv path.
      4. The python packages are installed from the wheels on the repo server using pip's --find-links option to ensure that they are preferred above the default pypi index.
      5. If there are any venv_packages_to_symlink then the appropriate python libraries installed into the system from those packages will be symlinked into the virtualenv. This provides for python libraries which have a tight coupling with C bindings which may not be portable as a wheel.
    2. If the wheel build was not enabled:
      1. The distribution packages required for compiling and at runtime by the python packages are installed.
      2. A python venv is created at venv_install_destination_path.
      3. The requirements and constraints files for the venv are prepared in the venv path. The constraints file in this case would contain the same content as the source-constraints file on the repo server where there is one.
      4. The python packages are installed from the default pip index. During the installation pip will do a git clone and build from it for any packages that have a git SHA as a constraint.
      5. If there are any venv_packages_to_symlink then the appropriate python libraries installed into the system from those packages will be symlinked into the virtualenv. This provides for python libraries which have a tight coupling with C bindings which may not be portable as a wheel.
  4. If any venv_facts_when_changed are set, then they are implemented on the target host in /etc/ansible/facts.d.

Default variables

../../defaults/main.yml

References

Documentation for the project can be found at:

https://docs.openstack.org/ansible-role-python_venv_build/latest/

The project home is at:

http://launchpad.net/openstack-ansible

Release notes for the project can be found at:

https://docs.openstack.org/releasenotes/ansible-role-python_venv_build/