The volume of logs we collect in CI jobs are extensive. This
patch ensures that the logs collected are compressed to reduce
the space taken in CI systems.
PYTHONUNBUFFERED is also set to ensure that the console log
from the CI jobs is recorded in the exact order of execution.
Change-Id: I60842285cd915d3c61b54003328408a8fc3d68f2
Related-Bug: #1620849
In order to make it easier to differentiate between the lists of
python packages, distribution packages, downloaded packages,
package pins and other similar variables the variable names are
being changed to ensure that they have a more explicit suffix
that defines the purpose and makes the naming more consistent.
This is to facilitate a lookup plugin which will be able to look
up all the package lists and present them as a consolidated piece
of data which may be used for artifact preparation.
Change-Id: I85fe0d5f4235c5d71ff1e8fc4066a9d1e0932684
This change enables log collection within the gate so that further analysis
on gate tasks can be performed post build. This is very useful when
debugging problems.
Change-Id: I8e95d9aa322ea1f1006dcbca169cc518c0e1361d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
It was duplicating the meaning/purpose of the role
variable `rally_role_project_group`
Also ensure that additional tasks are limited to only
a single host in the host group.
Change-Id: I3d5cc822cc0d3c2b0b3ba7b05a9fe1b6b9e3a839
As per [1] all linting tests can now use upper-constraints. This patch
removes all instances of the install_command override relating to lint
testing which were needed to negate the use of upper-constraints.
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html
Change-Id: Ib3576a0333dee510d3f08aca6f4a9bbb1eeda4e5
The numerous tags within the role have been condensed
to two tags: rally-install and rally-config
These tags have been chosen as they are namespaced
and cover the two major functions of the role.
Change-Id: I781eb1edc8ea1d8bb093a579b9bb88498ed2d534
The default value of "rally_all" remains, but this
allows deployers to target the role to multiple
groups now.
Change-Id: Id1bbda227032f66270fe5656250be7794a1dd4af
Some Linux distributions, such as CentOS 7 and Xenial, have trouble
validating SSL certificates when using get_url with servers
that use Server Name Indication (SNI).
This patch adds those packages to the list of required packages and
uses bindep to install them in developer test environments the same
way that the gate tests install them.
Change-Id: Ic3bf4b4238c1c6260d3c8dc0e4800ff26691ae93
The role gate will now actually run simple
Rally scenario against Keystone to validate that
the install and configuration is correct.
Change-Id: Ie5d52636700e8c276b0e793e815ae513d25415bb
This adds python packages required for SNI support to ensure that tox
installs the packages into the venv when running tests.
Change-Id: I320e63df935ffa9ad0b561f16d7e42c2f03c7aac
The plugin repo needs to be cloned for ansible-lint to
understand ansible plugins. The commands currently
reside in tox.ini under the ansible section and are not
currently included. This commit fixes that error.
Change-Id: I7d209c68ec1bf6760f84048d1f9983476ff82e68
OSLO logging currently defaults the 'use_stderr' option to True
which results duplicate logs in service daemon logs for both
upstart and systemd. To correct this issue the use_stderr
option has been set to false.
Change-Id: I3799265a48c8f92f9cef77a10b3f02fa6aa66b7f
Closes-Bug: 1588051
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch adds a var used by the repo_build process to determine
which host group to check for members of before building the pip
packages required by this role. This provides the ability to
optimise the repo build process to only building the packages
that are required for an environment.
Depends-On: Idda16b4f382eee57c7469af898859d6d81d4eb30
Change-Id: Iedb4c0c91daa291f881a9fc00c5b8c40bb312ce3
The current method of installing the distribution packages required is
set in the tasks and cannot be changed by a deployer.
Currently the apt task always installs the latest package. This results
in unexpected binary changes when a deployer may simply be trying to
execute a configuration change.
This patch adds the ability for a deployer to change the desired state
so that the results are predictable.
Change-Id: I1708ca8285ffa2bbd1a989b187ef3c6d9dd005c2
The 'docs' tox target executes the doc8 lint test which may result in
failures when testing documentation builds, but OpenStack-CI does not
execute that tox target.
In order to ensure that we catch all standard documentation syntax
errors and prevent them from merging, this patch includes the docs
target in the 'linters' chain of tests.
Fixes for any failures which result from executing this test are also
included in the patch.
Change-Id: Iacff67847b0708a5d41ab3e1accf59f40326497d
Just noting in the role metadata that the role supports Xenial
Change-Id: I70fba8cd35f13405a0261d3aafab4681f2ca3ebf
Implements: blueprint support-ubuntu-1604
The openstack_hosts role is not used in any of the test playbooks
or required by dependent roles and can be removed from the test
role requirements.
Change-Id: Id9395e745249c723c5d8135ae4c9661dcb0f7221
Unlike the Ansible apt module, the Ansible pip module does not
recognise a with_items list and process all the items at once.
To optimise the pip install tasks, this patch replaces the use
of with_items with a join filter so that the pip install task
does an install with all the packages in a list, ensuring that
the execution is one action instead of many.
Change-Id: I403e36a67d2e06f5cd32fe28222b597a17ef42a1
With the implementation of https://review.openstack.org/321331 the
human_log callback plugin is now part of the plugins repo.
This patch removes the retrofitted version in tox in favor of using
the version from the plugins repo instead.
Change-Id: I173f065f004d3336f6249625c8c36a609742389e
In order to prepare for the move of detailed configuration
content from the install guide to the roles, doc8 checks
are being implemented in the roles to ensure that any new
content submitted is properly checked.
Change-Id: I4da740890257f083b4ba12ff4ed321933109d01b
Partially-Implements: blueprint osa-install-guide-overhaul
This patch updates the sphinx configuration for docs and releasenotes
to make it easier to replicate across repositories and to comply with
pep8 testing without exceptions.
Change-Id: I3a80b33771e4142dae589b7f6fbf0f94255cabc8
In order to prepare for implementing requirements management by the
OpenStack requirements management process, and to improve the
reliability and effectiveness of test execution, this patch implements
some changes to the tox configuration:
- The minimum tox version is increased in order to be able to use
constraints for the python packages.
- The OpenStack upper-constraints are used when preparing the test
venv for the linters checks.
- Any proxy environment variables set on the test host are passed
into the venv to enable testing from behind a proxy.
- The environment variables used by Ansible tests are moved into
a new venv called 'ansible' and this environment is inherited
by all Ansible-related tests.
- The docs test will clean-up an existing build directory before
executing the docs build.
- The releasenotes build cannot use upper-constraints at this point,
so it doesn't.
- The Ansible role download will no longer ignore errors so that any
problems discovered will result in a failed test.
- The human readable logging callback plugin is implemented for
functional testing.
- The ansible test requirements are moved into tox.ini to ensure
compliance for requirements.txt/test-requirements.txt for the
global-requirements management contract.
- The ~/.ansible directory as a whole is not deleted. Instead only
the plugins and roles folders are deleted to ensure that zuul's
Ansible artifacts are left in-place.
- The ansible-lint version is updated to support execution against a
folder, and the test now executes against the entire role to ensure
that it captures all applicable files for lint testing.
This is a combined port of the following:
- https://review.openstack.org/323507
- https://review.openstack.org/338193
- https://review.openstack.org/332443
- https://review.openstack.org/338193
- https://review.openstack.org/339493
Change-Id: I03a60477abffb2b1c4a2dc7bd2979a22483a25a7
Remove all tasks and variables related to toggling between installation
of rally inside or outside of a Python virtual environment.
Installing within a venv is now the only supported deployment.
Additionally, a few changes have been made to make the creation of the
venv more resistant to interruptions during a run of the role.
* unarchiving a pre-built venv will now also occur when the venv
directory is created, not only after being downloaded
* virtualenv-tools is run against both pre-built and non pre-built venvs
to account for interruptions during or prior to unarchiving
Change-Id: I815dff623fce1440a5d60f64bf69f4f563371998
Implements: blueprint only-install-venvs
The pip_install and pip_lock_down roles have been merged.
Remove pip_lock_down from the role's meta dependencies and test
requirements.
Change-Id: Idf70fd3e14254ae7b633ca2731388dec307597da
A new release of flake8 is causing lint faiures. Our requirements
should match OpenStack requirements anyway.
This patch pins to the current master requirements from the
OpenStack requirements repository using global-requirements.
Change-Id: I722af00a4a530987a85c44fc5ae143b009eeb3ee
A new release of flake8 is causing lint faiures. Stable branches
should have requirements pinned anyway.
This patch pins to the current stable/mitaka requirements from
the OpenStack requirements repository.
Change-Id: Iefeb5e7e9b0f29fdcc8267e4d2a52a96ccf31fda
Debian-specific vars and logic have been moved to tasks
that will execute only on those distributions.
Change-Id: I6a9d04382b1f1badebf12483ce7d1d02b43f8903
Implements: blueprint multi-platform-host
Currently all pip install tasks only require the package to be
present. This means that when an environment undergoes a minor
upgrade the package is not upgraded to the same version that
was tested with. This ultimately results in a deployed
environment that does not match the tested environment.
While for the services installed into venvs this is not an
issue, it does affect those which do not use venvs and any
packages which are installed outside of a venv or on top
of a venv.
This patch changes the behaviour to ensure that the install
task will always use the latest available package. In
developer_mode this will mean using the version specified
in upper-constraints, and in an integrated build this will
mean the version which is available in the wheel repo's
folder for the tag.
Change-Id: I04455f7323e6c565e46203494876e1812c5d0c1e
Paramiko version 2.0 has been released. It now uses the Python library
cryptography. Installing this requires additional system packages. This
commit adds in the appropriate packages required by cryptography based
on its documentation [1].
An alternative approach would have been to constrain the version of
Paramiko however the project describes the 1.x versions as relying on
insecure dependencies [2].
[1] https://cryptography.io/en/latest/installation/
[2] http://www.paramiko.org/installing.html
Change-Id: I9ea9f7cbe946c9141189c4bcb9ba98656c8495b4
