146 Commits

Author SHA1 Message Date
Jimmy McCrory
c4526c0a61 Apply upper-constraints when building venvs
When building a venv for leap upgrades, checkout the applicable
requirements repo SHA and apply its upper-constraints file. If
packages within the venv are installed unconstrained, a project's
database migration scripts may fail.

Also remove tempest from being built within the venv.

Change-Id: Ic7e9aaa2d445bfe22c0f707a00a867cc8e84a38d
2017-04-07 23:50:49 -07:00
Kevin Carter
33da8fc8eb Ensure the components are isolated from the system
This creates a specific slice which all OpenStack services will operate
from. By creating an independent slice these components will be governed
away from the system slice allowing us to better optimise resource
consumption.

See the following for more information on slices:

* https://www.freedesktop.org/software/systemd/man/systemd.slice.html

See for following for more information on resource controls:

* https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html

Tools like ``systemd-cgtop`` and ``systemd-cgls`` will now give us
insight into specific processes, process groups, and resouce consumption
in ways that we've not had access to before. To enable some of this reporting
the accounting options have been added to the [Service] section of the unit
file.

Change-Id: Ife2e28ce6b3e0d0219b8a5ec2ca8d9dbe513d5a7
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-03-28 23:58:51 -05:00
Jenkins
a6f42e7510 Merge "Unify output formatting, add logging" 2017-03-24 18:30:59 +00:00
Jenkins
50f6d1fb56 Merge "Correct conditional for cinder is_metal check" 2017-03-21 19:02:31 +00:00
Weezer Su
db9edda9c7 Correct conditional for cinder is_metal check
change the is_metal == true to is_metal != true

Change-Id: I1836a3f354a951b6761442baf2c09d8d372c459f
2017-03-21 18:58:08 +00:00
Kevin Carter
1e5c0c5cb9
define the default kernel statically
Change-Id: I6d39460de826f73e0ae28e0feb6a173d384583a3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-03-16 19:24:00 -05:00
Nolan Brubaker
c2aeb03b10 Unify output formatting, add logging
Since there is no central storage for the restart actions and the tests
at the moment, they should have timestamped output to correlate events.
Using a standard format will make this easier to do programmatically,
too.

The output is sent both to stdout and a log file. Additionally, Ansible
output is suppressed now, so it doesn't pollute the console. Log times
are in UTC, to avoid timezone mismatches between nodes (observed in
multi-node AIOs).

The `configure_logging` function was copied into the tests/keystone.py
file instead of put into a common module in order to keep that file a
standalone unit for the time being.

Change-Id: I399d1827a8559896ca87e45c00241293ccd033d2
2017-03-15 16:36:32 -04:00
Weezer Su
9a48d3728d Fix the kernel version been deployed on VMs
Base on the http://packages.ubuntu.com/trusty/kernel/, the
linux-image-generic will install the 3.13.0.112.120 [amd64, i386]
kernel. leave it empty will do nothing to the kernel. this is
important when you deploy ubuntu 14.04.5 on VMs which use 4.4 rather
then 3.13.

Change-Id: I7984edaef5a65320ec3b10085646e8ad5c93f65d
2017-03-15 19:37:57 +00:00
Jenkins
eacb01c0a0 Merge "Add rolling downtime simulation tools" 2017-03-10 13:54:23 +00:00
Weezer Su
2a34b74d56 Remove the NETWORK_BASE from build.sh
remove the NETWORK_BASE from build.sh because it already been setup
at the variables.sh

Change-Id: I5143691ea83e9c8f8c7f91e6a56788fa62b4eee4
2017-03-09 15:56:32 -06:00
Nolan Brubaker
0ebc0430d7 Add rolling downtime simulation tools
These tools help simulate rolling downtime by taking down service
containers in a controlled fashion. While this can be done manually,
using a script to inspect the inventory for targets is much easier.
Having the rolling downtime happen automatically is also a bit less
error prone.

Tests for things like response times will be placed in the tests
directory, allowing for scripts to be written against different services
to try different things (like adding resources while rolling downtime
happens).

There is no central orchestration for the two components currently, in
order to keep things simple. Restarting containers and running the tests
is therefore best accomplished in a pair of tmux sessions.

Change-Id: I2d8e3484bbb1a71d2cd0c4124f824d49f76e5c2c
2017-03-09 16:29:13 -05:00
Nolan Brubaker
f93ec3d2be Allow environment variable overrides
As is, the variables.sh file will clobber envvars. Don't do that
anymore.

Change-Id: I6afc0eb354c19ebc96fc51f0dcf06d89aa3cabfe
2017-03-08 16:26:08 -05:00
OpenStack Proposal Bot
3ac8838598 Updated from global requirements
Change-Id: I6176e9d85c7ec9b64403b33bd99abb1c52d14f7a
2017-03-02 11:51:26 +00:00
Jimmy McCrory
ed717932c0 Fix downloading of pre-created leap release venvs
Precreated bundled venvs for OpenStack-Ansible have been created,
update the default VENV_URL with the path to them. Also fix the wget
command used to download them so that it returns a 0 on success and
deletes a failed download allowing the local build function to continue
on.

Change-Id: Ic4ae115384015dd7159da4e5850452bb5db181be
2017-02-23 15:24:56 -08:00
Jenkins
72ef877b93 Merge "Adds in scripts from multinode-aio - Ansible Xenial Deploy" 2017-02-23 20:54:19 +00:00
Rick Box
3b2c7ad457 Adds in scripts from multinode-aio - Ansible Xenial Deploy
Within the multi-node-aio-xenial-ansible folder the multi-node-aio
content has been copied over, Cobbler and Trusty specific parts have
been removed.

The intent here is to replace much of the bash which is present with
Ansible and not support Trusty. To do this a clean separate folder is
required so large changes which are made will not affect the current
working Trusty based install.

Change-Id: Ib64ee168ece4b7a00f950c8d5b1cd6dd4dcd7ba7
2017-02-23 16:47:52 +00:00
Jenkins
a31dc59f17 Merge "Allow unset varaibles when checking http_proxy" 2017-02-23 14:28:21 +00:00
Melissa Kam
fac7844a32 Allow unset varaibles when checking http_proxy
The build.sh script uses set -eu, which causes this script to
fail if http_proxy is not set.

Change-Id: I5a71106db13fcd7c7c5fe5bc3350441e556759c1
2017-02-22 14:59:44 -06:00
Jimmy McCrory
5b7f261735 Fix checkout of cloned leap upgrade repos
Perform a git branch checkout on cloned OSA repo directories when the
ignore-changes.marker file does not exist.

Change-Id: Ib062198f54e399d644e13c7aea7582e0ad6e1976
2017-02-22 12:13:42 -08:00
Jenkins
9d9580d091 Merge "Add kernel option into preseed" 2017-02-22 19:31:10 +00:00
Jenkins
590202f74a Merge "Fixes to leap upgrade preflight function" 2017-02-22 19:27:58 +00:00
Jimmy McCrory
164570a626 Fixes to leap upgrade preflight function
- Check for existence of the configuration directory
  (rpc_deploy/openstack_deploy) since only one should exist, dependent
  on the current stage of the upgrade
- Correct the path to upgrade-requirements.txt
- Use pip to install requirements found in upgrade-requirements.txt
  to avoid a 'double requirement given' error when installing pip

Change-Id: If39826c26eb948b91f14e7e27d0a5d0740c8349c
2017-02-22 10:49:30 -08:00
Melissa Kam
83201edfb5 Remove whitespaces before EOF
Change-Id: I6023839bf10e6b22cf41e7022b736cf9e7387f93
2017-02-21 16:07:44 -06:00
Jenkins
90d1757eaf Merge "Adds more values into variables.sh and places in extra scripts" 2017-02-20 19:57:54 +00:00
Jenkins
439dc5180c Merge "Adds in option to disable overriding of apt-sources list" 2017-02-20 19:57:48 +00:00
Jenkins
4c53790432 Merge "Adds in option to modify DNS - removes hardcoded values" 2017-02-20 16:16:32 +00:00
Rick Box
550b2824cc Adds more values into variables.sh and places in extra scripts
More values have been added to the variables.sh file. None of these have
been adjusted from the default values but this now provides a standard
goto place to have all these values and be able to change with ease.

An include for varibles.sh is now also at the top of each of the script
files.

Change-Id: I2354c87e985115ce0b7f168ab1d056d5032e5cff
2017-02-20 16:07:44 +00:00
Rick Box
d38b7b059a Adds in option to disable overriding of apt-sources list
By default AIO deploy overrides apt-sources, if things like a local
mirror are already set up then this script will override these. This
option allows for the override to be disabled.

Change-Id: Id84c2c5d12153125d7ac6698493c41a840993c5b
2017-02-20 15:32:15 +00:00
Rick Box
d261d1a6e8 Adds in option if proxy is set - push through to containers
This commit allows for a proxy to be pushed to each of the deployed
containers based on the standard environment variable "$http_proxy".

Change-Id: I94dc3a8afda9472f032fbd94991f8b676131cbfb
2017-02-20 15:16:05 +00:00
Rick Box
3baca05501 Adds in option to modify DNS - removes hardcoded values
Some networks may not allow 8.8.8.8 Google DNS servers. This patch
allows this to be specified in the variables file or to pull the value
from the systems resolv.conf file.

Change-Id: I494ab82c41bcd59ea85404d49cb2696029353a0e
2017-02-20 12:49:46 +00:00
Melissa Kam
1b40ea4cf3 Use Trusty sources for any 14.04 build
Change-Id: I42b2a63b899cd1254669569dee04f0821cb08ba0
2017-02-17 09:55:09 -06:00
Jenkins
eb8da37a36 Merge "Adds in Playbook and inventory for Ubuntu 16.04 roles" 2017-02-13 17:36:46 +00:00
Jenkins
a9be7df625 Merge "Adds in proxy role - Ubuntu 16.04" 2017-02-13 08:12:42 +00:00
Jenkins
d8aeb4cfca Merge "Adds in network config - removes hardcoded values" 2017-02-13 08:09:39 +00:00
Rick Box
116867f117 Adds in Playbook and inventory for Ubuntu 16.04 roles
Building on from the previous commit https://review.openstack.org/427869
this playbook calls all the roles and contains the required data to
deploy the pxe build environment.

Change-Id: I1a319b68558a4823832693f38f40921a72be783e
2017-02-13 08:09:35 +00:00
Rick Box
a8a44e2002 Adds in proxy role - Ubuntu 16.04
Building on from the previous commit https://review.openstack.org/427869
this add in a proxy role which sets up the environment and apt in case
this machine is based behind some form of proxy.

Change-Id: I1f2266f6443354aa954438801c0ddc84e78d6994
2017-02-13 08:09:15 +00:00
Jenkins
ef570d14c5 Merge "Updated from global requirements" 2017-02-12 18:53:56 +00:00
Jenkins
40ce84bcbc Merge "Replaces yaml.load() with yaml.safe_load()" 2017-02-12 08:12:48 +00:00
Jenkins
ea898130cc Merge "Replaces yaml.load() with yaml.safe_load()" 2017-02-12 08:12:41 +00:00
OpenStack Proposal Bot
b34253720d Updated from global requirements
Change-Id: I1f950b87447a585cffe44c27a18b27d0eed2daca
2017-02-11 17:48:31 +00:00
Rick Box
952f91f004 Adds in network config - removes hardcoded values
172.29 has the potential to conflict with internal addressing schemes.
Rather than have this hard coded these changes are to allow this to be
configured. Variables file has also been added to allow customization of
defined values.

Change-Id: I0b3e68443e7331b02794c3a54478e131d23fd5ec
2017-02-10 15:27:12 +00:00
OpenStack Proposal Bot
772b4f4239 Updated from global requirements
Change-Id: I2212f923769c0f9cfb64ddc7c2b9d3feecc0ddfb
2017-02-10 05:55:38 +00:00
Weezer Su
0df9b3dab1 Add kernel option into preseed
According to suggestion, add the kernel option in the preseed file,
leave it empty to get the lastest kernel, or specify the kernel
number.

Change-Id: Ie5b8f62a89ef9eb51954f0452becbaa9e81ee973
2017-02-08 10:24:14 -06:00
Jenkins
d388328465 Merge "Adds in TFTP Configure role - Ubuntu 16.04" 2017-02-06 13:41:36 +00:00
Jenkins
ce4f0aa634 Merge "Adds in DHCPD role - Ubuntu 16.04" 2017-02-06 13:41:11 +00:00
Jenkins
50cd9e4ab5 Merge "Adds in Apache2 role - Ubuntu 16.04" 2017-02-06 13:41:06 +00:00
Jenkins
91bea90f41 Merge "Add downgrade kernel script" 2017-02-06 13:41:00 +00:00
ji-xuepeng
62ab658962 Use https instead of http for git.openstack.org
Trivialfix

Change-Id: I41d526e84d2c5da3366f242e6a00f93a1c762cfa
2017-02-06 22:22:41 +08:00
gengchc2
64a549e6db Replaces yaml.load() with yaml.safe_load()
Yaml.load() return Python object may be dangerous if you receive
a YAML document from an untrusted source such as the Internet.
The function yaml.safe_load() limits this ability to simple Python
objects like integers or lists.

Reference:
https://security.openstack.org/guidelines/dg_avoid-dangerous-input-parsing-libraries.html

Change-Id: I78fde872948d6838957e35765c3f182bd4b9b512
2017-02-04 18:21:33 +08:00
gecong1973
af9d8ff093 Replaces yaml.load() with yaml.safe_load()
Yaml.load() return Python object may be dangerous if
you receive a YAML document from an untrusted source
such as the Internet. The function yaml.safe_load()
limits this ability to simple Python objects like
integers or lists.

Reference:
https://security.openstack.org/guidelines/dg_avoid-dangerous-input-parsing-libraries.html

Change-Id: Ib260be0cc604f2272e3c676930bcb307752e142b
2017-02-04 12:05:47 +08:00