162 Commits

Author SHA1 Message Date
Bjoern Teipel
2d2c1f1419 Telegraf playbook fixes
The vm_quota.py is missing as telegraf plugin but still
referenced inside the playbook-influx-telegraf.yml
playbook.
Additionally the my.cnf is not necessary to be present
on the telegraf hosts/containers, in order to function.
The override influxdb_protocol exposes the protocol to
be used for communicating with influxdb, usually HTTP

Change-Id: I90226d02e82d2516be4a4d84baff22e46ce709fb
2017-05-09 15:40:13 -05:00
Jesse Pretorius
f0a94da4d3 Use zuul-cloner for tests repo in OpenStack-CI
When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.

Change-Id: I93a4c523ef01824733f6a28122a3b6f884adb5a6
Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b
2017-05-04 17:13:15 +01:00
Weezer Su
eb5349bc78 Add the public ip to iptables rules
Adding the ip to the filter to make the DNAT request rightly.

Change-Id: I5e229444bd873cad59f5d038b82294c1f6acd853
2017-04-26 14:02:08 +00:00
Jenkins
5af3089f6a Merge "Add option to just configure prerouting rules" 2017-04-26 13:19:46 +00:00
Melissa Kam
0b581feae3 Add option to just configure prerouting rules
Change-Id: Ia297cc98201388231a0a4c4338790dd5734bb432
2017-04-25 14:31:48 -05:00
Jenkins
a17657df55 Merge "Fixing counter based graphs inside the Swift dashboard" 2017-04-25 16:49:50 +00:00
Bjoern Teipel
a4f8d4a972 Fixing counter based graphs inside the Swift dashboard
All counter based graphs inside the the Swift dashboard are fixed
and now correctly showing the timings per second rather than total
values per telegraf flush interval.
Additionally the High Response Time graphs are now using the timing_upper
metrics.

Minor issues inside the playbook-influx-telegraf.yml and telegraf.conf.j2
are fixed to support deployments without optional componentns like ironic,
cinder etc.

Change-Id: I0ac0d2004416cae7a6d137d98ab685b7abc22d3f
2017-04-25 10:41:04 -05:00
Melissa Kam
3dcb39b79d Fix variable exports for deployment
The newline was preventing some of the variables from
being exported properly.

Change-Id: If189f6e288796d828da749c5873983ea46764068
2017-04-21 16:16:56 -05:00
Jenkins
1874464be0 Merge "Add deploy node to multinode-aio enviroment." 2017-04-20 08:09:58 +00:00
Jenkins
7dcbf31979 Merge "Apply upper-constraints when building venvs" 2017-04-20 03:50:02 +00:00
Jenkins
73ae1cc8e1 Merge "Updates and bug fixes" 2017-04-20 03:11:34 +00:00
Weezer Su
06f1368490 Add deploy node to multinode-aio enviroment.
Add deploy node into the multinode-aio env, move the deployment
and HAproxy into it, and NAT the port 22, 80, 443 into the deploy
node.

Change-Id: I7c886f213e6b3398cb4ecbc36a4ee55fb7f7fd10
2017-04-19 16:57:06 -05:00
Bjoern Teipel
335d23f32b Adding Swift Proxy Server Dashboard
Provide initial version of a grafana OpenStack Swift Dashboard for the
Swift Proxy Server. The metrics are gathered by the built-in statsd
functionality of Swift and are forwarded via local telegraf daemons
to the influxDB.

Change-Id: Ieb7df97fbc7534e34ebde5a5fe365ff479de81fe
2017-04-13 16:19:09 -05:00
Jimmy McCrory
c4526c0a61 Apply upper-constraints when building venvs
When building a venv for leap upgrades, checkout the applicable
requirements repo SHA and apply its upper-constraints file. If
packages within the venv are installed unconstrained, a project's
database migration scripts may fail.

Also remove tempest from being built within the venv.

Change-Id: Ic7e9aaa2d445bfe22c0f707a00a867cc8e84a38d
2017-04-07 23:50:49 -07:00
Jenkins
08c6377645 Merge "Add a basic glance API tests" 2017-04-06 15:13:51 +00:00
d34dh0r53
e398389342 Updates and bug fixes
Several minor issues/annoyances fixed

* Changed the http_proxy tests to [ ! -z ] rather than [ -n ]
* Added http_proxy tests to no-cobbler-configure-vms.sh
* Added additional packages that are missing when a trusty OnMetal
  is upgraded to Xenial
* The isc-dhcp-server and atftpd services were not coming up so I
  have added specific restarts for both services to ensure that they
  come up
* Updated variables.sh to respect user set overrides, defaults have
  not been altered

Change-Id: I23ee641c2b6a36b18f4486f62af569d1ad28d5dc
2017-03-31 08:21:45 -05:00
Kevin Carter
33da8fc8eb Ensure the components are isolated from the system
This creates a specific slice which all OpenStack services will operate
from. By creating an independent slice these components will be governed
away from the system slice allowing us to better optimise resource
consumption.

See the following for more information on slices:

* https://www.freedesktop.org/software/systemd/man/systemd.slice.html

See for following for more information on resource controls:

* https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html

Tools like ``systemd-cgtop`` and ``systemd-cgls`` will now give us
insight into specific processes, process groups, and resouce consumption
in ways that we've not had access to before. To enable some of this reporting
the accounting options have been added to the [Service] section of the unit
file.

Change-Id: Ife2e28ce6b3e0d0219b8a5ec2ca8d9dbe513d5a7
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-03-28 23:58:51 -05:00
Jenkins
a6f42e7510 Merge "Unify output formatting, add logging" 2017-03-24 18:30:59 +00:00
Jenkins
50f6d1fb56 Merge "Correct conditional for cinder is_metal check" 2017-03-21 19:02:31 +00:00
Weezer Su
db9edda9c7 Correct conditional for cinder is_metal check
change the is_metal == true to is_metal != true

Change-Id: I1836a3f354a951b6761442baf2c09d8d372c459f
2017-03-21 18:58:08 +00:00
Nolan Brubaker
ede1af70b9 Add a basic glance API tests
One script will loop over the glance image list and collect response
times in a common logging format.

The other attempts to upload a small binary file to the glance store in
order to see a 'real' use case.

Change-Id: I2d2aabb802774e28746be7eabafb0578998fdd37
2017-03-20 15:36:15 -04:00
Kevin Carter
1e5c0c5cb9
define the default kernel statically
Change-Id: I6d39460de826f73e0ae28e0feb6a173d384583a3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-03-16 19:24:00 -05:00
Nolan Brubaker
c2aeb03b10 Unify output formatting, add logging
Since there is no central storage for the restart actions and the tests
at the moment, they should have timestamped output to correlate events.
Using a standard format will make this easier to do programmatically,
too.

The output is sent both to stdout and a log file. Additionally, Ansible
output is suppressed now, so it doesn't pollute the console. Log times
are in UTC, to avoid timezone mismatches between nodes (observed in
multi-node AIOs).

The `configure_logging` function was copied into the tests/keystone.py
file instead of put into a common module in order to keep that file a
standalone unit for the time being.

Change-Id: I399d1827a8559896ca87e45c00241293ccd033d2
2017-03-15 16:36:32 -04:00
Weezer Su
9a48d3728d Fix the kernel version been deployed on VMs
Base on the http://packages.ubuntu.com/trusty/kernel/, the
linux-image-generic will install the 3.13.0.112.120 [amd64, i386]
kernel. leave it empty will do nothing to the kernel. this is
important when you deploy ubuntu 14.04.5 on VMs which use 4.4 rather
then 3.13.

Change-Id: I7984edaef5a65320ec3b10085646e8ad5c93f65d
2017-03-15 19:37:57 +00:00
Jenkins
eacb01c0a0 Merge "Add rolling downtime simulation tools" 2017-03-10 13:54:23 +00:00
Weezer Su
2a34b74d56 Remove the NETWORK_BASE from build.sh
remove the NETWORK_BASE from build.sh because it already been setup
at the variables.sh

Change-Id: I5143691ea83e9c8f8c7f91e6a56788fa62b4eee4
2017-03-09 15:56:32 -06:00
Nolan Brubaker
0ebc0430d7 Add rolling downtime simulation tools
These tools help simulate rolling downtime by taking down service
containers in a controlled fashion. While this can be done manually,
using a script to inspect the inventory for targets is much easier.
Having the rolling downtime happen automatically is also a bit less
error prone.

Tests for things like response times will be placed in the tests
directory, allowing for scripts to be written against different services
to try different things (like adding resources while rolling downtime
happens).

There is no central orchestration for the two components currently, in
order to keep things simple. Restarting containers and running the tests
is therefore best accomplished in a pair of tmux sessions.

Change-Id: I2d8e3484bbb1a71d2cd0c4124f824d49f76e5c2c
2017-03-09 16:29:13 -05:00
Nolan Brubaker
f93ec3d2be Allow environment variable overrides
As is, the variables.sh file will clobber envvars. Don't do that
anymore.

Change-Id: I6afc0eb354c19ebc96fc51f0dcf06d89aa3cabfe
2017-03-08 16:26:08 -05:00
OpenStack Proposal Bot
3ac8838598 Updated from global requirements
Change-Id: I6176e9d85c7ec9b64403b33bd99abb1c52d14f7a
2017-03-02 11:51:26 +00:00
Jimmy McCrory
ed717932c0 Fix downloading of pre-created leap release venvs
Precreated bundled venvs for OpenStack-Ansible have been created,
update the default VENV_URL with the path to them. Also fix the wget
command used to download them so that it returns a 0 on success and
deletes a failed download allowing the local build function to continue
on.

Change-Id: Ic4ae115384015dd7159da4e5850452bb5db181be
2017-02-23 15:24:56 -08:00
Jenkins
72ef877b93 Merge "Adds in scripts from multinode-aio - Ansible Xenial Deploy" 2017-02-23 20:54:19 +00:00
Rick Box
3b2c7ad457 Adds in scripts from multinode-aio - Ansible Xenial Deploy
Within the multi-node-aio-xenial-ansible folder the multi-node-aio
content has been copied over, Cobbler and Trusty specific parts have
been removed.

The intent here is to replace much of the bash which is present with
Ansible and not support Trusty. To do this a clean separate folder is
required so large changes which are made will not affect the current
working Trusty based install.

Change-Id: Ib64ee168ece4b7a00f950c8d5b1cd6dd4dcd7ba7
2017-02-23 16:47:52 +00:00
Jenkins
a31dc59f17 Merge "Allow unset varaibles when checking http_proxy" 2017-02-23 14:28:21 +00:00
Melissa Kam
fac7844a32 Allow unset varaibles when checking http_proxy
The build.sh script uses set -eu, which causes this script to
fail if http_proxy is not set.

Change-Id: I5a71106db13fcd7c7c5fe5bc3350441e556759c1
2017-02-22 14:59:44 -06:00
Jimmy McCrory
5b7f261735 Fix checkout of cloned leap upgrade repos
Perform a git branch checkout on cloned OSA repo directories when the
ignore-changes.marker file does not exist.

Change-Id: Ib062198f54e399d644e13c7aea7582e0ad6e1976
2017-02-22 12:13:42 -08:00
Jenkins
9d9580d091 Merge "Add kernel option into preseed" 2017-02-22 19:31:10 +00:00
Jenkins
590202f74a Merge "Fixes to leap upgrade preflight function" 2017-02-22 19:27:58 +00:00
Jimmy McCrory
164570a626 Fixes to leap upgrade preflight function
- Check for existence of the configuration directory
  (rpc_deploy/openstack_deploy) since only one should exist, dependent
  on the current stage of the upgrade
- Correct the path to upgrade-requirements.txt
- Use pip to install requirements found in upgrade-requirements.txt
  to avoid a 'double requirement given' error when installing pip

Change-Id: If39826c26eb948b91f14e7e27d0a5d0740c8349c
2017-02-22 10:49:30 -08:00
Melissa Kam
83201edfb5 Remove whitespaces before EOF
Change-Id: I6023839bf10e6b22cf41e7022b736cf9e7387f93
2017-02-21 16:07:44 -06:00
Jenkins
90d1757eaf Merge "Adds more values into variables.sh and places in extra scripts" 2017-02-20 19:57:54 +00:00
Jenkins
439dc5180c Merge "Adds in option to disable overriding of apt-sources list" 2017-02-20 19:57:48 +00:00
Jenkins
4c53790432 Merge "Adds in option to modify DNS - removes hardcoded values" 2017-02-20 16:16:32 +00:00
Rick Box
550b2824cc Adds more values into variables.sh and places in extra scripts
More values have been added to the variables.sh file. None of these have
been adjusted from the default values but this now provides a standard
goto place to have all these values and be able to change with ease.

An include for varibles.sh is now also at the top of each of the script
files.

Change-Id: I2354c87e985115ce0b7f168ab1d056d5032e5cff
2017-02-20 16:07:44 +00:00
Rick Box
d38b7b059a Adds in option to disable overriding of apt-sources list
By default AIO deploy overrides apt-sources, if things like a local
mirror are already set up then this script will override these. This
option allows for the override to be disabled.

Change-Id: Id84c2c5d12153125d7ac6698493c41a840993c5b
2017-02-20 15:32:15 +00:00
Rick Box
d261d1a6e8 Adds in option if proxy is set - push through to containers
This commit allows for a proxy to be pushed to each of the deployed
containers based on the standard environment variable "$http_proxy".

Change-Id: I94dc3a8afda9472f032fbd94991f8b676131cbfb
2017-02-20 15:16:05 +00:00
Rick Box
3baca05501 Adds in option to modify DNS - removes hardcoded values
Some networks may not allow 8.8.8.8 Google DNS servers. This patch
allows this to be specified in the variables file or to pull the value
from the systems resolv.conf file.

Change-Id: I494ab82c41bcd59ea85404d49cb2696029353a0e
2017-02-20 12:49:46 +00:00
Melissa Kam
1b40ea4cf3 Use Trusty sources for any 14.04 build
Change-Id: I42b2a63b899cd1254669569dee04f0821cb08ba0
2017-02-17 09:55:09 -06:00
Jenkins
eb8da37a36 Merge "Adds in Playbook and inventory for Ubuntu 16.04 roles" 2017-02-13 17:36:46 +00:00
Jenkins
a9be7df625 Merge "Adds in proxy role - Ubuntu 16.04" 2017-02-13 08:12:42 +00:00
Jenkins
d8aeb4cfca Merge "Adds in network config - removes hardcoded values" 2017-02-13 08:09:39 +00:00