246 Commits

Author SHA1 Message Date
Zuul
da98030909 Merge "add ulimit support for kolla_docker" 2019-01-27 10:17:30 +00:00
Zuul
67d496b0e6 Merge "[prometheus] Support the prometheus openstack exporter" 2019-01-24 15:12:42 +00:00
Zuul
58772d53d5 Merge "Remove Vitrage Collector" 2019-01-24 09:34:52 +00:00
binhong.hua
3d3f5f1613 add ulimit support for kolla_docker
By default, docker containers inherit ulimit from limits of docker
deamon. On CentOS 7, docker daemon default NOFILE is 1048576.
It can found in /usr/lib/systemd/system/docker.service.
The big limit will cause many problem. we should control it in
production environment.

Change-Id: Iab962446a94ef092977728259d9818b86cfa7f68
2019-01-24 10:54:14 +08:00
Bartosz Zurkowski
7e85677020 Remove Vitrage Collector
Vitrage Collector service has been removed from Vitrage in change:
Ie713456b2df96e24d0b15d2362a666162bfb4300.

Change-Id: I45023940c1d2573bfed49d4ce3fac16ed2d559e4
Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>
Co-Authored-By: Kien Nguyen <kiennt65@viettel.com.vn>
2019-01-23 09:39:49 +07:00
Jorge Niedbalski
6c64b7c732 [prometheus] Support the prometheus openstack exporter
This patch implements the initial support for the
openstack-exporter[0] in the kolla-ansible
prometheus monitoring system.

The configuration and prechecks are reused from the other
exporters and a new template is provided for generating
a os-client-config file required by the exporter.

The default scrape interval is 60 seconds, but it can
be extended via a configuration option.

[0] https://github.com/Linaro/openstack-exporter

Change-Id: I4a34c4bb56e74b5cd544972cbd6540d9acb6e4a1
2019-01-21 10:41:35 -03:00
Mark Goddard
9c906a175b Wait for volume attachment in CI before detach
In some cases the volume attachment is not finished by the time we
attempt to detach it, causing the following error:

Volume status must be 'in-use' and attach_status must be 'attached' to detach.

This change waits for the volume to be attached before detaching.

Change-Id: Id23d114cf8a2b40f8f0784cbfb84e9fa0829adc0
2019-01-18 13:53:13 +00:00
Zuul
e9fda8664e Merge "Add Kolla Ceph filestore CI jobs" 2019-01-07 11:35:54 +00:00
tone.zhang
6786bc5b77 Add Kolla Ceph filestore CI jobs
Since Rocky bluestore is the default store type used by Kolla Ceph.
Filestore is still used by the customers. The patch allows deploy
both Ceph bluestore OSD and Ceph filestore OSD in Kolla-ansible CI
jobs.

Change-Id: Iaa7c1b2ff35eb649c3a6d9f7266463d8961cdc9e
Signed-off-by: tone.zhang <tone.zhang@arm.com>
2019-01-01 13:32:16 +08:00
Zuul
69666b6665 Merge "Support Docker CE in bootstrap-servers" 2018-12-28 12:21:26 +00:00
Eduardo Gonzalez
535aba79a9 Add NFV orchestration services CI job
Change-Id: I33484d0961f1cd17ed05dd124eba68d99b4db83f
2018-12-21 11:43:22 +01:00
Zuul
de71cf2f21 Merge "Test zun container creation in gates" 2018-12-20 09:31:49 +00:00
Eduardo Gonzalez
362b6ee40e Test zun container creation in gates
Change-Id: If5b4ba975a65e07d2704eb6bdb9d841d6a9c3d42
2018-12-19 19:50:59 +01:00
Mark Goddard
48aea5637f Support Docker CE in bootstrap-servers
Kolla Ansible's bootstrap-servers command provides support for
installing the Docker engine. This is currently done using the packages
at https://apt.dockerproject.org and https://yum.dockerproject.org.
These packages are outdated, with the most recent packages from May 2017
- docker-engine-17.05.

The source for up to date docker packages is
https://download.docker.com, which was introduced with the move to
Docker Community Edition (CE) and Docker Enterprise Edition (EE).

This change adds support to bootstrap-servers for Docker CE for CentOS
and Ubuntu.

It also adds a new variable, 'enable_docker_repo', which controls
whether a package repository for Docker will be enabled.

It also adds a new variable, 'docker_legacy_packages', which controls
whether the legacy packages at dockerproject.org will be used or the
newer packages at docker.com. The default value for this variable is
'false', meaning to use Docker CE.

Upgrading from docker-engine to docker-ce has been tested on CentOS 7.5
and Ubuntu 16.04, by running 'kolla-ansible bootstrap-servers' with
'docker_legacy_packages' set to 'false'. The upgrades were successful,
but result in all containers being stopped. For this reason, the
bootstrap-servers command checks running containers prior to upgrading
packages, and ensures they are running after the package upgrade is
complete.

As mentioned in the release note, care should be taken when upgrading
Docker with clustered services, which could lose quorum. To avoid this,
use --serial or --limit to apply the change in batches.

Change-Id: I6dfd375c868870f8646ef1a8f02c70812e8f6271
Implements: blueprint docker-ce
2018-12-17 14:04:43 +00:00
Mark Goddard
365bb5177d Create cells before starting nova services
Nova services may reasonably expect cell databases to exist when they
start. The current cell setup tasks in kolla run after the nova
containers have started, meaning that cells may or may not exist in the
database when they start, depending on timing. In particular, we are
seeing issues in kolla CI currently with jobs timing out waiting for
nova compute services to start. The following error is seen in the nova
logs of these jobs, which may or may not be relevant:

No cells are configured, unable to continue

This change creates the cell0 and cell1 databases prior to starting nova
services.

In order to do this, we must create new containers in which to run the
nova-manage commands, because the nova-api container may not yet exist.
This required adding support to the kolla_docker module for specifying a
command for the container to run that overrides the image's command.

We also add the standard output and error to the module's result when a
non-detached container is run. A secondary benefit of this is that the
output of bootstrap containers is now displayed in the Ansible output if
the bootstrapping command fails, which will help with debugging.

Change-Id: I2c1e991064f9f588f398ccbabda94f69dc285e61
Closes-Bug: #1808575
2018-12-14 19:26:42 +00:00
Zuul
d4dd1f0ac4 Merge "Deploy mistral-event-engine service" 2018-12-11 08:04:40 +00:00
Eduardo Gonzalez
846c15d8e1 Allow set tty for containers
This change adds support to comfigure tty,
it was enabled by default but a recent patch
removed it. Some services such as Karaf in opendaylight
requires a TTY during startup.

Closes-Bug: #1806662
Change-Id: Ia4335523b727d0e45505cbb1efb40ccf04c27db7
2018-12-05 11:44:37 +00:00
Eduardo Gonzalez
a7dbc39240 Suppress log copy output in gates
Change-Id: I01e58d3548d6adc4a2d6f1088773df7941da3865
2018-11-20 11:14:24 +00:00
Zuul
d0bb2fdcba Merge "Don't allocate pseudo-TTY when creating containers" 2018-11-19 12:18:15 +00:00
Zuul
5281e9e114 Merge "Use openstack command in CI" 2018-12-03 10:17:45 +00:00
Zuul
63eccc6372 Merge "Remove the deprecate the Glance Registry" 2018-12-01 07:08:36 +00:00
Mark Goddard
07d5351812 Use openstack command in CI
Switch the CI test script to use the openstack command rather than nova
etc.

Change-Id: Id1c6d35e1e50987d230adb7942da6a7ae5c0f050
2018-11-29 16:01:36 +00:00
caoyuan
d154919865 Remove the memcache container restart
the bug[0] has beed fix, restart container is unncessary, remove it
is safe.

[0]: https://bugs.launchpad.net/oslo.cache/+bug/1590779

Change-Id: Iadfb3666bad8160b5c071fa1a512e2dcdda84043
2018-11-26 23:43:39 +08:00
Zuul
c6e1ae982d Merge "Add neutron metering role into kolla-ansible" 2018-11-26 11:45:45 +00:00
caoyuan
03fd9715c5 Remove the deprecate the Glance Registry
A spec to Deprecate the Glance Registry Service[0] was accepted in Newton,
but it contained the ambiguous statement, "Mark the service as deprecated
and ready for removal in the Q release." kolla-ansible disable the
glance-registry in Q release[1], and since we are in S now,
remove glance-registry is safe.

[0]: http://specs.openstack.org/openstack/glance-specs/specs/newton/approved/glance/deprecate-registry.html
[1]: https://review.openstack.org/#/c/566804/

Change-Id: I48f794029e97aa6f76bbd500e33f28f51a3f2ac4
2018-11-21 20:51:51 +08:00
caoyuan
a5badc89f2 Add neutron metering role into kolla-ansible
Refer to neutron docs [1][2]
[1]: https://docs.openstack.org/neutron/pike/admin/archives/adv-config.html#l3-metering-service-driver
[2]: https://wiki.openstack.org/wiki/Neutron/Metering/Bandwidth
Co-Authored-By: ZhijunWei <wzj334965317@outlook.com>

Change-Id: I4a676d041bc2a86497cb139d9347365738b156df
2018-11-20 23:36:45 +08:00
Eduardo Gonzalez
26af377a3b Refactor tests/templates/globals.yml
Improve where openstack services are deployed
to a variable, so we will avoid many conditions
when more CI jobs are added with different configs.
Heat is enabled by default in globals.yml, lets deploy it
at least.

Change-Id: If0f009cf434c3ed5874b3171d8c75911ef8e4be1
2018-11-08 10:15:38 +00:00
Eduardo Gonzalez
db4e25836d Deploy mistral-event-engine service
This PS adds deployment of mistral-event-engine service
within mistral role.

Change-Id: If187e1a49a321d1e85fb3cfff2ea037e567b30dc
2018-11-06 17:50:54 +00:00
Mark Goddard
f92de48c28 Refactor CI testing scripts
This change is intended to ease the addition of new test scenarios, by
splitting the existing setup_gate.sh script up into several smaller
scripts. We may wish to further break these scripts up in future,
depending on the requirements of new test jobs.

Change-Id: Ifa442ac79a54f749c56d20f5f7383c3852d95268
2018-11-06 16:43:19 +00:00
Zuul
f941d47dcb Merge "Don't use 'become' in CI tests" 2018-11-06 16:31:25 +00:00
Zuul
c9a4d749f4 Merge "Support merging nested yaml config" 2018-11-06 10:26:28 +00:00
Doug Szumski
badb41a49c Support merging nested yaml config
Closes-Bug: 1786741
Change-Id: Ic217f3f8194cdcb5e28045e771a0160c1f12280e
2018-11-05 18:09:22 +00:00
Zuul
5f323afe9a Merge "Fix POST_FAILURE in CI when ARA DB doesn't exist" 2018-11-05 11:12:24 +00:00
Rabi Mishra
48bae90e8f Don't allocate pseudo-TTY when creating containers
With a pseudo terminal, service is not treated as a daemon
and signals would not work as expected.

Change-Id: I16aa29a7924df51659d973a81d8005ae3d86f57b
Related-Bug: #1799642
2018-11-03 22:35:20 +05:30
caoyuan
a737296a9b Fix the zun-wsproxy group missing in site.yml
zun-wsproxy group is used in multinode[1], but it missing from
site.yml this PS to fix this.

[1]: https://github.com/openstack/kolla-ansible/blob/master/ansible/inventory/multinode#L676

Change-Id: I3b06f15dba5b6569202b038a28fb1a2728cffd25
Closes-Bug: #1799234
2018-10-22 22:31:22 +08:00
Hongbin LU
f57c1aec6c Add Zun scenario job for ubuntu
Introduce a job 'kolla-ansible-ubuntu-source-zun' to test kolla
with Zun enabled. To reduce CI resource, this job will be triggered
only if there are changes on the Zun's ansible roles.

Change-Id: I0ba207e1d3761da2d6992c5834d4f59e7e1d6628
2018-10-22 14:24:36 +00:00
Mark Goddard
8399fc7813 Don't use 'become' in CI tests
Kolla Ansible now claims [1] to support executing as a user other than
root.  We should ensure that this is tested in CI.

This change removes the 'become' from hosts in the inventory, and sets
the remote user to 'kolla', as configured via the bootstrap-servers
command. The bootstrap-servers command and other ansible commands
executed before it still need to execute as the zuul user and not as
kolla, since kolla does not exist yet.

The autogenerated SSH private key in passwords.yml is now added to the
zuul user's SSH config, such that it can SSH as the kolla user, which
has authorised this key.

[1]
https://blueprints.launchpad.net/kolla-ansible/+spec/ansible-specific-task-become

Change-Id: I8f3587e2908bc5b8889cd6fbc01981a6186d33e6
2018-10-02 09:30:18 +01:00
Zuul
4431b1d0fa Merge "Update rabbitmq to use new conf & clustering" 2018-09-24 15:07:28 +00:00
Mark Goddard
652b78fc51 Fix POST_FAILURE in CI when ARA DB doesn't exist
If the main CI job fails before generating an ARA report, the SQLite
database file will not exist. This cases the job to fail with
POST_FAILURE, rather than FAILURE, and the following is seen in the
logs:

rsync: change_dir "/home/zuul/.ara" failed: No such file or directory
rsync error: some files/attrs were not transferred (see previous errors)
(code 23) at main.c(1655) [Receiver=3.1.1]
rsync: [Receiver] write error: Broken pipe (32)

This change fixes this by checking for an SQLite database file, and only
intiating the rsync transfer if it exists.

Change-Id: I370e5bc9f137abe552918a3215a025fa61e3a0ca
2018-09-24 09:32:02 +01:00
Zuul
0c03f6041e Merge "Remove compute hosts from etcd group" 2018-09-23 05:16:28 +00:00
Paul Bourke
b163cb02d1 Update rabbitmq to use new conf & clustering
Depends-On: I75e00312b36e1678b90a42cf58d24652323eff27
Change-Id: Ia716fabffca41eff816e59bbf9f4cab79ee8b72f
2018-09-21 16:47:59 +00:00
Zuul
95135e7f03 Merge "Temporarily remove the rabbitmq clusterer plugin" 2018-09-20 07:43:17 +00:00
ZhijunWei
6d5cf09f7e Fix the zookeeper group missing when run tests
Change-Id: I05960661b8efb32a3d57686dc3780d4138309ef5
Closes-Bug: #1793262
2018-09-17 10:01:41 -04:00
Zuul
2866704aa0 Merge "Remove_images in kolla_docker module" 2018-09-17 15:53:24 +00:00
Eduardo Gonzalez
fffdc6b3e5 Remove_images in kolla_docker module
Add support to remove images from kolla_docker
ansible module.

Change-Id: Ib6bcb98b5b295a2d590df3013188913d1f7f3584
2018-09-14 17:14:25 +02:00
Mark Goddard
ee7a29cf00 Remove compute hosts from etcd group
Since I701d495675178c3ed8ec1f00b31d09f198b38a6f merged, etcd only runs
on the control hosts, not the compute hosts. We therefore no longer
require the etcd group to include the compute hosts.

Since the group mapping is now static, we can remove the use of
host_in_groups from the etcd service, in favour of the simpler method of
specifying the group.

Change-Id: Id8f888d7321a30a85ff95e742f7e6c8e2b9c696f
Related-Bug: #1790415
2018-09-04 19:23:18 +01:00
wangwei
4e5e28fff5 Fix bluestore disk naming format in kolla-ansible
The current bluestore disk label naming is inconsistent with the
filestore. The filestore naming format is that the disk prefixes
belonging to the same osd are the same and the suffixes are
different.

This patch keeps the bluestore's disk naming as well.

Change-Id: I71dda29fc4a6765300ce7bb173d2c448c24f6eca
2018-08-31 09:55:09 +09:00
Paul Bourke
0d03fc2789 Temporarily remove the rabbitmq clusterer plugin
In order to migrate to the latest release of rabbitmq (3.7), we need to
first remove this deprecated plugin which is no longer supported (the
problems it solved are now addressed in rabbitmq itself).

This avoids a circular dependency in CI where the new images depend on
the new clustering and the new clustering depends on the new images.

Change-Id: I921459f3e40b9e0d4af9497384e49aabf0abe79b
2018-08-21 11:24:09 +00:00
Zuul
d1e427b7c1 Merge "Apply Resource Constraints to Openstack Services" 2018-07-26 14:18:34 +00:00
Lakshmi Prasanna Goutham Pratapa
76210a2d85 Apply Resource Constraints to Openstack Services
This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa
2018-07-25 17:05:04 +01:00