9766 Commits

Author SHA1 Message Date
Dai, Dang Van
8d5355dbc1 Fix bootstrap prometheus container location
This change to fix the case that
I won't use prometheus-mysqld-exporter

Change-Id: I1936bbae0172f4e65605d71066dced837bc30f7a
2018-12-27 12:46:22 +00:00
Zuul
4ded787e79 Merge "Allow custom horizon policy files for disabled openstack projects." 2018-12-27 10:19:58 +00:00
Zuul
4cad49284b Merge "Update hacking version" 2018-12-27 09:46:23 +00:00
confi-surya
b41309ef16 Update hacking version
Use latest release 1.1.0 and compatible changes w.r.t pep8

Change-Id: Ie5e5417a1d8481bb59d516a320bd7cc1dd89591e
2018-12-27 04:44:49 +00:00
wu.chunyang
4939fc28d2 remove cinder v1 endpoint
now,cinder does not support v1 endpoint,so no need to create it in
keystone

Change-Id: I4b3bbf24c19a996e73f4b360beeb20a110907a6d
2018-12-24 14:39:43 +00:00
Zuul
54da9fa92c Merge "Add become for Octavia configuration tasks" 2018-12-24 08:36:02 +00:00
Zuul
04fb9981e4 Merge "Fix Vitrage wrong configuration" 2018-12-24 07:36:17 +00:00
weiyj
7ace98d975 hinese quotes
Change-Id: I07e4e563538b4a47d9b1707b4e660531ccce7b9b
2018-12-24 09:35:53 +08:00
Kien Nguyen
90fd7aa03d Fix Vitrage wrong configuration
* In service_credentials section, project_name should be
admin and auth_url should use keystone public port (5000)
according [1].
* User vitrage should have role admin in project admin [2].

[1] https://github.com/openstack/vitrage/blob/master/devstack/plugin.sh#L164
[2] https://docs.openstack.org/vitrage/latest/install/install-rdo.html#initialize-vitrage

Change-Id: I8bf028c702df5659e4c58df0e17600ab5be14eea
2018-12-23 15:46:23 +00:00
Kien Nguyen
b1d7a8ad22 Add missing transport_url config
In oslo_messaging_notification section - vitrage conf,
kolla-ansible is missing transport_url.

Change-Id: I0100af9771dfdb9cd40ce5effff2a60c7bd43575
2018-12-23 22:32:38 +07:00
dommgifer
5c771df7d1 Add become for Octavia configuration tasks
This is required to support execution as a non-root user.

Change-Id: Idd2deec2918e5eb7743e60112a91cb31edf5aad3
Close-bug: #1809527
2018-12-22 15:04:13 +08:00
Martin Chlumsky
18dd74ac8d Allow custom horizon policy files for disabled openstack projects.
Currently, you cannot deploy custom policy files in horizon for disabled
openstack projects that don't have a horizon plugin.

This patch allows customizing disabled openstack projects policy files
used by horizon.  For services that do not have horizon plugins, it
creates variables of the form `enable_*_horizon_policy_file` and sets
these to the relevant `enable_*` variables by default. This ensures
backwards compatibility.

This patch is useful when you are migrating to kolla-ansible from
another deployment method one openstack project at a time and you have
custom policy files deployed by the old method.

Partially-Implements: blueprint docs-migrating-to-kolla
Closes-Bug: #1809314
Change-Id: Ifc33ef65759fa4831c24d177b272af5b45f80931
2018-12-21 11:39:53 -05:00
Zuul
17f9ff5a10 Merge "Add NFV orchestration services CI job" 2018-12-21 13:15:50 +00:00
Zuul
a3628f63a8 Merge "Add become for Prometheus configuration tasks" 2018-12-21 11:13:27 +00:00
Eduardo Gonzalez
535aba79a9 Add NFV orchestration services CI job
Change-Id: I33484d0961f1cd17ed05dd124eba68d99b4db83f
2018-12-21 11:43:22 +01:00
dommgifer
69823f8692 Add become for Prometheus configuration tasks
This is required to support execution as a non-root user.

Change-Id: I60d224407c2828d6b9f1701f7637385a25fbcced
Closes-Bug: #1809233
2018-12-21 16:59:18 +08:00
confi-surya
e73fe77d7d Use template for lower-constraints
Small cleanups:

* Use openstack-lower-constraints-jobs template, remove individual
  jobs.
* Sort list of templates

Change-Id: I67199fabe6a9f7b1fd38dac77a6157bf4fb465b9
Needed-By: https://review.openstack.org/623229
2018-12-21 07:33:19 +00:00
Zuul
d6ad05ca07 Merge "Support list of controller ip for the amphorae heartbeat receiver" 2018-12-20 17:34:46 +00:00
Zuul
de71cf2f21 Merge "Test zun container creation in gates" 2018-12-20 09:31:49 +00:00
Zuul
acda7ba592 Merge "Configure vitrage notification topic" 2018-12-20 03:26:54 +00:00
Eduardo Gonzalez
362b6ee40e Test zun container creation in gates
Change-Id: If5b4ba975a65e07d2704eb6bdb9d841d6a9c3d42
2018-12-19 19:50:59 +01:00
Duc Nguyen Cong
144d9bbc2d Support list of controller ip for the amphorae heartbeat receiver
In multi controller deployment, kolla will generate
"controller_ip_port_list option" in [health_manager] section with
ONLY IP of that node instead of a list of controller ip.
Therefor, "amphora-agent.conf" file of amphora instance will
contain IP of ONLY ONE controller node.
In case of that node fail, amphora agent won't send heartbeat
message to other health manager node, and the loadbalancer will
go to ERROR state.

Change-Id: I102ed6ba3fff2c12cc6d37f81ad59508eacc859c
Co-Authored-By: Hieu LE <hieulq2@viettel.com.vn>
2018-12-19 16:30:09 +00:00
Zuul
ac398c996c Merge "Add kolla-ansible to zuul job dependencies" 2018-12-19 07:04:33 +00:00
Zuul
81606a1b6c Merge "Fix fact gathering with --limit" 2018-12-18 20:49:06 +00:00
Zuul
57f069a336 Merge "Scalability improvements for disabled services" 2018-12-18 20:49:05 +00:00
Zuul
0b4587e947 Merge "Add support for Quobyte backend to Cinder and Nova" 2018-12-18 17:55:12 +00:00
Zuul
481b26b16a Merge "Fix prometheus prechecks for haproxy and memcached" 2018-12-18 11:44:52 +00:00
Mark Goddard
8d523dabd8 Add kolla-ansible to zuul job dependencies
This means we can pull in the job from other repositories without
explicitly adding the dependency on kolla-ansible in that project.

Change-Id: Ia7e4294508e6d445638c176359a939af32fdfb12
2018-12-18 10:37:20 +00:00
Nick Jones
005458d67e
[Designate] Configure named to also listen on 'dns_interface'
Update the template so that if 'dns_interface' is set, named listens on
this interface as well as the 'api_interface'.

Change-Id: I986ca46e5599e4767800fcc7f34a1c6e682efb55
Closes-Bug: 1808829
2018-12-17 16:06:15 +00:00
Zuul
a4bddfdb36 Merge "Create cells before starting nova services" 2018-12-17 15:28:21 +00:00
Pavel Sinkevych
0c8b4730af Fix prometheus prechecks for haproxy and memcached
Add missing `prometheus_memcached_exporter` container_fact
Fix conditional container_fact for haproxy_exporter

Change-Id: Id0f3b94af956f51e3c782c0244c6ce7a340119bd
Closes-Bug: #1808820
2018-12-17 18:03:34 +03:00
Mark Goddard
48aea5637f Support Docker CE in bootstrap-servers
Kolla Ansible's bootstrap-servers command provides support for
installing the Docker engine. This is currently done using the packages
at https://apt.dockerproject.org and https://yum.dockerproject.org.
These packages are outdated, with the most recent packages from May 2017
- docker-engine-17.05.

The source for up to date docker packages is
https://download.docker.com, which was introduced with the move to
Docker Community Edition (CE) and Docker Enterprise Edition (EE).

This change adds support to bootstrap-servers for Docker CE for CentOS
and Ubuntu.

It also adds a new variable, 'enable_docker_repo', which controls
whether a package repository for Docker will be enabled.

It also adds a new variable, 'docker_legacy_packages', which controls
whether the legacy packages at dockerproject.org will be used or the
newer packages at docker.com. The default value for this variable is
'false', meaning to use Docker CE.

Upgrading from docker-engine to docker-ce has been tested on CentOS 7.5
and Ubuntu 16.04, by running 'kolla-ansible bootstrap-servers' with
'docker_legacy_packages' set to 'false'. The upgrades were successful,
but result in all containers being stopped. For this reason, the
bootstrap-servers command checks running containers prior to upgrading
packages, and ensures they are running after the package upgrade is
complete.

As mentioned in the release note, care should be taken when upgrading
Docker with clustered services, which could lose quorum. To avoid this,
use --serial or --limit to apply the change in batches.

Change-Id: I6dfd375c868870f8646ef1a8f02c70812e8f6271
Implements: blueprint docker-ce
2018-12-17 14:04:43 +00:00
Zuul
3af135d4fb Merge "Allow disabling keepalived for external LBs" 2018-12-17 12:52:07 +00:00
Patrick O'Neill
f77cc87e81 Add support for Quobyte backend to Cinder and Nova
Add an enable_cinder_backend_quobyte option to etc/kolla/globals.yml to
enable use the Quobyte Cinder backend.
Change the bind mounts for /var/lib/nova/mnt to include the shared
propogation if Quobyte is enabled.
Update the documentation to include a section on configuring the Cinder.

Implements: blueprint cinder-quobyte-backend

Change-Id: I364939407ad244fe81cea40f880effdbcaa8a20d
2018-12-17 11:58:56 +00:00
Kien Nguyen
838cffd9c9 Configure vitrage notification topic
According [1], vitrage notification has to be configured in Nova,
Neutron, Cinder & Aodh config file.

[1] https://review.openstack.org/#/c/302802/

Change-Id: Iaf8cd7d40e6eb988adf4d208e6ad784f1004caa5
2018-12-17 08:10:39 +07:00
Bartosz Zurkowski
10d33f82bb Find Monasca agent plugins locally
Find module searches paths on managed server. Since role path and custom
Kolla config is located on deployment node and deployment node is not
considered to be a managed server, Monasca plugin files cannot be found.
After the deployment container running Monasca agent collector stucks in
restart mode due to missing plugin files.

The problem does not occur if deployment was started from a managed
server (eg. OSC). The problem occurs if the deployment was started from
a separate deployment server - a common case.

This change enforces running find module locally on deployment node.

Change-Id: Ia25daafe2f82f5744646fd2eda2d255ccead814e
Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>
2018-12-16 16:51:54 +01:00
Bartosz Zurkowski
c5d1e1d5f2 Call Grafana APIs only once
In multinode deployments creating default Grafana organization failed,
because Ansible attempted to call Grafana API in the context of each
host in the inventory. After creating organization via the first host,
subsequent attempts via the remaining hosts failed due to already
existing organization. This change enforces creating default
organization only once.

Other tasks using Grafana API have been enforced to be ran only once as
well.

Change-Id: I3a93a719b3c9b4e55ab226d3b22d571d9a0f489d
Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>
2018-12-16 16:30:58 +01:00
Mark Goddard
365bb5177d Create cells before starting nova services
Nova services may reasonably expect cell databases to exist when they
start. The current cell setup tasks in kolla run after the nova
containers have started, meaning that cells may or may not exist in the
database when they start, depending on timing. In particular, we are
seeing issues in kolla CI currently with jobs timing out waiting for
nova compute services to start. The following error is seen in the nova
logs of these jobs, which may or may not be relevant:

No cells are configured, unable to continue

This change creates the cell0 and cell1 databases prior to starting nova
services.

In order to do this, we must create new containers in which to run the
nova-manage commands, because the nova-api container may not yet exist.
This required adding support to the kolla_docker module for specifying a
command for the container to run that overrides the image's command.

We also add the standard output and error to the module's result when a
non-detached container is run. A secondary benefit of this is that the
output of bootstrap containers is now displayed in the Ansible output if
the bootstrapping command fails, which will help with debugging.

Change-Id: I2c1e991064f9f588f398ccbabda94f69dc285e61
Closes-Bug: #1808575
2018-12-14 19:26:42 +00:00
Benjamin Diaz
0729472256 Make Rabbitmq config files overridable
Rabbitmq config files can be overrided in node_custom_config directory.

Change-Id: Id724b44a3edd951fa8b06c9f2c347e9ed8c5ffd9
Closes-Bug: #1674446
2018-12-14 12:28:31 -03:00
Zuul
404d0b1c12 Merge "remove unused library" 2018-12-13 15:29:37 +00:00
Zuul
850f9eb870 Merge "Create /volumes when using external ceph" 2018-12-13 13:56:36 +00:00
Zuul
9434f605bd Merge "Change xtrabackup to mariadb-backup for mariadb 10.3" 2018-12-12 17:59:52 +00:00
wu.chunyang
4deefe3e92 remove unused library
trivial modify

Change-Id: I27d5b85d2c745fee5ff0643e7771b46faebd23a6
2018-12-12 10:30:20 +00:00
Zuul
9c73d4d8e4 Merge "Add cache configuration for ceilometer project" 2018-12-12 09:09:15 +00:00
Eduardo Gonzalez
4b273eeeec Change xtrabackup to mariadb-backup for mariadb 10.3
xtrabackup doesnt work with mariadb 10.3,
need to be changed to mariadb-backup tool.

For now only migrate galera, not kolla-backup tool
to fix the CI.

https://jira.mariadb.org/browse/MDEV-15774

Change-Id: Ie77ae41e419873feed4b036a307887b22455183b
Depends-On: Icefe3a77fb12d57c869521000d458e3f58435374
2018-12-11 18:20:25 +01:00
Jeffrey Zhang
55bf29ec6c Add cache configuration for ceilometer project
when using ceilometer+gnocchi, for every notification sample, ceilometer
will update the resource even if is not updated.

We should add [cache] section to make ceilometer cache the resource, and
stop send the useless update request.

Closes-Bug: #1807841
Change-Id: Ic33b4cd5ba8165c20878cab068f38a3948c9d31d
2018-12-11 17:12:25 +08:00
Kien Nguyen
835368524e Add Prometheus as Vitrage datasource
Vitrage has already supported Prometheus as
datasource. Kolla can config it automatically,
just need a little changes, for example in
wsgi config file [1].

Co-Authored-By: Hieu LE <hieulq2@viettel.com.vn>

[1] https://review.openstack.org/#/c/584649/8/devstack/apache-vitrage.template

Change-Id: I64028a0dfd9887813b980a31c30c2c1b1046da61
2018-12-11 16:05:05 +07:00
Zuul
d4dd1f0ac4 Merge "Deploy mistral-event-engine service" 2018-12-11 08:04:40 +00:00
Mark Goddard
56b4352f9e Fix fact gathering with --limit
Prior to this change, when the --limit argument is used, each host in the
limit gathers facts for every other host. This is clearly unnecessary, and
can result in up to (N-1)^2 fact gathers.

This change gathers facts for each host only once. Hosts that are not in
the limit are divided between those that are in the limit, and facts are
gathered via delegation.

This change also factors out the fact gathering logic into a separate
playbook that is imported where necessary.

Change-Id: I923df5af41a7f1b7b0142d0da185a9a0979be543
2018-12-07 14:49:27 +00:00
Mark Goddard
5d8403bdc8 Scalability improvements for disabled services
Currently, every service has a play in site.yml that is executed, and
the role is skipped if the service is disabled. This can be slow,
particularly with many hosts, since each play takes time to setup, and
evaluate.

This change creates various Ansible groups for hosts with services
enabled at the beginning of the playbook. If a service is disabled, this
new group will have no hosts, and the play for that service will be a
noop.

I have tested this on a laptop using an inventory with 12 hosts (each
pointing to my laptop via SSH), and a config file that disables every
service. Time taken to run 'kolla-ansible deploy':

Before change: 2m30s
After change: 0m14s

During development I also tried an approach using an 'include_role' task
for each service. This was not as good, taking 1m00s.

The downsides to this patch are that there is a large number of tasks at
the beginning of the playbook to perform the grouping, and every play
for a disabled service now outputs this warning message:

[WARNING]: Could not match supplied host pattern, ignoring: enable_foo_True

This is because if the service is disabled, there are no hosts in the
group. This seems like a reasonable tradeoff.

Change-Id: Ie56c270b26926f1f53a9582d451f4bb2457fbb67
2018-12-07 14:48:53 +00:00