54 Commits

Author SHA1 Message Date
Zuul
5126087af5 Merge "CentOS 8: Support variable image tag suffix" 2020-01-21 09:29:58 +00:00
James Kirsch
c15dc20341 Configure services to use Certificate Authority
Include a reference to the globally configured Certificate Authority to
all services. Services use the CA to verify HTTPs connections.

Change-Id: I38da931cdd7ff46cce1994763b5c713652b096cc
Partially-Implements: blueprint support-trusted-ca-certificate-file
2020-01-13 11:00:11 -08:00
Mark Goddard
9755c924be CentOS 8: Support variable image tag suffix
For the CentOS 7 to 8 transition, we will have a period where both
CentOS 7 and 8 images are available. We differentiate these images via a
tag - the CentOS 8 images will have a tag of train-centos8 (or
master-centos8 temporarily).

To achieve this, and maintain backwards compatibility for the
openstack_release variable, we introduce a new 'openstack_tag' variable.
This variable is based on openstack_release, but has a suffix of
'openstack_tag_suffix', which is empty except on CentOS 8 where it has a
value of '-centos8'.

Change-Id: I12ce4661afb3c255136cdc1aabe7cbd25560d625
Partially-Implements: blueprint centos-rhel-8
2020-01-10 09:56:04 +00:00
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Kris Lindgren
2fe0d98ebb Add a job that *only* deploys updated containers
Sometimes as cloud admins, we want to only update code that is running
in a cloud.  But we dont need to do anything else.  Make an action in
kolla-ansible that allows us to do that.

Change-Id: I904f595c69f7276e71692696471e32fd1f88e6e8
Implements: blueprint deploy-containers-action
2019-09-26 17:51:14 +01:00
Mark Goddard
cc555c4196 Add <project>_install_type for all projects
This allows the install type for the project to be different than
kolla_install_type

This can be used to avoid hitting bug 1786238, since kuryr only supports
the source type.

Change-Id: I2b6fc85bac092b1614bccfd22bee48442c55dda4
Closes-Bug: #1786238
2019-09-23 10:23:54 +00:00
Keith Plant
b95ff2d1db Configure Telegraf to monitor Docker containers
Added configuration to ansible/roles/telegraf/templates/telegraf.conf.j2 to
allow telegraf to grab telemetry data from docker directly.

Added option to etc/kolla/globals.yml to switch on/off the configuration to
ingest data from the docker daemon into telegraf.

Change-Id: Icbebc415d643a237fa128840d5f5a9c91d22c12d
Signed-off-by: Keith Plant <kplantjr@gmail.com>
2019-08-13 08:17:00 -04:00
Zuul
ca1de25fbf Merge "Add Kafka input to telegraf config" 2019-08-05 10:58:05 +00:00
Zuul
8f70bc22d6 Merge "Add extra volumes support for services that were not previously supported" 2019-08-05 09:02:04 +00:00
pangliye
93e868360d Add Kafka input to telegraf config
Change-Id: I9a8d3dc5f311d4ea4e5d9b03d522632abc66a7ac
2019-08-05 07:26:46 +00:00
Mark Goddard
de00bf491d Simplify handler conditionals
Currently, we have a lot of logic for checking if a handler should run,
depending on whether config files have changed and whether the
container configuration has changed. As rm_work pointed out during
the recent haproxy refactor, these conditionals are typically
unnecessary - we can rely on Ansible's handler notification system
to only trigger handlers when they need to run. This removes a lot
of error prone code.

This patch removes conditional handler logic for all services. It is
important to ensure that we no longer trigger handlers when unnecessary,
because without these checks in place it will trigger a restart of the
containers.

Implements: blueprint simplify-handlers

Change-Id: I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
2019-06-27 15:57:19 +00:00
ZijianGuo
e610a73e98 Add extra volumes support for services that were not previously supported
We don't add extra volumes support for all services in patch [1].
In order to unify the management of the volume, so we need add extra volumes
support for these services.

[1] 12ff28a693

Change-Id: Ie148accdd8e6c60df6b521d55bda12b850c0d255
Partially-Implements: blueprint support-extra-volumes
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-06-27 18:32:15 +08:00
Nick Jones
683f65aaf3
Fix Telegraf configuration template
Remove the erroneous curly braces that prevent the Telegraf
configuration template from being rendered.

Change-Id: Ibb8738b258245b4e43d5d877b156ed296fc66e4e
Closes-Bug: 1810505
2019-01-04 12:11:39 +00:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
pangliye
22d3857ce7 add monitoring of zookeeper
add monitoring of zookeeper

Change-Id: I1db23b67dd8f83f9341a376d9cd2d9a49abd4427
2018-11-13 03:46:20 +00:00
pangliye
daf5a637a3 Add monitoring of redis
add monitoring of redis

Change-Id: Ic6becef0c528083d48458c8dcaa10a2ca6c238a9
2018-11-08 06:23:26 +00:00
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Lakshmi Prasanna Goutham Pratapa
9f0db30fd1 Apply Resource-Constraints to all services.
This commit is the final commit to apply resource-constraints
to all OpenStack services.

Depends-on: I39004f54281f97d53dfa4b1dbcf248650ad6f186
Change-Id: I072d69be9698be54775cb0ae286ea2b6ed78776c
Implements: blueprint resource-constraints
2018-07-23 19:07:05 +05:30
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Zuul
01ccd0d394 Merge "Fix telegraf output for influxdb" 2018-03-13 13:55:04 +00:00
Duong Ha-Quang
9965cc46ff Specify 'become' for only neccesary tasks (all other roles)
Add become to only neccesary tasks in roles:

- aodh
- barbican
- bifrost
- ceilometer
- ceph
- chrony
- cinder
- cloudkitty
- collectd
- congress
- designate
- elasticsearch
- etcd
- freezer
- gnocchi
- grafana
- influxdb
- ironic
- iscsi
- karbor
- kibana
- kuryr
- magnum
- manila
- mistral
- mongodb
- multipathd
- murano
- octavia
- panko
- qdrouterd
- rally
- sahara
- searchlight
- senlin
- skydive
- solum
- swift
- swift
- tacker
- telegraf
- tempest
- trove
- vmtp
- watcher
- zun

Change-Id: I6e32d94d4172dd96d09d8609e8a5221ab5586a31
Partial-Implements: blueprint ansible-specific-task-become
2018-03-12 09:37:43 +07:00
Kevin TIBI
7764ed9894 Fix telegraf output for influxdb
Kolla add multiple url in one outputs.influxdb.
This configuration is when influxdb is in cluster mode.
We need to write one conf by influxdb server.

Change-Id: I1c788b3c8400e74d50cace5a334311408989c5e1
2018-03-06 09:36:12 +00:00
Alexandru Bogdan Pica
8e3b79440c Implement external MariaDB and pre-configured Databases support
This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121
2018-01-23 13:07:40 +00:00
Zuul
f7dfa20dd6 Merge "Add support for custom telegraf configuration" 2017-12-11 05:22:26 +00:00
Zuul
8ad8342bc1 Merge "Telegraf should only output to influxdb when influxdb is enabled" 2017-11-28 14:47:57 +00:00
Christian Berendt
6d710f595c Collect chrony metrics with telegraf
Change-Id: I4cf1716c94e4fd808b66169348b07fa42e5b242e
Depends-on: I8fbef57b99122b8a093150f6ca1efdd33e032917
2017-11-02 20:47:45 +01:00
Christian Berendt
7343a268a4 Add support for custom telegraf configuration
Change-Id: Iabf4f7da8a97793fae79bf6124b0da1feaf26d9a
2017-11-02 09:15:48 +01:00
Christian Berendt
72dbef13dd Telegraf should only output to influxdb when influxdb is enabled
Change-Id: Ib0512481aebf56b26f93955e682ea0cb80cb5bc4
2017-10-25 00:08:41 +02:00
guochao
d787b2bb21 Add collectd support to telegraf container
Telegraf can collect sample from collectd via UDP.
Configure collectd to exposed samples and telegraf to collect them.

Change-Id: Ie868a4c23cb793e6054e14ee1209a56256c130ac
Depends-On: I33b1a3fe606bf5bd4a08901c7bf274aed5303055
2017-07-21 03:35:30 +00:00
Jenkins
ae7fe6bdbb Merge "Modify the configuration file of telegraf" 2017-07-06 04:11:44 +00:00
Paul Bourke
d8fe3ea780 Add a new 'outward' rabbitmq instance
Certain services such as Murano and trove require access to a rabbitmq
instance from tenant networks. [0]

Exposing the internal rabbitmq to end users is a security hole, hence
there are two options, 1) use vhosts in the existing rabbitmq, or two a
separate rabbitmq instances. Given the importance of rabbitmq to the
OpenStack deployment, we have decided to go with a separate instance.
Refer to [1] for more detail on the various options.

This change makes the rabbitmq role generic so that it can be reused, in
this case to start 'outward_rabbitmq'. It needs to be exposed via
haproxy both for network isolation and also because this is what Murano
configuration requires.

Follow on patches will be added to add a vhost in this outward instance
for Murano and other services which require access.

Based on the original work by bdaca[2]

[0] http://murano.readthedocs.io/en/stable-liberty/intro/architecture.html
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-December/109091.html
[2] https://review.openstack.org/#/c/374525

Change-Id: Ib2bcc7ed4bf4f883a7cd1dfad3db89201e3cfd8d
Partial-Bug: #1620374
Depends-On: I020eb6219f89a310451becde41f6f1c7f54baadd
Co-Authored-By: Bartłomiej Daca <bartek.daca@gmail.com>
2017-06-15 11:12:22 +00:00
Bertrand Lallau
afdd11b9a2 Generalize api_interface_address variable usage
Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455
2017-05-23 08:35:15 +00:00
pangliye
2a8cb0b781 Modify the configuration file of telegraf
Influxdb is  standalone deployed by kolla-ansible,
The cluster mode is  charged now.
Previously configured, each telegraf monitors all influxdb.
If the telegraf is more, It is a disaster to influxdb.
So, telegraf should monitor influxdb deployed on the same server.

Change-Id: I9f4003a55de2a7be53c644b1d2f89d00cdf9478a
2017-05-18 07:37:16 +00:00
Bertrand Lallau
dfb115a047 Fix Telegraf container restarting issue
Telegraf always restart containers when doing reconfigure, due to
"pid_mode" attribut missing during compare_container.
Hence compare_container always evaluated to True.

Change-Id: I335816ebf99a218a5400b2baae5b55788459bfff
Closes-Bug: #1686349
2017-04-26 11:11:18 +02:00
Jenkins
ed0f2fc819 Merge "Fix Telegraf retention policy not found" 2017-04-05 11:32:25 +00:00
Bertrand Lallau
bfecaf3857 Telegraf: activate influxdb input plugin
The InfluxDB plugin will collect metrics on the given InfluxDB servers.

Change-Id: I23f0433f56047369ce19bbf3236f6dd7038eb06e
2017-04-05 08:32:53 +02:00
Bertrand Lallau
a914fb6671 Fix Telegraf retention policy not found
New version of influxdb use "autogen" as default retention policy name.
Please see https://github.com/influxdata/influxdb/issues/3733 for more
info.

Change-Id: I8aeb47f60b3aeb022e0cd7aaac630d7cad5b0099
Closes-Bug: #1673914
2017-04-04 10:22:05 +02:00
Bertrand Lallau
e05491bcd6 Fix Telegraf role reconfigure action
During "Optimize reconfigure action for Telegraf" previous commit, a
regression has been introduced on the "reconfigure" action.
https://review.openstack.org/#/c/424202/

The start.yml file has been deleted but reconfigure.yml still using it.

This patch fix it.

Change-Id: I8879b83ce7a1f886f6870d79f63c3f1cc706059b
Closes-Bug: #1676403
2017-03-27 17:47:35 +02:00
Bertrand Lallau
8e3cce34a2 Fix Telegraf startup config files issue
Telegraf allow plugin configuration in /etc/telegraf/telegraf.d
directory. These files comes from telegraf/templates/config/*.conf,
but this directory is optional.

Closes-Bug: #1673691

Change-Id: Idc695c86c5f6b6b0bd514728f2025321cd200907
2017-03-20 22:47:31 +01:00
Paul Bourke
5418ada148 Enable sanity checks from kolla-ansible
Add a new subcommand 'check' to kolla-ansible, used to run the
smoke/sanity checks.

Add stub files to all services that don't currently have checks.

Change-Id: I9f661c5fc51fd5b9b266f23f6c524884613dee48
Partially-implements: blueprint sanity-check-container
2017-03-09 10:37:06 +00:00
liyingjun
6b47d8284b Fix telegraf task
When enabling telegraf, task failed with "the field 'vars' has an invalid value,
which appears to include a variable that is undefined. The error was: 'dict
object' has no attribute 'telegraf_services'"

Change-Id: I6dd5c401bf9b673685ad1b4a20f85c55ce04abbc
Closes-bug: #1662807
2017-02-08 20:05:40 -05:00
caoyuan
b4632d2a24 Optimize reconfigure action for telegraf
Change-Id: Ia658b0774ce009feb6b4e73363114f7922833835
Partially-implements: blueprint better-reconfigure
2017-01-27 03:41:31 +00:00
Nenad Radojevic
78c66cd992 Missing mysql plugin in telegraf
Added mysql plugin for telegraf to be used by default.
Plugin gathers the statistic data from MySQL server.

Change-Id: I853af61b64739eedb70ab2b8dc78f0285a839671
Closes-Bug: #1652609
2016-12-27 23:36:18 +00:00
Jenkins
de0bae44cb Merge "Remove haproxy_socket volume from telegraf container" 2016-11-06 20:11:43 +00:00
Jenkins
86cd729c7d Merge "Remove inputs.filestat from telegraf configuration" 2016-11-06 04:58:26 +00:00
Jeffrey Zhang
fa45801946 Move precheck into its own role
* Merge prechecks.yml and site.yml playbook
* Create empty precheck.yml into all roles.

Change-Id: I8a138558a26c0a2a66c5fd48ed37be657c99c1dd
Implements: blueprint condition-pre-check
2016-11-03 14:48:58 +08:00
Christian Berendt
1f31fb94d5 Remove haproxy_socket volume from telegraf container
HAProxy stats pages are used to gather performance metrics from HAProxy.

Change-Id: I999fcd0785d90c1be0b760fd6cd8804399d5a94c
Closes-bug: #1634227
2016-10-24 11:56:48 +00:00
Christian Berendt
0c60cbe8de Collect haproxy/elasticsearch/memcached metrics only on related hosts
Change-Id: Ifa72d352e65c834afa078a7b08a7582155e6298b
Closes-bug: #1635032
2016-10-19 22:15:48 +02:00