58 Commits

Author SHA1 Message Date
Zuul
d10c5ac15c Merge "Ansible lint related fixes" 2020-05-14 09:52:29 +00:00
Michal Nasiadka
2128075c6e Ansible lint related fixes
Change-Id: I146ea3d84efb83ec5d7405644ad372e57ecafc1e
2020-05-12 17:39:07 +00:00
generalfuzz
f165b81e2a Use FQDN to communicate with Kibana and Elasticsearch
Switch URL composition from using VIP to FQDN to connect with Kibana and
Elasticsearch services.

Change-Id: I5d559ead1d6d5e928e76bb685e0f730868fd7b89
Closes-Bug: #1862419
2020-05-05 09:55:50 -07:00
Zuul
87984f5425 Merge "Add Ansible group check to prechecks" 2020-04-16 15:33:46 +00:00
Zuul
7f42813159 Merge "Refactor copy certificates task" 2020-04-16 14:03:37 +00:00
James Kirsch
4d155d69cd Refactor copy certificates task
Refactor service configuration to use the copy certificates task. This
reduces code duplication and simplifies implementing encrypting backend
HAProxy traffic for individual services.

Change-Id: I0474324b60a5f792ef5210ab336639edf7a8cd9e
2020-04-14 17:26:19 +00:00
Jason Anderson
8c489009c2
[elasticsearch] Add migration for Kibana 6.x index
Elasticsearch 6.x dropped support for mapping types[1], which by default
the Kibana index used. This means that when deploying ELK 6.x, the
Kibana index must be migrated to the new schema to preserve dashboards
and visualizations. There is a process defined[2], which involves
creating a new index with the specified schema, then reindexing the old
index's data into the new index, then doing a rename/delete.

This adds support for that workflow via Ansible. It takes place after
the ES container is restarted after an upgrade, so there will be a
(short) period of time where the Kibana index is not migrated. During
this time, Kibana still loads, but presents the user with a status
screen informing that the index needs migration.

[1]:
https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html
[2]: https://www.elastic.co/guide/en/kibana/6.x/migrating-6.0-index.html

Implements: blueprint elasticsearch-kibana-version-upgrade
Depends-On: https://review.opendev.org/709624
Change-Id: I4550629e2113f3da7f1cecfeab0d5fe0d899dae8
2020-04-06 16:08:17 -05:00
Zuul
31fe3c0651 Merge "Fix invalid JSON body in Elasticsearch API requests" 2020-03-18 10:47:57 +00:00
Zuul
2a2ce059dc Merge "Add notify restart container when cert changed" 2020-03-10 12:12:55 +00:00
yj.bai
d3cc2f670e Add notify restart container when cert changed
When change the cert file in /etc/kolla/certificate/.
The certificate in the container has not changed.
So I think can use kolla-ansible deploy when certificate is
changed. restart <container>

Partially-Implements: blueprint custom-cacerts

Change-Id: Iaac6f37e85ffdc0352e8062ae5049cc9a6b3db26
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2020-03-10 16:23:09 +08:00
Radosław Piliszek
266fd61ad7 Use "name:" instead of "role:" for *_role modules
Both include_role and import_role expect role's name to be given
via "name" param instead of "role".
This worked but caused errors with ansible-lint.
See: https://review.opendev.org/694779

Change-Id: I388d4ae27111e430d38df1abcb6c6127d90a06e0
2020-03-02 10:01:17 +01:00
Mark Goddard
49fb55f182 Add Ansible group check to prechecks
We assume that all groups are present in the inventory, and quite obtuse
errors can result if any are not.

This change adds a precheck that checks for the presence of all expected
groups in the inventory for each service. It also introduces a common
service-precheck role that we can use for other common prechecks.

Change-Id: Ia0af1e7df4fff7f07cd6530e5b017db8fba530b3
Partially-Implements: blueprint improve-prechecks
2020-02-28 16:23:14 +00:00
James Kirsch
256322a8fe Construct service configuration urls using kolla_internal_fqdn
Service configuration urls should be constructed using
kolla_internal_fqdn instead of kolla_internal_vip_address. Otherwise SSL
validation will fail when certificates are issued using domain names.

Change-Id: I21689e22870c2f6206e37c60a3c33e19140f77ff
Closes-Bug: 1862419
2020-02-22 08:28:01 -08:00
Mark Goddard
5db9eab042 Fix invalid JSON body in Elasticsearch API requests
The kibana, elasticsearch and monasca roles all use the uri module to
perform Elasticsearch configuration tasks via its API. The body of the
request should be JSON formatted, but these tasks now fail because it is
not.

The following error is seen:

TASK [monasca : Create default control plane organisation if it doesn't
exist]

invalid character '\\'' looking for beginning of object key string

The 'JSON' body in this case was:

{'name': 'monasca_control_plane@default'}

This was probably caused by the recent change to execute these tasks in
the kolla_toolbox container, but may also be caused by an Ansible
version bump (or something else).

This change fixes the issue by ensuring that the body is JSON-encoded in
all cases.

Change-Id: I7acc097381dd9a4af4e014525c1c88213abbde93
Closes-Bug: #1864177
2020-02-21 10:09:51 +00:00
James Kirsch
88418cbaa9 Use kolla_toolbox to execute REST methods
Delegate executing uri REST methods to the current module containers
using kolla_toolbox. This will allow self signed certificate that are
already copied into the container to be automatically validated. This
circumvents requiring Kolla Ansible to explicitly disable certificate
validation in the ansible uri module.

Partially-Implements: blueprint custom-cacerts

Change-Id: I2625db7b8000af980e4745734c834c5d9292290b
2020-01-28 14:03:32 -08:00
James Kirsch
511ba9f6a2 Copy CA into containers.
When kolla_copy_ca_into_containers is set to "yes", the Certificate
Authority in /etc/kolla/certificates will be copied into service
containers to enable trust for that CA. This is especially useful when
the CA is self signed, and would not be trusted by default.

Partially-Implements: blueprint custom-cacerts

Change-Id: I4368f8994147580460ebe7533850cf63a419d0b4
2020-01-28 14:03:32 -08:00
Marcin Juszkiewicz
934fea1aac Ansible lint: disable some checks
204 for very long url which is hard to break safely
306 for "echo | docker" as echo should not fall

Change-Id: I14df39d611d39ad1f6184ab92d628cb010881fbb
2020-01-17 13:51:43 +00:00
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Kris Lindgren
2fe0d98ebb Add a job that *only* deploys updated containers
Sometimes as cloud admins, we want to only update code that is running
in a cloud.  But we dont need to do anything else.  Make an action in
kolla-ansible that allows us to do that.

Change-Id: I904f595c69f7276e71692696471e32fd1f88e6e8
Implements: blueprint deploy-containers-action
2019-09-26 17:51:14 +01:00
Mark Goddard
de00bf491d Simplify handler conditionals
Currently, we have a lot of logic for checking if a handler should run,
depending on whether config files have changed and whether the
container configuration has changed. As rm_work pointed out during
the recent haproxy refactor, these conditionals are typically
unnecessary - we can rely on Ansible's handler notification system
to only trigger handlers when they need to run. This removes a lot
of error prone code.

This patch removes conditional handler logic for all services. It is
important to ensure that we no longer trigger handlers when unnecessary,
because without these checks in place it will trigger a restart of the
containers.

Implements: blueprint simplify-handlers

Change-Id: I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
2019-06-27 15:57:19 +00:00
Mark Goddard
b123bf6621 Use become for all docker tasks
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
2019-06-06 19:04:58 +01:00
Raimund Hook
726ba75341 Updated ansible uri module HEADERS_ field
This fixes a deprecation warning that gets displayed when running
the kibana/post_config 'Get kibana default indexes' task.

HEADERS_ has been deprecated since ansible 2.1 and will be
removed in 2.9.

https://docs.ansible.com/ansible/latest/modules/uri_module.html

TrivialFix

Change-Id: I177113c606119505c6cb69c66a326f7cbdaf2196
2019-05-04 18:48:03 +01:00
Duong Mai
2519451696 Edit ansible script for deploy kibana
Kibana deployment failed becaused of kibana_confs variable does not have attribute key,
So handlers failed to check conditional kibana_conf.changed | bool becaused of
kibana_confs.results|selectattr(item.key) does not exits.

Change variable name kibana_confs to kibana_conf.

Change-Id: If5e0a25b270a6f05c435a6dc32e2ac49406389c5
Closes-Bug: #1819246
2019-03-09 11:46:04 +07:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
Christian Berendt
c1acb65642 Fix template src of kibana configuration file
Change-Id: I4d409e052c00d83b451a2a13917f89531a88457f
2018-10-24 13:35:59 +02:00
Christian Berendt
c96633d454 Make custom kibana configuration files possible
Change-Id: Iedfad564f834504fa1f4bfd935cd735d1d9ee65f
2018-10-23 07:04:14 +02:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
binhong.hua
1e26b05fe0 fix condition of kibana pull action
Change-Id: Ia459d8b9925b22b23c90a33364da6ef40f4b1a70
2018-08-31 14:39:33 +08:00
Zuul
3288e0c4e3 Merge "Update usage of "|" to "is"" 2018-08-23 10:23:54 +00:00
caoyuan
471985dc2c Update usage of "|" to "is"
With the more recent versions of ansible, we should now use
"is" instead of the "|"

This should update it.

Change-Id: I6fba56fca182349972e8b0ee5452b37aa4090e0c
2018-08-13 12:40:10 +05:30
Mark Goddard
76c83a4096 Allow 400 response when creating kibana ES index
If the index already exists, kibana will respond with a 400 error. This
is required to support reconfigure and upgrade of kibana.

Change-Id: Ieee6d636fe81692dffb886f917b8398df10a525f
2018-08-03 12:50:21 +01:00
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Lakshmi Prasanna Goutham Pratapa
14bf524756 Apply Resource Constraints to Services.
This commit is to apply resource-constraints to a few more OpenStack services.
Commit to  apply constraints to the last set of services will be made in
the upcoming commit.

Depends-on: Icafa54baca24d2de64238222a5677b9d8b90e2aa
Change-Id: I39004f54281f97d53dfa4b1dbcf248650ad6f186
2018-07-26 11:35:28 +00:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
Jorge Niedbalski
317a107988 [kibana] enforce index creation for ELK > 5 (debian)
The default index isn't created automatically when using
ELK > 5.x. This commit adds a new task into the post-deploy
taskset to force the creation of the default index.

This patch also enforces the kibana config to set index
as defaultIndex without the need for the defaultIndex property
to already exists , preventing the error exposed on the public bug.

Also added a new option named kibana_default_index_options, with
a default index.mapper.dynamic set to true, this option
can be extended by the operator and they will be applied to the index.

Change-Id: Ica63a5fb30947f7ebae6cf8c80500a7dd0907211
Closes-Bug: #1772687
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@linaro.org>
2018-05-23 09:53:14 -04:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Duong Ha-Quang
9965cc46ff Specify 'become' for only neccesary tasks (all other roles)
Add become to only neccesary tasks in roles:

- aodh
- barbican
- bifrost
- ceilometer
- ceph
- chrony
- cinder
- cloudkitty
- collectd
- congress
- designate
- elasticsearch
- etcd
- freezer
- gnocchi
- grafana
- influxdb
- ironic
- iscsi
- karbor
- kibana
- kuryr
- magnum
- manila
- mistral
- mongodb
- multipathd
- murano
- octavia
- panko
- qdrouterd
- rally
- sahara
- searchlight
- senlin
- skydive
- solum
- swift
- swift
- tacker
- telegraf
- tempest
- trove
- vmtp
- watcher
- zun

Change-Id: I6e32d94d4172dd96d09d8609e8a5221ab5586a31
Partial-Implements: blueprint ansible-specific-task-become
2018-03-12 09:37:43 +07:00
Borne Mace
f27d89aa09 Increased mariadb and kibana timeouts
In some deployment scenarios the current timeouts
for mariadb bootstrap and kibana registration with
elasticsearch have been found to be too short. These
timeout increases shouldn't introduce any deployment
slowdown in current environment and eliminate
deployment failures in environments with slower
systems.

Change-Id: If33dfff2b42b23eff7ec2230c9b0c5a4c010072e
2017-12-12 12:07:26 -08:00
Paul Bourke
c0f2224e61 Dont use local connection for kibana post config
We should not assume the VIP is accessible from the deploy node.

Change-Id: I39640f98b4adddb9cb8dfdd09d9fb7ecba15a820
Closes-Bug: #1730651
2017-11-08 13:52:23 +00:00
Paul Bourke
2d9a028cb0 Kibana post_config checks only need to run locally
Recently added post_config for Kibana does some uri queries, these can
be problematic if run across every node.

Change-Id: I71d2611ae034ff2a85babb8d01419729f406a136
2017-05-19 09:39:18 +00:00
Jenkins
08f50d5e64 Merge "Configure Kibana default index pattern" 2017-05-10 10:54:11 +00:00
Jeffrey Zhang
b1f015616c Add timeout parameter for precheck tasks
wait_for module waits 300 seconds for the port started or stopped.  This
is meaningless and useless in precheck. This patch change timeout to 1
seconds.

Change-Id: I9b251ec4ba17ce446655917e8ef5e152ef947298
Closes-Bug: #1688152
2017-05-04 09:02:25 +08:00
Bertrand Lallau
2afc869d0f Configure Kibana default index pattern
When Kibana interface is opened for the first time, it requires creating
a default index pattern. To view, analyse and search logs, at least one
index pattern has to be created.
This patch automate "flog-*" default index pattern creation and set it
has default config.

Partially-implements: blueprint default-kibana-dashboard
Change-Id: Ie36696f9ad38ba7e49e65e0793a3b98d9f03ee8d
2017-05-02 22:30:47 +02:00
Bertrand Lallau
df259788ba Fix Kibana checking free port interface
In "Checking free port for Kibana Server" task the storage interface is
used instead of the api interface one.
This patch fix it.

Change-Id: I0b38688aa3774ae4a629cce3161a730fff907a9c
2017-04-20 23:04:48 +02:00
Paul Bourke
5418ada148 Enable sanity checks from kolla-ansible
Add a new subcommand 'check' to kolla-ansible, used to run the
smoke/sanity checks.

Add stub files to all services that don't currently have checks.

Change-Id: I9f661c5fc51fd5b9b266f23f6c524884613dee48
Partially-implements: blueprint sanity-check-container
2017-03-09 10:37:06 +00:00
zhubingbing
553a05c9d0 Fix 'kibana_config_jsons' is undefined
Change-Id: I88f84e21ec6ae6dd98f672112f34be8d53f2e2ad
2017-02-10 10:08:58 +00:00
caoyuan
13eed1ba57 Optimize reconfiguration for kibana
Change-Id: Ic4beb20361644b42dbed82fcffb7446c00f84e87
Partially-implements: blueprint better-reconfigure
2017-02-04 15:16:17 +00:00
qiankun.zhang
2d8d6e72ea Move kibana precheck into its own role
Co-Authored-By: caoyuan <cao.yuan@99cloud.net>

Change-Id: I38aa833febe7bf06bffb9c5e8ad6b7a79074fb50
Partially-implements: blueprint condition-pre-check
2017-01-12 10:51:44 +00:00
Christian Berendt
7904c4a4ec Add kolla_logs volume to kibana container
Change-Id: Ie3a04fa6c0f7406fcbc9033a38d9941509d9dc90
Closes-bug: #1640303
2016-11-08 22:14:31 +01:00