5440 Commits

Author SHA1 Message Date
Radosław Piliszek
9f14ad651a Fix multiple issues with MariaDB handling
These affected both deploy (and reconfigure) and upgrade
resulting in WSREP issues, failed deploys or need to
recover the cluster.

This patch makes sure k-a does not abruptly terminate
nodes to break cluster.
This is achieved by cleaner separation between stages
(bootstrap, restart current, deploy new) and 3 phases
for restarts (to keep the quorum).

Upgrade actions, which operate on a healthy cluster,
went to its section.

Service restart was refactored.

We no longer rely on the master/slave distinction as
all nodes are masters in Galera.

Closes-bug: #1857908
Closes-bug: #1859145
Change-Id: I83600c69141714fc412df0976f49019a857655f5
2020-01-15 20:15:09 +01:00
Zuul
6dde7ce833 Merge "Adds monasca-ui to horizon" 2020-01-08 16:27:10 +00:00
Zuul
03bd2e8fda Merge "Change Manila cephfs driver" 2020-01-08 15:36:41 +00:00
Zuul
82ee3e473c Merge "Configure Cinder to use lioadm on CentOS/RHEL 8" 2020-01-08 14:12:19 +00:00
Mark Goddard
350bb171ef Configure Cinder to use lioadm on CentOS/RHEL 8
In CentOS/RHEL 8 there is no scsi-target-utils package, nor is it
available in EPEL. It is removed from kolla in [1]. In RHEL 7 and beyond
the LIO kernel subsystem can be used instead of the tgtd daemon.

This change removes support for the SCSI target daemon on CentOS/RHEL 8.
The 'tgtd' image is no longer available for CentOS/RHEL 8.

[1] https://review.openstack.org/#/c/613815/5

Change-Id: I718fc16cde2dd177b2a1c2f79b932426034897fe
Related: blueprint centos-rhel-8
2020-01-08 10:39:54 +00:00
Michal Nasiadka
d512ba409d Change Manila cephfs driver
Currently used cephfs driver have been deprecated in Pike [1], change to use
the proper one.

[1]: https://opendev.org/openstack/manila/src/branch/master/releasenotes/notes/rename-cephfs-native-driver-3d9b4e3c6c78ee98.yaml

Closes-Bug: #1858773
Change-Id: I33bea1d0049accd48c61f85c1165bee1e1cf0c87
2020-01-08 11:00:20 +01:00
Zuul
6de50a11a0 Merge "Drop python 2 support from action plugins" 2020-01-07 14:31:45 +00:00
Zuul
94def91405 Merge "Fix duplicated words issue like "are are out of"" 2020-01-06 12:25:39 +00:00
Zuul
077b577416 Merge "Fix MariaDB galera IPv6 deployment on CentOS 7" 2020-01-06 11:50:30 +00:00
zhufl
1b90ed1c47 Fix duplicated words issue like "are are out of"
This is to fix the duplicated words issue like
"Other services that are are out of scope of this".

Change-Id: Ie4882dbb64d6e8774888b97895af20ba3855f0f8
2020-01-06 09:26:22 +00:00
Zuul
2257db2a9c Merge "Generate HAProxy configuration for all enabled servvices" 2020-01-04 12:01:43 +00:00
yj.bai
908bffcfc2 Fix MariaDB galera IPv6 deployment on CentOS 7
CentOS 7 uses old galera which has multiple issues handling
IPv6 addressing.
This patch applies two workarounds for CentOS 7.

Co-Authored-By: Jeffrey Zhang <jeffrey.zhang@99cloud.net>
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I7c178aba60c389e65075e0e6cbe4dfa5b8ce06ec
Closes-Bug: #1856532
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2020-01-02 20:20:29 +00:00
yj.bai
0449c4fc8e Fix /etc/kolla to node_config_directory
Change-Id: Ifa8fb271ee2d5642785097755f7347e3be00f8e9
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2019-12-30 10:38:32 +08:00
Hamed Bahadorzadeh
2eebf64275 Adds monasca-ui to horizon
Variable added to evaluate "ENABLE_MONASCA" env for 'kolla/horizon'. In
case 'enable_horizon_monasca' is true, 'policy_item' would be called for
Monasca.

Change-Id: Ie9ecb8ab5d4e74af9b83a5b00ccced5b630ab1ed
Implements: blueprint monasca-ui
Signed-off-by: Hamed Bahadorzadeh <h.bahadorzadeh@gmail.com>
2019-12-28 15:16:28 +03:30
Zuul
8e4556f020 Merge "Update section names in cloudkitty config" 2019-12-23 14:43:49 +00:00
Zuul
a94ab43f18 Merge "Fix unable to connect to epmd when deploy rabbitmq by train with ipv6" 2019-12-23 14:43:48 +00:00
Scott Solkhon
e6e6c41538 Generate HAProxy configuration for all enabled servvices
This change applys the HAProxy tag to the entire play, ensuring HAProxy
configuration is generated for all services when the HAProxy tag is
specified.

Change-Id: I67f57c831a713142d38c6e7b70f814a9ee8e5aae
Closes-Bug: #1855094
2019-12-21 15:22:17 +00:00
Zuul
f7a592f874 Merge "Fix fluentd parsing of WSGI logs" 2019-12-18 12:33:44 +00:00
yj.bai
7054b27dbb Fix unable to connect to epmd when deploy rabbitmq by train with ipv6
deploy rabbitmq cluster by train with ipv6 report:
unable to connect to epmd (port 4369) on control-1: address (cannot connect to host/port)

Closes-Bug: #1856725
Change-Id: I36ebb4e196ece8a304269e8c85e39dda72faae50
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2019-12-18 19:24:40 +08:00
Zuul
da105c20ee Merge "External Ceph: template cinder backend config when cinder_backend_ceph" 2019-12-16 19:34:44 +00:00
Pierre Riteau
ac61bc600d Update section names in cloudkitty config
The 'gnocchi_collector' and 'keystone_fetcher' section names were
deprecated in Stein.

Change-Id: I626dc7afe9eabfbeb6c08137a3e6bbeebde2b332
2019-12-16 18:35:34 +01:00
Michal Nasiadka
6a8cfbb933 External Ceph: template cinder backend config when cinder_backend_ceph
Currently External Ceph Cinder config requires the user to create cinder
service custom configuration.

This change alters the if/else statements to template out cinder backends
configuration when cinder_backend_ceph is True.

Change-Id: I143c3b44d2839e56d1dbf28484c0eaae0a753dc9
2019-12-16 14:08:04 +01:00
Zuul
d7c297ed59 Merge "Added senlin-conductor and senlin-health-manager" 2019-12-16 11:58:06 +00:00
Zuul
4307e2447e Merge "Fix Prometheus template generation" 2019-12-16 11:19:27 +00:00
Zuul
6c297e54c4 Merge "[neutron] Adjust neighbour table thresholds" 2019-12-16 11:19:26 +00:00
Zuul
a12aad6e61 Merge "Add also_notifies to Infoblox backend for Designate" 2019-12-16 11:19:24 +00:00
Mark Goddard
2b662cfb12 Allow ironic_ipxe to serve instance images
Ironic provides a feature to allow instance images to be served from a
local HTTP server [1]. This is the same server used for PXE images with
iPXE. This does not work currently because the ironic_ipxe container
does not have access to /var/lib/ironic/images (ironic docker volume),
where the images are cached. Note that to make use of this feature, the
following is required in ironic.conf:

[agent]
image_download_source = http

This change fixes the issue by giving ironic_ipxe container access to
the ironic volume.

[1] https://docs.openstack.org/ironic/latest/admin/interfaces/deploy.html#deploy-with-custom-http-servers

Change-Id: I501d02cfd40fbacea32d551c3912640c5661d821
Closes-Bug: #1856194
2019-12-12 14:41:00 +00:00
Mark Goddard
3f10f70840 Drop python 2 support from action plugins
These are executed on the local host where we run ansible-playbook,
and we have agreed to drop Python 2 support there.

Partially Implements: blueprint drop-py2-support
Change-Id: Id2190c3a22a56f4f048afbf0f7200daa8f41a292
2019-12-11 16:07:37 +00:00
Zuul
97bbad7c19 Merge "Template custom HAProxy service configuration" 2019-12-11 01:19:45 +00:00
Zuul
9b03f51c7b Merge "Make fluentd-elasticsearch configuration more robust" 2019-12-10 21:01:32 +00:00
Pierre Riteau
49f334284b Template custom HAProxy service configuration
Change Id84e3b6e62e544582d6917047534e846e026798d added support for
custom HAProxy service config using a plain copy of files in services.d.

Use a template action instead of a copy so that we can use variables and
iterate over group of hosts.

Change-Id: I1f07785932de4e4540422bd18af95241f05a67bf
2019-12-10 15:31:45 +01:00
Zuul
516cb3d95d Merge "Neutron: Added loading ip6_tables kernel module" 2019-12-10 13:56:10 +00:00
Zuul
a46292f6b6 Merge "Support configuration of Docker client timeout" 2019-12-10 12:38:57 +00:00
Zuul
8f3e6aca99 Merge "Use local python interpreter for keystone cron generator" 2019-12-10 11:15:55 +00:00
Zuul
558b745ae1 Merge "Support deploying Elasticsearch Curator" 2019-12-10 10:52:46 +00:00
Zuul
06cf664cfb Merge "Improve Apache logging" 2019-12-10 10:44:45 +00:00
Mark Goddard
e46d602536 Use local python interpreter for keystone cron generator
We generate the keystone cron schedule via a python script on localhost.
Currently this always uses 'python', however this may not be available
on some systems.

This change switches to use the same python interpreter as used by
ansible-playbook.

Partially-Implements: blueprint python-3

Change-Id: I6007f8d6880f418a503766cec21a330c44e5b80f
2019-12-09 17:07:10 +00:00
Jan Vondra
59b058e23d Neutron: Added loading ip6_tables kernel module
ip6_tables is needed for HybridFwDriver

Change-Id: I66c50b74a89c046dc59e59dd2422a80e0642ab72
2019-12-09 16:26:13 +00:00
Doug Szumski
d4381456fd Support deploying Elasticsearch Curator
This allows users to supply an Elasticsearch Curator actions file
to manage log retention [1]. Curator then runs on a cron job, which
defaults to every day. A default curator actions file is provided,
which can be customised by the end user if required.

[1] https://www.elastic.co/guide/en/elasticsearch/client/curator/current/actionfile.html

Change-Id: Ide9baea9190ae849e61b9d8b6cff3305bdcdd534
2019-12-09 14:49:05 +00:00
Mark Goddard
a32cddca49 Fix fluentd parsing of WSGI logs
WSGI log files use a different input configuration than OpenStack log
files. Currently this depends on log files matching either *-access.log
or *-error.log. Some services use *_access.log or *_error.log, so are
not parsed correctly.

This change modifies the fluentd configuration to accept an underscore
or hyphen for WSGI log file names.

Change-Id: I566d6cac0b6749054fd5422ec8f36f99dacb1db7
Closes-Bug: #1720371
2019-12-09 14:12:14 +00:00
Krzysztof Klimonda
0c573062fc Make fluentd-elasticsearch configuration more robust
Enable reconnect_on_error option so that ES plugin re-establishes
a new session to the ES cluster on errors. Also, enable buffering
to the file, so that the buffer survives container restarts.

Co-Authored-By: Michal Nasiadka <mnasiadka@gmail.com>
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Co-Authored-By: Doug Szumski <doug@stackhpc.com>
Closes-Bug: #1830724
Change-Id: Ia40685b9d4fc02194e03c8791ddeb3d29d7f07f6
2019-12-09 13:38:36 +00:00
Radosław Piliszek
58b5acbf65 Default to etcd3gw driver for etcd-based coordination
To fix instability and availability issues:

etcd3 is not available in repos for binary kolla images.

etcd3 does not support eventlet-based services [1].

[1] https://review.opendev.org/466098

Change-Id: I430bab735da204fc81696130b17931a89214c876
Closes-bug: #1852086
Closes-bug: #1854932
2019-12-07 08:52:12 +00:00
Michal Nasiadka
3f55b87069 Improve Apache logging
Currently we don't put global Apache error logs into /var/log/kolla,
this change adds statements that redirect those logs there.

Adapted the logfile names to catch into openstack wsgi logging fluentd
input config and existing logrotate cron entries.

Change-Id: I21216e688a1993239e3e81411a4e8b6f13e138c2
2019-12-06 13:11:49 +00:00
Scott Solkhon
68b9224425 Add also_notifies to Infoblox backend for Designate
Change-Id: Ia02f83dfaaba53f95e373b2b2be3f74cfb7ae578
Closes-Bug: #1855085
2019-12-04 15:30:13 +00:00
Zuul
f52082abf7 Merge "Fix Swift with kernel 4.19 and later" 2019-12-04 10:16:10 +00:00
Zuul
f8862c6cc2 Merge "[Train] Fix Tacker config for the local Glance store" 2019-12-03 10:21:34 +00:00
Erik Olof Gunnar Andersson
619ccf1242 Added senlin-conductor and senlin-health-manager
Depends-On: https://review.opendev.org/692948/
Depends-On: https://review.opendev.org/692691/
Change-Id: I07827b896d36c3723697540fcff164224f6729af
2019-11-30 16:49:33 -08:00
Scott Solkhon
991bdc5f55 Fix Prometheus template generation
In a deployment where Prometheus is enabled and
Alertmanager is disabled the task "Copying over
prometheus config file" in
'ansible/roles/prometheus/tasks/config.yml' will
fail to template the Prometheus configuration file
'ansible/roles/prometheus/templates/prometheus.yml.j2'
as the variable 'prometheus_alert_rules' does not
contain the key 'files'. This commit fixes this bug.

Change-Id: Idbe1e52dd3693a6f168d475f9230a253dae64480
Closes-Bug: #1854540
2019-11-30 22:54:22 +00:00
Radosław Piliszek
21488710d6 Fix Swift with kernel 4.19 and later
We mount Swift volumes with xfs.
The 'nobarrier' option we used was made noop [1]
and deprecated [2] (with warning) in kernel 4.10.
In 4.19 it was removed [3] resulting in an error
when using e.g. Debian Buster as host.
The noop patch was backported to CentOS 7 so
it is safe to remove this option with no behavior
change and backport to where needed.
Ubuntu Bionic uses 4.15 which only warns.
CentOS 8 uses 4.18 which only warns as well.
Debian Buster uses 4.19 exactly which breaks.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2291dab2c9d1880efd19469df2042e2277c8b7a4
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cf4573d899cd80d8578c050061dc342f99f3a32
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1c02d502c20809a2a5f71ec16a930a61ed779b81

Change-Id: I006dea21321146c7fc738d0b41c401b72d271a99
Closes-bug: #1800132
2019-11-30 12:56:36 +01:00
Mark Goddard
01050dc01f Support configuration of Docker client timeout
Adds support for configuration of the Docker client timeout via
'docker_client_timeout'.

This change also increases the default timeout to 120 seconds, as we
sometimes see timeouts in CI and heavily loaded or underpowered
environments. Increasing 'docker_client_timeout' further may be helpful
in cases where Docker reports 'Read timed out'.

Change-Id: I73745771078cb2c0ebae2b1d87ba2c4c12958d82
Closes-Bug: #1809844
2019-11-28 10:10:00 +00:00