5633 Commits

Author SHA1 Message Date
Dincer Celik
4b5df0d866 Introduce /etc/timezone to Debian/Ubuntu containers
Some services look for /etc/timezone on Debian/Ubuntu, so we should
introduce it to the containers.

In addition, added prechecks for /etc/localtime and /etc/timezone.

Closes-Bug: #1821592
Change-Id: I9fef14643d1bcc7eee9547eb87fa1fb436d8a6b3
2020-04-09 18:53:36 +00:00
Zuul
7c92e56cfd Merge "Separate per-service host configuration tasks" 2020-04-05 16:40:27 +00:00
Zuul
3122e6151c Merge "manila share container name variable" 2020-04-05 16:40:24 +00:00
linpeiwen
fa16190910 manila share container name variable
manila share container name variable is fixed in some places,
but in the defaults directory, manila share container_name variable
is variable. If the manila share container_name variable is changed
during deployment, it will not be assigned to container name,
but a fixed 'manila_share' name.

Change-Id: Iea23c62518add8d6820b76b16edd3221906b0ffb
2020-04-04 22:53:31 -04:00
Zuul
e631367122 Merge "Remove default(omit) from openstack_cacert in templates" 2020-04-04 12:50:52 +00:00
Mark Goddard
0edad7138c Remove default(omit) from openstack_cacert in templates
The use of default(omit) is for module parameters, not templates. We
define a default value for openstack_cacert, so it should never be
undefined anyway.

Change-Id: Idfa73097ca168c76559dc4f3aa8bb30b7113ab28
2020-04-03 14:49:11 +01:00
Zuul
5b86b94307 Merge "Avoid unconditional fact gathering" 2020-04-02 16:50:26 +00:00
Zuul
47e2baf9f4 Merge "Support setting Kafka storage volume" 2020-04-02 15:11:05 +00:00
Mark Goddard
fdea19a305 Separate per-service host configuration tasks
Currently there are a few services that perform host configuration
tasks. This is done in config.yml. This means that these changes are
performed during 'kolla-ansible genconfig', when we might expect not to
be making any changes to the remote system.

This change separates out these host configuration tasks into a
config-host.yml file, which is included directly from deploy.yml.

One change in behaviour is that this prevents these tasks from running
during an upgrade or genconfig. This is probably what we want, but we
should be careful when any of these host configuration tasks are
changed, to ensure they are applied during an upgrade if necessary.

Change-Id: I001defc75d1f1e6caa9b1e11246abc6ce17c775b
Closes-Bug: #1860161
2020-04-02 13:51:56 +00:00
Mark Goddard
e0ba55a85d Avoid unconditional fact gathering
One way to improve the performance of Ansible is through fact caching.
Rather than gather facts in every play, we can configure Ansible to
cache them in a persistent store. An example Ansible configuration for
doing this is as follows:

[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./facts
fact_caching_timeout = 86400

This does not affect Kolla Ansible however, since we use the setup
module which unconditionally gathers facts regardless of the state of
the cache. This gets worse with large inventories limited to a small
batch of hosts via --limit or serial, since the limited hosts must
gather facts for all others.

One way to detect whether facts exist for a host is via the
'module_setup' variable, which exists only when facts exist. This change
uses the 'module_setup' fact to determine whether facts need to be
gathered for hosts outside of the batch. For hosts in the batch, we
switch from using the setup module to gather_facts on the play, which
can use the 'smart' gathering logic.

Change-Id: I04841fb62b2e1d9e97ce4b75ce3a7349b9c74036
Partially-Implements: blueprint performance-improvements
2020-04-02 12:39:46 +00:00
Zuul
c791943356 Merge "Fix HAProxy prechecks during scale-out with limit" 2020-04-02 09:38:57 +00:00
Zuul
1a9dfa3101 Merge "kolla-toolbox container name variable" 2020-04-02 09:38:56 +00:00
Zuul
5ffbdfdff7 Merge "Add clients ca_file in heat" 2020-04-02 09:38:54 +00:00
Zuul
7387220132 Merge "Add glance_ca_certificates_file when using self sign cert in glance" 2020-04-02 09:38:52 +00:00
Radosław Piliszek
c033ddca08 Fix ovs fw driver for the other ovs agent
In [1] only neutron-openvswitch-agent was fixed and not xenapi.
That merged in Ussuri and went cleanly into Train.
In Stein and Rocky, the backport was not clean and
accidentally fixed xenapi instead of the regular one.

Neither the original bug nor its incomplete fix were released,
except for Rocky. :-(
Hence this patch also removes the confusing reno instead of
adding a new one.

[1] https://review.opendev.org/713129

Change-Id: I331417c8d61ba6f180bcafa943be697418326645
Closes-bug: #1869832
Related-bug: #1867506
2020-04-01 20:15:07 +02:00
Doug Szumski
b758883418 Support setting Kafka storage volume
Not everyone wants Kafka data stored on a Docker volume. This
change allows a user to flexibly control where the data is stored.

Change-Id: I2ba8c7a85c7bf2564f954a43c6e6dbb3257fe902
2020-03-30 11:00:03 +02:00
linpeiwen
5659177026 keystone roles container name variable
keystone and keystone_fernet container name variable is fixed
in some places, but in the defaults directory, keystone
and keystone_fernet container_name variable is variable.
If the keystone and keystone_fernet container_name variable is
changed during deployment, it will not be assigned to keystone
and keystone_fernet, but a fixed 'keystone' and 'keystone_fernet' name.

Change-Id: Ifc8ac69e6abc4586f0e4fd820b9022aea9f76396
2020-03-26 23:38:34 -04:00
LinPeiWen
8721ca35d8 kolla-toolbox container name variable
kolla-toolbox container name variable is fixed in some places,
but in the defaults directory, kolla-toolbox container_name variable
is variable. If the kolla-toolbox container_name variable is changed
during deployment, it will not be assigned to kolla-toolbox,
but a fixed 'kolla-toolbox' name.

Change-Id: I9579017761ff47477dba597282be9ae6fab4242a
2020-03-26 08:58:54 -04:00
Jeffrey Zhang
34a331ab2f Add clients ca_file in heat
This patch fix creating statck resource failure in heat.

Change-Id: I00c23f8b89765e266d045cc463ce4d863d0d6089
Closes-Bug: #1869137
2020-03-26 18:15:33 +08:00
Jeffrey Zhang
04382c8031 Add glance_ca_certificates_file when using self sign cert in glance
Change-Id: I9395ae32378f4ff1fd57be78d7daec7745579e04
Closes-Bug: #1869133
2020-03-26 15:12:45 +08:00
Mark Goddard
f3350d4e13 Fix HAProxy prechecks during scale-out with limit
Deploy HAProxy on one or more servers. Add another server to the
inventory in the haproxy group, and run the following:

kolla-ansible prechecks --limit <new host>

The following task will fail:

    TASK [haproxy : Checking if kolla_internal_vip_address and
    kolla_external_vip_address are not pingable from any node]

This happens because ansible does not execute on hosts where
haproxy/keepalived is running, and therefore does not know that the VIP
should be active.

This change skips VIP prechecks when not all HAProxy hosts are in the
play.

Closes-Bug: #1868986

Change-Id: Ifbc73806b768f76f803ab01c115a9e5c2e2492ac
2020-03-25 17:10:48 +00:00
Zuul
c9ca7e5529 Merge "Fix kolla-ansible stop with heterogeneous hosts" 2020-03-25 15:09:30 +00:00
Zuul
6af7efe349 Merge "Fix telegraf with zookeeper (wrong port variable reference)" 2020-03-25 12:55:33 +00:00
LinPeiWen
8a206699d4 mariadb container name variable
mariadb container name variable is fixed in some places,
but in the defaults directory, mariadb container_name variable
is variable. If the mariadb container_name variable is changed
during deployment, it will not be assigned to container_name,
but a fixed 'mariadb' name.

Change-Id: Ie8efa509953d5efa5c3073c9b550be051a7f4f9b
2020-03-25 01:17:29 -04:00
Mark Goddard
89df07e8ef Fix kolla-ansible stop with heterogeneous hosts
The 'kolla-ansible stop' command can be used to stop the services
running on hosts. However, if you run this command in an environment
with heterogeneous nodes (most real world scenarios have at least
control/compute), then it fails. This is because it only checks
whether a container is enabled, and not whether the host is in the
correct group. For example, it fails with nova-libvirt:

    No such container: nova_libvirt to stop.

This change fixes the issue by only attempting to stop containers on
hosts to which they are mapped.

Change-Id: Ibecac60d1417269bbe25a280996ca9de6e6d018f
Closes-Bug: #1868596
2020-03-23 17:21:53 +00:00
Marcin Juszkiewicz
9bde29a30b cirros: upgrade to 0.5.1
We released CirrOS 0.5.1. Time to move then.

Change-Id: Ibca24836f19b3cbf6166fa39a3702883938feda8
2020-03-21 15:40:10 +00:00
Doug Szumski
505cded29b Support disabling Prometheus server
This is useful to people who manage their Prometheus Server
externally to Kolla Ansible, or want to use the exporters with
another framework such as Monasca.

Change-Id: Ie3f61e2e186c8e77e21a7b53d2bd7d2a27eee18e
2020-03-20 17:18:39 +00:00
Zuul
1740463987 Merge "Fix deployment of fluentd without any enabled OpenStack services" 2020-03-19 17:07:14 +00:00
Zuul
270fb4cf0b Merge "CentOS 8: Use master image tag by default" 2020-03-19 12:55:34 +00:00
Radosław Piliszek
561b33cbb2 Fix deployment of fluentd without any enabled OpenStack services
Fluentd cannot accept empty 'path' parameter.

I refactored the service list following the general pattern
we have.

Change-Id: I83d820efcc7e86bac9f8bda26a8f8bece72159e6
Closes-bug: #1867953
2020-03-18 21:17:54 +01:00
Zuul
31fe3c0651 Merge "Fix invalid JSON body in Elasticsearch API requests" 2020-03-18 10:47:57 +00:00
Zuul
c5afb1bbd3 Merge "Disable Fluentd Monasca plugin retry limit" 2020-03-17 18:47:12 +00:00
Zuul
5046463b10 Merge "Make Fluentd config folders readable" 2020-03-17 18:41:32 +00:00
Doug Szumski
c92378d788 Make Fluentd config folders readable
Currently, config folders lack the execute bit so Fluentd
cannot read the config and just does nothing when it starts up. This
change explicitly sets the execute bit on folders which need it,
rather than doing it in a more generic way which is more risky from
a security perspective.

Change-Id: Ia840f4b67043df4eaa654f47673dcdc973f13d9c
Closes-Bug: #1867754
2020-03-17 12:06:50 +00:00
Radosław Piliszek
cf918fbc29 Fix native openvswitch firewall driver in neutron-openvswitch-agent
ovs-ofctl is still being run by neutron-openvswitch-agent.
Potential removal is scheduled for Victoria.
Until then, we have to mount /run/openvswitch in there.

Change-Id: Ia73b5665cece523bb822f6a223335f6fae94fb6a
Closes-bug: #1867506
2020-03-16 19:36:40 +00:00
Mark Goddard
bab770a88b CentOS 8: Use master image tag by default
While supporting both CentOS 7 and 8, we used the tag 'master-centos8'
for CentOS 8 images. We are now ready to drop CentOS 7 support, and
Kolla is switching to publish CentOS 8 images using the master tag on
the master branch, so we should use this.

Depends-On: https://review.opendev.org/713265

Partially-Implements: blueprint centos-rhel-8

Change-Id: I07d2c285e3214a6dc827a8e8eacf263048ee099b
2020-03-16 15:58:55 +00:00
Jeffrey Zhang
6aceaee4ad Use proper es schema in fluentd when use internal tls
Fix elasticsearch schema in fluentd when kolla_enable_tls_internal is
true.

Change-Id: I51286d2def7a762d569740c1abc5b924b682ad9d
Closes-Bug: #1867481
2020-03-15 08:47:07 +08:00
Zuul
dd55cf6ed0 Merge "Copy ca certificates also to horizon container" 2020-03-12 21:11:35 +00:00
Radosław Piliszek
939b6f7958 Fix telegraf with zookeeper (wrong port variable reference)
Change-Id: I29f65c83b9bd45e463d868cf9a55611f33fe3177
Closes-bug: #1867179
2020-03-12 19:15:56 +01:00
yj.bai
ed2df25ebc Copy ca certificates also to horizon container
Add copy ca file to horizon container.
because:
Could not find a suitable TLS CA certificate bundle,
invalid path: /etc/pki/ca-trust/source/anchors/kolla-customca-haproxy-internal.crt

Closes-Bug: #1867121

Change-Id: I64d4dbeebd53048705005b61eb3c5b2104e8f2ed
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2020-03-12 16:59:53 +00:00
Mark Goddard
96151a35d0 Host OS prechecks follow up
We only log the release in the 'Checking host OS release or version'
precheck, but we allow either the release or version to be included in
the list. For example, on CentOS 7:

    CentOS release Core is not supported. Supported releases are: 8

Include the version in the failure message too.

Change-Id: I0302cd4fc94a0c3a6aa1dbac7b9fedf37c11b81e
Related: blueprint improve-prechecks
2020-03-11 17:08:17 +00:00
Zuul
cced7e04e5 Merge "Check supported host OS distributions in prechecks" 2020-03-11 11:19:54 +00:00
Zuul
e49448439b Merge "Fix HAProxy monitor VIP precheck" 2020-03-11 02:43:00 +00:00
Zuul
f867373a73 Merge "support ipv6 for grafana.ini.j2" 2020-03-11 02:29:54 +00:00
yj.bai
3e582a05fa support ipv6 for grafana.ini.j2
grafana not support ipv6 in grafana.ini.j2.

Closes-Bug: #1866141

Change-Id: Ia89a9283e70c10a624f25108b487528dbb370ee4
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2020-03-10 17:47:34 +00:00
Will Szumski
a1c51b73c7 Use macro to avoid repetition
I didn't use a for loop as the logic for omitting the
comma for the final element dirties the logic.

Change-Id: Id29d5deebcc5126d69a1bd8395e0df989f2081f0
2020-03-10 13:00:34 +00:00
Zuul
2a2ce059dc Merge "Add notify restart container when cert changed" 2020-03-10 12:12:55 +00:00
Zuul
98bd59455d Merge "Fix elasticsearch configuration in ipv6 environments" 2020-03-10 12:05:16 +00:00
Zuul
5fbadcd8c3 Merge "Delete stale fluent config on restart" 2020-03-10 12:05:15 +00:00
Mark Goddard
d20c65ed48 Check supported host OS distributions in prechecks
This should help to ensure that users are running tested and supported
host OS distributions.

Change-Id: I6ee76463d284ad4f3646af1c7ec2b7e50e2f3b15
Partially-Implements: blueprint improve-prechecks
2020-03-10 11:04:29 +00:00