11674 Commits

Author SHA1 Message Date
Michal Nasiadka
1f50f1be79 CI: Add centos linuxbridge job
Change-Id: I8aa02ac2caf320934be5ca439b9b9a24ac6aa573
2020-07-30 10:57:44 +02:00
Marcin Juszkiewicz
18e84f4d04 CI: drop not used Xenial nodeset definitions
Change-Id: I96c477063338c48b033694dd85482a8723fd498b
2020-07-29 17:41:55 +02:00
likui
3660c77637 Remove deprecated options in Trove
Option "network_label_regex" from group "DEFAULT" is
deprecated for removal.

Change-Id: I8aab2ca322159e61e4cbe9a5b30825a71a991e7e
2020-07-29 17:39:00 +08:00
Radosław Piliszek
3018199f0b Add timesync prechecks
If not running containerised chrony, we need to check that host
has its own means of system clock synchronization.

Change-Id: I31b3e9ed625d63a4bf82c674593522268c20ec4c
Partial-Bug: #1885689
2020-07-28 18:35:27 +00:00
Mark Goddard
56a07702bc Performance: use a single config file for fluentd
Currently we generate multiple fluentd configuration files for inputs,
filters, formatters and outputs.
These are then included from the main td-agent.conf configuration file.
With a large number of hosts, this can take a long time to template.

Benchmarking of templating is available at [1].

This change switches to a single fluentd configuration file, with the
include done locally. For the default template files included with Kolla
Ansible we use Jinja includes, but this does not work with templates in
a different directory. We therefore use the Ansible template lookup
plugin, which has a slightly higher overhead than a jinja include, but
far lower than generating multiple templates. This should drastically
improve the performance of this task.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/template.md

Partially-Implements: blueprint performance-improvements

Change-Id: Ia8623be0aa861fea3e54d2c9e1c971dfd8e3afa9
2020-07-28 12:14:48 +01:00
Mark Goddard
f329af7dfa Performance: use a single config file for logrotate
Currently we generate a logrotate configuration file for each enabled
service. These are then included from a logrotate.d directory. With a
large number of hosts, this can take a long time to template.

Benchmarking of templating is available at [1].

This change switches to a single logrotate configuration file for all
services, with the include done locally using jinja. This should
drastically improve the performance of this task.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/template.md

Partially-Implements: blueprint performance-improvements

Change-Id: I39cfa70bef6560f615cad516c43aaef6a523b964
2020-07-28 12:14:47 +01:00
Mark Goddard
9702d4c3c3 Performance: use import_tasks for check-containers.yml
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the check-containers.yml include, the
included file only has a single task, so the overhead of skipping this
task will not be greater than the overhead of the task import. It
therefore makes sense to switch to use import_tasks there.

Partially-Implements: blueprint performance-improvements

Change-Id: I65d911670649960708b9f6a4c110d1a7df1ad8f7
2020-07-28 12:10:59 +01:00
Zuul
da1d14b186 Merge "Move TLS documentation to its own page" 2020-07-28 10:30:41 +00:00
Zuul
2966766fc2 Merge "Drop a no-longer-relevant note" 2020-07-28 10:30:40 +00:00
Radosław Piliszek
fffe9021ff Drop a no-longer-relevant note
Modern Ansible handles this just fine.

Change-Id: Iea4d0499b92e2449ef8bc01651af6d3548ceab20
2020-07-27 17:34:54 +02:00
Radosław Piliszek
e1e8533c89 Drop RDP console variables
These are noop after Hyper-V support was removed.

Change-Id: Ib451b154893e5cedc366aed83c35f48d92c7ab82
2020-07-27 15:38:56 +02:00
Justinas Balciunas
9fc98be11a prometheus-openstack-exporter config service filtering
This change disables services in the Prometheus openstack-exporter
if they are not enabled in the deployment. Such behaviour allows
to avoid warnings and errors in the log files and keep the
log file contents clean and informative.

Change-Id: I4dcac976620a5f451e3d273183199aefe400994a
2020-07-27 13:30:26 +00:00
Zuul
21f5a02604 Merge "Remove Hyper-V integration" 2020-07-27 12:47:33 +00:00
Zuul
34ace98ff4 Merge "Improve Grafana DB bootstrap" 2020-07-27 11:57:49 +00:00
Zuul
cd9afc5ba3 Merge "Set Kafka default replication factor" 2020-07-27 11:57:45 +00:00
Zuul
676cfa5c1f Merge "fluentd: log to a file instead of stdout" 2020-07-27 10:57:43 +00:00
Mark Goddard
3870c74d0b Move TLS documentation to its own page
Moved the TLS documentation from "advanced-configuration" doc to its
own TLS document. This is in preparation for improving it.

Change-Id: I4c83f1810ef1222aaa3560174c1ba39328853c4e
Co-Authored-By: James Kirsch <generalfuzz@gmail.com>
2020-07-27 10:05:58 +00:00
Christian Berendt
6eb02245d6 Remove Hyper-V integration
Change-Id: I2e22ec47f644de2f1509a0111c9e1fffe8da0a1a
2020-07-27 10:25:46 +01:00
Dincer Celik
fc7ce6cabe [docker] Added a new flag to disable default iptables rules
Docker is manipulating iptables rules by default to provide network
isolation, and this might cause problems if the host already has an
iptables-based firewall.

This change introduces docker_disable_default_iptables_rules to
disable the iptables manipulation by putting "iptables: false" [1] to
daemon.json

For better defaults, this feature will be enabled by default in
Victoria.

[1] https://docs.docker.com/network/iptables/

Closes-Bug: #1849275

Change-Id: I165199fc98fb98f227f2a20284e1bab03ef65b5b
2020-07-27 09:09:45 +00:00
Doug Szumski
2c730590d7 Improve Grafana DB bootstrap
This fixes an issue where multiple Grafana instances would race
to bootstrap the Grafana DB. The following changes are made:

- Only start additional Grafana instances after the DB has been
  configured.

- During upgrade, don't allow old instances to run with an
  upgraded DB schema.

Change-Id: I3e0e077ba6a6f43667df042eb593107418a06c39
Closes-Bug: #1888681
2020-07-27 08:23:05 +00:00
Doug Szumski
a273e28e20 Set Kafka default replication factor
This ensures that when using automatic Kafka topic creation, with more than one
node in the Kafka cluster, all partitions in the topic are automatically
replicated. When a single node goes down in a >=3 node cluster, these topics will
continue to accept writes providing there are at least two insync replicas.

In a two node cluster, no failures are tolerated. In a three node cluster, only a
single node failure is tolerated. In a larger cluster the configuration may need
manual tuning.

This configuration follows advice given here:

[1] https://docs.cloudera.com/documentation/kafka/1-2-x/topics/kafka_ha.html#xd_583c10bfdbd326ba-590cb1d1-149e9ca9886--6fec__section_d2t_ff2_lq

Closes-Bug: #1888522

Change-Id: I7d38c6ccb22061aa88d9ac6e2e25c3e095fdb8c3
2020-07-27 08:23:05 +00:00
Michal Nasiadka
696533f228 fluentd: log to a file instead of stdout
fluentd logs currently to stdout, which is known to produce big docker logs
in /var/lib/docker. This change makes fluentd to log to /var/log/kolla/fluentd.

Closes-Bug: #1888852
Change-Id: I8fe0e54cb764a26d26c6196cef68aadc6fd57b90
2020-07-27 07:13:13 +00:00
Zuul
61e32bb131 Merge "Revert "Remove the waiting for ironic-api to be accessible"" 2020-07-25 09:57:55 +00:00
Mark Goddard
4a6050a333 Revert "Remove the waiting for ironic-api to be accessible"
This reverts commit 8fc86893893685e828600e21ddba147b64f0adc3.

It appears that it is still necessary to wait for ironic to be up, otherwise inspector may fail to start:

The baremetal service for 192.0.2.10:None exists but does not have any supported versions.

Change-Id: Ibc8314c91113618ce9e92b8933a63eba3cf3bbe1
2020-07-24 14:55:13 +00:00
Zuul
9a141eb144 Merge "Fix some CloudKitty API responses when behind SSL" 2020-07-24 10:38:57 +00:00
Zuul
ef38c505f8 Merge "Add support for encrypting etcd service" 2020-07-24 07:53:50 +00:00
Zuul
98f773d0be Merge "Masakari: copy TLS certificates into containers" 2020-07-24 07:53:48 +00:00
Zuul
39909a600c Merge "Performance: remove unnecessary conditions from includes" 2020-07-24 07:52:37 +00:00
Zuul
2db1da1ea6 Merge "add missing conditional for octavia" 2020-07-23 15:18:26 +00:00
Zuul
44280c0069 Merge "Update glance configuration file for backend" 2020-07-23 14:31:50 +00:00
wu.chunyang
5c0f18793c add missing conditional for octavia
octavia deploy failed due to mount a empyt directroy into container

Change-Id: Ifd95126da59f649b02ab39c0b209df4750bdcfce
2020-07-23 20:05:16 +08:00
Mark Goddard
0b4c8a3c3d Masakari: copy TLS certificates into containers
From Ussuri, if CA certificates are copied into
/etc/kolla/certificates/ca/, these should be copied into all containers.
This is not being done for masakari currently.

Additionally, we are not setting the [DEFAULT] nova_ca_certificates_file
option in masakari.conf. This depends on masakari bug 1873736 being
fixed to work.

This change fixes these issues.

Change-Id: I9a3633f58e5eb734fa32edc03a3022a500761bbb
Closes-Bug: #1888655
2020-07-23 12:06:24 +01:00
Pierre Riteau
cd55c8f4b2 Fix some CloudKitty API responses when behind SSL
Some CloudKitty API responses include a Location header using http
instead of https. Seen with `openstack rating module enable hashmap`.

Change-Id: I11158bbfd2006e3574e165b6afc9c223b018d4bc
Closes-Bug: #1888544
2020-07-22 18:59:36 +02:00
Zuul
6bbc713885 Merge "Fix fluentd warnings caused by "type copy"" 2020-07-22 12:35:38 +00:00
Zuul
ca578c98b6 Merge "fix deploy freezer failed when kolla_dev_mod enabled" 2020-07-22 12:32:45 +00:00
Zuul
6033959b53 Merge "CI: Update ceph-ansible to v5 - deploy Ceph Octopus" 2020-07-22 12:32:43 +00:00
Zuul
8519ceb443 Merge "Update trove config file" 2020-07-22 12:32:42 +00:00
Zuul
b0407ffb17 Merge "Make /dev/kvm permissions handling more robust" 2020-07-22 12:32:40 +00:00
likui
3d04199692 Update glance configuration file for backend
global file glance_backend_file parameters not take effect

Closes-Bug: #1888501

Change-Id: I3afd117633a84d342effb6baadf16fa42c16776c
2020-07-22 20:30:29 +08:00
Zuul
8fc8dec3f7 Merge "CI: Add CentOS KVM job" 2020-07-22 11:54:28 +00:00
Zuul
f039de861d Merge "CI: Add Ubuntu KVM job" 2020-07-22 11:54:26 +00:00
Pierre Riteau
5b05781241 Fix fluentd warnings caused by "type copy"
A "@type copy" statement is already present at the beginning of each
match element, so extra "type copy" are not needed. They are causing the
following warnings in fluentd logs:

[warn]: parameter 'type' in <match syslog.local0.**>
[warn]: parameter 'type' in <match syslog.local1.**>

This commit also harmonizes indentation of the Monasca config block.

Change-Id: I779c2b942d007acbdd43d999f2fc0cdc131d431f
Related-Bug: #1885873
2020-07-21 14:35:43 +02:00
Pierre Riteau
cf97aeeb83 Configure prometheus-openstack-exporter to use internal endpoints
Change-Id: Ia134a518b63bb59cfad631cc488181f5245160e6
2020-07-21 09:38:49 +02:00
wu.chunyang
7dc471323c fix deploy freezer failed when kolla_dev_mod enabled
we should clone freezer code before run bootstray,
otherwise, the directory /opt/stack/freezer which is empty will
mount into freezer_api container.

Closes-Bug: #1888242

Change-Id: I7c22dd380fd5b1dff7b421109c4ae37bab11834a
2020-07-21 10:32:21 +08:00
likui
712b27daea Update trove config file
Option "trove_auth_url/os_region_name" from group "DEFAULT" is deprecated.
Use option "auth_url/region_name" from group service_credentials

Change-Id: I15d6891582c92c7fc813f280a2b47ebaaca77eba
2020-07-21 09:53:11 +08:00
Radosław Piliszek
202365e702 Make /dev/kvm permissions handling more robust
This makes use of udev rules to make it smarter and override
host-level packages settings.
Additionally, this masks Ubuntu-only service that is another
pain point in terms of /dev/kvm permissions.
Fingers crossed for no further surprises.

Change-Id: I61235b51e2e1325b8a9b4f85bf634f663c7ec3cc
Closes-bug: #1681461
2020-07-17 17:51:18 +00:00
Radosław Piliszek
f2a5d374ca CI: Add CentOS KVM job
Follow-up on [1] which added Ubuntu.

[1] https://review.opendev.org/735454

Change-Id: If0baef37db392324c285798fa55d5e85892edb9f
2020-07-17 17:51:08 +00:00
Zuul
376988125a Merge "Add py38 package metadata" 2020-07-17 17:01:32 +00:00
Zuul
9e7f055f1a Merge "Use Confluent Kafka client in remaining Monasca services" 2020-07-17 15:43:57 +00:00
Zuul
9a8341c2a7 Merge "Performance: Run common role in a separate play" 2020-07-17 15:43:22 +00:00