10510 Commits

Author SHA1 Message Date
Zuul
4b4c10af2b Merge "Use default BGP speaker driver in bgp_dragent.ini" 2019-08-19 07:31:13 +00:00
Zuul
b93e33e78e Merge "Remove nova [DEFAULT]firewall_driver option" 2019-08-19 07:07:01 +00:00
Radosław Piliszek
b693746cb0 Fix Zun connectivity to itself and Cinder
Zun was misconfigured and defaulted to using public endpoints
which are likely inaccessible from the internal network.
This patch fixes that and removes unused and deprecated
options. Validity of options confirmed from Queens to Train
against respective docs.

Change-Id: I25cc8792351c43eb9ff45465e49fa72ceccd6cb5
Closes-bug: #1840572
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-18 17:13:04 +02:00
Zuul
d9dd536cf8 Merge "CI: Zun jobs" 2019-08-17 11:58:46 +00:00
Zuul
5394cf187d Merge "Allow to configure docker for Zun" 2019-08-17 11:58:45 +00:00
Zuul
58cca6801c Merge "Allow cinder coordination backend to be configured" 2019-08-16 16:06:01 +00:00
Zuul
483304dc03 Merge "update horizon configuration for python3 migrating" 2019-08-16 16:05:59 +00:00
Zuul
83d8b1053c Merge "CI: Add docker inspect output to docker_info logs" 2019-08-16 15:40:38 +00:00
Mark Goddard
a14eee24d1 Check for CRITICAL, WARNING and ERROR log messages in CI
At the end of a CI run, check all log files.

Change-Id: I99afc1c5207757e35beabf7daebd86c56151c96d
2019-08-16 15:33:54 +00:00
Radosław Piliszek
d4de1d7520 CI: Zun jobs
- Test Zun on CentOS too
- Make etcd change also trigger Zun jobs (like kuryr and zun)
- Test multinode Zun deployments instead of AIO
  (more likely to break)
- In Zun scenario, stop configuring docker for legacy swarm mode
  (Zun is no swarm)
- Separate test-zun.sh testing script
- Show appcontainer to see which node it has been started on

Change-Id: I289b1009fe00aedb9b78cbd83298b14da5fd9670
Depends-On: https://review.opendev.org/676736
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-16 17:11:00 +02:00
Radosław Piliszek
44f88d16ac Allow to configure docker for Zun
Change-Id: Icf3f01516185afb7b9f642407b06a0204c36ecbe
Closes-Bug: #1840315
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-16 15:35:11 +02:00
Scott Solkhon
09e02ef8f1 Support configuration of trusted CA certificate file
This commit adds the functionality for an operator to specify
their own trusted CA certificate file for interacting with the
Keystone API.

Implements: blueprint support-trusted-ca-certificate-file
Change-Id: I84f9897cc8e107658701fb309ec318c0f805883b
2019-08-16 12:47:42 +00:00
Michal Nasiadka
8cf24bcc81 CI: Add docker inspect output to docker_info logs
Change-Id: I081f2f4762651bca935f08a67b20f21946aaf051
2019-08-16 09:30:16 +00:00
Radosław Piliszek
de2f7be981 TrivialFix: remove unused template var
Change-Id: I7d0ed4ad94e3d07220de131b2a0fcd399d942782
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-16 08:25:22 +02:00
Zuul
aa135e37f7 Merge "Standardize the configuration of "oslo_messaging" section" 2019-08-15 20:04:56 +00:00
Zuul
fac646406f Merge "Testing Masakari role in gate" 2019-08-15 17:26:56 +00:00
Zuul
bf372c2502 Merge "Add Masakari Ansible role" 2019-08-15 16:36:44 +00:00
Rafael Weingärtner
22a6223b1b Standardize the configuration of "oslo_messaging" section
After all of the discussions we had on
"https://review.opendev.org/#/c/670626/2", I studied all projects that
have an "oslo_messaging" section. Afterwards, I applied the same method
that is already used in "oslo_messaging" section in Nova, Cinder, and
others. This guarantees that we have a consistent method to
enable/disable notifications across projects based on components (e.g.
Ceilometer) being enabled or disabled. Here follows the list of
components, and the respective changes I did.

* Aodh:
The section is declared, but it is not used. Therefore, it will
be removed in an upcomming PR.

* Congress:
The section is declared, but it is not used. Therefore, it will
be removed in an upcomming PR.

* Cinder:
It was already properly configured.

* Octavia:
The section is declared, but it is not used. Therefore, it will
be removed in an upcomming PR.

* Heat:
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Ceilometer:
Ceilometer publishes some messages in the rabbitMQ. However, the
default driver is "messagingv2", and not ''(empty) as defined in Oslo;
these configurations are defined in ceilometer/publisher/messaging.py.
Therefore, we do not need to do anything for the
"oslo_messaging_notifications" section in Ceilometer

* Tacker:
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Neutron:
It was already properly configured.

* Nova
It was already properly configured. However, we found another issue
with its configuration. Kolla-ansible does not configure nova
notifications as it should. If 'searchlight' is not installed (enabled)
the 'notification_format' should be 'unversioned'. The default is
'both'; so nova will send a notification to the queue
versioned_notifications; but that queue has no consumer when
'searchlight' is disabled. In our case, the queue got 511k messages.
The huge amount of "stuck" messages made the Rabbitmq cluster
unstable.

https://bugzilla.redhat.com/show_bug.cgi?id=1478274
https://bugs.launchpad.net/ceilometer/+bug/1665449

* Nova_hyperv:
I added the same configurations as in Nova project.

* Vitrage
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Searchlight
I created a mechanism similar to what we have in AODH, Cinder, Nova,
and others.

* Ironic
I created a mechanism similar to what we have in AODH, Cinder, Nova,
and others.

* Glance
It was already properly configured.

* Trove
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Blazar
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Sahara
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Watcher
I created a mechanism similar to what we have in AODH, Cinder, Nova,
and others.

* Barbican
I created a mechanism similar to what we have in Cinder, Nova,
and others. I also added a configuration to 'keystone_notifications'
section. Barbican needs its own queue to capture events from Keystone.
Otherwise, it has an impact on Ceilometer and other systems that are
connected to the "notifications" default queue.

* Keystone
Keystone is the system that triggered this work with the discussions
that followed on https://review.opendev.org/#/c/670626/2. After a long
discussion, we agreed to apply the same approach that we have in Nova,
Cinder and other systems in Keystone. That is what we did. Moreover, we
introduce a new topic "barbican_notifications" when barbican is
enabled. We also removed the "variable" enable_cadf_notifications, as
it is obsolete, and the default in Keystone is CADF.

* Mistral:
It was hardcoded "noop" as the driver. However, that does not seem a
good practice. Instead, I applied the same standard of using the driver
and pushing to "notifications" queue if Ceilometer is enabled.

* Cyborg:
I created a mechanism similar to what we have in AODH, Cinder, Nova,
and others.

* Murano
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Senlin
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Manila
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Zun
The section is declared, but it is not used. Therefore, it will
be removed in an upcomming PR.

* Designate
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

* Magnum
It was already using a similar scheme; I just modified it a little bit
to be the same as we have in all other components

Closes-Bug: #1838985

Change-Id: I88bdb004814f37c81c9a9c4e5e491fac69f6f202
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2019-08-15 13:18:16 -03:00
Mark Goddard
d54c8fbdcc Use internal API for heat -> heat communication
Heat has a new option (server_keystone_endpoint_type), which can be used
to set the keystone endpoint used by instances to make callbacks to
heat. This needs to be public, since we can't assume users have access
to the internal API. However, the current method of setting
[clients_heat] endpoint_type means that communication from heat to its
own API (e.g. when a stack is a resource in another stack) uses the
public network also, and this might not work if TLS is enabled.

This change uses server_keystone_endpoint_type to keep instance traffic
on the public API, and removes the [clients_heat] endpoint_type option
to use the default in [clients] endpoint_type of internalURL.

This feature was added to heat in https://review.opendev.org/#/c/650967.

Change-Id: I932ea55a3c2a411557c34361db08bcb3a2b27eaf
Closes-Bug: #1812864
Related-Bug: #1762754
Related-Bug: #1688331
2019-08-15 16:04:44 +01:00
Kien Nguyen
577bb50a04 Add Masakari Ansible role
Masakari provides Instances High Availability Service for
OpenStack clouds by automatically recovering failed Instances.

Depends-On: https://review.openstack.org/#/c/615469/
Change-Id: I0b3457232ee86576022cff64eb2e227ff9bbf0aa
Implements: blueprint ansible-masakari
Co-Authored-By: Gaëtan Trellu <gaetan.trellu@incloudus.com>
2019-08-15 09:58:53 -04:00
Scott Solkhon
03cd7eb356 Wait for MariaDB to be accessible via HAProxy
Explicitly wait for the database to be accessible via the load balancer.
Sometimes it can reject connections even when all database services are up,
possibly due to the health check polling in HAProxy.

Closes-Bug: #1840145
Change-Id: I7601bb710097a78f6b29bc4018c71f2c6283eef2
2019-08-15 10:00:36 +00:00
Zuul
6db0892fc7 Merge "Fix idempotency of fluentd customisations" 2019-08-15 08:40:34 +00:00
Radosław Piliszek
03b4c706fa Allow cinder coordination backend to be configured
This is to allow operator to prevent enabling redis and/or
etcd from magically configuring cinder coordinator.

Note this change is backwards-compatible.

Change-Id: Ie10be55968e43e3b9cc347b1b58771c1f7b1b910
Related-Bug: #1840070
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-15 07:48:28 +00:00
Zuul
dda1885151 Merge "Enable iscsid on cinder-backup hosts" 2019-08-15 03:34:37 +00:00
Zuul
f27a19680b Merge "Add missing when condition for swift config files" 2019-08-14 20:07:12 +00:00
chenxing
673bbd5678 update horizon configuration for python3 migrating
Depends-On: https://review.opendev.org/675581
Closes-Bug: #1838719
Partially Implements: blueprint python3-support

Change-Id: Ib8bfb130b8490b583539cc264c2d2a2a034b270c
2019-08-14 17:58:04 +00:00
Zuul
9401aab752 Merge "CI: Sanity check that nodepool.private_ipv4 is assigned" 2019-08-14 17:21:29 +00:00
Scott Solkhon
8acbb32b95 Add missing when condition for swift config files
Change-Id: If5bba855a6e34c971fdb1ceb6f10dba62e54b811
2019-08-14 16:52:42 +00:00
Kien Nguyen
fbac54c5f5 Testing Masakari role in gate
Add Masakari testing into the Gate.

Change-Id: I52df33f963e7d2ae4059887df3d24d9e6642134e
Depends-On: https://review.opendev.org/#/c/615469/
Depends-On: https://review.opendev.org/#/c/615715
Implements: blueprint ansible-masakari
Co-Authored-By: Gaëtan Trellu <gaetan.trellu@incloudus.com>
2019-08-14 12:32:51 -04:00
Scott Solkhon
dcaa5f0b3d Fix idempotency of fluentd customisations
Fix fluentd config from overwriting custom config with the same filename

Closes-Bug: #1840166
Change-Id: I42c5446381033015f590901b2120950d602f847f
2019-08-14 15:53:49 +00:00
Zuul
b599f78dd7 Merge "Add missing Octavia policy file to Horizon" 2019-08-14 15:27:38 +00:00
Zuul
17029c7e71 Merge "Configure Telegraf to monitor Docker containers" 2019-08-14 14:00:17 +00:00
Zuul
495be668d8 Merge "Add support for Swift S3 API" 2019-08-14 12:28:19 +00:00
Scott Solkhon
b3d07a4b52 Add missing Octavia policy file to Horizon
This commit adds the missing policy file for Octavia
in Horizon, thus enabling the panel where appropriate.

Change-Id: I60f1a52de71519f2d8bd84baa8aba5700fa75b1c
2019-08-14 12:00:59 +00:00
Scott Solkhon
46f9ad3a96 HAProxy backend connection limits
The default connection limits for backends is 2000
however, mariadb defaults to a max of 10000 conections,
therefore changing this limit to match the mariadb limit.

'haproxy_max_connections' also needs to be bumped
for this to work.

Change-Id: I5ded328485855f3f3d4390282040b0d89d08d997
2019-08-14 10:44:31 +00:00
Scott Solkhon
d72b27f2d1 Add support for Swift S3 API
This feature is disabled by default, and can be enabled by setting
'enable_swift_s3api' to 'true' in globals.yml.

Two middlewares are required for Swift S3 - s3api and s3token. Additionally, we
need to configure the authtoken middleware to delay auth decisions to give
s3token a chance to authorise requests using EC2 credentials.

Change-Id: Ib8e8e3a1c2ab383100f3c60ec58066e588d3b4db
2019-08-14 09:55:35 +00:00
Zuul
64d587b819 Merge "Fix swift log level configuration" 2019-08-13 17:03:47 +00:00
Scott Solkhon
dea87cde97 Fix swift log level configuration
Change-Id: I7f980640e75a9328a14a3e14e9c55358955f3182
2019-08-13 12:28:38 +00:00
Keith Plant
b95ff2d1db Configure Telegraf to monitor Docker containers
Added configuration to ansible/roles/telegraf/templates/telegraf.conf.j2 to
allow telegraf to grab telemetry data from docker directly.

Added option to etc/kolla/globals.yml to switch on/off the configuration to
ingest data from the docker daemon into telegraf.

Change-Id: Icbebc415d643a237fa128840d5f5a9c91d22c12d
Signed-off-by: Keith Plant <kplantjr@gmail.com>
2019-08-13 08:17:00 -04:00
Isaac Prior
3010d4c391 Fixes Monasca log transformer UTC offset exception
Monasca log transformer currently throws exceptions on encountering a 
non-UTC time offset (+0000):
"""
"exception": "Invalid format: \"2019-08-08 17:39:45 +0100\" is malformed at \" +0100\"",
"config_parsers":"yyyy-MM-dd HH:mm:ss +0000,ISO8601"}
"""

This fix allows logstash to interpret any valid ISO8601 offset.

Change-Id: Id70c3dd9cdcf681e955931f18a054e19cc284c0a
Closes-Bug: #1839597
2019-08-13 08:46:29 +00:00
Zuul
5c70e0a615 Merge "Set 'distro_python_version' variable" 2019-08-13 04:00:55 +00:00
Zuul
571c89712d Merge "CI: Collect docker and systemd configs" 2019-08-12 17:19:36 +00:00
Marcin Juszkiewicz
bf7ed6be04 Set 'distro_python_version' variable
We use that variable in Kolla in many places. There are places in
'kolla-ansible' where we also need it.

Change-Id: Iea78c4a7cb0fd1405ea7299cdcf0841f63820c8c
2019-08-12 13:23:42 +00:00
Zuul
b16bb0d787 Merge "Do not require EPEL repo on RHEL-based target hosts" 2019-08-10 00:33:53 +00:00
Zuul
4468250b95 Merge "Remove support for Docker legacy packages" 2019-08-09 15:27:09 +00:00
Zuul
3a37131f1d Merge "Fix FWaaS service provider (v2, Stein issue)" 2019-08-09 12:05:56 +00:00
Radosław Piliszek
85a5fb55c4 Fix FWaaS service provider (v2, Stein issue)
Because we merged both [1] and [2] in master,
we got broken FWaaS.
This patch unbreaks it and is required to backport
to Stein due to [2] backport waiting for merge,
while [1] is already backported.

[1] https://review.opendev.org/661704
[2] https://review.opendev.org/668406

Change-Id: I74427ce9b937c42393d86574614603bd788606af
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-08 20:26:57 +02:00
Doug Szumski
339ea2bdeb Support namespacing RabbitMQ logs
The RabbitMQ role supports namespacing the service via the
project_name. For example, if you change the project_name, the
container name and config directory will be renamed accordingly. However
the log folder is currently fixed, even though the service tries to
write to one named after the project_name. This change fixes that.

Whilst you might generally use vhosts, running multiple RabbitMQ
services on a single node is useful at the very least for testing,
or for running 'outward RabbitMQ' on the same node.

This change is part of the work to support Cells v2.

Partially Implements: blueprint support-nova-cells
Change-Id: Ied2c24c01571327ea532ba0aaf2fc5e89de8e1fb
2019-08-08 16:46:32 +00:00
Zuul
ee5e99fcf5 Merge "Stop using MountFlags=shared in Docker configuration" 2019-08-08 10:57:03 +00:00
Doug Szumski
65b9756127 Add support for using custom Logstash patterns
A user may want to define and use Logstash patterns. This
commit adds support to copy them into the Monasca Log
Transformer container. In the future support could be
added for other Logstash containers.

Change-Id: Id8cde14af6dc7f49714f6b1cb878882d0048d293
2019-08-08 10:48:35 +01:00