5584 Commits

Author SHA1 Message Date
Zuul
80b895a45b Merge "Fix kolla-ansible not reflect environment changed" 2020-08-21 18:04:38 +00:00
Zuul
e53dae8eff Merge "Add cinder auth config to nova-cell nova.conf.j2" 2020-08-21 15:45:02 +00:00
Michal Arbet
ed2a2704d1 Fix kolla-ansible not reflect environment changed
Neutron's containers should use ENV from kolla_docker module's
environment parameter (defined in roles/neutron/defaults/main.yml)
after reconfigure, not only when deploying.

Currently this is working only for deploy, not for reconfigure.

How to test it ?
- Deploy neutron with "neutron_legacy_iptables" set to yes/no.
- Change value of "neutron_legacy_iptables" to opposite value as before.
- Reconfigure neutron.

Current result :
 - "KOLLA_LEGACY_IPTABLES" in container's ENV is not changed

Expected result :
 - "KOLLA_LEGACY_IPTABLES:" in container's ENV should be changed

This patch is fixing this behaviour by adding missing
environment parameter to neutron's "Check neutron containers" task.

Change-Id: Ibfbe2d4f49261fa766acbb6ff45da9994118bda8
Closes-Bug: #1853776
2020-08-21 14:51:41 +00:00
Michal Nasiadka
88c53da22f Fix ansible-lint after upgrade to 4.3.1
Change-Id: I3a31c74608b48919c1563bac88313a8c4219dedc
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2020-08-21 10:06:06 +00:00
Pierre Riteau
5a52d8e4a0 Add workaround for keystonemiddleware/neutron memcached issue
There is an issue where keystonemiddleware connections to memcached from
neutron-server grow beyond configured values [1], eventually reaching
the maximum number of connections accepted by memcached servers. Other
services do not appear to be affected by this issue.

A workaround is to use the advanced memcached pool. Despite its
documentation claiming to only work with Python 2, it appears to work
fine on Python 3.

[1] https://bugs.launchpad.net/keystonemiddleware/+bug/1883659

Change-Id: Ifbbc2022839cbc575848d830600241c61603c80b
Closes-Bug: #1892210
2020-08-19 17:56:30 +02:00
Jegor van Opdorp
de16013bd6 Add cinder auth config to nova-cell nova.conf.j2
Fixes an issue during deleting evacuated instances with encrypted block
devices.

Change-Id: I9b9b689ef7e1e41b597e2c5f6b96f3ed011193c5
Closes-Bug: 1891462
Related-Bug: 1850279
2020-08-19 07:25:20 +00:00
Florian LEDUC
56710de59d Use iSCSI multipath for libvirt
* Multipath daemon allows to reach block devices
via multiple paths for better resiliency and performance.
Multipathd periodically checks the failed iscsi paths
and maintains a list of valid paths. Libvirt can use more
than one iSCSI path when option volume_use_multipath is set
and when multipathd enabled.

Change-Id: I54629656803c4989f7673e8c69d2a820609b5960
Implements: blueprint nova-libvirt-multipath-iscsi
2020-08-19 07:24:51 +00:00
Rafael Weingärtner
f425c0678f Standardize use and construction of endpoint URLs
The goal for this push request is to normalize the construction and use
 of internal, external, and admin URLs. While extending Kolla-ansible
 to enable a more flexible method to manage external URLs, we noticed
 that the same URL was constructed multiple times in different parts
 of the code. This can make it difficult for people that want to work
 with these URLs and create inconsistencies in a large code base with
 time. Therefore, we are proposing here the use of
 "single Kolla-ansible variable" per endpoint URL, which facilitates
 for people that are interested in overriding/extending these URLs.

As an example, we extended Kolla-ansible to facilitate the "override"
of public (external) URLs with the following standard
"<component/serviceName>.<companyBaseUrl>".
Therefore, the "NAT/redirect" in the SSL termination system (HAproxy,
HTTPD or some other) is done via the service name, and not by the port.
This allows operators to easily and automatically create more friendly
 URL names. To develop this feature, we first applied this patch that
 we are sending now to the community. We did that to reduce the surface
  of changes in Kolla-ansible.

Another example is the integration of Kolla-ansible and Consul, which
we also implemented internally, and also requires URLs changes.
Therefore, this PR is essential to reduce code duplicity, and to
facility users/developers to work/customize the services URLs.

Change-Id: I73d483e01476e779a5155b2e18dd5ea25f514e93
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2020-08-19 07:22:17 +00:00
Radosław Piliszek
479fb85d0d Fix tz handling for newly introduced services
We missed to ensure new neutron switch use the new condition for
timezone file mounting.

Change-Id: I838e526b930d5276d3ce24f5188262af7eb33280
Closes-Bug: #1882553
2020-08-19 07:21:05 +00:00
Zuul
ad50147194 Merge "Deploy neutron-mlnx-agent and neutron-eswitchd containers" 2020-08-17 19:02:45 +00:00
Zuul
9275fc1368 Merge "Add support to use bifrost-deploy behind proxy" 2020-08-17 15:17:37 +00:00
Bartosz Bezak
9da39345ad Add support to use bifrost-deploy behind proxy
Change-Id: If90c2dfd32c8bc50671f6dd38e5a82b434c07151
Depends-On: https://review.opendev.org/#/c/720338
2020-08-17 10:29:29 +02:00
Zuul
a726cda8d1 Merge "Add support for encrypting Nova API" 2020-08-16 20:04:10 +00:00
Zuul
a4aab1b520 Merge "Add missing mistral services log files" 2020-08-15 20:58:11 +00:00
James Kirsch
d6251506f7 Add support for encrypting Nova API
This patch introduces an optional backend encryption for the Nova API
service. When used in conjunction with enabling TLS for service API
endpoints, network communcation will be encrypted end to end, from
client through HAProxy to the Nova service.

Change-Id: I48e1540b973016079d5686b328e82239dcffacfd
Partially-Implements: blueprint add-ssl-internal-network
2020-08-15 13:22:44 +00:00
Bharat Kunwar
4809462f4e Deploy neutron-mlnx-agent and neutron-eswitchd containers
Change-Id: I173669bdf92b1f2ea98907ba16808ca3c914944c
2020-08-13 23:33:57 +01:00
Zuul
d1e5de2120 Merge "Add Keep Alive Timeout for httpd" 2020-08-13 15:27:39 +00:00
Mark Goddard
8389140f05 Prevent overwriting existing Keystone Fernet keys
Steps to reproduce:

* Deploy a cloud
* Add another controller to the inventory
* Deploy to the new controller using --limit:

kolla-ansible deploy --limit new-controller

Expected results:

The new controller uses the cluster's existing fernet keys.

Actual results:

New fernet keys are generated on the new controller, and pushed out to
the existing controllers. This invalidates tokens created from those
keys.

This change prevents the above scenario from happening, by failing the
deployment if there are no hosts with existing Ferney keys to
distribute, and not all Keystone hosts are in the target host list.

Closes-Bug: #1891364

Change-Id: If0c0e038b77fc010a3a017f9841a674d53b16457
2020-08-13 15:43:15 +01:00
James Kirsch
19b028e660 Add Keep Alive Timeout for httpd
This patch introduces a global keep alive timeout value for services
that leverage httpd + wsgi to handle http/https requests. The default
value is one minute.

Change-Id: Icf7cb0baf86b428a60a7e9bbed642999711865cd
Partially-Implements: blueprint add-ssl-internal-network
2020-08-13 09:52:40 +00:00
Pierre Riteau
68a128d74f Synchronize REST_API_REQUIRED_SETTINGS with Horizon
Backport to Ussuri unmodified. Backport to Train and Stein without
DEFAULT_BOOT_SOURCE.

Closes-Bug: #1891024

Change-Id: If8fe490c3f698ab3eb37735fbfcb8ab0d5fa8a06
2020-08-13 08:02:12 +00:00
Zuul
516658f489 Merge "Mount /etc/timezone based on host OS" 2020-08-12 22:09:19 +00:00
Zuul
580f929dfa Merge "ubuntu: move to 20.04 Focal" 2020-08-11 15:26:39 +00:00
Marcin Juszkiewicz
352f91ac10 ubuntu: move to 20.04 Focal
There is a time once every 2 years when ubuntu team releases new LTS
release. And then UCA joins with binary packages for current OpenStack
development cycle.

It is this time for Ubuntu 20.04 'focal'.

Includes CI fix to pass:

[CI] Temporarily block new Ansible

The proper fix [1] needs fixing older branches before newer.
This one allows to fix CI first, in the usual order.

To revert after [1] gets merged in all relevant branches.

[1] https://review.opendev.org/745648

Old-Change-Id: Ifbd37d8addd4322773118e2e9d46494741a8ae66
Related-Bug: #1891145

Depends-on: https://review.opendev.org/#/c/738994/
Change-Id: Ib8b70ee40ec2d19509cc84c0f530612f81907721
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2020-08-11 13:55:01 +02:00
Zuul
5117eeb9fb Merge "keystone: all distros are Python3 - use /usr/bin/python3" 2020-08-10 20:48:44 +00:00
Zuul
8dfab9675c Merge "Add trove-guestagent.conf" 2020-08-10 12:21:30 +00:00
Mark Goddard
146b00efa7 Mount /etc/timezone based on host OS
Previously we mounted /etc/timezone if the kolla_base_distro is debian
or ubuntu. This would fail prechecks if debian or ubuntu images were
deployed on CentOS. While this is not a supported combination, for
correctness we should fix the condition to reference the host OS rather
than the container OS, since that is where the /etc/timezone file is
located.

Change-Id: Ifc252ae793e6974356fcdca810b373f362d24ba5
Closes-Bug: #1882553
2020-08-10 10:14:18 +01:00
likui
3888196334 Add trove-guestagent.conf
Add trove-guestagent.conf templates for trove-guestagent service.
Default the Guest Agent config file to be injected during instance creation.

Change-Id: Id0750b84fef8e19658b27f8ae16a857e1394216e
2020-08-10 16:14:24 +08:00
Mark Goddard
97e26b49cd Fix Barbican client (Castellan) with TLS (part 2)
This patch is a continuation of
I6a174468bd91d214c08477b93c88032a45c137be for the nova-cell role, which
was missed.

The Castellan (Barbican client) has different parameters to control
the used CA file.
This patch uses them.
Moreover, this aligns Barbican with other services by defaulting
its client config to the internal endpoint.

See also [1].

[1] https://bugs.launchpad.net/castellan/+bug/1876102

Closes-Bug: #1886615

Change-Id: I056f3eebcf87bcbaaf89fdd0dc1f46d143db7785
2020-08-07 14:16:04 +01:00
nikparasyr
6033b71d5e Enable glance role to copy extra configuration
Glance role copies glance-image-import.conf
when enabled to allow configuration of
glance interoperable image import. Property
protection can be enabled and file is copied.

Change-Id: I5106675da5228a5d7e630871f0882269603e6571
Closesl-Bug: #1889272
Signed-off-by: nikparasyr <nik.parasyr@protonmail.com>
2020-08-06 18:43:50 +02:00
Marcin Juszkiewicz
d7d4df2684 keystone: all distros are Python3 - use /usr/bin/python3
Change-Id: I59a15186bbe931efd8d99a990a3ceafbd264e1df
2020-08-06 11:30:15 +02:00
Kien Nguyen
5e845ee0cf Add missing mistral services log files
Change-Id: Ib08544a265fe1e0d599a6243cb9d38ed9a7769e1
2020-08-06 10:17:41 +07:00
Radosław Piliszek
c1a6ca0d21 Fix actions for Aodh and Swift
These two roles were missing 'stop' and 'deploy-containers',
respectively.

Change-Id: Iaf434be9baf1973323bb177fad799aea39210fba
2020-08-04 14:32:37 +02:00
Zuul
4e62c86236 Merge "Add timesync prechecks" 2020-08-04 09:12:43 +00:00
Zuul
c58a824e88 Merge "[docker] Added a new flag to disable default iptables rules" 2020-08-04 09:11:28 +00:00
Zuul
0cb9fca9ca Merge "linuxbridge: Fix name of securitygroup section" 2020-08-03 11:04:57 +00:00
likui
908845d32e Update conf for magnum
Deprecated: Option "cafile" from group "keystone_authtoken" is deprecated.
Use option "cafile" from group "keystone_auth".

Change-Id: Ia372b1b73afc0bea6a68dcd156cf963c01e3f3ab
2020-08-03 10:35:13 +08:00
Zuul
202dc899f4 Merge "Fix Masakari role missing deploy-containers" 2020-07-31 13:25:05 +00:00
Zuul
0048e3dd11 Merge "prometheus-openstack-exporter config service filtering" 2020-07-31 10:54:46 +00:00
Zuul
0a6a30b77a Merge "Remove deprecated options in Trove" 2020-07-30 15:29:19 +00:00
Radosław Piliszek
5d3ca8b09e Fix Masakari role missing deploy-containers
Masakari was introduced parallelly to deploy-containers action and
so we missed to add this functionality to it.

Change-Id: Ibef198d20d481bc92b38af786cdf0292b246bb12
Closes-Bug: #1889611
2020-07-30 15:41:37 +02:00
Nick Jones
07f67f1b92 linuxbridge: Fix name of securitygroup section
With an incorrectly named section, whatever's defined in here is
actually ignored which can result in unexpected behaviour.

Closes-Bug: 1889455

Change-Id: Ib2e2b53e9a3c0e62a2e997881c0cd1f92acfb39c
Signed-off-by: Nick Jones <nick@dischord.org>
2020-07-30 09:43:51 +00:00
likui
3660c77637 Remove deprecated options in Trove
Option "network_label_regex" from group "DEFAULT" is
deprecated for removal.

Change-Id: I8aab2ca322159e61e4cbe9a5b30825a71a991e7e
2020-07-29 17:39:00 +08:00
Radosław Piliszek
3018199f0b Add timesync prechecks
If not running containerised chrony, we need to check that host
has its own means of system clock synchronization.

Change-Id: I31b3e9ed625d63a4bf82c674593522268c20ec4c
Partial-Bug: #1885689
2020-07-28 18:35:27 +00:00
Mark Goddard
56a07702bc Performance: use a single config file for fluentd
Currently we generate multiple fluentd configuration files for inputs,
filters, formatters and outputs.
These are then included from the main td-agent.conf configuration file.
With a large number of hosts, this can take a long time to template.

Benchmarking of templating is available at [1].

This change switches to a single fluentd configuration file, with the
include done locally. For the default template files included with Kolla
Ansible we use Jinja includes, but this does not work with templates in
a different directory. We therefore use the Ansible template lookup
plugin, which has a slightly higher overhead than a jinja include, but
far lower than generating multiple templates. This should drastically
improve the performance of this task.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/template.md

Partially-Implements: blueprint performance-improvements

Change-Id: Ia8623be0aa861fea3e54d2c9e1c971dfd8e3afa9
2020-07-28 12:14:48 +01:00
Mark Goddard
f329af7dfa Performance: use a single config file for logrotate
Currently we generate a logrotate configuration file for each enabled
service. These are then included from a logrotate.d directory. With a
large number of hosts, this can take a long time to template.

Benchmarking of templating is available at [1].

This change switches to a single logrotate configuration file for all
services, with the include done locally using jinja. This should
drastically improve the performance of this task.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/template.md

Partially-Implements: blueprint performance-improvements

Change-Id: I39cfa70bef6560f615cad516c43aaef6a523b964
2020-07-28 12:14:47 +01:00
Mark Goddard
9702d4c3c3 Performance: use import_tasks for check-containers.yml
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the check-containers.yml include, the
included file only has a single task, so the overhead of skipping this
task will not be greater than the overhead of the task import. It
therefore makes sense to switch to use import_tasks there.

Partially-Implements: blueprint performance-improvements

Change-Id: I65d911670649960708b9f6a4c110d1a7df1ad8f7
2020-07-28 12:10:59 +01:00
Justinas Balciunas
9fc98be11a prometheus-openstack-exporter config service filtering
This change disables services in the Prometheus openstack-exporter
if they are not enabled in the deployment. Such behaviour allows
to avoid warnings and errors in the log files and keep the
log file contents clean and informative.

Change-Id: I4dcac976620a5f451e3d273183199aefe400994a
2020-07-27 13:30:26 +00:00
Zuul
21f5a02604 Merge "Remove Hyper-V integration" 2020-07-27 12:47:33 +00:00
Zuul
34ace98ff4 Merge "Improve Grafana DB bootstrap" 2020-07-27 11:57:49 +00:00
Zuul
cd9afc5ba3 Merge "Set Kafka default replication factor" 2020-07-27 11:57:45 +00:00