255 Commits

Author SHA1 Message Date
Radosław Piliszek
1bfed045cf Do not set net.ipv4.ip_forward sysctl
To prevent a security issue.
More details in the reno.

Change-Id: I8bb398e299aa68147004723a18d3a1ec459011e5
Closes-Bug: #1945453
2021-09-29 15:19:12 +00:00
Pierre Riteau
72e0673989 Fix neutron upgrade using host limit without controllers
When running kolla-ansible upgrade with a host limit that does not
include controllers, the neutron upgrade fails.

Change-Id: I7125a6ef1f180db6997026ff27e84feb04ee239d
Closes-Bug: #1939691
2021-09-10 22:22:40 +02:00
Radosław Piliszek
9ff2ecb031 Refactor and optimise image pulling
We get a nice optimisation by using a filtered loop instead
of task skipping per service with 'when'.

Partially-Implements: blueprint performance-improvements
Change-Id: I8f68100870ab90cb2d6b68a66a4c97df9ea4ff52
2021-08-10 11:57:54 +00:00
Zuul
860c32de76 Merge "Revert "Performance: Use import_tasks in the main plays"" 2020-12-15 19:52:24 +00:00
Mark Goddard
db4fc85c33 Revert "Performance: Use import_tasks in the main plays"
This reverts commit 9cae59be51e8d2d798830042a5fd448a4aa5e7dc.

Reason for revert: This patch was found to introduce issues with fluentd customisation. The underlying issue is not currently fully understood, but could be a sign of other obscure issues.

Change-Id: Ia4859c23d85699621a3b734d6cedb70225576dfc
Closes-Bug: #1906288
2020-12-14 10:36:55 +00:00
Radosław Piliszek
71e9c603b8 Do not set 'always' tag where unnecessary
Makes 'import_tasks' not change behaviour compared to
'include_tasks'.

Change-Id: I600be7c3bd763b3b924bd4a45b4e7b4dca7a33e3
2020-10-27 19:51:46 +01:00
Radosław Piliszek
9cae59be51 Performance: Use import_tasks in the main plays
Main plays are action-redirect-stubs, ideal for import_tasks.

This avoids 'include' penalty and makes logs/ara look nicer.

Fixes haproxy and rabbitmq not to check the host group as well.

Change-Id: I46136fc40b815e341befff80b54a91ef431eabc0
Partially-Implements: blueprint performance-improvements
2020-10-27 19:09:32 +01:00
Radosław Piliszek
3411b9e420 Performance: optimize genconfig
Config plays do not need to check containers. This avoids skipping
tasks during the genconfig action.

Ironic and Glance rolling upgrades are handled specially.

Swift and Bifrost do not use the handlers at all.

Partially-Implements: blueprint performance-improvements
Change-Id: I140bf71d62e8f0932c96270d1f08940a5ba4542a
2020-10-12 19:30:06 +02:00
James Kirsch
93ad57f47e Add support for encrypting backend Neutron API Server
Add TLS support for backend Neutron API Server communication using
HAProxy to perform TLS termination. When used in conjunction with
enabling TLS for service API endpoints, network communication will be
encrypted end to end, from client through HAProxy to the Neutron
service.

Change-Id: Ib333a1f1bd12491df72a9e52d961161210e2d330
Partially-Implements: blueprint add-ssl-internal-network
2020-10-12 17:27:44 +00:00
Zuul
6c5e9321e4 Merge "Allow to skip and unset sysctl vars" 2020-10-08 10:21:31 +00:00
Zuul
7c61f72108 Merge "Control Neutron migrations" 2020-10-07 16:43:32 +00:00
Zuul
21a96db1be Merge "Add support for changing sysctl.conf path" 2020-10-07 16:33:31 +00:00
Michal Nasiadka
c52a89ae04 Use Docker healthchecks for core services
This change enables the use of Docker healthchecks for core OpenStack
services.
Also check-failures.sh has been updated to treat containers with
unhealthy status as failed.

Implements: blueprint container-health-check
Change-Id: I79c6b11511ce8af70f77e2f6a490b59b477fefbb
2020-10-05 08:35:47 +00:00
Radosław Piliszek
8d2d37064e Control Neutron migrations
Since [1] and [2] merged, K-A has to control Neutron migrations
to migrate all required projects.

This patch additionally fixes the other observed issue.

[1] https://review.opendev.org/750075
[2] https://review.opendev.org/753543

Change-Id: I09e1b421e9066890b50bd82331a3050de252464f
Closes-Bug: #1894380
Depends-On: https://review.opendev.org/755346
2020-10-01 10:13:19 +02:00
Radosław Piliszek
bce266201b Allow to skip and unset sysctl vars
via KOLLA_SKIP and KOLLA_UNSET

Change-Id: I7d9af21c2dd8c303066eb1ee4dff7a72bca24283
Related-Bug: #1837551
2020-09-21 13:13:58 +02:00
Radosław Piliszek
6be51fa67a Add support for changing sysctl.conf path
via kolla_sysctl_conf_path

Change-Id: I09b20fa008a7fecedcb599b4792f24215179b853
2020-09-21 11:47:05 +02:00
Mark Goddard
496904d650 Performance: use import_tasks for register and bootstrap
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the register.yml and bootstrap.yml
includes, all of the tasks in the included file use run_once: True.
The run_once flag improves performance at scale drastically, so
importing these tasks unconditionally will have a lower overhead than a
conditional include task.  It therefore makes sense to switch to use
import_tasks there.

See [1] for benchmarks of run_once.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/run-once.md

Change-Id: Ic67631ca3ea3fb2081a6f8978e85b1522522d40d
Partially-Implements: blueprint performance-improvements
2020-08-28 16:31:04 +00:00
Mark Goddard
b685ac44e0 Performance: replace unconditional include_tasks with import_tasks
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. For unconditionally included tasks, switching to
import_tasks provides a clear benefit.

Benchmarking of include vs. import is available at [1].

This change switches from include_tasks to import_tasks where there is
no condition applied to the include.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md#task-include-and-import

Partially-Implements: blueprint performance-improvements

Change-Id: Ia45af4a198e422773d9f009c7f7b2e32ce9e3b97
2020-08-28 16:12:03 +00:00
Michal Arbet
ed2a2704d1 Fix kolla-ansible not reflect environment changed
Neutron's containers should use ENV from kolla_docker module's
environment parameter (defined in roles/neutron/defaults/main.yml)
after reconfigure, not only when deploying.

Currently this is working only for deploy, not for reconfigure.

How to test it ?
- Deploy neutron with "neutron_legacy_iptables" set to yes/no.
- Change value of "neutron_legacy_iptables" to opposite value as before.
- Reconfigure neutron.

Current result :
 - "KOLLA_LEGACY_IPTABLES" in container's ENV is not changed

Expected result :
 - "KOLLA_LEGACY_IPTABLES:" in container's ENV should be changed

This patch is fixing this behaviour by adding missing
environment parameter to neutron's "Check neutron containers" task.

Change-Id: Ibfbe2d4f49261fa766acbb6ff45da9994118bda8
Closes-Bug: #1853776
2020-08-21 14:51:41 +00:00
Bharat Kunwar
4809462f4e Deploy neutron-mlnx-agent and neutron-eswitchd containers
Change-Id: I173669bdf92b1f2ea98907ba16808ca3c914944c
2020-08-13 23:33:57 +01:00
Mark Goddard
9702d4c3c3 Performance: use import_tasks for check-containers.yml
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the check-containers.yml include, the
included file only has a single task, so the overhead of skipping this
task will not be greater than the overhead of the task import. It
therefore makes sense to switch to use import_tasks there.

Partially-Implements: blueprint performance-improvements

Change-Id: I65d911670649960708b9f6a4c110d1a7df1ad8f7
2020-07-28 12:10:59 +01:00
gugug
c7d92ed668 Remove the ml2_conf.ini merging for agents
planned removal

Change-Id: Ib37ea4d42f82096a682cebc724c45c9dd39c8b47
2020-07-08 15:31:49 +00:00
gugug
1c7ca29664 Remove the neutron-fwaas roles since it retired
Depends-On: I561504160e5548c54d1af31821c3366ab34cf0ec
Change-Id: I15e5c0e0a956ee181873cf002229532a15ff959d
Co-Authored-By: jacky06 <zhang.min@99cloud.net>
2020-06-20 12:12:07 +00:00
Zuul
f73f3e6d04 Merge "Add missing become to some VMWare tasks" 2020-06-10 18:30:03 +00:00
Zuul
e74cada7c1 Merge "permission denied when enable_kolla_dev_mod" 2020-06-10 02:32:45 +00:00
Christian Berendt
60e03d7bf3 Remove XenAPI integration
Change-Id: Iea3f4f3d2e5c6040c1e0bc7bfae8719cc7d8ac55
2020-06-09 13:56:17 +02:00
wu.chunyang
3e9a648601 permission denied when enable_kolla_dev_mod
non-root user has no permission to create directory under /opt
directory. use "become: true" to resolve it.

Change-Id: I155efc4b1e0691da0aaf6ef19ca709e9dc2d9168
2020-06-07 19:36:42 +08:00
x191859
640bbc38ab Add missing become to some VMWare tasks
Fixed on ``Copying VMware vCenter CA file`` and ``Copying over nsx.ini``.

Change-Id: If909f59e7e4b241594c6b2567784ecad23e74226
Closes-Bug: #1882252
2020-06-05 16:53:27 +00:00
Michal Nasiadka
2128075c6e Ansible lint related fixes
Change-Id: I146ea3d84efb83ec5d7405644ad372e57ecafc1e
2020-05-12 17:39:07 +00:00
Zuul
87984f5425 Merge "Add Ansible group check to prechecks" 2020-04-16 15:33:46 +00:00
Zuul
7f42813159 Merge "Refactor copy certificates task" 2020-04-16 14:03:37 +00:00
Zuul
b6801477db Merge "Support Neutron policy file in both .json and .yaml format" 2020-04-16 13:32:01 +00:00
Zuul
975db2b2b4 Merge "OVN Support" 2020-04-15 20:51:57 +00:00
Scott Solkhon
92deddca88 Support Neutron policy file in both .json and .yaml format
Change-Id: I22a995195a1d12bb759cba9777527c23475124f2
2020-04-15 14:30:45 +01:00
James Kirsch
4d155d69cd Refactor copy certificates task
Refactor service configuration to use the copy certificates task. This
reduces code duplication and simplifies implementing encrypting backend
HAProxy traffic for individual services.

Change-Id: I0474324b60a5f792ef5210ab336639edf7a8cd9e
2020-04-14 17:26:19 +00:00
Michal Nasiadka
8a0740df97 OVN Support
Implement OVN Ansible role.

Implements: blueprint ovn-controller-neutron-ansible

Depends-On: https://review.opendev.org/713422
Change-Id: Icd425dea85d58db49c838839d8f0b864b4a89a78
2020-04-09 07:40:12 +02:00
Mark Goddard
1d70f509e3 Perform host configuration during upgrade
This is a follow up to I001defc75d1f1e6caa9b1e11246abc6ce17c775b. To
maintain previous behaviour, and ensure we catch any host configuration
changes, we should perform host configuration during upgrade.

Change-Id: I79fcbf1efb02b7187406d3c3fccea6f200bcea69
Related-Bug: #1860161
2020-04-08 17:03:22 +01:00
Mark Goddard
fdea19a305 Separate per-service host configuration tasks
Currently there are a few services that perform host configuration
tasks. This is done in config.yml. This means that these changes are
performed during 'kolla-ansible genconfig', when we might expect not to
be making any changes to the remote system.

This change separates out these host configuration tasks into a
config-host.yml file, which is included directly from deploy.yml.

One change in behaviour is that this prevents these tasks from running
during an upgrade or genconfig. This is probably what we want, but we
should be careful when any of these host configuration tasks are
changed, to ensure they are applied during an upgrade if necessary.

Change-Id: I001defc75d1f1e6caa9b1e11246abc6ce17c775b
Closes-Bug: #1860161
2020-04-02 13:51:56 +00:00
Zuul
2a2ce059dc Merge "Add notify restart container when cert changed" 2020-03-10 12:12:55 +00:00
yj.bai
d3cc2f670e Add notify restart container when cert changed
When change the cert file in /etc/kolla/certificate/.
The certificate in the container has not changed.
So I think can use kolla-ansible deploy when certificate is
changed. restart <container>

Partially-Implements: blueprint custom-cacerts

Change-Id: Iaac6f37e85ffdc0352e8062ae5049cc9a6b3db26
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2020-03-10 16:23:09 +08:00
Radosław Piliszek
266fd61ad7 Use "name:" instead of "role:" for *_role modules
Both include_role and import_role expect role's name to be given
via "name" param instead of "role".
This worked but caused errors with ansible-lint.
See: https://review.opendev.org/694779

Change-Id: I388d4ae27111e430d38df1abcb6c6127d90a06e0
2020-03-02 10:01:17 +01:00
Mark Goddard
49fb55f182 Add Ansible group check to prechecks
We assume that all groups are present in the inventory, and quite obtuse
errors can result if any are not.

This change adds a precheck that checks for the presence of all expected
groups in the inventory for each service. It also introduces a common
service-precheck role that we can use for other common prechecks.

Change-Id: Ia0af1e7df4fff7f07cd6530e5b017db8fba530b3
Partially-Implements: blueprint improve-prechecks
2020-02-28 16:23:14 +00:00
James Kirsch
511ba9f6a2 Copy CA into containers.
When kolla_copy_ca_into_containers is set to "yes", the Certificate
Authority in /etc/kolla/certificates will be copied into service
containers to enable trust for that CA. This is especially useful when
the CA is self signed, and would not be trusted by default.

Partially-Implements: blueprint custom-cacerts

Change-Id: I4368f8994147580460ebe7533850cf63a419d0b4
2020-01-28 14:03:32 -08:00
Zuul
6c297e54c4 Merge "[neutron] Adjust neighbour table thresholds" 2019-12-16 11:19:26 +00:00
Jan Vondra
59b058e23d Neutron: Added loading ip6_tables kernel module
ip6_tables is needed for HybridFwDriver

Change-Id: I66c50b74a89c046dc59e59dd2422a80e0642ab72
2019-12-09 16:26:13 +00:00
Zuul
734e86a02f Merge "Split out OVS/Linuxbridge agent configs from ml2_conf.ini" 2019-11-27 18:14:43 +00:00
Pierre Riteau
4234cc4b5b [neutron] Adjust neighbour table thresholds
When clouds have a large number of hosts, the default size of the ARP
cache is too small. The cache can overflow, which means that the system
has no way to reach some IP addresses.

Increasing threshold limits addresses the situation, in a reasonably
safe way (the maximum impact is 5MB or so of additional RAM used).

More context on this issue:

* http://man7.org/linux/man-pages/man7/arp.7.html
* https://bugs.launchpad.net/charm-nova-compute/+bug/1780348
* https://bugs.launchpad.net/fuel/+bug/1488938
* https://bugs.launchpad.net/tripleo/+bug/1690087
* 0583a0c949
* 3dd21ea62a
* 1651a1805a

Change-Id: I60c871e8eb9f2c086818ff077987f2390930800c
Closes-Bug: #1844349
2019-11-27 16:32:47 +01:00
Michal Nasiadka
0947974ff6 Split out OVS/Linuxbridge agent configs from ml2_conf.ini
Change-Id: I799993728112a525e34cfbc4e786a10f0ed03be9
2019-11-27 13:42:48 +00:00
Michal Nasiadka
1009931162 Change local_action to delegate_to: localhost
As part of the effort to implement Ansible code linting in CI
(using ansible-lint) - we need to implement recommendations from
ansible-lint output [1].

One of them is to stop using local_action in favor of delegate_to -
to increase readability and and match the style of typical ansible
tasks.

[1]: https://review.opendev.org/694779/

Partially implements: blueprint ansible-lint

Change-Id: I46c259ddad5a6aaf9c7301e6c44cd8a1d5c457d3
2019-11-22 15:04:44 +00:00
Mark Goddard
0bd60388a8 Remove transitional support for neutron-lbaas removal
Support for deploying neutron-lbaas was removed in the Train release. We
no longer need the task to remove the container in the upgrade process.

Change-Id: Ie336f68c710616de29f34dd4011e137ec056973b
2019-11-14 11:48:56 +00:00