5589 Commits

Author SHA1 Message Date
Mark Goddard
db4fc85c33 Revert "Performance: Use import_tasks in the main plays"
This reverts commit 9cae59be51e8d2d798830042a5fd448a4aa5e7dc.

Reason for revert: This patch was found to introduce issues with fluentd customisation. The underlying issue is not currently fully understood, but could be a sign of other obscure issues.

Change-Id: Ia4859c23d85699621a3b734d6cedb70225576dfc
Closes-Bug: #1906288
2020-12-14 10:36:55 +00:00
Radosław Piliszek
9cae59be51 Performance: Use import_tasks in the main plays
Main plays are action-redirect-stubs, ideal for import_tasks.

This avoids 'include' penalty and makes logs/ara look nicer.

Fixes haproxy and rabbitmq not to check the host group as well.

Change-Id: I46136fc40b815e341befff80b54a91ef431eabc0
Partially-Implements: blueprint performance-improvements
2020-10-27 19:09:32 +01:00
Zuul
b77b4eb829 Merge "Fix keystone-startup.sh - remove Fernet key age check" 2020-10-23 21:03:59 +00:00
Mark Goddard
ba8c27f554 Fix keystone-startup.sh - remove Fernet key age check
Currently we check the age of the primary Fernet key on Keystone
startup, and fail if it is older than the rotation interval. While this
may seem sensible, there are various reasons why the key may be older
than this:

* if the rotation interval is not a factor of the number of seconds in a
  week, the rotation schedule will be lumpy, with the last rotation
  being up to twice the nominal rotation interval
* if a keystone host is unavailable at its scheduled rotation time,
  rotation will not happen. This may happen multiple times

We could do several things to avoid this issue:

1. remove the check on the age of the key
2. multiply the rotation interval by some factor to determine the
   allowed key age

This change goes for the more simple option 1. It also cleans up some
terminology in the keystone-startup.sh script.

Closes-Bug: #1895723

Change-Id: I2c35f59ae9449cb1646e402e0a9f28ad61f918a8
2020-10-22 09:20:02 +01:00
Zuul
8bc37f5282 Merge "Fix fernet cron path on Ubuntu/Debian" 2020-10-22 08:13:00 +00:00
Michal Nasiadka
07d42bff01 Fix fernet cron path on Ubuntu/Debian
The correct path according to Ubuntu cron manpage [1] is
/var/spool/cron/crontabs/$USER

[1]: http://manpages.ubuntu.com/manpages/trusty/man8/cron.8.html

Closes-Bug: #1898765
Change-Id: Id5fc354e3e32cae2468cd2557a2967859e3b4e16
2020-10-19 16:18:59 +00:00
Zuul
db4e12e9d3 Merge "baremetal: Install iptables for Docker if enabled" 2020-10-14 09:34:59 +00:00
Radosław Piliszek
3411b9e420 Performance: optimize genconfig
Config plays do not need to check containers. This avoids skipping
tasks during the genconfig action.

Ironic and Glance rolling upgrades are handled specially.

Swift and Bifrost do not use the handlers at all.

Partially-Implements: blueprint performance-improvements
Change-Id: I140bf71d62e8f0932c96270d1f08940a5ba4542a
2020-10-12 19:30:06 +02:00
James Kirsch
93ad57f47e Add support for encrypting backend Neutron API Server
Add TLS support for backend Neutron API Server communication using
HAProxy to perform TLS termination. When used in conjunction with
enabling TLS for service API endpoints, network communication will be
encrypted end to end, from client through HAProxy to the Neutron
service.

Change-Id: Ib333a1f1bd12491df72a9e52d961161210e2d330
Partially-Implements: blueprint add-ssl-internal-network
2020-10-12 17:27:44 +00:00
Zuul
42d0b89361 Merge "Apply bool filter to all enable_prometheus_* variables" 2020-10-12 14:02:24 +00:00
Pierre Riteau
6985e9a67c Apply bool filter to all enable_prometheus_* variables
Change-Id: I639145a709f1d3b9882bbdfb20a754646d1f5270
2020-10-09 18:51:38 +02:00
wu.chunyang
4fe4820b1f remove redundant line and replace octavia user
remove redundant space line
replace octavia user with {{ octavia_keystone_user }}

Change-Id: I284acc580a1a530eede3e0227febe8667dea5d47
2020-10-09 17:26:37 +08:00
Zuul
0617ca7a22 Merge "add octavia openrc file" 2020-10-09 09:10:18 +00:00
Zuul
b9d77bf954 Merge "octavia: generate certificates automatically" 2020-10-09 08:41:08 +00:00
Mark Goddard
bc1d02d896 baremetal: Install iptables for Docker if enabled
If iptables is not installed, e.g. in the CentOS 8 cloud image, and
Docker iptables management is enabled, we get the following errors:

Failed to find iptables: exec: \"iptables\": executable file not found
in $PATH failed to start daemon: Error initializing network controller:
error obtaining controller instance: failed to create NAT chain DOCKER:
Iptables not found

This change installs the iptables package Docker iptables management is
enabled.

Change-Id: I3ba5318debccafb28c3cbce8e4e9813c28b086fc
Closes-Bug: #1899060
2020-10-08 17:48:57 +01:00
Radosław Piliszek
3f9d30a28f Fix haproxy bundle generation
This fixes the `certificates` command to not include CSRs in
the haproxy bundle.
The regex was wrong.

Change-Id: If25a6d5dd40f507fea4470be01baeeb7c8a790b4
2020-10-08 16:57:16 +02:00
wu.chunyang
195269d758 add octavia openrc file
we use octavia user to upload image currently, so it is better to
create a octavia openrc file for user

Implements: blueprint implement-automatic-deploy-of-octavia

Change-Id: Ib53d00fa4a6ee59b8a0b2245f83786a6af0cbf53
2020-10-08 14:50:52 +00:00
Mark Goddard
894f4912ac octavia: generate certificates automatically
implemented as a separate command (kolla-ansible octavia-certificates)

Implements: blueprint implement-automatic-deploy-of-octavia

Co-Authored-By: wu.chunyang <wuchunyang@yovole.com>
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Change-Id: I2c5b26ce9e363f35c523865904a582f7960aa682
2020-10-08 16:50:30 +02:00
Zuul
6c5e9321e4 Merge "Allow to skip and unset sysctl vars" 2020-10-08 10:21:31 +00:00
Zuul
8604dee6b0 Merge "Add support for ACME http-01 challenge" 2020-10-07 23:31:58 +00:00
Zuul
950b038a7a Merge "Fix common role when using external mariadb" 2020-10-07 20:54:07 +00:00
Zuul
f8822a2f25 Merge "Update ironic iPXE configuration" 2020-10-07 20:31:24 +00:00
Zuul
dd0d903d88 Merge "Coordinate haproxy and keepalived restarts" 2020-10-07 19:26:31 +00:00
Zuul
7c1acb96a0 Merge "Add support for with_frontend and with_backend" 2020-10-07 16:48:12 +00:00
Zuul
7c61f72108 Merge "Control Neutron migrations" 2020-10-07 16:43:32 +00:00
Zuul
21a96db1be Merge "Add support for changing sysctl.conf path" 2020-10-07 16:33:31 +00:00
nikparasyr
fc4327d946 Allow overwrite of placement-api wsgi config
Use with_first_found on placement-api-wsgi to allow
overwrite from users and keep consistency with other
roles.

Change-Id: I11c84db6df1bb5be61db5b6b0adf8c160a2bd931
Closes-Bug: #1898766
2020-10-06 20:43:52 +02:00
Zuul
e65c9b4dc9 Merge "[baremetal]: Use $releasever in docker-ce repo" 2020-10-06 13:32:54 +00:00
Mark Goddard
dd8264176c Update ironic iPXE configuration
* ipxe_enabled was removed in Ussuri, now there is a separate ipxe boot
  interface.
* iPXE now has its own set of configuration for the bootfile and config
  template, and the values previously set when iPXE is enabled are now
  the default in ironic. The overrides have been removed, since they
  match the iPXE defaults.

Change-Id: I9d9f030ee4be979d0a849b59e5eb991f2d82f6a4
2020-10-06 13:20:03 +01:00
Zuul
5e638b757b Merge "Use Docker healthchecks for core services" 2020-10-06 08:26:21 +00:00
Michal Nasiadka
a220c81fb4 horizon: stop using deprecated django.py
[1]: https://review.opendev.org/#/c/561802/

Change-Id: Id335502ad464aa417162b2576ffae3818d30cba1
2020-10-05 12:46:49 +02:00
Michal Nasiadka
c52a89ae04 Use Docker healthchecks for core services
This change enables the use of Docker healthchecks for core OpenStack
services.
Also check-failures.sh has been updated to treat containers with
unhealthy status as failed.

Implements: blueprint container-health-check
Change-Id: I79c6b11511ce8af70f77e2f6a490b59b477fefbb
2020-10-05 08:35:47 +00:00
Radosław Piliszek
c2d0bf30ea Coordinate haproxy and keepalived restarts
Keepalived and haproxy cooperate to provide control plane HA in
kolla-ansible deployments.
Certain care should be exerted to avoid prolonged availability
loss during reconfigurations and upgrades.
This patch aims to provide this care.
There is nothing special about keepalived upgrade compared to
reconfig, hence it is simplified to run the same code as for
deploy.
The broken logic of safe upgrade is replaced by common handler
code which's goal is to ensure we down current master only after
we have backups ready.

This change introduces a switch to kolla_docker module that allows
to ignore missing containers (as they are logically stopped).
ignore_missing is the switch's name.
All tests are included.

Change-Id: I22ddec5f7ee4a7d3d502649a158a7e005fe29c48
2020-10-04 16:58:24 +02:00
Zuul
4c4ad2b87b Merge "Implement automatic deploy of octavia" 2020-10-02 15:04:46 +00:00
wu.chunyang
4a58f4238c Implement automatic deploy of octavia
this patchset has implemented:
  - network (lb-mgmt-net)
  - security groups and rules (used by amphora and health manager)
  - amphora flavor (used by amphora)
  - nova keypair (used by amphora at the time of debugging)

Add a octavia_amp_listen_port variable which used by amphora
Add amp_image_owner_id in octavia.conf

Implements: blueprint implement-automatic-deploy-of-octavia
Co-Authored-By: zhangchun <zhangchun@yovole.com>

Depends-On: https://review.opendev.org/652030

Change-Id: I67009d046925cfc02c1e0073c80085c1471975f6
2020-10-02 14:05:00 +02:00
Zuul
586357ca74 Merge "Change the default haproxy template to split variant" 2020-10-01 12:49:02 +00:00
Radosław Piliszek
8d2d37064e Control Neutron migrations
Since [1] and [2] merged, K-A has to control Neutron migrations
to migrate all required projects.

This patch additionally fixes the other observed issue.

[1] https://review.opendev.org/750075
[2] https://review.opendev.org/753543

Change-Id: I09e1b421e9066890b50bd82331a3050de252464f
Closes-Bug: #1894380
Depends-On: https://review.opendev.org/755346
2020-10-01 10:13:19 +02:00
Zuul
4441038e29 Merge "Make keep-alive timeout configurable" 2020-09-30 17:01:56 +00:00
Zuul
ba933f16e9 Merge "Support TLS encryption of RabbitMQ client-server traffic" 2020-09-29 11:31:03 +00:00
Michal Nasiadka
883b79a1a5 [baremetal]: Use $releasever in docker-ce repo
Update to CentOS 8 versions of packages in docker-ce repo (that are
now available)

Change-Id: I50d28ea31c3c29322974b91a72a2bd7999324ac7
2020-09-28 17:27:23 +00:00
Zuul
0dd44b7675 Merge "Reduce the use of SQLAlchemy connection pooling" 2020-09-28 17:14:55 +00:00
Radosław Piliszek
2fd72a39e9 Add support for ACME http-01 challenge
All docs are included.

Change-Id: Ie29ff7ca340812c8dc0dac493518c87cf7bf137b
Partially-Implements: blueprint letsencrypt-https
2020-09-26 20:29:20 +02:00
Zuul
29b2d4284a Merge "Fix keystone-startup.sh" 2020-09-25 13:44:13 +00:00
Zuul
07cbec194f Merge "Add support for encrypting Ironic API" 2020-09-25 11:47:49 +00:00
Michal Nasiadka
d78673e77f Fix keystone-startup.sh
keystone-startup.sh is using fernet_token_expiry instead of
fernet_key_rotation_interval - which effects in restart loop of keystone
containers - when restarted after 2-3 days.

Closes-Bug: #1895723

Change-Id: Ifff77af3d25d9dc659fff34f2ae3c6f2670df0f4
2020-09-25 10:19:44 +00:00
James Kirsch
7c2df87ded Add support for encrypting Ironic API
This patch introduces an optional backend encryption for the Ironic API
service. When used in conjunction with enabling TLS for service API
endpoints, network communcation will be encrypted end to end, from
client through HAProxy to the Ironic service.

Change-Id: I9edf7545c174ca8839ceaef877bb09f49ef2b451
Partially-Implements: blueprint add-ssl-internal-network
2020-09-24 10:09:13 -07:00
Zuul
43a0a1ca3d Merge "Allow setting container_proxy per service" 2020-09-24 10:05:50 +00:00
Zuul
01a47b927d Merge "Bump minimum Ansible version to 2.9" 2020-09-24 09:40:15 +00:00
Pierre Riteau
c5c6d995d3 Bump minimum Ansible version to 2.9
Change-Id: I5befc72a4894d625ca352b27df9d3aa84a2f5b2c
2020-09-23 17:48:01 +02:00
Mark Goddard
6882013309 Fix common role when using external mariadb
If the common role is executed against a set of hosts that are not all
in the fluentd group, the run_once tasks that find customisations may be
skipped. This causes a later failure when accessing the registered
variables for those tasks.

This issue was raised on the mailing list:
http://lists.openstack.org/pipermail/openstack-discuss/2020-September/016932.html

This issue only affects the master branch, due to addition of groups
for the common role in I6a4676bf6efeebc61383ec7a406db07c7a868b2a.

This change fixes the issue by always running the find tasks, if fluentd
is enabled.

Change-Id: I559c4b94d18c7f36d43e1d88629ed44668abf859
2020-09-22 17:32:37 +01:00