13713 Commits

Author SHA1 Message Date
Zuul
f86ad4d389 Merge "quorum: add missing octavia and masakari" 2023-12-19 16:12:23 +00:00
Zuul
36f0bb2235 Merge "CI: Drop masakari jobs from project.yaml" 2023-12-19 13:00:04 +00:00
Michal Nasiadka
f3c3ea61eb CI: Drop masakari jobs from project.yaml
Those have been failing for long time and Masakari is deprecated
in Kolla (not mentioning the state of the project itself).

Change-Id: I06d32f8f4921b2025fd176be1198dc0cbcfa5f3b
2023-12-19 12:11:54 +00:00
Michal Nasiadka
be5dc32c86 quorum: add missing octavia and masakari
Change-Id: Ibf9a9a0c18938f638c8e8b00b6017c64f1523b23
2023-12-19 12:13:53 +01:00
Michal Nasiadka
71df8e6208 CI: treat ansible-lint warnings as errors
Taken from [1].

[1]: https://ansible.readthedocs.io/projects/lint/configuring/#ansible-lint-configuration

Change-Id: I29c1abb4d518a4da6788e06804f28c5f6853c581
2023-12-19 11:05:49 +00:00
Sven Kieske
176aa5a42c
CI: fix two ansible-lint warnings
Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: I81a9b2dab7e9a4e2c8facaa0f32538f2884e3ca9
2023-12-18 19:15:38 +01:00
Michal Nasiadka
d05a1560bc CI: Drop duplicated variables in base.yaml
Change-Id: I168f3bb81731e56ae338c4438ab4d024fa69d827
2023-12-15 13:17:19 +00:00
Zuul
7fc76fb4e9 Merge "CI: Move openstack clients setup to a role" 2023-12-14 21:01:51 +00:00
Michal Nasiadka
56cbf8c031 CI: Move openstack clients setup to a role
Change-Id: Id5c53b63e88502c999b89cbc62405bb8953fef3a
2023-12-14 16:38:26 +01:00
Zuul
17a76d2a0e Merge "Add precheck for RabbitMQ quorum queues" 2023-12-14 14:54:40 +00:00
Zuul
82c5c027f0 Merge "Fix Docker health check for sahara_engine" 2023-12-14 14:28:42 +00:00
Zuul
82c1c11a4d Merge "RabbitMQ: correct docs on Quorum Queue migrations" 2023-12-14 12:32:29 +00:00
Pierre Riteau
693c5c8b23 Fix Docker health check for sahara_engine
The wrong process name was being used.

Closes-Bug: #2046268
Change-Id: I5a5d4f227205e811732331ee6e020ccea67b6fab
2023-12-14 09:53:03 +00:00
Zuul
407991c825 Merge "CI: Pin docker to <7 in setup_gate.sh" 2023-12-14 09:50:47 +00:00
Zuul
c0cddb0967 Merge "Configures the tap-as-a-service neutron plugin" 2023-12-13 16:11:36 +00:00
Matt Crees
61f84e3beb Add precheck for RabbitMQ quorum queues
Adds a precheck to fail if non-quorum queues are found in RabbitMQ.

Currently excludes fanout and reply queues, pending support in
oslo.messaging [1].

[1]: https://review.opendev.org/c/openstack/oslo.messaging/+/888479

Closes-Bug: #2045887
Change-Id: Ibafdcd58618d97251a3405ef9332022d4d930e2b
2023-12-13 14:49:05 +00:00
Michal Nasiadka
70ebf95c2b CI: Pin docker to <7 in setup_gate.sh
Change-Id: I46c05a54171cbf43a51594998561db94af7d17e6
2023-12-13 14:05:28 +00:00
Zuul
7a29abb590 Merge "CI: Move test-ovn before test-core-openstack" 2023-12-07 16:30:17 +00:00
Zuul
8046b80324 Merge "CI: Add SCENARIO env var to upgrade.sh" 2023-12-07 14:22:54 +00:00
Matt Crees
1dfdf0904e RabbitMQ: correct docs on Quorum Queue migrations
RabbitMQ does not need to be reconfigured when migrating from non-HA
queues to quorum queues.

Change-Id: I69d409732ff8eb7ae736e95f8a5d290efd33fa32
2023-12-06 16:10:52 +00:00
Zuul
ca8260eb09 Merge "Fix broken list concatenation in horizon role" 2023-12-06 15:58:46 +00:00
Michal Nasiadka
4a3d6b6e51 CI: Add SCENARIO env var to upgrade.sh
Change-Id: I373f6d13809674c521155ca51962785a8b1ac598
2023-12-06 14:59:23 +01:00
Andrey Kurilin
97cd173177 Fix broken list concatenation in horizon role
Starting with ansible-core 2.13, list concatenation format is changed
and does not support concatenation operations outside of the jinja template.

The format change:

  "[1] + {{ [2] }}" -> "{{ [1] + [2] }}"

This affects the horizon role that iterates over existing policy files to
override and concatenate them into a single variable.

Co-Authored-By: Dr. Jens Harbott <harbott@osism.tech>

Closes-Bug: #2045660
Change-Id: I91a2101ff26cb8568f4615b4cdca52dcf09e6978
2023-12-05 16:59:20 +01:00
Mark Goddard
af6e1ca4fd Support Ansible max_fail_percentage
This allows us to continue execution until a certain proportion of hosts
to fail. This can be useful at scale, where failures are common, and
restarting a deployment is time-consuming.

The default max failure percentage is 100, keeping the default
behaviour. A global max failure percentage may be set via
kolla_max_fail_percentage, and individual services may define a max
failure percentage via <service>_max_fail_percentage.

Note that all hosts in the inventory must be reachable for fact
gathering, even those not included in a --limit.

Closes-Bug: #1833737
Change-Id: I808474a75c0f0e8b539dc0421374b06cea44be4f
2023-12-05 11:49:42 +01:00
7112d5f117 Update master for stable/2023.2
Add file to the reno documentation build to show release notes for
stable/2023.2.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.2.

Sem-Ver: feature
Change-Id: I96140b1e3be06b60909f3eed739f1a944e119c7e
2023-12-01 12:14:57 +00:00
Christian Berendt
c939504da6 magnum: support kubeconfig configuration file
If a file {{ node_custom_config }}/magnum/kubeconfig exists, it is
copied to /var/lib/magnum/.kube/config in all Magnum Service Containers.
At this location, the vexxhost/magnum-cluster-api will loo for the Kubeconfig
configuration file to control the Cluster API Control Plane. If the
vexxhost/magnum-cluster-api is installed in the Magnum container images,
control of a cluster API control plane can then take place via the Magnum API.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/902101
Change-Id: I986c5192fe96b9c480a2d8fa87d719a50ce78186
17.0.0.0rc1
2023-12-01 07:15:25 +00:00
Michal Nasiadka
bdd2aa37b2 fluentd: Fix getting podman labels
podman_image_info returns Config dict, not ContainerConfig.

Change-Id: I9f813c90b42246c4835d7d7b18476a021d80548b
2023-12-01 07:52:14 +01:00
Zuul
21236c2e90 Merge "CI: Install rich on depends-on podman builds" 2023-11-30 18:58:51 +00:00
Zuul
016cb7d427 Merge "enable quorum queues" 2023-11-30 17:46:57 +00:00
Zuul
9bbfd2f954 Merge "fluentd: Use labels for transition to v5" 2023-11-30 16:51:22 +00:00
Michal Nasiadka
6fb1220b65 CI: Install rich on depends-on podman builds
Change-Id: I54f94c383ae5a1185b364495422e1ab79cbd1afb
2023-11-30 15:29:43 +01:00
Sven Kieske
64575519aa enable quorum queues
This implements a global toggle `om_enable_rabbitmq_quorum_queues`
to enable quorum queues for each service in RabbitMQ, similar to
what was done for HA[0].

Quorum Queues are enabled by default.

Quorum queues are more reliable, safer, simpler and faster than
replicated mirrored classic queues[1].

Mirrored classic queues are deprecated and scheduled for removal
in RabbitMQ 4.0[2].

Notice, that we do not need a new policy in the RabbitMQ definitions
template, because their usage is enabled on the client side and can't
be set using a policy[3].

Notice also, that quorum queues are not yet enabled in oslo.messaging
for the usage of reply_ and fanout_ queues (transient queues).
This will change once[4] is merged.

[0]: https://review.opendev.org/c/openstack/kolla-ansible/+/867771
[1]: https://www.rabbitmq.com/quorum-queues.html
[2]: https://blog.rabbitmq.com/posts/2021/08/4.0-deprecation-announcements/
[3]: https://www.rabbitmq.com/quorum-queues.html#declaring
[4]: https://review.opendev.org/c/openstack/oslo.messaging/+/888479

Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: I6c033d460a5c9b93c346e9e47e93b159d3c27830
2023-11-30 13:53:00 +00:00
Zuul
9b6a70e441 Merge "etcd: update to v3.4" 2023-11-30 01:10:27 +00:00
Zuul
1551c4d2de Merge "CI: Fail on fluentd log parsing errors" 2023-11-30 01:10:25 +00:00
Jan Gutter
ccfa2a6c16 etcd: update to v3.4
* Updates etcd to v3.4
* Updated the config to use v3.4's logging mechanism
* Deprecated etcd CA parameters aren't used, so we are not affected
  by their removal.
* Note that we are not currently guarding against skip-version updates for
  etcd.

Notable non-voting jobs exercising some of this:
* kolla-ansible-ubuntu-upgrade-cephadm (cinder->tooz->etcd3gw->etcd)
* kolla-ansible-ubuntu-zun (see
  https://review.opendev.org/c/openstack/openstack-ansible/+/883194 )

Depends-On: https://review.opendev.org/c/openstack/kolla/+/890464
Change-Id: I086e7bbc7db64421445731a533265e7056fbdb43
2023-11-29 17:43:33 +00:00
Jan Gutter
ae21f31785 etcd: deduplicate environments for containers
* etcd service containers usually have a set of
  environment parameters required to boot the container.
* The short-lived etcd bootstrap containers pass extra
  ETCD_INITIAL_* environment variables, but still need to
  pass the ones that the service containers use.
* This uses ansible's `combine` filter to cut down on the
  duplication.
* This is intended to be just a straightforward refactor.

Change-Id: I04e95f92a8f365553afd618d58b99de595d48312
2023-11-29 17:43:22 +00:00
Michal Nasiadka
ba54f8cdda CI: Fail on fluentd log parsing errors
Change-Id: Ie3963f5ed20f7fb61ef2e03f0cf12a4ea1c87c9c
2023-11-29 17:42:44 +00:00
Dr. Jens Harbott
0b1a59dd8c podman: install "rich" dependency
This dependency was added to podman-py in version 4.8.0, but not added
properly to their requirements. Install it explicitly for our tox
and integration testing as a workaround.

[0] https://github.com/containers/podman-py/issues/350

Change-Id: I61a5fdfc4e505f2577185f0c0f1297cf2709be2c
2023-11-29 17:04:06 +00:00
Zuul
0b7d9c368e Merge "Update etcd docs and reno" 2023-11-29 14:16:03 +00:00
Zuul
29e1827bd1 Merge "CI: Add oslo_db.exception.DBConnectionError to check-logs.sh" 2023-11-29 14:15:59 +00:00
Dr. Jens Harbott
8e7a67bfd7
Update etcd docs and reno
Just some minor formatting and wording updates

Change-Id: Ia42944512f8d14a1993bd4ae3d09f0f2ab431322
2023-11-29 12:07:53 +00:00
Zuul
e971d0c795 Merge "etcd: Add support for more scenarios" 2023-11-29 11:13:18 +00:00
Michal Nasiadka
6e9e66b892 CI: Add oslo_db.exception.DBConnectionError to check-logs.sh
Change-Id: Ia1de6d9452e2c900169e9b4ccb7dfc1280283909
2023-11-29 10:04:44 +01:00
Zuul
65c2196975 Merge "Fix Horizon WSGI application log parsing" 2023-11-28 18:53:58 +00:00
Jan Gutter
ed3b27cc92 etcd: Add support for more scenarios
This commit addresses a few shortcomings in the etcd service:
  * Adding or removing etcd nodes required manual intervention.

  * The etcd service would have brief outages during upgrades or
    reconfigures because restarts weren't always serialised.

This makes the etcd service follow a similar pattern to mariadb:
  * There is now a distiction between bootstrapping the cluster
    and adding / removing another member.

  * This more closely follows etcd's upstream bootstrapping
    guidelines.

  * The etcd role now serialises restarts internally so the
    kolla_serial pattern is no longer appropriate (or necessary).

This does not remove the need for manual intervention in all
failure modes: the documentation has been updated to address the
most common issues.

Note that there's repetition in the container specifications: this
is somewhat deliberate. In a future cleanup, it's intended to reduce
the duplication.

Change-Id: I39829ba0c5894f8e549f9b83b416e6db4fafd96f
2023-11-28 18:43:56 +01:00
Zuul
9a0ac440df Merge "Revert "Enable RabbitMQ HA queues by default"" 2023-11-28 16:45:06 +00:00
Michal Nasiadka
06baa8f63f fluentd: Use labels for transition to v5
Depends-On: https://review.opendev.org/c/openstack/kolla/+/901508
Change-Id: I8c7d3de95d0f1f8e57a993b8c3417d90459e19be
2023-11-28 15:13:00 +00:00
Doug Szumski
4168b46cff Fix Horizon WSGI application log parsing
Like other WSGI services in Kolla Ansible, the Horizon WSGI application
handles log output via the `wsgi.errors` object. See [1] for further
information. The problem is that this log output is written to a file called
`horizon.log`, causing it to processed as an 'Oslo log' in the Fluentd
processing pipeline. Since the log format doesn't match the expected format,
this results in parsing errors.

This fix renames the log file and adjusts the format to match other WSGI
applications. The logs are then processed in the same way as other WSGI
application logs, resolving the issue.

[1] https://modwsgi.readthedocs.io/en/master/user-guides/debugging-techniques.html

Change-Id: I93777d1c53920f5470c78356e6b3a4064fbe04b4
Closes-Bug: #1898174
2023-11-28 14:43:18 +00:00
Zuul
db79eb0a55 Merge "Rename kolla_docker to kolla_container" 2023-11-28 12:06:09 +00:00
Matt Crees
cdda49ecb9 Revert "Enable RabbitMQ HA queues by default"
This reverts commit b86c304a2987d0e60570bcd43cbb08da9b64c241.

Reason for revert: We want to enable Quorum Queues by default in Caracal, without requiring two queue migrations between releases. See etherpad for details: https://etherpad.opendev.org/p/kolla-ansible-rmq-quorum-queues-proposal

Change-Id: Ia19ab97f538125475297976347c5da332a7fdda7
2023-11-28 10:07:12 +00:00