3824 Commits

Author SHA1 Message Date
Sergiy Markin
d0b3f1c1d2 [mariadb-operator] Fix mariadb TLS
This PS fixed some imcompatibilities of inherited mariadb config with
docker-entrypoint.sh script that is now used to perform initial mariadb
nodes setup and mariadb-upgrade at startup.
Also added x509 requirement for root and audit users
connections.


Change-Id: Ic5ad2e692b64927fc73962fe0cc250a9d682114c
2024-01-08 17:43:17 +00:00
Zuul
359ab4b8d7 Merge "Update template for ingress 1.9" 2024-01-05 16:32:34 +00:00
Ritchie, Frank (fr801x)
f66bb53509 Update template for ingress 1.9
The names of a few configuration variables have changed in version 1.9.

EnableRealIp to EnableRealIP
HttpAccessLogPath to HTTPAccessLogPath
whitelist to allowlist
Whitelist to Allowlist

Additionally,

ajp_temp_path

is no longer valid.

Change-Id: I2ebb658bd237216c43306dab6cd7f7a1ca6388ac
2024-01-04 18:32:56 -05:00
Zuul
4b6654abf1 Merge "[mariadb-operator] Enable auto-upgrade" 2024-01-04 21:06:27 +00:00
Zuul
3678f4a9d1 Merge "Use host network for ovn controller pods" 2024-01-04 13:32:01 +00:00
Zuul
1566fd2c5f Merge "Add Ubuntu Focal and Jammy overrides for openvswitch" 2024-01-04 13:23:30 +00:00
Sergiy Markin
2627138d98 [mariadb-operator] Enable auto-upgrade
This PS enables auto-upgrade feature from official mariadb docker
entrypoint script.

Also switching mariadb image to the official from docker.io/mariadb
repo and adding temp volime mount to mariadb-server pods created by
mariadb-operator.

Change-Id: Ie3a02e546fd2a56948177b97c009eab35b42776a
2024-01-03 22:59:25 +00:00
Vladimir Kozhukalov
4d5919b070 Use host network for ovn controller pods
Change-Id: I9f852ff54cfc42536387fa51a73f019b56070345
2024-01-03 10:08:56 -06:00
Vladimir Kozhukalov
1a112e9fba Add Ubuntu Focal and Jammy overrides for openvswitch
Change-Id: Ifc4fa0cbc9c7b4f2a9785edcecd562beb00abab3
2024-01-02 08:59:09 -06:00
Anselme, Schubert (sa246v)
7532c7700e
Enable addition of default consumer prefetch count
Change-Id: Ib1e29be00ec6accf78a01c4931d62fadf1ea28a3
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
2023-12-22 13:58:33 -05:00
Zuul
d7a45a864a Merge "Add license headers to deploy-env tasks files" 2023-12-19 22:25:10 +00:00
Zuul
463290fbc6 Merge "Fix ovn ovsdb port number" 2023-12-19 21:23:35 +00:00
Zuul
53b4ba5313 Merge "Add ovn overrides" 2023-12-19 21:21:19 +00:00
Zuul
9ced5ece88 Merge "[backups] Add throttlling of remote backups" 2023-12-19 14:43:54 +00:00
Sergiy Markin
13c1d8cd38 [backups] Add throttlling of remote backups
This PS adds a possibility to limit (to throttle) the number of
simultaneously uploaded backups while keeping the logic on the client
side using flag files on remote side. The main idea is to have an
ability to limit number of simultaneous remote backups upload sessions.

Change-Id: I5464004d4febfbe20df9cd41ca62ceb9fd6f0c0d
2023-12-18 20:39:45 +00:00
Vladimir Kozhukalov
6e4045097d Fix ovn ovsdb port number
Change-Id: I7a83b5f51748d75c748180ba9288758f8528db1b
2023-12-18 14:15:05 -06:00
Vladimir Kozhukalov
39712c3725 Add ovn overrides
- ubuntu_focal
- ubuntu_jammy

Change-Id: Id6e55a86b810b6a43eb0a30d7bd6253f4b4fb509
2023-12-18 14:07:09 -06:00
Ritchie, Frank (fr801x)
5cbce03f21 Enable management api metrics collection
The default rabbitmq image disables metrics collection via the management
api. This is implemented by adding a file named:

/etc/rabbitmq/conf.d/management_agent.disable_metrics_collector.conf

with the contents:

management_agent.disable_metrics_collector = true

The prometheus exporter currently used by osh requires this value to be
false.

This change was introduced when rabbit introduced the integrated
prometheus exporter:

https://github.com/docker-library/rabbitmq/issues/419

Change-Id: I9a94f49a7827bb4725ed3fd98404e637bfefa086
2023-12-18 12:55:35 -05:00
Sergiy Markin
f66c924b2f [backups] Mariadb backups improvements
This PS removes mariadb-verify-server sidecar container from
mariadb-backup cronjob in order to make backup process more resilient.

Change-Id: I2517c2de435ead34397ca0483610f511c8035bdf
2023-12-15 16:18:35 +00:00
Vladimir Kozhukalov
e45cbaf088 Add license headers to deploy-env tasks files
Change-Id: Ic0ed6d2cdc02e5f55019f9f38a3811af6b39a5ea
2023-12-13 14:19:50 -06:00
Zuul
a656fc4875 Merge "Update k8s packages repo" 2023-12-13 18:53:02 +00:00
Vladimir Kozhukalov
9e256fd8b0 Update k8s packages repo
For details see the announcement
https://kubernetes.io/blog/2023/08/31/legacy-package-repository-deprecation/

Also bump K8s version up to 1.28.4

Change-Id: Ic6b3478e53504622804b6f003ca176a679573d5b
2023-12-12 17:58:20 -06:00
Zuul
28ce54fcb8 Merge "Update curator for es v8" 2023-12-12 21:26:49 +00:00
Ritchie, Frank (fr801x)
7167b9bf31 Update curator for es v8
This PS is to update es curator for elasticsearch v8. Curator 5.x
is not compatible with es v8.

Changes are needed for config.yml:

https://github.com/elastic/curator#new-client-configuration

No changes are required for the actions file.

Change-Id: I6968e22c7ae5f630e1342f47feee0c2c494b767f
2023-12-12 11:16:14 -05:00
Vladimir Kozhukalov
10a171eb18 Increase the number of inotify instances
For TLS test jobs on Ubuntu Jammy when we run
dnsmasq on the master node needed for testing
we get the error:

"failed to create inotify: Too many open files"

By default the number of inotify instances on Jammy
is 128. We increase this up to 256.

Change-Id: I07c8a0f909608b6e44040ffeefc6ab576236c93f
2023-12-11 13:07:09 -06:00
Stephen Taylor
426c71f753 Initialize registry_namespaces unconditionally if not initialized
The deploy-env playbook can fail with an error stating that
registry_namespaces is not defined in some cases. This change moves
the initialization of registry_namespaces so that buildset_registry
is not required for it to be set when other conditions are not met.

Change-Id: I160e7d479008fd3afd460382691673b92bd042c9
2023-12-06 12:15:57 -07:00
Zuul
ee22dd9b4e Merge "Fix ceph-adapter-rook.sh script" 2023-12-06 06:35:12 +00:00
Zuul
428b8d261f Merge "Make curator path configurable" 2023-12-06 05:31:32 +00:00
Vladimir Kozhukalov
bba74aefde Fix ceph-adapter-rook.sh script
Change-Id: I6ebcceb105781e2ca2a39ca84d4e4bc9171a5f15
2023-12-05 22:24:15 -06:00
Ritchie, Frank (fr801x)
e36b5d6dab Make curator path configurable
Some es curator images do not use /usr/bin/curator for the executable. This PS
makes the path configurable via values.yaml.

Change-Id: I640e0f4928683810ef0b4a6d4dbac9bdf865aa2a
2023-12-05 17:11:15 -05:00
Vladimir Kozhukalov
978507351f Add ceph-adapter-rook chart
When using Rook for managing Ceph clusters we have
to provision a minimal set of assets (keys, endpoints, etc.)
to make Openstack-Helm charts work with these Ceph clusters.

Rook provides CRDs that can be used for managing Ceph assets
like pools/keyrings/buckets etc. but Openstack-Helm can not
utilize these CRDs. To support these CRDs in OSH would
require having lots of conditionals in OSH templates since
we still want OSH to work with OSH ceph-* charts.

Change-Id: If7fe29052640e48c37b653e13a74d95e360a6d16
2023-12-05 14:27:57 -06:00
Sergiy Markin
4a95f75b6b [backups] Added staggered backups
This PS adds staggered backups possibility by adding anti-affinity rules
to backups cronjobs that can be followed across several namespaces to
decrease load on remote backup destination server making sure that at
every moment in time there is only one backup upload is in progress.

Change-Id: If49791f866a73a08fb98fa0e0b4854042d079c66
2023-12-05 04:10:22 +00:00
Sergiy Markin
29f2b616cc [mariadb-operator] Mariadb-cluster chart
This PS adds mariadb-cluster chart based on mariadb-operator. Also for
some backward compartibility this PS adds mariadb-backup chart and
prometheus-mysql-exporter chart as a separate ones.

Change-Id: I3f652375cce2e3b45e095e08d2e6f4ae73b8d8f0
2023-11-29 21:51:48 -06:00
Vladimir Kozhukalov
3d64d4c832 Update get-values-overrides.sh script
The PR synchronized this script with that
used in the openstack-helm repo.

Let's use the same script in both repos.
The related PR for the openstack-helm repo
is coming.

Change-Id: I5cfaad8ebfd08790ecabb3e8fa480a7bf2bb7e1e
2023-11-29 22:56:42 +00:00
Vladimir Kozhukalov
730488ca53 Disable metrics gathering for Rook Ceph cluster
We don't need this for tests and it is better to
keep the test env minimal since the test hardware
is limited.

Change-Id: I0b3f663408c1ef57ad25a4d031b706cb6abc87a9
2023-11-29 15:47:20 -06:00
Vladimir Kozhukalov
1c5610c2c0 Uncomment erroneously commented jobs in check pipeline
Change-Id: Icae3903cb3818e5eb5a15e93b751b3ba4ccad32e
2023-11-27 17:13:03 -06:00
Vladimir Kozhukalov
7f783dba51 Update elasticsearch chart to work with Rook Ceph
When using Rook for managing Ceph we can use
Rook CRDs to create S3 buckets and users.

This PR adds bucket claim template to the
elasticsearch chart. Rook creates a bucket for
a bucket claim and also creates a secret
containing the credentials to get access to this
bucket. So we also add a snippet to expose
these credentials via environment variables to
containers where they are needed.

Change-Id: Ic5cd35a5c64a914af97d2b3cfec21dbe399c0f14
2023-11-26 19:34:42 -06:00
Vladimir Kozhukalov
145e9df9b7 Run Rook job on multi-node env
Change-Id: Idce9fd9f4817e0dd07b49c291fa6a0a887384073
2023-11-24 19:14:37 -06:00
Vladimir Kozhukalov
510cea0c23 Deploy Ceph on multi-node envs
- In case we deploy Ceph on a multi-node env we have
  to prepare the loop devices on all nodes. For this
  we moved loop devices setup to the deploy-env
  Ansible role.

  For simplicity we need the same device on all nodes,
  so we create a loop device with a big
  minor number (/dev/loop100 by default) hoping
  that only low minor numbers could be busy.

- For test jobs we don't need to use different devices
  for OSD data and metadata. There is no
  any benefit from this for the test environment.
  So let's keep it simple and put both OSD data and metadata
  on the same device.

- On multi-node env Ceph cluster needs cluster members
  see each other, so let's use pod network CIDR.

Change-Id: I493b6c31d97ff2fc4992c6bb1994d0c73320cd7b
2023-11-24 19:06:08 -06:00
Ali Safari
ab14348f97 Add labels to rabbitmq service
Change-Id: I53d18ee535ff563d33387ba633776a060cd1d389
2023-11-07 22:40:43 +00:00
Leontii Istomin
42d86b17ca Remove versions from doc/requirements.txt to avoid confusion
Versions from TOX_CONSTRAINTS_FILE are used which is defaulted to
  https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt

Change-Id: I547c244f9d79f3a0f4d0269f546495504f2340cd
2023-11-07 21:00:14 +00:00
Stephen Taylor
86aa30fc72 [ceph-rgw] Multiple namespace support for the ceph-rgw-pool job
The ClusterRole and ClusterRoleBinding definitions for the
ceph-rgw-pool job don't take the namespace into account. This isn't
an issue for deployments that include a single Ceph cluster, but
this change adds the namespace to the names of those resources to
allow the job to be deployed correctly in multiple namespaces.

Change-Id: I98a82331a52702c623941f839d1258088813f70e
2023-11-06 17:57:10 +00:00
Vladimir Kozhukalov
c047fce569 Fix path for setup-client.sh script
Change-Id: Ieb7549d2f00d981efa1d4bc2d6d8a57a067ef6c7
2023-11-06 10:34:20 -06:00
Stephen Taylor
d070774bfc [ceph-rgw] Add a ceph-rgw-pool job to re-run the ceph-rbd-pool job
The Reef release disallows internal pools from being created by
clients, which means the ceph-client chart is no longer able to
create the .rgw.root pool and configure it. The new ceph-rgw-pool
job deletes and re-creates the ceph-rbd-pool job after ceph-rgw has
been deployed so that job can configure the .rgw.root pool
correctly.

Change-Id: Ic3b9d26de566fe379227a2fe14dc061248e84a4c
2023-11-02 07:05:37 -06:00
Zuul
bad0169ece Merge "Fix deploy-env when buildset_registry is defined" 2023-11-01 19:10:07 +00:00
Vladimir Kozhukalov
b5b66f1489 Fix deploy-env when buildset_registry is defined
It used to configure /etc/hosts in two different places.
The buildset registry record was added while configuing
Containerd and then this record was removed while
configuring Kubernetes.

The PR adds the buildset registry record to the /etc/hosts
template and the task is moved to the tasks/main.yaml.

Change-Id: I7d1ae6c7d33a33d8ca80b63ef9d69decb283e0a6
2023-10-31 17:34:12 -05:00
SPEARS, DUSTIN (ds443n)
b769895a60 Update openvswitch to support cgroups v2
Adds check and if cgroups v2 is active use
cgroups v2 file structure for setting cpus

Change-Id: I603271a1b043d192988694c50ea7411a567b16ca
2023-10-30 17:33:38 -04:00
Zuul
ff552d5969 Merge "Create osh-bandit role" 2023-10-27 22:12:27 +00:00
Zuul
1fcb3f1b23 Merge "Fix deploy-env role" 2023-10-27 21:48:05 +00:00
Vladimir Kozhukalov
f9f487ce4a Fix deploy-env role
The role tried to include non-existing file
which was forgotten while we moved the role to this repo.
This inclusion is only actual for cases when we
consume images from a buildset registry.

Change-Id: I1510edf7bdc78f9c61f7722e2c7848e152edf892
2023-10-27 13:59:19 -05:00