Commit Graph

179 Commits

Author SHA1 Message Date
Vladimir Kozhukalov
7fddc1a8d3 Bump K8s to v1.31
Change-Id: I384b10ef7b2da42d2227b4134e4ece4c5f9aa6d1
2024-12-02 22:00:51 -06:00
Vladimir Kozhukalov
5d943fd72d Remove 2023.1 build jobs
The 2023.1 release is unmaintained since 2024-10-30.
See https://releases.openstack.org/

Change-Id: I8375b16338b172a5875b7a379df085020490305c
2024-11-15 14:40:51 -06:00
Vladimir Kozhukalov
7086815c74 [libvirt] Add 2023.1 overrides
Recently we fixed the libvirt.sh script
and removed the conditionals cgroup commands
which were introduced for smooth transition
to Jammy and cgroups v2

https://review.opendev.org/c/openstack/openstack-helm-infra/+/929401

But because we didn't have overrides for 2023.1
we used to run 2023.1 with the default libvirt image
openstackhelm/libvirt:latest-ubuntu_focal
which does not work with cgroups v2 on the host
system with this recent fix (see above).

So the 2023.1 Ubuntu Jammy compute-kit test jobs fails.
This PR fixes this job by means of introducing
explicit image overrides for 2023.1.

Change-Id: Ie81f8fb412362388274ea92ad7fa5d3d176c0441
2024-09-24 21:49:34 -05:00
Vladimir Kozhukalov
466e2ed931 Add compute-kit-2023-1-ubuntu_focal job
This is necessary to test if libvirt changes
are compatible with cgroups v1.

Change-Id: I3cfb4e747a4cd23bc2d7051ef526fd58dc38aaf8
2024-09-16 14:59:40 -05:00
Vladimir Kozhukalov
f2bdcae040 Update deploy-env role
- Use kubeadm configuration to not set taints
  on control plain nodes (instead of removing them after
  deployment).
- Fix ssh client key permissions.
- Update the Mariadb ingress test job so it is inherinted
  from the plain compute-kit test job. And also remote
  it from the check pipeline.

Change-Id: I92c73606ed9b9161f39ea1971b3a7db7593982ff
2024-09-03 17:32:28 -05:00
Sergiy Markin
43fd714348 Ingress-nginx controller upgrade for mariadb
This PS bumps up ingress-nginx controller version
to v1.11.2 in mariadb chart due to CVE
vulnerability.

nginx.tmpl from mariadb chart has been updated to
match the latest 1.11.2 ingress-controller image.

Change-Id: Ie2fd811f8123515f567afde62bbbb290d58dd1b2
2024-08-27 15:36:23 +00:00
Vladimir Kozhukalov
1178ded805 Upgrade env
- K8s 1.30.3
- Helm 3.14.0
- Crictl 1.30.1
- Calico 3.27.4
- Cilium 1.16.0
- Ingress-nginx Helm chart 4.11.1

Change-Id: I3d5a3d855b0b4b0b66e42d94e1e9704f7f91f88b
2024-07-29 05:47:12 -05:00
Vladimir Kozhukalov
05f2f45971 Add 2024.1 overrides to some charts
- Add 2024.1 overrides to those charts where
  there are overrides for previous releases.
- Update some jobs to use 2024.1 overrides.
- Update default images in  grafana, postgresql,
  nagios, ceph-rgw, ceph-provisioners,
  kubernetes-node-problem-detector
- Install tzdata package on K8s nodes. This
  is necessary for kubernetes-node-problem-detector
  chart which mounts /etc/localtime from hosts.

Change-Id: I343995c422b8d35fa902d22abf8fdd4d0f6f7334
2024-07-23 07:35:50 -05:00
Vladimir Kozhukalov
62f12e3a1f Add Flannel deployment to deploy-env role
Change-Id: I72f3f29196ea1d433655c8862ac34718df18c7ea
2024-07-15 12:12:48 -05:00
Vladimir Kozhukalov
1d34fbba2a Test job for legacy OSH Ceph to Rook migration
At the moment the recommended way of managing Ceph clusters
is using Rook-Ceph operator. However some of the users
still utilize legacy OSH Ceph* charts. Since Ceph is
a critical part of the infrastructure we suggest a migration
procedure and this PR is to test it.

Change-Id: I837c8707b9fa45ff4350641920649188be1ce8da
2024-07-02 12:41:54 -05:00
Vladimir Kozhukalov
bc45596483 Add Cilium deployment to deploy-env role
Change-Id: I7cec2d3ff09ec3f85992162bbdb8c351660f7de8
2024-07-02 08:58:30 +00:00
Vladimir Kozhukalov
b460c559bb Simplify ceph-adapter-rook
- Do not deploy anything in the ceph namespace
- Prepare admin key secret in the openstack namespace.
  Get admin key from the Ceph tools pod
- Prepare Ceph client config with the mon_host
  taken from the rook-ceph-mon-endpoints configmap
  as recommended in the Rook documentation.

Change-Id: Idd4134efab49de032a389283e611c4959a6cbf24
2024-06-19 06:41:03 -05:00
Vladimir Kozhukalov
cbc4dffb30 Bump K8s version to 1.29.5
Change-Id: I4a3c7a17f32b5452145e1677e3c5072875dc9111
2024-05-15 13:47:28 -05:00
Vladimir Kozhukalov
427b0163eb Cleanup unused scripts
Change-Id: I3bad13cc332fd439b3b56cfa5fc596255bc466f2
2024-05-13 16:17:12 -05:00
astebenkova
93aec7e807 [chromedriver] Loosen compatibility up with Chrome
Chromedriver had strict version selection. This commit allows
it to pick the closest patch version to google-chrome-stable

Change-Id: I435985573f69ee4bb0f6009416452649f302c0fe
2024-05-01 13:29:24 -05:00
Vladimir Kozhukalov
d31027cfb4 Install OSH Helm plugin
Depends-On: I71ab6ad104beb491b5b15b7750e2fc0988db82bf
Change-Id: I8f30fbdf94d76ef9fa2985a25c033df290995326
2024-04-17 19:06:48 -05:00
Vladimir Kozhukalov
cdbecfb7f4 Update test jobs
- Remove openstack-helm-infra-openstack-support* jobs.
  Instead of these jobs we run compute-kit, cinder and tls
  jobs defined in the openstack-helm repo.
- Remove all experimental jobs since they are outdated and
  do not work. We will later add some of the test cases
  including apparmor, network policy, tenant Ceph and others.

Change-Id: I8f3379c06b4595ed90de025d32c89de29614057d
2024-04-14 20:47:46 -05:00
Vladimir Kozhukalov
51c7affc72 Run more test jobs when helm-toolkit updated
Specifically we would like at least the following
deployments to be tested when helm-toolkit is updated
- compute-kit
- cinder
- tls

Change-Id: I3991d6984563813d5a3a776eabd52e2e89933bd8
2024-04-03 14:07:23 -05:00
Vladimir Kozhukalov
6ca83be780 Rename dpdk job name to reflect Openstack version
Change-Id: I9c04a60ae8b7fde35a8a970e3b74bcaad7bd564f
2024-03-26 11:22:04 -05:00
Vladimir Kozhukalov
81828430e5 Bump RabbitMQ version 3.9.0 -> 3.13.0
Also
- Update default Heat image to 2023.2 used for
  init and test jobs
- Add overrides for
  - yoga-ubuntu_focal
  - zed-ubuntu_focal
  - zed-ubuntu_jammy
  - 2023.1-ubuntu_focal
  - 2023.1-ubuntu_jammy
  - 2023.2-ubuntu_jammy

Change-Id: I516c655ea1937f9bd1d363ea86d35e05e3d54eed
2024-03-25 17:56:06 -05:00
Vladimir Kozhukalov
5b1879aa09 Refactor deploy-env role
- Make it less mixed. Each task file
  deploys one feature.
- Deploy Metallb
- Deploy Openstack provider network gateway

Change-Id: I41f0353b286f817cb562b3bd59992e4baa473568
2024-03-25 14:45:00 -05:00
Vladimir Kozhukalov
fb90642b18 Update ovn controller init script
- OVN init script must be able to attach an interface
  to the provider network bridge and migrate IP from the
  interface to the bridge exactly like Neutron OVS agent
  init script does it.

- OVN init script sets gateway option to those OVN controller
  instances which are running on nodes with l3-agent=enabled
  label.

Change-Id: I24345c1f85c1e75af6e804f09d35abf530ddd6b4
2024-03-21 16:03:51 -05:00
Sergiy Markin
0e086e4c12 [mariadb] Switch to ingress-less mariadb
This PS switches mariadb to use primary service by
default instead of ingress based deployment. The
primary service that is getting created and
automatically updated based on the leader election
process in start.py entrypoint script.

Mariadb primary service was introduced by this PS:

https://review.opendev.org/c/openstack/openstack-helm-infra/+/905797

Change-Id: I4992276d0902d277a7a81f2730c22635b15794b0
2024-02-26 18:59:22 +00:00
astebenkova
2216cbfec4 Add compute-kit job with DPDK enabled
+ add role for enabling hugepages

Change-Id: I89d3c09ea3bedcba6cb51178c8d1ac482a57af01
Depends-On: I2f9d954258451f64eb87d03affc079b71b00f7bd
2024-02-14 13:23:34 -06:00
Vladimir Kozhukalov
6a452ecb49 Remove some aio jobs
These two jobs openstack-helm-infra-aio-monitoring and
openstack-helm-infra-aio-logging were only needed for
backward compatibility.

Depends-On: I9c3b8cd18178aa57ce44564490ef1b61f275ae29
Change-Id: I09d0e48128a3fd98fa9148b8e520df75d6e5be50
2024-02-05 14:17:39 -08:00
Zuul
010c21cfc9 Merge "Bump Calico version to v3.27.0" 2024-02-03 00:06:33 +00:00
Vladimir Kozhukalov
cf2cdd7821 Fix prevent trailing whitespace lint command
Recently we added a jpg file to OSH documentation
but the lint job didn't run due to the job configuration.

But then for the next PR link job did run and failed
due to trailing whitespace in the jpg file.

Change-Id: I9abf8f93a4566411076190965f282375846dc5db
2024-02-02 14:12:26 -06:00
Vladimir Kozhukalov
cfff60ec10 Bump Calico version to v3.27.0
Change-Id: I8daa54e70c66cec41733d6b9fd5c9dd4597ff9c1
2024-02-02 13:54:22 -06:00
Zuul
c48dffaaa2 Merge "Use upstream ingress-nginx chart" 2024-02-01 21:42:47 +00:00
Vladimir Kozhukalov
88ad17a84b Use upstream ingress-nginx chart
Change-Id: I90a1a1e27f0b821bbecfe493057eada81d4f9424
2024-02-01 10:45:05 -06:00
Vladimir Kozhukalov
03225aad49 Use containerized Openstack client
Change-Id: I17c841b74bf92fc3ac375404b27fa2562603604f
2024-01-31 13:42:43 -06:00
Sergiy Markin
07bd8c92a2 [mariadb] Add mariadb-server-primary service
This PS adds mariadb-server-primary service that is getting created
and automatically updated based on the leader election process in
start.py entrypoint script.

Change-Id: I1d8a8db0ce8102e5e23f7efdeedd139726ffff28
Signed-off-by: Sergiy Markin <smarkin@mirantis.com>
2024-01-18 00:17:47 +00:00
Vladimir Kozhukalov
9e256fd8b0 Update k8s packages repo
For details see the announcement
https://kubernetes.io/blog/2023/08/31/legacy-package-repository-deprecation/

Also bump K8s version up to 1.28.4

Change-Id: Ic6b3478e53504622804b6f003ca176a679573d5b
2023-12-12 17:58:20 -06:00
Vladimir Kozhukalov
978507351f Add ceph-adapter-rook chart
When using Rook for managing Ceph clusters we have
to provision a minimal set of assets (keys, endpoints, etc.)
to make Openstack-Helm charts work with these Ceph clusters.

Rook provides CRDs that can be used for managing Ceph assets
like pools/keyrings/buckets etc. but Openstack-Helm can not
utilize these CRDs. To support these CRDs in OSH would
require having lots of conditionals in OSH templates since
we still want OSH to work with OSH ceph-* charts.

Change-Id: If7fe29052640e48c37b653e13a74d95e360a6d16
2023-12-05 14:27:57 -06:00
Sergiy Markin
29f2b616cc [mariadb-operator] Mariadb-cluster chart
This PS adds mariadb-cluster chart based on mariadb-operator. Also for
some backward compartibility this PS adds mariadb-backup chart and
prometheus-mysql-exporter chart as a separate ones.

Change-Id: I3f652375cce2e3b45e095e08d2e6f4ae73b8d8f0
2023-11-29 21:51:48 -06:00
Vladimir Kozhukalov
7f783dba51 Update elasticsearch chart to work with Rook Ceph
When using Rook for managing Ceph we can use
Rook CRDs to create S3 buckets and users.

This PR adds bucket claim template to the
elasticsearch chart. Rook creates a bucket for
a bucket claim and also creates a secret
containing the credentials to get access to this
bucket. So we also add a snippet to expose
these credentials via environment variables to
containers where they are needed.

Change-Id: Ic5cd35a5c64a914af97d2b3cfec21dbe399c0f14
2023-11-26 19:34:42 -06:00
Vladimir Kozhukalov
145e9df9b7 Run Rook job on multi-node env
Change-Id: Idce9fd9f4817e0dd07b49c291fa6a0a887384073
2023-11-24 19:14:37 -06:00
Vladimir Kozhukalov
510cea0c23 Deploy Ceph on multi-node envs
- In case we deploy Ceph on a multi-node env we have
  to prepare the loop devices on all nodes. For this
  we moved loop devices setup to the deploy-env
  Ansible role.

  For simplicity we need the same device on all nodes,
  so we create a loop device with a big
  minor number (/dev/loop100 by default) hoping
  that only low minor numbers could be busy.

- For test jobs we don't need to use different devices
  for OSD data and metadata. There is no
  any benefit from this for the test environment.
  So let's keep it simple and put both OSD data and metadata
  on the same device.

- On multi-node env Ceph cluster needs cluster members
  see each other, so let's use pod network CIDR.

Change-Id: I493b6c31d97ff2fc4992c6bb1994d0c73320cd7b
2023-11-24 19:06:08 -06:00
Vladimir Kozhukalov
c047fce569 Fix path for setup-client.sh script
Change-Id: Ieb7549d2f00d981efa1d4bc2d6d8a57a067ef6c7
2023-11-06 10:34:20 -06:00
Zuul
ff552d5969 Merge "Create osh-bandit role" 2023-10-27 22:12:27 +00:00
Sergiy Markin
a430d16bd5 Control ceph loopback devices path
This PS adds control over location of Ceph loopback devices path.

Change-Id: Ib0738c1127ff37633cdd035b3978cc137c5eaf71
2023-10-26 17:28:45 +00:00
Vladimir Kozhukalov
45b209ac79 Create osh-bandit role
The motivation is to reduce the code base and get rid
of unnecessary duplications. This PR is moves bandit
tasks from the osh-infra-bandit.yaml playbook
to the osh-bandit role. Then we can use this role for the
same job in OSH.

Change-Id: I9489a8c414e6679186e6c399243a7c0838df812a
2023-10-25 17:59:56 -05:00
Sergiy Markin
f9b0360418 Mount extra 80Gb volume
This PS mounts extra 80Gb volume if available and mounts it to
/opt/ext_vol. It also alters docker and containerd configs to move their
root folder to that extra volume. This helps zuul gates to succeed when
a node with 40Gb volume is assigned to a zuul gate.

Change-Id: I1c91b13c233bac5ebfe6e3cb16d4288df2c2fe80
2023-10-24 18:39:51 +00:00
Stephen Taylor
a58f80599b [ceph] Add support for deploying and managing Ceph with Rook
This change adds an openstack-support-rook zuul job to test
deploying Ceph using the upstream Rook helm charts found in the
https://charts.rook.io/release repository. Minor changes to the
storage keyring manager job and the mon discovery service in the
ceph-mon chart are also included to allow the ceph-mon chart to be
used to generate auth keys and deploy the mon discovery service
necessary for OpenStack.

Change-Id: Iee4174dc54b6a7aac6520c448a54adb1325cccab
2023-10-03 07:16:02 -06:00
Vladimir Kozhukalov
ae91cf3fc3 Use deploy-env role for all deployment jobs
To make it easier to maintain the jobs all experimental
jobs (those which are not run in check and gate pipelines)
are moved to a separate file. They will be revised later
to use the same deploy-env role.

Also many charts use Openstack images for testing this
PR adds 2023.1 Ubuntu Focal overrides for all these charts.

Change-Id: I4a6fb998c7eb1026b3c05ddd69f62531137b6e51
2023-09-22 15:02:07 -05:00
Samuel Liu
664f4c9dfb Remove PodSecurityPolicy
PodSecurityPolicy was deprecated in Kubernetes v1.21, and removed from Kubernetes in v1.25.[1]

In Kubernetes 1.21, PodSecurityPolicy is deprecated. As with all Kubernetes feature deprecations, PodSecurityPolicy will continue to be available and fully-functional for several more releases. PodSecurityPolicy, previously in the beta stage, is planned for removal in Kubernetes 1.25.[2]

[1] https://kubernetes.io/docs/concepts/security/pod-security-policy/
[2] https://kubernetes.io/blog/2021/04/08/kubernetes-1-21-release-announcement/#podsecuritypolicy-deprecation

Change-Id: Ic060d925b6e97e5651e74a1a1161906aef740a8c
2023-06-08 21:53:29 +03:00
astebenkova
92d16f3a29 [osh-selenium] Upgrade image to latest-ubuntu_focal
+ migrate all Python tests to use Selenium v4 (bionic image had v3 installed):
https://www.selenium.dev/documentation/webdriver/getting_started/upgrade_to_selenium_4/
+ amend selenium role in order to install ChromeDriver compatible with Google Chrome:
https://chromedriver.chromium.org/downloads/version-selection
+ run selenium tests AFTER the charts are deployed

Change-Id: I46200b7dc173bd0e1e6bf3545d9a26c252a21927
2023-05-23 18:09:16 +03:00
Gage Hugo
74652cb181 Bump ubuntu and openstack version in jobs
The current zuul jobs definitions still use an older release and
distro version of ubuntu. This change modifies the versions to run
ubuntu focal and the Xena release of openstack.

Change-Id: I653fd9ed42972c7bba5fa94519cd413c0d15b2c9
2023-02-16 06:02:48 +00:00
Gage Hugo
6be6d638b3 Update ubuntu nodesets to focal
The openstack-single-node nodeset still is using ubuntu-bionic,
which is nearly 4 years old now. This change updates it to use
the newer ubuntu focal release.

Depends-on: https://review.opendev.org/c/openstack/openstack-helm-infra/+/839996
Depends-on: https://review.opendev.org/c/openstack/openstack-helm-infra/+/840370

Change-Id: Ia43cb31e13bc059541116064aa2092526186b831
2022-05-03 19:57:15 +00:00
Gage Hugo
d37fd936bf Remove keystone-auth job
The kubernetes-keystone-auth check job has not been ran in a long
time and has not been maintained. This change removes it from the
list of jobs defined and jobs ran in the osh-infra project.

Change-Id: If0275524fda92d8fd8baa689521e2e841210ce51
2022-03-31 17:00:16 -05:00