3380 Commits

Author SHA1 Message Date
Gupta, Sangeet (sg774j)
ba998fc142 cert-rotation: Return true if grep finds no match
If grep does not find a match, it return 1 which fails the shell
script. Hence made it return true if no match is found.
Also, removed returning of error from the script becasue any failure
will cause the job to re-run which may re-renew certificates and
restart the pods again. And this can continue if the error persists.

Chaange-Id: I2a38b59789fd522e8163ff9b12ff847eb1fe2f3a
Change-Id: Ica456ef6c5bec2bd29f51aaeef7b5ce5e8681beb
2021-08-06 17:58:28 +00:00
Lo, Chi (cl566n)
830df06628 Enable TLS path between Prometheus-elasticsearch-exporter and Elasticsearch
Elasticsearch is TLS enabled.  Prometheus-elasticsearch-exporter
needs to be configured to use cacert when communicating with Elasticsearch.

Change-Id: I4a87226fed541777df78733f3650363859ff01b8
2021-08-06 10:02:18 -07:00
Gage Hugo
a4f300e3da Update helm 2 version to latest
The version of helm 2 that OSH has been using was older and seems
to have been removed from the googleapi repo that the jobs are
setup to use, this was causing job failures.

This change updates the version to the latest v2 release.

Change-Id: I675f539b24ea9c2355ac9eacc7dd8122c5236e5f
2021-08-06 10:56:50 -05:00
Zuul
b49919283d Merge "cert-rotation: New chart for certificate rotation" 2021-08-05 22:13:57 +00:00
Gupta, Sangeet (sg774j)
f94aed3c7a cert-rotation: New chart for certificate rotation
This chart creates a cronjob which monitors the expiry of the
certificates created by jetstack cert-manager. It rotates the
certificates and restarts the pods that mounts the certificate
secrets so that the new certificate can take effect.

Change-Id: I492b5f319cf0f2e7ccbbcf516953e17aafc1c59f
2021-08-05 17:46:15 +00:00
Chinasubbareddy Mallavarapu
7117c93772 [ceph-osd] Change var crash mount propagation to HostToContainer
- As it will be a security violation to mount anything under /var
partition to pods , changing the mount propagation to HostToContainer

Change-Id: If7a27304507a9d1bcb9efcef4fc1146f77080a4f
2021-08-05 14:33:06 +00:00
Zuul
8d00380469 Merge "Add Alertmanager dashboard to Grafana" 2021-08-03 18:25:20 +00:00
Zuul
48e4ce50ac Merge "namespace-config: Grant access to existing PSP" 2021-08-03 00:44:58 +00:00
Lo, Chi (cl566n)
5a290e1d83 Add Alertmanager dashboard to Grafana
This patch set adds a new Alertmanager dashboard to Grafana.  Note
that a new configmap is created for this instead of using the
same configmap which includes all the dashboards. Using the same
configmap will eventually run into issue with configmap size limitation.

Change-Id: I10561c0b0b464c3b67d4a738f9f2cb70ef601b3d
2021-08-02 16:05:47 -07:00
Zuul
8351fdd0f1 Merge "Use focal libvirt image for victoria and wallaby" 2021-08-02 18:36:31 +00:00
DeJaeger, Darren (dd118r)
f26d4db145 Update mon-check with latest monmap outputs
This PS updates the mon-check reap-zombies python script to consider
the more recent Ceph changes, including the fact that there is now
a v1 and v2 backend. In addition, it executes the reap-zombies script
with the python3 binary, as the basic 'python' binary does not exist
in the container.

Change-Id: Id079671f03cc5ddbe694f2aa8c9d2480dc573983
2021-08-02 13:16:39 +00:00
Phil Sphicas
3c4ebf0172 namespace-config: Grant access to existing PSP
This change updates the namespace-config chart to (optionally) create
RBAC rules allowing service accounts in the namespace 'use' access to an
existing Pod Security Policy in the cluster. The policy is specified as:

    podSecurityPolicy:
      existingPsp: name-of-existing-psp

This aligns with the PSP deprecation guidance provided to date [0],
which suggests easing the transition to the "PSP Replacement Policy" by
establishing the standard PSPs (Restricted, Baseline, and Privileged),
assigning a cluster-wide default, and binding more-permissive policies
as needed in certain namespaces.

[0] https://kubernetes.io/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/

Change-Id: I46da230abf822e0cc3553561fd779444439c34a7
2021-08-02 01:36:36 +00:00
Andrii Ostapenko
15b43d939e Use focal libvirt image for victoria and wallaby
Change-Id: I70a989aeaac3d763b110cc854e00fa33d5f8861a
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2021-07-31 20:54:08 +00:00
Zuul
9797d1b034 Merge "Revoke all privileges for PUBLIC role in postgres dbs" 2021-07-30 22:53:04 +00:00
Zuul
de4d8a02b0 Merge "Limit Ceph OSD Container Security Contexts" 2021-07-30 22:44:18 +00:00
Maximilian Weiss
bc754e088e Revoke all privileges for PUBLIC role in postgres dbs
Change-Id: I98102bd9c72264c7e364b50e0683e4777b42b0e7
2021-07-30 17:16:58 +00:00
Parsons, Cliff (cp769u)
6e794561ac Limit Ceph Provisioner Container Security Contexts
Wherever possible, the ceph-provisioner containers need to run
with the least amount of privilege required. In some cases there
are privileges granted but are not needed. This patchset modifies
those container's security contexts to reduce them to only what
is needed.

Change-Id: I74bd31df4af5cacc26834e645b0816bf285e8428
2021-07-29 20:25:07 +00:00
Parsons, Cliff (cp769u)
b55143dec2 Limit Ceph OSD Container Security Contexts
Wherever possible, the ceph-osd containers need to run with the
least amount of privilege required. In some cases there are
privileges granted but are not needed. This patchset modifies
those container's security contexts to reduce them to only what
is needed.

Change-Id: I0d6633efae7452fee4ce98d3e7088a55123f0a78
2021-07-29 20:24:37 +00:00
Chinasubbareddy Mallavarapu
bf5f545c1c [ceph-provisioner] Add check for empty ceph endpoint
This is to add check to find out empty ceph mon endpoint while
generating ceph etc configmap for clients.

Change-Id: I6579a268c5f4bc458120dda66667988e5a529ee9
2021-07-29 12:23:26 +00:00
Haider, Nafiz (nh532m)
adab36be22 Helm-Toolkit: Make Rabbit-init job more robust
Change-Id: I36ef7b2cdcf747ed2503ca5d27bc7803349f287d
2021-07-27 20:19:56 +00:00
Zuul
5e20097998 Merge "[ceph-osd] Mount /var/crash inside ceph-osd pods" 2021-07-27 15:40:59 +00:00
Ritchie, Frank (fr801x)
0acc0ce3dd Fix placement target delete function
You must specify the zone or zonegroup.

Change-Id: Id2bb6d5576ba39fb3671f7426e48f174fcf0016b
2021-07-23 17:21:44 -05:00
Stephen Taylor
c2ca599923 [ceph-osd] Mount /var/crash inside ceph-osd pods
This change adds /var/crash as a host-path volume mount for
ceph-osd pods in order to facilitate core dump capture when
ceph-osd daemons crash.

Change-Id: Ie517c64e08b11504f71d7d570394fbdb2ac8e54e
2021-07-20 15:30:19 -06:00
Neely, Travis (tn720x)
6169504761 Update db backup/restore retry for sending to remote
There is an additional error status 'Service Unavailable' which can
indicate the service is temporary unavailable. Adding that error
status to the retry list in case the issue is resolved during the
backup timeframe.

Change-Id: I9e2fc1a9b33dea3858de06b10d512da98a635015
2021-07-20 10:47:38 -05:00
Zuul
2c1a7b772b Merge "RabbitMQ add preStop and prep 3.8.x feature flag" 2021-07-14 18:49:22 +00:00
Zuul
d2822cbe16 Merge "Enable probes override from values.yaml for libvirt" 2021-07-14 07:06:26 +00:00
Anjeev Kumar
b11b4ae6c3 Enable probes override from values.yaml for libvirt
This PS enables overriding liveness/readiness probes configurations
for libvirt pods via values.yaml. In addition, updating the values
for some of the fields of the probes as the default values seem to
be too aggresive.

Change-Id: I64033a1d67461851d8f2d86905ef7068c2ec43b6

Co-authored-by: Huy Tran <ht095u@att.com>
Change-Id: Ib10379829e2989d3de385ad6d1944565b2f9953f
2021-07-13 14:08:59 -05:00
Roy Tang
479a1c7335 RabbitMQ add preStop and prep 3.8.x feature flag
This ps updates the following:
- Add preStop action to allow rabbitmq node a chance to more
  graceful shutdown
- Add support for RABBITMQ_FEATURE_FLAG in preparation for
  future 3.8.x upgrade.

Change-Id: I25d1e4fdb9dee370382e97a5a97b2b098f5ef11f
2021-07-13 14:57:03 -04:00
Zuul
4d2f78fee2 Merge "Added the helm hook for create user job for exporter" 2021-07-12 21:17:32 +00:00
Zuul
06a90742a1 Merge "Disable RGW crash dumps" 2021-07-12 21:12:03 +00:00
Zuul
db7aad8b84 Merge "Added helm hook for rabbitmq job cluster wait" 2021-07-12 20:45:59 +00:00
Ritchie, Frank (fr801x)
56ae7ae52c Disable RGW crash dumps
While ceph crash dumps are needed for mons and osds there is no
need for rgw.

Change-Id: I62c3c9a089d29528f79653c412fba5200fd1595e
2021-07-08 13:45:19 -05:00
xuxant02@gmail.com
812aba01b8 Added helm hook for rabbitmq job cluster wait
Job wait cluster was failing due to the field immutability which was
resulting in the manual delete of the job for every helm upgrade to be
successful. Reason being job being upgraded before the other manifest
that are required been updated. It can be avoided by using helm-hook
post-install and post-upgrade which will force the job manifest to be
applied only after all other manifest are applied. Hook annotation is
provided "5" so that the if other jobs are annotated, exporter job will
be last to created in case hooks are added to the other jobs in chart.
Also helm3_hook value is used for condition.

Change-Id: Ib83f1d4bef6300c2b76aa54f08927b74346184c7
2021-07-08 22:07:59 +05:45
xuxant02@gmail.com
9133218e83 Added the helm hook for create user job for exporter
exporter-jpb-create-user was failing due to the field immutability
which was resulting in the manual delete of the job for every helm
upgrade to be successful. Reason being job being upgraded before the
other manifest that are required been updated. It can be avoided by
using helm-hook post-install and post-upgrade which will force the
job manifest to be applied only after all other manifest are applied.
Hook annotation is provided "5" so that the if other jobs are annotated,
exporter job will be last to created.
helm3_hook value is used for the condition which will enable the disable
of the hook.

Change-Id: I2039abb5bad07a19fd09fc5e245485c3c772beca
2021-07-08 22:04:36 +05:45
Ritchie, Frank (fr801x)
3c4828935a Add hash for ceph rgw keystone user
This is needed for rgw pods will restart when keystone user secret
changes.

Change-Id: I2092bdc9a4e8a328aec393cf07f8be196be26c32
2021-07-07 13:34:41 -05:00
Stephen Taylor
07ceecd8d7 Export crash dumps when Ceph daemons crash
This change configures Ceph daemon pods so that
/var/lib/ceph/crash maps to a hostPath location that persists
when the pod restarts. This will allow for post-mortem examination
of crash dumps to attempt to understand why daemons have crashed.

Change-Id: I53277848f79a405b0809e0e3f19d90bbb80f3df8
2021-06-30 14:24:15 -06:00
Zuul
a2b9fe53e3 Merge "Nagios: Mount internal TLS CA certificate" 2021-06-30 16:59:40 +00:00
Gupta, Sangeet (sg774j)
7057def52b Nagios: Mount internal TLS CA certificate
Mounted internal TLS CA certificate to be able to communicate with
prometheus and elasticsearch.

Change-Id: I1fc5e1e7c46a95f50487eea5924a13bdcad51b51
2021-06-29 13:23:55 +00:00
Zuul
4b0bb674ed Merge "Use local auth before keystone for s3" 2021-06-28 17:38:27 +00:00
Zuul
6d3272a04f Merge "chore(openssl): updates cert generation" 2021-06-28 17:35:21 +00:00
Zuul
30e179b166 Merge "Uplift ingress to 0.42.0" 2021-06-28 16:46:55 +00:00
Zuul
60ea0b3883 Merge "kafka broker hosts should be defined with a comma separated list" 2021-06-28 16:46:49 +00:00
Tin Lam
00052793dd chore(openssl): updates cert generation
This patch removes the dependency on cfssl to generate certificates and
removes unused constructs in the script.

Change-Id: Ia933420157f456bf99a6ec5416e6dbb63bfa5258
Signed-off-by: Tin Lam <t@lam.wtf>
2021-06-27 06:10:04 +00:00
Zuul
d067a0bb7a Merge "Ceph OSD Init Improvements" 2021-06-25 21:56:26 +00:00
Smith, David (ds3330)
0ecb9bf288 kafka broker hosts should be defined with a comma separated list
The broker attribute should use a comma separated list with the port
definition included

Example: kafka1:9092,kafka2:9092,kafka:9092

The kafka client will connect to the first available host this
will provide resiliency if a host is not available

Change-Id: I5f82e96f2aa274379b6d808291d4b5109709bf72
2021-06-25 14:06:26 +00:00
jinyuanliu
cc9ad68b30 Clean up residual file
The soft link file has been deleted, This path cleans up the redundant files.

Change-Id: I71b9f7de7bf2cfac71984047b8d44d707ed2b07f
2021-06-25 13:53:01 +08:00
Parsons, Cliff (cp769u)
b3ebb46ce2 Ceph OSD Init Improvements
Some minor improvements are made in this patchset:
1) Move osd_disk_prechecks to the very beginning to make sure the
   required variables are set before running the bulk of the script.
2) Specify variables in a more consistent manner for readability.
3) Remove variables from CLI commands that are not used/set.

Change-Id: I6167b277e111ed59ccf4415e7f7d178fe4338cbd
2021-06-24 17:12:34 +00:00
Ritchie, Frank (fr801x)
787e692ea0 Use local auth before keystone for s3
This change is to have RGW use local authentication before Keystone
when both are enabled. This can improve performance:

https://cloudblog.switch.ch/2020/02/10/radosgw-keystone-integration-performance-issues-finally-solved/

Given that we do not duplicate local users in keystone with different
passwords this should be a safe change.

Change-Id: I976a47a5d68884ffb54a0ddd8ab802d69cecbf44
2021-06-24 10:47:27 -05:00
Stephen Taylor
62f5cab770 [ceph-rgw] Fix a bug in placement target deletion for new targets
A deployment that specifies a placement target with "delete: true"
should delete that placement target if it exists. For a clean
deployment the expectation is that the placement target should be
created and immediately deleted; however, the check for existence
happens before its creation and the delete doesn't execute as a
result. This change adds a recheck for existence immediately after
creation to remedy that.

Change-Id: I26f7fa79c5c851070e94af758d0a0438aa7efa52
2021-06-22 07:11:25 -06:00
Zuul
196592ded6 Merge "Enable Ceph CSI Provisioner to Stand Alone" 2021-06-21 04:17:52 +00:00