2706 Commits

Author SHA1 Message Date
Zuul
e94ecb7e8f Merge "[ceph-osd] enhancement to the "post-apply" job" 2020-08-24 19:06:58 +00:00
Chinasubbareddy Mallavarapu
8adc6216bc [CEPH] Disable ceph pg autoscaler on pools by reading from values
This is to disable unintentionally enabled pg autoscaler on pools
by reading it from values.

Change-Id: Ib919ae7786ec1d4cbe7a309d28fd6571aa6195de
2020-08-21 16:55:33 -05:00
Zuul
754d8e93b4 Merge "Add Alerta feature to osh-infra" 2020-08-19 21:19:18 +00:00
Xiaoguang(William) Zhang
83a55fd19e Add Alerta feature to osh-infra
Change-Id: Id8dc3f86b8d6754df4ba3c0c720a78731e3f54d5
2020-08-19 13:35:40 +00:00
Zuul
f37f6f5c93 Merge "Add capabilitity to backup only a single database" 2020-08-19 03:16:15 +00:00
Kabanov, Dmitrii
badfff4d19 [ceph-osd] enhancement to the "post-apply" job
The PS adds changes which allow to count available OSDs and compare an amount
of "ready" OSDs with total quantity of OSDs. Also it allows to pass the check
if the amount of "ready" OSD is more then required ("required_percent_of_osds").
Otherwise, the check will fail (including the case when one or several pods
in the namespace are not ready after timeout.)

Change-Id: I3cf6dbc6393b62423ee5929167f03b8fc7bbac68
2020-08-18 21:39:02 +00:00
Parsons, Cliff (cp769u)
233197fc0b Add capabilitity to backup only a single database
This PS adds the capability to Mariadb and Postgresql to backup a
single database (as an optional parameter to the backup script).

Change-Id: I9bc1eb0173063638b2cf58465c063f602ed20bc1
2020-08-18 18:30:31 +00:00
Zuul
611fda0171 Merge "Ingress: Configure Default SSL Certificate" 2020-08-18 16:36:39 +00:00
Zuul
622bc4d972 Merge "Remove remaining test pods before new test run" 2020-08-18 16:21:00 +00:00
Huang, Sophie (sh879n)
ba601e0cba Add TLS handling for the retrieval of MariaDB grants
In this patchset, when TLS is enabled, command line options
are added to provide the needed certificates for the invocation
of pt-show-grants during the MariaDB backup process.

Change-Id: I38eacb27ee0051e96c9fb2ba62773e84725ee868
2020-08-17 18:10:32 +00:00
Zuul
1851b47fdf Merge "Add Application armor to Postgresql-backup pods" 2020-08-14 23:36:27 +00:00
Chinasubbareddy Mallavarapu
4214e85a77 [CEPH] Add missing ceph cluster name for helm tests
This is to export the ceph cluster name as environment variable
since its getting referred by scripts.
also to fix the query to get inactive pgs.

Change-Id: I1db5cfbd594c0cc6d54f748f22af5856d9594922
2020-08-14 16:09:19 -05:00
diwakar thyagaraj
acf6276f49 Add Application armor to Postgresql-backup pods
Change-Id: Idb4d214803bb98f1846154bb27d571f44ca74dba
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-08-14 18:23:02 +00:00
Kabanov, Dmitrii
4557f6fbe8 [ceph] Update queries to filter pgs correctly
The PS updates queries in wait_for_pgs function in ceph-client and
ceph-osd charts. It allows more accurately check the status of PGs.
The output of the "ceph pg ls" command may contain many PG statuses,
like "active+clean", "active+undersized+degraded", "active+recovering",
"peering" and etc. But along with these statuses there may be such as
"stale+active+clean". To avoid the wrong interpretation of the status
of the PSs the filter was changed from "startswith(active+)" to
"contains(active)".
Also PS adds a delay after restart of the pods to post-apply job.
It allows to reduce the number of useless queries to kubernetes.

Change-Id: I0eff2ce036ad543bf2554bd586c2a2d3e91c052b
2020-08-13 22:45:01 -07:00
Phil Sphicas
341e9b29df Ingress: Configure Default SSL Certificate
Adds configuration options for the --default-ssl-certificate feature of
NGINX Ingress Controller, which provides a default certificate for
requests that do not match any configured server names.[0]

To enable with a new certificate, specify:
  .conf.default_ssl_certificate.enabled=true
  .endpoints.ingress.host_fqdn_override.public.tls.crt="PEM cert data"
  .endpoints.ingress.host_fqdn_override.public.tls.key="PEM key data"
  .manifests.secret_ingress_tls=true

To enable using a TLS cert in an existing secret, specify:
  .conf.default_ssl_certificate.enabled=true
  .conf.default_ssl_certificate.name="name of the secret"
  .conf.default_ssl_certificate.namespace="namespace of the secret"

0: https://kubernetes.github.io/ingress-nginx/user-guide/tls/#default-ssl-certificate

Change-Id: Idd704fd880f56137923d4c38cc188b130ee3b56d
2020-08-14 05:32:51 +00:00
Zuul
adf9fc7fc4 Merge "Add TLS handling to mariadb backup and restore" 2020-08-14 04:48:47 +00:00
Zuul
486ad9a25f Merge "[ceph-osd] Move back to old naming convention for VGs" 2020-08-13 21:05:50 +00:00
Zuul
6df0a36443 Merge "Add apparmor profile to keystone-webhook container" 2020-08-13 20:40:48 +00:00
Zuul
c19ee4ab94 Merge "[ceph-client] Fix crush weight comparison in reweight_osds()" 2020-08-13 20:40:46 +00:00
Zuul
592153734a Merge "[ceph] Check for osds deployed with zero crush weight" 2020-08-13 20:03:31 +00:00
Kabanov, Dmitrii
4bc95447b6 [ceph-osd] Move back to old naming convention for VGs
The PS switches back to old naming convention for VGs. The old naming
convention have to be used until the changes which allow to handle
update of VG names are merged ( https://review.opendev.org/#/c/745166/ ).
Otherwise, OSDs will not come up after an upgrade.

Change-Id: I1bf9ca93149a93dfd5f79813533ace3a1fe58002
2020-08-13 19:14:01 +00:00
Gayathri Devi Kathiri
a57190fd8a Remove remaining test pods before new test run
If the test pod still exists, then the new test run
fails with ERROR: pods "abc-test" already exists

So, Removing remaining test pods before new test run

Change-Id: I3b3ed5ceaf420aa39a669b4a50a838ad154b1fdd
Closes-Bug: #1882030
2020-08-13 18:12:25 +00:00
Taylor, Stephen (st053q)
f66f9fe560 [ceph-client] Fix crush weight comparison in reweight_osds()
The recently-added crush weight comparison in reweight_osds() that
checks weights for zero isn't working correctly because the
expected weight is being calculated to two decimal places and then
compared against "0" as a string. This updates the comparison
string to "0.00" to match the calculation.

Change-Id: I29387a597a21180bb7fba974b4daeadf6ffc182d
2020-08-13 12:00:32 -06:00
Chinasubbareddy Mallavarapu
64b423cee0 [ceph] Check for osds deployed with zero crush weight
This is to check for osds deployed with zero crush weight from
helm tests.

Change-Id: Ie8d9c65b33bf7a026a342d1d7e81ec37cb981db3
2020-08-13 14:39:38 +00:00
KHIYANI, RAHUL (rk0850)
3332968caa Add apparmor profile to keystone-webhook container
Change-Id: I583c4c01e2c92c16705420fe726e3e7648a16705
2020-08-12 18:57:21 -05:00
Huang, Sophie (sh879n)
48c4d8c900 Add TLS handling to mariadb backup and restore
Added certification handling to restore_mariadb script.

Change-Id: Iff3a15c0b4e84857cafd5b70c6cbea5363734751
2020-08-12 23:13:49 +00:00
Zuul
8631c79548 Merge "Add security context template for keystone-webhook container" 2020-08-12 22:51:41 +00:00
Kabanov, Dmitrii
787052a975 [ceph-osd] update post apply job
The PS updates post apply job and moves execution of the command outside
of if statement. The output of the command stored in a variable
which will be checked in if statement. Added "-z" to correct comparison
of the length of the string (variable). It was accidentally missed in
the initial PS.

Change-Id: I907f75d0a9e5ef27fba5306ddb86199e94b01b3b
2020-08-12 15:49:20 +00:00
Zuul
dbd465c4c6 Merge "prometheus exporter to monitor health check for web URL" 2020-08-12 00:51:52 +00:00
Zuul
8bf7505396 Merge "Run node-problem-detector from 0 user" 2020-08-11 22:35:41 +00:00
KHIYANI, RAHUL (rk0850)
117fae68e6 Run node-problem-detector from 0 user
This adds the runAsUser flag to node-problem-detector at pod level

Change-Id: I0db38599e037ac38b24e50bbc2228ef1fd264c4e
2020-08-11 19:36:15 +00:00
Vipul Jain
628186f05e prometheus exporter to monitor health check for web URL
1) A separate prometheus job need to provide target and scrap metrics
2) it is based on https://github.com/prometheus/blackbox_exporter

Adding script file for deployment and job under zuul.d
Resolving conflict
Change-Id: Ia15ab7d8ef882886fe0e37cc2599e6815d7bcc6c
2020-08-11 22:39:54 +05:30
Taylor, Stephen (st053q)
f1e9a6ba83 [ceph-client] Refrain from reweighting OSDs to 0
If circumstances are such that the reweight function believes
OSD disks have zero size, refrain from reweighting OSDs to 0.
This can happen if OSDs are deployed with the noup flag set.

Also move the setting and unsetting of flags above this
calculation as an additional precautionary measure.

Change-Id: Ibc23494e0e75cfdd7654f5c0d3b6048b146280f7
2020-08-11 09:48:53 -06:00
KHIYANI, RAHUL (rk0850)
a58a78ff83 Add security context template for keystone-webhook container
This implements security context override at pod level and adds
readOnly-fs to keystone-webhook container

Change-Id: Ia67947b7323e41363a5ee379c0dfb001936b5107
2020-08-11 09:45:08 -05:00
Zuul
10fd77b6e4 Merge "Update alertmanager include snmp_notifier function" 2020-08-11 06:16:10 +00:00
Zuul
b05c64033f Merge "Add missing security context template to mariadb-backup pod" 2020-08-11 05:28:51 +00:00
Zuul
d2d3b05273 Merge "Prometheus: Render Rules as Templates" 2020-08-10 20:31:45 +00:00
Steven Fitzpatrick
fb7fc87d23 Prometheus: Render Rules as Templates
This change allows us to substitute values into our rules files.

Example:

- alert: my_region_is_down
  expr: up{region="{{ $my_region }}"} == 0
  
To support this change, rule annotations that used the expansion
{{ $labels.foo }} had to be surrounded with "{{` ... `}}" to render
correctly.

Change-Id: Ia7ac891de8261acca62105a3e2636bd747a5fbea
2020-08-10 18:16:35 +00:00
Zuul
87d7ddaf56 Merge "Add Application Armor to elastic-apm" 2020-08-07 19:35:35 +00:00
Kabanov, Dmitrii
f99444f37a [ceph-osd] update post_apply job
The PS updates wait_for_pods function and adds query to filter the pods
which are not in Running or Succeeded state.
Also the PS reduces the amount of 'kubectl get' requests.

Change-Id: Ie2abdaf0a87ca377f5ce287a3de9e87d1ca6c0d4
2020-08-07 17:16:53 +00:00
Xiaoguang(William) Zhang
7c94deae43 Update alertmanager include snmp_notifier function
Change-Id: I5aedbdcdbba397a9fddde19a0898cb91de08553a
2020-08-07 12:25:33 -04:00
dt241s@att.com
25d03950b0 Add Application Armor to Elastic FileBeat
Change-Id: Ifa6eb48e56e1a92d090ea9dbaaaf5890f95ae032
2020-08-07 14:34:37 +00:00
dt241s@att.com
564cada4ad Add Application Armor to elastic-apm
Change-Id: Id1e6b70db03f71b87539f6e3e466f39d8440b773
2020-08-07 14:34:27 +00:00
Andrii Ostapenko
817efe966c Pass parameter to skip helm tests for component in the same script
Pass parameter from job allowing to parallelize helm tests using
separate scripts.

Change-Id: I3e06c5590d51c75448dc5ff5978dc7fc90daca6f
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-08-06 00:51:15 +00:00
Andrii Ostapenko
fdcbd03784 Allow parallelization in gate runner
Change-Id: I393a22cfcaecb00d14d8ac643bd4b7ffbba03b12
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-08-06 00:51:11 +00:00
Zuul
a6a9146de4 Merge "Add missing security context template to nagios init container" 2020-08-05 19:06:21 +00:00
Zuul
fde1d53cd4 Merge "[Ceph-OSD] Refactor the code of OSD init script" 2020-08-05 18:59:12 +00:00
KHIYANI, RAHUL (rk0850)
cf131bacb2 Add missing security context template to nagios init container
This change adds security context template at container level
to implement readOnly-fs flag

Change-Id: Ibd4f8a916bcd74c1d89aa360e89d4477cd01d367
2020-08-05 10:29:30 -05:00
Zuul
8cd4e26a42 Merge "Fluentd: Allow for Multiple Config Files" 2020-08-05 14:48:13 +00:00
Kabanov, Dmitrii
cbc8001255 [Ceph-OSD] Refactor the code of OSD init script
The PS adds the changes which simplifies the code and
removes unnecessary steps.

Change-Id: I66e2b661e4d2cbc4a08d7690514321f9f3127e19
2020-08-05 14:45:50 +00:00