2461 Commits

Author SHA1 Message Date
Steven Fitzpatrick
fa77f3fc6b Reduce osh-infra check jobs
This change moves the following non-voting checks to experimental:

  - openstack-helm-infra-federated-monitoring
  - openstack-helm-infra-kafka
  - openstack-helm-infra-local-storage
  - openstack-helm-infra-aio-network-policy
  - openstack-helm-infra-apparmor
  - openstack-helm-infra-aio-logging-apparmor
  - openstack-helm-infra-openstack-support-apparmor
  - openstack-helm-infra-metacontroller

Change-Id: I6bb70be0dc1b012742405818d049e9d0e155f671
2020-04-08 08:57:36 -05:00
Tin Lam
01ccb0b861 fix(mariadb): handle empty grastate value
In the scenario where grastate values cannot be found, we will set the
configmap to 'None' and log a warning.. This should also prevent a possible
type incompatibility issue in error scenario.

Change-Id: I0fb08b329a3fb05c65bead5781c84a592ae4c263
Signed-off-by: Tin Lam <tin@irrational.io>
2020-04-07 22:54:12 -05:00
Zuul
9e48974fcf Merge "fix(mariadb): handle IndexError" 2020-04-08 00:55:23 +00:00
Tin Lam
6e84da7671 fix(mariadb): encode Popen() returns
subprocess.Popen() returns byte object by defect which has issue with
operations that treats thing as str. This ensure Popen() encodes the
return as utf-8 before we do anything.

Change-Id: I321771f69cfcb492be1308c61313a0598b1e766a
Signed-off-by: Tin Lam <tin@irrational.io>
2020-04-07 16:29:47 -05:00
Tin Lam
0544c7079c fix(mariadb): handle IndexError
This patch set handles an unexpected IndexError stacktrace when the
galera cluster's data file does not return with an expected key with a
colon (:) in the string.

Change-Id: I4f58e97753a0f68468a02b98676e031176145e44
Signed-off-by: Tin Lam <tin@irrational.io>
2020-04-07 21:14:43 +00:00
Pete Birley
19a1fbf8f7 Ceph: Dont mount log directories to host
This PS updates the bind mounts for ceph logs directorys to be
emptydirs. This ensures we do not polute the hosts permanantly
with ceph logs, which should be directed to stdout.

Change-Id: I6d72c0864b9ecc493cd62564e0e0450d90cfcf00
Signed-off-by: Pete Birley <pete@port.direct>
2020-04-07 18:05:04 +00:00
jacky06
c7d4270af4 [ussuri][goal] Updates for python 2.7 drop
OpenStack is dropping the py2.7 support in ussuri cycle.

Only update required is in tox to define common baspython as py3 os that
all tox env including [testenv:functional] will use py3 instead of py2.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: I36ee8166e5c6d84303a868d8efe044bfc0b55722
2020-04-06 05:02:16 +00:00
Zuul
36310e39a2 Merge "[FIX] Fixes libvirt apparmor and gate jobs" 2020-04-04 06:10:48 +00:00
diwakar thyagaraj
6632b114b8 [FIX] Fixes libvirt apparmor and gate jobs
This change refactors the apparmor job to utilize the feature
gates system instead of relying on separate scripts.

Change-Id: I51b36c1972ff3ee8d4366bf2d5027e433721d740
2020-04-03 18:17:58 +00:00
Chinasubbareddy Mallavarapu
6913435cc2 [ceph-osd] Fix creating osd-id file logic for log-runner container
This is to fix the issue with osd-id file creation since current
logic failing to create /tmp/osd-id file when we restart the pod/container.

Change-Id: Ie9f810fa3a705eccaf625a3cbd91d3d8ee4e05d1
2020-04-03 09:22:04 -05:00
Zuul
e42a628243 Merge "Fluentd: Switch to Native Metrics Plugin" 2020-04-02 20:50:53 +00:00
diwakar thyagaraj
868bd18cf1 Remove Duplicate Apparmor configs from values
Since apparmor configs are moved to value overrides, removing this.

Change-Id: Ia23c34c2ed76fceb78f68e609066139b69e09e61
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-01 18:41:05 +00:00
Zuul
0541bdc6b5 Merge "elasticsearch: Add metadata labels to CronJob" 2020-04-01 09:23:32 +00:00
Tin Lam
df4cc7827d [Mariadb] Initialize variable
There are scenarios where the wsrep_rec_pos variable is being returned
without it being first initialized when the .communicate() method
returns a blank. This patchset sets up a default initialization, so
the readiness check does not error out with an exception.

Change-Id: Ifea922f446bf3cbc9220f39a41dffc2763e6a5f3
Signed-off-by: Tin Lam <tin@irrational.io>
2020-03-31 16:17:10 -05:00
Chinasubbareddy Mallavarapu
af9ac277e8 [ceph-provisioner] update msgr2 port for etc configmap
This is to update msgr2 port for clients who uses ceph etc file
from configmap

Change-Id: If6079e860afc67c5e466faaa8bea040f67b1f842
2020-03-30 17:42:00 -05:00
Chinasubbareddy Mallavarapu
1a0ca47b51 [ceph-client] update helm tests logic for pg autoscaler
This is to update helm tests logic depends on pg autoscaler
enabled or not.

Change-Id: I14e86c9f14260bff10f098caaa96709550372ee3
2020-03-30 14:18:20 -05:00
Steven Fitzpatrick
9d2e08f1a4 Fluentd: Switch to Native Metrics Plugin
This change updates the fluentd chart to use the native
fluent-plugin-prometheus for metric production. This plugin
provides more detailed metrics about fluentd's operations,
specifically regarding input and output statistics.

https://github.com/fluent/fluent-plugin-prometheus

Using the plugin, each fluentd pod produces metrics, so
scape annotations have been added to the pod spec. The
zuul check on metric producers has been updated to account
for this.

Depends-On: https://review.opendev.org/714167
Change-Id: I809356d92b0cff1e31cb2062102bbedefd4843fd
2020-03-30 00:44:58 -05:00
Zuul
6201e6b1e5 Merge "[ceph-daemons] Redirect all the logs to stdout" 2020-03-29 22:34:35 +00:00
Zuul
b5bf0946e5 Merge "Ceph-OSD: Add log tailer to ceph-osd" 2020-03-29 22:31:15 +00:00
Chinasubbareddy Mallavarapu
977a5a2f97 [ceph-daemons] Redirect all the logs to stdout
This is to redirect all the logs from daemons to stdout to avoid
accumulating large sized log files on filesystem.

NOTE: The ceph-osd daemon won't work this way and is addressed
separately in https://review.opendev.org/715295. All other Ceph
daemons are included here.

Change-Id: I3045d6e941791aba14979472fac1bca09776d3bf
2020-03-29 10:27:44 -06:00
Chinasubbareddy Mallavarapu
872f68ada7 [ceph-pools] Enable autoscaler by reading from values
This is to make enabling autosclaer feature optional from values since
its new feature and few deployments may not required it.

Change-Id: Ie7cbdb71f6a1bf636db2d61ae0f3a4c19af2ca34
2020-03-28 20:26:33 -05:00
Pete Birley
5701d26a13 Ceph-OSD: Add log tailer to ceph-osd
This change adds in a log tailer script in a pod that tails the ceph
logs, periodicly truncating them.

Change-Id: Idbe03d4123f86b1b88e277fea6d13f58104f94b0
Signed-off-by: Pete Birley <pete@port.direct>
2020-03-27 21:14:11 +00:00
Kabanov, Dmitrii
13f54b0e03 [Ceph] Add msgr1 port for ceph-provisioners
Change-Id: Ifa9b44074d927006f47dfcc449361cf3f6aa9413
2020-03-27 08:28:58 -07:00
Luna Das
ea7fdef8e5 [FIX] Add whitelisting of fields of various types to configMap.
This Patch Set enables whitelisting of proper field names
of different types(i.e string, integer)in configMap volume.
It makes the pauseImage for daemonjob configurable.

Change-Id: Ia2062c5bc9ba1d8783e9573d7f9ea315c34f7fe7
2020-03-27 19:47:06 +05:30
Zuul
aa01846f60 Merge "Disable podsecuritypolicy job" 2020-03-26 21:00:08 +00:00
Zuul
35552e6c9f Merge "Enable Apparmor to fluentd" 2020-03-26 17:15:51 +00:00
diwakar thyagaraj
c0bec2c744 Enable Apparmor to fluentd
Change-Id: I21640c263cbf7871319d2710160e37a9dddf0eb2
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-03-25 21:50:53 +00:00
Wickersham, Brian (bw6938)
b607f8654b [ceph-client] Set num PGs to small value for a pg autoscaling bug
This is to workaround a pg merging bug in ceph. The number of PGs
is set to a small value. The reason this works is because the
pg autoscaler is set to work unconditionally. This will need
to change once the autoscaler is optional.

Change-Id: I6ec404f281e201023fa974601d7083f4ce3cfd30
2020-03-25 20:46:46 +00:00
Zuul
344440ef27 Merge "Fixes the pod prestop hook" 2020-03-25 17:45:27 +00:00
Zuul
965fd345c3 Merge "Revert "Enable Apparmor for fluentd"" 2020-03-25 00:01:34 +00:00
Steve Wilkerson
d898a65a2d Revert "Enable Apparmor for fluentd"
This reverts commit a3110abd66727e8fab881a044110d01d2f405019.

Change-Id: I90180d5caa6cd7873220fcc91570c92ae7a234e5
2020-03-24 19:46:19 +00:00
Zuul
f8a1fad09c Merge "Update gather-prom-metrics service selection" 2020-03-24 18:53:53 +00:00
Steven Fitzpatrick
510f55a48b Update gather-prom-metrics service selection
This change updates how the gather-prom-metrics playbook role
chooses which services and ports to scrape when gathering metrics
at the end of a zuul build.

We can hit more metric endpoints by finding services with a "metrics"
port, as not all applications have a service labeled "component=exporter"

Change-Id: Ib8db7ea2e7034063eefadad74828d0396407275b
2020-03-23 22:19:55 -05:00
Hemachandra Reddy
b5c12377da Fixes the pod prestop hook
Change-Id: I94988da32bdcf2d1a947abecf4faa8cd84a54741
2020-03-23 23:45:04 +00:00
Zuul
eb5bccc4be Merge "Fix Kibana Selenium tests" 2020-03-23 16:18:10 +00:00
Zuul
0415bdef08 Merge "Fix Grafana Selenium tests" 2020-03-23 14:22:46 +00:00
Zuul
3e8aad1fc7 Merge "Adjust RabbitMQ Exporter Probes" 2020-03-20 16:20:07 +00:00
Zuul
bdf901ea5c Merge "Add more fields to daemonjob crd spec." 2020-03-20 05:50:20 +00:00
Zuul
ff9634dc0f Merge "gnocchi: Add metadata labels to CronJob" 2020-03-20 04:57:02 +00:00
Zuul
977012ec91 Merge "postgresql: Add metadata labels to CronJob" 2020-03-20 04:57:01 +00:00
Gage Hugo
a52604bd80 Disable podsecuritypolicy job
The current podsecuritypolicy job has not been passing for quite
a while. Disable it for now until it's fixed to avoid wasting
infra resources.

Change-Id: I14b184cf03e625cbbaa829a4de101dc2142a7e27
2020-03-20 04:05:15 +00:00
Luna Das
55beab680f Add more fields to daemonjob crd spec.
Change-Id: I1690035c7e35887245bbdcfdc97b19409fd6ab8a
2020-03-20 04:03:13 +00:00
Zuul
d0f808a64e Merge "[libvirt] update values.yaml to use train libvirt image" 2020-03-20 00:00:04 +00:00
dt241s@att.com
a3110abd66 Enable Apparmor for fluentd
Change-Id: I6b759beff9fd3166a9868a4d71319836f272bc25
2020-03-18 21:26:48 +00:00
Cliff Parsons
72afe093aa Adjust RabbitMQ Exporter Probes
The currently defined RabbitMQ Exporter probes make a call to the
"/metrics" path of the exporter service, which downloads a huge file and
takes a very long time to download. An http probe should be based on a very
simple and short url response from the service. So this changes the
probes to just call the base path "/" of the url and set the timeout to
something reasonable like 5 seconds.

Change-Id: Ie106490b2fb8d61660663f39a992bf4dc1a61222
2020-03-18 16:58:14 +00:00
Huang, Sophie (sh879n)
4f30b1361e Mariadb: Add ingress network policy overrides
This patch set adds in default mariadb ingress network policy
overrides for openstack namespace.

Change-Id: I037de30f868dfeb0dedb1c32209b8be6d4690962
2020-03-17 20:00:42 +00:00
Zuul
1a3de19e4f Merge "Enable AppArmor profile to Openvswitch." 2020-03-17 19:57:10 +00:00
Zuul
f5132a0d61 Merge "[ceph-client] Set target size ratio of pools for pg autoscaling" 2020-03-17 19:15:44 +00:00
Zuul
f0e3ce9a08 Merge "Add ability to set the domain name in the Nagios chart" 2020-03-17 17:52:47 +00:00
Wickersham, Brian (bw6938)
4a35fb3faf [ceph-client] Set target size ratio of pools for pg autoscaling
Setting the target size ratio of each pool initially which will
autoscale the num of pgs immediately before any data is written to
the pools. This will reduce backfilling as data is written as the
autoscaling would have been done when the pool was first created.

Change-Id: I00b5372d669068621577ae0fe370219a4aa53b6f
2020-03-17 16:53:42 +00:00