2330 Commits

Author SHA1 Message Date
KHIYANI, RAHUL (rk0850)
19fe6ca7ab Memcached: Add apparmor profile to memcached init containers
Change-Id: I194e24da1bc5813179b582ad89bc9c41213fe4d0
2020-05-04 16:37:58 +00:00
Steve Taylor
a12ea0244e [ceph-osd] Use lvm commands instead of ceph-volume to get OSD properties
This change removes "ceph-volume inventory" and "ceph-volume lvm
list" commands from the ceph-volume OSD initialization script and
Bluestore start script and replaces them with "pvdisplay" and
"lvs" to retrieve lvm tags directly from lvm volumes instead.
Ceph-volume makes repeated calls to blkid, which is very slow in
some cases and deadlocks in others when there are RBDs mapped on
the host.

Change-Id: Ia999770d4a59729e38dbb494b34c30e5a1b36a8b
2020-04-30 16:21:13 +00:00
Kabanov, Dmitrii
8dbd488605 [Ceph-OSD] Update getting and releasing of lock
The PS moves the relese of lock out from the if-else statement.
Otherwise we are not releasing the lock in two of three cases.
And the lock will be released only if we are going though
the last "else" statement. The PS supposed to fix that.
Also it increases the timeout value.

Change-Id: I284cff06a34ddb864484d2a63780b4fdb9075a0a
2020-04-29 18:19:19 +00:00
Zuul
bce44bb981 Merge "Add Project Specific Contributor and PTL Docs" 2020-04-29 17:14:43 +00:00
diwakar thyagaraj
ccaa11b649 Enable Apparmor to ceph client bootstrap Pods
Change-Id: Ia8fd1e50a2478743f0ff625ffdd8801610f05ee1
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-29 14:43:50 +00:00
KHIYANI, RAHUL (rk0850)
5bc24e78a4 Ingress: Run nginx ingress as non-root user
Running nginx ingress with www-data user

Change-Id: I769577cdedd05cc1b8b035928e67ad7ed70568db
2020-04-27 16:12:21 -05:00
Roman Gorshunov
e4096d84e7 Add Project Specific Contributor and PTL Docs
This patch amends contributor guidelines for the Ussuri goals.

Change-Id: Idc88bf4e0f6e41c2ac1b89c700314cd9038982d4
Task: 38545
Story: 2007236
2020-04-24 20:59:44 +02:00
Zuul
bbfa550361 Merge "Enable Apparmor to ceph-bootstrap Pods" 2020-04-24 17:28:51 +00:00
Cliff Parsons
41342cdc4a Fix MariaDB Backup Problems
This patch fixes 2 problems with MariaDB backup:
1) If a user with grants to a database has a hyphenated name, the backup
script errors out and the grants for this user won't be saved in the backup.
2) While restoring databases from a backup, if connections are allowed
during the restore operation, there is potential for deadlock. Table
level locks are added to the backup sql file in order to try to prevent
these deadlock situations.

Change-Id: If612e7b9f3f4d75fc67018eea17609f07a0c0b0f
2020-04-24 14:03:59 +00:00
diwakar thyagaraj
fb0cd00a56 Enable Apparmor to ceph-bootstrap Pods
Change-Id: Ifa8d43a2a68fffaea554f04a5df63fb6b7ea5422
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-24 02:22:17 +00:00
Zuul
51bff9687e Merge "Postgresql backup/restore enhancements" 2020-04-23 17:16:37 +00:00
Cliff Parsons
382d113a87 Postgresql backup/restore enhancements
1) Added a new backup container for accessing RGW via Openstack Swift API.
2) Modified the backup script so that tarballed databases can be sent to the RGW.
3) Added new script to send the database backup to the RGW.
4) Modified the restore script so that databases can be retrieved from the RGW.
5) Added new script to retrieve the database backups from the RGW.

Change-Id: Id17a8fcb63f5614ea038c58acdc256fb4e05f434
2020-04-22 22:31:48 +00:00
Chinasubbareddy Mallavarapu
8d9b7fdd2b [ceph-mon-check] fix the command to connect correct ceph cluster
This is to fix the command to connect to the cluster name instead of
namesapce.

Change-Id: I8b8f7c10d7667245a8f6cb02fb5b69dd122099e5
2020-04-22 20:02:14 +00:00
Taylor, Stephen (st053q)
9057c770a6 Enable cephfs tests
Cephfs tests were disabled in order to merge
https://review.opendev.org/695568 due to gate failures that were
blocking it. CephFS isn't used in openstack-helm-infra, so it
wasn't required for that work. This change re-enables the cephfs
tests so we can work through any issues that are causing further
failures.

Since the the issue got fixed in 14.2.8 , upgrading all daemons to 14.2.8.
(https://tracker.ceph.com/issues/43770)

Change-Id: I376d39b7ee00ccb1ab8046b58f92b19a822272e1
2020-04-21 22:25:52 +00:00
Gage Hugo
fa0876750a Fixing docs build for osh-infra
There was a change[0] in zuul that made the docs job fail
if the build/html dir was empty. Our docs job was putting the
built docs in the source file by mistake, this ps fixes
this issue.

[0] https://review.opendev.org/#/c/721221/

Change-Id: I9ef336cf32dad96ff4a4fcc2743363a8f40f4718
2020-04-21 20:05:39 +00:00
diwakar thyagaraj
965e1b6079 [FIX] Fix Typo in Apparmor profile to Ingress Charts
Change-Id: I548b441d6c712fd63fdf7dacb475683533a001ca
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-19 00:13:49 +00:00
Zuul
359b1c7be7 Merge "[ceph-osd] A rack's OSDs are not marked out after down_out interval" 2020-04-18 04:18:15 +00:00
John Lawrence
35872a9f25 Process Exporter: Override Children Parameter
Currently  resource usage of subprocesses is added
in to their parent's usage. And this will provide
option to override.

Change-Id: I22c36e5a6f354f6318e72798ce9865011a85b2af
2020-04-16 16:45:15 +00:00
Wickersham, Brian (bw6938)
5952acdd93 [ceph-osd] A rack's OSDs are not marked out after down_out interval
An entire rack's OSDs are not being marked out after
down_out interval. This manifested itself during
resiliency testing when all interfaces were brought
down on a control plan host and the down_interval
was surpassed.

Change-Id: I6f4a69ec442c3e768feb7bd74c7d610aa9d4aa67
2020-04-16 16:13:21 +00:00
Kabanov, Dmitrii
35ff844cbb [Ceph-RGW] Update annotation for the ingress controller
The PS corrects the annotation for the ingress controller.

Change-Id: I16dd75c357ee6e40eb86ba9cfb64b8b4a869ac1a
2020-04-14 09:31:51 -07:00
Andreas Jaeger
34d54f2812 Cleanup py27 support and docs
This repo is now testing docs only with Python 3, so let's make
a few cleanups:
- Remove obsolete sections from setup.cfg
- Switch to using sphinx-build
- Cleanup doc/source/conf.py to remove now obsolete content.
- Use newer openstackdocstheme version
- Remove install_command from tox.ini, the default is fine

Change-Id: Iffd00261e6a15e2dd7c98e96bbcc9db3e86c0c65
2020-04-11 15:26:50 +02:00
Zuul
60cb50eeae Merge "Reduce osh-infra check jobs" 2020-04-09 00:07:40 +00:00
Steven Fitzpatrick
fa77f3fc6b Reduce osh-infra check jobs
This change moves the following non-voting checks to experimental:

  - openstack-helm-infra-federated-monitoring
  - openstack-helm-infra-kafka
  - openstack-helm-infra-local-storage
  - openstack-helm-infra-aio-network-policy
  - openstack-helm-infra-apparmor
  - openstack-helm-infra-aio-logging-apparmor
  - openstack-helm-infra-openstack-support-apparmor
  - openstack-helm-infra-metacontroller

Change-Id: I6bb70be0dc1b012742405818d049e9d0e155f671
2020-04-08 08:57:36 -05:00
Tin Lam
01ccb0b861 fix(mariadb): handle empty grastate value
In the scenario where grastate values cannot be found, we will set the
configmap to 'None' and log a warning.. This should also prevent a possible
type incompatibility issue in error scenario.

Change-Id: I0fb08b329a3fb05c65bead5781c84a592ae4c263
Signed-off-by: Tin Lam <tin@irrational.io>
2020-04-07 22:54:12 -05:00
Zuul
9e48974fcf Merge "fix(mariadb): handle IndexError" 2020-04-08 00:55:23 +00:00
Tin Lam
6e84da7671 fix(mariadb): encode Popen() returns
subprocess.Popen() returns byte object by defect which has issue with
operations that treats thing as str. This ensure Popen() encodes the
return as utf-8 before we do anything.

Change-Id: I321771f69cfcb492be1308c61313a0598b1e766a
Signed-off-by: Tin Lam <tin@irrational.io>
2020-04-07 16:29:47 -05:00
Tin Lam
0544c7079c fix(mariadb): handle IndexError
This patch set handles an unexpected IndexError stacktrace when the
galera cluster's data file does not return with an expected key with a
colon (:) in the string.

Change-Id: I4f58e97753a0f68468a02b98676e031176145e44
Signed-off-by: Tin Lam <tin@irrational.io>
2020-04-07 21:14:43 +00:00
Pete Birley
19a1fbf8f7 Ceph: Dont mount log directories to host
This PS updates the bind mounts for ceph logs directorys to be
emptydirs. This ensures we do not polute the hosts permanantly
with ceph logs, which should be directed to stdout.

Change-Id: I6d72c0864b9ecc493cd62564e0e0450d90cfcf00
Signed-off-by: Pete Birley <pete@port.direct>
2020-04-07 18:05:04 +00:00
jacky06
c7d4270af4 [ussuri][goal] Updates for python 2.7 drop
OpenStack is dropping the py2.7 support in ussuri cycle.

Only update required is in tox to define common baspython as py3 os that
all tox env including [testenv:functional] will use py3 instead of py2.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: I36ee8166e5c6d84303a868d8efe044bfc0b55722
2020-04-06 05:02:16 +00:00
Zuul
36310e39a2 Merge "[FIX] Fixes libvirt apparmor and gate jobs" 2020-04-04 06:10:48 +00:00
diwakar thyagaraj
6632b114b8 [FIX] Fixes libvirt apparmor and gate jobs
This change refactors the apparmor job to utilize the feature
gates system instead of relying on separate scripts.

Change-Id: I51b36c1972ff3ee8d4366bf2d5027e433721d740
2020-04-03 18:17:58 +00:00
Chinasubbareddy Mallavarapu
6913435cc2 [ceph-osd] Fix creating osd-id file logic for log-runner container
This is to fix the issue with osd-id file creation since current
logic failing to create /tmp/osd-id file when we restart the pod/container.

Change-Id: Ie9f810fa3a705eccaf625a3cbd91d3d8ee4e05d1
2020-04-03 09:22:04 -05:00
Zuul
e42a628243 Merge "Fluentd: Switch to Native Metrics Plugin" 2020-04-02 20:50:53 +00:00
diwakar thyagaraj
868bd18cf1 Remove Duplicate Apparmor configs from values
Since apparmor configs are moved to value overrides, removing this.

Change-Id: Ia23c34c2ed76fceb78f68e609066139b69e09e61
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-01 18:41:05 +00:00
Zuul
0541bdc6b5 Merge "elasticsearch: Add metadata labels to CronJob" 2020-04-01 09:23:32 +00:00
Tin Lam
df4cc7827d [Mariadb] Initialize variable
There are scenarios where the wsrep_rec_pos variable is being returned
without it being first initialized when the .communicate() method
returns a blank. This patchset sets up a default initialization, so
the readiness check does not error out with an exception.

Change-Id: Ifea922f446bf3cbc9220f39a41dffc2763e6a5f3
Signed-off-by: Tin Lam <tin@irrational.io>
2020-03-31 16:17:10 -05:00
Chinasubbareddy Mallavarapu
af9ac277e8 [ceph-provisioner] update msgr2 port for etc configmap
This is to update msgr2 port for clients who uses ceph etc file
from configmap

Change-Id: If6079e860afc67c5e466faaa8bea040f67b1f842
2020-03-30 17:42:00 -05:00
Chinasubbareddy Mallavarapu
1a0ca47b51 [ceph-client] update helm tests logic for pg autoscaler
This is to update helm tests logic depends on pg autoscaler
enabled or not.

Change-Id: I14e86c9f14260bff10f098caaa96709550372ee3
2020-03-30 14:18:20 -05:00
Steven Fitzpatrick
9d2e08f1a4 Fluentd: Switch to Native Metrics Plugin
This change updates the fluentd chart to use the native
fluent-plugin-prometheus for metric production. This plugin
provides more detailed metrics about fluentd's operations,
specifically regarding input and output statistics.

https://github.com/fluent/fluent-plugin-prometheus

Using the plugin, each fluentd pod produces metrics, so
scape annotations have been added to the pod spec. The
zuul check on metric producers has been updated to account
for this.

Depends-On: https://review.opendev.org/714167
Change-Id: I809356d92b0cff1e31cb2062102bbedefd4843fd
2020-03-30 00:44:58 -05:00
Zuul
6201e6b1e5 Merge "[ceph-daemons] Redirect all the logs to stdout" 2020-03-29 22:34:35 +00:00
Zuul
b5bf0946e5 Merge "Ceph-OSD: Add log tailer to ceph-osd" 2020-03-29 22:31:15 +00:00
Chinasubbareddy Mallavarapu
977a5a2f97 [ceph-daemons] Redirect all the logs to stdout
This is to redirect all the logs from daemons to stdout to avoid
accumulating large sized log files on filesystem.

NOTE: The ceph-osd daemon won't work this way and is addressed
separately in https://review.opendev.org/715295. All other Ceph
daemons are included here.

Change-Id: I3045d6e941791aba14979472fac1bca09776d3bf
2020-03-29 10:27:44 -06:00
Chinasubbareddy Mallavarapu
872f68ada7 [ceph-pools] Enable autoscaler by reading from values
This is to make enabling autosclaer feature optional from values since
its new feature and few deployments may not required it.

Change-Id: Ie7cbdb71f6a1bf636db2d61ae0f3a4c19af2ca34
2020-03-28 20:26:33 -05:00
Pete Birley
5701d26a13 Ceph-OSD: Add log tailer to ceph-osd
This change adds in a log tailer script in a pod that tails the ceph
logs, periodicly truncating them.

Change-Id: Idbe03d4123f86b1b88e277fea6d13f58104f94b0
Signed-off-by: Pete Birley <pete@port.direct>
2020-03-27 21:14:11 +00:00
Kabanov, Dmitrii
13f54b0e03 [Ceph] Add msgr1 port for ceph-provisioners
Change-Id: Ifa9b44074d927006f47dfcc449361cf3f6aa9413
2020-03-27 08:28:58 -07:00
Zuul
aa01846f60 Merge "Disable podsecuritypolicy job" 2020-03-26 21:00:08 +00:00
Zuul
35552e6c9f Merge "Enable Apparmor to fluentd" 2020-03-26 17:15:51 +00:00
diwakar thyagaraj
c0bec2c744 Enable Apparmor to fluentd
Change-Id: I21640c263cbf7871319d2710160e37a9dddf0eb2
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-03-25 21:50:53 +00:00
Wickersham, Brian (bw6938)
b607f8654b [ceph-client] Set num PGs to small value for a pg autoscaling bug
This is to workaround a pg merging bug in ceph. The number of PGs
is set to a small value. The reason this works is because the
pg autoscaler is set to work unconditionally. This will need
to change once the autoscaler is optional.

Change-Id: I6ec404f281e201023fa974601d7083f4ce3cfd30
2020-03-25 20:46:46 +00:00
Zuul
344440ef27 Merge "Fixes the pod prestop hook" 2020-03-25 17:45:27 +00:00