2469 Commits

Author SHA1 Message Date
Zuul
9df8b4ee94 Merge "Elasticsearch Secret Vars" 2020-05-14 15:48:28 +00:00
Zuul
3408809e3d Merge "Elasticsearch remote logging user" 2020-05-14 15:16:52 +00:00
KHIYANI, RAHUL (rk0850)
845385de38 Mariadb: Add apparmor profile to init-containers
This change adds apparmor profile to mariadb init containers and
ingress init container

Change-Id: I843baf221a82d234104b14db5b02026fe87e6063
2020-05-14 09:57:21 -05:00
Smith, David (ds3330)
9178fd1dac Elasticsearch remote logging user
This patchset adds the ability to define an elasticsearch account to
use for remote logging and centralized logging functions

Change-Id: Iec61a130db6d94218893d3544e5a82c8ca04055b
2020-05-14 13:11:05 +00:00
Tin Lam
3dd6d0e7a0 chore(images): update to stein bionic images
Some infra charts still have old ocata xenial images as default. This
should bring them up to date with the OSH charts.

Change-Id: If8454b6d0fe52387bf6327501ee4ff87f56e87b8
Signed-off-by: Tin Lam <tin@irrational.io>
2020-05-14 07:21:41 -05:00
Steven Fitzpatrick
570024013a Fluentd: Attach uuid to fluentd config etc
By tying the fluent condfiguration to the release, it will be re-rendered
if the release is upgraded. This is useful in combination with [0], allowing
powerful configuration updates using helm upgrade. For example

Values:

    .Values.pod.env.fluentd.vars.OUTPUT_ENABLED: true

fluent.conf:

    ...
    {{- if .Values.pod.env.fluentd.vars.OUTPUT_ENABLED }}
    <match **>
    # Output Configuration here
    </match>
    {{- end }}

To disable this output section, issue a helm upgrade command and set the
apprpriate value to false.

    helm upgrade fluentd ./fluentd --set pod.env.fluentd.vars.OUTPUT_ENABLED=false

[0] https://review.opendev.org/#/c/726880/

Change-Id: I3dce9e5c4eaf588569e8cc3e1ea3cf3bebd0c3c5
2020-05-14 07:19:43 +00:00
Zuul
a7df3d89bb Merge "Backup/restore enhancements" 2020-05-13 23:19:17 +00:00
Parsons, Cliff (cp769u)
5a2babd514 Backup/restore enhancements
This patchset introduces the framework by which all OSH-based database
systems can use to backup and restore their databases. The framework
is refactored from the Postgresql backup and restore logic. This will
prevent alot of code duplication in the backup restore scripts across
each cluster.

In the process, some improvements needed to be made:
1) Removing the need for 2 separate containers to do the backup
   and restore work to a remote gateway. This simplifies the design
   and enables a higher level of robustness.
2) Adding separate "days to keep" config value for remote backup files,
   as there may be different requirements for the remote files than the
   local backup files.
3) Adding capability to send Storage_Policy when creating the remote
   RGW swift container.
4) Making coding style improvement for readability and maintainability.
5) Fixing a deployment bug that occurs when remote backup is disabled.

Change-Id: I3a3482ad67320e89f04305b17da79abf7ad6eb45
2020-05-13 16:34:21 +00:00
Steven Fitzpatrick
23191ef5a3 Elasticsearch Secret Vars
This change adds the HTK Secret Environment Variables function
to Elasticsearch. This may be required to store auth or certificate
details needed to establish remote clustering.

Change-Id: I3f1167f1c015101f768ad223024ce2490d355d83
2020-05-13 11:31:46 -05:00
Tin Lam
e97ee512c4 fix(tpl): correct a rendering issue
The mariadb statefulset template, while renders properly in helm2, does
not render in helm3. An extra "-" gobbles up a needed newline causing an
error when you run "helm template mariadb".

Change-Id: Idc1bee8e94c209a485ff2453ba2531dcddb63fc8
Signed-off-by: Tin Lam <tin@irrational.io>
2020-05-13 07:25:33 -05:00
Chinasubbareddy Mallavarapu
c14d8c6514 [CEPH-OSD] Move to 'OnDelete' upgrade strategy for ceph-osd daemonsets
This is to move to onDelete upgrade strategy  for ceph-osd daemonsets
so that osd upgrade cane be performed  by failure domains as current
upgrade strategy(RollingUpdate) will randomly pick the osd pods for upgrade.

This will be more helpful when we have rack based failure domains
on the ceph clusters.

This ps will add a new job called post-apply to restart
the osd pods rack by rack

- post-apply job will make sure osds gets restart rack by rack which
  will save upgrade time.
- its less/no distruptive since we are upgrading per failure domain.

also this job will be enabled only when we have OnDelete upgrade strategy in values.

Change-Id: I2e977e75616e08fee780f714bbd267743c42c74d
2020-05-12 20:03:40 +00:00
Zuul
e53d28718d Merge "Remove OSH Authors copyright" 2020-05-12 20:00:38 +00:00
Steve Taylor
b1658e7033 [ceph-osd] Helper function clean-up
Fix a bug in get_lvm_tag_from_volume when no logical volume is
provided and remove unnecessary echo commands from helper
functions.

Change-Id: I8e89d1f8e5a3c7c8148a7cd46ef3fcdf4f6f82bd
2020-05-11 21:36:25 +00:00
Zuul
8f05761ee5 Merge "Enable Apparmor to openvswitch init pods" 2020-05-11 20:11:52 +00:00
Zuul
b1526dadc1 Merge "Fluentd: Render Config as Template" 2020-05-11 18:22:28 +00:00
Steven Fitzpatrick
2039805314 Fluentd: Render Config as Template
This change updates the fluentd configmap-etc to render
.Values.conf.fluentd.template as a template, allowing for greater
flexibility in configuration.

Change-Id: I8809767c679c377e319ecc53960c55ae18e1b558
2020-05-11 10:13:44 -05:00
diwakar thyagaraj
d2e48fc131 Enable Apparmor to openvswitch init pods
Change-Id: Ib71f7e4a2ea21efaa648ddf13a8ee3378609deb2
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-11 14:27:57 +00:00
diwakar thyagaraj
53b5fda1c6 Enable Apparmor to Kibana Completed pods
Change-Id: Idf408846f6a6f4350ce5c78247338cfebb280e38
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-08 23:07:52 +00:00
Zuul
f8c5d06769 Merge "Settings for Remote Elasticsearch Clusters" 2020-05-08 22:22:06 +00:00
Steven Fitzpatrick
95e3c21df4 Settings for Remote Elasticsearch Clusters
This change adds a new Deployment to the Elasticsearch chart to add a
set of "gateway" nodes to the Elasticsearch cluster. These nodes will
facilitate Elasticsearch remote cluster, for features such as cross
cluster search.

Co-Authored-By: David Smith <ds3330@att.com>
Change-Id: Ic4ac988a922a12addce3c65e0ef4099d46bbc784
2020-05-08 13:07:54 -05:00
diwakar thyagaraj
64ac469eb6 Enable Apparmor to Prometheus-init-containers
Change-Id: Ibea27338437c9c039b10bff02a28d60d3f5cf4b1
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-08 17:24:54 +00:00
KHIYANI, RAHUL (rk0850)
8617c8c1e0 Ingress: Add apparmor profile to ingress init container
Change-Id: I2217a8ab8c76b8f6a14f477c3159e4133ef186f9
2020-05-07 13:51:29 +00:00
Gage Hugo
d14d826b26 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
2020-05-07 02:11:15 +00:00
Zuul
eabe320a99 Merge "Enable Docker default Apparmor for all Prometheus init Containers" 2020-05-06 22:32:34 +00:00
Zuul
4e700828b7 Merge "[FIX] Add whitelisting of fields of various types to configMap." 2020-05-06 19:10:39 +00:00
diwakar thyagaraj
ebfcec03e2 Enable Docker default Apparmor for all Prometheus init Containers
Change-Id: I036882f7e443d3494e3fb38b2d5ded4bfa11a9b1
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-06 17:18:16 +00:00
Zuul
9cab29ba0c Merge "Enable Apparmor to postgresql init containers" 2020-05-06 15:33:52 +00:00
diwakar thyagaraj
7c5479fb83 Enable Apparmor to postgresql init containers
Change-Id: If679428710dbb8c9c8a5da4248c48e05a2fb0844
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-06 01:55:12 +00:00
Zhipeng Liu
d82325edf7 Fix ipv6 address cannot be parsed in mariadb-ingress pod
There is a bug in nginx.tmpl that it will not enclose ipv6
addresses in square brackets resulting in them being unable to
be parsed.

Test pass on both ipv4 and ipv6 simplex setup for StarlingX
project

Change-Id: I16e586f5d8e3dfcb5e94f0486409c9637ba197b2
Signed-off-by: Zhipeng Liu <zhipengs.liu@intel.com>
2020-05-06 01:53:36 +00:00
diwakar thyagaraj
aaeb0b1abb Enable Apparmor to Grafana Completed pods
This also adds init containers.

Change-Id: Ia70db208a1583b9a44a32d9a3d485ca7dc8a3ce2
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-05 15:59:22 +00:00
Zuul
79043176a1 Merge "Memcached: Add apparmor profile to memcached init containers" 2020-05-04 22:12:37 +00:00
Zuul
34f991bc19 Merge "Enabling Apparmor fo rabbimq Init conatiners" 2020-05-04 21:30:37 +00:00
Zuul
2e743df516 Merge "Set ctl socket ownership" 2020-05-04 21:24:39 +00:00
Dodda, Prateek
da58791418 Enabling Apparmor fo rabbimq Init conatiners
Change-Id: I7b8306288503d066113a2bf410ef4a77a64c553c
2020-05-04 17:10:25 +00:00
KHIYANI, RAHUL (rk0850)
19fe6ca7ab Memcached: Add apparmor profile to memcached init containers
Change-Id: I194e24da1bc5813179b582ad89bc9c41213fe4d0
2020-05-04 16:37:58 +00:00
Steve Taylor
a12ea0244e [ceph-osd] Use lvm commands instead of ceph-volume to get OSD properties
This change removes "ceph-volume inventory" and "ceph-volume lvm
list" commands from the ceph-volume OSD initialization script and
Bluestore start script and replaces them with "pvdisplay" and
"lvs" to retrieve lvm tags directly from lvm volumes instead.
Ceph-volume makes repeated calls to blkid, which is very slow in
some cases and deadlocks in others when there are RBDs mapped on
the host.

Change-Id: Ia999770d4a59729e38dbb494b34c30e5a1b36a8b
2020-04-30 16:21:13 +00:00
Kabanov, Dmitrii
8dbd488605 [Ceph-OSD] Update getting and releasing of lock
The PS moves the relese of lock out from the if-else statement.
Otherwise we are not releasing the lock in two of three cases.
And the lock will be released only if we are going though
the last "else" statement. The PS supposed to fix that.
Also it increases the timeout value.

Change-Id: I284cff06a34ddb864484d2a63780b4fdb9075a0a
2020-04-29 18:19:19 +00:00
Zuul
bce44bb981 Merge "Add Project Specific Contributor and PTL Docs" 2020-04-29 17:14:43 +00:00
diwakar thyagaraj
ccaa11b649 Enable Apparmor to ceph client bootstrap Pods
Change-Id: Ia8fd1e50a2478743f0ff625ffdd8801610f05ee1
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-29 14:43:50 +00:00
KHIYANI, RAHUL (rk0850)
5bc24e78a4 Ingress: Run nginx ingress as non-root user
Running nginx ingress with www-data user

Change-Id: I769577cdedd05cc1b8b035928e67ad7ed70568db
2020-04-27 16:12:21 -05:00
rajesh.kudaka
f2b8bacb26 Set ctl socket ownership
This is moved from neutron-ovs-agent init script[0] to
openvswitch poststart to ensure that ovs ctl socket
has the required permissions even if the openvswitch
pods are restarted.

[0] https://github.com/openstack/openstack-helm/blob/master/neutron/templates/bin/_neutron-openvswitch-agent-init.sh.tpl#L22-L25

Change-Id: I09e604576e2408a8dec9905d1ad070422351ed99
2020-04-27 15:06:01 +00:00
Roman Gorshunov
e4096d84e7 Add Project Specific Contributor and PTL Docs
This patch amends contributor guidelines for the Ussuri goals.

Change-Id: Idc88bf4e0f6e41c2ac1b89c700314cd9038982d4
Task: 38545
Story: 2007236
2020-04-24 20:59:44 +02:00
Zuul
bbfa550361 Merge "Enable Apparmor to ceph-bootstrap Pods" 2020-04-24 17:28:51 +00:00
Cliff Parsons
41342cdc4a Fix MariaDB Backup Problems
This patch fixes 2 problems with MariaDB backup:
1) If a user with grants to a database has a hyphenated name, the backup
script errors out and the grants for this user won't be saved in the backup.
2) While restoring databases from a backup, if connections are allowed
during the restore operation, there is potential for deadlock. Table
level locks are added to the backup sql file in order to try to prevent
these deadlock situations.

Change-Id: If612e7b9f3f4d75fc67018eea17609f07a0c0b0f
2020-04-24 14:03:59 +00:00
diwakar thyagaraj
fb0cd00a56 Enable Apparmor to ceph-bootstrap Pods
Change-Id: Ifa8d43a2a68fffaea554f04a5df63fb6b7ea5422
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-24 02:22:17 +00:00
Zuul
51bff9687e Merge "Postgresql backup/restore enhancements" 2020-04-23 17:16:37 +00:00
Cliff Parsons
382d113a87 Postgresql backup/restore enhancements
1) Added a new backup container for accessing RGW via Openstack Swift API.
2) Modified the backup script so that tarballed databases can be sent to the RGW.
3) Added new script to send the database backup to the RGW.
4) Modified the restore script so that databases can be retrieved from the RGW.
5) Added new script to retrieve the database backups from the RGW.

Change-Id: Id17a8fcb63f5614ea038c58acdc256fb4e05f434
2020-04-22 22:31:48 +00:00
Chinasubbareddy Mallavarapu
8d9b7fdd2b [ceph-mon-check] fix the command to connect correct ceph cluster
This is to fix the command to connect to the cluster name instead of
namesapce.

Change-Id: I8b8f7c10d7667245a8f6cb02fb5b69dd122099e5
2020-04-22 20:02:14 +00:00
Taylor, Stephen (st053q)
9057c770a6 Enable cephfs tests
Cephfs tests were disabled in order to merge
https://review.opendev.org/695568 due to gate failures that were
blocking it. CephFS isn't used in openstack-helm-infra, so it
wasn't required for that work. This change re-enables the cephfs
tests so we can work through any issues that are causing further
failures.

Since the the issue got fixed in 14.2.8 , upgrading all daemons to 14.2.8.
(https://tracker.ceph.com/issues/43770)

Change-Id: I376d39b7ee00ccb1ab8046b58f92b19a822272e1
2020-04-21 22:25:52 +00:00
Gage Hugo
fa0876750a Fixing docs build for osh-infra
There was a change[0] in zuul that made the docs job fail
if the build/html dir was empty. Our docs job was putting the
built docs in the source file by mistake, this ps fixes
this issue.

[0] https://review.opendev.org/#/c/721221/

Change-Id: I9ef336cf32dad96ff4a4fcc2743363a8f40f4718
2020-04-21 20:05:39 +00:00