2651 Commits

Author SHA1 Message Date
Tin Lam
7cb3ef69ae feat(tls): add tls support to helm-toolkit
This patch set:

- allows options in the bootstrap job to load the proper TLS secret into
  the  proper envvar so the openstack client can connect properly to
  perform bootstrap;
- adds in certificates to make rally work properly with TLS endpoints;
- adds methods to handle TLS secret volume and volumeMount;
- updates ingress to handle secure backends.

Change-Id: I322cda393f18bfeed0b9f8b1827d101f60d6bdeb
Signed-off-by: Tin Lam <tin@irrational.io>
2020-06-26 00:32:57 +00:00
Chris Wedgwood
6d032c3971 [rabbitmq] Upgrade to 3.7.26
Staying current.  Many bugfixes.

Change-Id: Ib95c30380d89c336774d5c74e02ce5cbd9efb5d7
2020-06-25 23:32:50 +00:00
Zuul
5e316a9ba0 Merge "Mariadb backup/restore enhancements" 2020-06-25 18:48:08 +00:00
Zuul
e48feaefb2 Merge "[ceph-rgw] Add rwg restart job" 2020-06-25 17:17:26 +00:00
Zuul
b4c66cea6a Merge "Fix problems with DB utilities in HTK and Postgresql" 2020-06-25 16:17:17 +00:00
Alexander Vlasov
70b0b9b266 [ceph-rgw] Add rwg restart job
Some updates to rgw config like zone or zonegroup changes that can
be done during bootstrap process require rgw restart.
Add restart job which when enabled will use
'kubectl rollout restart deployment'
in order to restart rgw

This will be more useful in greenfield scenarios where
we need to setup zone/zonegroups right after rgw svc up which
needs to restart rgw svc.

Change-Id: I6667237e92a8b87a06d2a59c65210c482f3b7302
2020-06-25 13:15:56 +00:00
Zuul
9655817eae Merge "Remove duplicate lint job entry and script" 2020-06-25 04:11:51 +00:00
Huang, Sophie (sh879n)
573ac49939 Mariadb backup/restore enhancements
Below enhancements are made to Mariadb backup:
1) Used new helm-toolkit function to send/retrieve Mariadb
   backups to/from RGW via OpenStack Swift API.
2) Modified the backup script such that the database backup
   tarball can be sent to RGW.
3) Added a keystone user for RGW access.
4) Added a secret for OpenStack Swift API access.
5) Changed the cronjob image and runAsUser
6) Modified the restore script so that archives stored remotely
   on RGW can be used for the restore data source.
7) Added functions to the restore script to retrieve data
   from an archive for tables, table rows and table schema of a databse
8) Added a secret containing all the backup/restore related
   configuration needed for invoking the backup/restore operation
   from a different application or namespace.

Change-Id: Iadb9438fe419cded374897b43337039609077e61
2020-06-24 21:13:21 +00:00
Cliff Parsons
1da7a5b0f8 Fix problems with DB utilities in HTK and Postgresql
This PS fixes:
1) Removes printing of the word "Done" after the restore/list command
   executes, which is not needed and clutters the output.
2) Fixes problem with list_tables related to command output.
3) Fixes parameter ordering problem with list_rows and list_schema
4) Adds the missing menu/parameter parsing code for list_schema
5) Fixes backup-restore secret and handling of PD_DUMPALL_OPTIONS.
6) Fixes single db restore, which wasn't dropping the database, and
   ended up adding duplicate rows.
7) Fixes cronjob deficiencies - added security context and init containers,
   fixed backup related service account related typos.
8) Fixes get_schema so that it only finds the table requested, rather
   than other tables that also start with the same substring.
9) Fixes swift endpoint issue where it sometimes returns the wrong
   endpoint, due to bad grep command.

Change-Id: I0e3ab81732db031cb6e162b622efaf77bbc7ec25
2020-06-24 19:16:04 +00:00
Singh, Jasvinder (js581j)
fd8cdb66af Updating nagios cluster role for rbd monitoring
This patchset is required for the patch set https://review.opendev.org/#/c/737629.
The kuberntes python api requires these permissions, for this script to work properly.

Change-Id: I69f2ca40ab6068295a4cb2d85073183ca348af1e
2020-06-23 17:59:17 -04:00
Zuul
401d4e70ce Merge "Add node-problem-detector chart" 2020-06-22 23:11:27 +00:00
Steve Wilkerson
a31bb2b049 Add node-problem-detector chart
This adds a chart for the node problem detector. This chart
will help provide additional insight into the status of the
underlying infrastructure of a deployment.

Updated the chart with new yamllint checks.

Change-Id: I21a24b67b121388107b20ab38ac7703c7a33f1c1
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2020-06-22 13:00:55 -05:00
Gage Hugo
16676b5b63 Remove duplicate lint job entry and script
osh-infra currently has a duplicate linter playbook that is not
being used, since the other is used for both osh and osh-infra.
This change removes the duplicate entry and playbook.

Change-Id: If7040243a45f2166973dc5f0c8cd793431916942
2020-06-22 12:31:25 -05:00
chinasubbareddy mallavarapu
91f60d2884 Revert "[ceph-client] Update ceph-mon port."
Reverting this ps since we tried to solve the  problem here for
the old clients prior to nautilus but nautilus clients thinks
its v2 port and try to communicate with server and getting some
warnings as shown below:

lets make v2 port as default and ovverride  mon_host config for
old clients prior  to nautilus as we did in this ps
(https://review.opendev.org/#/c/711648/).

better solution will be moving out of old ceph clients by changing
the images wherever old ceph clients are installed.

log:

+ ceph auth get-or-create client.cinder mon 'profile rbd' osd
'profile rbd' -o /tmp/tmp.k9PBzKOyCq.keyring
2020-06-19 15:56:13.100 7febee088700 -1 --2-
172.29.0.139:0/2835096817 >> v2:172.29.0.141:6790/0 conn(0x7febe816b4d0
0x7febe816b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0
rx=0 tx=0)._handle_peer_banner peer v2:172.29.0.141:6790/0 is using msgr V1 protocol

This reverts commit acde91c87d5e233d1180544df919cb6603e306a9.

Change-Id: I08ef968b3e80c80b973ae4ec1f80ba1618f0e0a5
2020-06-19 22:16:16 +00:00
Gage Hugo
26350f37aa Add new python roles to playbooks
With the latest infra update, the images used no longer contain
python by default and projects are expected to use the new
ensure roles to use packages as needed.

This change adds some of the ensure roles to a few playbooks,
additional cleanup can be done using these in future changes.

Change-Id: Ie14ab297e71195d4fee070af253edf4d25ee5d27
2020-06-19 18:07:13 +00:00
Zuul
e4167fd248 Merge "[ceph-osd] Allow ceph-volume to deploy OSDs on dirty disks" 2020-06-18 22:12:26 +00:00
Tin Lam
587182c779 fix(ovs): add capability to openvswitch
While OpenVSwitch works in the gate using kubernetes 1.16, running this
in kubernetes 1.18 causes a permission denied error while executing
chroot in an init container script [0]. This adds the SYS_CHROOT
capability to address the error.

[0] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/openvswitch/templates/bin/_openvswitch-vswitchd-init-modules.sh.tpl#L18-L20

Change-Id: I62c01678cce6cd4e98418ed5518613ccd5eecbf9
Signed-off-by: Tin Lam <tin@irrational.io>
2020-06-18 17:07:40 +00:00
Zuul
eaaf0062e4 Merge "(fix) Changed pip to pip3" 2020-06-18 15:47:03 +00:00
Zuul
ee12b4c5db Merge "Don't run linter on docs changes" 2020-06-18 15:47:01 +00:00
Brian Wickersham
567a7c6c1e [ceph-osd] Allow ceph-volume to deploy OSDs on dirty disks
Currently there are conditions that can prevent Bluestore OSDs
from deploying correctly if the disk used was previously deployed
as an OSD in another Ceph cluster. This change fixes the
ceph-volume OSD init script so it can handle these situations
correctly if OSD_FORCE_REPAIR is set.

Additionally, there is a race condition that may occur which
causes logical volumes to not get tagged with all of the
necessary metadata for OSDs to function. This change fixes
that issue as well.

Change-Id: I869ba97d2224081c99ed1728b1aaa1b893d47c87
2020-06-18 14:04:02 +00:00
Zuul
0a35fd827e Merge "Enable key-duplicates and octal-values yamllint checks" 2020-06-18 04:49:03 +00:00
Zuul
017f16274d Merge "ceph-osd: Log the script name, lineno and funcname" 2020-06-18 04:01:58 +00:00
Zuul
7935018d8f Merge "Don't rely on pip and tox installed on zuul node" 2020-06-18 03:31:44 +00:00
Zuul
6217a5eda3 Merge "[ceph-osd, ceph-client] Weight OSDs as they are added" 2020-06-18 02:22:53 +00:00
Gage Hugo
16ff2531e4 Don't rely on pip and tox installed on zuul node
Change-Id: I3b715a4cc5ae064b458694ab98feb2b6cc226e65
2020-06-18 01:00:31 +00:00
Zuul
16414767e0 Merge "Enable yamllint rules for templates" 2020-06-18 00:09:28 +00:00
Gage Hugo
6b5d1a1d4a Don't run linter on docs changes
This change modifies the linting job to not run when a patchset
only modifies openstack-helm documentation.

Change-Id: I0ed0fd5fff10d81dd34351b7da930d1a340b10d8
2020-06-17 18:06:34 -05:00
Stephen Taylor
59b825ae48 [ceph-osd, ceph-client] Weight OSDs as they are added
Currently OSDs are added by the ceph-osd chart with zero weight
and they get reweighted to proper weights in the ceph-client chart
after all OSDs have been deployed. This causes a problem when a
deployment is partially completed and additional OSDs are added
later. In this case the ceph-client chart has already run and the
new OSDs don't ever get weighted correctly. This change weights
OSDs properly as they are deployed instead. As noted in the
script, the noin flag may be set during the deployment to prevent
rebalancing as OSDs are added if necessary.

Added the ability to set and unset Ceph cluster flags in the
ceph-client chart.

Change-Id: Ic9a3d8d5625af49b093976a855dd66e5705d2c29
2020-06-17 21:49:39 +00:00
Zuul
26ee617d77 Merge "[Ceph-OSD] Add comparision of releases of Daemonsets" 2020-06-17 21:10:05 +00:00
Ahmad Mahmoudi
b7d3311649 (fix) Changed pip to pip3
Changed pip to pip3 to address zuul gate issues.

Change-Id: Id265b405b293af5b51b4774ca4c7465c9e9457b6
2020-06-17 18:15:07 +00:00
Andrii Ostapenko
83e27e600c Enable key-duplicates and octal-values yamllint checks
With corresponding code changes.

Change-Id: I11cde8971b3effbb6eb2b69a7d31ecf12140434e
2020-06-17 13:14:30 -05:00
Andrii Ostapenko
dfb32ccf60 Enable yamllint rules for templates
- braces
- brackets
- colons
- commas
- comments
- comments-indentation
- document-start
- hyphens
- indentation

With corresponding code changes.

Also idempotency fix for lint script.

Change-Id: Ibe5281cbb4ad7970e92f3d1f921abb1efc89dc3b
2020-06-17 13:13:53 -05:00
KHIYANI, RAHUL (rk0850)
ddfa7a4741 Ceph-mon: Add pod/container security context
This updates the ceph-mon chart to include the pod
security context on the pod template

This also adds the container security context to set
readOnlyRootFilesystem flag to true

Change-Id: I4c9e292eaf3d76ee80f50553d1cbc8cdc6f57cac
2020-06-16 23:59:34 +00:00
Gage Hugo
548356e86c Update keyword to become instead of sudo
Zuul updated ansible to 2.9 and broke one of the playbooks
that had the old sudo keyword, which is no longer valid in 2.9.

This change updates the offending file to use "become" instead,
which is the valid keyword instead of sudo.

Change-Id: I2057de7470d65a60c4c984cb99d0715c9d43b3a8
2020-06-16 21:52:00 +00:00
Zuul
c8a5edb218 Merge "Ingress: add apparmor profile to ingress init containers" 2020-06-15 23:22:27 +00:00
Phil Sphicas
67d762eef3 ceph-osd: Log the script name, lineno and funcname
For the scripts in the ceph-osd daemonset that source common.sh (i.e.
those that run in the osd-init and ceph-osd-default containers), updates
the PS4 prompt so that the script name, line number, and function are
included in xtrace output.

Change-Id: Ieebbb82b64db4cf363ed4396289c823744d4a860
2020-06-15 23:14:44 +00:00
Zuul
1065b48025 Merge "Introduces templates linting" 2020-06-15 19:37:06 +00:00
Kabanov, Dmitrii
2a9aa76ae9 [Ceph-OSD] Add comparision of releases of Daemonsets
The PS adds comparison of releases of Daemonsets. If there is
more than one release we need run post-apply job.

Change-Id: If0c55aba4e6450815972586701f0611505d41af5
2020-06-14 21:15:14 -07:00
KHIYANI, RAHUL (rk0850)
ae278c674e Ingress: add apparmor profile to ingress init containers
concat is not available in helm 14.1, using work around
to emulate this functionality.

Change-Id: Iccbc5de567be1c899b7f5d47cf43b6e962c27e91
2020-06-12 21:02:32 +00:00
KHIYANI, RAHUL (rk0850)
d88b553727 Adding python3-pip package to installation script
python3-pip package is required to support openstack-client

Change-Id: I285a32a766dacf6c0e956216144ca1fcf300da77
2020-06-12 11:23:35 -05:00
Andrii Ostapenko
8f24a74bc7 Introduces templates linting
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.

Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.

Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
2020-06-11 23:29:42 -05:00
Taylor, Stephen (st053q)
88f7d5f42c [ceph-osd] Optimize symlink creation in udev_settle
This change optimizes the creation of symlinks in udev_settle by
only looking at the disks related to the OSD being
deployed/started and skipping the ln command for existing symlinks.
A second "udevadm settle" command is also added after the creation
of the symlinks in order to allow any logical volumes related to
new symlinks to become established.

Change-Id: I3283021fd80c8a05f0aa0c9917bb7ba0ea144303
2020-06-11 14:36:06 +00:00
Zuul
9e7c3a607a Merge "ceph-osd: Simplify failure domain config using map" 2020-06-10 23:33:54 +00:00
Zuul
fc4eba11b6 Merge "[ceph-client] Update ceph-mon port." 2020-06-10 21:28:52 +00:00
Gage Hugo
377286efd8 Add 32GB nodeset definition
This change defines the 32GB node option to be used in rare cases
when a particular check requires more resources than a normal
run can provide.

Change-Id: I9ff79f98a0f1874411c0df19cb07b5473d82992a
2020-06-09 11:01:40 -05:00
Zuul
145c1df9df Merge "Set OS_ENDPOINT_TYPE in keystone openrc vars" 2020-06-08 22:47:53 +00:00
Sphicas, Phil (ps3910)
577dcd5d19 ceph-osd: Simplify failure domain config using map
Using a subset of the characters in the hostname to determine the
failure domain is not always possible, and using overrides based on
hostnames is in some ways overkill.

This change provides a simple way to map hostnames to failure domains.
It is used only when 'failure_domain' is set other than 'host', and when
'failure_domain_by_hostname' is 'false'. Any hosts not referenced in the
map will be given the default treatment (root=default host=hostname)

Example usage:

    conf:
      storage:
        failure_domain: rack
        failure_domain_by_hostname_map:
          hostfoo: rack1
          hostbar: rack1
          hostbaz: rack2
          hostqux: rack2

Change-Id: Ia98fec8c623486f80054877e40e0753e4b939e8e
2020-06-08 19:45:00 +00:00
Zuul
01705232be Merge "mariadb: use utf8_general_ci collation as default" 2020-06-06 05:17:01 +00:00
Zuul
c382b37f82 Merge "Add backup/restore configuration secret" 2020-06-05 14:23:10 +00:00
Zuul
75dfc47176 Merge "Remove retired congress" 2020-06-05 04:29:20 +00:00