1198 Commits

Author SHA1 Message Date
Pete Birley
939923db64 Rabbit: Be more verbose in test logs
This PS increases the feedback give by the rabbitmq test pod

Change-Id: If8aa713017eccaf100c6186cd569a6a0f4b021e9
Signed-off-by: Pete Birley <pete@port.direct>
2019-02-16 01:33:48 +00:00
Pete Birley
69cf377fd6 RabbitMQ: Improve robustness of readyness checks
This PS moves the readyness check to simply checking if the ampq
port is open, both simplifying it and also correctly indicating if
the process is ready to serve requests.

Change-Id: I38416c8bf3b242fa344875da13f81e5bbc1983c7
Signed-off-by: Pete Birley <pete@port.direct>
2019-02-15 18:17:55 -06:00
Steve Wilkerson
8e8193f9e1 Update kubeadm kubernetes version to 1.12.2
This updates the kubernetes version used when deploying via
kubeadm to v1.12.2, which matches what is deployed via minikube
for the single node jobs

This required updating the apiVersion in the kubeadm configuration
file template, as well as removing the --cadvisor-port flag from
the kubelet args, as this has been removed entirely

Change-Id: I26573de35529ce44e91e6d4d4530f608b8cee476
2019-02-15 10:42:57 -06:00
Steve Wilkerson
ef3adc4d0e Execute helm tests in osh-infra single node jobs
This executes the helm tests for the single node jobs in
openstack-helm-infra for rabbitmq, elasticsearch, fluent-logging,
prometheus, and grafana.

Change-Id: I0109cfbe6adeb9e24d513c8313d964323634b8da
2019-02-11 14:16:06 +00:00
Pete Birley
d3c3c35365 Libvirt: Move ceph config to remain on host
Change-Id: If525389191b82dc7fb10cd2941bbe48fb0bbdb3c
Signed-off-by: Pete Birley <pete@port.direct>
2019-02-08 16:20:49 -06:00
Zuul
045e64067b Merge "Ceph-RGW: Support rotation of s3 key pairs" 2019-02-07 20:32:26 +00:00
Zuul
b4ec10151c Merge "[CEPH] Expand the Keystone Helm tests for RGW" 2019-02-07 19:11:42 +00:00
Zuul
6620f08e49 Merge "fix type error to streamline single-replica mariadb startup" 2019-02-07 17:21:07 +00:00
Steve Wilkerson
cf0ed142f6 Ceph-RGW: Support rotation of s3 key pairs
This updates the helm-toolkit script for creating rgw s3 users
to first check if a user exists, then create the user if it does
not exist or modify the user's keys if it does exist. This is
accomplished by using jq to identify all existing access keys for
the specified user, removing those key pairs using the access key,
then modifies the existing user with the supplied access/secret
key pair for the given user

This also updates the ceph-rgw chart to use the helm-toolkit s3
user script for creating the admin s3 user instead of using a
similar script defined directly in the ceph-rgw chart

Change-Id: I575b66415d44db7bb752102e45595305d86e623b
2019-02-07 10:33:49 -06:00
Matthew Heler
e8ed3bcf76 [CEPH] Expand the Keystone Helm tests for RGW
- Since the admin key has been removed, we need to also replace
radosrgw-admin with openstack container commands.

- Additionally expand the helm tests for keystone to also upload
and validate an object in RGW (similiar to S3 helm tests).

Change-Id: I4be603121fc227dd48f83704e99bba94341c4c09
2019-02-06 20:43:20 -06:00
Chris Friesen
d99b1412e2 fix type error to streamline single-replica mariadb startup
The mariadb startup script was trying to optimize the single-replica
case but missed the fact that the variable it was checking was a
string rather than an int.

Converting it to an int before doing the comparison makes it work
as expected.

Change-Id: I8612e9e8ef5ec8ff61ecf0782f262a5feafd501a
Signed-off-by: Chris Friesen <chris.friesen@windriver.com>
2019-02-06 17:34:27 -06:00
Steve Wilkerson
f4aa5dc574 Ceph-RGW: Update storage-init job application labels
This changes the application label for the ceph-rgw storage init
job to 'ceph' to match the other jobs defined for the chart, rather
than use 'ceph-rgw'

Change-Id: Ia0b679567161e91241250f0c250d24a45c5ebb92
2019-02-06 14:43:24 -06:00
Pete Birley
87de515727 Ceph-OSD: Restore bootstrap job
At some point the bootrap job was partially removed from the ceph
osd chart, this PS resores it.

Change-Id: I2f51deda64b299fe980e1f191b860bfe173e6aca
Signed-off-by: Pete Birley <pete@port.direct>
2019-02-05 21:54:06 -06:00
Steve Wilkerson
cb021c2124 Add Armada job for testing update of chart passwords
This adds both a periodic and experimental job for deploying Ceph
and the LMA components via Armada. This job will then generate new
passphrases for the LMA components, render an updated manifest for
the LMA components including the new passphrases, then applies the
updated LMA manifest to validate the ability for all deployed
charts to update those passphrases successfully

Change-Id: I966ebeadd3823a087239aa7d198444a084e5d242
2019-02-04 22:44:52 +00:00
Zuul
ada418f756 Merge "Add Armada job for testing update of release uuids" 2019-02-04 22:42:25 +00:00
Zuul
2aceca1e29 Merge "[CEPH] Extend RadosGW S3 configuration support" 2019-02-04 22:27:13 +00:00
Steve Wilkerson
8361b74926 Add Armada job for testing update of release uuids
This adds both a periodic and experimental job for deploying Ceph
and the LMA components via Armada. This job will then generate a
new release uuid, render an updated manifest for all previously
deployed releases, then apply that manifest to validate the
ability for all deployed charts to update successfully with the
new release uuid annotation

Change-Id: Ic1eed1bd949279f4630fb3964fbb03788536213c
2019-02-04 21:14:02 +00:00
Steve Wilkerson
12529ddf0a Nagios: Update default workers back to 4
This moves the default workers defined for Nagios back to 4, as
reducing the number to one has made Nagios deployments in our jobs
inconsistent

Change-Id: I288554c4f9d57714b3d70a241942b2e4fd334500
2019-02-04 14:21:15 -06:00
Matthew Heler
ff18dbe5fd [CEPH] Extend RadosGW S3 configuration support
- Support using custom client params for S3 configurations
- Move common tuning for S3 and Keystone into there own
configuration option
- Cleanup the rgw helm tests, since copying the ceph admin key is
no longer required
- Cleanup duplicate portions of the code for configuring the RGW
backend and frontend port
- Add an rgw helm test check for the osh-infra-logging gates

Change-Id: I46dbb4c45b0b96f5cf555077e49d2e09a1171424
2019-02-04 14:08:09 -06:00
Zuul
eceacafbba Merge "Add Armada deployment job to openstack-helm-infra" 2019-02-04 19:21:48 +00:00
Zuul
340f4bdbe1 Merge "Mariadb: Use correct credentials for exporter in secret" 2019-02-04 16:53:51 +00:00
Steve Wilkerson
9422e970a7 Add Armada deployment job to openstack-helm-infra
This adds both a periodic and experimental job for deplying Ceph
and the LMA components via Armada to openstack-helm-infra

Change-Id: Ia3b557801d4f4b667d82eb47a6ef1825394ee526
2019-02-04 14:07:07 +00:00
Zuul
d1c14976ba Merge "Disable nonfunctional experimental check jobs" 2019-02-04 13:11:15 +00:00
Steve Wilkerson
6e2ea01ae0 Mariadb: Use correct credentials for exporter in secret
This updates the mariadb chart to use the correct auth values for
the mariadb prometheus exporter. The correct credentials to use
are the credentials in the oslo_db endpoint

Change-Id: I2d325167d7ffdf911a56fe97b879cb13b0d4c195
2019-02-04 06:23:33 -06:00
Zuul
d3531f1fba Merge "[CEPH] Enable iostat mgr module" 2019-02-04 09:45:13 +00:00
Steve Wilkerson
fc6ac671fb Disable nonfunctional experimental check jobs
This disables the centos and fedora experimental checks, as they
need work to resolve issues before they can begin to add value

Change-Id: If523a9163426dc4bc78b5e19a000d0d677cb20e4
2019-02-04 09:02:36 +00:00
Pete Birley
3e012063e7 HTK: Simplify s3 bucket creation job and support locked down acl
This PS simplifies the s3 bucket creation job and updates it to
work with newer versions of s3cli

Change-Id: Idd1f3baf8356329db8a6ede54d53f7599b56f00b
Signed-off-by: Pete Birley <pete@port.direct>
2019-02-03 22:15:27 -06:00
Zuul
e0a858b8c2 Merge "Ceph: Update default to use OSH image" 2019-02-04 02:45:41 +00:00
Zuul
a467aa4039 Merge "Nagios: Reduce Nagios concurrent checks and workers" 2019-02-03 05:53:28 +00:00
Zuul
d77772a147 Merge "Postgresql: Remove unused exporter user configuration" 2019-02-03 05:49:43 +00:00
Matthew Heler
5de299b9b1 [CEPH] Enable iostat mgr module
Enable the iostat mgr module for Ceph. This module show the
current throughput and IOPS done on a Ceph cluster.

Change-Id: I2fe5b47401c15e349a49f345bacd99da39889373
2019-02-02 16:44:58 -06:00
Steve Wilkerson
8cc89fb63f Postgresql: Remove unused exporter user configuration
This removes the auth credentials from the postgresql exporter's
endpoint configuration, as the secret and job for creating that
user come from the auth credentials defined in the postgresql
endpoint instead

Change-Id: Id17578b8e22a1808d7c3323bda9cd005c2584c97
2019-02-02 11:08:02 -06:00
Steve Wilkerson
4d170bea32 MariaDB: Remove unused exporter password credentials
This removes auth credentials from the mariadb exporter endpoint,
as the credentials used for the actual user are consumed from
the oslo_db endpoint instead.

Change-Id: I1419f50b7e0059763588028b55c982ad83446a33
2019-02-02 10:55:00 -06:00
Zuul
1c33bc8719 Merge "RGW: Fix helm-test pod template" 2019-02-01 23:13:44 +00:00
Zuul
82007140fc Merge "Monitoring job: Remove nagios file mount check" 2019-02-01 22:41:46 +00:00
Zuul
6116245e83 Merge "[MariaDB] Fix non-default setting for admin user" 2019-02-01 21:36:15 +00:00
Pete Birley
6ea6a85198 Ceph: Update default to use OSH image
This PS udpates the default image in the chart to the latest OSH image.

Change-Id: Ib8d2a72ad48049fe02560dc4405f0088890b6f64
Signed-off-by: Pete Birley <pete@port.direct>
2019-02-01 21:25:13 +00:00
Pete Birley
0d55b9064c RGW: Fix helm-test pod template
This PS updates the helm test driven pod template:
 * places rgw keystone conditional to correct location
 * removes unrequired roles and bindings
 * adds dependency on the rgw being running
 * corrects spelling error
 * corrects s3cmd to work with version 1.6.1

Change-Id: I665dba9fdca1d840f4d864e32f07b6185af51d25
Signed-off-by: Pete Birley <pete@port.direct>
2019-02-01 21:21:10 +00:00
Steve Wilkerson
44b5c008f1 Monitoring job: Remove nagios file mount check
This removes the elasticsearch query clause json file check from
the single node monitoring job, as it's become a bit unreliable.
Instead, we'll rely on the periodic multinode job to validate this
works as intended

Change-Id: I8d33a2625d5d666af280467dc21d76ed0302f837
2019-02-01 15:20:03 -06:00
Steve Wilkerson
3614d025dc Fluentbit: Remove database used in tail inputs
This updates the fluentbit configuration for tail inputs to remove
the values for utilizing mysqlite databases to track its location
in each file it's configured to tail.  This is intended to reduce
the pressure fluentbit exerts on the host through writing to
/var/log/foo.db. To help mitigate large amounts of traffic
sent from fluentbit to fluentd upon a pod restart, this also
adds a throttle filter to fluentbit.

As a result, Fluentbit no longer needs a writable mount to its
hostPath on /var/log on the host.  Thus, this change includes
updating the Fluentbit daemonset's mount on /var/log to be
readOnly

Change-Id: If4381f4ff47e887f3ea10beded4f6172edaf08ba
2019-02-01 16:56:31 +00:00
Steve Wilkerson
442e4985c3 Nagios: Reduce Nagios concurrent checks and workers
This reduces the maximum concurrent checks Nagios will execute to
prevent process sprawl on the host. This also reduces the number
of default workers to a single worker, to prevent Nagios from
forking off multiple processes that then execute service checks
and commands in parallel

Change-Id: I0d8445a265740b4a2491bdfd739cb0f27955f06d
2019-02-01 08:31:31 -06:00
Steve Wilkerson
25e4e5662e Update network-policy ldap deployment and test
This updates the script for deploying ldap in the network policy
job to accept ingress traffic from prometheus pods.

This also updates the network policy test to account for return
values with more than one result when checking for a pod to use,
as well as selecting pods by application and component labels
instead of simply grepping for a name (as this could cause issues
with grepping for 'fluentd', when that could return both fluentd
and fluentd-exporter pods, for example)

Change-Id: I12a4029f574ea7d5b250709adef21b07d8cf0220
2019-01-31 21:29:40 +00:00
Zuul
6ef3f58fb8 Merge "Add pre-fixes to the Selenium jobs and remove "|| true"" 2019-01-31 20:39:40 +00:00
Chris Wedgwood
b15605029e [MariaDB] Fix non-default setting for admin user
This allows the database administration username to be changed in
addition to the password (the configuration and documentation implies
this is possible, but the current implementation assumes root in a few
places).

Remove some unused env declarations.

Tested with and without Prometheus exporter enabled.

Change-Id: I8a48dac7a84fe583c7dc5ff0644ca14b1e5190d8
2019-01-31 17:29:52 +00:00
Zuul
b30012a616 Merge "[CEPH] Fixes for the OSD defrag cronjob" 2019-01-31 16:05:14 +00:00
Matthew Heler
fc76091261 [CEPH] Fixes for the OSD defrag cronjob
Fix a naming issue with the cronjob's binary, and schedule the cron
job to run every 15 minutes for the gates. Additonally check to
to ensure we are only running on block devices. Also update the
script to work with ceph-volume created devices.

Change-Id: I8aedab0ac41c191ef39a08034fff3278027d7520
2019-01-31 06:13:05 -06:00
Deokjin Kim
cbb9ec0748 Fix calling wrong variable name in gnocchi
Checking test_version seems right. test_mimic is not existing.

Change-Id: I2cbfed0f7da0b22eb753ed7bce833872a7ff707f
Signed-off-by: Deokjin Kim <deokjin81.kim@samsung.com>
2019-01-31 00:34:21 +00:00
Zuul
c3a8063fdb Merge "Fluentd: remove unused configuration section" 2019-01-30 23:30:50 +00:00
Zuul
3bd3b70e51 Merge "[Calico] Configuration robustness improvements" 2019-01-30 22:16:55 +00:00
Steve Wilkerson
f01e9d2391 Fluentd: remove unused configuration section
This removes an unused section of configuration for fluentd, as
well as cleans up the values for filtering fluentd logs

Change-Id: I0c58d3ac236af7723c64c3b9fcba877736b1f606
2019-01-30 16:03:59 -06:00