2104 Commits

Author SHA1 Message Date
Tin Lam
c199addf3c Update apiVersion
This patch set updates and tests the apiVersion for rbac.authorization.k8s.io
from v1beta1 to v1 in preparation for its removal in k8s 1.20.

Change-Id: I4e68db1f75ff72eee55ecec93bd59c68c179c627
Signed-off-by: Tin Lam <tin@irrational.io>
2020-01-09 08:59:48 +00:00
Zuul
a78ee39dcb Merge "Prometheus: Add feature-gate support in deployment scripts" 2020-01-08 05:28:40 +00:00
Zuul
f6775eaae0 Merge "Ingress: nginx-ingress-controller 0.26.1+ support" 2020-01-08 00:35:21 +00:00
Steve Wilkerson
ddd5a74319 Prometheus: Add feature-gate support in deployment scripts
This updates the deployment scripts for Prometheus to leverage the
feature gate functionality rather than bash generation of the list
of override files to use for alerting rules

Change-Id: Ie497ae930f7cc4db690a4ddc812a92e4491cde93
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2020-01-07 22:06:19 +00:00
Chinasubbareddy Mallavarapu
e2a60422df [ceph-osd] fix the issue with ceph-osd init
This is to fix the issue with ceph-osd initilization when deployed
with wal and db on same disk as pod restart always trying to prepare
the disk.

this ps will make sure to handle the case and skip the ceph-volume prepare
step in case of already deployed osd disk.

Change-Id: I5c37568f342cb4362a0de0a9c11a52b7aea3e147
2020-01-06 15:27:19 +00:00
Zuul
954a06b3fd Merge "Add bandit check to osh-infra" 2020-01-03 20:51:03 +00:00
Phil Sphicas
927c018d29 Fix ansible docker support task
Removes become: and become_user: when including another role (that
already defines become: true and become_user: root)

Fixes an error occurring in the gates:
ERROR! 'become_user' is not a valid attribute for a IncludeRole

Change-Id: I362eefbe5b09ad64e97b3b541d07db2e6b990613
2020-01-03 08:37:11 -08:00
Phil Sphicas
e7b4242c3b Ingress: nginx-ingress-controller 0.26.1+ support
nginx-ingress-controller 0.26.1 introduces configurable parameters for
streamPort and profilerPort, and changes the default for statusPort.

This change allows those parameters to be configured, while maintaining
compatibility with earlier versions of nginx-ingress.controller. It also
modifies the default status port value from 18080 to 10246.

Reference: https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0261

Change-Id: I88a7315f2ed47c31b8c2862ce1ad47b590b32137
2020-01-01 13:34:00 -08:00
Phil Sphicas
bcecbad652 Ingress: k8s and ingress version compatibility
k8s 1.14 first enabled Ingress in the networking.k8s.io/v1beta1 API
group, while still serving it in the extensions/v1beta1 API group. The
extensions/v1beta1 API endpoint is deprecated in 1.16 and scheduled for
removal in 1.20. [0]

ingress-nginx 0.25.0 actually uses the networking.k8s.io/v1beta1 API,
which requires updated RBAC rules. [1]

This change updates the ClusterRole used by the ingress service account
to grant access to Ingress resources via either the extensions/v1beta1
or networking.k8s.io/v1beta1 API, aligning with the static manifests
from the kubernetes/ingress-nginx repo [2]. It does not change the
apiVersion used when creating Ingress resources.

[0] https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/
[1] https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.25.0
[2] 870be3bcd8/deploy/static/mandatory.yaml (L50-L106)

Change-Id: I67d4dbdb3834ca4ac8ce90ec51c8d6414ce80a01
2020-01-01 13:34:00 -08:00
Zuul
e389f51865 Merge "Kafka - Implement SASL Authentication" 2019-12-31 19:02:38 +00:00
Zuul
e9d93980f3 Merge "Disable kubeadm-aio jobs while issues addressed" 2019-12-31 19:02:37 +00:00
Gage Hugo
30c66126ee Add bandit check to osh-infra
This change adds a non-voting bandit check to openstack-helm-infra
similar to what is ran in the openstack-helm repo.

This check will be made voting in a future change once the current
failures are addressed.

Similarly this check will be modified in a future change to
only be ran when affected python files are changed.

Change-Id: I177940f7b050fbe8882d298628c458bbd935ee89
2019-12-31 18:29:01 +00:00
Kabanov, Dmitrii
cd6f3442e9 [Ceph-OSD] Update exit code for flock
The PS updates the exit code for flock. Now we are using default value (1)
if timeout happened.

Change-Id: I2d1cd051c61695a12aa11af1ecb356f91b9e8279
2019-12-30 09:41:35 -08:00
Zuul
e95a8b8b09 Merge "Enable runtime apparmor for ceph-mon,ceph-mds & ceph-mgr." 2019-12-18 20:24:23 +00:00
Steve Wilkerson
803f0e8435 Disable kubeadm-aio jobs while issues addressed
This disables the keystone-auth single node job and all multinode
periodic and experimental jobs while standing issues with the
kubeadm-aio image deployment are sorted out

Change-Id: I3ce0afba155e923b6dd50f83fa6b529908b9a79b
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-18 13:56:21 -06:00
Zuul
98e0d86f8d Merge "[Elasticsearch] update ceph Nautilus images" 2019-12-18 18:27:14 +00:00
diwakar thyagaraj
841feb7e82 Enable runtime apparmor for ceph-mon,ceph-mds & ceph-mgr.
Also changed ceph apparmor gate job.
Change-Id: I92b9a467b2a77d607dd431f031ec566cc18a86a4
2019-12-18 14:52:12 +00:00
Chinasubbareddy Mallavarapu
0c18f272df [Elasticsearch] update ceph Nautilus images
This is to update ceph images  to Nautilus based images since
ceph cluster is now upgraded to Nautilus.

Change-Id: Ib57f29a4dba89de762a9824ba398ad49b0bd397b
2019-12-18 01:45:10 +00:00
Chinasubbareddy Mallavarapu
027c8497a2 [ceph-client] remove rbd pool init step
This it to remove "rbd"  pool intilization since its not
required as appliction enable in next step does the same.

Change-Id: I07ecdbe417f0156005ebf7cef8fd9e40bace3920
2019-12-18 01:44:55 +00:00
Zuul
13b2384134 Merge "[ceph-client] update pool validation logic" 2019-12-18 01:43:59 +00:00
Chinasubbareddy Mallavarapu
61419dd305 [ceph-client] force to set ceph-mgr module configs
This is to force setting the config values for all moduels since nautilus
version will not let us set them  before mgr starts.

Change-Id: I0e370b525b628fce040b33ab2e403b8b71e948cb
2019-12-17 21:46:19 +00:00
Chinasubbareddy Mallavarapu
02f63af995 [ceph-client] update pool validation logic
Starting in Nautilus, setting pgp_num step is no longer necessary
as long as pgp_num and pg_num currently match, pgp_num will automatically
track any pg_num changes.
More importantly, the adjustment of pgp_num to migrate data and (eventually)
converge to pg_num is done gradually to limit the data migration load on the system.

Change-Id: I491b6eac35b486698c0eef256ca91dac217f8929
2019-12-17 21:46:04 +00:00
Stephen Taylor
016b56e586 Ceph Nautilus compatibility
This change updates the Ceph charts to use Ceph Nautilus images
built on Ubuntu Bionic instead of Xenial. The mirror that hosts
Ceph packages only provides Nautilus packages for Bionic at
present, so this is necessary for Nautilus deployment.

There are also several configuration and scripting changes
included to provide compatibility with Ceph Nautilus. Most of
these simply allow existing logic to execute for Nautilus
deployments, but some logical changes are required to support
Nautilus as well.

NOTE: The cephfs test has been disabled because it was failing
the gate. This test has passed in multiple dev environments, and
since cephfs isn't used by any openstack-helm-infra components we
don't want this to block getting this change merged. The gate
issue will be investigated and addressed in a subsequent patch
set.

Change-Id: Id2d9d7b35d4dc66e93a0aacc9ea514e85ae13467
2019-12-17 18:47:24 +00:00
Steve Wilkerson
edd6ffd712 Reduce osh-infra-logging job scope
This updates the osh-infra-logging single node job to omit the
fluentbit deployment step, as having multiple logging daemonsets
deployed to the single node jobs is causing IO issues. Also, it
was noted that the fluentd-deployment step was missing the
overrides to move the fluentd-deployment release from utilizing a
daemonset to a deployment. This resulted in 3 logging daemons
being deployed to a single host

Change-Id: I4a0c5550e6ea6a331aab0082a975f161e65704bf
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-17 12:43:12 -06:00
Steven Fitzpatrick
472097d7eb Kafka - Implement SASL Authentication
This change implements SASL authentication in the Kafka chart.
Kafka and Exporter credentials are defined in the endpoints section,
while other credentials for producers and consumers can be defined
in the jaas section.

Additionally, a few server settings are provided to enable SASL auth,
and a jvm_options key is introduced. Any options specified here will
be set when starting Kafka, including the location of the jaas file
in this case.

Change-Id: I43469c5bb5734b62cf69be924fe9cf7078e82a9c
2019-12-16 16:55:55 -06:00
Zuul
3494fe8489 Merge "Add support to set vhost-iommu-support as global option" 2019-12-16 22:39:41 +00:00
Zuul
6bb7039368 Merge "Fix apparmor annotation of libvirt." 2019-12-16 19:22:22 +00:00
Zuul
69b9c7dd0b Merge "Fix incompatible curator version" 2019-12-16 19:14:19 +00:00
Zuul
79da084a07 Merge "[ceph-client][ceph-osd] Adjust required_percent_of_osds parameter" 2019-12-16 17:39:30 +00:00
Kaspars Skels
4fb0b49169 Fix incompatible curator version
Elasticsearch version 7.1.0 incompatible with this version of Curator (5.6.0)

Change-Id: If9323a6d742ddf2915ca9ec167eb8585c694cf1e
2019-12-13 15:20:57 -06:00
Itxaka
cace08c016
Fix openstack release in gates
Change I7def8df68371deda0b75a685363c8a73b818dd45 removed one
line by mistake passing the openstack release var down to the zuul
jobs, so all jobs are currently running under ocata.

This patch restores the missing line, thus fixing the opensuse jobs
and making sure the other jobs run under the correct release.

Change-Id: Ia7a488928e521de1afb821f141d77d2b0268ff0a
2019-12-12 10:51:19 +01:00
Roy Tang
57c2c0620c Fix apparmor annotation of libvirt.
Change-Id: I9b37bea8fe3476e4586c261156f15ca63f4658be
2019-12-11 10:44:53 -06:00
Steve Wilkerson
03580ec90a Elasticsearch: Make node selectors more granular
This updates the Elasticsearch chart to make the values keys used
for defining node selectors for the various elasticsearch
components more granular

Change-Id: Ic1ac343b1d6ee48fc7cb456afe4cd9588c4aa13b
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-10 15:55:38 -06:00
Zuul
3af64d1111 Merge "Fluentd: Add support for arbitrary secret env variables" 2019-12-09 22:37:11 +00:00
Roy Tang
9a8516867a Add support to set vhost-iommu-support as global option
Also update other optional parm

Change-Id: I9a36acd6a331255d01722ed5961b08e1fbca80d1
2019-12-09 22:19:21 +00:00
Zuul
cecd63ef62 Merge "Grafana: Remove default dashboards from chart" 2019-12-09 21:34:53 +00:00
Zuul
160370eec9 Merge "Fix feature gate envvar overriding" 2019-12-09 21:25:08 +00:00
Steve Wilkerson
005ece16d4 Fluentd: Add support for arbitrary secret env variables
This adds a helm-toolkit util for consuming arbitrary secret env
variables via pod env variables. It also updates the Fluentd chart
to add a release secret that is used to house the secret env
variables defined in the chart's values.yaml. This can be used as
an example to expand to other charts where this functionality is
desired

Change-Id: I9ef606840af92e54b2204e637c58442085e2c748
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-09 14:15:40 -06:00
Steve Wilkerson
a9652653fb Update Armada manifests for osh-infra
This updates the Armada manifests to remove the explicit enabling
of helm tests (as the default armada behavior is now to test by
default) and updates the ceph-osd chart document to set the native
helm wait behavior to false (required for the update-uuid job to
complete successfully)

Change-Id: Ia84f20ce0f38be5f07dedce70b3fbe424a037ba2
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-09 13:50:12 +00:00
Steve Wilkerson
3a6df3b544 Grafana: Remove default dashboards from chart
This removes the default dashboards from the Grafana chart and
instead places them in the values_overrides directory, similar to
what was done for the Prometheus rules. As Grafana dashboards
will likely be heavily dependent upon end-user needs, the old
default dashboard configs should only be used as a reference
instead of opinionated defaults that are difficult to override.
The previous defaults made using specialized labels for dashboard
variables difficult, as they were making dangerous assumptions
about deployed namespaces and host fqdns. By removing the defaults
entirely, end users can define their own dashboards to meet their
specialized needs

Change-Id: I7def8df68371deda0b75a685363c8a73b818dd45
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-09 13:39:13 +00:00
Zuul
385803eef2 Merge "Fix OVS-DPDK readiness probe for OVS < v2.10.0" 2019-12-08 20:54:19 +00:00
Zuul
9b77752723 Merge "Initialize PCI module if used before DPDK for hotplug to work" 2019-12-08 10:28:52 +00:00
Tin Lam
ac18e6acf9 Fix feature gate envvar overriding
Currently using envsubst to perform substitution of value overrides in
the feature gate caused conflicts as gotpl gets templated into those
overrides. This adds in '%%%REPLACE_${var}%%%' and uses sed to perform
the substitution instead to address the issue.

Change-Id: I9d3d630b53a2f3d828866229a5072bb04440ae15
Signed-off-by: Tin Lam <tin@irrational.io>
2019-12-07 12:22:16 -06:00
Pete Birley
eb5ae14dcd HTK: Omit port from uri for http/https when 80/443 used
This PS updates htk to omit the port used in the url when this
corresponds to the standard ports for the http and https protocols.

Change-Id: I46e2237dde99460fd096bd6fe58fe154b220041f
Signed-off-by: Pete Birley <pete@port.direct>
2019-12-06 17:48:38 -06:00
Zuul
d216fbf731 Merge "Elasticsearch: Remove unnecessary rbac definitions" 2019-12-06 18:16:06 +00:00
Zuul
bb7c2787c3 Merge "Elasticsearch/Kibana: Update version to 7.1.0" 2019-12-06 18:16:05 +00:00
bw6938
53a1343400 [ceph-client][ceph-osd] Adjust required_percent_of_osds parameter
and incorporate in ceph-osd testing.

This parameter is used by helm test to check if a set % of OSDs
out of the total are considered in & up. Adjusting to 75% and
adding to helm-test.sh for ceph-osd along with more robust
ceph osd validation function

Change-Id: Ib1f37b901f8656f0c6f2ed6a3ec27f0357e82278
2019-12-06 16:16:00 +00:00
Hemachandra Reddy
066a7e0ab4 Initialize PCI module if used before DPDK for hotplug to work
Change-Id: I9f552035eb70fd2a828ff7fa4852ba66055b885f
2019-12-06 15:56:14 +00:00
Tin Lam
daefed7218 Add feature gate capability to OSH-Infra
This patch set adds the feature gate capability to OpenStack-Helm-Infra
repository without depending on the main OpenStack-Helm repository.

Change-Id: I70b8fac4fd2365f8eedcf50519f125eb34534f2f
Signed-off-by: Tin Lam <tlam@omegaprime.dev>
Signed-off-by: Tin Lam <tin@irrational.io>
2019-12-03 16:55:00 -06:00
Zuul
8bd11d1ad2 Merge "[ceph-client] Validate failure domain support for replica count per pool" 2019-12-03 22:23:08 +00:00