1818 Commits

Author SHA1 Message Date
Steve Wilkerson
d72299db4b Nagios: Use helm-toolkit template renderer for config files
This updates the Nagios chart to use the helm-toolkit template
renderer snippet for generating the Nagios configuration files.
This was done to make the exposure of the configuration files
simpler for those who are more familiar with traditional Nagios
configuration files, as well as allowing for values overrides for
adding custom host names or custom object definitions to nagios
objects (as Nagios doesn't easily allow for this via environment
accessible macros).

Change-Id: I84d5c83d84d6438af5f3ab57997e80e8b1fc8312
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-06 19:35:32 +00:00
Zuul
d23e847e1f Merge "Uplift nfs-provisioner" 2019-06-06 19:28:51 +00:00
Zuul
a3f4337952 Merge "Ceph-client: Fix security context for pod/container" 2019-06-06 18:47:30 +00:00
Chinasubbareddy Mallavarapu
6767d0ad48 [ceph-client] Handle pool min_size during ceph cluster expansion
This is to update logic for  pool min_size parameter  as  this is
not getting changed when replication changes from intilization.

Change-Id: I30f99aaf92c3dc83afce10534b1d2ac9402b7fa7
2019-06-06 15:20:43 +00:00
RAHUL KHIYANI
d3dec3d257 Ceph-mon: Fix security context for pod/container
This changes the  user from root to the nobody user instead
in ceph-mon chart wherever needed

Change-Id: I5d3c1fbc8f983688807b73867773bfa2d83b91b3
2019-06-06 12:32:01 +00:00
Zuul
5245f04d4c Merge "Ingress: Add pod/container security context" 2019-06-06 11:53:11 +00:00
Zuul
82291cfe0e Merge "Ceph-osd: Fix security context for pod/container" 2019-06-06 11:53:10 +00:00
Zuul
3ba03ed8ea Merge "Grafana: Add Selenium tests to helm test pod" 2019-06-05 22:25:09 +00:00
Evgeny L
b77385c82f Uplift nfs-provisioner
* Update the version of nfs-provisioner to the latest image.
* Allow nfs-provisioner user to manage endpoints, this is
  required because the newest version uses `leaderelection`
  package from k8s, this packages leverages labels on endpoints
  to track leader election information.

Change-Id: Ie2727bd6bcc26e57875bea38f0f665d4a0e85bd7
2019-06-05 16:57:40 +00:00
Steve Wilkerson
25e105f26a Grafana: Add Selenium tests to helm test pod
This adds selenium tests for the grafana chart to the helm test
pod to help ensure the Grafana deployment is functional and
accessible

Change-Id: Idc8d97e5111628d1ed4f25145086d54c5e0136e7
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-05 16:09:04 +00:00
RAHUL KHIYANI
c9a1c412e7 Ceph-osd: Fix security context for pod/container
This changes the  user from root to the nobody user instead
in ceph-osd chart wherever needed

This also permits read-only filesystems to back the containers by setting
the default to true

Change-Id: Ia777bf212e0e3414909c70a4bd839e12d4919bb2
2019-06-05 15:00:20 +00:00
Georg Kunz
5596247f3f Enable hugepage support in HTK resources snippet
This patch simplifies the resource snippet in helm-toolkit to allow for
specifying hugepage limits. Specifically, this patch replaces the
individual checks for specific system resources (e.g., cpu, memory) by
just copying over the entire resource component as defined in a
values.yaml or a corresponding overwrite.

This change is a prerequisite for enable hugepage handling in other
charts such as openvswitch or postgresql.

Change-Id: I786ff6c7aa5fb6b08b54d2e21878551e5e1e3818
2019-06-05 11:37:43 +02:00
RAHUL KHIYANI
a0d67a1117 Ingress: Add pod/container security context
This updates the etcd chart to include the pod
security context on the pod template.

This also adds the container security context to set
readOnlyRootFilesystem to true

Change-Id: I9bf05ab5c21f9afbe269e1566cfecd20b3c086c0
2019-06-04 15:19:21 -05:00
Steve Wilkerson
0d2ac10fc2 Update apparmor job after fluent-logging split
This updates the apparmor job to account for the splitting of the
fluent-logging chart, as it was missed during that change.  Now,
the apparmor job will deploy fluentbit as well as fluentd deployed
as a daemonset running as a collecting agent

Change-Id: Iefa50f474b57a10c5e7e5a9032c7b23d26d97640
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-04 12:58:33 +00:00
Zuul
b7f0fd8dff Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-06-04 08:57:21 +00:00
Zuul
e238b7a611 Merge "Remove readonly rootfs Elasticsearch" 2019-06-03 19:39:55 +00:00
Zuul
547cf9b6b8 Merge "Fluentd: Update configuration defaults and overrides" 2019-06-03 18:01:51 +00:00
Steve Wilkerson
dc6c1f2145 Elasticsearch: Update cluster-wait job script
This updates the cluster-wait job script to include a sleep for
when no nodes of a given type are detected.  This check was
previously executed only when a node count of (0 < x < expected)
was detected.  This update reduces the number of queries executed
against the Elasticsearch http endpoint

Change-Id: I15cb39250a5ab9a7f6df0d62c35289a55e109dbd
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-03 16:00:51 +00:00
Steve Wilkerson
aee9708b06 Fluentd: Update configuration defaults and overrides
This updates the default fluentd configuration to include
recommended settings for preventing the elasticsearch plugin from
reloading the connection after 10000 requests (default for the
ruby gem). This also updates the configuration overrides for the
fluentd-daemonset deployment to provide input parity with the
default fluentbit configuration by adding inputs for the docker
and kubelet systemd units, inputs for ceph, libvirt, kernel logs,
and auth logs on the host.  Finally, this updates the fluentd
template to include environment variables for the host name and
the fluentd pod name so they can be added to logged events through
fluentd filter plugins

Change-Id: I21f7a89a325c44f8b058ff01a20191bea1a210b4
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-03 16:00:44 +00:00
Randeep Jalli
79cb67544c Remove readonly rootfs Elasticsearch
This removes ReadonlyRootfs from
Elasticsearch data pods as this is
required in order for the data pods
to recover from outages

Change-Id: I603d3a25b6580eab20e2b20e1b1cd0cf740c7ab2
2019-06-03 15:02:51 +00:00
Zuul
2f0c2e328d Merge "Elasticsearch: kill process after data node drain" 2019-05-31 23:48:34 +00:00
Steve Wilkerson
1ee476b197 Elasticsearch: Allow override of job back off limits and deadlines
this updates the Elasticsearch cluster wait and snapshot repo jobs
to include values overrides for the job backoff limits and the
active deadline seconds field. This allows for tweaking beyond the
standard defaults for kubernetes jobs

Change-Id: I1f95a635ab4dfdb3718d5d4fa668c64a9095e899
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-05-31 15:52:16 -05:00
Steve Wilkerson
277b54fc65 Fluentd: Use fqdn hostname for Elasticsearch service
This moves Fluentd to use the helm-toolkit endpoint lookup for
using the fqdn for the Elasticsearch hostname instead of the
standard short host name

Change-Id: Ibe640979002331693f0a9b6155c9014572294664
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-05-31 14:42:09 -05:00
caoyuan
040edeb79a Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I0e3af4a3385f5b2a7705bc19b775863b16c2e08e
2019-05-31 01:52:10 +00:00
Steve Wilkerson
b0706544f7 Elasticsearch: kill process after data node drain
This updates the Elasticsearch entrypoint override for the data
nodes to include a signal to kill the process after the trap to
drain each data node completes

Change-Id: Iccd4342fe16d06787cb24342d9a57e4de12e6980
2019-05-30 15:12:37 -05:00
Zuul
c34dbeeec8 Merge "mariadb: fix hostpath values" 2019-05-30 19:46:15 +00:00
Zuul
f353fc4ab9 Merge "Elasticsearch: Add data node specific entrypoint override" 2019-05-30 19:24:40 +00:00
Zuul
6febe7333c Merge "Elasticsearch: Update client and data pod deps" 2019-05-30 18:52:51 +00:00
Zuul
62576654a6 Merge "Fluentd: Use OSH Fluentd image with systemd plugin" 2019-05-30 17:47:41 +00:00
Steve Wilkerson
d334c5b68a Elasticsearch: Update client and data pod deps
This updates the Elasticsearch client and data pod dependencies
to allow for sequential bring up of the cluster components. As
we want the order to be master->client->data, we add the discovery
service endpoint as a dependency for the client pods and add both
the discovery and client service endpoints as dependencies for
the data pods

Change-Id: Iec6d6f259dc8b7b4f2309b492409cc0e5feab669
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-05-30 15:28:59 +00:00
Steve Wilkerson
1b5cb4bea2 Fluentd: Use OSH Fluentd image with systemd plugin
This updates the default fluentd image to use the fluentd image
built with the systemd input plugin from the openstack-helm-images
repository

Change-Id: I7c75cd19d62f3dbc3fa4708642119f1781e58677
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-05-30 15:28:37 +00:00
Steve Wilkerson
0a8b710083 Elasticsearch: Add job history to Curator, update schedule key
This updates the Elastic Curator cron job to include configuration
for successful and failed job history limits, similar to the other
cron jobs we deploy. This also moves the key for configuring the
cron schedule from under .Values.conf.curator to a new top level
jobs key to maintain consistency

This also fixes an indentation issue with the deployment overrides
for Curator as well as adds the overrides for the Armada job

Change-Id: I9c720df9677215bdd2bf18be77959bd5f671c0ca
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-05-30 15:28:30 +00:00
Steve Wilkerson
91178c31bf Elasticsearch: Add data node specific entrypoint override
This updates the Elasticsearch chart to include a specific start
script for the Elasticsearch data nodes that includes a trap on
signals that removes a data node from allocation eligible nodes
before shutting down.  This results in all shards being moved from
a node on shut down to alleviate issues with planned down nodes,
such as during upgrade scenarios

Change-Id: I22f4957f90e4113831a8ddf48691cb14f811c1e5
2019-05-30 10:25:03 -05:00
RAHUL KHIYANI
789fa7a4e5 Ceph-client: Fix security context for pod/container
This changes the  user from root to the nobody user instead
in ceph-client chart wherever needed

Change-Id: I4c56b97f85093bbbaaef617f1981f67215a8bc00
2019-05-29 15:32:28 +00:00
Steve Wilkerson
8cb4051fc8 Elasticsearch: Fix typos in cluster wait script
This fixes typos in the cluster wait script to ensure the messages
reflect the types of nodes being checked

Change-Id: I5964b5517b3099fbfe8d574b2ca869d366c9bb17
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-05-29 09:58:44 -05:00
Itxaka
ce086f7e41
mariadb: fix hostpath values
Use the correct declaration of the hostpath

Change-Id: I7661083ad24abfb8469700ab2039b4b7be5dd681
2019-05-29 14:06:21 +02:00
Doug Aaser
9efb353b83 Patroni inclusion work for HA Postgres
This patchset aims to add HA Clustering support for Postgres. HA Clustering
provides automatic failover in the event of the database going down in addition
to keeping replicas of the database for rebuilding in the event of a node
going down. To achieve this clustering we use
[Patroni](https://github.com/zalando/patroni) which offers HA clustering
support for Postgres.

Patroni is a daemon that runs in the background and keeps track of which
node in your cluster is currently the leader node and routes all traffic
on the Postgresql endpoint to that node. If the leader node goes down,
Patroni holds an election to chose a new leader and updates the endpoint
to route traffic accordingly. All communication between nodes is done by
a Patroni created endpoint, seperate from the externally facing Postgres
endpoint.

Note that, although the postgresql helm chart can be upgraded from
non-patroni to patroni clustering, the previous `postgresql`
endpoints object (which is not directly managed by helm) must be
deleted via an out-of-band mechanism so that it may be replaced by the
patroni-managed endpoints.  If Postgres itself is leveraged for the
deployment process, this must be done with careful timing.  Note that
the old endpoints had a port named "db", and the new endpoints has
a port named "postgresql".

- Picking up patchset: https://review.openstack.org/#/c/591663

Co-authored-by: Tony Sorrentino <as1413@att.com>
Co-authored-by: Randeep Jalli <rj2083@att.com>
Co-authored-by: Pete Birley <pete@port.direct>
Co-authored-by: Matt McEuen <mm9745@att.com>

Change-Id: I721b745017dc1ea7ae05dfd9f8d5dd08d0965985
2019-05-28 19:13:13 +00:00
Zuul
630efb7fb0 Merge "Fluentd: Support Daemonset deployment" 2019-05-28 17:50:43 +00:00
Zuul
0513501201 Merge "Fix python template" 2019-05-28 17:39:25 +00:00
Steve Wilkerson
bdaf866a4e Fluentd: Support Daemonset deployment
This adds required changes to the Fluentd chart to allow for
deploying Fluentd as either a deployment or a daemonset. This
follows the pattern laid out by the ingress chart. This also
updates the single and multinode jobs to deploy fluentd as both
a daemonset and a deployment for validation

Change-Id: I84353a2daa2ce56ff59882a8d33203286ed27e06
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-05-28 08:23:44 -05:00
Jean-Philippe Evrard
ed200cb3b0 Wait for pods for openstack-support scenario
The wait for pods is not consistently used in the
openstack-support scenario.

This is a problem, as some helm charts deploys are
basically masking issues that can arise.

This should fix it.

Change-Id: Ib3e8f16bea701bf20375d4deec7c7869e7bf85c2
2019-05-28 07:51:34 +00:00
Zuul
fd2eb31920 Merge "fix typo errors" 2019-05-27 17:05:26 +00:00
Tin Lam
6bae1b020d Fix python template
This patch set removes an unused import that is not python3 compatible.

Change-Id: I360989c8eb23065d8e655d4583eb97338244412d
Signed-off-by: Tin Lam <tin@irrational.io>
2019-05-26 21:43:19 +00:00
Zuul
5cb47aed80 Merge "Separate fluentbit and fluentd charts" 2019-05-24 17:34:27 +00:00
Zuul
a12892f724 Merge "Ceph-rgw: Fix securityContext" 2019-05-24 15:44:00 +00:00
Tin Lam
752bbf48c4 Migrate ubuntu jobs from Xenial to Bionic
This patch set is based on [0] that also fixed up a handful of Bionic-
specific changes required for the gates to pass.

[0] https://review.openstack.org/#/c/649698/

Co-Authored-By: ghanshyam <gmann@ghanshyammann.com>

Change-Id: I217a27c53eec2a51ddbea7226a23042558c5946b
2019-05-24 13:52:08 +00:00
Steve Wilkerson
abb5e0f713 Separate fluentbit and fluentd charts
This begins to split the fluent-logging chart into two separate
charts, one for fluentbit and one for fluentd. This is to help
isolate each chart and its dependencies better, and to treat each
service as its own entity.

This also moves the job for creating Elasticsearch templates to
the Elasticsearch chart, as the elasticsearch chart should have
ownership of creating the templates for its indices.

This also performs some general cleanup of values keys that are
not currently used

Change-Id: I827277d5faa62b8b59c5960330703d23c297ca47
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-05-24 06:31:09 -05:00
Zuul
1a3788ddf1 Merge "CephFs-provisioner: fix securitycontext in the pod" 2019-05-24 03:08:04 +00:00
Zuul
ff2d62c9bf Merge "Fluent-logging: Remove utils for generating configuration files" 2019-05-24 03:08:03 +00:00
Zuul
2de8acf4a4 Merge "Fluentd: Update ClusterRole object" 2019-05-24 03:08:01 +00:00