This patchset aims to add HA Clustering support for Postgres. HA Clustering
provides automatic failover in the event of the database going down in addition
to keeping replicas of the database for rebuilding in the event of a node
going down. To achieve this clustering we use
[Patroni](https://github.com/zalando/patroni) which offers HA clustering
support for Postgres.
Patroni is a daemon that runs in the background and keeps track of which
node in your cluster is currently the leader node and routes all traffic
on the Postgresql endpoint to that node. If the leader node goes down,
Patroni holds an election to chose a new leader and updates the endpoint
to route traffic accordingly. All communication between nodes is done by
a Patroni created endpoint, seperate from the externally facing Postgres
endpoint.
Note that, although the postgresql helm chart can be upgraded from
non-patroni to patroni clustering, the previous `postgresql`
endpoints object (which is not directly managed by helm) must be
deleted via an out-of-band mechanism so that it may be replaced by the
patroni-managed endpoints. If Postgres itself is leveraged for the
deployment process, this must be done with careful timing. Note that
the old endpoints had a port named "db", and the new endpoints has
a port named "postgresql".
- Picking up patchset: https://review.openstack.org/#/c/591663
Co-authored-by: Tony Sorrentino <as1413@att.com>
Co-authored-by: Randeep Jalli <rj2083@att.com>
Co-authored-by: Pete Birley <pete@port.direct>
Co-authored-by: Matt McEuen <mm9745@att.com>
Change-Id: I721b745017dc1ea7ae05dfd9f8d5dd08d0965985
This adds required changes to the Fluentd chart to allow for
deploying Fluentd as either a deployment or a daemonset. This
follows the pattern laid out by the ingress chart. This also
updates the single and multinode jobs to deploy fluentd as both
a daemonset and a deployment for validation
Change-Id: I84353a2daa2ce56ff59882a8d33203286ed27e06
Signed-off-by: Steve Wilkerson <sw5822@att.com>
The wait for pods is not consistently used in the
openstack-support scenario.
This is a problem, as some helm charts deploys are
basically masking issues that can arise.
This should fix it.
Change-Id: Ib3e8f16bea701bf20375d4deec7c7869e7bf85c2
This patch set is based on [0] that also fixed up a handful of Bionic-
specific changes required for the gates to pass.
[0] https://review.openstack.org/#/c/649698/
Co-Authored-By: ghanshyam <gmann@ghanshyammann.com>
Change-Id: I217a27c53eec2a51ddbea7226a23042558c5946b
This begins to split the fluent-logging chart into two separate
charts, one for fluentbit and one for fluentd. This is to help
isolate each chart and its dependencies better, and to treat each
service as its own entity.
This also moves the job for creating Elasticsearch templates to
the Elasticsearch chart, as the elasticsearch chart should have
ownership of creating the templates for its indices.
This also performs some general cleanup of values keys that are
not currently used
Change-Id: I827277d5faa62b8b59c5960330703d23c297ca47
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This removes the utilities for generating the fluentd, fluentbit,
and parser configuration files from yaml and moves to instead
consume the configuration files as strings from the values.yaml.
This allows for easier creation and maintenance of configuration
files for fluentd and fluentbit, as the utilities became unwieldy
with complex configuration files.
This necessitated the removal of the core test executed by the
charts helm tests, but this would be required as we move to split
the charts regardless
Change-Id: Ied4a76bbdf58b54a6d702db04a7120b64f54dcac
Signed-off-by: Steve Wilkerson <sw5822@att.com>
- Update ceph-client chart to
1) By default, enable ceph-client helm test. Update enabler
key in values.yaml to follow pattern as in other charts
2) Add needed dependancy for ceph-client helm tests
3) Update helm test script to reduce output and update
error msgs
4) Removed unwanted ENV variables SPECS and EXPECTED_POOLMINSIZE
- Update gate scripts to run helm test command
Change-Id: I6a0e4f5107e49dac081ac2037bcc0f9c0864793f
This updates the scripts for deploying fluentd to include
overrides for enabling prometheus monitoring. Despite not
deploying prometheus in the osh-infra-logging job, we can still
leverage the post run job to gather metrics from the exporters
service. This gives us the means for verifying the functionality
of the exporter
Change-Id: Id98474de89d86419157635007e2f114f0947498e
This adds the password for the mariadb sst user to the armada-lma
manifest, as it was previously missed
Change-Id: I8768569fff96bf15cb4b2a577a0f667972fda886
This updates the Elasticsearch chart to allow for setting the
heap size per node type instead of for all nodes equally. This
also adds the required environment variable to configure whether
a node is an ingest node. This is set to false, as suggested for
elasticsearch versions <= 6.x
This also removes the ES_PLUGINS_INSTALL environment variable as
it is not used for anything in the current charts
Change-Id: I9096774db46dcbcd48b8a5448f0510984bf4108f
This adds test specific timeout to all LMA components in
Armada. This also removes test enabled true because
Armada enables tests by default
Change-Id: I893342b36ba27cbe7d3ce8767f73795e84a11732
This fixes the ceph deployment in the apparmor job as the previous
overrides weren't entirely correct. This also reorders the
deployment steps in the apparmor job to enforce the sequential
naming scheme used
Change-Id: I161bae649d4ff67307abeadc12b3c7d321af31c7
This updates the airship-divingbell repository defined in the
single node check job to use the correct repository namespace.
This also updates the divingbell check job to use the standard
osh-infra-gate-runner playbook as well as the minikube based
kubernetes deployment
Change-Id: Iff53279b3e09058deb323d092955cbf87230b5e5
This PS remvoes the stable helm repo, if present, to improve the
build time of patches.
Change-Id: Id6ec86e5ff426994b12adf4ca8e80eda2e52f147
Signed-off-by: Pete Birley <pete@port.direct>
This adds ingress network policy for the fluent-logging, kibana
and Elasticsearch charts. This leverages the helm-toolkit template
that was used in openstack-helm for the openstack services
Change-Id: I2a89b62f1002851346e9a25de40113078e9c518f
Trivial fix. This patch set fixes inconsistent indentations in YAML file.
Change-Id: I98ed9680d93f9c21e44b7da8462c9ce3607350bd
Signed-off-by: Tin Lam <tin@irrational.io>
In order to align with ceph-provisioner storageclass
.Values we need to update ceph-mon .Values and tenant-
ceph scripts
Change-Id: I36fb07de9e791ac33cf0b4c38b3e4d63337d3e72
Story: storageclass schema changes
Signed-off-by: kranthi guttikonda <kranthi.guttikonda@att.com>
Signed-off-by: kranthikirang <kranthi.guttikonda@b-yond.com>
This PS adds a basic sanity test to the mariadb chart, using
mysqlslap.
Change-Id: I7450ea8a66364d123022bc773ee90047f9e69b1c
Signed-off-by: Pete Birley <pete@port.direct>
This updates the apparmor job to only use the docker default
profile for memcached, as the custom apparmor profiles used didnt
allow for a successful deployment. This also updates the libvirt
overrides, as the current change to use daemonset-overrides
required updating the container name.
Co-authored-by: wilkers.steve@gmail.com
Co-authored-by: ld366r@att.com
Change-Id: I00cb4c62a38e0e1178e45b4e34c946b3b53da6d5
This moves both minikube and kubeadm deployments back to using
the ceph mimic repositories instead of ceph luminous repositories
Change-Id: I1b6b6af6ecb82e7c690a735286198bc7d0ab7c8d
This adds a deployment step for postgresql to the single node
monitoring job to validate the exporter works as intended
Change-Id: I2680b3e40ca4466e27daf6145cef064c312a7b57
This PS adds the ability to change the admin user credentials
and erlang session cookie. To do so requires `--recreate-pods` to
be passed to helm on a release upgrade.
Change-Id: Ib04ad43a7c303a8ddc31fd0de288a2f7f3294a12
Signed-off-by: Pete Birley <pete@port.direct>
This updates the kubernetes version used when deploying via
kubeadm and minikube to v1.13.4
This required updating the apiVersion in the kubeadm configuration
file template, as well as removing the --cadvisor-port flag from
the kubelet args, as this has been removed entirely
Change-Id: I3088b65ece0a5c9c5ef2669247ac293d6a6f66ed
- Move the cronjob from ceph-mon to ceph-client
- Adding ceph-rbd-pool job as dependencies for cronjob
- checkPGs manifest set to true so it will always run
in gate.
Co-Authored-By: Chinasubbareddy Mallavarapu <cr3938@att.com>,
Renis Makadia <renis.makadia@att.com>
Change-Id: I9855d8d22265e78c7e2f5fa7ece69c9ff532ecb2
Provide support to add annotations to the podsecuritypolicy. This will
allow to add annotations related to seccomp and apparmor in psp.
Change-Id: I78718ae1f60e8ebee8ac8ba86145bb9ae26491d5
