This change updates how the Elasticsearch chart handles
S3 configuration and snapshot repository registration.
This allows for
- Multiple snapshot destinations to be configued
- Repositories to use a specific placement target
- Management of multiple account credentials
Change-Id: I12de918adc5964a4ded46f6f6cd3fa94c7235112
Unrestrict octal values rule since benefits of file modes readability
exceed possible issues with yaml 1.2 adoption in future k8s versions.
These issues will be addressed when/if they occur.
Also ensure osh-infra is a required project for lint job, that matters
when running job against another project.
Change-Id: Ic5e327cf40c4b09c90738baff56419a6cef132da
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
This change adds a new Deployment to the Elasticsearch chart to add a
set of "gateway" nodes to the Elasticsearch cluster. These nodes will
facilitate Elasticsearch remote cluster, for features such as cross
cluster search.
Co-Authored-By: David Smith <ds3330@att.com>
Change-Id: Ic4ac988a922a12addce3c65e0ef4099d46bbc784
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.
This change removes all references to this copyright by the
non-existent group and any blank lines underneath.
Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
This updates the Elasticsearch chart to make the values keys used
for defining node selectors for the various elasticsearch
components more granular
Change-Id: Ic1ac343b1d6ee48fc7cb456afe4cd9588c4aa13b
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This removes the cluster role definition from the Elasticsearch
component templates, as these are not needed for the service to
function correctly.
Change-Id: I671272affbed8984a47121187024e4b831937123
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Elasticsearch and Kibana charts to deploy
version 7.1.0. This move required significant changes to both
charts, including: changing elasticsearch masters to a statefulset
to utilize reliable dns names for the discovery process, config
updates to reflect deprecated/updated/removed values, use the
kibana saved objects api for managing index patterns and setting
the default index, and updating the elasticsearch entrypoint
scripts to reflect the use of elastic-keystore for storing s3
credentials instead of defining them in the configuration file
Change-Id: I270d905f266fc15492e47d8376714ba80603e66d
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Elasticsearch chart to include a specific start
script for the Elasticsearch data nodes that includes a trap on
signals that removes a data node from allocation eligible nodes
before shutting down. This results in all shards being moved from
a node on shut down to alleviate issues with planned down nodes,
such as during upgrade scenarios
Change-Id: I22f4957f90e4113831a8ddf48691cb14f811c1e5
This adds the helm-toolkit function for defining the update
strategy for the elasticsearch-data statefulset and sets the chart
default to RollingUpdate
Change-Id: Ia10ea7bf000474e597bdb36778118a96d85b93c1
This updates the Curator image to use version 5.6.0, which adds
additional actions for use, such as the ability to shrink indices.
This also adds a separate configmap and config secret for Curator,
as this allows us to use separate configmap annotations on the
Elasticsearch component pods to prevent Curator config updates
from triggering recreation of Elasticsearch components. This helps
alleviate overhead associated with Elasticsearch service restarts.
Change-Id: I0aec7756b0dc09bc3981ede950dc88f821aeca4b
This updates the Elasticsearch chart to allow for setting the
heap size per node type instead of for all nodes equally. This
also adds the required environment variable to configure whether
a node is an ingest node. This is set to false, as suggested for
elasticsearch versions <= 6.x
This also removes the ES_PLUGINS_INSTALL environment variable as
it is not used for anything in the current charts
Change-Id: I9096774db46dcbcd48b8a5448f0510984bf4108f
This PS adds emptydirs backing the /tmp directory in pods, which
is required in most cases for full operation when using a read only
filesystem backing the container.
Additionally some yaml indent issues are resolved.
Change-Id: I8b7f1614da059783254aa6efc09facf23fca3cad
Signed-off-by: Pete Birley <pete@port.direct>
This adds the release-annotation to the pod spec for the charts in
openstack-helm-infra. This also adds missing configmap annotations
to charts in openstack-helm-infra
Change-Id: Ie23f0c16a7a21d3929e98928db2bbcef69ae6490
This ps adds the ability to use the ceph radosgw s3 api for
snapshot repositories. It removes the ability to use a RWM pvc, as
the radosgw solution provides a more robust approach for storing
index snapshots
Change-Id: Ie56ac41ccdc61bfadcac52b400cceb35403e9fae
This PS adds the ability to attach a release uuid to pods and rc
objects as desired. A follow up ps will add the ability to add arbitary
annotations to the same objects.
Change-Id: Iceedba457a03387f6fc44eb763a00fd57f9d84a5
Signed-off-by: Pete Birley <pete@port.direct>
This updates the osh-infra charts to use a secret for their
configuration files instead of a configmap, allowing for the
storage of sensitive information
Change-Id: Ia32587162288df0b297c45fd43b55cef381cb064
In most cases, the ingress controller's nodeSelector key and value
are "node-role.kubernetes.io/ingress" and "true".
Using quote to treat the nodeSelector value as a string.
Change-Id: Ie1745629b90795e4d888d85f35565e6d6350e09b
This adds missing readiness probes to the following charts in
openstack-helm-infra: elasticsearch, fluent-logging, kibana,
nagios, prometheus-kube-state-metrics, prometheus-node-exporter,
and prometheus-openstack-exporter
Change-Id: I6a2635b08667c31eadb1b05ba848c658935a17e5
This PS moves to use the current API version for kubernetes rcs'
that were previously using `apps/v1beta1`.
Story: 2002205
Task: 21735
Change-Id: Icb4e7aa2392da6867427a58926be2da6f424bd56
Signed-off-by: Pete Birley <pete@port.direct>
This moves the charts in openstack-helm-infra closer towards a
standard structure. It addresses multiple deviations, including:
missing resources for init containers, incorrect indents for
disabled resources in some charts, incorrect indents for volumes
and volumemounts added via values, missing resources for some
helm test templates, missing helm-toolkit image functions, and
moving the resource template declarations to be under the image
template declarations
Change-Id: I4834a5d476ef7fc69c5583caacc0229050f20a76
This ps adds more granular node selectors for the charts in osh
infra to match what is currently done in osh
Change-Id: I8957a95053b9fb3ea329fd37ff049cd223a7695d
This PS simplify the logic for dyanmicly merging the image management
depenencies into pod deps when active.
Change-Id: I0cf6c93173bc5fbce697ac15be8697d3b1326d0a
This reverts the changes made to Elasticsearch, Kibana and fluent
logging charts in https://review.openstack.org/#/c/550229/7.
Specifically, this moves the images back to previous used versions
and makes the required changes to the fluent-logging elasticsearch
template job to include the correct mapping directives for the
elasticsearch template.
This change was made to give more time for evaluating a more
robust solution for switching to the official upstream images that
will not cause intermittent gate failures as seen since 550229 was
merged
Change-Id: I9f70b3412a8edc5cb1d80937b158aa2fe7b1ec82
This PS moves elasticsearch to use the endpoints section and
lookups to set the port it serves on.
Change-Id: I4a73893124b6d988cd1f885cfc3dd62abeb4ae8c
This moves Elasticsearch and Kibana to use the latest version
(6.2.2), as the images we were using are no longer supported with
the 6.x release. There was a change in the doc reference in the
log entries that prevented the previous ES version from indexing
those entries, resulting in a busted gate. Moving Kibana to 6.2.2
was required to match major/minor versions with Elasticsearch
The Elasticsearch version change also required changing config file
locations, changing the entrypoint used for launching the service,
changing the running user for the elasticsearch service, and
updated the ES tests as some of the API responses changed between
versions
This also required updating the elasticsearch template job as the
mapping definition entries changed between versions
Change-Id: Ia4cd9a66851754a1bb8f225c7e24513c43568e93
This PS moves existing dynamic common dependencies under a
'dynamic.common' key to simplify the yaml tree.
Change-Id: I4332bcfdf11197488e7bd5d8cf4c25565ea1c7b6
This PS moves static dependencies unser a 'static' key to allow
expansion to cover dynamic dependencies.
Change-Id: Ia0e853564955e0fbbe5a9e91a8b8924c703b1b02
This PS includes the release name in the cluster role to prevent
colision if the chart is deployed multiple times in the same
cluster.
Change-Id: I7166e5ee25b3d4c89879393c5f84c869585a2681
Adds "helm-toolkit.utils.merge" which is a replacement for the
upstream sprig "merge" function which didn't quite do what we
wanted, specifically it didn't merge slices, it just overrode
one with the other. This PS also updates existing callsites
of the sprig merge with "helm-toolkit.utils.merge".
Change-Id: I456349558d4cf941d1bcb07fc76d0688b0a10782
This adds node selectors to all templates in the elasticsearch
chart, as they were previously missed
Change-Id: I34ea5751663b2e993c5f73a78a1f91133919752c
This dynamically adds the elasticsearch path.repo configuration
entry if it's not defined. This solves issues arising when the
storage settings are disabled in favor of emptydirs for simpler
ES deployments. If elasticsearch attempts to configure the repo
path with an invalid entry (inaccesible external or shared fs
path), the service will crash.
Change-Id: I089b77104107dfb1f8e6ea2d8a560384718e63f9
This PS brings ElasticSearch inline with other charts by placing the
RBAC roles and bindings in the same template as the pod rc they are
assocated with.
Change-Id: I6d541a18d6750d42d31326f77a9aacb06195ddac
This brings the elasticsearch configmaps, volume and mount names
inline with other charts by naming them after the service.
This also moves the configuration for elasticsearch to the values
file to bring it inline with other charts that do the same
Change-Id: I61f7c740d830a9a0567f8b72a0f815a09407b90c
This provides an example action in the curator config for taking
snapshots of the elasticsearch indexes. As the snapshot action
requires a repository registered with Elasticsearch, this also
adds a PVC for a filesystem repository backed with NFS and a job
for registering the repository with Elasticsearch.
Change-Id: I26b788c58f52844e997bde5002459bddc1bb685e
Currently, services have two serviceaccounts: one specified in the
chart that cannot read anything, and one injected via helm-toolkit
that can read everything. This patch set refactors the logic to:
- cleanup the roles and their binding automatically when the helm
chart is deleted;
- remove the need to separately mount a serviceaccount with secret;
- better handling of namespaces resource restriction.
Co-Authored-By: portdirect <pete@port.direct>
Change-Id: I47d41e0cad9b5b002f59fc9652bad2cc025538dc
The elasticsearch tests fail because the pods don't have
readiness or liveliness probes in the templates. This adds those
definitions
Change-Id: I4fd25aec5ae02d89ae1b933d8b083a3e9cafc55a
This moves the Elasticsearch chart to OSH infra, along with rbac
rules for running Elasticsearch. It includes a cronjob for running
ES Curator for cleaning up old indices
Change-Id: I69fcbe8b77de8b594eba5340a6e4340f389ba5bf
