tmp is an emptyDir volume mount with 0777 permissions. Some versions
of Ruby require the sticky bit to be set.
Change-Id: Ib6f4daa2068e7b29c62b5858848774b7117f6808
Based on spec in openstack-helm repo,
support-OCI-image-registry-with-authentication-turned-on.rst
Each Helm chart can configure an OCI image registry and
credentials to use. A Kubernetes secret is then created with these
info. Service Accounts then specify an imagePullSecret specifying
the Secret with creds for the registry. Then any pod using one
of these ServiceAccounts may pull images from an authenticated
container registry.
Change-Id: Iebda4c7a861aa13db921328776b20c14ba346269
This change updates the default image value where openstack is
used to Wallaby from newton in the fluentd chart.
Change-Id: I794062bee9e5d316e1c754544c0970da93e46dbf
This change updates the helm-toolkit path in each chart as part
of the move to helm v3. This is due to a lack of helm serve.
Change-Id: I011e282616bf0b5a5c72c1db185c70d8c721695e
The broker attribute should use a comma separated list with the port
definition included
Example: kafka1:9092,kafka2:9092,kafka:9092
The kafka client will connect to the first available host this
will provide resiliency if a host is not available
Change-Id: I5f82e96f2aa274379b6d808291d4b5109709bf72
This will ease mirroring capabilities for the docker official images.
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I0f9177b0b83e4fad599ae0c3f3820202bf1d450d
This change enables TLS between Elasticsearch and Kibana
data path. Note that TLS terminates at apache-proxy container
of the Elasticsearch-client pod, not directly to port 9200 of
elasticsearch-client container.
Since all data traffic goes through apache-proxy container,
fluentd output to Elasticsearch are configured to have TLS
enabled as well.
In additon, other Elasticsearch pods that communicate with
Elasticsearch-client endpoint are modified to provide
the cacert option with curl.
Change-Id: I3373c0c350b30c175be4a34d25a403b9caf74294
This change updates the fluentd chart to use HTK probe templates
to allow configuration by value overrides
Change-Id: I97a3cc0832554a31146cd2b6d86deb77fd73db41
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0
Change-Id: I15950b735b4f8566bc0018fe4f4ea9ba729235fc
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
Added chart lint in zuul CI to enhance the stability for charts.
Fixed some lint errors in the current charts.
Change-Id: I9df4024c7ccf8b3510e665fc07ba0f38871fcbdb
This chart could deploy fluentd either as a Deployment
or a Daemonset. Both options would use the deployment-fluentd
template with various sections toggled off based on values.yaml
I'd like to know - Does anyone run this chart as a Deployment?
We can simplify the chart, and zuul gates, by changing the chart
to deploy a Daemonset specifically.
Change-Id: Ie88ceadbf5113fc60e5bb0ddef09e18fe07a192c
Unrestrict octal values rule since benefits of file modes readability
exceed possible issues with yaml 1.2 adoption in future k8s versions.
These issues will be addressed when/if they occur.
Also ensure osh-infra is a required project for lint job, that matters
when running job against another project.
Change-Id: Ic5e327cf40c4b09c90738baff56419a6cef132da
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.
Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.
Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
By tying the fluent condfiguration to the release, it will be re-rendered
if the release is upgraded. This is useful in combination with [0], allowing
powerful configuration updates using helm upgrade. For example
Values:
.Values.pod.env.fluentd.vars.OUTPUT_ENABLED: true
fluent.conf:
...
{{- if .Values.pod.env.fluentd.vars.OUTPUT_ENABLED }}
<match **>
# Output Configuration here
</match>
{{- end }}
To disable this output section, issue a helm upgrade command and set the
apprpriate value to false.
helm upgrade fluentd ./fluentd --set pod.env.fluentd.vars.OUTPUT_ENABLED=false
[0] https://review.opendev.org/#/c/726880/
Change-Id: I3dce9e5c4eaf588569e8cc3e1ea3cf3bebd0c3c5
This change updates the fluentd configmap-etc to render
.Values.conf.fluentd.template as a template, allowing for greater
flexibility in configuration.
Change-Id: I8809767c679c377e319ecc53960c55ae18e1b558
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.
This change removes all references to this copyright by the
non-existent group and any blank lines underneath.
Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
This change updates the fluentd chart to use the native
fluent-plugin-prometheus for metric production. This plugin
provides more detailed metrics about fluentd's operations,
specifically regarding input and output statistics.
https://github.com/fluent/fluent-plugin-prometheus
Using the plugin, each fluentd pod produces metrics, so
scape annotations have been added to the pod spec. The
zuul check on metric producers has been updated to account
for this.
Depends-On: https://review.opendev.org/714167
Change-Id: I809356d92b0cff1e31cb2062102bbedefd4843fd
This patch set updates and tests the apiVersion for rbac.authorization.k8s.io
from v1beta1 to v1 in preparation for its removal in k8s 1.20.
Change-Id: I4e68db1f75ff72eee55ecec93bd59c68c179c627
Signed-off-by: Tin Lam <tin@irrational.io>
This change adds in missing network policy overrides for
fluent-daemonset and prometheus-exporter, as well as removes
existing mariadb network policies overrides that were causing
the network policy check job to fail.
Change-Id: Ib7a33f3d14617f9a9fda264f32cde7729a923193
This adds a helm-toolkit util for consuming arbitrary secret env
variables via pod env variables. It also updates the Fluentd chart
to add a release secret that is used to house the secret env
variables defined in the chart's values.yaml. This can be used as
an example to expand to other charts where this functionality is
desired
Change-Id: I9ef606840af92e54b2204e637c58442085e2c748
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This proposes adding a kafka chart to osh-infra that aligns
with the design patterns laid out by the other charts in osh-infra
and osh.
danielqsj's kafka-exporter image is leveraged to deploy a prometheus
exporter for kafka alongside the main application if enabled in
values.yaml
Change-Id: I5997b0994fc3aef9bd1b222c373cc3a013112566
Co-Authored-By: Meghan Heisler <mh783g@att.com>
This updates charts that consume images built from osh-images to
use tags other than the :latest tags. This will be followed up
with the definition of jobs to allow for vetting out of updated
images, as reliance on :latest tags assumes any change merged into
osh-images will result in functionally correct behavior (which has
shown to not be the case traditionally)
Change-Id: I181aa56ed187604dc7583d8081e53cc69eb27310
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintained
Change-Id: I5bfdc156ae228ab16da57569ac6b05a9a125cb6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the kubeadm and minikube Kubernetes deployments to
deploy version 1.16.2
Change-Id: I324f9665a24c9383c59376fb77cdb853facd0f18
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This adds support for configuring fluentd's update strategy when
deployed as a daemonset, as this was previously missed when
the changes to support both daemonsets and deployments were made
Change-Id: I5ac4fbfc0e64caaf207de42cd71c893f8d0f6ff1
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Fluentd deployment template to use the helm
toolkit util for generating environment variables through the
chart's values.yaml. This adds flexibility in defining fluentd
outputs, as arbitrary environment variables can be injected and
consumed in fluentd's filters and outputs
Change-Id: I72a2c476378cc555bde1387781b4a06f13b51bc6
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the fluentd chart to add support for leveraging a
Kafka output. This required adding a kafka endpoint entry to the
chart's values.yaml, as well as the required template updates to
the fluentd deployment template and the addition of a secret for
kafka credentials
Depends-On: https://review.opendev.org/#/c/679297/
Change-Id: I80a487a0538f0b3704fb598da38c07feedaccb0e
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Fluentd clusterrole to allow for getting
namespaces, as this is required for the fluentd kubernetes
plugin to function correctly
Change-Id: Id9d735310c53a922a62c6a82121edd332e7df724
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This fixes the whitespace chomps for adding extra volumes and
volume mounts via values.yaml for the Fluentd chart, as currently
too much whitespace is removed and the extra volumes and mounts
are not added correctly
Change-Id: I9cf67c3321339078ac795a7290f441b16cc41d41
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This adds a conditional check on the deployment type of the
Fluentd chart to determine whether to enable the current liveness
and readiness probes or not. The current probes are designed
around using fluentd as an aggregator and do not function properly
when fluentd is deployed as a daemonset. When run as a daemonset
and configured to tail files via the tail input plugin, fluentd
will prioritize reading the entirety of those files before
processing other input types, including opening the forward source
socket required for the current probes to function correctly. This
results in scenarios where the current probes will fail when in
fact fluentd is functioning correctly.
Daemonset focused probes to come as a follow on once a proper path
forward has been determined
Change-Id: I8a164bd47ce1950e0bd6c5043713f4cde9f85d79
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This removes the readOnly flag from the /var/log mount for the
fluentd pod to allow for using the file buffer mechanism when
desired
Change-Id: I23f0f03824eec5b142d3f2e8e42e8d07cddfe618
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the default fluentd configuration to include
recommended settings for preventing the elasticsearch plugin from
reloading the connection after 10000 requests (default for the
ruby gem). This also updates the configuration overrides for the
fluentd-daemonset deployment to provide input parity with the
default fluentbit configuration by adding inputs for the docker
and kubelet systemd units, inputs for ceph, libvirt, kernel logs,
and auth logs on the host. Finally, this updates the fluentd
template to include environment variables for the host name and
the fluentd pod name so they can be added to logged events through
fluentd filter plugins
Change-Id: I21f7a89a325c44f8b058ff01a20191bea1a210b4
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This moves Fluentd to use the helm-toolkit endpoint lookup for
using the fqdn for the Elasticsearch hostname instead of the
standard short host name
Change-Id: Ibe640979002331693f0a9b6155c9014572294664
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the default fluentd image to use the fluentd image
built with the systemd input plugin from the openstack-helm-images
repository
Change-Id: I7c75cd19d62f3dbc3fa4708642119f1781e58677
Signed-off-by: Steve Wilkerson <sw5822@att.com>
