121 Commits

Author SHA1 Message Date
Steve Wilkerson
bfa237d347 Charts: Update helm test pod templates
This updates the helm test pod templates in the charts with helm
tests defined. This change includes the addition of:

- Generate test pod cluster roles and role bindings
- Generate service accounts for test pods
- Add node selectors to the test pods
- Add service accounts to the test pods
- Addition of entrypoint container to the test pods
- Indentation fix for rabbitmq test pod template

Change-Id: I9a0dd8a1a87bfe5eaf1362e92b37bc004f9c2cdb
2018-10-09 21:00:00 +00:00
Mike Pham
0ad481d9c6 Add missing labels to cronJobs
This PS adds the missing labels so it is able to target by the network policy.

Change-Id: I5da7e0a9b17d00a5e98488780d72c77a93af999f
2018-10-09 13:58:17 -04:00
Steve Wilkerson
30f339a080 Elasticsearch: Add node selector to Curator cron job
This adds the node selector key and value configuration to the
Curator cron job for Elasticsearch, as it was previously omitted

Change-Id: Id702007fa827a1e1f90dee9b2a855e4197f4567c
2018-10-09 11:01:41 -05:00
Steve Wilkerson
1f4a890343 Elasticsearch: Update log4j2 configuration settings
This updates the configuration settings used for the log4j2
template for Elasticsearch. The previous settings weren't
compatible with the version of Elasticsearch currently being used
(5.6.4)

Change-Id: Id4b02ad022c46d599ae02ef77bb0f81f7e62c9e4
2018-10-09 08:01:39 -05:00
Steve Wilkerson
a084769410 Elasticsearch S3 repo
This ps adds the ability to use the ceph radosgw s3 api for
snapshot repositories. It removes the ability to use a RWM pvc, as
the radosgw solution provides a more robust approach for storing
index snapshots

Change-Id: Ie56ac41ccdc61bfadcac52b400cceb35403e9fae
2018-09-19 15:53:21 -05:00
Zuul
a98b14d541 Merge "Elasticsearch: Add ingress, remove node ports" 2018-09-19 02:57:09 +00:00
Pete Birley
bb3ff98d53 Add release uuid to pods and rc objects
This PS adds the ability to attach a release uuid to pods and rc
objects as desired. A follow up ps will add the ability to add arbitary
annotations to the same objects.

Change-Id: Iceedba457a03387f6fc44eb763a00fd57f9d84a5
Signed-off-by: Pete Birley <pete@port.direct>
2018-09-13 05:35:35 +00:00
Steve Wilkerson
cd88fc44fc Elasticsearch: Add ingress, remove node ports
This adds an ingress to the Elasticsearch chart, allowing for the
exposure of the Elasticsearch cluster externally if required.

This also removes the node ports from the data and discovery
services, as these ports should not be used beyond service
discovery by the elasticsearch nodes. It moves the node port for
the client service under the network.elasticsearch key to match
the network tree for the other services

Change-Id: Ia989eff87b8c9f112c697ae309bbb971dc699aa5
2018-09-04 14:19:13 +00:00
Steve Wilkerson
9a311475ba Charts: Use secrets for configs in chart
This updates the osh-infra charts to use a secret for their
configuration files instead of a configmap, allowing for the
storage of sensitive information

Change-Id: Ia32587162288df0b297c45fd43b55cef381cb064
2018-08-24 15:56:53 -05:00
Seungkyu Ahn
a430533e6a Quoting node_select_value in Ingress Controller
In most cases, the ingress controller's nodeSelector key and value
are "node-role.kubernetes.io/ingress" and "true".
Using quote to treat the nodeSelector value as a string.

Change-Id: Ie1745629b90795e4d888d85f35565e6d6350e09b
2018-08-01 02:39:05 +00:00
Zuul
9c7169e40e Merge "Resources: Fix erroneous resource definitions" 2018-07-30 18:29:10 +00:00
Steve Wilkerson
4f78e1f6fc Drive apache proxy configuration via values templates
This proposes defining the apache proxy hosts entirely via values
templates. While complicated on its face, this gives flexibility
by allowing the ability to define the desired authentication
mechanism via values templates. These options can range from
using http basic auth for development purposes to defining more
complex ldap configurations without a need to modify the chart
directly

Change-Id: Ief1b6890444ff90cc9c0ca872087af74836c0771
Signed-off-by: Pete Birley <pete@port.direct>
2018-07-30 07:52:26 -05:00
Steve Wilkerson
397eebf995 Resources: Fix erroneous resource definitions
This fixes the resource trees for the fluent-logging and
openstack-exporter charts to match the other charts. This
also fixes the elasticsearch master template to use the
correct indentation level for the resource template

Change-Id: Ic6ec270a880216daff10d1f22128c6377ebf9933
2018-07-27 16:35:37 -05:00
Steve Wilkerson
dc16a897d7 Add missing labels to helm test pods
This adds missing labels to the helm test pods in osh-infra

Change-Id: I618d9089bfde2d847411f5f876f0ff6afd9cce7f
2018-07-10 08:55:40 -05:00
Steve Wilkerson
cb7bf2c0b3 Add missing readiness probes to openstack-helm-infra charts
This adds missing readiness probes to the following charts in
openstack-helm-infra: elasticsearch, fluent-logging, kibana,
nagios, prometheus-kube-state-metrics, prometheus-node-exporter,
and prometheus-openstack-exporter

Change-Id: I6a2635b08667c31eadb1b05ba848c658935a17e5
2018-06-26 12:25:36 +00:00
Steve Wilkerson
2dd5bf0594 Update ordering of auth providers in apache reverse proxy
This updates the ordering of the basic auth providers in the
elasticsearch and nagios chart to check the file provider first
before going out to check the configured ldap server.

Change-Id: I47ff8a1c7b2cefa8425914c5d4d7a76aa8d43216
Signed-off-by: Steve Wilkerson <wilkers.steve@gmail.com>
2018-06-25 12:43:06 -05:00
Pete Birley
fa629cdbbd Daemonsets: Use current kubernetes daemonset api version
This PS moves to use the current ga version for kubernetes daemonsets,
additionally any remaining deployments that were using the
`extensions/v1beta1` have been updated to `apps/v1`.

Story: 2002205
Task: 21735

Change-Id: If9703162dc472af1e6096bf2b9062802fd5ce8ab
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-13 21:53:18 +00:00
Pete Birley
b6a51fb57f Use current kubernetes API version
This PS moves to use the current API version for kubernetes rcs'
that were previously using `apps/v1beta1`.

Story: 2002205
Task: 21735

Change-Id: Icb4e7aa2392da6867427a58926be2da6f424bd56
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-12 17:35:13 -05:00
Steve Wilkerson
f6fe167278 Elasticsearch: Update tests to clean up test data and index
This ps adds a function for cleaning up the test data used to
verify Elasticsearch is functioning properly. It removes the test
index created and populated with test data to ensure the resulting
elasticsearch cluster is clean and does not contain random data

Change-Id: Ibdeb90e3f3b6307bf16c68469556bef256ed2d78
2018-06-10 12:56:57 +00:00
Steve Wilkerson
de9c46bcfa Charts: Tidy up openstack-helm-infra charts
This moves the charts in openstack-helm-infra closer towards a
standard structure. It addresses multiple deviations, including:
missing resources for init containers, incorrect indents for
disabled resources in some charts, incorrect indents for volumes
and volumemounts added via values, missing resources for some
helm test templates, missing helm-toolkit image functions, and
moving the resource template declarations to be under the image
template declarations

Change-Id: I4834a5d476ef7fc69c5583caacc0229050f20a76
2018-05-21 12:58:22 -07:00
Steve Wilkerson
9e11fc11af Update resource tree for elasticsearch/kibana
This adds the entry for resources for the apache proxy running in
the elasticsearch client and kibana pods. This also fixes an
incorrect enabled flag for resources in the kibana chart

Change-Id: Ifcd33a680167d7debfae2c4d71bdcb693632fce9
2018-05-15 20:55:24 +00:00
Steve Wilkerson
3c692abd6e Add ldap support in elasticsearch/kibana apache proxies
This adds required configuration for enabling LDAP through
the apache proxy in the elasticsearch and kibana charts by
default

Change-Id: Iaff8f328ff50944ddad94ec86b1134ca73750176
2018-05-14 13:53:30 +00:00
Sean Eagan
f402171e42 Move to v0.3.1 of kubernetes-entrypoint
Move to v0.3.1 of kubernetes-entrypoint which has 2 breaking changes to
pod dependencies, and also adds support for depending on jobs via
labels.

Change-Id: I2bafc2153ddd46b3833b253a2e7950bccbccf8ed
2018-04-25 12:38:44 -05:00
Zuul
f059e9a6df Merge "Update curator auth config and actions" 2018-04-19 21:35:08 +00:00
Zuul
e36dfcd21d Merge "Add manifest for image_repo_sync job" 2018-04-19 20:55:15 +00:00
Steve Wilkerson
5a4d56d068 Update curator auth config and actions
Curator is unable to use environment variables for configuration
values if the configured option contains more than the env
variable.  In the case of the http_auth value (which expects
user:password), using ${USER}:${PASS} prevents curator from
successfully authenticating to elasticsearch.  This moves to
dynamically define these values in the configmap if the value is
empty

This also updates values for curators actions to target logstash-
indices for its actions

Change-Id: Id5b49171e00847432e4ab0cf4be60005b70c21e3
2018-04-19 10:01:10 -05:00
Steve Wilkerson
e166432a98 Add manifest for image_repo_sync job
This ps proposes adding a common template for the image_repo_sync
jobs for consumption by the charts

Change-Id: I48476d1e4fd94bd1b08b13b46983e3d999f8d8ca
2018-04-19 14:10:08 +00:00
Steve Wilkerson
ee7516f565 add elasticsearch, fluent-logging, grafana registry endpoints
This adds the local image registry endpoint to elasticsearch,
fluent-logging and grafana.  This endpoint was missing from the
values.yaml in those charts

Change-Id: I30dc1f0cab40ccf8a493e13f407e2f0d37af1eee
2018-04-19 01:12:47 +00:00
Zuul
d93649da5f Merge "Update kibana authentication values in endpoints and deployment" 2018-04-19 00:59:04 +00:00
Steve Wilkerson
a31afb2f85 Update kibana authentication values in endpoints and deployment
The Kibana username and password needs to match the Elasticsearch
username and password, as Kibana requires an authorized elasticsearch
user to make queries against the elasticsearch backend to display
its dashboards and set up the initial .kibana index. This changes
the apache proxy running in front of kibana to consume the
elasticsearch username and password via the elasticsearch secret in
the chart to ensure kibana has proper access

Change-Id: Ife3fd916e8d9a3f8877d01a9048a892f92e412d8
2018-04-18 21:55:38 +00:00
Zuul
b16b4354d0 Merge "Add actions to Elastic Curator configuration" 2018-04-18 19:56:49 +00:00
Steve Wilkerson
e7da89ee05 Add actions to Elastic Curator configuration
This ps updates Curator with reference actions for deleting and
snapshotting Elasticsearch indices and also modifies Curator
to account for Elasticsearch auth and hostname configuration via
endpoint lookup

Change-Id: Ic68a2506c2ea96fc7269a7bb639ebba9c9b1ef20
2018-04-18 12:31:43 -05:00
Steve Wilkerson
aaffc4caf0 OSH-Infra: Update labels for chart components
This ps adds more granular node selectors for the charts in osh
infra to match what is currently done in osh

Change-Id: I8957a95053b9fb3ea329fd37ff049cd223a7695d
2018-04-13 08:44:33 -05:00
Pete Birley
b9336ca613 Helm-Toolkit: Kubernetes Entrypoint, simplify image dependencies
This PS simplify the logic for dyanmicly merging the image management
depenencies into pod deps when active.

Change-Id: I0cf6c93173bc5fbce697ac15be8697d3b1326d0a
2018-04-13 08:42:37 -05:00
Sean Eagan
db15b5e30b Support pod dependencies
Adds support for a new feature of kubernetes-entrypoint, pod
dependencies, that was added in v0.3.0.

Change-Id: I78d9e0545ca3b837cd2386783386a253f7f5a2d6
2018-03-20 10:53:53 -05:00
Zuul
33cb0e8433 Merge "Revert Elasticsearch/Kibana image change" 2018-03-12 19:13:34 +00:00
Steve Wilkerson
8e4da9da55 Revert Elasticsearch/Kibana image change
This reverts the changes made to Elasticsearch, Kibana and fluent
logging charts in https://review.openstack.org/#/c/550229/7.

Specifically, this moves the images back to previous used versions
and makes the required changes to the fluent-logging elasticsearch
template job to include the correct mapping directives for the
elasticsearch template.

This change was made to give more time for evaluating a more
robust solution for switching to the official upstream images that
will not cause intermittent gate failures as seen since 550229 was
merged

Change-Id: I9f70b3412a8edc5cb1d80937b158aa2fe7b1ec82
2018-03-12 10:27:35 -05:00
Zuul
eb3cbf0f95 Merge "yaml cleanup: trim multiline strings" 2018-03-10 07:01:35 +00:00
Zuul
404ce5b01d Merge "Elasticsearch: use endpoints section and lookups to set port" 2018-03-09 01:24:32 +00:00
Zuul
0dfdaf52c2 Merge "Remove Api version requirement for CronJob Curator" 2018-03-08 23:11:48 +00:00
Chris Wedgwood
3a8c00764c yaml cleanup: trim multiline strings
Change-Id: I7e8f423be2efb84f3116258beca805265ca388f7
2018-03-08 20:18:53 +00:00
Steve Wilkerson
3f44f4586a Elasticsearch: use endpoints section and lookups to set port
This PS moves elasticsearch to use the endpoints section and
lookups to set the port it serves on.

Change-Id: I4a73893124b6d988cd1f885cfc3dd62abeb4ae8c
2018-03-08 20:00:54 +00:00
Steve Wilkerson
d681396412 Address errors with Elasticsearch and Kibana
This moves Elasticsearch and Kibana to use the latest version
(6.2.2), as the images we were using are no longer supported with
the 6.x release.  There was a change in the doc reference in the
log entries that prevented the previous ES version from indexing
those entries, resulting in a busted gate.  Moving Kibana to 6.2.2
was required to match major/minor versions with Elasticsearch

The Elasticsearch version change also required changing config file
locations, changing the entrypoint used for launching the service,
changing the running user for the elasticsearch service, and
updated the ES tests as some of the API responses changed between
versions

This also required updating the elasticsearch template job as the
mapping definition entries changed between versions

Change-Id: Ia4cd9a66851754a1bb8f225c7e24513c43568e93
2018-03-08 10:27:06 -06:00
Hemanth Nakkina
5392ac810b Remove Api version requirement for CronJob Curator
Curator job in Elasticsearch helm chart has a condition on
api version batch/v2alpha1. Cronjob resource is deprecated
in batch/v2alpha1 from k8s 1.8 and batch/v1beta1 is enabled
by default.

Remove the condition on API version as it is no more required.

Closes-Bug: #1753524
Change-Id: Ia296b3742e655fae508e5d4402e7f3881db31688
2018-03-06 08:51:08 +05:30
Zuul
d6e9fa2c5a Merge "Add clusterrolebinding and clusterrole for elasticsearch-master" 2018-03-01 10:21:11 +00:00
Pete Birley
3c101a6324 dependencies: move dynamic common deps under a 'dynamic.common' key
This PS moves existing dynamic common dependencies under a
'dynamic.common' key to simplify the yaml tree.

Change-Id: I4332bcfdf11197488e7bd5d8cf4c25565ea1c7b6
2018-02-24 17:42:10 -05:00
Pete Birley
e0c688d7ee dependencies: move static dependencies under a 'static' key
This PS moves static dependencies unser a 'static' key to allow
expansion to cover dynamic dependencies.

Change-Id: Ia0e853564955e0fbbe5a9e91a8b8924c703b1b02
2018-02-24 17:39:55 -05:00
Steve Wilkerson
94d6e2899e Add clusterrolebinding and clusterrole for elasticsearch-master
The template for elasticsearch-master was missing the clusterrole
and clusterrolebinding. This adds them to bring it in line with
the other templates

Change-Id: I34bc7e889018411b3791c1b7f24d150e1f6a24e5
2018-02-23 10:51:34 -06:00
portdirect
515494ca98 RBAC: Include release name in cluster roles to prevent collision
This PS includes the release name in the cluster role to prevent
colision if the chart is deployed multiple times in the same
cluster.

Change-Id: I7166e5ee25b3d4c89879393c5f84c869585a2681
2018-02-19 13:13:56 -06:00
Zuul
f808c525f8 Merge "Add deep merge utility to helm-toolkit" 2018-02-14 01:29:27 +00:00