45 Commits

Author SHA1 Message Date
Sreejith Punnapuzha
7314edc57d Fix Grafana helm test exception error
Grafana helm test is failing with the below error
"NameError: name 'exception' is not defined"
This is because exception is defined in smaller case. changing
exception to Exception fixes this issue

Change-Id: I533ae822babb4f063242fee1cd42b5b821519b5f
Signed-off-by: Sreejith Punnapuzha <Sreejith.Punnapuzha@outlook.com>
2019-07-08 14:02:42 +00:00
Steve Wilkerson
c7290b7ffe Grafana: Remove tests that query API
This removes the tests that query the Grafana API for checking
whether the prometheus datasource has been provisioned and for
checking the number of active dashboards against the number of
expected dashboards determined via the chart's values.yaml.

The reason for removing these is that Grafana can be configured
to use data source types beyond just Prometheus and additional
dashboards can be added to Grafana via the Grafana UI.  In cases
where dashboards are added via the Grafana UI, they are persisted
in the grafana database which will cause helm test failures during
upgrade scenarios.  Now that we have selenium tests executed as
part of the Grafana helm tests that validate Grafana is
functional, these API tests add little value

Change-Id: I9f20ca28e9c840fb3f4fa0707a43c9419fafa2c1
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-26 09:25:47 -05:00
Steve Wilkerson
b4b1dd9528 Add missing affinity keys to chart pod specs
This adds the affinity key to the pod spec for the grafana,
nagios, kube-state-metrics, and openstack-exporter charts as it
was previously missed

Change-Id: Ifefa88d7f33607b4d595effa5fbf72f3387e5081
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-13 19:15:42 +00:00
Steve Wilkerson
25e105f26a Grafana: Add Selenium tests to helm test pod
This adds selenium tests for the grafana chart to the helm test
pod to help ensure the Grafana deployment is functional and
accessible

Change-Id: Idc8d97e5111628d1ed4f25145086d54c5e0136e7
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-05 16:09:04 +00:00
Tin Lam
6bae1b020d Fix python template
This patch set removes an unused import that is not python3 compatible.

Change-Id: I360989c8eb23065d8e655d4583eb97338244412d
Signed-off-by: Tin Lam <tin@irrational.io>
2019-05-26 21:43:19 +00:00
Zuul
9388c2ba5a Merge "Grafana: Add security context to chart and read-only-fs" 2019-05-13 21:45:01 +00:00
Rahul Khiyani
d8ca55c685 Logs format standardization for LMA component
Added file name, line number and function name to logging message format
for troubleshooting purpose
    - This change is related to Grafana's session-db-sync job

Change-Id: Iaadbedfda0fd9cd7fe4b5c09fc05cb6181c400d1
2019-05-06 21:13:27 +00:00
RAHUL KHIYANI
e1c9a35230 Grafana: Add security context to chart and read-only-fs
This PS adds the security context macros to the grafana chart,
and moves the default to read-only-rootfs for all containers

Change-Id: Ie79e3bfc6af07b16cd53eddae17eceac3d9f8613
2019-04-23 03:22:21 +00:00
Pete Birley
2abf62ff4d OSH-Infra: Add emptydirs for tmp
This PS adds emptydirs backing the /tmp directory in pods, which
is required in most cases for full operation when using a read only
filesystem backing the container.

Additionally some yaml indent issues are resolved.

Change-Id: I8b7f1614da059783254aa6efc09facf23fca3cad
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-20 20:50:59 +00:00
Steve Wilkerson
84f30ec103 Add release-annotation to pod spec, add missing annotations
This adds the release-annotation to the pod spec for the charts in
openstack-helm-infra. This also adds missing configmap annotations
to charts in openstack-helm-infra

Change-Id: Ie23f0c16a7a21d3929e98928db2bbcef69ae6490
2019-03-21 09:10:48 -05:00
Rahul Khiyani
33897b9a01 Grafana securityContext
securityContext with readOnlyRootFilesystem is implemented at container
level and leveraged the helm-toolkit snippet

Change-Id: I98ca4211e0e236beb3dfe0e11cf5bb10a91b16a6
2019-03-18 14:07:52 +00:00
Rahul Khiyani
28e0493955 readOnlyRootFilesystem: true for Grafana chart
Fix for adding readOnlyRootFilesystem flag at pod
level

Change-Id: Icf0244ca0e5c5eb1b96b17e8b7a64819d1093f0d
2019-03-07 23:05:25 +00:00
Steve Wilkerson
65ce9c73d7 Grafana: Add job to update admin password
This change adds a job to the Grafana chart that  allows for the
changing of the grafana admin user password if required, as
Grafana only allows the changing of this password via the
grafana-admin CLI or via an http call that requires both the old
and new password

Change-Id: I59a5d26edc4aa4da16e80c5454ecdebbae3a1d15
2019-02-12 09:59:45 -06:00
Steve Wilkerson
bf5840fa7a Grafana: Add container security context
This adds the container security context to grafana, which
explicitly sets allowPrivilegeEscalation to false

Change-Id: I3723a0c96699b9a517dafa2df08bf8cc916bf117
2019-01-03 16:19:03 -06:00
Steve Wilkerson
680f920312 Grafana: Add pod security context for grafana user
This updates the Grafana chart to include the pod security context
on the grafana pod. This changes the pod's user from root to the
grafana user instead

Change-Id: Id64853640f1941001b83566865defe93227b4291
2019-01-03 12:42:52 -06:00
Tin Lam
92e68d33ea Add network policy toolkit function
This patch set implements the helm toolkit function to generate a
kubernetes network policy manifest based on overrideable values.
This also adds a chart that shuts down all the ingress and egress
traffics in the namespace. This can be used to ensure the
whitelisted network policy works as intended.

Additionally, implementation is done for some infrastructure charts.

Change-Id: I78e87ef3276e948ae4dd2eb462b4b8012251c8c8
Co-Authored-By: Mike Pham <tp6510@att.com>
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-15 13:50:50 +00:00
Steve Wilkerson
bfa237d347 Charts: Update helm test pod templates
This updates the helm test pod templates in the charts with helm
tests defined. This change includes the addition of:

- Generate test pod cluster roles and role bindings
- Generate service accounts for test pods
- Add node selectors to the test pods
- Add service accounts to the test pods
- Addition of entrypoint container to the test pods
- Indentation fix for rabbitmq test pod template

Change-Id: I9a0dd8a1a87bfe5eaf1362e92b37bc004f9c2cdb
2018-10-09 21:00:00 +00:00
Pete Birley
bb3ff98d53 Add release uuid to pods and rc objects
This PS adds the ability to attach a release uuid to pods and rc
objects as desired. A follow up ps will add the ability to add arbitary
annotations to the same objects.

Change-Id: Iceedba457a03387f6fc44eb763a00fd57f9d84a5
Signed-off-by: Pete Birley <pete@port.direct>
2018-09-13 05:35:35 +00:00
Steve Wilkerson
9a311475ba Charts: Use secrets for configs in chart
This updates the osh-infra charts to use a secret for their
configuration files instead of a configmap, allowing for the
storage of sensitive information

Change-Id: Ia32587162288df0b297c45fd43b55cef381cb064
2018-08-24 15:56:53 -05:00
Steve Wilkerson
8652e14acb Add auth for prometheus
This adds authentication to Prometheus with an apache reverse
proxy, similar to elasticsearch, kibana and nagios. This adds an
admin user and password via htpasswd along with adding ldap
support.

This required modifying the grafana chart to configure the
prometheus datasource's basic auth credentials in the data sources
provisioning configuration file by checking whether basic auth is
enabled and injecting the username/password defined in the
corresponding endpoint definition.

This also modifies the nagios chart to use the authenticated
endpoint for prometheus, which is required for nagios to
successfully query the prometheus endpoint for its service
checking mechanism

Change-Id: Ia4ccc3c44a89b2c56594be1f4cc28ac07169bf8c
2018-08-08 18:49:45 +00:00
Seungkyu Ahn
a430533e6a Quoting node_select_value in Ingress Controller
In most cases, the ingress controller's nodeSelector key and value
are "node-role.kubernetes.io/ingress" and "true".
Using quote to treat the nodeSelector value as a string.

Change-Id: Ie1745629b90795e4d888d85f35565e6d6350e09b
2018-08-01 02:39:05 +00:00
Steve Wilkerson
dc16a897d7 Add missing labels to helm test pods
This adds missing labels to the helm test pods in osh-infra

Change-Id: I618d9089bfde2d847411f5f876f0ff6afd9cce7f
2018-07-10 08:55:40 -05:00
Steve Wilkerson
c26a1b53f6 Update TLS secret templates, remove nagios readiness probe
This updates the TLS secret templates to include the backend
service in the dict supplied to the manifest template, as it is
required for the TLS secret to render correctly.

This also removes the readiness probe from the nagios container in
the deployment for the nagios chart, as it wasn't functioning as
intended due to the port not being available for the probe

Change-Id: Iabcfd40c74938e0497d08ffeeebc98ab722fa660
2018-06-27 18:56:45 -05:00
Zuul
714bc3e6da Merge "Ingress: Add initial TLS Support for osh-infra public endpoints" 2018-06-26 23:07:28 +00:00
Steve Wilkerson
b823954787 Ingress: Add initial TLS Support for osh-infra public endpoints
Adds support for TLS on overriden fqdns for public endpoints for
the services that have them in openstack-helm-infra. Currently this
implementation is limited, in that it does not provide support for
dynamically loading CAs into the containers, or specifying them manually
via configuration. As a result only well known or CA's added manually
to containers will be recognised.

Change-Id: I4ab4bbe24b6544b64cd365467e8efb2a421ac3f4
2018-06-26 14:47:19 -05:00
Steve Wilkerson
68fa1d6fbe Grafana: Provision data sources via dynamic template in values
This moves to define the datasources provisioned by grafana via
a template defined in the values.yaml. This allows us to define
multiple datasource types that can be mapped directly to the
corresponding entries in endpoints, which enables us to generate
the data source urls via endpoint lookups rather than hardcoding
this. This is the first step to support multiple data sources in
a singular grafana deployment

Change-Id: Iac7f4b1e07aaf83ae4d2a0c923cd06817f0d8c0d
2018-06-26 13:57:46 -05:00
Steve Wilkerson
497959371d Grafana: Update LDAP configuration, update volume mounts
This updates the LDAP configuration for grafana, using a template
defined in the values.yaml file. Using the template allows us to
dynamically define LDAP configuration values, such as the bind dn,
search base and group search base paths, the password, and the
LDAP fqdn.  This also updates the volume mount for the
provisioning directory to be defined by the configuration value in
the values.yaml file

Change-Id: I1e4866d1189cf40b08b3443dc725646a1b76094c
2018-06-26 07:36:15 -05:00
Pete Birley
abb00e97fd Gotpl: remove quote and trunc to suppress output
This PS removes the use of the `quote and truncate` approach to
suppress output from gotpl actions in templates and replaces it
with the recommended practice of defining `$_` instead.

Change-Id: I5fedc3471dcbecef37d2fe1302bf9760b3163467
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-16 16:37:08 -05:00
Pete Birley
fa629cdbbd Daemonsets: Use current kubernetes daemonset api version
This PS moves to use the current ga version for kubernetes daemonsets,
additionally any remaining deployments that were using the
`extensions/v1beta1` have been updated to `apps/v1`.

Story: 2002205
Task: 21735

Change-Id: If9703162dc472af1e6096bf2b9062802fd5ce8ab
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-13 21:53:18 +00:00
Steve Wilkerson
de9c46bcfa Charts: Tidy up openstack-helm-infra charts
This moves the charts in openstack-helm-infra closer towards a
standard structure. It addresses multiple deviations, including:
missing resources for init containers, incorrect indents for
disabled resources in some charts, incorrect indents for volumes
and volumemounts added via values, missing resources for some
helm test templates, missing helm-toolkit image functions, and
moving the resource template declarations to be under the image
template declarations

Change-Id: I4834a5d476ef7fc69c5583caacc0229050f20a76
2018-05-21 12:58:22 -07:00
Steve Wilkerson
e081c19fe8 Add ldap support to grafana, update version, add helm tests
This adds ldap support to the grafana chart. This required updating
the version of Grafana to 5.0, as this version allows for using
configuration files to bootstrap the datasources and dashboards
instead of using the grafana http api. This was a necessary change
as using ldap for grafana presented issues trying to create the
datasource via the http api

This also adds a basic helm test for grafana. This test simply
verifies whether the prometheus datasource configured exists and
whether the number of dashboards reported by the admin api matches
the number of dashboards expected

Change-Id: I2e987cb425adba9f909722ffdb25b83f82710c4d
2018-05-15 01:42:04 +00:00
Steve Wilkerson
e166432a98 Add manifest for image_repo_sync job
This ps proposes adding a common template for the image_repo_sync
jobs for consumption by the charts

Change-Id: I48476d1e4fd94bd1b08b13b46983e3d999f8d8ca
2018-04-19 14:10:08 +00:00
Zuul
49e9084679 Merge "OSH-Infra: Update labels for chart components" 2018-04-18 18:47:08 +00:00
Zuul
626b94e0c8 Merge "Helm-Toolkit: Kubernetes Entrypoint, simplify image dependencies" 2018-04-17 15:11:00 +00:00
Steve Wilkerson
7757400edc OSH-infra: move charts to use ingress manifest in htk
This moves all relevant charts in osh-infra to use the htk manifest
template for ingresses, bringing them in line with the charts in
openstack-helm

Change-Id: Ic9c3cc6f0051fa66b6f88ec2b2725698b36ce824
2018-04-13 15:41:12 -05:00
Steve Wilkerson
aaffc4caf0 OSH-Infra: Update labels for chart components
This ps adds more granular node selectors for the charts in osh
infra to match what is currently done in osh

Change-Id: I8957a95053b9fb3ea329fd37ff049cd223a7695d
2018-04-13 08:44:33 -05:00
Pete Birley
b9336ca613 Helm-Toolkit: Kubernetes Entrypoint, simplify image dependencies
This PS simplify the logic for dyanmicly merging the image management
depenencies into pod deps when active.

Change-Id: I0cf6c93173bc5fbce697ac15be8697d3b1326d0a
2018-04-13 08:42:37 -05:00
Zuul
eb3cbf0f95 Merge "yaml cleanup: trim multiline strings" 2018-03-10 07:01:35 +00:00
Chris Wedgwood
3a8c00764c yaml cleanup: trim multiline strings
Change-Id: I7e8f423be2efb84f3116258beca805265ca388f7
2018-03-08 20:18:53 +00:00
Steve Wilkerson
934bef2f0d Grafana: use endpoints section and lookups to set port
This PS moves grafana to use the endpoints section and lookups to
set the port it serves on.

Change-Id: I51d4c10297f3423569539dcbf7fbecad24d5e47f
2018-03-08 20:01:05 +00:00
Pete Birley
3c101a6324 dependencies: move dynamic common deps under a 'dynamic.common' key
This PS moves existing dynamic common dependencies under a
'dynamic.common' key to simplify the yaml tree.

Change-Id: I4332bcfdf11197488e7bd5d8cf4c25565ea1c7b6
2018-02-24 17:42:10 -05:00
Pete Birley
e0c688d7ee dependencies: move static dependencies under a 'static' key
This PS moves static dependencies unser a 'static' key to allow
expansion to cover dynamic dependencies.

Change-Id: Ia0e853564955e0fbbe5a9e91a8b8924c703b1b02
2018-02-24 17:39:55 -05:00
Sean Eagan
641c79c902 Add deep merge utility to helm-toolkit
Adds "helm-toolkit.utils.merge" which is a replacement for the
upstream sprig "merge" function which didn't quite do what we
wanted, specifically it didn't merge slices, it just overrode
one with the other.  This PS also updates existing callsites
of the sprig merge with "helm-toolkit.utils.merge".

Change-Id: I456349558d4cf941d1bcb07fc76d0688b0a10782
2018-02-13 10:08:50 -06:00
portdirect
21dc4f8af8 Grafana: support multiple replicas and data persistance
This PS adds support for multiple replicas of Grafana to be
deployed, and adds MySQL based data persistance to the chart.

Change-Id: Ife44985a6d5024cc2074346340fba1d8efdecbfa
2018-01-11 02:39:34 -05:00
Steve Wilkerson
a1f608ed74 Add Grafana chart to OSH infra
Moves the grafana chart to OSH infra along with basic rbac rules
that may be tightened with future work.

Change-Id: Ie14627530a73d4b7b01eb93ca5f7174d99d9caec
2018-01-04 09:10:55 -06:00