307 Commits

Author SHA1 Message Date
Zuul
8691034b6d Merge "Update prometheus metrics gathering in post-job" 2018-02-07 23:04:15 +00:00
Steve Wilkerson
094e0103a9 Add elasticsearch exporter
This adds a chart for an elasticsearch exporter to expose metrics
for prometheus. This also moves the exporter to be included as
part of the chart it's meant to target as opposed to its own
separate chart

Change-Id: I491f4d1efba633827d8a6255218daeb9d427f922
2018-02-07 10:59:27 -06:00
Zuul
27c46b5603 Merge "Provide additional flexibility for the calico chart" 2018-02-06 22:10:21 +00:00
Steve Wilkerson
6af03cd6d9 Update prometheus metrics gathering in post-job
This updates the prometheus metrics gathering role to dynamically
detect deployed namespaces instead of using hardcoded tasks for
each namespace. This allows the use of this job in openstack-helm
without needing to manually add additional namespaces

Change-Id: I2304aceab8dcda5471af708f57924193f03ee8f3
2018-02-05 18:21:49 -06:00
Tin Lam
ae54ad7cd0 Fix namespace label
This patch set fixes the namespace labels and adds in a new log file to
describe all namespaces to troubleshoot gate issues where labels are not
applied properly.

Change-Id: I34c43c345f9a49df8d7fcf2e7824220a22698d46
Signed-off-by: Tin Lam <tin@irrational.io>
2018-02-05 15:43:40 -06:00
Zuul
266e96abbc Merge "Update kibana image to 5.6.4 tag" 2018-02-04 16:54:26 +00:00
Zuul
d5c6d42ecc Merge "Disable Elasticsearch NFS snapshot repository by default" 2018-02-04 16:54:25 +00:00
James E. Blair
e60bce18f7 Zuul: Remove project name
Zuul no longer requires the project-name for in-repo configuration.
Omitting it makes forking or renaming projects easier.

Change-Id: I35277bfa2de254337854c6058e32f91ac1d621c8
2018-02-02 08:50:55 -08:00
Steve Wilkerson
e04be06ee3 Update kibana image to 5.6.4 tag
Updates the kibana version to match the elasticsearch version

Change-Id: I4ba2410ebf00ce2b269806f46f2c0a14652b71dc
2018-02-01 14:26:48 -06:00
Zuul
cf02db5570 Merge "Add node selectors to elasticsearch chart" 2018-02-01 20:22:28 +00:00
Zuul
61792ba9af Merge "Remove helm_tests image key from openstack-exporter" 2018-02-01 15:54:56 +00:00
Zuul
5ab1e126d8 Merge "Rollback helm version to 2.7.2" 2018-02-01 15:23:29 +00:00
portdirect
2d725fd00b Rollback helm version to 2.7.2
The update to helm 2.8.0 resulted in issues with releases not
registering as ready before timing out. This rolls the version back
until those issues are addressed

Change-Id: Id58f85f150054d82c047bd360258ebe9e571360f
2018-01-31 21:28:58 +00:00
Steve Wilkerson
220310bd2f Add node selectors to elasticsearch chart
This adds node selectors to all templates in the elasticsearch
chart, as they were previously missed

Change-Id: I34ea5751663b2e993c5f73a78a1f91133919752c
2018-01-31 15:19:28 -06:00
Steve Wilkerson
b15d0ed0d2 Fix alertmanager volumemount
The volume mount name for the permissions init container was
overlooked when the PVC for alertmanager was removed and the
volume renamed. This changes the mount appropriately

Change-Id: I5db6594a3192ec78354e5f3d3d41e96317488664
2018-01-30 19:57:11 -06:00
Zuul
1fa00dec4a Merge "Alertmanager: Use volumeclaimtemplate for storage" 2018-01-30 23:52:47 +00:00
Steve Wilkerson
fa5fbd0e1f Remove helm_tests image key from openstack-exporter
The openstack-exporter doesn't have helm tests currently, so there
is no need for a helm_tests image key.

Change-Id: I7d1f737170a343e4ec66c4d7ace3ba76cdffb329
2018-01-30 16:09:48 -06:00
Zuul
3036dd24c6 Merge "kube-state-metrics for kubernetes version 1.8" 2018-01-30 17:37:52 +00:00
Zuul
0cdee45661 Merge "K8s/Helm: Bump versions to current release" 2018-01-30 17:18:34 +00:00
Alan Meadows
c10713bde9 Provide additional flexibility for the calico chart
* Ingests the bird templates so that we can override them
to support things such as custom BGP ports (listen) and
neighbors (remote)

* Supports announcing addresses that are within the
.Values.networking.bgp.ipv4|6.additional_cidrs list
in support of ingress controllers that can create
dummy interfaces and assign addresses to be announced

* Introduces a new job to perform calicoctl manipulation to
support manipulating the mesh, adding peers, and changing the
ipPool settings which is value driven

* Support custom port binding and specific interface binding
to allow custom BGP port selection for IPv4 and IPv6

* Instantiates calicoctl as a utility on hosts

* Adds a new function to helm-toolkit to retrieve the http
or https prefix for an endpoint

* Supports https based etcd backends with new certificate
parameters

* Finally, introduces more strict bgp listening to allow
multiple hostNet bgp speakers to run in parallel

Change-Id: Ib4d00befddbd8498b9dcc693409b8b2577458497
2018-01-30 08:43:04 -08:00
Zuul
8971ba79ff Merge "Remove kibana template in elasticsearch templates" 2018-01-30 16:25:01 +00:00
Zuul
713ce66963 Merge "Prometheus: Use volume claims to support RWO storage class" 2018-01-30 16:15:29 +00:00
Steve Wilkerson
977c561a8f Alertmanager: Use volumeclaimtemplate for storage
This removes the pvc in Alertmanager and changes the default
storage_class to readwriteonce.  Now that Alertmanager uses peer
meshing, it's not required for the replicas to share a common
volume claim

Change-Id: I24290264cb0e552a143a56faa753289f073c47b9
2018-01-30 09:32:49 -06:00
Steve Wilkerson
012f5ec8d0 Disable Elasticsearch NFS snapshot repository by default
This disables the Elasticsearch snapshot repository backed by
NFS by default as the curator job for snapshots is disabled by
default, and should make no assumption that NFS is deployed by
default

Change-Id: Idc74cfb80fcb4c4741d82c6d0ce63fd90a8c919f
2018-01-29 16:25:27 +00:00
Steve Wilkerson
0bfc5bd12f Remove kibana template in elasticsearch templates
The _kibana.sh.tpl template ended up in the elasticsearch chart
templates folder. This removes it

Change-Id: I636d5b716c266d7ccd266a5f9c051a6eec56e3e1
2018-01-29 13:04:56 +00:00
Steve Wilkerson
8a523c0afd Tune prometheus default intervals and block durations
This increases the default scrape and evaluation intervals to match
those of the prometheus service upstream with the aim to reduce
prometheus's resource consumption. It also adds configuration
parameters for the min and max block durations that series can
span

Change-Id: I7f9352413a273fbf680b892ba26e30cf27bae232
2018-01-29 13:04:34 +00:00
Siri Kim
74db52a349 Prometheus: Use volume claims to support RWO storage class
This PS uses volumeClaimTemplates to provide multiple
pvcs and pvs to prometheus statefulset's multiple pods.
This gives ability to provide read-write-many access
backend by a read-write-once storage class.

Change-Id: I53d1b866c0c87f0833941b612d3acfbe5742744f
2018-01-29 05:46:46 +00:00
Siri Kim
d9d2ba547a kube-state-metrics for kubernetes version 1.8
This PS is kube-state-metrics for kubernetes version 1.8.
Using kube-state-metrics:v1.2.0 image makes kube-state-
metric pod work properly. Also, gives authority to list
endpoints, persistentvolumes, and horizontalpodautoscalers
by adding them to clusterrole.

Change-Id: I705b29c321b0162740744afa8573dc6ae75bcc60
2018-01-29 05:45:57 +00:00
portdirect
27a2b2e16d K8s/Helm: Bump versions to current release
This PS bumps the versions of helm and K8s to their current GA
release.

Change-Id: Ia205f47d4f6b89d6d54bb2a71fde28a5730d2a47
2018-01-28 23:34:52 -05:00
Steve Wilkerson
3ec7f5f0ff Gate fix: httpd image, elasticsearch, openstack-exporter
There was a change in the upstream reference httpd image for
apache that changed how modules were built for apache.
This change adds the required fix to accomodate the change.
See isssue here https://github.com/docker-library/httpd/pull/87

The Elasticsearch image tag was updated to accomodate the kernel
versions used in the gate as part of the kernel update playbook
See https://github.com/elastic/elasticsearch/issues/28349#issuecomment-360233779

The openstack-exporter binary was changed to reflect changes made
to the openstack-exporter image

Change-Id: I1deb9e7cde794421dd33fade566c2a9fdb5007e6
2018-01-28 15:07:24 -06:00
Zuul
b380b68bba Merge "Gate: collect host level info" 2018-01-20 21:29:05 +00:00
Zuul
0cb4cfbd2e Merge "Change flush interval for fluentbit" 2018-01-19 18:35:57 +00:00
Zuul
4844e2f312 Merge "Update Ceph dashboards to use ceph-mgr prometheus module" 2018-01-19 18:35:56 +00:00
Zuul
2675d25f6d Merge "helm-toolkit prometheus service annotation clean up" 2018-01-19 18:35:55 +00:00
Zuul
2b7a708761 Merge "Prometheus: Disable prometheus enable-admin-api by default" 2018-01-19 18:35:54 +00:00
Zuul
dd4a33529f Merge "kube-state-metrics: remove unused replica key" 2018-01-19 17:52:02 +00:00
Zuul
08a6266654 Merge "Add label to namespaces" 2018-01-19 14:50:08 +00:00
Steve Wilkerson
a9320d4acc kube-state-metrics: remove unused replica key
the prometheus-kube-state-metrics/values.yaml file had a key for
prometheus replicas, which was likely a result of copy/paste

Change-Id: Id5b915c3814f9caa313c16dfbca7796e7f8284e2
2018-01-18 09:55:52 -06:00
Steve Wilkerson
f4257cdc23 Prometheus: Disable prometheus enable-admin-api by default
This disables the Prometheus admin http api by default to fall in
line with the service defaults, as enabling this exposes endpoints
for deleting and modifying time series over http.

Change-Id: I797dce32d625c4c8e927a6b0649dbba0db7f6905
2018-01-18 09:23:24 -06:00
portdirect
6a150bd0f9 KubeADM: Increase pod limit for hosts
This PS increases the pod limit per hosts - useful in dev envs.

Change-Id: I3bdb933d7f5001cb91db6f7f227101e587532a54
2018-01-17 18:57:12 -05:00
Steve Wilkerson
4075336dfb Change flush interval for fluentbit
The default flush interval for fluentbit should be set to the
service's default value (5s) rather than flushing the buffer every
second

Change-Id: I9a77d42681af4c59e383553a5f3716afc372bedd
2018-01-17 17:44:49 -06:00
Tin Lam
f6347e7d28 Add label to namespaces
This patch set adds label to "kube-system" and "default"
namespaces used for podSelector.

Change-Id: Ia67a0e4d9adf6f5575d74aebf77673aae3660c62
Signed-off-by: Tin Lam <tin@irrational.io>
2018-01-17 16:08:24 -06:00
Zuul
c1a8572584 Merge "KubeADM-AIO: Move to upstream debian base image" 2018-01-17 08:03:29 +00:00
Steve Wilkerson
9ffc748979 helm-toolkit prometheus service annotation clean up
This adds checks for the fields in the service annotations for
prometheus, similar to the checks made for the pod annotations.
It also moves prometheus annotations under a prometheus: key
under a top-level monitoring tree to allow for other monitoring
mechanisms independent of the endpoints tree

Change-Id: I4be6d6ad8e74e8ca52bd224ceddad785577bf6c7
2018-01-16 20:35:50 +00:00
Steve Wilkerson
b63afdd10c Run kibana behind apache
Run kibana behind apache as a reverse proxy to supply basic auth
for kibana, as xpack requires a suscription to support security
for kibana

Change-Id: I82168fc47fad29e26bcb02964709a04200dac467
2018-01-16 08:15:06 -06:00
Steve Wilkerson
d197c4f9a2 Run elasticsearch behind apache
Run elasticsearch behind apache as a reverse proxy to supply basic
auth for elasticsearch, as xpack requires a suscription to support
security for elasticsearch

Change-Id: I72d06ed9cd2179ead86ddc67db33c68a1e40c437
2018-01-16 08:14:47 -06:00
Steve Wilkerson
67f2a8ce4f Update Ceph dashboards to use ceph-mgr prometheus module
This updates the ceph dashboards in grafana to use the metrics
provided by the ceph-mgr prometheus module instead of the digital
ocean ceph exporter

Change-Id: I449bc924034f0beabca3c6e8aab17765a706fd50
2018-01-15 23:59:34 +00:00
portdirect
0025eb9444 Gate: collect host level info
This PS adds basic host level log collection to the gate.

Change-Id: I5ee3905e134b2d9abdad08121ecb7e257c7165c8
2018-01-14 17:41:25 -05:00
portdirect
9b40b8656d Prometheus Openstack Exporter: tidy chart and add ks user
This PS adds keystone user management to the prometheus-openstack-exporter
chart, and also performs some spring cleaning.

Change-Id: I69e40c523867f751ecd8c63169aefdfdf4eb5cd2
2018-01-14 09:51:06 -05:00
portdirect
76d0f57e64 KubeADM-AIO: Move to upstream debian base image
This PS moves KubeADM-AIO to use the upstream debian base image
that is used to build the Kubernetes release images:
 * https://github.com/kubernetes/kubernetes/tree/master/build/debian-base

Change-Id: I153a3aee43613b429793c179b2707c916821d39a
2018-01-13 23:46:33 -05:00