389 Commits

Author SHA1 Message Date
Zuul
0cdee45661 Merge "K8s/Helm: Bump versions to current release" 2018-01-30 17:18:34 +00:00
Alan Meadows
c10713bde9 Provide additional flexibility for the calico chart
* Ingests the bird templates so that we can override them
to support things such as custom BGP ports (listen) and
neighbors (remote)

* Supports announcing addresses that are within the
.Values.networking.bgp.ipv4|6.additional_cidrs list
in support of ingress controllers that can create
dummy interfaces and assign addresses to be announced

* Introduces a new job to perform calicoctl manipulation to
support manipulating the mesh, adding peers, and changing the
ipPool settings which is value driven

* Support custom port binding and specific interface binding
to allow custom BGP port selection for IPv4 and IPv6

* Instantiates calicoctl as a utility on hosts

* Adds a new function to helm-toolkit to retrieve the http
or https prefix for an endpoint

* Supports https based etcd backends with new certificate
parameters

* Finally, introduces more strict bgp listening to allow
multiple hostNet bgp speakers to run in parallel

Change-Id: Ib4d00befddbd8498b9dcc693409b8b2577458497
2018-01-30 08:43:04 -08:00
Zuul
8971ba79ff Merge "Remove kibana template in elasticsearch templates" 2018-01-30 16:25:01 +00:00
Zuul
713ce66963 Merge "Prometheus: Use volume claims to support RWO storage class" 2018-01-30 16:15:29 +00:00
Steve Wilkerson
977c561a8f Alertmanager: Use volumeclaimtemplate for storage
This removes the pvc in Alertmanager and changes the default
storage_class to readwriteonce.  Now that Alertmanager uses peer
meshing, it's not required for the replicas to share a common
volume claim

Change-Id: I24290264cb0e552a143a56faa753289f073c47b9
2018-01-30 09:32:49 -06:00
Steve Wilkerson
012f5ec8d0 Disable Elasticsearch NFS snapshot repository by default
This disables the Elasticsearch snapshot repository backed by
NFS by default as the curator job for snapshots is disabled by
default, and should make no assumption that NFS is deployed by
default

Change-Id: Idc74cfb80fcb4c4741d82c6d0ce63fd90a8c919f
2018-01-29 16:25:27 +00:00
Steve Wilkerson
0bfc5bd12f Remove kibana template in elasticsearch templates
The _kibana.sh.tpl template ended up in the elasticsearch chart
templates folder. This removes it

Change-Id: I636d5b716c266d7ccd266a5f9c051a6eec56e3e1
2018-01-29 13:04:56 +00:00
Steve Wilkerson
8a523c0afd Tune prometheus default intervals and block durations
This increases the default scrape and evaluation intervals to match
those of the prometheus service upstream with the aim to reduce
prometheus's resource consumption. It also adds configuration
parameters for the min and max block durations that series can
span

Change-Id: I7f9352413a273fbf680b892ba26e30cf27bae232
2018-01-29 13:04:34 +00:00
Siri Kim
74db52a349 Prometheus: Use volume claims to support RWO storage class
This PS uses volumeClaimTemplates to provide multiple
pvcs and pvs to prometheus statefulset's multiple pods.
This gives ability to provide read-write-many access
backend by a read-write-once storage class.

Change-Id: I53d1b866c0c87f0833941b612d3acfbe5742744f
2018-01-29 05:46:46 +00:00
Siri Kim
d9d2ba547a kube-state-metrics for kubernetes version 1.8
This PS is kube-state-metrics for kubernetes version 1.8.
Using kube-state-metrics:v1.2.0 image makes kube-state-
metric pod work properly. Also, gives authority to list
endpoints, persistentvolumes, and horizontalpodautoscalers
by adding them to clusterrole.

Change-Id: I705b29c321b0162740744afa8573dc6ae75bcc60
2018-01-29 05:45:57 +00:00
portdirect
27a2b2e16d K8s/Helm: Bump versions to current release
This PS bumps the versions of helm and K8s to their current GA
release.

Change-Id: Ia205f47d4f6b89d6d54bb2a71fde28a5730d2a47
2018-01-28 23:34:52 -05:00
Steve Wilkerson
3ec7f5f0ff Gate fix: httpd image, elasticsearch, openstack-exporter
There was a change in the upstream reference httpd image for
apache that changed how modules were built for apache.
This change adds the required fix to accomodate the change.
See isssue here https://github.com/docker-library/httpd/pull/87

The Elasticsearch image tag was updated to accomodate the kernel
versions used in the gate as part of the kernel update playbook
See https://github.com/elastic/elasticsearch/issues/28349#issuecomment-360233779

The openstack-exporter binary was changed to reflect changes made
to the openstack-exporter image

Change-Id: I1deb9e7cde794421dd33fade566c2a9fdb5007e6
2018-01-28 15:07:24 -06:00
Zuul
b380b68bba Merge "Gate: collect host level info" 2018-01-20 21:29:05 +00:00
Zuul
0cb4cfbd2e Merge "Change flush interval for fluentbit" 2018-01-19 18:35:57 +00:00
Zuul
4844e2f312 Merge "Update Ceph dashboards to use ceph-mgr prometheus module" 2018-01-19 18:35:56 +00:00
Zuul
2675d25f6d Merge "helm-toolkit prometheus service annotation clean up" 2018-01-19 18:35:55 +00:00
Zuul
2b7a708761 Merge "Prometheus: Disable prometheus enable-admin-api by default" 2018-01-19 18:35:54 +00:00
Zuul
dd4a33529f Merge "kube-state-metrics: remove unused replica key" 2018-01-19 17:52:02 +00:00
Zuul
08a6266654 Merge "Add label to namespaces" 2018-01-19 14:50:08 +00:00
Steve Wilkerson
a9320d4acc kube-state-metrics: remove unused replica key
the prometheus-kube-state-metrics/values.yaml file had a key for
prometheus replicas, which was likely a result of copy/paste

Change-Id: Id5b915c3814f9caa313c16dfbca7796e7f8284e2
2018-01-18 09:55:52 -06:00
Steve Wilkerson
f4257cdc23 Prometheus: Disable prometheus enable-admin-api by default
This disables the Prometheus admin http api by default to fall in
line with the service defaults, as enabling this exposes endpoints
for deleting and modifying time series over http.

Change-Id: I797dce32d625c4c8e927a6b0649dbba0db7f6905
2018-01-18 09:23:24 -06:00
portdirect
6a150bd0f9 KubeADM: Increase pod limit for hosts
This PS increases the pod limit per hosts - useful in dev envs.

Change-Id: I3bdb933d7f5001cb91db6f7f227101e587532a54
2018-01-17 18:57:12 -05:00
Steve Wilkerson
4075336dfb Change flush interval for fluentbit
The default flush interval for fluentbit should be set to the
service's default value (5s) rather than flushing the buffer every
second

Change-Id: I9a77d42681af4c59e383553a5f3716afc372bedd
2018-01-17 17:44:49 -06:00
Tin Lam
f6347e7d28 Add label to namespaces
This patch set adds label to "kube-system" and "default"
namespaces used for podSelector.

Change-Id: Ia67a0e4d9adf6f5575d74aebf77673aae3660c62
Signed-off-by: Tin Lam <tin@irrational.io>
2018-01-17 16:08:24 -06:00
Zuul
c1a8572584 Merge "KubeADM-AIO: Move to upstream debian base image" 2018-01-17 08:03:29 +00:00
Steve Wilkerson
9ffc748979 helm-toolkit prometheus service annotation clean up
This adds checks for the fields in the service annotations for
prometheus, similar to the checks made for the pod annotations.
It also moves prometheus annotations under a prometheus: key
under a top-level monitoring tree to allow for other monitoring
mechanisms independent of the endpoints tree

Change-Id: I4be6d6ad8e74e8ca52bd224ceddad785577bf6c7
2018-01-16 20:35:50 +00:00
Steve Wilkerson
b63afdd10c Run kibana behind apache
Run kibana behind apache as a reverse proxy to supply basic auth
for kibana, as xpack requires a suscription to support security
for kibana

Change-Id: I82168fc47fad29e26bcb02964709a04200dac467
2018-01-16 08:15:06 -06:00
Steve Wilkerson
d197c4f9a2 Run elasticsearch behind apache
Run elasticsearch behind apache as a reverse proxy to supply basic
auth for elasticsearch, as xpack requires a suscription to support
security for elasticsearch

Change-Id: I72d06ed9cd2179ead86ddc67db33c68a1e40c437
2018-01-16 08:14:47 -06:00
Steve Wilkerson
67f2a8ce4f Update Ceph dashboards to use ceph-mgr prometheus module
This updates the ceph dashboards in grafana to use the metrics
provided by the ceph-mgr prometheus module instead of the digital
ocean ceph exporter

Change-Id: I449bc924034f0beabca3c6e8aab17765a706fd50
2018-01-15 23:59:34 +00:00
portdirect
0025eb9444 Gate: collect host level info
This PS adds basic host level log collection to the gate.

Change-Id: I5ee3905e134b2d9abdad08121ecb7e257c7165c8
2018-01-14 17:41:25 -05:00
portdirect
9b40b8656d Prometheus Openstack Exporter: tidy chart and add ks user
This PS adds keystone user management to the prometheus-openstack-exporter
chart, and also performs some spring cleaning.

Change-Id: I69e40c523867f751ecd8c63169aefdfdf4eb5cd2
2018-01-14 09:51:06 -05:00
portdirect
76d0f57e64 KubeADM-AIO: Move to upstream debian base image
This PS moves KubeADM-AIO to use the upstream debian base image
that is used to build the Kubernetes release images:
 * https://github.com/kubernetes/kubernetes/tree/master/build/debian-base

Change-Id: I153a3aee43613b429793c179b2707c916821d39a
2018-01-13 23:46:33 -05:00
portdirect
217385a0dd Gate: Simplify and speed up pod and k8s object log collection
This PS simplifys and accelerates the collection of pod container
logs and k8s object info in the zuul gates, which can currently take
40 minutes for the 5 node OSH gate to complete.

Change-Id: Ie9b23174fade3df4a87f2b771ea654e2081b4f4e
2018-01-13 21:39:42 -05:00
Zuul
ac53413b47 Merge "Remove unneeded context in prometheus service annotation" 2018-01-13 00:26:56 +00:00
Steve Wilkerson
182c0c5618 Remove unneeded context in prometheus service annotation
Removes an unused context declaration from the prometheus service
annotation template in helm-toolkit, and removes all references to
it

Change-Id: I57612c1504cf046f367ee10d26ef3062ebe528d3
2018-01-12 08:28:48 -06:00
portdirect
286a5de9e2 Gate: Five node nodegroups
This PS adds five node nodegroups to OSH-Infra, that will ultimately
be consumed by the OSH gates.

Change-Id: I212ecdb0ffca1855d5f7e0fbbb2e8a11afb1919c
2018-01-12 03:44:49 +00:00
portdirect
3dfcde8849 KubeADM-AIO: allow cluster domain to be customised
This PS allows the cluster domain to be customised if desired.

Change-Id: I86b9271b6248a36ce39a367b814cffa2bea6d4b3
2018-01-11 16:22:27 +00:00
Zuul
3e68d02fc2 Merge "Grafana: support multiple replicas and data persistance" 2018-01-11 16:02:28 +00:00
portdirect
21dc4f8af8 Grafana: support multiple replicas and data persistance
This PS adds support for multiple replicas of Grafana to be
deployed, and adds MySQL based data persistance to the chart.

Change-Id: Ife44985a6d5024cc2074346340fba1d8efdecbfa
2018-01-11 02:39:34 -05:00
Zuul
1cbc437c82 Merge "Gate: Increase timeouts for awating node to come back from reboot" 2018-01-11 05:03:36 +00:00
Zuul
7379397946 Merge "Elasticsearch Snapshot: Fix permisions for PVC" 2018-01-11 05:03:36 +00:00
Zuul
e239914dd5 Merge "Prometheus-Alertmanager: Fix permisions for PVC" 2018-01-11 04:22:17 +00:00
Zuul
6336a57a64 Merge "Prometheus: Fix permisions for PVC" 2018-01-11 04:19:23 +00:00
portdirect
97d60dcccd Elasticsearch Snapshot: Fix permisions for PVC
This PS fixes the permisions for the PVC backing the
Elasticsearch Snapshot

Change-Id: I7b9897a7e0f34096ce1f2a04aceab7796d3a89c5
2018-01-10 13:40:11 -05:00
portdirect
666f7de6a1 Prometheus-Alertmanager: Fix permisions for PVC
This PS fixes the permisions for the PVC backing
Prometheus-Alertmanager

Change-Id: I8cfb2b999c1f2add9c1647238603c3940ef0bc0a
2018-01-10 13:05:36 -05:00
portdirect
f59d3b0011 Prometheus: Fix permisions for PVC
This PS fixes the permisions for the PVC backing Prometheus

Change-Id: I19b96296fe3553df5b433c22c05cbdfff1b8402a
2018-01-10 12:43:06 -05:00
portdirect
3d633fca7a Gate: Increase timeouts for awating node to come back from reboot
This PS increases the timeout in waiting for the node to come back
online following a reboot.

Change-Id: I92d1e5b665006bf9693f56ad8272330c19e6ccfa
2018-01-10 10:48:02 -05:00
sungil
85011f9c48 Add Permission for k8s plugin in fluent-logging
This PS adds permissions for k8s plugin in fluent-logging.
The k8s plugin in fluentbit gets information per pod and adds it to the
message(log) before output. But the plugin cannot get the pod in current chart.
This PS fix this issue.

Change-Id: Icdce8a0a5ed0975c4d6e72ba50df8ef9a3b76ca6
2018-01-10 11:02:02 +09:00
portdirect
13852ceee4 Gate: Deploy HWE kernel on ubuntu hosts
This PS deploys the HWE kernel on Ubuntu Hosts, which is required
for CephFS:
 * https://github.com/kubernetes-incubator/external-storage/issues/345

Change-Id: I2ebd46eadf5a4c7a857d42302f388511691ab0db
2018-01-09 04:04:18 +00:00
portdirect
abd7e78c65 Fluentd: tidy rbac roles and bindings to live with appropriate rc
This PS brings Fluentd (&bit) inline with other charts by placing the
RBAC roles and bindings in the same template as the pod rc they are
assocated with.

Change-Id: I622a2adfc0dc9f5044202cd6318e3ed803088c5f
2018-01-08 02:17:38 +00:00