112 Commits

Author SHA1 Message Date
Pete Birley
5750d2a01f Gate: update Ansible conventions
Using tests as filters is deprecated, and will be removed in 2.9.

Change-Id: I2bc31177cdb3d59319c4cb04f77db573f3217479
2018-04-18 23:32:35 -05:00
Zuul
6dd5f87a3d Merge "Helm: update helm version to v2.8.2" 2018-04-19 00:22:26 +00:00
Pete Birley
a7da953e39 Helm: update helm version to v2.8.2
This PS updates the helm version in the gate to helm 2.8.2

Change-Id: Ied757f6d653255f198de524f41111cd2928a03dc
2018-04-18 21:55:00 +00:00
Pete Birley
37d836c8c7 Gate/Dev: RPM and structure improvements
This PS restores the use of the distro provided docker for RPM based
distros. It also removes the roles subdirectory from the playbooks
and replaces it with a symlink for local dev use.

Change-Id: Ic11adb85813e92488f5ebbe6714ad7da8d3938e2
2018-04-18 13:56:47 -05:00
Pete Birley
6921006103 Gate: update paths for pip 10 installation
Pip>=10 moves the entrypoint to /usr/local/bin from /usr/bin, this
ps forces the shell to forget all locations following upgrade
to allow it to adapt to the new location.

Change-Id: I1ed92b75f689e982397cd4fc87ac262256e161e8
2018-04-15 16:54:56 +00:00
Steve Wilkerson
9dd81954fd Move roles and playbooks to top level
This proposes moving the playbooks and roles to the top level of
osh-infra to be consumed by osh

Change-Id: I7e3516da8e516060f94b8b9c8875918504de7446
2018-04-09 20:52:18 -05:00
portdirect
59d4141d6a Update to F27 and update docker packages
This PS moves the fedora gates to use F27. It also updates
the docker packages used for both fedora and centos

Change-Id: Ic5ab5793d7176a81a8ea70c82b1d53a1fdf5f740
2018-04-06 17:21:41 +00:00
Steve Wilkerson
99befc2484 Nagios Chart
This adds the nagios chart to osh-infra to provide additional
monitoring functionality. It uses helper functions to consume
yaml definitions for services, commands, hosts and hostgroups
to generate the required configurations for those entities in
nagios's configuration

Change-Id: I6238bb8cb1e5c8dc48594ddea50693f3e7b0a176
2018-03-23 13:45:40 +00:00
Zuul
f461e53e52 Merge "Provide ability to specify proxy" 2018-03-14 14:10:57 +00:00
Paul Carver
e01268bb50 Correcting a spelling error
perfoming -> performing

Change-Id: I639d8a4d0021dbdcd66ff711f86c25f1313a823f
2018-03-12 20:26:30 -04:00
Tin Lam
391bbf69d7 Provide ability to specify proxy
This patch set provides ability to configure the proxy setting in
the ansible playbook used to run the gate.

Change-Id: I7ae66cb16fa6db7d46a0a3f23c289cc625e583c4
Signed-off-by: Tin Lam <tin@irrational.io>
2018-03-12 04:15:30 +00:00
Pete Birley
21a6217c58 Gate: get full process list from host
This PS adds the ps command to the gate log collection task.

Change-Id: Ife1710854547b28a6dfa6b3e4d527ebf4d04179b
2018-03-09 12:45:44 +00:00
Zuul
3a552b4200 Merge "KubeADM: force calico interface to match to same as kubelet" 2018-03-09 00:32:34 +00:00
Pete Birley
c9ffaea770 KubeADM: force calico interface to match to same as kubelet
This PS forces calico to use the same interface as the kubelet
for inter-node communication.

Change-Id: I7d03beec46cccba4a57e092bf108695e8e88996f
2018-03-08 22:20:56 +00:00
Hemanth Nakkina
5392ac810b Remove Api version requirement for CronJob Curator
Curator job in Elasticsearch helm chart has a condition on
api version batch/v2alpha1. Cronjob resource is deprecated
in batch/v2alpha1 from k8s 1.8 and batch/v1beta1 is enabled
by default.

Remove the condition on API version as it is no more required.

Closes-Bug: #1753524
Change-Id: Ia296b3742e655fae508e5d4402e7f3881db31688
2018-03-06 08:51:08 +05:30
Pete Birley
ca3929025b Gate: collect more host level logs
This PS collects more host level logs in the gate scripts.

Change-Id: I0ec13f45fd4561fec59d08b08eb78390a3866156
2018-02-26 07:20:25 +00:00
Pete Birley
1bf13af26c Node-Labels: Add linuxbridge node label to nodes
This PS adds the linxubridge node label to nodes to simplify
deployment and testing of the neutron linuxbridge backend.

Change-Id: I3095c21a75ad3057539f4fed431cb18c68e49a9b
2018-02-26 03:57:49 +00:00
Pete Birley
e0c688d7ee dependencies: move static dependencies under a 'static' key
This PS moves static dependencies unser a 'static' key to allow
expansion to cover dynamic dependencies.

Change-Id: Ia0e853564955e0fbbe5a9e91a8b8924c703b1b02
2018-02-24 17:39:55 -05:00
Steve Wilkerson
4f1f180e8e Add prometheus annotations to tiller-deploy service
This adds the prometheus annotations to the tiller chart, allowing
for the scraping of metrics from tiller.  currently, tiller exports
metrics specific to the grpc calls being made against it

Change-Id: Ibe10098689cc37a9de6306af86f4d63d8aab318b
2018-02-20 10:05:05 -06:00
portdirect
3713266f4c K8s: udpate KubeADM-AIO to use K8s v1.9.3
This PS updates the version of K8s used in the gate to v1.9.3

Change-Id: I284083e0cdf828701c439c44edfd3a975dcd69c3
2018-02-17 15:21:06 -05:00
Steve Wilkerson
5d95b0e2cb Add fluentd prometheus exporter for logging metrics
This adds templates for a prometheus exporter for fluentd to
adequately capture log metrics and fluentd service metrics for
consumption by prometheus

Change-Id: I6d6a8c2be07af819dc6d99b8ce5f1d4b635a69f0
2018-02-13 19:09:35 +00:00
portdirect
7a322f0a08 KubeADM: Label namespaces
This PS removes the duplicate labeling of K8s namespaces everytime
the kubeadm image is run - as this action is performed by the
container itself while deploying k8s. It also updates the playbook
to label the kube-public ns.

Change-Id: Icfaabe54d263e0bbc52f2ac11835cb5d92ad32a4
2018-02-11 13:59:15 -05:00
Zuul
3b130197e7 Merge "Fix namespace label" 2018-02-09 16:30:26 +00:00
Zuul
4297f786b0 Merge "Add elasticsearch exporter" 2018-02-07 23:30:04 +00:00
Steve Wilkerson
094e0103a9 Add elasticsearch exporter
This adds a chart for an elasticsearch exporter to expose metrics
for prometheus. This also moves the exporter to be included as
part of the chart it's meant to target as opposed to its own
separate chart

Change-Id: I491f4d1efba633827d8a6255218daeb9d427f922
2018-02-07 10:59:27 -06:00
Steve Wilkerson
6af03cd6d9 Update prometheus metrics gathering in post-job
This updates the prometheus metrics gathering role to dynamically
detect deployed namespaces instead of using hardcoded tasks for
each namespace. This allows the use of this job in openstack-helm
without needing to manually add additional namespaces

Change-Id: I2304aceab8dcda5471af708f57924193f03ee8f3
2018-02-05 18:21:49 -06:00
Tin Lam
ae54ad7cd0 Fix namespace label
This patch set fixes the namespace labels and adds in a new log file to
describe all namespaces to troubleshoot gate issues where labels are not
applied properly.

Change-Id: I34c43c345f9a49df8d7fcf2e7824220a22698d46
Signed-off-by: Tin Lam <tin@irrational.io>
2018-02-05 15:43:40 -06:00
portdirect
2d725fd00b Rollback helm version to 2.7.2
The update to helm 2.8.0 resulted in issues with releases not
registering as ready before timing out. This rolls the version back
until those issues are addressed

Change-Id: Id58f85f150054d82c047bd360258ebe9e571360f
2018-01-31 21:28:58 +00:00
portdirect
27a2b2e16d K8s/Helm: Bump versions to current release
This PS bumps the versions of helm and K8s to their current GA
release.

Change-Id: Ia205f47d4f6b89d6d54bb2a71fde28a5730d2a47
2018-01-28 23:34:52 -05:00
Steve Wilkerson
3ec7f5f0ff Gate fix: httpd image, elasticsearch, openstack-exporter
There was a change in the upstream reference httpd image for
apache that changed how modules were built for apache.
This change adds the required fix to accomodate the change.
See isssue here https://github.com/docker-library/httpd/pull/87

The Elasticsearch image tag was updated to accomodate the kernel
versions used in the gate as part of the kernel update playbook
See https://github.com/elastic/elasticsearch/issues/28349#issuecomment-360233779

The openstack-exporter binary was changed to reflect changes made
to the openstack-exporter image

Change-Id: I1deb9e7cde794421dd33fade566c2a9fdb5007e6
2018-01-28 15:07:24 -06:00
Zuul
b380b68bba Merge "Gate: collect host level info" 2018-01-20 21:29:05 +00:00
Zuul
08a6266654 Merge "Add label to namespaces" 2018-01-19 14:50:08 +00:00
portdirect
6a150bd0f9 KubeADM: Increase pod limit for hosts
This PS increases the pod limit per hosts - useful in dev envs.

Change-Id: I3bdb933d7f5001cb91db6f7f227101e587532a54
2018-01-17 18:57:12 -05:00
Tin Lam
f6347e7d28 Add label to namespaces
This patch set adds label to "kube-system" and "default"
namespaces used for podSelector.

Change-Id: Ia67a0e4d9adf6f5575d74aebf77673aae3660c62
Signed-off-by: Tin Lam <tin@irrational.io>
2018-01-17 16:08:24 -06:00
Zuul
c1a8572584 Merge "KubeADM-AIO: Move to upstream debian base image" 2018-01-17 08:03:29 +00:00
portdirect
0025eb9444 Gate: collect host level info
This PS adds basic host level log collection to the gate.

Change-Id: I5ee3905e134b2d9abdad08121ecb7e257c7165c8
2018-01-14 17:41:25 -05:00
portdirect
9b40b8656d Prometheus Openstack Exporter: tidy chart and add ks user
This PS adds keystone user management to the prometheus-openstack-exporter
chart, and also performs some spring cleaning.

Change-Id: I69e40c523867f751ecd8c63169aefdfdf4eb5cd2
2018-01-14 09:51:06 -05:00
portdirect
76d0f57e64 KubeADM-AIO: Move to upstream debian base image
This PS moves KubeADM-AIO to use the upstream debian base image
that is used to build the Kubernetes release images:
 * https://github.com/kubernetes/kubernetes/tree/master/build/debian-base

Change-Id: I153a3aee43613b429793c179b2707c916821d39a
2018-01-13 23:46:33 -05:00
portdirect
217385a0dd Gate: Simplify and speed up pod and k8s object log collection
This PS simplifys and accelerates the collection of pod container
logs and k8s object info in the zuul gates, which can currently take
40 minutes for the 5 node OSH gate to complete.

Change-Id: Ie9b23174fade3df4a87f2b771ea654e2081b4f4e
2018-01-13 21:39:42 -05:00
portdirect
3dfcde8849 KubeADM-AIO: allow cluster domain to be customised
This PS allows the cluster domain to be customised if desired.

Change-Id: I86b9271b6248a36ce39a367b814cffa2bea6d4b3
2018-01-11 16:22:27 +00:00
Zuul
3e68d02fc2 Merge "Grafana: support multiple replicas and data persistance" 2018-01-11 16:02:28 +00:00
portdirect
21dc4f8af8 Grafana: support multiple replicas and data persistance
This PS adds support for multiple replicas of Grafana to be
deployed, and adds MySQL based data persistance to the chart.

Change-Id: Ife44985a6d5024cc2074346340fba1d8efdecbfa
2018-01-11 02:39:34 -05:00
portdirect
3d633fca7a Gate: Increase timeouts for awating node to come back from reboot
This PS increases the timeout in waiting for the node to come back
online following a reboot.

Change-Id: I92d1e5b665006bf9693f56ad8272330c19e6ccfa
2018-01-10 10:48:02 -05:00
portdirect
13852ceee4 Gate: Deploy HWE kernel on ubuntu hosts
This PS deploys the HWE kernel on Ubuntu Hosts, which is required
for CephFS:
 * https://github.com/kubernetes-incubator/external-storage/issues/345

Change-Id: I2ebd46eadf5a4c7a857d42302f388511691ab0db
2018-01-09 04:04:18 +00:00
portdirect
2e0b57ad93 KubeADM-AIO: Drive basic CNI configuration via values
This PS drives basic CNI options via ansible playbook in the
KubeADM-AIO container and modifies the calico chart to support
configuration via values.

Change-Id: Iaf2f9807438c3a34e797c62c2c6913edb677997c
2018-01-07 12:32:40 -05:00
Zuul
227df8bf87 Merge "Gate: Harden helm serve playbook" 2018-01-07 06:19:32 +00:00
Zuul
604fb28fd3 Merge "Gate: Move pull images to its own playbook" 2018-01-07 03:57:20 +00:00
portdirect
f5ccebb791 Gate: Harden helm serve playbook
This PS make the helm server more robust by moving the server to be
managed via a systemd unit.

Change-Id: I651bf3b2670812ea255c3692bd933bb6092397f9
2018-01-06 20:14:19 -05:00
Zuul
8d14de64a4 Merge "Perserve env variables" 2018-01-06 23:30:18 +00:00
portdirect
229c5dd45c Gate: Move pull images to its own playbook
This PS moves build images to its own playbook, reduducing the time
taken to run the gates when not required.

Change-Id: I17203a6fa2e3c7898175389f13d680fe9347b920
2018-01-06 17:32:42 -05:00