88 Commits

Author SHA1 Message Date
Pete Birley
be3f300623 Gate: place role default vars within the role
This PS places the role default vars within the appropriate role,
in addition it exposes kubeadm's selfhosted param and deploys the
kubeadm binary to master nodes.

This PS exposes the selfhosted param, and deploys kubeadm to master
nodes.

Change-Id: I5ad1b593a711ffe353b012394d54044dede0691d
2018-05-14 03:28:31 +00:00
Steve Wilkerson
b492ee54c3 OSH-infra multinode gate rework
This moves to using bash scripts for deploying the charts for the
multinode gates in osh-infra as opposed to using the ansible
chart builder

Change-Id: I2951ccf57250a5e6e272872f2d6f7a81bd11f184
2018-05-10 00:06:04 +00:00
Pete Birley
39ab1c3b5d Gate: Improve centos support
This PS updates the CentOS Gate.

Change-Id: Ic29b9e5949fb3fb3433802f7933208f3cabfef72
2018-04-20 11:28:36 -05:00
Pete Birley
37d836c8c7 Gate/Dev: RPM and structure improvements
This PS restores the use of the distro provided docker for RPM based
distros. It also removes the roles subdirectory from the playbooks
and replaces it with a symlink for local dev use.

Change-Id: Ic11adb85813e92488f5ebbe6714ad7da8d3938e2
2018-04-18 13:56:47 -05:00
Steve Wilkerson
9dd81954fd Move roles and playbooks to top level
This proposes moving the playbooks and roles to the top level of
osh-infra to be consumed by osh

Change-Id: I7e3516da8e516060f94b8b9c8875918504de7446
2018-04-09 20:52:18 -05:00
portdirect
59d4141d6a Update to F27 and update docker packages
This PS moves the fedora gates to use F27. It also updates
the docker packages used for both fedora and centos

Change-Id: Ic5ab5793d7176a81a8ea70c82b1d53a1fdf5f740
2018-04-06 17:21:41 +00:00
Steve Wilkerson
99befc2484 Nagios Chart
This adds the nagios chart to osh-infra to provide additional
monitoring functionality. It uses helper functions to consume
yaml definitions for services, commands, hosts and hostgroups
to generate the required configurations for those entities in
nagios's configuration

Change-Id: I6238bb8cb1e5c8dc48594ddea50693f3e7b0a176
2018-03-23 13:45:40 +00:00
Zuul
f461e53e52 Merge "Provide ability to specify proxy" 2018-03-14 14:10:57 +00:00
Paul Carver
e01268bb50 Correcting a spelling error
perfoming -> performing

Change-Id: I639d8a4d0021dbdcd66ff711f86c25f1313a823f
2018-03-12 20:26:30 -04:00
Tin Lam
391bbf69d7 Provide ability to specify proxy
This patch set provides ability to configure the proxy setting in
the ansible playbook used to run the gate.

Change-Id: I7ae66cb16fa6db7d46a0a3f23c289cc625e583c4
Signed-off-by: Tin Lam <tin@irrational.io>
2018-03-12 04:15:30 +00:00
Pete Birley
21a6217c58 Gate: get full process list from host
This PS adds the ps command to the gate log collection task.

Change-Id: Ife1710854547b28a6dfa6b3e4d527ebf4d04179b
2018-03-09 12:45:44 +00:00
Pete Birley
ca3929025b Gate: collect more host level logs
This PS collects more host level logs in the gate scripts.

Change-Id: I0ec13f45fd4561fec59d08b08eb78390a3866156
2018-02-26 07:20:25 +00:00
Pete Birley
1bf13af26c Node-Labels: Add linuxbridge node label to nodes
This PS adds the linxubridge node label to nodes to simplify
deployment and testing of the neutron linuxbridge backend.

Change-Id: I3095c21a75ad3057539f4fed431cb18c68e49a9b
2018-02-26 03:57:49 +00:00
Pete Birley
e0c688d7ee dependencies: move static dependencies under a 'static' key
This PS moves static dependencies unser a 'static' key to allow
expansion to cover dynamic dependencies.

Change-Id: Ia0e853564955e0fbbe5a9e91a8b8924c703b1b02
2018-02-24 17:39:55 -05:00
Steve Wilkerson
4f1f180e8e Add prometheus annotations to tiller-deploy service
This adds the prometheus annotations to the tiller chart, allowing
for the scraping of metrics from tiller.  currently, tiller exports
metrics specific to the grpc calls being made against it

Change-Id: Ibe10098689cc37a9de6306af86f4d63d8aab318b
2018-02-20 10:05:05 -06:00
portdirect
3713266f4c K8s: udpate KubeADM-AIO to use K8s v1.9.3
This PS updates the version of K8s used in the gate to v1.9.3

Change-Id: I284083e0cdf828701c439c44edfd3a975dcd69c3
2018-02-17 15:21:06 -05:00
Steve Wilkerson
5d95b0e2cb Add fluentd prometheus exporter for logging metrics
This adds templates for a prometheus exporter for fluentd to
adequately capture log metrics and fluentd service metrics for
consumption by prometheus

Change-Id: I6d6a8c2be07af819dc6d99b8ce5f1d4b635a69f0
2018-02-13 19:09:35 +00:00
portdirect
7a322f0a08 KubeADM: Label namespaces
This PS removes the duplicate labeling of K8s namespaces everytime
the kubeadm image is run - as this action is performed by the
container itself while deploying k8s. It also updates the playbook
to label the kube-public ns.

Change-Id: Icfaabe54d263e0bbc52f2ac11835cb5d92ad32a4
2018-02-11 13:59:15 -05:00
Zuul
3b130197e7 Merge "Fix namespace label" 2018-02-09 16:30:26 +00:00
Zuul
4297f786b0 Merge "Add elasticsearch exporter" 2018-02-07 23:30:04 +00:00
Steve Wilkerson
094e0103a9 Add elasticsearch exporter
This adds a chart for an elasticsearch exporter to expose metrics
for prometheus. This also moves the exporter to be included as
part of the chart it's meant to target as opposed to its own
separate chart

Change-Id: I491f4d1efba633827d8a6255218daeb9d427f922
2018-02-07 10:59:27 -06:00
Steve Wilkerson
6af03cd6d9 Update prometheus metrics gathering in post-job
This updates the prometheus metrics gathering role to dynamically
detect deployed namespaces instead of using hardcoded tasks for
each namespace. This allows the use of this job in openstack-helm
without needing to manually add additional namespaces

Change-Id: I2304aceab8dcda5471af708f57924193f03ee8f3
2018-02-05 18:21:49 -06:00
Tin Lam
ae54ad7cd0 Fix namespace label
This patch set fixes the namespace labels and adds in a new log file to
describe all namespaces to troubleshoot gate issues where labels are not
applied properly.

Change-Id: I34c43c345f9a49df8d7fcf2e7824220a22698d46
Signed-off-by: Tin Lam <tin@irrational.io>
2018-02-05 15:43:40 -06:00
portdirect
2d725fd00b Rollback helm version to 2.7.2
The update to helm 2.8.0 resulted in issues with releases not
registering as ready before timing out. This rolls the version back
until those issues are addressed

Change-Id: Id58f85f150054d82c047bd360258ebe9e571360f
2018-01-31 21:28:58 +00:00
portdirect
27a2b2e16d K8s/Helm: Bump versions to current release
This PS bumps the versions of helm and K8s to their current GA
release.

Change-Id: Ia205f47d4f6b89d6d54bb2a71fde28a5730d2a47
2018-01-28 23:34:52 -05:00
Steve Wilkerson
3ec7f5f0ff Gate fix: httpd image, elasticsearch, openstack-exporter
There was a change in the upstream reference httpd image for
apache that changed how modules were built for apache.
This change adds the required fix to accomodate the change.
See isssue here https://github.com/docker-library/httpd/pull/87

The Elasticsearch image tag was updated to accomodate the kernel
versions used in the gate as part of the kernel update playbook
See https://github.com/elastic/elasticsearch/issues/28349#issuecomment-360233779

The openstack-exporter binary was changed to reflect changes made
to the openstack-exporter image

Change-Id: I1deb9e7cde794421dd33fade566c2a9fdb5007e6
2018-01-28 15:07:24 -06:00
Zuul
b380b68bba Merge "Gate: collect host level info" 2018-01-20 21:29:05 +00:00
portdirect
0025eb9444 Gate: collect host level info
This PS adds basic host level log collection to the gate.

Change-Id: I5ee3905e134b2d9abdad08121ecb7e257c7165c8
2018-01-14 17:41:25 -05:00
portdirect
9b40b8656d Prometheus Openstack Exporter: tidy chart and add ks user
This PS adds keystone user management to the prometheus-openstack-exporter
chart, and also performs some spring cleaning.

Change-Id: I69e40c523867f751ecd8c63169aefdfdf4eb5cd2
2018-01-14 09:51:06 -05:00
portdirect
217385a0dd Gate: Simplify and speed up pod and k8s object log collection
This PS simplifys and accelerates the collection of pod container
logs and k8s object info in the zuul gates, which can currently take
40 minutes for the 5 node OSH gate to complete.

Change-Id: Ie9b23174fade3df4a87f2b771ea654e2081b4f4e
2018-01-13 21:39:42 -05:00
portdirect
3dfcde8849 KubeADM-AIO: allow cluster domain to be customised
This PS allows the cluster domain to be customised if desired.

Change-Id: I86b9271b6248a36ce39a367b814cffa2bea6d4b3
2018-01-11 16:22:27 +00:00
Zuul
3e68d02fc2 Merge "Grafana: support multiple replicas and data persistance" 2018-01-11 16:02:28 +00:00
portdirect
21dc4f8af8 Grafana: support multiple replicas and data persistance
This PS adds support for multiple replicas of Grafana to be
deployed, and adds MySQL based data persistance to the chart.

Change-Id: Ife44985a6d5024cc2074346340fba1d8efdecbfa
2018-01-11 02:39:34 -05:00
portdirect
3d633fca7a Gate: Increase timeouts for awating node to come back from reboot
This PS increases the timeout in waiting for the node to come back
online following a reboot.

Change-Id: I92d1e5b665006bf9693f56ad8272330c19e6ccfa
2018-01-10 10:48:02 -05:00
portdirect
13852ceee4 Gate: Deploy HWE kernel on ubuntu hosts
This PS deploys the HWE kernel on Ubuntu Hosts, which is required
for CephFS:
 * https://github.com/kubernetes-incubator/external-storage/issues/345

Change-Id: I2ebd46eadf5a4c7a857d42302f388511691ab0db
2018-01-09 04:04:18 +00:00
Zuul
227df8bf87 Merge "Gate: Harden helm serve playbook" 2018-01-07 06:19:32 +00:00
Zuul
604fb28fd3 Merge "Gate: Move pull images to its own playbook" 2018-01-07 03:57:20 +00:00
portdirect
f5ccebb791 Gate: Harden helm serve playbook
This PS make the helm server more robust by moving the server to be
managed via a systemd unit.

Change-Id: I651bf3b2670812ea255c3692bd933bb6092397f9
2018-01-06 20:14:19 -05:00
Zuul
8d14de64a4 Merge "Perserve env variables" 2018-01-06 23:30:18 +00:00
portdirect
229c5dd45c Gate: Move pull images to its own playbook
This PS moves build images to its own playbook, reduducing the time
taken to run the gates when not required.

Change-Id: I17203a6fa2e3c7898175389f13d680fe9347b920
2018-01-06 17:32:42 -05:00
Zuul
91548a6648 Merge "Kubernetes: Move to version 1.9.1" 2018-01-06 17:01:07 +00:00
lamt
9c31548c9b Perserve env variables
This patch set adds in -E flag for ``sudo pip install`` so
any environment variable is preserved for the install.  This
allows for some proxy-related variables such as ``http_proxy``
to pass through for pip.

This mirrors the change in [0].

[0] https://review.openstack.org/#/c/531270/1

Change-Id: Ib513fce862f31d784634645ac6b038c6a35f4cd6
2018-01-05 08:57:25 -06:00
rakesh-patnaik
fc52cda24a Openstack metrics exporter for prometheus
This exporter provides a means for Prometheus to gather openstack
service metrics related to overlying openstack-helm deployments
Change-Id: I5f1789c62b4547add0c67edb51540f712bf43da8
2018-01-05 07:51:31 -06:00
portdirect
7b6b2274ab Kubernetes: Move to version 1.9.1
This PS moves the version of kubernetes used in the gates to v1.9.1

Change-Id: Ida17cb20b0a5ce11e03ccfafddb0f58fadda76f1
2018-01-05 00:56:07 -05:00
Zuul
17b13c42d7 Merge "Add Grafana chart to OSH infra" 2018-01-05 04:59:51 +00:00
Steve Wilkerson
a1f608ed74 Add Grafana chart to OSH infra
Moves the grafana chart to OSH infra along with basic rbac rules
that may be tightened with future work.

Change-Id: Ie14627530a73d4b7b01eb93ca5f7174d99d9caec
2018-01-04 09:10:55 -06:00
Zuul
2e4b15f16e Merge "Docker: Run docker without iptables" 2018-01-02 21:42:03 +00:00
Zuul
0d5a2a64a3 Merge "Add support for out of branch variables" 2017-12-31 03:14:59 +00:00
portdirect
bbf32935dc Docker: Run docker without iptables
This PS updates the docker daemon settings to run without
apply iptables rules. This simplifies host network management by
removing one of the actors interacting with iptables.

Change-Id: I335247afddf736b60212d199a3b860c3c792977f
2017-12-30 21:59:19 -05:00
Matt McEuen
134d7cab0c Add support for out of branch variables
This change allows users of the gate script to optionally
override the location of the ansible inventory and variables files,
so that they can live outside of the source tree.

Change-Id: Ibe7666ce366c7ad34e8ee6ff5ef3f23589aa10ce
2017-12-30 13:58:56 -06:00