40 Commits

Author SHA1 Message Date
Zuul
0d5a2a64a3 Merge "Add support for out of branch variables" 2017-12-31 03:14:59 +00:00
Matt McEuen
134d7cab0c Add support for out of branch variables
This change allows users of the gate script to optionally
override the location of the ansible inventory and variables files,
so that they can live outside of the source tree.

Change-Id: Ibe7666ce366c7ad34e8ee6ff5ef3f23589aa10ce
2017-12-30 13:58:56 -06:00
Zuul
41c2a6e8f3 Merge "Gate: simplify helm release log gathering" 2017-12-29 18:28:08 +00:00
portdirect
6c3786aef3 Gate: simplify helm release log gathering
This PS simplifys the helm release log gathering, to collect logs
from all charts released into the cluster, and also adds a `logs`
sub-target to the dev-deploy target in the Makefile.

Change-Id: I016a5e08163eaccf13331db6faa45fab1e9cf4f2
2017-12-28 15:40:32 -05:00
Steve Wilkerson
45ba95a2de Elasticsearch: Add curator snapshot action and PVC for fs repo
This provides an example action in the curator config for taking
snapshots of the elasticsearch indexes. As the snapshot action
requires a repository registered with Elasticsearch, this also
adds a PVC for a filesystem repository backed with NFS and a job
for registering the repository with Elasticsearch.

Change-Id: I26b788c58f52844e997bde5002459bddc1bb685e
2017-12-28 13:42:47 -06:00
Steve Wilkerson
b45e8ddcbd Move kibana to OSH infra
This moves the Kibana chart to OSH infra, which finalizes moving
the logging components to OSH infra

Change-Id: Iacbfde8d5d7099fcb4dde8a437e030c2d4936de6
2017-12-26 18:26:58 -06:00
portdirect
8b6d6c43cb Gate: collect infor about more kubernetes objects
This PS fleshes out the list of objects that info is collected for
in the gate.

Change-Id: I8f9560a05f5a5c7f5b27dcc7108ea3edc991206f
2017-12-19 09:56:34 -05:00
Steve Wilkerson
938bce7370 Include prometheus- prefix for select monitoring charts
This adds the prometheus- prefix to the alertmanager,
kube-state-metrics and node exporter charts to reflect their
intended usage as part of a prometheus centric monitoring solution

This will imply a logical grouping of these components, similar to
their deployment in the osh-infra gates

Change-Id: I4f391a10b64389022f01a94ea3704c110f8f9bb5
2017-12-17 23:22:50 -05:00
portdirect
c189522fdb Kubernetes: Update to v1.9.0
This PS updates the OpenStack-Infra Gate to use Kubernetes v1.9.0

Change-Id: I2c2d8180a6e05ac6babc72f6347f00a19bf7e0fd
2017-12-17 17:06:40 +00:00
portdirect
f472531ace Docker: user json-file on Fedora and CentOS
The current fluent-bit implementation only supports the json-file
log driver for docker, this PS moves CentOS and Fedora to use that
until we can support Journald.

Change-Id: I8aa876aa96119d9a1a0e06c28873e3c4c1e3ace5
2017-12-17 17:06:16 +00:00
tin.l.lam
e4de36d97b CentOS: Fix jq and pip installation
This PS fixes pip and jq installation on CentOS. It also removes
some duplicate code in the gate playbooks.

Co-Authored-By: portdirect <pete@port.direct>

This patch set should fix an issue where centos cannot find
python-pip in the EPEL.

Change-Id: If3a437e0756a363b8cefaa9a8bdd1c3498fedbfd
2017-12-17 17:05:47 +00:00
Zuul
2fb6a79b49 Merge "Fluent-logging helm chart" 2017-12-15 20:29:50 +00:00
sungil
2862f038e2 Fluent-logging helm chart
This introduces an initial helm chart for fluent logging.
It provides a functional fluent-bit and fluentd deployment to
use in conjunction with elasticsearch and kibana to consume
and aggregate logs from all resource types in a cluster.
It can deliver logs to kafka for external tools to consume.

This PS moves fluent-logging chart from osh-addons, osh to
osh-infra repo.
previous ps(addon): https://review.openstack.org/#/c/507023/
previous ps(osh): https://review.openstack.org/#/c/514622/

Specification: https://review.openstack.org/#/c/505491/
Partially implements: blueprint osh-logging-framework

Change-Id: I72e580aa3a197550060fc07af8396a7c8368d40b
2017-12-15 10:52:16 -06:00
Ganesh Maharaj Mahalingam
b3e63a9b47 Enable ceph-mgr label on nodes to support luminous
Change-Id: I17359df62a720cbd0b3ff79b1d642f99b3e81b3f
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2017-12-13 10:06:15 -08:00
Zuul
39968768e9 Merge "Helm: update to 2.7.2" 2017-12-12 00:27:31 +00:00
Steve Wilkerson
605faded2e Gather deployment information after checks run
Adds tasks to gather descriptions of kubernetes objects, logs from
deployed pods, logs from helm test pods, the status of each
helm release, and metrics from endpoints exposed to prometheus

Change-Id: I606797c6a5d75ba446ed2c16a9710f7b0227f910
2017-12-11 09:57:28 -06:00
Pete Birley
6faa1b4c0b Helm: update to 2.7.2
This PS updates the version of helm to v2.7.2 which includes some
security fixes.

Change-Id: Ic6de39985a88c4e0fb062c35f9ee6c256b3d20d9
2017-12-05 15:07:58 -06:00
Steve Wilkerson
4b94e47c94 Add Elasticsearch to OSH Infra
This moves the Elasticsearch chart to OSH infra, along with rbac
rules for running Elasticsearch. It includes a cronjob for running
ES Curator for cleaning up old indices

Change-Id: I69fcbe8b77de8b594eba5340a6e4340f389ba5bf
2017-12-01 17:42:23 -06:00
Zuul
9051ec4ac2 Merge "Prometheus monitoring for OSH infra" 2017-11-30 23:06:45 +00:00
Steve Wilkerson
429a4edd86 Prometheus monitoring for OSH infra
This will move prometheus to OSH-infra to be included as part of
the basic infrastructure deploy for openstack-helm. It includes
charts for Prometheus, Node Exporter, Kube-State-Metrics, and
Alertmanager. It provides a base for monitoring and alerting
for the underlying infrastructure

Partially Implements: blueprint osh-monitoring

Change-Id: Ie453373b54c5f1825339ce0566e4b5d0f74abc20
2017-11-30 09:43:17 -06:00
Steve Wilkerson
2a915c4a15 Add support for running helm tests for charts
Adds a common-helm-test task executed as part of the
deploy-helm-packages playbook. It allows the ability to run helm
tests against a chart by including a helm_test key in the chart
definition

Change-Id: I52bd5ca2fafa4eb704369590f7903c127133b090
2017-11-30 09:09:46 -06:00
portdirect
70e26302f7 Gate: Fix vars for local development use
This PS resolves an issue in the way ansible variables are merged
re-enableing use of the gate scripts for local development use.

Change-Id: Ia18a9f92816e4cc49f1c5423c04bed8a80abc6dc
2017-11-21 15:37:25 -05:00
Zuul
775b1f9f88 Merge "Gate: Modularise gate for development use" 2017-11-20 23:44:59 +00:00
Zuul
172844c662 Merge "Gate: make pod subnet configurable" 2017-11-17 20:48:11 +00:00
portdirect
87f22838f2 Gate: Modularise gate for development use
This PS breaks down the gate to enable use as a development aid.

Change-Id: I376da8940ed085b7575dd528ec4082f42da1748c
2017-11-16 12:47:19 -05:00
Zuul
e3327455c7 Merge "Gate: force use of bash for ansible script module use" 2017-11-14 21:14:53 +00:00
intlabs
2029e94d1b Gate: force use of bash for ansible script module use
This PS forces the use of bash for ansible tasks that use the
script module.

Change-Id: I131768d48081834806eeb912d84c562eeada1f46
2017-11-13 13:17:54 -06:00
Tin Lam
5d70c0ec8f Gate: Add linter gate and path options to OSH-Infra
This patch set adds a zuul v3 trailing whitespace linter gate to
the OSH-infra repository.

It additionally updates the work_dir path to account for being
launched as a dependant project in zuulv3.

Change-Id: I680a5942f008a6246641bebca7d9880a7824f2ce
2017-11-13 10:06:16 -06:00
intlabs
80e1d68fa0 Gate: Move to k8s v1.8.3 and expose params
This PS moves to use version 1.8.3 of k8s and also exposes all the
versions as build params to the kubeadm-aio dockerfile.

Change-Id: I8e6ef8090f3fa178e950a42605a5d86e298e0c99
2017-11-13 01:17:18 +00:00
intlabs
8ac50d34af Gate: Add task to deploy yq and deps
This PS adds a task to deploy yq and it's deps

Change-Id: I8404060d4cc24542e59af8cbfd7c69c5c11c0834
2017-11-12 05:07:46 -06:00
Matt McEuen
2b185f5735 Gate: make pod subnet configurable
Make the CNI pod subnet configurable/overridable.

Change-Id: I5fa3a2178ad3a61c1bf939fc3c08c787169d350b
2017-11-05 18:52:42 -06:00
intlabs
fe62a25081 Helm: Move to version 2.7.0
This PS moves the OpenStack-Helm Infra gates to v2.7.0

Change-Id: I8f2067019022cfcbc81a53927c397720297fe547
2017-11-02 23:43:27 -05:00
Zuul
72b644b66a Merge "Gate: remove zuul v2 stub" 2017-11-03 02:07:39 +00:00
intlabs
4ef2747f3a RBAC: enforce standard k8s RBAC policy
This PS removes the lax RBAC policy used in OSH to date, and the
standard rules are applied.

Change-Id: Ic2cf6063a4c0b7c4a6e73094f55ad424effc965c
2017-11-02 16:34:15 -05:00
intlabs
436845541b Image local repo jobs and multiple namespace support
This PS introduces support for using a local docker repo to
store images if desired, and adds multiple namespace support
to the entrypoint lookup functions.

Change-Id: Ib51aa30d3cc033795fe13f6c40a57d46171ad586
2017-11-02 16:23:32 -05:00
intlabs
ddf38fd47b Gate: remove zuul v2 stub
This PS removes the stub v2 gate setup script.

Change-Id: Ieb78bf466111e8b152e54ad2ee3ec32ff68b202d
2017-11-01 21:10:39 +00:00
intlabs
fd683f4241 Chart Runner: Add NFS provisoner and Docker Registry
This PS adds the NFS provisioner and Docker Registry to the OSH-Infra Repo,
these are being used as initial candidates for moving as they are simple
charts and allow the chart runner logic to be developed in the gate
playbooks.

Change-Id: Ie80b8578aafd3fe7252d3dcb603ea6af7586776e
2017-11-01 11:40:09 -05:00
intlabs
8c9f70401a Helm-Toolkit: Import into infra repo from primary
This PS imports helm-toolkit into the Helm Infra repo

Change-Id: Ife3451f9af5a40dbd5aa2414fc6226b26f81ede1
2017-10-26 19:33:36 -05:00
intlabs
bd85bad919 Zuul V3 gate
This PS sets up the V3 gate for openstack-helm-infra.

Change-Id: I07ffa591cb5e08f5e2f1f5cbc94e810c3aa1f97b
2017-10-24 20:27:34 -05:00
intlabs
366a175447 Remove old helm-toolkit
This PS removes the old helm toolkit in preperation for the repo to
be initialised with openstack-helm-infra and the zuul v3 gate.

Change-Id: I4fa3be6bc240c061620dc3b5533136107a99065c
2017-10-24 17:42:50 -05:00