104 Commits

Author SHA1 Message Date
Steven Fitzpatrick
fa77f3fc6b Reduce osh-infra check jobs
This change moves the following non-voting checks to experimental:

  - openstack-helm-infra-federated-monitoring
  - openstack-helm-infra-kafka
  - openstack-helm-infra-local-storage
  - openstack-helm-infra-aio-network-policy
  - openstack-helm-infra-apparmor
  - openstack-helm-infra-aio-logging-apparmor
  - openstack-helm-infra-openstack-support-apparmor
  - openstack-helm-infra-metacontroller

Change-Id: I6bb70be0dc1b012742405818d049e9d0e155f671
2020-04-08 08:57:36 -05:00
diwakar thyagaraj
6632b114b8 [FIX] Fixes libvirt apparmor and gate jobs
This change refactors the apparmor job to utilize the feature
gates system instead of relying on separate scripts.

Change-Id: I51b36c1972ff3ee8d4366bf2d5027e433721d740
2020-04-03 18:17:58 +00:00
Zuul
aa01846f60 Merge "Disable podsecuritypolicy job" 2020-03-26 21:00:08 +00:00
diwakar thyagaraj
c0bec2c744 Enable Apparmor to fluentd
Change-Id: I21640c263cbf7871319d2710160e37a9dddf0eb2
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-03-25 21:50:53 +00:00
Steve Wilkerson
d898a65a2d Revert "Enable Apparmor for fluentd"
This reverts commit a3110abd66727e8fab881a044110d01d2f405019.

Change-Id: I90180d5caa6cd7873220fcc91570c92ae7a234e5
2020-03-24 19:46:19 +00:00
Gage Hugo
a52604bd80 Disable podsecuritypolicy job
The current podsecuritypolicy job has not been passing for quite
a while. Disable it for now until it's fixed to avoid wasting
infra resources.

Change-Id: I14b184cf03e625cbbaa829a4de101dc2142a7e27
2020-03-20 04:05:15 +00:00
dt241s@att.com
a3110abd66 Enable Apparmor for fluentd
Change-Id: I6b759beff9fd3166a9868a4d71319836f272bc25
2020-03-18 21:26:48 +00:00
dt241s@att.com
1871b830af Enable Apparmor for ElasticSearch
Change-Id: Ie7dc4399be4f0a62576e336c1de02810eeb09212
2020-03-16 20:08:18 +00:00
dt241s@att.com
d59b6e5944 Enable Apparmor to Kibana
Also added new apparmor zuul gates jobs for Logging, as initial 
apparmor is  getting timeout.

Change-Id: Iea0a5055238d75f401caf9ddb0ddd9985a091aab
2020-03-14 04:37:32 +00:00
Luna Das
9b9913d64a Add Helm Charts for MetaController and DaemonJobController.
These charts bootstraps a metacontroller on a
Kubernetes cluster using the Helm package manager.
This enables you to deploy custom controllers as
service + deployment pairs. A DaemonJobController
chart bootstraps the CompositeController and register
DaemonJob CRD, the daemonjob controller executes
DaemonJob(CR's) in kubernetes Cluster.

Change-Id: Ic946f564ea1cf07e89c90a598e59230dc240950c
2020-03-10 01:04:42 +05:30
dt241s@att.com
920bddde64 Enable Apparmor to nagios
Change-Id: I5927d32903cabd93b9d78c0c47994a94162deb1c
2020-02-28 13:31:50 +00:00
Zuul
46c3b737c5 Merge "Add Apparmor to Grafana" 2020-02-25 22:12:39 +00:00
dt241s@att.com
cc392aaa85 Add Apparmor to Grafana
Added apparmor feature gate and Zuul Gate Job

Change-Id: I9ce522f77447b1cb3f189ab7023c5c711e577618
2020-02-25 19:25:12 +00:00
dt241s@att.com
f9c346fa37 [FIX] Fixes apparmor for ingress
This also adds Apparmor Gate Script.

Change-Id: Id25d9cddc8f8c4c98dd8cd044f679201a908c875
2020-02-25 06:10:17 +00:00
Pete Birley
bda5983180 Add namespace configuration chart
This chart provides default configuration objects for a
kubernetes namespace.

Change-Id: If1b1545956064bb0897c8d67d9f13ef606ed2ba3
Signed-off-by: Pete Birley <pete@port.direct>
2020-02-21 13:51:56 -06:00
Prateek Dodda
b84f536ebd Add Docker default AppArmor profile to rabbitmq exporter chart
Change-Id: If1b420f91d1d23cc454a9ca8eff95a88a7e0b414
2020-02-19 12:40:08 -06:00
Zuul
d8c937f608 Merge "Enable Docker default Apparmor for Postgresql and prometheus-postgresql." 2020-02-18 20:58:17 +00:00
Zuul
6a0a7f3cc4 Merge "Add Docker default AppArmor profile to mariadb exporter chart" 2020-02-18 19:29:24 +00:00
Zuul
75474c01b8 Merge "Address bandit gate failures" 2020-02-18 17:35:45 +00:00
KHIYANI, RAHUL (rk0850)
2712f54117 Add Docker default AppArmor profile to mariadb exporter chart
Change-Id: I6d5fcbb511f4f9cdb31727421fe320beeff1a882
2020-02-18 04:49:44 +00:00
dt241s@att.com
f633555f16 Enable Docker default Apparmor for Postgresql and prometheus-postgresql.
Change-Id: I013ca5f99e5032c44f0d679e467da9e928c02a6b
2020-02-17 23:01:06 +00:00
dt241s@att.com
8bd4a2624a [FIX] Add apparmor to prometheus.
This also fixes Elasticsearch apparmor Jobs.

Change-Id: I8f2a9aa12beffe3ca394a2e9dd00aba7e5292f29
2020-02-14 23:13:38 +00:00
Gage Hugo
86e56b2aee Address bandit gate failures
This change addresses the results that were found when running
bandit against the templated python files in the various charts.

This also makes the bandit gate only run when python template
files are changed as well as makes the job voting.

Change-Id: Ia158f5f9d6d791872568dafe8bce69575fece5aa
2020-02-04 15:33:17 -06:00
Zuul
042ac52397 Merge "Update Elastic Beats versions to 7.1.0" 2020-01-14 00:09:44 +00:00
Zuul
c2ece6a45a Merge "Support for local storage" 2020-01-09 23:18:16 +00:00
Smruti Soumitra Khuntia
2ac08b59b4 Support for local storage
This change adds a means of introducing new storage classes
and local persistent volumes.

Change-Id: I340c75f3d0a1678f3149f3cf62e4ab104823cc49
Co-Authored-By: Steven Fitzpatrick <steven.fitzpatrick@att.com>
2020-01-09 10:24:31 -06:00
Steve Wilkerson
45ac5fbe0f Update Elastic Beats versions to 7.1.0
This updates the Elastic Beats charts to 7.1.0 to keep them
aligned with the Kibana and Elasticsearch chart versions, which
is required for compatibility

This also updates the experimental job to use the single node
minikube deployment as opposed to the standard 5 node multinode
deployment

Change-Id: I4baba6ca2ea2f3785f11905138b67979a4501caa
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2020-01-09 13:11:43 +00:00
Gage Hugo
f9713b0051 Get osh-infra netpol gate passing
This change adds in missing network policy overrides for
fluent-daemonset and prometheus-exporter, as well as removes
existing mariadb network policies overrides that were causing
the network policy check job to fail.

Change-Id: Ib7a33f3d14617f9a9fda264f32cde7729a923193
2020-01-09 04:38:25 +00:00
Zuul
954a06b3fd Merge "Add bandit check to osh-infra" 2020-01-03 20:51:03 +00:00
Gage Hugo
30c66126ee Add bandit check to osh-infra
This change adds a non-voting bandit check to openstack-helm-infra
similar to what is ran in the openstack-helm repo.

This check will be made voting in a future change once the current
failures are addressed.

Similarly this check will be modified in a future change to
only be ran when affected python files are changed.

Change-Id: I177940f7b050fbe8882d298628c458bbd935ee89
2019-12-31 18:29:01 +00:00
Steve Wilkerson
803f0e8435 Disable kubeadm-aio jobs while issues addressed
This disables the keystone-auth single node job and all multinode
periodic and experimental jobs while standing issues with the
kubeadm-aio image deployment are sorted out

Change-Id: I3ce0afba155e923b6dd50f83fa6b529908b9a79b
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-18 13:56:21 -06:00
Steve Wilkerson
edd6ffd712 Reduce osh-infra-logging job scope
This updates the osh-infra-logging single node job to omit the
fluentbit deployment step, as having multiple logging daemonsets
deployed to the single node jobs is causing IO issues. Also, it
was noted that the fluentd-deployment step was missing the
overrides to move the fluentd-deployment release from utilizing a
daemonset to a deployment. This resulted in 3 logging daemons
being deployed to a single host

Change-Id: I4a0c5550e6ea6a331aab0082a975f161e65704bf
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-17 12:43:12 -06:00
Steven Fitzpatrick
e8f3d84ccc Create Chart to Deploy Apache Kafka
This proposes adding a kafka chart to osh-infra that aligns
with the design patterns laid out by the other charts in osh-infra
and osh.

danielqsj's kafka-exporter image is leveraged to deploy a prometheus
exporter for kafka alongside the main application if enabled in
values.yaml

Change-Id: I5997b0994fc3aef9bd1b222c373cc3a013112566
Co-Authored-By: Meghan Heisler <mh783g@att.com>
2019-12-03 11:37:54 -06:00
Zuul
05c3ec119b Merge "Grafana: Support multiple datasources" 2019-11-22 16:43:18 +00:00
Steve Wilkerson
97e029e606 Grafana: Support multiple datasources
This updates the Grafana chart to support the definition of
multiple datasources. This moves to defining a template in the
chart's values.yaml file that allows for inline gotpl for
defining an arbitrary number of datasources. This also updates the
grafana dashboards to include a selector for the Prometheus
datasource to use via a drop down selector. This is vetted out in
the federated monitoring job

Change-Id: I55171fed5c2b343130d135d0b42bc96ff11c4712
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-22 14:45:04 +00:00
Steve Wilkerson
0b86616c6f Make keystone-auth job nonvoting
This makes the keystone-auth job nonvoting, until adequate work
can be done to help make the job more reliable. At the moment,
this job seems to be responsible for the majority of the gate job
failures due to what seems to be limitations with the single node
nodesets available

Change-Id: I08f1f10b79e9a5fd82ef7c6d887a03ccb55cceed
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-22 14:44:52 +00:00
Zuul
873838f11f Merge "Prometheus: Update chart to support federation" 2019-11-22 05:27:18 +00:00
Steve Wilkerson
eabc9fad64 Re-enable experimental jobs in osh-infra
This adds the experimental jobs back to osh-infra, as they were
erroneously disabled via comments in a previously merged change

Change-Id: Id92c24223f8c22f1a0ff82b62c222b2920ecd929
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-21 13:53:00 -06:00
Steve Wilkerson
fbd34421f2 Prometheus: Update chart to support federation
This updates the Prometheus chart to support federation. This
moves to defining the Prometheus configuration file via a template
in the values.yaml file instead of through raw yaml. This allows
for overriding the chart's default configuration wholesale, as
this would be required for a hierarchical federated setup. This
also strips out all of the default rules defined in the chart for
the same reason. There are example rules defined for the various
aspects of OSH's infrastructure in the prometheus/values_overrides
directory that are executed as part of the normal CI jobs. This
also adds a nonvoting federated-monitoring job that vets out the
ability to federate prometheus in a hierarchical fashion with
extremely basic overrides

Change-Id: I0f121ad5e4f80be4c790dc869955c6b299ca9f26
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-21 12:39:56 +00:00
Steve Wilkerson
4e7b8a183e Remove elasticsearch ldap test from osh-infra-logging
This removes the elasticsearch-ldap.sh script from the single node
osh-infra-logging job, as this step does not provide any real
value and is tightly coupled to the elasticsearch version used.
This sort of validation should be reserved for smoke tests in
future helm tests for charts

Change-Id: I7ca4805a8809568cb09c8bab6c239c008528fd6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-20 12:52:51 +00:00
Steve Wilkerson
608d75ec8d Add zookeeper chart to osh-infra
This proposes adding a zookeeper chart to osh-infra that aligns
with the design patterns laid out by the other charts in osh-infra
and osh.

Change-Id: I25edc58fc951e7f81f7275ade6cf9c97e0afae02
Signed-off-by: Steve Wilkerson <sw5822@att.com>
Co-Authored-By: Steven Fitzpatrick <steven.fitzpatrick@att.com>
2019-11-14 19:51:20 +00:00
Michael Polenchuk
f2d32c8598 Bring in PowerDNS chart
Also bring in endpoint_token_lookup function to
get service token from endpoints schema.

Change-Id: Iffa68d8b2c70799a2013b99d15c9fd55e858babb
2019-09-11 12:46:34 +04:00
Steve Wilkerson
d547063c37 Disable cephfs provisioner in multinode jobs
This disables the cephfs provisioner in the multinode
periodic jobs. It seems the helm tests for the ceph
provisioner chart that test cephfs fail more often than
not in the multinode jobs while passing reliably in the
single node check and gate jobs. As cephfs is still
gated, disabling the cephfs provisioner in the periodic
jobs allows for further investigation into this issue
without causing potential regressions

Change-Id: I36e68cc2e446afac8769fb9ab753105909341f24
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-08-13 14:49:27 +00:00
Zuul
e29022f8ae Merge "Revert "CI: Make openstack-support and keystone-auth jobs nonvoting"" 2019-07-18 19:47:54 +00:00
cheng li
776885458a Revert "CI: Make openstack-support and keystone-auth jobs nonvoting"
This reverts commit 5e3f729ffe5692e6e37d0fe6378906662d94bbd0.

Change-Id: I65cb5d24f0538fbd0d6cd28e5e6313e679d87655
2019-07-17 14:06:21 +00:00
Steve Wilkerson
c1d9063a86 Ignore Selenium test failures in jobs
This adds the ability to tolerate failures of the selenium tests
in our jobs, as we intermittenly see these tests fail. The failure
of these tests should not necessarily indicate failure of the job
overall, so this change prevents exactly that

Change-Id: I4f97fad96f63d42fdb3bb5b8958dbed3dfd7dfc7
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-07-02 13:30:13 -05:00
Drew Walters
5e3f729ffe CI: Make openstack-support and keystone-auth jobs nonvoting
Change-Id: I17e7a6a499f8e7c86c5359452c3317fc4cbfe533
2019-06-25 22:46:39 +00:00
MegHeisler
d5af61b751 Remove fluent-loggin from network-policy
This removes the old fluent-logging chart from network
policy and replaces it with the new fluentbit and fluentd
charts. This will return the network policy gate back to
passing

Change-Id: I060c6c3034fa798a131a053b9d496e5d8781c55d
2019-06-24 16:14:05 +00:00
Zuul
65f78190ba Merge "Add zuul-jobs roles to job job that uses them" 2019-06-22 12:19:09 +00:00
Kostiantyn Kalynovskyi
0e0858a7c7 Add zuul-jobs roles to job job that uses them
Job openstack-helm-infra uses role named "start-zuul-console" that is part
of another project named "zuul/zuul-jobs". If this job is
ever used by another project as "parent job", it would fail, because
wouldn't find the role in any of the default pathes. This patch adds the
roles from zuul/zuul-jobs project, to the job that uses these roles from
the project

Change-Id: Ib3b7e0e43008b7a4f394b49b75529bfde9780d2f
2019-06-22 10:13:45 +00:00