780 Commits

Author SHA1 Message Date
Tin Lam
92e68d33ea Add network policy toolkit function
This patch set implements the helm toolkit function to generate a
kubernetes network policy manifest based on overrideable values.
This also adds a chart that shuts down all the ingress and egress
traffics in the namespace. This can be used to ensure the
whitelisted network policy works as intended.

Additionally, implementation is done for some infrastructure charts.

Change-Id: I78e87ef3276e948ae4dd2eb462b4b8012251c8c8
Co-Authored-By: Mike Pham <tp6510@att.com>
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-15 13:50:50 +00:00
Zuul
be7dbf6c28 Merge "[MariaDB] Update/remove deprecated configuration" 2018-10-13 21:40:48 +00:00
Zuul
75ea67e591 Merge "Fluent-logging: Update helm tests for checking index entries" 2018-10-13 03:11:39 +00:00
Zuul
c39b29e351 Merge "Fluentd: Update logging interval values" 2018-10-13 03:02:04 +00:00
Zuul
016cc39c9f Merge "Gate: Cleanup scripts for k8s keystone auth gate" 2018-10-12 23:38:52 +00:00
Zuul
bfb1c2a498 Merge "Replace docker-py with docker" 2018-10-12 22:15:15 +00:00
Zuul
cd50e50eb3 Merge "Charts: Update heat image used for jobs and helm tests" 2018-10-12 20:35:14 +00:00
Pete Birley
8bb71f6659 Gate: Cleanup scripts for k8s keystone auth gate
This PS cleans up the scripts for the k8s k8s keystone auth gate.

Change-Id: I248439f9b8ffa372dfaba5acba0c8c587231d901
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-12 13:43:41 -05:00
Jean-Philippe Evrard
100c900da0 Regroup OpenStack-Helm* gating under a folder
This move definitions of openstack-helm-infra into
a newly created zuul.d folder.

The advantage is to simplify readability of gating, and
makes it easier for contributors to step into the gating
of the openstack-helm-* projects.

- zuul.d/playbooks will contain all the playbooks used for gating
- zuul.d/nodesets.yaml contains all the specific nodesets
  required by OpenStack-Helm* projects
- zuul.d/project.yaml will be defined in each repo, and will
  contain the repo's pipelines information (so this repository's
  project.yaml only contains openstack-helm-infra pipelines)
- zuul.d/jobs.yaml will contain all the openstack-helm-*
  repositories jobs

This patch also introduces a first common 'lint' playbook
and 'openstack-helm-lint' job, showing how a job can be
re-used across repositories without requiring repetition of
job definition/plays in other repositories.

Change-Id: Id055ddac4da4971b1fb13ac075a7659369cd2b24
2018-10-12 15:13:12 +02:00
Steve Wilkerson
c7cbb9f4dd Charts: Update heat image used for jobs and helm tests
This changes the image used for various jobs and helm tests in the
osh-infra charts. This replaces the kolla heat image with the loci
based heat image used for jobs and helm tests in openstack-helm in
order to drive consistency

Change-Id: Ie9deedadb7507282fe62723ec4641dd508040364
2018-10-11 14:47:58 -05:00
Steve Wilkerson
78283495f0 Fluent-logging: Update helm tests for checking index entries
This updates the helm tests for the fluent-logging chart to make
them more robust in being able to check for indexes defined in the
chart.  This is done by calculating the combined flush interval
for both fluentbit and fluentd, and sleeping for at least one
flush cycle to ensure all functional indexes have received logged
events.

Then, the test determines what indexes should exist by checking
all Elasticsearch output configuration entries, determining
whether to use the default logstash-* index or the logstash_prefix
configuration value if it exists.  For each of these indexes, the
test checks whether the indexes have successful hits (ie: there
have been successful entries into these indexes)

Change-Id: I36ed7b707491e92da6ac4b422936a1d65c92e0ac
2018-10-11 13:28:30 -05:00
Chris Wedgwood
8554bdcbef [MariaDB] Update/remove deprecated configuration
Change-Id: I18aa87602b63ecd051c21e007aff8cadccdd0cda
2018-10-11 15:31:31 +00:00
Steve Wilkerson
9b5d4d9f17 Fluentd: Update logging interval values
This updates the logging interval values for the Elasticsearch
outputs to integers (20) vs the previous string value (20s)

Change-Id: I681bdaf807ba0136fef3b6dc1c7ddaa689ae77a3
2018-10-11 09:05:00 -05:00
Zuul
e231a7c5fd Merge "Elasticsearch: Update log4j2 configuration settings" 2018-10-10 19:23:47 +00:00
Zuul
922d7d3d26 Merge "Charts: Update helm test pod templates" 2018-10-10 06:00:44 +00:00
Zuul
c84dfd8122 Merge "Add configMap hash to annotation" 2018-10-09 22:23:10 +00:00
Zuul
9f63330f0b Merge "Add missing labels to cronJobs" 2018-10-09 21:49:41 +00:00
Steve Wilkerson
bfa237d347 Charts: Update helm test pod templates
This updates the helm test pod templates in the charts with helm
tests defined. This change includes the addition of:

- Generate test pod cluster roles and role bindings
- Generate service accounts for test pods
- Add node selectors to the test pods
- Add service accounts to the test pods
- Addition of entrypoint container to the test pods
- Indentation fix for rabbitmq test pod template

Change-Id: I9a0dd8a1a87bfe5eaf1362e92b37bc004f9c2cdb
2018-10-09 21:00:00 +00:00
Zuul
b7a459ae2a Merge "Elasticsearch: Add node selector to Curator cron job" 2018-10-09 20:26:28 +00:00
Mike Pham
0ad481d9c6 Add missing labels to cronJobs
This PS adds the missing labels so it is able to target by the network policy.

Change-Id: I5da7e0a9b17d00a5e98488780d72c77a93af999f
2018-10-09 13:58:17 -04:00
Steve Wilkerson
30f339a080 Elasticsearch: Add node selector to Curator cron job
This adds the node selector key and value configuration to the
Curator cron job for Elasticsearch, as it was previously omitted

Change-Id: Id702007fa827a1e1f90dee9b2a855e4197f4567c
2018-10-09 11:01:41 -05:00
Zuul
c685327406 Merge "Gate: refactor gate script running" 2018-10-09 16:00:28 +00:00
Zuul
bc7593b54c Merge "VBMC: Move and update vbmc image to osh-infra" 2018-10-09 15:49:37 +00:00
Tin Lam
f0f89b9425 Add configMap hash to annotation
This patch set adds in the annotation for the configmap-bin-hash for the
memcached chart.

Change-Id: I8d0e624af18165a1b146680eefa86f1184ddd924
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-09 14:16:17 +00:00
Zuul
2fa40d7603 Merge "[Calico] Update to v3.2.3" 2018-10-09 13:59:22 +00:00
Steve Wilkerson
1f4a890343 Elasticsearch: Update log4j2 configuration settings
This updates the configuration settings used for the log4j2
template for Elasticsearch. The previous settings weren't
compatible with the version of Elasticsearch currently being used
(5.6.4)

Change-Id: Id4b02ad022c46d599ae02ef77bb0f81f7e62c9e4
2018-10-09 08:01:39 -05:00
Tin Lam
1c92644007 Fix indentation
Fix an indentation in gotpl where the things are indented 9 spaces.

Change-Id: Ifa1fd1bb16a262b6f17287f6f9b3746db30486ba
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-08 21:11:18 -05:00
Gage Hugo
650214e149 Replace docker-py with docker
docker-py's last release is outdated[0], last updated Nov 2016 while
the more up-to-date "docker" release[1] is still maintained. This
changes the use of "docker-py" to "docker".

[0] https://pypi.org/project/docker-py/
[1] https://pypi.org/project/docker/

Change-Id: I78fe5e426631c5ea5e0d128dc30fd55c81cca2e0
2018-10-08 21:58:00 +00:00
Pete Birley
7f4a37440f VBMC: Move and update vbmc image to osh-infra
This PS updates and moves the vmbc image to osh infra.

Change-Id: I9f8d21df8974d1484d9f087ee296fede2a87e545
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-08 09:38:24 -05:00
Pete Birley
c7f7091739 Gate: refactor gate script running
This is the 1st in a series of commits to reduce the complexity
of the gate playbooks, follow on work will target gates more
specificly to both reduce infra load and improve coverage.

This PS also removes the unused ldap-deploy playbook.

Change-Id: Ie4ddabe86d71008611c6cc5015a927e32c54fa35
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-08 04:51:59 +00:00
Zuul
7c839c82b2 Merge "Gate: Move to K8s 1.10.8" 2018-10-06 06:30:08 +00:00
Chris Wedgwood
1e82ab4c45 [Calico] Update to v3.2.3
Update everything to v3.2.3 consistently.

Change-Id: I4728faeec1572e6d4921f5118a9baee1cd7422ed
2018-10-06 05:41:29 +00:00
Zuul
8d9c892c69 Merge "fix tox python3 overrides" 2018-10-06 05:16:18 +00:00
Zuul
77393ab143 Merge "Libvirt: Fix image" 2018-10-06 00:58:41 +00:00
Pete Birley
f8880d27ad Libvirt: Fix image
This PS fixes the libvirt image, buy removing the ubuntu-cloud
archive repo and pinning to a good version.

Change-Id: I5097d8893b92d020f7a5a1cb5925dec0b01d4da2
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-05 15:41:12 -05:00
Zuul
b4c49db89e Merge "Disable Ceph RADOS Gateway dynamic bucket resharding. Problem was discovered regarding issues being caused by RGW dynamic bucket resharding. It is at this time recommended to disable this feature." 2018-10-05 20:32:58 +00:00
Jean-Charles Lopez
05391252f8 Disable Ceph RADOS Gateway dynamic bucket resharding.
Problem was discovered regarding issues being caused by RGW dynamic
bucket resharding. It is at this time recommended to disable this feature.

Change-Id: Id524415f4ed08ee5374f7fd3b53f6e36c3ab084e
2018-10-05 16:20:10 +00:00
Doug Hellmann
e3641940dc fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I2ced79bfdfb8dd17f966fdf985e06b2f835cfa13
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-10-05 16:19:35 +00:00
Gupta, Sangeet (sg774j)
a34a7d8e50 Adding Falco
This commit adds falco daemonset of the node for behavioral activity
monitor designed to detect anomalous activity.

Change-Id: I783a2acc03592471c81a8a54e1dc0df140b34a42
2018-10-05 16:18:51 +00:00
Pete Birley
19376ee9e6 Gate: Move to K8s 1.10.8
This PS moves to use k8s 1.10.8, which includes a couple of fixes
for PVC mounts.

* https://github.com/kubernetes/kubernetes/pull/66863

Change-Id: Ica30950a8200f5755897b51fd2b4d24c69a10e61
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-05 07:44:42 -05:00
Chris Wedgwood
26e1b9cde6 [Calico] Update to Calico v3.2
Change-Id: I2214fea8d8c1563b08c4015c9e91a29cf071af5a
2018-10-05 01:49:42 +00:00
Pete Birley
46935734af Gate: remove unused pull images role and playbook
This PS removes the unused pull images role and playbook.

Change-Id: Ic26035c3f58efb6269fd58e570601cccfdd84949
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-05 01:43:40 +00:00
Pete Birley
feeeed4d5d Gate: Remove unused helm chart deployment role and playbook
This PS removes the unused helm chart deployment role and playbook.

Change-Id: I01c58a628589ec35af2557c8cc93ea47fe084089
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-05 01:43:33 +00:00
Pete Birley
16a6f7af8f Gate: Tolerate failures in log collection tasks
This PS updates the gate tasks to tollerate failures in the post
run log collection tasks.

Change-Id: I8b982112955f4112e8107a7eae35680aa68c87ab
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-05 01:42:56 +00:00
Zuul
98f310eee4 Merge "Gate: Update multinode jobs to five nodes" 2018-10-04 16:00:32 +00:00
Steve Wilkerson
18346cad8a Gate: Update multinode jobs to five nodes
This updates the multinode jobs to the five node jobs to attempt
to address resource issues encountered in the multinode jobs

Change-Id: If96a33099997aae2c7914a98332380ea32f2a3fe
2018-10-03 14:35:07 -05:00
Pete Birley
25985f7b43 Libvirt: escape kube cgroups and pid reaper
This PS moves to run the Libvirt process as a transient unit
on the host, free fom k8s controlled cgroups. In addition it
also uses the cloud archive provided libvirt/qemu packages.

Change-Id: Idfe9ae6f072acd86f877df0c3dfe3db4c20902d6
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-03 19:11:00 +00:00
Pete Birley
500698398d Gate: move centos to experimental untill we have optimised gates
This PS moves the centos job to experimental untill we have
done some optimisation on the gates

Change-Id: I3bfa8be9ac86025199060ec1ad9e7485bff30901
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-03 12:33:35 -05:00
Chris Wedgwood
45275ffefd [Calico] Realign Calico v2 chart for upcoming v3.2 upgrade
This PS realigns Calico v2 with the pending Calico v3.2 chart in order
to minimize differences.  It's mostly refactoring with a few small fixes.

Change-Id: Ie5157b4ae324b6eb4c8ccb5cc07d8b9bc5a83ebd
2018-10-03 03:38:25 +00:00
Zuul
681dee71b7 Merge "Fluentbit: Add kernel, kubelet, and dockerd logs" 2018-10-02 17:17:18 +00:00