1028 Commits

Author SHA1 Message Date
Zuul
5347636108 Merge "Elasticsearch: Add security context for exporter pod/container" 2019-01-07 16:26:08 +00:00
Zuul
0770465962 Merge "Uplift Ceph charts to the Mimic release" 2019-01-05 19:39:57 +00:00
Zuul
b70fe971c5 Merge "Helm-toolkit: Update job for creating s3 buckets" 2019-01-05 19:36:35 +00:00
Matthew Heler
c0d028e245 Uplift Ceph charts to the Mimic release
Change the release of Ceph from 12.2.3 (Luminous) to latest 13.2.2
(Mimic). Additionally use supported RHEL/Centos Images rather then
Ubuntu images, which are now considered deprecated by Redhat.

- Uplift all Ceph images to the latest 13.2.2 ceph-container images.
- RadosGW by default will now use the Beast backend.
- RadosGW has relaxed settings enabled for S3 naming conventions.
- Increased RadosGW resource limits due to backend change.
- All Luminous specific tests now test for both Luminous/Mimic.
- Gate scripts will remove all none required ceph packages. This is
required to not conflict with the pid/gid that the Redhat container
uses.

Change-Id: I9c00f3baa6c427e6223596ade95c65c331e763fb
2019-01-05 14:38:38 +00:00
Steve Wilkerson
8180635733 Helm-toolkit: Update job for creating s3 buckets
This updates the helm-toolkit manifest template and scipts for
creating an S3 bucket and linking it to a user. This moves away
from the previous python implementation that used rgwadmin, and
instead uses s3cmd for a cleaner approach that can support more
recent versions of ceph

Change-Id: I305062a5daa063bfe21a12448d7a3957bca00bf4
2019-01-05 14:37:47 +00:00
Zuul
ff51fd77e1 Merge "Parameterize hugepage pod cgroup" 2019-01-05 09:17:11 +00:00
weiyj
f5d1d6938c spelling error
Change-Id: Idd9e6bc6a1459e74703599b94f0b410fdb9f94ef
2019-01-05 04:39:23 +00:00
Zuul
754758e8a7 Merge "Kube-State-Metrics: Add pod/container security context" 2019-01-05 03:14:11 +00:00
Zuul
0b66795342 Merge "Grafana: Add pod security context for grafana user" 2019-01-04 10:08:33 +00:00
Zuul
9eb31f0374 Merge "'NOP' cleanup for more consistent white-space use in charts" 2019-01-04 07:32:39 +00:00
Zuul
6adecf2bea Merge "Ceph: Fix provisioner name substituation typo" 2019-01-04 05:08:28 +00:00
Chris Wedgwood
0c4e37391f 'NOP' cleanup for more consistent white-space use in charts
Where we have the style '{{ ...' we should use the style '... }}'.

Change-Id: Ic3e779e4681370d396f95d3804ca27db5b9d3642
2019-01-03 22:45:49 +00:00
Anderson, Craig (ca846m)
ff7985e391 Parameterize hugepage pod cgroup
Change-Id: I5a52397185610e19ce5861ce3c3b3303006a296b
2019-01-03 14:13:13 -08:00
Steve Wilkerson
0679ed49bd Elasticsearch: Add security context for exporter pod/container
This adds the security context snippet for the elasticsearch
prometheus exporter pod. This changes the pod's user from root to
the nobody user instead

This also adds the container security context to explicitly set
allowPrivilegeEscalation to false

Change-Id: If692fccaf4dd362b28fecb4656036289a3a97122
2019-01-03 16:12:17 -06:00
Steve Wilkerson
4d50e6fa7a Kube-State-Metrics: Add pod/container security context
This updates the kube-state-metrics chart to include the pod
security context on the pod template. This changes the pod's
user from root to the nobody user instead

This also adds the container security context to explicitly set
allowPrivilegeEscalation to false

Change-Id: I17748b299a6e7a394cae63a0e713c49fbf68b4eb
2019-01-03 16:08:22 -06:00
Zuul
3819986398 Merge "Fluent-logging: Add input and output for qemu logs" 2019-01-03 20:13:07 +00:00
Zuul
22e1eba890 Merge "Create Kibana Index Patterns" 2019-01-03 20:13:06 +00:00
Pete Birley
f8b8ac9623 Ceph: Fix provisioner name substituation typo
This PS fixes a typo in the cephfs provisioner name, which was being given the
same key as rbd to look for.

Change-Id: I84dc541a103fc61feb1998ab41edd602c17e2b6f
Signed-off-by: Pete Birley <pete@port.direct>
2019-01-03 13:55:44 -06:00
Steve Wilkerson
680f920312 Grafana: Add pod security context for grafana user
This updates the Grafana chart to include the pod security context
on the grafana pod. This changes the pod's user from root to the
grafana user instead

Change-Id: Id64853640f1941001b83566865defe93227b4291
2019-01-03 12:42:52 -06:00
Huang, Scott (sh2725)
ed7b5e4024 Create Kibana Index Patterns
Use Kibana REST API to create Kibana index patterns and set a default
index pattern.
Script calling Kibana REST API is executed using a Job, and the specific
index patterns are configurable in values.yaml.

Change-Id: I1ca6dd9609e6d62d1ce749ee09e1490d51659709
2019-01-02 17:26:38 -05:00
Steve Wilkerson
5c4e77d816 Fluent-logging: Add input and output for qemu logs
This adds an input to Fluentbit for capturing all qemu instance
logs in /var/log/libvirt/qemu/, and adds an Elasticsearch output
for those entries

Change-Id: I0802023f9861a5944e7989fd5469133c325349e7
2019-01-02 15:29:03 -06:00
Zuul
8dba8cb648 Merge "[CEPH] Cleanup the ceph-osd helm-chart" 2019-01-02 16:42:37 +00:00
Zuul
98f298fda8 Merge "[Calico] Update to v3.4.0" 2019-01-02 16:20:05 +00:00
Zuul
0cefed2c0f Merge "[Calico] Update to v3.3.2" 2019-01-02 16:19:18 +00:00
Chris Wedgwood
239cc91f7a [Calico] Update to v3.4.0
Calico v3.4.0 update.

Notable features upstream:

 * Advertising Kubernetes Service ClusterIPs
   https://docs.projectcalico.org/master/usage/service-advertisement

 * Host * endpoints (wildcard interfaceName)
   https://docs.projectcalico.org/v3.4/reference/calicoctl/resources/hostendpoint

Change-Id: Iaebfcae08d622b75005e4e6df55e9eac34317cc2
2018-12-30 06:17:05 +00:00
Chris Wedgwood
49604fbd17 [Calico] Update to v3.3.2
Calico v3.3.2 update with:

 * update container images
 * update configuration
   * expose  ipPool blockSize
 * update bird templates
 * update rbac

Change-Id: I72d218cb55a70b72c4d7d940109e718fe44a094f
2018-12-30 06:15:50 +00:00
Matthew Heler
e581a79807 [CEPH] Cleanup the ceph-osd helm-chart
- Split off duplicate code across multiple bash scripts into a common
file.
- Simplify the way journals are detected for block devices.
- Cleanup unused portions of the code.
- Standardize the syntax across all the code.
- Use sgdisk for zapping disks rather then ceph-disk.

Change-Id: I13e4a89cab3ee454dd36b5cdedfa2f341bf50b87
2018-12-28 13:09:21 -06:00
Steve Wilkerson
281b0799f0 Write libvirt logs to host
This modifies the libvirt chart to write logs directly to the
host by default. This also modifies the fluentbit and fluentd
charts to capture libvirt logs from the host and index them into
Elasticsearch

Change-Id: I0bbc49d2c0d4cf4895f797e48f309f308ffd021f
2018-12-28 17:43:12 +00:00
Zuul
13a58c5530 Merge "[Calico] Update to v3.2.4" 2018-12-27 20:07:16 +00:00
Zuul
5cca3e74d4 Merge "[CEPH] Fix race conditions with OSD POD initialization" 2018-12-24 22:48:53 +00:00
Matthew Heler
89745aad06 [Ceph] Update rbd-provisioner and cephfs-provisioner
- Move from docker tag v0.1.1 to v1.1.0-k8s1.10

Change-Id: I5a2afbdeb87c732a17da64916de8bb301f12cbb3
2018-12-22 17:31:29 +00:00
Matthew Heler
30b57ba671 [CEPH] Fix race conditions with OSD POD initialization
Under POD restart conditions there is a race condition with lsblk
causing the helm chart to zap a fully working OSD disk. We refactor
the code to remove this requirement.

Additonally the new automatic journal partitioning code has a race
condition in which the same journal partition could be picked twice
for OSDs on the same node. To resolve this we share a common tmp
directory from the node to all of the OSD pods on that node.

Change-Id: I807074c4c5e54b953b5c0efa4c169763c5629062
2018-12-21 15:05:54 -06:00
Zuul
0513c779bd Merge "(calico) Add network policy safety valve" 2018-12-21 08:06:29 +00:00
Zuul
5ea831964e Merge "[gnocchi] don't randomize job names" 2018-12-21 04:34:38 +00:00
Scott Hussey
048b18a50f (calico) Add network policy safety valve
- If a rule set in the network policy override for the calico
  chart is empty, it causes the calico-settings job to fail. This
  safety valve should handle the empty list gracefully.

Change-Id: I4b8a39941f05a8eb86734ff129b2d73830883236
2018-12-20 11:02:32 -06:00
Chris Wedgwood
41508d39e2 [Calico] Update to v3.2.4
Upstream container updates only, no chart changes required.

Change-Id: I3cdc6f23269a5beac231575ac1b5faf654e424b7
2018-12-19 17:18:32 +00:00
Matthew Heler
e1a3819a0d [CEPH] Support a troubleshooting option to reset PG metadata
Ceph upstream bug: https://tracker.ceph.com/issues/21142 is
impacting the availability of our sites in pipeline. Add an option
to reset the past interval metadata time on an OSDs PG to solve for
this issue if it occurs.

Change-Id: I1fe0bee6ce8aa402c241f1ad457bbf532945a530
2018-12-18 23:26:18 -06:00
Zuul
4233c25308 Merge "[Ceph] Tunables for rgw buckets" 2018-12-18 17:37:10 +00:00
Zuul
1b0d47bb01 Merge "[Ceph] Update ceph helm tests" 2018-12-17 18:23:51 +00:00
Matthew Heler
54efa7922d [Ceph] Tunables for rgw buckets
Set rgw_override_bucket_index_max_shards to 8 (default: 0)

By default create 8 shards per a bucket with Ceph RagosGW. This allows
up to ~800k-1M objects to be in a bucket before seeing performance slow-
downs. The only downside to this change is that a directory listing for
a bucket may take slightly longer to finish.

Change-Id: I96c7ac81501a41d29927e102a6029bf432bd3d21
2018-12-16 19:35:00 +00:00
Zuul
bc32affe0c Merge "[Calico] Logging fixes/updates" 2018-12-16 16:52:46 +00:00
Zuul
6d354f0f7b Merge "Revert "Add Egress Helm-toolkit function & enforce the nework policy at OSH-INFRA"" 2018-12-16 08:57:09 +00:00
Chris Wedgwood
3f79066797 [Calico] Logging fixes/updates
Expose the early logging level for calico-node.

Use conf.node.FELIX_LOGSEVERITYSCREEN to set logging level in
BGPConfiguration and FelixConfiguration (whilst this is an odd
name/location it backwards compatible and will in most cases set
things as expected).

Change-Id: I70c3028423eddb4721456f645c4475da4af7ced5
2018-12-16 07:21:31 +00:00
Pete Birley
0bf3674539 Revert "Add Egress Helm-toolkit function & enforce the nework policy at OSH-INFRA"
This reverts commit 8d33a2911cda0c9e88406b9eeacbd8dfa70286f2.

Change-Id: Ic861b9bf9b337449b47a3558da8355e7a5bcacee
2018-12-16 04:21:46 +00:00
Zuul
158c223256 Merge "Ceph: Allow multiple test pods for ceph-client to be present in clusters" 2018-12-16 03:30:03 +00:00
Matthew Heler
de69c68365 [Ceph] Update ceph helm tests
- Ensure the helm tests are logging all commands and variables

Change-Id: I4f4c553a3fbb4d77e9d1ab41c1c0c763c963cfd3
2018-12-15 13:47:43 -06:00
Zuul
b90bf10b89 Merge "Add Egress Helm-toolkit function & enforce the nework policy at OSH-INFRA" 2018-12-15 09:32:21 +00:00
Mike Pham
8d33a2911c Add Egress Helm-toolkit function & enforce the nework policy at OSH-INFRA
This PS implements the helm toolkit function to generate the
Egress in kubernetes network policy manifest based on overrideable values.
It also enbale the K8s network policy at Osh-infra gate.

Change-Id: Icbe2a18c98dba795d15398dcdcac64228f6a7b4c
2018-12-14 16:32:40 -05:00
Zuul
96ef3188aa Merge "Revert "helm-toolkit: Support standard kubernetes/helm labels"" 2018-12-14 20:36:42 +00:00
Zuul
ef0f26988f Merge "Helm-toolkit: Fix hasKey call for security context snippet" 2018-12-14 20:01:53 +00:00