64 Commits

Author SHA1 Message Date
Steve Wilkerson
b50fae62a4 Update kubernetes-entrypoint image reference
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintained

Change-Id: I5bfdc156ae228ab16da57569ac6b05a9a125cb6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-10-18 18:20:11 +00:00
Kabanov, Dmitrii
8acb9e0f39 [ceph-client] Remove explicit call to Python2
The PS removes the explicit call to Python2 from _checkPGs.py file.

Change-Id: If8cf5b9e1347997bb5b2fe13155d377736f288da
2019-10-14 23:42:27 +00:00
Andrii Ostapenko
fdcc9b7e0e Make all prints python3 compatible
Change-Id: Ie5a08859010453d276b42253f5f2130f80b82224
2019-10-01 01:28:35 +00:00
Chinasubbareddy Mallavarapu
acd5d11bc2 Ceph-Client: update configmap name for defragosds cronjob
This is to update configmap names using by defragosds cronjob.

Change-Id: I29608cd8b6ce1e30615a0f92853939d7bbae9972
2019-08-01 12:22:48 -05:00
Daniel Pawlik
0b58aea135 Fix mon_host hosts when hostname contains 'ip'
Ceph-mon template script parse mon_host in wrong way, when
hostname contains'ip' word, e.g.: airship.

Change-Id: I0a097443d42ad2e9b6be6c61facd7932ddb4b3bb
Story: 2006255
2019-07-19 10:49:50 +00:00
Zuul
bf752cdf89 Merge "[Ceph Enhancement] Move ceph-defragosds cron job to ceph-client chart" 2019-07-04 00:36:13 +00:00
Renis Makadia
25f4f17f8e [Ceph Enhancement] Move ceph-defragosds cron job to ceph-client chart
- Move the cron manifests to ceph-client chart
- Keep the script that actually does the work in Ceph-OSD
- with this PS, ceph-defragosds will be started after Ceph-Client chart
gets deployed. In the cronjob, it will exec to a running OSD pod and
execute the script.

Change-Id: I6e7f7b32572308345963728f2f884c1514ca122d
2019-07-03 21:08:37 +00:00
Chinasubbareddy Mallavarapu
8275ad7d2f Ceph-Client: Cleanup for unused values
This is just to cleanup client chart for provisioner references.

Change-Id: I4b2e4b0eee244d102b503bee4adbbae857863c8d
2019-07-03 12:32:34 +00:00
Zuul
4ed7bf33b3 Merge "Ceph-Client: Wait for inactive pgs in ceph cluster" 2019-07-03 04:28:45 +00:00
Chinasubbareddy Mallavarapu
897ebbc75c Ceph-client: update pg validation in helm tests.
This is to update the logic to check for incomplete pgs in ceph
cluster and proceed if there are no incomplete/inactive pgs and
will not wait for healthy ceph cluster.

Change-Id: I026d6cc378053e805680c31d75fdfb40bbb636f5
2019-07-02 12:38:44 -05:00
Chinasubbareddy Mallavarapu
b69f393b54 Ceph-Client: Wait for inactive pgs in ceph cluster
This is to add wait logic for inactive pgs after adjusting pgs in
manage pool job.

Change-Id: I3353262644ae649ed9f495ac83a2567d9da263ae
2019-07-02 11:50:20 -05:00
Chinasubbareddy Mallavarapu
961a58dada Ceph-Client: allow helm test to proceed with 80% of osds avialble
This is to adjust helm test logic to  proceed the deployment if 80% of
osds are up  and running in the cluster .

Change-Id: I128266fd374426f75928332690e275b7f0175318
2019-07-01 15:11:43 +00:00
Zuul
b37405dbe6 Merge "Ceph-Clients: Explicity set prom exporter params" 2019-06-27 16:58:52 +00:00
Pete Birley
57e1e3ce40 Ceph-Clients: Explicity set prom exporter params
Occasionally the default config can result in attempts
to bind to ipv6 which fail - so we explicity set the
host to ipv4.

Change-Id: I3c01ed0ef7c84cf779d88386c14f7c7bd2003310
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-27 14:03:58 +00:00
Pete Birley
248df772fd Ceph-Mgr: update how we set config params
This PS updates the start script to use `config set`, rather
than `config-key set` which has been depricated in Mimic.

Change-Id: I97d0c4385b016d73aa362c0fc293d235b532810c
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-27 01:37:02 +00:00
Dejaeger, Darren (dd118r)
083956e37b Add node selector to Ceph test pods
This PS looks to add a node selector into the Ceph test pod's
specs.

Change-Id: If73a5036c5e6a651393f81a136874b9e8a52b4f1
2019-06-24 17:41:09 +00:00
Taylor, Stephen (st053q)
d74b579f1d [Ceph] Implement pool quotas on pools as they are created/managed
This patch set implements pool quotas on each pool in the Ceph
cluster by obtaining the total capacity of the cluster in bytes,
multiplying that by the defined percentage of total data expected
to reside in each pool and by the cluster quota, and setting a
byte quota on each pool that is equal to its expected percentage
of the total cluster quota.

Change-Id: I1686822a74c984e99e9347f55b98219c47decec1
2019-06-20 10:16:47 -05:00
Venkata, Krishna (kv988c)
ed574f456f Switch from default values being populated
for upgrade strategy for ceph components

This PS uses HelmToolKit function to add
upgrade strategy parameters to ceph Components

Change-Id: I54e71d2a52bd639b3e93fc899c1bf2cd075b5396
2019-06-18 01:42:38 +00:00
Chinasubbareddy Mallavarapu
9e653096e1 [ceph-client] update dependncy for test pod
This is to update test pod dependency  since its getting started
right after mgr service availbe and mgr pods are in init state and
waiting for rbd-pool job.

Change-Id: Iaf9af3ffcf1f4940c1b661a853df0ec4edd99d39
2019-06-06 16:50:06 -05:00
Zuul
e2cd5d99f9 Merge "[ceph-client] Handle pool min_size during ceph cluster expansion" 2019-06-06 20:45:48 +00:00
Chinasubbareddy Mallavarapu
6767d0ad48 [ceph-client] Handle pool min_size during ceph cluster expansion
This is to update logic for  pool min_size parameter  as  this is
not getting changed when replication changes from intilization.

Change-Id: I30f99aaf92c3dc83afce10534b1d2ac9402b7fa7
2019-06-06 15:20:43 +00:00
RAHUL KHIYANI
789fa7a4e5 Ceph-client: Fix security context for pod/container
This changes the  user from root to the nobody user instead
in ceph-client chart wherever needed

Change-Id: I4c56b97f85093bbbaaef617f1981f67215a8bc00
2019-05-29 15:32:28 +00:00
Venkata, Krishna (kv988c)
1ad5857957 Fetch Total_OSD value from
ceph-client/values.yaml

Change-Id: I6af79af0627f85550655f5066dd6278bf0b5365a
2019-05-21 17:15:48 -05:00
Renis Makadia
5985b61286 Ceph-Client: Update, Enable and Cleanup helm tests
- Update ceph-client chart to
1) By default, enable ceph-client helm test. Update enabler
key in values.yaml to follow pattern as in other charts
2) Add needed dependancy for ceph-client helm tests
3) Update helm test script to reduce output and update
error msgs
4) Removed unwanted ENV variables SPECS and EXPECTED_POOLMINSIZE
- Update gate scripts to run helm test command

Change-Id: I6a0e4f5107e49dac081ac2037bcc0f9c0864793f
2019-05-18 03:09:45 +00:00
Roy Tang (rt7380)
85bd731562 Expose Anti-Affinity Weight Setting
This ps exposes the anti-affinity weight value, including
default, that will be consumed by the updated htk function.

Change-Id: Id8eb303674764ef8b0664f62040723aaf77e0a54
2019-05-14 17:04:52 -05:00
Dmitrii Kabanov
b7c07a595c Update image links for Ceph.
Updated the links according to the decision here [1].
The convention for images is <imagename>:<tagging>
where <tagging> takes the format <version>-<distro>.

[1]: http://eavesdrop.openstack.org/irclogs/%23openstack-helm/%23openstack-helm.2019-03-07.log.html#t2019-03-07T17:50:58

Change-Id: I84f8ce731e428f8ef035c008ff615e712c78a1f4
2019-04-24 14:53:42 +00:00
Pete Birley
f569cf0d5c Ceph-Client: Add security context to chart and read-only-fs
This PS adds the security context macros to the ceph-client chart,
and moves the default to read-only-rootfs for all containers.

Change-Id: I2fe03f31cc59e1cda2bf0396ae6e3aca5c440a16
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-21 19:06:24 +00:00
Pete Birley
dece008337 Ceph: Make /etc/ceph and /run emptydirs uniformly across all pods
This PS updates the ceph charts to make /etc/ceph an emptydir
uniformly across all charts, both ensuring no default config is loaded,
and also permitting read-only filesystems to back the containers.

Additionally /run is uniformly applied across all long running pods
as a memory backed emptydir.

Change-Id: I00d1b15758b7eb4476fb950ddcb38db9a5149ad0
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-21 19:06:18 +00:00
Pete Birley
2abf62ff4d OSH-Infra: Add emptydirs for tmp
This PS adds emptydirs backing the /tmp directory in pods, which
is required in most cases for full operation when using a read only
filesystem backing the container.

Additionally some yaml indent issues are resolved.

Change-Id: I8b7f1614da059783254aa6efc09facf23fca3cad
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-20 20:50:59 +00:00
Steve Wilkerson
84f30ec103 Add release-annotation to pod spec, add missing annotations
This adds the release-annotation to the pod spec for the charts in
openstack-helm-infra. This also adds missing configmap annotations
to charts in openstack-helm-infra

Change-Id: Ie23f0c16a7a21d3929e98928db2bbcef69ae6490
2019-03-21 09:10:48 -05:00
Steve Taylor
65de349d58 Move ceph-mon's checkPGs cron job to ceph-client
- Move the cronjob from ceph-mon to ceph-client
- Adding ceph-rbd-pool job as dependencies for cronjob
- checkPGs manifest set to true so it will always run
in gate.

Co-Authored-By: Chinasubbareddy Mallavarapu <cr3938@att.com>,
                Renis Makadia <renis.makadia@att.com>

Change-Id: I9855d8d22265e78c7e2f5fa7ece69c9ff532ecb2
2019-03-19 20:53:08 +00:00
Zuul
d3531f1fba Merge "[CEPH] Enable iostat mgr module" 2019-02-04 09:45:13 +00:00
Matthew Heler
5de299b9b1 [CEPH] Enable iostat mgr module
Enable the iostat mgr module for Ceph. This module show the
current throughput and IOPS done on a Ceph cluster.

Change-Id: I2fe5b47401c15e349a49f345bacd99da39889373
2019-02-02 16:44:58 -06:00
Pete Birley
6ea6a85198 Ceph: Update default to use OSH image
This PS udpates the default image in the chart to the latest OSH image.

Change-Id: Ib8d2a72ad48049fe02560dc4405f0088890b6f64
Signed-off-by: Pete Birley <pete@port.direct>
2019-02-01 21:25:13 +00:00
Chinasubbareddy M
1abcde851e Ceph : cleanup ceph charts
This is  to clean ceph charts for unused variables and left over from
ceph chart split

Change-Id: Iec50599a031ae7acacc8eb0504f7146647450306
2019-01-11 19:53:36 +00:00
Matthew Heler
c0d028e245 Uplift Ceph charts to the Mimic release
Change the release of Ceph from 12.2.3 (Luminous) to latest 13.2.2
(Mimic). Additionally use supported RHEL/Centos Images rather then
Ubuntu images, which are now considered deprecated by Redhat.

- Uplift all Ceph images to the latest 13.2.2 ceph-container images.
- RadosGW by default will now use the Beast backend.
- RadosGW has relaxed settings enabled for S3 naming conventions.
- Increased RadosGW resource limits due to backend change.
- All Luminous specific tests now test for both Luminous/Mimic.
- Gate scripts will remove all none required ceph packages. This is
required to not conflict with the pid/gid that the Redhat container
uses.

Change-Id: I9c00f3baa6c427e6223596ade95c65c331e763fb
2019-01-05 14:38:38 +00:00
Chris Wedgwood
0c4e37391f 'NOP' cleanup for more consistent white-space use in charts
Where we have the style '{{ ...' we should use the style '... }}'.

Change-Id: Ic3e779e4681370d396f95d3804ca27db5b9d3642
2019-01-03 22:45:49 +00:00
Zuul
1b0d47bb01 Merge "[Ceph] Update ceph helm tests" 2018-12-17 18:23:51 +00:00
Matthew Heler
de69c68365 [Ceph] Update ceph helm tests
- Ensure the helm tests are logging all commands and variables

Change-Id: I4f4c553a3fbb4d77e9d1ab41c1c0c763c963cfd3
2018-12-15 13:47:43 -06:00
Pete Birley
5695a15f93 Ceph: Allow multiple test pods for ceph-client to be present in clusters
This ps allows multiple ceph test pods to be present in cluster with
more than one ceph deployment.

Change-Id: Ib8be8fc58e3a374dfcf6845988668433cf43655a
Signed-off-by: Pete Birley <pete@port.direct>
2018-12-12 10:03:15 -06:00
Renis Makadia
5aaa59f23f Disable Ceph helm test while being updated.
Change-Id: I6011ee6f41044647dea43fa2848058d723e194b8
2018-12-06 15:47:08 +00:00
Matthew Heler
35cce6cb43 Switch Ceph to IPs when DNS is down
Add helper scripts that are called by a POD to switch
Ceph from DNS to IPs. This POD will loop every 5 minutes
to catch cases where the DNS might be unavailable.

On a POD's Service start switch ceph.conf to using IPs rather
then DNS.

Change-Id: I402199f55792ca9f5f28e436ff44d4a6ac9b7cf9
2018-12-03 10:51:37 -06:00
Renis Makadia
b1005b23b4 Helm tests for Ceph-OSD and Ceph-Client charts
Change-Id: If4a846f0593b8679558662205a8560aa3cbb18ae
2018-12-01 08:08:00 +00:00
Matthew Heler
6e8c289c13 Add failure domains, and device classes for custom CRUSH rules
Largely inspired and taken from Kranthi's PS.

 - Add support for creating custom CRUSH rules based off of failure
domains and device classes (ssd & hdd)
- Basic logic around the PG calculator to autodetect the number of
OSDs globally and per device class (required when using custom crush
rules that specify device classes).

Change-Id: I13a6f5eb21494746c2b77e340e8d0dcb0d81a591
2018-11-27 09:37:30 -06:00
Matthew Heler
5ce9f2eb3b Enable Ceph charts to be rack aware for CRUSH
Add support for a rack level CRUSH map. Rack level CRUSH support is
enabled by using the "rack_replicated_rule" crush rule.

Change-Id: I4df224f2821872faa2eddec2120832e9a22f4a7c
2018-11-20 09:07:36 -06:00
Zuul
fca344900f Merge "Enable the mgr balancer module by default." 2018-11-02 22:36:13 +00:00
Matthew Heler
a79562a28b Enable the mgr balancer module by default.
The balancer module will distribute PGs more evenly across OSDs. 
While CRUSH does a good job at this, it is not perfect and hot spots
(where an OSD has more PGs then it's peers) can occur.

Change-Id: Ic45a6bf745bdd09a3f5782e9e8bda89c3d3da2aa
2018-11-01 15:52:51 +00:00
Matthew Heler
6ef48d3706 Further performance tuning changes for Ceph
- Throttle down snap trimming as to lessen it's performance impact
(Setting just osd_snap_trim_priority isn't effective enough to throttle
down the impact)
osd_snap_trim_sleep: 0.1 (default 0)
osd_pg_max_concurrent_snap_trims: 1 (default 2)

- Align filestore_merge_threshold with upstream Ceph values
(A negative number disables this function, no change in behavior)
filestore_merge_threshold: -10 (formerly -50, default 10)

- Increase RGW pool thread size for more concurrent connections
rgw_thread_pool_size: 512 (default 100)

- Disable in-memory logs for the ms subsytem.
debug_ms: 0/0 (default 0/5)

- Formating cleanups

Change-Id: I4aefcb6e774cb3e1252e52ca6003cec495556467
2018-10-26 15:10:50 +00:00
Pete Birley
1144ccbbb2 Ceph: Update MGR check to allow use on hosts with fqdns defined
This PS updates the mgr check to allow use on hosts with fqdns
defined.

Change-Id: If1cb740e8093fbcafce846234c96db931409b436
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-24 00:57:12 +00:00
Chinasubbareddy M
e23e372120 ceph-mgr: make prometheus module port configurable
this is to give example for prometheus module port configurable

Change-Id: I66844bb8ee59a58f7bfd3e3002a183779810e881
2018-10-23 15:40:43 -05:00