78 Commits

Author SHA1 Message Date
Chinasubbareddy Mallavarapu
da289c78cb [CEPH] Uplift from Nautilus to Octopus release
This is to uplift ceph charts from 14.X release to 15.X

Change-Id: I4f7913967185dd52d4301c218450cfad9d0e2b2b
2021-02-03 22:34:53 +00:00
Ritchie, Frank (fr801x)
75a115ea29 Run mon container as ceph user
This PS is to address security best practices concerning running
containers as a non-privileged user and disallowing privilege
escalation.

Change-Id: If4c0e9fe446091ba75d1a9818ffd3a0933285af4
2021-01-26 20:14:27 +00:00
Frank Ritchie
9b1ac0ffcb Enable shareProcessNamespace in mon daemonset
This is to address zombie processes found in ceph-mon containers due
to the mon-check.sh monitoring script. With shareProcessNamespace the
/pause container will properly handle the defunct processes.

Change-Id: Ic111fd28b517f4c9b59ab23626753e9c73db1b1b
2020-12-11 11:57:39 -05:00
Andrii Ostapenko
1532958c80
Change helm-toolkit dependency version to ">= 0.1.0"
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0

Change-Id: I15950b735b4f8566bc0018fe4f4ea9ba729235fc
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-24 12:19:28 -05:00
Mohammed Naser
c7a45f166f Run chart-testing on all charts
Added chart lint in zuul CI to enhance the stability for charts.
Fixed some lint errors in the current charts.

Change-Id: I9df4024c7ccf8b3510e665fc07ba0f38871fcbdb
2020-09-11 18:02:38 +03:00
diwakar thyagaraj
5d50433362 Enable Application Armor to all ceph key-generator pods.
1) Changed the pod name and container name to pick name dynamically for
   osd,mon,mgr and mds.

2) Added Init container for ceph-provisioners.

Change-Id: I3e27d51c055010cff982ddb0951d01ea8adac234
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-07-20 01:29:42 +00:00
Andrii Ostapenko
41f02d3c98
Fix service account name for ceph-mon keyring generator
Fix issues introduced by https://review.opendev.org/#/c/735648
with extra 'ceph-' in service_account and security context not
rendered for keyring generator containers.

Change-Id: Ie53b3407dbd7345d37c92c60a04f3badf735f6a6
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-07 15:06:23 -05:00
Andrii Ostapenko
824f168efc Undo octal-values restriction together with corresponding code
Unrestrict octal values rule since benefits of file modes readability
exceed possible issues with yaml 1.2 adoption in future k8s versions.
These issues will be addressed when/if they occur.

Also ensure osh-infra is a required project for lint job, that matters
when running job against another project.

Change-Id: Ic5e327cf40c4b09c90738baff56419a6cef132da
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-07 15:42:53 +00:00
Andrii Ostapenko
83e27e600c Enable key-duplicates and octal-values yamllint checks
With corresponding code changes.

Change-Id: I11cde8971b3effbb6eb2b69a7d31ecf12140434e
2020-06-17 13:14:30 -05:00
Andrii Ostapenko
dfb32ccf60 Enable yamllint rules for templates
- braces
- brackets
- colons
- commas
- comments
- comments-indentation
- document-start
- hyphens
- indentation

With corresponding code changes.

Also idempotency fix for lint script.

Change-Id: Ibe5281cbb4ad7970e92f3d1f921abb1efc89dc3b
2020-06-17 13:13:53 -05:00
KHIYANI, RAHUL (rk0850)
ddfa7a4741 Ceph-mon: Add pod/container security context
This updates the ceph-mon chart to include the pod
security context on the pod template

This also adds the container security context to set
readOnlyRootFilesystem flag to true

Change-Id: I4c9e292eaf3d76ee80f50553d1cbc8cdc6f57cac
2020-06-16 23:59:34 +00:00
Andrii Ostapenko
8f24a74bc7 Introduces templates linting
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.

Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.

Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
2020-06-11 23:29:42 -05:00
Andrii Ostapenko
731a6b4cfa Enable yamllint checks
- document-end
- document-start
- empty-lines
- hyphens
- indentation
- key-duplicates
- new-line-at-end-of-file
- new-lines
- octal-values

with corresponding code adjustment.

Change-Id: I92d6aa20df82aa0fe198f8ccd535cfcaf613f43a
2020-05-29 19:49:05 +00:00
Kabanov, Dmitrii
46930fcd06 [Ceph] Upgrade Ceph from 14.2.8 to 14.2.9 version
The PS upgrades Ceph to 14.2.9 version.

Change-Id: I72a2e39a7b4294ac8fd42b1dbc78579c2c0ae791
2020-05-28 15:46:47 +00:00
Kabanov, Dmitrii
2aa6b3cf1c [Ceph] Add kubernetes tolerations for ceph deployments
The PS adds kubernetes tolerations for deployments from ceph-client,
ceph-mon, ceph-provisioners and ceph-rgw charts.

Change-Id: If96f5f2058fca6e145e537e95af39089f441ccbb
2020-05-20 19:32:06 +00:00
Gage Hugo
d14d826b26 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
2020-05-07 02:11:15 +00:00
diwakar thyagaraj
ccaa11b649 Enable Apparmor to ceph client bootstrap Pods
Change-Id: Ia8fd1e50a2478743f0ff625ffdd8801610f05ee1
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-29 14:43:50 +00:00
diwakar thyagaraj
fb0cd00a56 Enable Apparmor to ceph-bootstrap Pods
Change-Id: Ifa8d43a2a68fffaea554f04a5df63fb6b7ea5422
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-24 02:22:17 +00:00
Chinasubbareddy Mallavarapu
8d9b7fdd2b [ceph-mon-check] fix the command to connect correct ceph cluster
This is to fix the command to connect to the cluster name instead of
namesapce.

Change-Id: I8b8f7c10d7667245a8f6cb02fb5b69dd122099e5
2020-04-22 20:02:14 +00:00
Taylor, Stephen (st053q)
9057c770a6 Enable cephfs tests
Cephfs tests were disabled in order to merge
https://review.opendev.org/695568 due to gate failures that were
blocking it. CephFS isn't used in openstack-helm-infra, so it
wasn't required for that work. This change re-enables the cephfs
tests so we can work through any issues that are causing further
failures.

Since the the issue got fixed in 14.2.8 , upgrading all daemons to 14.2.8.
(https://tracker.ceph.com/issues/43770)

Change-Id: I376d39b7ee00ccb1ab8046b58f92b19a822272e1
2020-04-21 22:25:52 +00:00
Wickersham, Brian (bw6938)
5952acdd93 [ceph-osd] A rack's OSDs are not marked out after down_out interval
An entire rack's OSDs are not being marked out after
down_out interval. This manifested itself during
resiliency testing when all interfaces were brought
down on a control plan host and the down_interval
was surpassed.

Change-Id: I6f4a69ec442c3e768feb7bd74c7d610aa9d4aa67
2020-04-16 16:13:21 +00:00
Pete Birley
19a1fbf8f7 Ceph: Dont mount log directories to host
This PS updates the bind mounts for ceph logs directorys to be
emptydirs. This ensures we do not polute the hosts permanantly
with ceph logs, which should be directed to stdout.

Change-Id: I6d72c0864b9ecc493cd62564e0e0450d90cfcf00
Signed-off-by: Pete Birley <pete@port.direct>
2020-04-07 18:05:04 +00:00
diwakar thyagaraj
868bd18cf1 Remove Duplicate Apparmor configs from values
Since apparmor configs are moved to value overrides, removing this.

Change-Id: Ia23c34c2ed76fceb78f68e609066139b69e09e61
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-04-01 18:41:05 +00:00
Chinasubbareddy Mallavarapu
977a5a2f97 [ceph-daemons] Redirect all the logs to stdout
This is to redirect all the logs from daemons to stdout to avoid
accumulating large sized log files on filesystem.

NOTE: The ceph-osd daemon won't work this way and is addressed
separately in https://review.opendev.org/715295. All other Ceph
daemons are included here.

Change-Id: I3045d6e941791aba14979472fac1bca09776d3bf
2020-03-29 10:27:44 -06:00
Chinasubbareddy Mallavarapu
f098f760f0 [ceph-mon] update stop script not to remove mons from monmap
This is to update ceph-mon stop script  not to remove mons from
monmap as in multinode clusters three mons in the monmap are required
to handle the quorum properly.

Change-Id: I0dd643007ea0558244bfecae1d90db78828e9834
2020-03-09 20:11:23 +00:00
Chinasubbareddy Mallavarapu
7425e3e5c0 [CEPH] update all ceph daemons startup scripts to support msgr2
This is to update all ceph daemons startup scripts as per msgr2 protocol and
also to update v2 port for mon_host config.
This also removes setting mon_addr config since we already have mon_host config.

v1 default port: 6789
V2 default port: 3300

Change-Id: I3d95edbd89f5ac8b40a34f41c1099311cee4f875
2020-03-04 23:22:02 -06:00
Chinasubbareddy Mallavarapu
07c62ddfd5 [ceph-mon] Add mon hosts as per msgr2 protocol
This is to update mon_host configuration to support both v1 and v2
of messenger.

ex: mon_host = [v1:172.29.0.11:6790/0,v2:172.29.0.11:3300/0]

Change-Id: I02785ea42c07d1aecbef2cf0c32dd6a1a236659f
Signed-off-by: Pete Birley <pete@port.direct>
2020-02-27 20:36:16 +00:00
diwakar thyagaraj
344c0543fa Enable runtime Apparmor default for All Ceph Components
Change-Id: Id62fe453846ffe6ab01198177d5d8046378d61bf
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-02-24 15:52:11 +00:00
Chinasubbareddy Mallavarapu
281b2018c2 CEPH: upgrade ceph version from 14.2.5 to 14.2.7
This is to upgrade ceph version from 14.2.5 from 14.2.7  and also
to update ceph provisioners to use latest code from quay.io

  - rbd-provisioner: quay.io/external_storage/rbd-provisioner:v2.1.1-k8s1.11
  - cephfs-provisioner: quay.io/external_storage/cephfs-provisioner:v2.1.0-k8s1.11

This also updates verbs for proivioner's clusterrole to support new code.

Change-Id: Ia94129574610bb5c800a6941804e58ca3aefce65
2020-02-18 21:29:23 +00:00
Zuul
75474c01b8 Merge "Address bandit gate failures" 2020-02-18 17:35:45 +00:00
Chinasubbareddy Mallavarapu
622f604cbe [Ceph-Mon] Check for ceph-mon messenger V2
This adds a new check to make sure msgr2 is enabled if it is
supported by all of the mons. When mon quorum is lost the
mons revert to the v1 protocol, which results in a Ceph
warning state if v2 is supported by all of the available
mons.

Change-Id: Ib85243d38f122c1993aba945b7ae943eed262dbf
2020-02-10 16:43:19 -06:00
Gage Hugo
86e56b2aee Address bandit gate failures
This change addresses the results that were found when running
bandit against the templated python files in the various charts.

This also makes the bandit gate only run when python template
files are changed as well as makes the job voting.

Change-Id: Ia158f5f9d6d791872568dafe8bce69575fece5aa
2020-02-04 15:33:17 -06:00
Tin Lam
c199addf3c Update apiVersion
This patch set updates and tests the apiVersion for rbac.authorization.k8s.io
from v1beta1 to v1 in preparation for its removal in k8s 1.20.

Change-Id: I4e68db1f75ff72eee55ecec93bd59c68c179c627
Signed-off-by: Tin Lam <tin@irrational.io>
2020-01-09 08:59:48 +00:00
diwakar thyagaraj
841feb7e82 Enable runtime apparmor for ceph-mon,ceph-mds & ceph-mgr.
Also changed ceph apparmor gate job.
Change-Id: I92b9a467b2a77d607dd431f031ec566cc18a86a4
2019-12-18 14:52:12 +00:00
Stephen Taylor
016b56e586 Ceph Nautilus compatibility
This change updates the Ceph charts to use Ceph Nautilus images
built on Ubuntu Bionic instead of Xenial. The mirror that hosts
Ceph packages only provides Nautilus packages for Bionic at
present, so this is necessary for Nautilus deployment.

There are also several configuration and scripting changes
included to provide compatibility with Ceph Nautilus. Most of
these simply allow existing logic to execute for Nautilus
deployments, but some logical changes are required to support
Nautilus as well.

NOTE: The cephfs test has been disabled because it was failing
the gate. This test has passed in multiple dev environments, and
since cephfs isn't used by any openstack-helm-infra components we
don't want this to block getting this change merged. The gate
issue will be investigated and addressed in a subsequent patch
set.

Change-Id: Id2d9d7b35d4dc66e93a0aacc9ea514e85ae13467
2019-12-17 18:47:24 +00:00
Steve Wilkerson
cbeb7f149b Move charts off using the :latest built tags
This updates charts that consume images built from osh-images to
use tags other than the :latest tags. This will be followed up
with the definition of jobs to allow for vetting out of updated
images, as reliance on :latest tags assumes any change merged into
osh-images will result in functionally correct behavior (which has
shown to not be the case traditionally)

Change-Id: I181aa56ed187604dc7583d8081e53cc69eb27310
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-21 19:57:07 +00:00
Steve Wilkerson
b50fae62a4 Update kubernetes-entrypoint image reference
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintained

Change-Id: I5bfdc156ae228ab16da57569ac6b05a9a125cb6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-10-18 18:20:11 +00:00
Phil Sphicas
b4004c58bb ceph-mon fix logging when no zombie mons found
Change-Id: Ie45320bce6945cc1e3ea7ac4d6f46a1e50abf621
2019-10-08 22:42:01 +00:00
Andrii Ostapenko
fdcc9b7e0e Make all prints python3 compatible
Change-Id: Ie5a08859010453d276b42253f5f2130f80b82224
2019-10-01 01:28:35 +00:00
Daniel Pawlik
0b58aea135 Fix mon_host hosts when hostname contains 'ip'
Ceph-mon template script parse mon_host in wrong way, when
hostname contains'ip' word, e.g.: airship.

Change-Id: I0a097443d42ad2e9b6be6c61facd7932ddb4b3bb
Story: 2006255
2019-07-19 10:49:50 +00:00
Venkata, Krishna (kv988c)
ed574f456f Switch from default values being populated
for upgrade strategy for ceph components

This PS uses HelmToolKit function to add
upgrade strategy parameters to ceph Components

Change-Id: I54e71d2a52bd639b3e93fc899c1bf2cd075b5396
2019-06-18 01:42:38 +00:00
RAHUL KHIYANI
d3dec3d257 Ceph-mon: Fix security context for pod/container
This changes the  user from root to the nobody user instead
in ceph-mon chart wherever needed

Change-Id: I5d3c1fbc8f983688807b73867773bfa2d83b91b3
2019-06-06 12:32:01 +00:00
Chinasubbareddy Mallavarapu
a40d93cef1 Ceph: Decrease ceph mon data store space to 15%
Ceph issues a HEALTH_WARN status in cluster log when the available
disk space of the monitor’s data store is lower than or equal to this percentage.

This is to decrease the value from default value 30% to 15%.

Change-Id: Id412969f1dfb08b07356747f13aa31f00b41130d
2019-05-17 14:39:37 -05:00
Zuul
b69584bd65 Merge "Expose Anti-Affinity Weight Setting" 2019-05-16 17:17:03 +00:00
Roy Tang (rt7380)
85bd731562 Expose Anti-Affinity Weight Setting
This ps exposes the anti-affinity weight value, including
default, that will be consumed by the updated htk function.

Change-Id: Id8eb303674764ef8b0664f62040723aaf77e0a54
2019-05-14 17:04:52 -05:00
Pete Birley
656f51d8d9 Ceph: Dont attempt to reuse monmap if it exists
This PS forces the monmap to be clobbered each time the container starts
which is required to recover from ome senarios when using an emptydir
to back /etc/ceph.

Change-Id: I2cf271593591ce07435893336cff98a8b1c72166
Signed-off-by: Pete Birley <pete@port.direct>
2019-05-12 16:21:39 +00:00
Zuul
8ea9be76ab Merge "ceph-mon: Fix security context" 2019-05-10 02:29:34 +00:00
Dmitrii Kabanov
b7c07a595c Update image links for Ceph.
Updated the links according to the decision here [1].
The convention for images is <imagename>:<tagging>
where <tagging> takes the format <version>-<distro>.

[1]: http://eavesdrop.openstack.org/irclogs/%23openstack-helm/%23openstack-helm.2019-03-07.log.html#t2019-03-07T17:50:58

Change-Id: I84f8ce731e428f8ef035c008ff615e712c78a1f4
2019-04-24 14:53:42 +00:00
RAHUL KHIYANI
a5e8953bd4 ceph-mon: Fix security context
This PS fixes the use of the security context macros for the
ceph-mon chart.

Change-Id: Ibde448481c44f2753ddfe57e590ea7d05671793a
2019-04-23 23:18:21 -05:00
Pete Birley
dece008337 Ceph: Make /etc/ceph and /run emptydirs uniformly across all pods
This PS updates the ceph charts to make /etc/ceph an emptydir
uniformly across all charts, both ensuring no default config is loaded,
and also permitting read-only filesystems to back the containers.

Additionally /run is uniformly applied across all long running pods
as a memory backed emptydir.

Change-Id: I00d1b15758b7eb4476fb950ddcb38db9a5149ad0
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-21 19:06:18 +00:00