65 Commits

Author SHA1 Message Date
Chinasubbareddy Mallavarapu
734b344bf6 [ceph-provisioners] Update ceph_mon config as per new ceph clients
As new ceph clients expecting the ceph_mon config as shown below , this
ps will update the configmap.

mon_host = [v1:172.29.1.139:6789/0,v2:172.29.1.139:3300/0],
[v1:172.29.1.140:6789/0,v2:172.29.1.140:3300/0],
[v1:172.29.1.145:6789/0,v2:172.29.1.145:3300/0]

Change-Id: I6b96bf5bd4fb29bf1e004fc2ce8514979da706ed
2021-03-29 15:02:08 +00:00
Kiran Kumar Surapathi (ks342f)
4b42f3f57f Fix Helm tests for the Ceph provisioners
We are adding the node selectors to helm tests for Ceph provisioners

Change-Id: I0fc9a78dcd27a92486dc724ce9294da96826eac9
2021-03-11 17:34:12 +00:00
Kabanov, Dmitrii
cda359ef1f [Ceph] Add Ceph CSI plugin
The PS adds Ceph CSI plugin (RBD only)

Change-Id: I3ddc69e49d12ff178263f38ac10aff90bb82b902
2021-03-09 16:54:45 +00:00
Chinasubbareddy Mallavarapu
da289c78cb [CEPH] Uplift from Nautilus to Octopus release
This is to uplift ceph charts from 14.X release to 15.X

Change-Id: I4f7913967185dd52d4301c218450cfad9d0e2b2b
2021-02-03 22:34:53 +00:00
Chinasubbareddy Mallavarapu
7c8ca55ac0 [ceph-provisioners] Validate each storageclass created
This is to include every storageclass getting created part of
helm tests.

Change-Id: I62dc11600d00fe2ec7babb1688e61d3eaa50100c
2020-10-28 22:14:49 +00:00
Andrii Ostapenko
1532958c80
Change helm-toolkit dependency version to ">= 0.1.0"
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0

Change-Id: I15950b735b4f8566bc0018fe4f4ea9ba729235fc
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-24 12:19:28 -05:00
Mohammed Naser
c7a45f166f Run chart-testing on all charts
Added chart lint in zuul CI to enhance the stability for charts.
Fixed some lint errors in the current charts.

Change-Id: I9df4024c7ccf8b3510e665fc07ba0f38871fcbdb
2020-09-11 18:02:38 +03:00
dt241s@att.com
4c46b2662a Add Application Armor to Ceph-Provisioners-config test
1) Added  to service account name insted of traditional pod name
   to resolve for dynamic release names.

Change-Id: Ibf4c69415e69a7baca2e3b96bcb23851e68d07d8
2020-08-03 16:42:53 +00:00
diwakar thyagaraj
936397b36a Add Application Armor to Ceph-Provisioners-key-generator
1) Added  to service account name insted of traditional pod name.

Change-Id: I1c7ba9081ccf396b037861b496110251f2248fd2
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-07-23 14:15:04 +00:00
diwakar thyagaraj
5d50433362 Enable Application Armor to all ceph key-generator pods.
1) Changed the pod name and container name to pick name dynamically for
   osd,mon,mgr and mds.

2) Added Init container for ceph-provisioners.

Change-Id: I3e27d51c055010cff982ddb0951d01ea8adac234
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-07-20 01:29:42 +00:00
Andrii Ostapenko
824f168efc Undo octal-values restriction together with corresponding code
Unrestrict octal values rule since benefits of file modes readability
exceed possible issues with yaml 1.2 adoption in future k8s versions.
These issues will be addressed when/if they occur.

Also ensure osh-infra is a required project for lint job, that matters
when running job against another project.

Change-Id: Ic5e327cf40c4b09c90738baff56419a6cef132da
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-07 15:42:53 +00:00
chinasubbareddy mallavarapu
91f60d2884 Revert "[ceph-client] Update ceph-mon port."
Reverting this ps since we tried to solve the  problem here for
the old clients prior to nautilus but nautilus clients thinks
its v2 port and try to communicate with server and getting some
warnings as shown below:

lets make v2 port as default and ovverride  mon_host config for
old clients prior  to nautilus as we did in this ps
(https://review.opendev.org/#/c/711648/).

better solution will be moving out of old ceph clients by changing
the images wherever old ceph clients are installed.

log:

+ ceph auth get-or-create client.cinder mon 'profile rbd' osd
'profile rbd' -o /tmp/tmp.k9PBzKOyCq.keyring
2020-06-19 15:56:13.100 7febee088700 -1 --2-
172.29.0.139:0/2835096817 >> v2:172.29.0.141:6790/0 conn(0x7febe816b4d0
0x7febe816b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0
rx=0 tx=0)._handle_peer_banner peer v2:172.29.0.141:6790/0 is using msgr V1 protocol

This reverts commit acde91c87d5e233d1180544df919cb6603e306a9.

Change-Id: I08ef968b3e80c80b973ae4ec1f80ba1618f0e0a5
2020-06-19 22:16:16 +00:00
Andrii Ostapenko
83e27e600c Enable key-duplicates and octal-values yamllint checks
With corresponding code changes.

Change-Id: I11cde8971b3effbb6eb2b69a7d31ecf12140434e
2020-06-17 13:14:30 -05:00
Andrii Ostapenko
dfb32ccf60 Enable yamllint rules for templates
- braces
- brackets
- colons
- commas
- comments
- comments-indentation
- document-start
- hyphens
- indentation

With corresponding code changes.

Also idempotency fix for lint script.

Change-Id: Ibe5281cbb4ad7970e92f3d1f921abb1efc89dc3b
2020-06-17 13:13:53 -05:00
Andrii Ostapenko
8f24a74bc7 Introduces templates linting
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.

Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.

Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
2020-06-11 23:29:42 -05:00
Kabanov, Dmitrii
acde91c87d [ceph-client] Update ceph-mon port.
The PS updates ceph-mon port for clients.

Change-Id: I1a41f0ad042c916e63bd4505ddea7a84b162b188
2020-06-03 15:53:40 +00:00
Andrii Ostapenko
731a6b4cfa Enable yamllint checks
- document-end
- document-start
- empty-lines
- hyphens
- indentation
- key-duplicates
- new-line-at-end-of-file
- new-lines
- octal-values

with corresponding code adjustment.

Change-Id: I92d6aa20df82aa0fe198f8ccd535cfcaf613f43a
2020-05-29 19:49:05 +00:00
Kabanov, Dmitrii
46930fcd06 [Ceph] Upgrade Ceph from 14.2.8 to 14.2.9 version
The PS upgrades Ceph to 14.2.9 version.

Change-Id: I72a2e39a7b4294ac8fd42b1dbc78579c2c0ae791
2020-05-28 15:46:47 +00:00
Kabanov, Dmitrii
2aa6b3cf1c [Ceph] Add kubernetes tolerations for ceph deployments
The PS adds kubernetes tolerations for deployments from ceph-client,
ceph-mon, ceph-provisioners and ceph-rgw charts.

Change-Id: If96f5f2058fca6e145e537e95af39089f441ccbb
2020-05-20 19:32:06 +00:00
diwakar thyagaraj
163c5aa780 Enable Apparmor to all osh-infra test pods
Also Changed container names to static.

Change-Id: I51f53b480d18aaa38a9707429f01052ee122e7e9
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-19 15:36:07 +00:00
Gage Hugo
d14d826b26 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
2020-05-07 02:11:15 +00:00
Taylor, Stephen (st053q)
9057c770a6 Enable cephfs tests
Cephfs tests were disabled in order to merge
https://review.opendev.org/695568 due to gate failures that were
blocking it. CephFS isn't used in openstack-helm-infra, so it
wasn't required for that work. This change re-enables the cephfs
tests so we can work through any issues that are causing further
failures.

Since the the issue got fixed in 14.2.8 , upgrading all daemons to 14.2.8.
(https://tracker.ceph.com/issues/43770)

Change-Id: I376d39b7ee00ccb1ab8046b58f92b19a822272e1
2020-04-21 22:25:52 +00:00
Chinasubbareddy Mallavarapu
af9ac277e8 [ceph-provisioner] update msgr2 port for etc configmap
This is to update msgr2 port for clients who uses ceph etc file
from configmap

Change-Id: If6079e860afc67c5e466faaa8bea040f67b1f842
2020-03-30 17:42:00 -05:00
Chinasubbareddy Mallavarapu
977a5a2f97 [ceph-daemons] Redirect all the logs to stdout
This is to redirect all the logs from daemons to stdout to avoid
accumulating large sized log files on filesystem.

NOTE: The ceph-osd daemon won't work this way and is addressed
separately in https://review.opendev.org/715295. All other Ceph
daemons are included here.

Change-Id: I3045d6e941791aba14979472fac1bca09776d3bf
2020-03-29 10:27:44 -06:00
Kabanov, Dmitrii
13f54b0e03 [Ceph] Add msgr1 port for ceph-provisioners
Change-Id: Ifa9b44074d927006f47dfcc449361cf3f6aa9413
2020-03-27 08:28:58 -07:00
Chinasubbareddy Mallavarapu
7425e3e5c0 [CEPH] update all ceph daemons startup scripts to support msgr2
This is to update all ceph daemons startup scripts as per msgr2 protocol and
also to update v2 port for mon_host config.
This also removes setting mon_addr config since we already have mon_host config.

v1 default port: 6789
V2 default port: 3300

Change-Id: I3d95edbd89f5ac8b40a34f41c1099311cee4f875
2020-03-04 23:22:02 -06:00
diwakar thyagaraj
344c0543fa Enable runtime Apparmor default for All Ceph Components
Change-Id: Id62fe453846ffe6ab01198177d5d8046378d61bf
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-02-24 15:52:11 +00:00
Chinasubbareddy Mallavarapu
281b2018c2 CEPH: upgrade ceph version from 14.2.5 to 14.2.7
This is to upgrade ceph version from 14.2.5 from 14.2.7  and also
to update ceph provisioners to use latest code from quay.io

  - rbd-provisioner: quay.io/external_storage/rbd-provisioner:v2.1.1-k8s1.11
  - cephfs-provisioner: quay.io/external_storage/cephfs-provisioner:v2.1.0-k8s1.11

This also updates verbs for proivioner's clusterrole to support new code.

Change-Id: Ia94129574610bb5c800a6941804e58ca3aefce65
2020-02-18 21:29:23 +00:00
Tin Lam
c199addf3c Update apiVersion
This patch set updates and tests the apiVersion for rbac.authorization.k8s.io
from v1beta1 to v1 in preparation for its removal in k8s 1.20.

Change-Id: I4e68db1f75ff72eee55ecec93bd59c68c179c627
Signed-off-by: Tin Lam <tin@irrational.io>
2020-01-09 08:59:48 +00:00
Stephen Taylor
016b56e586 Ceph Nautilus compatibility
This change updates the Ceph charts to use Ceph Nautilus images
built on Ubuntu Bionic instead of Xenial. The mirror that hosts
Ceph packages only provides Nautilus packages for Bionic at
present, so this is necessary for Nautilus deployment.

There are also several configuration and scripting changes
included to provide compatibility with Ceph Nautilus. Most of
these simply allow existing logic to execute for Nautilus
deployments, but some logical changes are required to support
Nautilus as well.

NOTE: The cephfs test has been disabled because it was failing
the gate. This test has passed in multiple dev environments, and
since cephfs isn't used by any openstack-helm-infra components we
don't want this to block getting this change merged. The gate
issue will be investigated and addressed in a subsequent patch
set.

Change-Id: Id2d9d7b35d4dc66e93a0aacc9ea514e85ae13467
2019-12-17 18:47:24 +00:00
Steve Wilkerson
cbeb7f149b Move charts off using the :latest built tags
This updates charts that consume images built from osh-images to
use tags other than the :latest tags. This will be followed up
with the definition of jobs to allow for vetting out of updated
images, as reliance on :latest tags assumes any change merged into
osh-images will result in functionally correct behavior (which has
shown to not be the case traditionally)

Change-Id: I181aa56ed187604dc7583d8081e53cc69eb27310
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-21 19:57:07 +00:00
Steve Wilkerson
b50fae62a4 Update kubernetes-entrypoint image reference
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintained

Change-Id: I5bfdc156ae228ab16da57569ac6b05a9a125cb6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-10-18 18:20:11 +00:00
Kabanov, Dmitrii
22ef25ab29 [Ceph] Update helm tests.
The PS updates helm tests for Ceph-RGW and Ceph-provisioners:
- Checking several randomly generated objects instead of one static object.
- Improved the output of the tests.

Change-Id: I0733d7c47a2a8bdf30b0d6a97c1a0331eb5030c8
2019-10-02 12:51:40 -07:00
Steve Wilkerson
ae3c07b853 Ceph: Update default test pod timeout for provisioners
This mvoes the default timeout for the ceph provisioners helm test
pod to 600 seconds, as 120 seconds is fairly aggressive.  This
also adds the required --timeout flag to the helm test command in
each job for the ceph provisioners, as well as adding the required
helm test configuration to the armada-lma manifest

Change-Id: I5a3b98de9132fe83cf09b1e5b3fcc513bd496650
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-07-12 13:43:38 +00:00
Chinasubbareddy Mallavarapu
ddbe20a30b Ceph-Provisioner: wait for helm tests related pods
This is to update logic to wait for  pods created by helm tests.

Change-Id: Ic1ee293613a0b050be9fa01a29642bfc9e8de2d7
2019-07-05 10:18:38 -05:00
Dejaeger, Darren (dd118r)
083956e37b Add node selector to Ceph test pods
This PS looks to add a node selector into the Ceph test pod's
specs.

Change-Id: If73a5036c5e6a651393f81a136874b9e8a52b4f1
2019-06-24 17:41:09 +00:00
Kabanov, Dmitrii (dk370c)
e580e0204b [Ceph] Update helm tests for ceph-provisioners
The changes allow to provide more information in case of failure.

Change-Id: Ie03cafac33ef9b2cf457e0d483d838170eadaef4
2019-06-20 11:19:51 -07:00
Venkata, Krishna (kv988c)
ed574f456f Switch from default values being populated
for upgrade strategy for ceph components

This PS uses HelmToolKit function to add
upgrade strategy parameters to ceph Components

Change-Id: I54e71d2a52bd639b3e93fc899c1bf2cd075b5396
2019-06-18 01:42:38 +00:00
Renis Makadia
c7f5c9979c Add helm tests for Ceph Provisioners chart
- Adding helm tests for Ceph provisioner chart
- Helm test should only executed when deploying chart with
client_secrets: true.

Co-Authored-By: Chinasubbareddy Mallavarapu <cr3938@att.com>

Change-Id: I33421249246dfaf6ea4f835e76a74813dfb3b595
2019-06-12 12:32:30 -05:00
Chinasubbareddy Mallavarapu
cb767a5694 CephFs-provisioner: fix securitycontext in the pod
This is to update correct user id in security context for
cephfs provisoner pod as there is  no user with 99 exist
in the container.

Change-Id: I1bbe46df555b35b8afe636327fa83015fd784db0
2019-05-23 22:03:41 +00:00
Chinasubbareddy Mallavarapu
53bdbf00ed CephFs-provisioner: fix cephfs storageclass
This is to remove invalid key "userSecretName" for
cephfs storageclass as we are having toruble to provision
a pvc with cephfs storageclass with "userSecretName" key .

Failed to provision volume with StorageClass
"cephfs": invalid option "userSecretName"

Change-Id: Ide52987c9f8ef8fc2327bf30747395e70dc05f99
2019-05-23 11:41:44 -05:00
Roy Tang (rt7380)
85bd731562 Expose Anti-Affinity Weight Setting
This ps exposes the anti-affinity weight value, including
default, that will be consumed by the updated htk function.

Change-Id: Id8eb303674764ef8b0664f62040723aaf77e0a54
2019-05-14 17:04:52 -05:00
Kabanov, Dmitrii (dk370c)
f95cd4ac69 [Ceph] Update image links for CephFS and RBD provisioners.
This PS allows to use Ubuntu based images for CephFS and
RBD provisioners.

Change-Id: Ie7afe1344891e54639dc0d9a076be97d8047ee07
2019-05-08 02:47:07 +00:00
Zuul
212d9f96a7 Merge "Ceph-provisioners: Fix security context" 2019-04-25 15:35:08 +00:00
Dmitrii Kabanov
b7c07a595c Update image links for Ceph.
Updated the links according to the decision here [1].
The convention for images is <imagename>:<tagging>
where <tagging> takes the format <version>-<distro>.

[1]: http://eavesdrop.openstack.org/irclogs/%23openstack-helm/%23openstack-helm.2019-03-07.log.html#t2019-03-07T17:50:58

Change-Id: I84f8ce731e428f8ef035c008ff615e712c78a1f4
2019-04-24 14:53:42 +00:00
RAHUL KHIYANI
b1900bbfc2 Ceph-provisioners: Fix security context
This PS fixes the use of the security context macros for the
ceph-provisioners chart

Change-Id: Iddeb643139f2e7798282e67e319f38d3a22cd10d
2019-04-24 04:57:23 +00:00
Pete Birley
dece008337 Ceph: Make /etc/ceph and /run emptydirs uniformly across all pods
This PS updates the ceph charts to make /etc/ceph an emptydir
uniformly across all charts, both ensuring no default config is loaded,
and also permitting read-only filesystems to back the containers.

Additionally /run is uniformly applied across all long running pods
as a memory backed emptydir.

Change-Id: I00d1b15758b7eb4476fb950ddcb38db9a5149ad0
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-21 19:06:18 +00:00
Pete Birley
2abf62ff4d OSH-Infra: Add emptydirs for tmp
This PS adds emptydirs backing the /tmp directory in pods, which
is required in most cases for full operation when using a read only
filesystem backing the container.

Additionally some yaml indent issues are resolved.

Change-Id: I8b7f1614da059783254aa6efc09facf23fca3cad
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-20 20:50:59 +00:00
Rahul Khiyani
c8b6b4b073 ceph-provisioners: Add pod/container security context
This updates the ceph-provisioners  chart to include the pod
security context on the pod template

This also adds the container security context to set allowPrivilegeEscalation
to false and readOnlyRootFilesystem to true

Change-Id: Iee49ffe17f2cd08fc978461269b654d3b2cb4406
2019-04-16 17:57:46 +00:00
Steve Wilkerson
7f04846c52 Ceph charts: Add missing templates, update dependency values
This addresses slight issues with the ceph-osd, ceph-provisioners,
and ceph-rgw charts. Those issues include:
- Remove duplicate test: key in ceph-osd dependencies
- Add missing image repo sync job to ceph-provisioner and rgw
- Use correct job name for image repo sync dependencies in charts
- Remove incorrect keystone service dependency for ceph-rgw, as
  the keystone jobs are dependent on the keystone service

This also updates the ceph-rgw chart to use dynamic dependencies
based on whether keystone auth or s3 auth is used

Change-Id: Id3b3f289bdd4ca4d1b2e9b6267b12427e422a08d
2019-04-07 06:42:29 +00:00