78 Commits

Author SHA1 Message Date
Zuul
91df918e87 Merge "Update build-chart playbook" 2020-12-07 22:27:13 +00:00
Zuul
e4683420d7 Merge "Revert "Don't use opendev docker proxy"" 2020-12-07 14:51:47 +00:00
Gage Hugo
4047ff0fd4 Update build-chart playbook
The build-chart playbook task to point each chart to helm-toolkit
has a find command that when used with another repo, will
include the charts for osh-infra as well.

This change modifies the playbook to only modify requirements
in charts in the repo being published.

Change-Id: I493b4c64fe2525bac0acae06bd40c3896c918e20
2020-12-04 17:31:18 -06:00
Gage Hugo
7fdf282271 Revert "Don't use opendev docker proxy"
This reverts commit 42f3b3eaf5a8794b1f247915fffbef68137e6c1c.

Reason for revert: dockerhub now sets a hard limit on daily pulls, lets switch back to using the opendev docker proxy.

Change-Id: I87e399c89d5736f39d7bdba2011655e5f5766180
2020-12-03 19:42:47 +00:00
Mohammed Naser
ca60e1d875 Make publish jobs more generic
This will help in allowing the openstack-helm repo cleanly
publish to the seperate folder.

Change-Id: I2651c2f81191802a8f30314c4eebffdf0c2a53af
2020-11-11 19:22:55 -05:00
Andrii Ostapenko
42f3b3eaf5
Don't use opendev docker proxy
Look like using docker proxy is slower and less stable than pulling from
dockerhub directly and contributes to some part of unstable builds.

This reverts commit e3f14aaff35364b84acedf53b3778111cbae0373.

Change-Id: I9735ad35ce9240f610479a56eaa38715defa2e04
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-27 10:33:40 -05:00
Tin Lam
62b10c7d49 chore(pkg): updates the chart packaging
Part 2. This patch set adjusts the url once the initial packages are
make available.

Change-Id: Idfb69146d606b43c98c552d1d2c5680ccd503282
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-21 00:58:16 -05:00
Tin Lam
738c89b342 fix(job): fixes the post job
This corrects the ability to sync artifacts to tarballs.o.o.

Change-Id: Icb2b6653f263aaab173d1479d05c0209e7390c50
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-20 22:43:10 -05:00
Tin Lam
da81705a47 fix(post): fixes publish job
This fixes a typo of the publish job.

Change-Id: I077feb29a8764a0b3031b34b462779c911baaee3
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-19 11:53:46 -05:00
Tin Lam
e5c776e5c4 chore(pkg): updates the chart packaging
This patch set updates the ability to package (and subsequent publish)
of the charts in the OpenStack-Helm-Infra repository.

Change-Id: I6175325b0e7a668c22a7ec3ab08cae51ad4f9ab8
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-17 08:42:53 +00:00
Tin Lam
3a2d0f83b4 chore(charts): addresses issues with chart publish
This changes attempts to address the chart publish issue. Also makes
the job periodic.

Change-Id: I806da82a7eb07ce8e83ae8c023a014fa3b917193
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-16 15:15:35 +00:00
Andrii Ostapenko
7a0558bd78 Override images provided as artifacts from buildset registry
Switch from using images from defined in docker_images to provided as
zuul artifacts. Currently to be used in conjunction with [0] in
openstack-helm-images pipelines.

[0] https://review.opendev.org/741551

Change-Id: I43dbd38906e8854c87a361f2e5e479f57850252f
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-28 17:38:29 +00:00
Mohammed Naser
52093576aa publish: fix missing task for creating new index
The task was missed which was causing the post pipeline to fail, this
patch should fix it by adding the missing task.

Change-Id: I13955b1c9ac3899325f7397da6bf5379b3991241
2020-09-18 16:16:50 -04:00
Zuul
87535287f3 Merge "lint: enable running on changed charts only" 2020-09-18 19:21:17 +00:00
Zuul
487c50a40a Merge "Publish Helm charts into tarballs.openstack.org" 2020-09-18 17:08:07 +00:00
Mohammed Naser
d4e2228f03 lint: enable running on changed charts only
At the moment, we are using --all which means run the linting on all of
the charts.  However, the problem with using --all is that it disables
version checking which means we can't enforce version changes on Helm
charts.

This patch drops it which means the chart-testing logic will go over the
changed files and make sure that it lints those charts which have
undergone changes.

Because we use a mdoel of 1 commit per merge within Gerrit, this should
still give us the exact coverage that we need without potentially
missing any linting changes.

Change-Id: I64c7896b25c1f3daaa4f61723de8a6c722aaf3a6
2020-09-17 13:50:26 -04:00
Mohammed Naser
887327aff8 Publish Helm charts into tarballs.openstack.org
This will start publishing all of the charts into tarballs.openstack.org
which should allow for easier public consumption of these charts.  This
patch covers adding publishing for openstack-helm-infra first.

Change-Id: Iaa14629c0d0c36b98c2295119af3008f14c0cd39
2020-09-11 19:13:22 +00:00
Mohammed Naser
c7a45f166f Run chart-testing on all charts
Added chart lint in zuul CI to enhance the stability for charts.
Fixed some lint errors in the current charts.

Change-Id: I9df4024c7ccf8b3510e665fc07ba0f38871fcbdb
2020-09-11 18:02:38 +03:00
Andrii Ostapenko
fdcbd03784 Allow parallelization in gate runner
Change-Id: I393a22cfcaecb00d14d8ac643bd4b7ffbba03b12
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-08-06 00:51:11 +00:00
Andrii Ostapenko
e3f14aaff3
Use zuul docker mirror for functional jobs
Change-Id: I4899a69ef2c961094d808821b8a14d58ed02d621
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-22 09:06:27 -05:00
Andrii Ostapenko
e8caaac6b4
Add ensure-pip role to functional jobs
Change-Id: Idcead2d24fd30d950130a6f1f121beac039e656f
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-16 10:52:40 -05:00
Andrii Ostapenko
0ff44fc155
Support images overrides
Adds ability to override images in osh and osh-i repositories
if docker_images dict is provided with tags following zuul-jobs
upload-docker-image naming convention.

This allows to inherit osh job from osh-images project providing
required vars.

Change-Id: I5e7acb0dd0edd6838b6f7ddb5111db0fa34912d4
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-12 11:41:49 -05:00
Andrii Ostapenko
2b4cf6a2d9 Completely switch to python3 for developers installation
This addresses an issue with using py2 as interpreter while
installing required dependencies with py3.

Also switch kubeadm-aio image to bionic.

Change-Id: I5a9e6678c45fad8288aa6971f57988b46001c665
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-03 09:47:43 +00:00
Gage Hugo
16676b5b63 Remove duplicate lint job entry and script
osh-infra currently has a duplicate linter playbook that is not
being used, since the other is used for both osh and osh-infra.
This change removes the duplicate entry and playbook.

Change-Id: If7040243a45f2166973dc5f0c8cd793431916942
2020-06-22 12:31:25 -05:00
Gage Hugo
26350f37aa Add new python roles to playbooks
With the latest infra update, the images used no longer contain
python by default and projects are expected to use the new
ensure roles to use packages as needed.

This change adds some of the ensure roles to a few playbooks,
additional cleanup can be done using these in future changes.

Change-Id: Ie14ab297e71195d4fee070af253edf4d25ee5d27
2020-06-19 18:07:13 +00:00
Ahmad Mahmoudi
b7d3311649 (fix) Changed pip to pip3
Changed pip to pip3 to address zuul gate issues.

Change-Id: Id265b405b293af5b51b4774ca4c7465c9e9457b6
2020-06-17 18:15:07 +00:00
Andrii Ostapenko
8f24a74bc7 Introduces templates linting
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.

Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.

Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
2020-06-11 23:29:42 -05:00
Andrii Ostapenko
6779ff041c Add yamllint check to lint job
Initial commit with bootstrapping non-voting configuration
for yamllint. Yamllint checks will be switched from 'warning'
to 'enabled' in subsequent commits together with code adjustments.

Change-Id: Ie372cb9fefb310bd044b4b03064e183f0c8c003b
2020-05-20 12:42:28 -05:00
Gage Hugo
d14d826b26 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
2020-05-07 02:11:15 +00:00
Gage Hugo
30c66126ee Add bandit check to osh-infra
This change adds a non-voting bandit check to openstack-helm-infra
similar to what is ran in the openstack-helm repo.

This check will be made voting in a future change once the current
failures are addressed.

Similarly this check will be modified in a future change to
only be ran when affected python files are changed.

Change-Id: I177940f7b050fbe8882d298628c458bbd935ee89
2019-12-31 18:29:01 +00:00
Kabanov, Dmitrii (dk370c)
1db1ddf0ba [Kibana] Add improvements to Selenium tests for Kibana
This PS adds several fixes to Selenium tests (for Kibana) and adds
role which allows to collect the results.

Change-Id: If9fb5f50e395379fdd3ccc46e945a93606dcbabe
2019-06-25 20:51:01 -07:00
Steve Wilkerson
2c11798c65 Update airship-divingbell check job
This updates the airship-divingbell repository defined in the
single node check job to use the correct repository namespace.

This also updates the divingbell check job to use the standard
osh-infra-gate-runner playbook as well as the minikube based
kubernetes deployment

Change-Id: Iff53279b3e09058deb323d092955cbf87230b5e5
2019-04-23 10:09:27 -05:00
OpenDev Sysadmins
29b124e3dc OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:34:11 +00:00
Roman Gorshunov
31e3469d28 Update Helm to version 2.13.0
This updates Helm from version v2.12.3 to v2.13.0.
The `merge` function has changed behavior, and is now called
`mergeOverwrite`.

Change-Id: Ie3364256c8abb714b748b3bf3658bd6cd1e3ce35
2019-03-11 05:53:08 +00:00
Steve Wilkerson
cb021c2124 Add Armada job for testing update of chart passwords
This adds both a periodic and experimental job for deploying Ceph
and the LMA components via Armada. This job will then generate new
passphrases for the LMA components, render an updated manifest for
the LMA components including the new passphrases, then applies the
updated LMA manifest to validate the ability for all deployed
charts to update those passphrases successfully

Change-Id: I966ebeadd3823a087239aa7d198444a084e5d242
2019-02-04 22:44:52 +00:00
Steve Wilkerson
8361b74926 Add Armada job for testing update of release uuids
This adds both a periodic and experimental job for deploying Ceph
and the LMA components via Armada. This job will then generate a
new release uuid, render an updated manifest for all previously
deployed releases, then apply that manifest to validate the
ability for all deployed charts to update successfully with the
new release uuid annotation

Change-Id: Ic1eed1bd949279f4630fb3964fbb03788536213c
2019-02-04 21:14:02 +00:00
Steve Wilkerson
9422e970a7 Add Armada deployment job to openstack-helm-infra
This adds both a periodic and experimental job for deplying Ceph
and the LMA components via Armada to openstack-helm-infra

Change-Id: Ia3b557801d4f4b667d82eb47a6ef1825394ee526
2019-02-04 14:07:07 +00:00
Pete Birley
a8b3787a1b Gate: simplify playbooks
This PS simplifies the gate playbooks

Change-Id: I4dd7c892090f8eec10edf083b68dc3e3cc99ece9
Signed-off-by: Pete Birley <pete@port.direct>
2019-01-14 07:55:32 +00:00
Meg Heisler
c3bef9e88f Selenium Tests for OSH Infra
This adds scripts using Selenium Webdriver to verify
the dashboards for Gafana, Nagios, and Prometheus are
reachable and functioning as expected. The scripts
create screenshots of each dashboard as well as
pages that can be navigated to.

It also adds the scripts to the gates for the single
and multinode deployments.

Change-Id: I1699e0ba8ff82ce8f59342cc71aad10cff7d2516
2019-01-07 15:59:42 -06:00
Aaron Sheffield
8b201ea0eb Adding AppArmor profile to Calico v3
- Adds AppArmor profile to the privileged pod
  using kubernetes_manadatory_access_control_annotation.
- Added apparmor install to the gate jobs.

Change-Id: I8b53e0b8ddc2695fa278481edf5688efa23ab06b
2018-12-10 16:23:35 +00:00
Anderson, Craig (ca846m)
ab4c330b65 Add divingbell to zuul for test of htk overrides
Change-Id: I1e5cec885b4ae20f34733e6aaf1ee0a8a170ac32
2018-12-05 08:36:50 +00:00
Pete Birley
c7f7091739 Gate: refactor gate script running
This is the 1st in a series of commits to reduce the complexity
of the gate playbooks, follow on work will target gates more
specificly to both reduce infra load and improve coverage.

This PS also removes the unused ldap-deploy playbook.

Change-Id: Ie4ddabe86d71008611c6cc5015a927e32c54fa35
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-08 04:51:59 +00:00
Gupta, Sangeet (sg774j)
a34a7d8e50 Adding Falco
This commit adds falco daemonset of the node for behavioral activity
monitor designed to detect anomalous activity.

Change-Id: I783a2acc03592471c81a8a54e1dc0df140b34a42
2018-10-05 16:18:51 +00:00
Pete Birley
46935734af Gate: remove unused pull images role and playbook
This PS removes the unused pull images role and playbook.

Change-Id: Ic26035c3f58efb6269fd58e570601cccfdd84949
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-05 01:43:40 +00:00
Pete Birley
feeeed4d5d Gate: Remove unused helm chart deployment role and playbook
This PS removes the unused helm chart deployment role and playbook.

Change-Id: I01c58a628589ec35af2557c8cc93ea47fe084089
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-05 01:43:33 +00:00
Steve Wilkerson
a084769410 Elasticsearch S3 repo
This ps adds the ability to use the ceph radosgw s3 api for
snapshot repositories. It removes the ability to use a RWM pvc, as
the radosgw solution provides a more robust approach for storing
index snapshots

Change-Id: Ie56ac41ccdc61bfadcac52b400cceb35403e9fae
2018-09-19 15:53:21 -05:00
Steve Wilkerson
bc6e22f392 Gate: Trim dev-deploy gates
This attempts to trim down the dev-deploy gates until further
gate refactoring is complete. This disables the elasticsearch and
fluentd exporters and removes the openstack exporter from the
single node deployment gates to ease the load on nodepool vms

Change-Id: If211511e8f52fe39d293966abbd7e62b45b65970
2018-09-17 13:56:51 +00:00
Steve Wilkerson
6b944f557b Libvirt: Move chart to openstack-helm-infra
This moves the libvirt chart to openstack-helm-infra as part of
the effort to move charts to their appropriate repositories

Change-Id: I02ce197f8d100da74c086d84e2f9d2b902a69e97
Story: 2002204
Task: 21723
2018-09-10 09:45:55 -06:00
Steve Wilkerson
3dcbfae101 Openvswitch: Move chart to openstack-helm-infra
This moves the openvswitch chart to openstack-helm-infra as part of
the effort to move charts to their appropriate repositories

Change-Id: I6e00231b8de54c01bc9bb31e0433753a9f281542
Story: 2002204
Task: 21730
2018-09-07 12:35:40 +00:00
Zuul
40e0ddba33 Merge "Gate: Add process exporter to gate deployments and docs" 2018-09-05 20:52:24 +00:00