77 Commits

Author SHA1 Message Date
Radosław Piliszek
43f68d948e CI: Move to centos-8-stream
Co-Authored-By: Michal Nasiadka <mnasiadka@gmail.com>

Change-Id: I413f484d60d7cdfa03f3f8d779f07462be15d2dd
2021-04-06 16:13:09 +00:00
Zuul
cef8c8b3de Merge "CI: octavia: create and test a load balancer" 2021-03-05 11:00:56 +00:00
wuchunyang
366ba8526e CI: octavia: create and test a load balancer
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Change-Id: I20c1d9bddee5a046b521b9378bade2ab50612be0
2021-03-04 09:27:38 +08:00
Michał Nasiadka
65a16a08e2 CI: Move from ceph-ansible to cephadm
Change-Id: I81a4f8f8b8faa7559740531bb16d8aec7fc23f9b
2021-03-02 17:49:12 +01:00
Doug Szumski
47fee115eb CI: Add monasca scenario
Adds the following new Zuul job for testing deployment of Monasca and
associated services:

* kolla-ansible-centos8-source-monasca

All core OpenStack services except for Keystone are disabled to ensure
enough memory is available.

A follow up patch will replace the basic tests here with Tempest.

Co-Authored-By: Doug Szumski <doug@stackhpc.com>

Change-Id: I5d33fd3d7b69798ba0aa23509f7b809065f61c19
2021-01-19 09:25:24 +00:00
Radosław Piliszek
604d85b6d2 Drop lower-constraints
Per our PTG resolution [1] and general OpenStack resolution [2], lower-constraints
are not worth the extra work and confusion they introduce.
This patch drops them along with all mentions.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-October/018445.html
[2] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019521.html

Change-Id: I4c65a428facdf6d4ce28d97a868589aeae4c856e
2020-12-18 20:05:27 +01:00
Radosław Piliszek
90d0900e82 [CI] Drop periodics
We agreed that, due to DockerHub pull limits, it is more
appropriate to spend pull credits on real reviews than these
extra periodic runs.
This also makes us more lightweight on CI in general.

Change-Id: I5c2ee0042e9e07108a8177983fbf5fc6bb8163f0
2020-12-10 21:21:07 +01:00
Radosław Piliszek
f2b2b9cf5e Remove a CentOS IPv6 FIXME meant for Ussuri
CentOS 8 should work fine without the workaround.

This change adds the missing CentOS 8 IPv6 CI job as well.

Change-Id: I58af7a09b5ae09a10b9efc33c1f30c2efc6613f7
2020-11-08 09:37:35 +01:00
35488ecc4b Add Python3 wallaby unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for wallaby.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I9811dd8fb7fcb094277d4d8757104c98f3291828
2020-11-05 10:12:05 +00:00
Mark Goddard
d2326712d4 CI: add magnum scenario, also covering octavia
Adds a new Zuul job, kolla-ansible-centos8-source-magnum, for testing
deployment of Magnum, Octavia and associated services.

Change-Id: I61b293ba6bb52064ea98a73e2dff0023fa01a2a2
2020-09-17 15:01:53 +00:00
Michal Nasiadka
1f50f1be79 CI: Add centos linuxbridge job
Change-Id: I8aa02ac2caf320934be5ca439b9b9a24ac6aa573
2020-07-30 10:57:44 +02:00
Zuul
8fc8dec3f7 Merge "CI: Add CentOS KVM job" 2020-07-22 11:54:28 +00:00
Zuul
f039de861d Merge "CI: Add Ubuntu KVM job" 2020-07-22 11:54:26 +00:00
Radosław Piliszek
f2a5d374ca CI: Add CentOS KVM job
Follow-up on [1] which added Ubuntu.

[1] https://review.opendev.org/735454

Change-Id: If0baef37db392324c285798fa55d5e85892edb9f
2020-07-17 17:51:08 +00:00
Mark Goddard
f44876c406 CI: add prometheus-efk scenario
Tests prometheus, grafana, and centralised logging.

The tests could be improved in future by querying logs in elasticsearch,
and metrics in prometheus.

Change-Id: Iabad035d583d291169f23be3d71931cb260e87ae
2020-07-10 07:55:14 +00:00
Radosław Piliszek
88e7f8006e CI: Add Ubuntu KVM job
This also uses the recommended machinery to set qemu instead
of relying on config file override so that we test what we
really want to test.

Change-Id: I560e4f9d0a69c347e6aaf3b970331157c1a56f18
2020-06-14 12:03:32 +02:00
824e3ea46d Add Python3 victoria unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for victoria.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: If89122615a5ba9ff1d053bf70e738f5e7c5dfa2c
2020-05-22 18:31:03 +00:00
Michal Nasiadka
d8f31e0a5e CI: Add ansible-lint to tox
* Reworked tox pep8 into linters job, that runs:
  - pep8
  - bandit
  - bashate
  - doc8
  - yamllint
  - ansible-lint (validate-all-files.py + ansible-lint)

* Skip E701 - missing galaxy_info in meta and E602 see [1].
* Skip E301 and E503 - followup later in a separate change
* Added ansible-role-jobs to zuul.d/project.yaml which will run
  openstack-tox-linters job in check queue
* Fixed remaining style issue
* Made tox and docs reference the new env for linters
* Dropped pype environment (not supported)

[1]: https://github.com/ansible/ansible-lint/issues/457

Change-Id: I494b4b151804aac8173120e6c6e42bc2fdb00234
2020-05-17 17:02:38 +02:00
Marcin Juszkiewicz
d8672795fa CI: add Debian source upgrade job on AArch64
Change-Id: I6242a3a1c24f0fa07f949897365a826f577caaac
2020-04-20 09:30:27 +00:00
Michal Nasiadka
8a0740df97 OVN Support
Implement OVN Ansible role.

Implements: blueprint ovn-controller-neutron-ansible

Depends-On: https://review.opendev.org/713422
Change-Id: Icd425dea85d58db49c838839d8f0b864b4a89a78
2020-04-09 07:40:12 +02:00
Zuul
9643dd54e2 Merge "CI: CentOS 8: Enable Masakari job and periodics" 2020-03-19 18:55:38 +00:00
Marcin Juszkiewicz
ed4607ec56 CI: add Debian/source upgrade job
Change-Id: If3f95e0bfc779879ca7d2cf45188fb76b4b772da
2020-03-19 10:03:01 +01:00
Mark Goddard
368ad387d0 CI: CentOS 8: Enable Masakari job and periodics
Depends-On: https://review.opendev.org/702706

Change-Id: I5344abe93f682418a0f61c894d64b360bc129592
Partially-Implements: blueprint centos-rhel-8
2020-03-13 10:17:49 +00:00
Zuul
011ef713f4 Merge "CI: CentOS 8: Enable Ceph Ansible upgrade check job" 2020-03-10 18:52:49 +00:00
Zuul
879707f06d Merge "CI: Add CentOS 8 upgrade to gate" 2020-03-10 11:51:13 +00:00
Mark Goddard
5b9b882ca7 CI: CentOS 8: Enable Ceph Ansible upgrade check job
Change-Id: Id83870942d8488ad0b16e0917c190b8ff3a4dcef
Partially-Implements: blueprint centos-rhel-8
2020-03-10 10:50:06 +00:00
Radosław Piliszek
a4f162ebed CI: Add CentOS 8 upgrade to gate
Change-Id: If5afd6f7dbf45ccee95e99ddf080b276466a2eda
2020-03-09 21:52:17 +01:00
Radosław Piliszek
464e7f8061 CI: Enable CentOS 8 bifrost job
Depends-on: https://review.opendev.org/711293
Change-Id: Ida5332cf2a8ac24a25eccc487e8a11c1e975ddcc
2020-03-09 21:42:19 +01:00
Marcin Juszkiewicz
f487fc5e5c zuul: drop all CentOS 7 jobs
Change-Id: If63e1d00f0e56b65b636d38a6533c63d2ee1e0e2
2020-03-06 11:13:54 +00:00
Mark Goddard
29f223c02a CentOS 8: CI: Add upgrade job
Test upgrade from CentOS 8 train to CentOS 8 master.

Change-Id: Ibff2c7f8844dec4758945cbc7aa8df80d70a3dfd
Partially-Implements: blueprint centos-rhel-8
2020-03-03 10:29:31 +00:00
Michal Nasiadka
f7bc4d78d9 CI: Add linuxbridge jobs
global_physnet_mtu needs to be set in neutron.conf, because linuxbridge-agent
discovers underlying vxlan0 interface mtu and returns an error when creating
vxlan port

CentOS8 job will not be added, because CentOS 8 iptables-ebtables package
is missing broute (--among-src) tables support required for linuxbridge agent,
see [1].

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1720637

Change-Id: I6b12f7ba95401d3342359c57ceeee8bec8aefe49
2020-02-20 17:08:04 +01:00
Zuul
6266d569e8 Merge "CI: Add CentOS 8 ceph-ansible job" 2020-02-14 11:49:34 +00:00
Michal Nasiadka
1a68c53fe0 CI: Add CentOS 8 ceph-ansible job
Change-Id: If354fbdeb2fd8d026faa0210b84da0a688c3ae0a
2020-02-14 09:29:05 +01:00
Zuul
4b26c4dd2c Merge "CI: Fix deploy guide jobs" 2020-02-13 16:43:18 +00:00
Michal Nasiadka
4e6fe7a6da Remove kolla-ceph
Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].

This change removes the Ansible code and associated CI jobs.

[1]: https://review.opendev.org/669214

Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2
2020-02-11 11:42:06 +01:00
Radosław Piliszek
2564b391ed CI: Fix deploy guide jobs
This fixes files relevant for them because k-a uses common files
for both docs and deploy guide.
We need this to finally get rid of Rocky mentions despite our
recent fixes to docs.

Change-Id: I46c564f3d3e53c54148186fbadfd7380c4f2fd30
2020-02-10 14:07:53 +00:00
Radosław Piliszek
287adab05e CentOS 8: Add deploy jobs in CI
Adds new CI job definitions for CentOS 8:

- kolla-ansible-centos8-source
- kolla-ansible-centos8-binary
- kolla-ansible-centos8-source-ceph-ansible
- kolla-ansible-centos8-source-cinder-lvm
- kolla-ansible-centos8-source-mariadb
- kolla-ansible-centos8-source-bifrost
- kolla-ansible-centos8-source-zun
- kolla-ansible-centos8-source-swift
- kolla-ansible-centos8-source-scenario-nfv
- kolla-ansible-centos8-source-ironic
- kolla-ansible-centos8-binary-ironic
- kolla-ansible-centos8-source-masakari
- kolla-ansible-centos8-source-cells

The following jobs are added to the check pipeline:

- kolla-ansible-centos8-source
- kolla-ansible-centos8-binary
- kolla-ansible-centos8-source-cinder-lvm
- kolla-ansible-centos8-source-mariadb
- kolla-ansible-centos8-source-zun
- kolla-ansible-centos8-source-swift
- kolla-ansible-centos8-source-scenario-nfv
- kolla-ansible-centos8-source-ironic
- kolla-ansible-centos8-binary-ironic
- kolla-ansible-centos8-source-cells

The following jobs are not yet passing so are not added to the check
pipeline:

- kolla-ansible-centos8-source-ceph-ansible
- kolla-ansible-centos8-source-bifrost
- kolla-ansible-centos8-source-masakari

The kolla-ansible-centos8-source job is added to the gate.

Upgrade jobs will be added when CentOS 8 support exists in Train.

Depends-On: https://review.opendev.org/704337
Depends-On: https://review.opendev.org/704848
Depends-On: https://review.opendev.org/704965

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: Ibd806feee71721b122b77d7eff33228ca1cc2853
Partially-Implements: blueprint centos-rhel-8
2020-02-06 11:43:50 +00:00
Radosław Piliszek
5b0894299e CI: Replace cinder-lvm scenario with zun scenario
As discussed during the meeting. [1]

[1] http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-02-05-15.00.html

Change-Id: I339fc7f01f640a32ff35d19ded54b6f8e2fdec15
2020-02-05 18:25:18 +01:00
Michal Nasiadka
f9046c2dd9 Ceph-Ansible upgrade jobs
Change-Id: Ifc87adbac6bef0aaab2ad0e80739654b3e3123ad
2020-02-05 08:15:29 +00:00
Radosław Piliszek
beed12f354 CI: Refactor base jobs
Since Zuul was restarted after applying [1], we can keep relevant
files listings in the base jobs per scenario without blocking
periodic job runs.
This greatly simplifies the jobs and project definitions.

[1] https://review.opendev.org/678273

Change-Id: If2e4fff6a514e6174709c2fc9f76b103f81a54ae
2020-01-27 17:12:48 +01:00
Michal Nasiadka
d8c15ad4e8 CI: Add Ceph-Ansible jobs
* Adding zuul centos-source/ubuntu-source ceph-ansible jobs
* Jobs will deploy all Ceph integrated OpenStack components, i.e.
  cinder, glance, nova
* Will utilize core openstack testing script

Depends-On: https://review.opendev.org/685032
Depends-On: https://review.opendev.org/698301

Implements: blueprint ceph-ansible
Change-Id: I233082b46785f74014177f579aeac887a25b2ae2
2020-01-24 22:37:03 +01:00
Zuul
cf6e157b79 Merge "Add AArch64 CI job" 2020-01-02 19:41:29 +00:00
Marcin Juszkiewicz
4f8326a894 Add AArch64 CI job
Debian/source to match Kolla CI test.

Change-Id: Ica45d5673db8ba1ebd4bfeb6e3d37b5ad5dd412b
2019-12-19 09:37:16 +01:00
Radosław Piliszek
7714ea724f CI: Test Swift
Partially-implements: blueprint kolla-deployment-scenario-testing
Change-Id: I82266829dba47eac81f440a53706c2e9064b8e62
2019-12-10 13:19:17 +00:00
Radosław Piliszek
a2fc684164 CI: Refactor a lot
Separate upgrade logic to is_upgrade job var and rename
scenarios to match.

Rename "ACTION" to "SCENARIO" (as it is a scenario).

Separate testing of dashboard (aka Horizon) and increase
its timeout to 5 minutes (CentOS 7 slow as always).

Separate initialization of core OpenStack.

Use gate setup script from ./tests/

Remove useless tox setupenv.

Do not deploy Heat when not really necessary.

Change-Id: I4fca319ccc3de7188f8b7b44c9c71321e3899467
2019-11-26 17:30:14 +01:00
Marcin Juszkiewicz
e0f4613622 CI: Switch to Ussuri jobs and drop py27 one
Change-Id: Ie42e4f96df38888b3f64515cfe84504727d3def3
2019-11-19 15:33:57 +01:00
Mark Goddard
ed996ef90d CI: Add mariadb test
Tests the following operations for MariaDB:

* Stop
* Recovery

Backup and restore will be added in a separate change.

Depends-On: https://review.opendev.org/693329
Change-Id: I836d91554715cce0e82c1bbebb7430c457418b2d
2019-11-07 18:03:07 +00:00
Doug Szumski
78a828ef42 Support multiple nova cells
This patch adds initial support for deploying multiple Nova cells.

Splitting a nova-cell role out from the Nova role allows a more granular
approach to deploying and configuring Nova services.

A new enable_cells flag has been added that enables the support of
multiple cells via the introduction of a super conductor in addition to
cell-specific conductors. When this flag is not set (the default), nova
is configured in the same manner as before - with a single conductor.

The nova role now deploys the global services:

* nova-api
* nova-scheduler
* nova-super-conductor (if enable_cells is true)

The nova-cell role handles services specific to a cell:

* nova-compute
* nova-compute-ironic
* nova-conductor
* nova-libvirt
* nova-novncproxy
* nova-serialproxy
* nova-spicehtml5proxy
* nova-ssh

This patch does not support using a single cell controller for managing
more than one cell. Support for sharing a cell controller will be added
in a future patch.

This patch should be backwards compatible and is tested by existing CI
jobs. A new CI job has been added that tests a multi-cell environment.

ceph-mon has been removed from the play hosts list as it is not
necessary - delegate_to does not require the host to be in the play.

Documentation will be added in a separate patch.

Partially Implements: blueprint support-nova-cells
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Change-Id: I810aad7d49db3f5a7fd9a2f0f746fd912fe03917
2019-10-16 17:42:36 +00:00
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Marcin Juszkiewicz
b9d6cc2a7f Add Debian/source CI job
Change-Id: I0628b16e3ebdb3fa8196acdc1bd9c63e75bcfb09
2019-10-04 08:57:55 +00:00