195 Commits

Author SHA1 Message Date
Zuul
62d6ffa6b3 Merge "Revert "Disable enforcing scopes in Neutron temporary"" 2022-02-08 22:41:18 +00:00
Zuul
7799e74a05 Merge "Bump fedora-latest to F35" 2022-02-08 22:34:32 +00:00
Zuul
f9f6208960 Merge "Add openstack-two-node-centos-8-stream" 2022-02-07 22:42:44 +00:00
Ian Wienand
343e351627 Bump fedora-latest to F35
Generally this is straight forward, but Horizon has a dependency issue
with pyScss (described in [1]) so it is disabled, for now.

[1] https://bugs.launchpad.net/horizon/+bug/1960204

Co-Authored-By: Dr. Jens Harbott <harbott@osism.tech>
Depends-On: https://review.opendev.org/c/openstack/devstack/+/827578
Change-Id: I7c4bf0945f9ac5bd563fe0a698c09b8571c97c5e
2022-02-08 08:08:19 +11:00
Slawek Kaplonski
081c9b716f Revert "Disable enforcing scopes in Neutron temporary"
This reverts commit be7b5bf671b4cdc082fb9b7bb73ec55cab0054dd.

As related bug is fixed, lets enabled scope enforcement in Neutron
again.

Related-bug: #1959196
Change-Id: I72db7ef533e78a10734d105e6a0debef288e41a1
2022-01-28 09:52:53 +01:00
Ade Lee
1fd45940f3 Add openstack-two-node-centos-8-stream
This will allow multinode FIPS testing

Change-Id: I82b3b8fe56275aed72e13f6d1bd9170c50e5da0d
2022-01-27 15:57:13 +00:00
Slawek Kaplonski
be7b5bf671 Disable enforcing scopes in Neutron temporary
After patch [1] was merged in Neutron, enforcing scopes there
is broken.
So lets disable it temporary to unblock Devstack's gate for now.

[1] https://review.opendev.org/c/openstack/neutron/+/821208

Related-Bug: #1959196
Change-Id: I24da6f3897a638749d16f738329a873a5f9a291d
2022-01-27 16:04:32 +01:00
Zuul
3c98c21fec Merge "Fix mysqladmin failure for Fedora 34 and mariadb" 2022-01-04 19:23:59 +00:00
Zuul
858aca019c Merge "Don't enable the dstat service in CI jobs" 2022-01-03 18:00:28 +00:00
Miguel Lavalle
c1a75c6a50 Fix mysqladmin failure for Fedora 34 and mariadb
mysqladmin is incorrectly installed in Fedora 34 with mariadb. This
causes the failure of Zuul Fedora based jobs. The issue is a conflict
between mariadb and community mysql that is described in [1] and [2].

The workaround is to explicitly install package "mariadb"

Also configure an increased swap size like for the other platform jobs
in order to avoid OOM issues.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2026933
[2] https://lists.launchpad.net/maria-discuss/msg06179.html

Closes-Bug: #1956116
Change-Id: Icf6d7e1af5130689ea10b29d37cc9b188b2c9754
2022-01-03 15:33:35 +01:00
Zuul
3155217fb6 Merge "openEuler 20.03 LTS SP2 support" 2021-12-30 12:04:44 +00:00
Dr. Jens Harbott
134205c138
Don't enable the dstat service in CI jobs
We still are seeing regular job failures because the pcp package fails
to install. Assume that we can still enable it on demand when someone
needs to debug specific job issues, let us just disable it by default.

Related-Bug: 1943184
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I32ef8038e21c818623db9389588b3c6d3f98dcad
2021-12-23 12:29:14 +01:00
Kevin Zhao
7880ba665e openEuler 20.03 LTS SP2 support
openEuler is an open-source Linux based operating system. The current
openEuler kernel is based on Linux and supports multi arch, such as X86_64
and aarch64. It fully unleashes the potential of computing chips. As an
efficient, stable, and secure open-source OS built by global open-source
contributors, openEuler applies to database, big data, cloud computing,
and AI scenarios. openEuler is using RPM for package management.

Note:
Currently there is no available package for uwsgi-plugin-python3 and ovn, so that
openEuler needs manually install them from source.

Website: https://www.openeuler.org/en/

Change-Id: I169a0017998054604a63ac6c177d0f43f8a32ba6
Co-Authored-By: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
2021-12-22 14:47:27 +08:00
Slawek Kaplonski
24b65adc9c Deploy Neutron with enforced new RBAC rules
This patch adds new config option NEUTRON_ENFORCE_NEW_DEFAULTS which
if set to True will deploy Neutron with enforce new rbac defaults and
scopes.
It will also use SYSTEM_ADMIN user to interact with Neutron where it is
needed.

Depends-On: https://review.opendev.org/c/openstack/neutron/+/798821

Change-Id: I14d934f0deced34d74003b92824cad3c44ec4f5e
2021-12-20 14:42:35 +01:00
Alfredo Moralejo
5ea4c3c18c Support CentOS Stream 9
This patch includes changes required to run devstack on CentOS Stream 9
which has been already published in official repos by CentOS team [1]:

- Add RDO deps repository for CS9.
- remove xinetd package from installation for swift. Note that
  rsync-daemon is installed which should work fine.
- Replace genisoimage by xorriso in CS9.
- Use /etc/os-release to identify the distro in CS9 as it doesn't
  provide lsb_release command.
- Use pip from rpm package instead of from get-pip.py as done in Fedora.
- Add non-voting job devstack-platform-centos-9-stream to the check
  pipeline.

Change-Id: Ic67cddabd5069211dc0611994b8b8360bcd61bef
2021-12-02 09:10:48 +01:00
Dr. Jens Harbott
f8e00b86ae
Run Bullseye with more swap
Since Bullseye like Centos 8 Stream needs more memory due to changed
default settings in newer qemu versions, set the swap size to 4G, which
is the same setting already being used for the CS8 jobs successfully.

Change-Id: I83ea34d6aa647d2ab9d4d78ed354904fce836e68
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
2021-11-04 15:28:50 +01:00
Zuul
6923f7b5e6 Merge "Enable running on Debian Bullseye" 2021-10-20 14:56:29 +00:00
Zuul
d603c2a459 Merge "Remove cinder from service names" 2021-10-11 08:27:27 +00:00
Jens Harbott
959a7c262a Enable running on Debian Bullseye
Some adaption in database handling is all that is missing. Also add a
platform job that tests this.

Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I6dd3e48444dd415d84df5e7f5c74540847cdd6db
2021-10-04 17:27:33 +02:00
Lee Yarwood
982b03c605 zuul: Remove dedicated devstack-async job
I83d56c9363d481bb6d5921f5e1f9b024f136044b switched the default of
DEVSTACK_PARALLEL over to True so this dedicated job is no longer
required as *all* jobs should now be using it.

Change-Id: I0f475ab177c2cd49eeb6be861cdd11581e8e0b97
2021-09-30 13:08:35 +01:00
Rajat Dhasmana
65b46a503a Remove cinder from service names
In devstack job, cinder is not a valid service name and logs error
in gate[1] so remove it.

2021-09-28 05:44:47.791807 | controller | + functions-common:service_check:1603      :   for service in ${ENABLED_SERVICES//,/ }
2021-09-28 05:44:47.795506 | controller | + functions-common:service_check:1605      :   sudo systemctl is-enabled devstack@cinder.service
2021-09-28 05:44:47.809647 | controller | Failed to get unit file state for devstack@cinder.service: No such file or directory

[1] https://e978bdcfc0235dcd9417-6560bc3b6382c1d289b358872777ca09.ssl.cf1.rackcdn.com/801989/7/check/tempest-integrated-storage/779d1e7/job-output.txt

Change-Id: I7ca105201d82b72c7e56778425d3bce7c76047db
2021-09-28 03:13:28 -04:00
Zuul
e102559f87 Merge "Set swap size to 4G for c8 jobs" 2021-08-25 16:28:43 +00:00
Ghanshyam Mann
31f0418bad Make Ironic job as voting
ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa
job is voting on Ironic and neutron gate which mean it is
stable enough and make sense to make it voting on devstack gate too.

ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa is alias
job of ironic-tempest-bios-ipmi-direct-tinyipa so using the original
job instead of alias
- https://opendev.org/openstack/ironic/src/branch/master/zuul.d/ironic-jobs.yaml#L784

Change-Id: I95c67ad69e6eae6a72d25a851a71b7de85e56fd2
2021-08-17 08:49:28 -05:00
Ghanshyam Mann
ed323805f2 make swift-dsvm-functional job as voting
swift-dsvm-functional job test swift under python3 and
voting on swift gate whihc means this is a stable job now,
let's make this voting to devstack gate too.

Removing swift-dsvm-functional-py3 job as it does not exist anymore
after- https://review.opendev.org/c/openstack/swift/+/731318

swift-dsvm-functional itself is py3 job now.

Change-Id: I58847f74306194eaad132680815101a134fb4022
2021-08-17 08:45:02 -05:00
60b5538c33 Set swap size to 4G for c8 jobs
Tempest is failing randomly with different reasons
as mentioned in the bug, updating swap size those
issues are not seen.

Before [1] default swap size used to be 8GB but was dropped
to 1G so need to configure it in required job itself.

Did couple of tests in [2] and with 4GB+ swap jobs are
running green. On investigation found that with qemu-5
both Ubuntu and CentOS jobs have memory crunch, currently
Ubuntu jobs are not impacted as they are running with
qemu-4.

[1] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/750941
[2] https://review.opendev.org/c/openstack/devstack/+/803144

Closes-Bug: #1938914
Change-Id: I57910b5fde5ddf2bd37d93e06c1aff77c6e231e9
2021-08-12 21:27:13 +05:30
Ghanshyam Mann
6fecfd4fff Add devstack-enforce-scope job to enable the rbac scope checks
keystone has system scope feature implemented since
queens release. Now Devstack also started moving towards the new RBAC.

This commit adds a new job 'devstack-enforce-scope' which enable the
scope checks on service side and see if devstack setting are fine or not.

This job will be expanded to enable the scope checks for the other service
also once they start supporting the system scope.

This will help us to test the scope check setting.

Change-Id: Ie9cd9c7e7cd8fdf8c8930e59ae9d297f86eb9a95
2021-08-11 10:32:42 -05:00
Zuul
3e58254d38 Merge "Switch fedora-latest to use fedora-34" 2021-07-29 23:49:31 +00:00
Zuul
5d380b6f7f Merge "zuul: Add /etc/libvirt to log collection" 2021-07-26 20:34:10 +00:00
Lee Yarwood
2175ff3108 zuul: Add /etc/libvirt to log collection
Useful when debugging libvirtd issues such as bug #1912310.

Related-Bug: #1912310
Change-Id: Ic8504bd61316e44215672cc44436a3b9a19e114d
2021-06-25 10:59:52 +01:00
Rodolfo Alonso Hernandez
bf13075632 Make explicit the network backend used in the CI jobs
All Neutron CI jobs (except for unit, functional and fullstack jobs),
have explicitly defined the network backend used:
- linuxbridge
- ovs
- ovn

That was discussed and approved during the Neutron CI meetings [1].

[1]https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-06-15-15.00.log.html

Depends-On: https://review.opendev.org/c/openstack/neutron/+/797051
Change-Id: Ib14542311e9b1d49829bef54f433b8a04709a9fd
2021-06-23 13:03:52 +00:00
Artem Goncharov
89baa314c1 Temporary add feature pragma
OpenStackSDK has a feature branch "feature/r1" as a preparation for the
R1.0 release. Due to different branch naming functional tests with
devstack are not running. Add temporarily (for the duration of the
feature branch) pragma to the zuul.yaml to allow Zuul run functional
tests. It will be dropped once SDK gets next major release.
Previous attemp didn't work well for other projects, therefore
explicitly include master as well.

Change-Id: I3a5722873f395bc52cc55a0fd6bcea0ebe3b74fc
2021-06-11 17:06:16 +02:00
Mohammed Naser
20d6a21e8a Switch fedora-latest to use fedora-34
Fedora 32 is now EOL, we should test against the newly released
version of Fedora which is 34.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/795604
Change-Id: I10d868aca20d1a10d3e7fcfeb78f6fda4c896ee8
2021-06-09 14:20:11 -04:00
Radosław Piliszek
81937a230a [CI] Drop CentOS Linux 8 job and nodeset
CentOS Stream 8 (aka CentOS 8 Stream) is the currently
supported runtime platform. [0]

DevStack works with CentOS Stream only now. [1]

The only usage of the nodeset being dropped is handled by the
Depends-On.

[0] https://governance.openstack.org/tc/reference/runtimes/xena.html
[1] https://review.opendev.org/c/openstack/devstack/+/759122

Depends-On: https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/795159
Change-Id: Ic0f696b46dce3dba529b53a8f9de8cda6b913c7b
2021-06-07 17:28:38 +00:00
Radosław Piliszek
96509ea025 Check centos-8-stream
CentOS Stream 8 (aka CentOS 8 Stream) is the currently
supported runtime platform. [0]

Some background history:
The Manila team has asked QA to test centos-8-stream
in the common gate.
A bit later it turned out the point releases of CentOS 8 (aka
CentOS Linux 8) will stop happening entirely by the end of 2021.
[1]

Includes a workaround to the edk2-ovmf issue on CentOS Stream 8
x86_64.

[0] https://governance.openstack.org/tc/reference/runtimes/xena.html
[1] https://lists.centos.org/pipermail/centos-devel/2020-December/075451.html

Change-Id: Iee5a262af757f27f79ba1d6f790e949427dca190
2021-06-07 06:54:20 +00:00
Zuul
4c4b9bc9ef Merge "Revert "Temporary add feature pragma"" 2021-06-05 15:05:53 +00:00
Ghanshyam
3ad1661384 Revert "Temporary add feature pragma"
This reverts commit 6843bc798c3fe5f17286e1c07ede95171bb49a25.

Reason for revert: not sure why but this end up disabling the integration job on check pipeline,

Change-Id: Icfaf8ea17b3ce2e405414c23f8075b18d297bf8b
example: latest recheck on PS12 check pipeline job for neutron - https://review.opendev.org/c/openstack/neutron/+/790060
2021-06-04 21:32:17 +00:00
Zuul
f42347285a Merge "Temporary add feature pragma" 2021-06-04 17:52:46 +00:00
Monty Taylor
6843bc798c Temporary add feature pragma
OpenStackSDK has a feature branch "feature/r1" as a preparation for the
R1.0 release. Due to different branch naming functional tests with
devstack are not running. Add temporarily (for the duration of the
feature branch) pragma to the zuul.yaml to allow Zuul run functional
tests. It will be dropped once SDK gets next major release.

Change-Id: I671b589150fe731125e16316a994a5942219920b
2021-06-04 15:28:30 +00:00
Lucas Alvares Gomes
22038a9a8c [OVN] Configure public bridge enabled by default
This patch makes the OVN_L3_CREATE_PUBLIC_NETWORK configuration True by
default. This option makes the OVN lib in DevStack create & configure
the external bridge, matching the same behavior from the OVS driver
in DevStack.

Change-Id: Icda53b95fdc3c169ac48a6ec4343c87ba404baa4
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2021-05-27 16:17:06 +01:00
Zuul
a634c7f91c Merge "[CI] Add no-tls-proxy job" 2021-05-26 19:23:31 +00:00
Ghanshyam Mann
7604e085b4 Pin nodeset for unit test job
devstack unit test job does not set any nodeset
and so does use default nodeset defined in base jobs
in opendev. When opendev switches the default nodeset to
the latest distro version, devstack unit test job can start
failing. Example:

- https://review.opendev.org/q/I01408f2f2959b0788fe712ac268a526502226ee9
- https://review.opendev.org/q/Ib1ea47bc7384e1f579cb08c779a32151fccd6845

To avoid such a situation in future, let's set the working nodeset
for this job also so that when we cut the stable branch we can
run it on the working distro version. 

Change-Id: I302140778fedf08bc8ba72c453837fa7b8f8f9ae
2021-05-25 18:10:44 +00:00
Radosław Piliszek
ddb66f2344 [CI] Add no-tls-proxy job
Some gates run devstack like this and it follows different code
paths.
Let's ensure we don't break it now and then.

Change-Id: I6ee1bfc30bced53f6d7fb841e01714069919fd88
Reference: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/015997.html
Reference: http://eavesdrop.openstack.org/meetings/qa/2021/qa.2021-05-11-14.00.log.html
2021-05-24 17:45:06 +00:00
Lucas Alvares Gomes
e38a39ad40 Change default network backend driver to ML2/OVN
This patch is changing the default network backend driver in DevStack to
OVN.

This is a long effort from the Neutron community that has been
discussed on previous PTGs and agreed upon.

A similar patch to this one [0] have been merged in the past but got
reverted [1] because it did break some zuul jobs. This one also include
fixes for such jobs and was verified at [2]

[0] https://review.opendev.org/c/openstack/devstack/+/735097
[1] https://review.opendev.org/c/openstack/neutron/+/775632
[2] https://review.opendev.org/c/zuul/zuul-jobs/+/791117

Change-Id: I8c2be82f33ed9f6c36f5509b3b210ee1a38e87ca
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2021-05-24 09:48:35 +01:00
Ghanshyam
ff073a5643 Revert "Change Neutron's default ML2 driver to OVN"
This reverts commit 5c304d817682d6c807b532b50a2f105479ac3fa2.

Reason for revert: There are more things to fix/move like done in 791085 and 791282 Also let's change all required default in devstack scripts instead of devstack's zuul job side. Basically do this change without any change in .zuul.yaml

Change-Id: Ie0f59d1b9a4b97ad9fd8131819054dfb616f31fd
2021-05-13 16:25:17 +00:00
Lucas Alvares Gomes
5c304d8176 Change Neutron's default ML2 driver to OVN
As part of the Victoria PTG the Neutron community [0] agreed on changing
the default backend driver from ML2/OVS to ML2/OVN in DevStack. A lot of
changes have been submitted towards this goal including but not limted
to:

* Moving the OVN module to DevStack:
  https://review.opendev.org/c/openstack/devstack/+/734621

* Updating the OVN module to use distro packages instead of compiling
  OVN from source: https://review.opendev.org/c/openstack/devstack/+/763402o

And now this patch is changing the the actual Q_AGENT,
Q_ML2_TENANT_NETWORK_TYPE and Q_ML2_PLUGIN_MECHANISM_DRIVERS values in
devstack to what is expected by OVN as well as updating the Zuul
templates to enable the OVN services.

[0] https://etherpad.opendev.org/p/neutron-victoria-ptg

Change-Id: I92054ce9d2ab7a42746ed5dececef583b0f8a833
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2021-05-10 14:43:28 +01:00
Ghanshyam Mann
7ad4cd07c8 Drop Bionic support
Since victoria cycle, we have moved upstream testing to
Ubuntu Focal (20.04) and so does no Bionic distro in
Xena cycle testing runtime[1]. Grenade jobs also started
running on Focal since victoria was released.

Only thing left was legacy jobs which were not migrated to
Ubuntu Focal in Victoria and as per another community-wide
goal[2], all the lgeacy jobs were suppsoed to be migrated
to zuulv3 native jobs in victoria cycle itself. One of the
pending job was in nova (nova-grenade-multinode) which is also
migrated to zuulv3 native now
- https://review.opendev.org/c/openstack/nova/+/778885

If there is any job running on bionic, we strongly recommend
to migrate it to Ubuntu Focal.

[1] https://governance.openstack.org/tc/reference/runtimes/xena.html
[2] https://governance.openstack.org/tc/goals/selected/victoria/native-zuulv3-jobs.html

Change-Id: I39e38e4a6c2e52dd3822c9fdea354258359a9f53
2021-04-29 09:33:59 -05:00
Lee Yarwood
c062792709 cinder: Default CINDER_ISCSI_HELPER to lioadm
As outlined in bug #1917750 the use of tgtadm in multinode environments
with multiple c-vol services can cause volumes to use duplicate WWNs.

This has been shown to cause some encrypted volume test failures as
os-brick returns a /dev/disk/by-id path to n-cpu that can point to the
wrong underlying volume when multiple volumes with the same WWN are
connected to a host.

There is also some speculation that the duplicate WWNs are also causing
libvirt to fail to detach volumes from instances but as yet this has not
been proven.

This change aims to avoid all of the above by switching the default of
CINDER_ISCSI_HELPER to lioadm for all deployments instead of just EL and
SUSE based deployments.

The Bionic platform job however is pinned to tgtadm as there issues
installing python3-rtslib-fb.

Closes-Bug: #1917750
Change-Id: If5c860d1e69aaef9a9236303c370479a7714ad43
2021-04-23 10:53:10 +01:00
Zuul
556f84aea9 Merge "Async task support" 2021-02-11 22:33:01 +00:00
Dan Smith
30d9bf9a6d Async task support
We have a *ton* of stuff in devstack that is very linear, specifically
the ten-ish minutes we spend loading osc to run a single API command
against something. We also generate configs, sync databases, and other
things that use one core of our worker and make our runtime longer
than it really needs to be.

The idea in this patch is to make it super simple to run some things
in the background and then wait for them to finish before proceeding
to something that will require them to be done. This avoids the
interleaving you would expect by redirecting the async tasks to a log
file, and then cat'ing that log file synchronously during the wait
operation. The per-task log file remains so it's easier to examine
it in isolation.

Multiple people have reported between 22-30% improvement in the
time it takes to stack with this. More can be done, but what is here
already makes a significant difference.

Change-Id: I270a910b531641b023c13f75dfedca057a1f1031
2021-02-09 15:57:04 -08:00
Jens Harbott
b37240382d Drop opensuse platform job
It has been broken for over a month. Feel free to revert in combination
with a fix, better with a commitment to keep the job in working shape
permanently.

Change-Id: I2604374c23716d56de29e16a459b7c7f45b84891
2021-01-08 09:41:56 +01:00