209 Commits

Author SHA1 Message Date
Clark Boylan
50e3c06ec2 Fix dbcounter installation on Jammy
There are two problems with dbcounter installation on Jammy. The first
is straightforward. We have to use `py_modules` instead of `modules` to
specify the source file. I don't know how this works on other distros
but the docs [0] seem to clearly indicate py_modules does this.

The second issue is quite an issue and requires story time. When
pip/setuptools insteall editable installs (as is done for many of the
openstack projects) it creates an easy-install.pth file that tells the
python interpreter to add the source dirs of those repos to the python
path. Normally these paths are appended to your sys.path. Pip's isolated
build env relies on the assumption that these paths are appeneded to the
path when it santizes sys.path to create the isolated environemnt.

However, when SETUPTOOLS_SYS_PATH_TECHNIQUE is set to rewrite the paths
are not appended and are inserted in the middle. This breaks pip's
isolated build env which broke dbcounter installations. We fix this by
not setting SETUPTOOLS_SYS_PATH_TECHNIQUE to rewrite. Upstream indicates
the reason we set this half a decade ago has since been fixed properly.

The reason Jammy and nothing else breaks is that python3.10 is the first
python version to use pip's isolated build envs by default.

I've locally fiddled with a patch to pip [1] to try and fix this
behavior even when rewrite is set. I don't plan to push this upstream
but it helps to illustrate where the problem lies. If someone else would
like to upstream this feel free.

Finally this change makes the jammy platform job voting again and adds
it to the gate to ensure we don't regress again.

[0] https://docs.python.org/3/distutils/sourcedist.html#specifying-the-files-to-distribute
[1] https://paste.opendev.org/show/bqVAuhgMtVtfYupZK5J6/

Change-Id: I237f5663b0f8b060f6df130de04e17e2b1695f8a
2022-05-20 10:35:18 -07:00
Dr. Jens Harbott
083eeee5af Make jammy platform jobs non-voting
We missed to add the jobs to the gate queue and so they have already
regressed before they were actually in place. Make them non-voting for
now until the issues are fixed.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I5d1f83dfe23747096163076dcf80750585c0260e
2022-05-19 13:55:35 +02:00
Zuul
08254ca312 Merge "Add Ubuntu 22.04 LTS (jammy) platform job" 2022-05-19 08:16:18 +00:00
Zuul
071374fa05 Merge "Collect status of all services" 2022-05-17 18:40:28 +00:00
Dr. Jens Harbott
5c765cb8a1 Add Ubuntu 22.04 LTS (jammy) platform job
The new Ubuntu LTS release has been made last week, start running
devstack on it as a platform job.

Horizon has issues with py310, so gets disabled for now.

Run variants with OVS and OVN(default).

Co-Authored-By: yatinkarel <ykarel@redhat.com>
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I47696273d6b009f754335b44ef3356b4f5115cd8
2022-05-12 13:54:02 +02:00
Zuul
d450e146cc Merge "Global option for enforcing scope (ENFORCE_SCOPE)" 2022-05-07 10:51:35 +00:00
42be2425d8 Collect status of all services
Would be helpful in troubleshooting services
which either fails to start or takes time to
start.

Related-Bug: #1970679
Change-Id: Iba2fce5f8b1cd00708f092e6eb5a1fbd96e97da0
2022-04-29 16:40:32 +00:00
Grzegorz Grasza
8615563df4 Global option for enforcing scope (ENFORCE_SCOPE)
This updates each devstack service library, to use it as the
default value for service-specific RBAC configuration.

Change-Id: I41061d042206c411ee3dd94ce91098e612af7ae7
2022-04-26 14:17:20 +02:00
Ghanshyam Mann
c6dfd169ae Drop centos 8 stream testing
In Zed cycle testing runtime, we are targetting the centos 9 stream
- https://governance.openstack.org/tc/reference/runtimes/zed.html

With dropping the python 3.6 support, project started adding python 3.8
as minimum, example nova:
- 56b5aed08c/setup.cfg (L13)

with that, centos 8 stream job is failing 100%
- https://zuul.openstack.org/build/970d029dc96742c3aa0f6932a35e97cf
- https://zuul.openstack.org/builds?job_name=devstack-platform-centos-8-stream&skip=0

This commit drops centos-8-stream testing so that we focus on centos-9-stream.

Change-Id: I045e67b1ca79aba1b2a7be9f88d7804c69c6d781
2022-04-25 15:24:39 -05:00
Zuul
45f71b10ef Merge "Gather performance data after tempest" 2022-04-21 23:06:35 +00:00
Dan Smith
c2772c2984 Gather performance data after tempest
This makes us gather a bunch of consistent statistics after we run
tempest that can be use to measure the impact of a given change. These
are stable metrics such as "number of DB queries made" and "how much
memory is each service using after a tempest run."

Note that this will always run after devstack to generate the JSON
file, but there are two things that control its completeness:

 - MYSQL_GATHER_PERFORMANCE must be enabled to get per-db stats
 - Unless tls-proxy is enabled, we will only get API stats for keystone

Change-Id: Ie3b1504256dc1c9c6b59634e86fa98494bcb07b1
2022-04-20 13:07:22 -07:00
afariasa
f4a703661e Add OpenStack two nodes nodeset for Centos 9
Change-Id: I01c8e5e0e88d0dcfe778f19548a2e268406ef6bf
2022-04-06 15:24:09 +00:00
Ghanshyam Mann
45b029064f Move openEuler job to experimental pipeline
OpenEuler job fails 100% of the time. As discussed in QA meeting,
we agreed to move OpenEuler job to experimental pipeline.
- https://meetings.opendev.org/meetings/qa/2022/qa.2022-03-22-15.00.log.html#l-76

Once it is fixed, we can think of adding back to regular pipeline.

Change-Id: I831889a09fabe5bed5522d17e352ec8009eac321
2022-03-29 07:18:40 +00:00
Ghanshyam Mann
369042b74f Make centos-9-stream job voting
bug#1960346 is fixed by the below series
- https://review.opendev.org/q/(topic:bug/1960346+OR+topic:wait_until_sshable_pingable)+status:merged
and now centos-9-stream job is passing and made voting
on tempest gate.

This commit makes devstack centos9 steam platform job
as voting and add it gate pipeline too.

Change-Id: Ic35420c5d58926ae90a136045a1558112accc533
2022-03-22 10:59:07 -05:00
Zuul
62d6ffa6b3 Merge "Revert "Disable enforcing scopes in Neutron temporary"" 2022-02-08 22:41:18 +00:00
Zuul
7799e74a05 Merge "Bump fedora-latest to F35" 2022-02-08 22:34:32 +00:00
Zuul
f9f6208960 Merge "Add openstack-two-node-centos-8-stream" 2022-02-07 22:42:44 +00:00
Ian Wienand
343e351627 Bump fedora-latest to F35
Generally this is straight forward, but Horizon has a dependency issue
with pyScss (described in [1]) so it is disabled, for now.

[1] https://bugs.launchpad.net/horizon/+bug/1960204

Co-Authored-By: Dr. Jens Harbott <harbott@osism.tech>
Depends-On: https://review.opendev.org/c/openstack/devstack/+/827578
Change-Id: I7c4bf0945f9ac5bd563fe0a698c09b8571c97c5e
2022-02-08 08:08:19 +11:00
Slawek Kaplonski
081c9b716f Revert "Disable enforcing scopes in Neutron temporary"
This reverts commit be7b5bf671b4cdc082fb9b7bb73ec55cab0054dd.

As related bug is fixed, lets enabled scope enforcement in Neutron
again.

Related-bug: #1959196
Change-Id: I72db7ef533e78a10734d105e6a0debef288e41a1
2022-01-28 09:52:53 +01:00
Ade Lee
1fd45940f3 Add openstack-two-node-centos-8-stream
This will allow multinode FIPS testing

Change-Id: I82b3b8fe56275aed72e13f6d1bd9170c50e5da0d
2022-01-27 15:57:13 +00:00
Slawek Kaplonski
be7b5bf671 Disable enforcing scopes in Neutron temporary
After patch [1] was merged in Neutron, enforcing scopes there
is broken.
So lets disable it temporary to unblock Devstack's gate for now.

[1] https://review.opendev.org/c/openstack/neutron/+/821208

Related-Bug: #1959196
Change-Id: I24da6f3897a638749d16f738329a873a5f9a291d
2022-01-27 16:04:32 +01:00
Zuul
3c98c21fec Merge "Fix mysqladmin failure for Fedora 34 and mariadb" 2022-01-04 19:23:59 +00:00
Zuul
858aca019c Merge "Don't enable the dstat service in CI jobs" 2022-01-03 18:00:28 +00:00
Miguel Lavalle
c1a75c6a50 Fix mysqladmin failure for Fedora 34 and mariadb
mysqladmin is incorrectly installed in Fedora 34 with mariadb. This
causes the failure of Zuul Fedora based jobs. The issue is a conflict
between mariadb and community mysql that is described in [1] and [2].

The workaround is to explicitly install package "mariadb"

Also configure an increased swap size like for the other platform jobs
in order to avoid OOM issues.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2026933
[2] https://lists.launchpad.net/maria-discuss/msg06179.html

Closes-Bug: #1956116
Change-Id: Icf6d7e1af5130689ea10b29d37cc9b188b2c9754
2022-01-03 15:33:35 +01:00
Zuul
3155217fb6 Merge "openEuler 20.03 LTS SP2 support" 2021-12-30 12:04:44 +00:00
Dr. Jens Harbott
134205c138
Don't enable the dstat service in CI jobs
We still are seeing regular job failures because the pcp package fails
to install. Assume that we can still enable it on demand when someone
needs to debug specific job issues, let us just disable it by default.

Related-Bug: 1943184
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I32ef8038e21c818623db9389588b3c6d3f98dcad
2021-12-23 12:29:14 +01:00
Kevin Zhao
7880ba665e openEuler 20.03 LTS SP2 support
openEuler is an open-source Linux based operating system. The current
openEuler kernel is based on Linux and supports multi arch, such as X86_64
and aarch64. It fully unleashes the potential of computing chips. As an
efficient, stable, and secure open-source OS built by global open-source
contributors, openEuler applies to database, big data, cloud computing,
and AI scenarios. openEuler is using RPM for package management.

Note:
Currently there is no available package for uwsgi-plugin-python3 and ovn, so that
openEuler needs manually install them from source.

Website: https://www.openeuler.org/en/

Change-Id: I169a0017998054604a63ac6c177d0f43f8a32ba6
Co-Authored-By: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
2021-12-22 14:47:27 +08:00
Slawek Kaplonski
24b65adc9c Deploy Neutron with enforced new RBAC rules
This patch adds new config option NEUTRON_ENFORCE_NEW_DEFAULTS which
if set to True will deploy Neutron with enforce new rbac defaults and
scopes.
It will also use SYSTEM_ADMIN user to interact with Neutron where it is
needed.

Depends-On: https://review.opendev.org/c/openstack/neutron/+/798821

Change-Id: I14d934f0deced34d74003b92824cad3c44ec4f5e
2021-12-20 14:42:35 +01:00
Alfredo Moralejo
5ea4c3c18c Support CentOS Stream 9
This patch includes changes required to run devstack on CentOS Stream 9
which has been already published in official repos by CentOS team [1]:

- Add RDO deps repository for CS9.
- remove xinetd package from installation for swift. Note that
  rsync-daemon is installed which should work fine.
- Replace genisoimage by xorriso in CS9.
- Use /etc/os-release to identify the distro in CS9 as it doesn't
  provide lsb_release command.
- Use pip from rpm package instead of from get-pip.py as done in Fedora.
- Add non-voting job devstack-platform-centos-9-stream to the check
  pipeline.

Change-Id: Ic67cddabd5069211dc0611994b8b8360bcd61bef
2021-12-02 09:10:48 +01:00
Dr. Jens Harbott
f8e00b86ae
Run Bullseye with more swap
Since Bullseye like Centos 8 Stream needs more memory due to changed
default settings in newer qemu versions, set the swap size to 4G, which
is the same setting already being used for the CS8 jobs successfully.

Change-Id: I83ea34d6aa647d2ab9d4d78ed354904fce836e68
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
2021-11-04 15:28:50 +01:00
Zuul
6923f7b5e6 Merge "Enable running on Debian Bullseye" 2021-10-20 14:56:29 +00:00
Zuul
d603c2a459 Merge "Remove cinder from service names" 2021-10-11 08:27:27 +00:00
Jens Harbott
959a7c262a Enable running on Debian Bullseye
Some adaption in database handling is all that is missing. Also add a
platform job that tests this.

Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I6dd3e48444dd415d84df5e7f5c74540847cdd6db
2021-10-04 17:27:33 +02:00
Lee Yarwood
982b03c605 zuul: Remove dedicated devstack-async job
I83d56c9363d481bb6d5921f5e1f9b024f136044b switched the default of
DEVSTACK_PARALLEL over to True so this dedicated job is no longer
required as *all* jobs should now be using it.

Change-Id: I0f475ab177c2cd49eeb6be861cdd11581e8e0b97
2021-09-30 13:08:35 +01:00
Rajat Dhasmana
65b46a503a Remove cinder from service names
In devstack job, cinder is not a valid service name and logs error
in gate[1] so remove it.

2021-09-28 05:44:47.791807 | controller | + functions-common:service_check:1603      :   for service in ${ENABLED_SERVICES//,/ }
2021-09-28 05:44:47.795506 | controller | + functions-common:service_check:1605      :   sudo systemctl is-enabled devstack@cinder.service
2021-09-28 05:44:47.809647 | controller | Failed to get unit file state for devstack@cinder.service: No such file or directory

[1] https://e978bdcfc0235dcd9417-6560bc3b6382c1d289b358872777ca09.ssl.cf1.rackcdn.com/801989/7/check/tempest-integrated-storage/779d1e7/job-output.txt

Change-Id: I7ca105201d82b72c7e56778425d3bce7c76047db
2021-09-28 03:13:28 -04:00
Zuul
e102559f87 Merge "Set swap size to 4G for c8 jobs" 2021-08-25 16:28:43 +00:00
Ghanshyam Mann
31f0418bad Make Ironic job as voting
ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa
job is voting on Ironic and neutron gate which mean it is
stable enough and make sense to make it voting on devstack gate too.

ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa is alias
job of ironic-tempest-bios-ipmi-direct-tinyipa so using the original
job instead of alias
- https://opendev.org/openstack/ironic/src/branch/master/zuul.d/ironic-jobs.yaml#L784

Change-Id: I95c67ad69e6eae6a72d25a851a71b7de85e56fd2
2021-08-17 08:49:28 -05:00
Ghanshyam Mann
ed323805f2 make swift-dsvm-functional job as voting
swift-dsvm-functional job test swift under python3 and
voting on swift gate whihc means this is a stable job now,
let's make this voting to devstack gate too.

Removing swift-dsvm-functional-py3 job as it does not exist anymore
after- https://review.opendev.org/c/openstack/swift/+/731318

swift-dsvm-functional itself is py3 job now.

Change-Id: I58847f74306194eaad132680815101a134fb4022
2021-08-17 08:45:02 -05:00
60b5538c33 Set swap size to 4G for c8 jobs
Tempest is failing randomly with different reasons
as mentioned in the bug, updating swap size those
issues are not seen.

Before [1] default swap size used to be 8GB but was dropped
to 1G so need to configure it in required job itself.

Did couple of tests in [2] and with 4GB+ swap jobs are
running green. On investigation found that with qemu-5
both Ubuntu and CentOS jobs have memory crunch, currently
Ubuntu jobs are not impacted as they are running with
qemu-4.

[1] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/750941
[2] https://review.opendev.org/c/openstack/devstack/+/803144

Closes-Bug: #1938914
Change-Id: I57910b5fde5ddf2bd37d93e06c1aff77c6e231e9
2021-08-12 21:27:13 +05:30
Ghanshyam Mann
6fecfd4fff Add devstack-enforce-scope job to enable the rbac scope checks
keystone has system scope feature implemented since
queens release. Now Devstack also started moving towards the new RBAC.

This commit adds a new job 'devstack-enforce-scope' which enable the
scope checks on service side and see if devstack setting are fine or not.

This job will be expanded to enable the scope checks for the other service
also once they start supporting the system scope.

This will help us to test the scope check setting.

Change-Id: Ie9cd9c7e7cd8fdf8c8930e59ae9d297f86eb9a95
2021-08-11 10:32:42 -05:00
Zuul
3e58254d38 Merge "Switch fedora-latest to use fedora-34" 2021-07-29 23:49:31 +00:00
Zuul
5d380b6f7f Merge "zuul: Add /etc/libvirt to log collection" 2021-07-26 20:34:10 +00:00
Lee Yarwood
2175ff3108 zuul: Add /etc/libvirt to log collection
Useful when debugging libvirtd issues such as bug #1912310.

Related-Bug: #1912310
Change-Id: Ic8504bd61316e44215672cc44436a3b9a19e114d
2021-06-25 10:59:52 +01:00
Rodolfo Alonso Hernandez
bf13075632 Make explicit the network backend used in the CI jobs
All Neutron CI jobs (except for unit, functional and fullstack jobs),
have explicitly defined the network backend used:
- linuxbridge
- ovs
- ovn

That was discussed and approved during the Neutron CI meetings [1].

[1]https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-06-15-15.00.log.html

Depends-On: https://review.opendev.org/c/openstack/neutron/+/797051
Change-Id: Ib14542311e9b1d49829bef54f433b8a04709a9fd
2021-06-23 13:03:52 +00:00
Artem Goncharov
89baa314c1 Temporary add feature pragma
OpenStackSDK has a feature branch "feature/r1" as a preparation for the
R1.0 release. Due to different branch naming functional tests with
devstack are not running. Add temporarily (for the duration of the
feature branch) pragma to the zuul.yaml to allow Zuul run functional
tests. It will be dropped once SDK gets next major release.
Previous attemp didn't work well for other projects, therefore
explicitly include master as well.

Change-Id: I3a5722873f395bc52cc55a0fd6bcea0ebe3b74fc
2021-06-11 17:06:16 +02:00
Mohammed Naser
20d6a21e8a Switch fedora-latest to use fedora-34
Fedora 32 is now EOL, we should test against the newly released
version of Fedora which is 34.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/795604
Change-Id: I10d868aca20d1a10d3e7fcfeb78f6fda4c896ee8
2021-06-09 14:20:11 -04:00
Radosław Piliszek
81937a230a [CI] Drop CentOS Linux 8 job and nodeset
CentOS Stream 8 (aka CentOS 8 Stream) is the currently
supported runtime platform. [0]

DevStack works with CentOS Stream only now. [1]

The only usage of the nodeset being dropped is handled by the
Depends-On.

[0] https://governance.openstack.org/tc/reference/runtimes/xena.html
[1] https://review.opendev.org/c/openstack/devstack/+/759122

Depends-On: https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/795159
Change-Id: Ic0f696b46dce3dba529b53a8f9de8cda6b913c7b
2021-06-07 17:28:38 +00:00
Radosław Piliszek
96509ea025 Check centos-8-stream
CentOS Stream 8 (aka CentOS 8 Stream) is the currently
supported runtime platform. [0]

Some background history:
The Manila team has asked QA to test centos-8-stream
in the common gate.
A bit later it turned out the point releases of CentOS 8 (aka
CentOS Linux 8) will stop happening entirely by the end of 2021.
[1]

Includes a workaround to the edk2-ovmf issue on CentOS Stream 8
x86_64.

[0] https://governance.openstack.org/tc/reference/runtimes/xena.html
[1] https://lists.centos.org/pipermail/centos-devel/2020-December/075451.html

Change-Id: Iee5a262af757f27f79ba1d6f790e949427dca190
2021-06-07 06:54:20 +00:00
Zuul
4c4b9bc9ef Merge "Revert "Temporary add feature pragma"" 2021-06-05 15:05:53 +00:00
Ghanshyam
3ad1661384 Revert "Temporary add feature pragma"
This reverts commit 6843bc798c3fe5f17286e1c07ede95171bb49a25.

Reason for revert: not sure why but this end up disabling the integration job on check pipeline,

Change-Id: Icfaf8ea17b3ce2e405414c23f8075b18d297bf8b
example: latest recheck on PS12 check pipeline job for neutron - https://review.opendev.org/c/openstack/neutron/+/790060
2021-06-04 21:32:17 +00:00