1655 Commits

Author SHA1 Message Date
Zuul
08254ca312 Merge "Add Ubuntu 22.04 LTS (jammy) platform job" 2022-05-19 08:16:18 +00:00
Dan Smith
fe52d7f0a8 Change DB counting mechanism
The mysql performance_schema method for counting per-database queries
is very heavyweight in that it requires full logging (in a table) of
every query. We do hundreds of thousands in the course of a tempest
run, which ends up creating its own performance problem.

This changes the approach we take, which is to bundle a very tiny
sqlalchemy plugin module which counts just what we care about in
a special database.

It is more complex than just enabling the features in mysql, but it
is a massively smaller runtime overhead. It also provides us the
opportunity to easily zero the counters just before a tempest run.

Change-Id: I361bc30bb970cdaf18b966951f217862d302f0b9
2022-05-12 07:55:02 -07:00
Dr. Jens Harbott
5c765cb8a1 Add Ubuntu 22.04 LTS (jammy) platform job
The new Ubuntu LTS release has been made last week, start running
devstack on it as a platform job.

Horizon has issues with py310, so gets disabled for now.

Run variants with OVS and OVN(default).

Co-Authored-By: yatinkarel <ykarel@redhat.com>
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I47696273d6b009f754335b44ef3356b4f5115cd8
2022-05-12 13:54:02 +02:00
Sean Mooney
eca9783a0a ensure /usr/local/bin in in path
osc is typicaly installed in /usr/local/bin
to avoid command not found errors when invoking osc
in devstack ensure that /usr/local/bin is included
in the PATH.

Change-Id: I605fbc4b131149bf5d1b6307b360fe365c680b1a
2022-04-07 13:31:50 +01:00
Ian Wienand
343e351627 Bump fedora-latest to F35
Generally this is straight forward, but Horizon has a dependency issue
with pyScss (described in [1]) so it is disabled, for now.

[1] https://bugs.launchpad.net/horizon/+bug/1960204

Co-Authored-By: Dr. Jens Harbott <harbott@osism.tech>
Depends-On: https://review.opendev.org/c/openstack/devstack/+/827578
Change-Id: I7c4bf0945f9ac5bd563fe0a698c09b8571c97c5e
2022-02-08 08:08:19 +11:00
Dr. Jens Harbott
807330ac37
Fix cloning requirements when GIT_DEPTH is set
We always need the master branch of requirements in order to be able to
install tempest with it, so override GIT_DEPTH when cloning that repo.

Closes-Bug: 1956616
Change-Id: Id0b409bfadd73f2c30314724178d6e199121050b
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
2022-01-07 11:42:26 +01:00
Zuul
7b0251cf37 Merge "Fix stacking without preconfigured DATABASE_PASSWORD" 2022-01-04 18:33:26 +00:00
Dr. Jens Harbott
353c3f9cb1
Fix stacking without preconfigured DATABASE_PASSWORD
When we need to read a DATABASE_PASSWORD from the user, make sure we
actually use it in our database URLs.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I5ebf6b0280e82f2c87a63cbee7a9957c6bd26898
2021-12-23 12:07:29 +01:00
Kevin Zhao
7880ba665e openEuler 20.03 LTS SP2 support
openEuler is an open-source Linux based operating system. The current
openEuler kernel is based on Linux and supports multi arch, such as X86_64
and aarch64. It fully unleashes the potential of computing chips. As an
efficient, stable, and secure open-source OS built by global open-source
contributors, openEuler applies to database, big data, cloud computing,
and AI scenarios. openEuler is using RPM for package management.

Note:
Currently there is no available package for uwsgi-plugin-python3 and ovn, so that
openEuler needs manually install them from source.

Website: https://www.openeuler.org/en/

Change-Id: I169a0017998054604a63ac6c177d0f43f8a32ba6
Co-Authored-By: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
2021-12-22 14:47:27 +08:00
Zuul
6c849e3713 Merge "Support CentOS Stream 9" 2021-12-04 18:18:06 +00:00
Alfredo Moralejo
5ea4c3c18c Support CentOS Stream 9
This patch includes changes required to run devstack on CentOS Stream 9
which has been already published in official repos by CentOS team [1]:

- Add RDO deps repository for CS9.
- remove xinetd package from installation for swift. Note that
  rsync-daemon is installed which should work fine.
- Replace genisoimage by xorriso in CS9.
- Use /etc/os-release to identify the distro in CS9 as it doesn't
  provide lsb_release command.
- Use pip from rpm package instead of from get-pip.py as done in Fedora.
- Add non-voting job devstack-platform-centos-9-stream to the check
  pipeline.

Change-Id: Ic67cddabd5069211dc0611994b8b8360bcd61bef
2021-12-02 09:10:48 +01:00
Julia Kreger
c96993d138 Make OS_CLOUD be able to be configured
OS_CLOUD is used to communiate to CLI tools what cloud
credentials to utilize.
The change I86ffa9cd52454f1c1c72d29b3a0e0caa3e44b829
unfortunately set an explicit OS_CLOUD account which breaks
any jobs which are expecting a previosuly set OS_CLOUD which
may be different to work. For example, OS_CLOUD being set
as devstack-system-admin to facilitate Secure RBAC testing.

Change-Id: Iee900e552584dda622f57eea3508df48dff2e071
2021-11-18 10:39:36 -08:00
Dr. Jens Harbott
f56f7a557a
Stop creating userrc_early
We can use the devstack-admin cloud configuration everywhere now
and don't need to set environment variables with cloud credentials
any longer.

Fix the swift setup, where some more options need to be explicitly
specified now and the default OS_CLOUD setting overridden.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I86ffa9cd52454f1c1c72d29b3a0e0caa3e44b829
2021-11-16 19:35:32 +01:00
Zuul
fb2e741008 Merge "Stop creating a keystone admin site" 2021-11-03 12:30:07 +00:00
Zuul
bca0448438 Merge "Create clouds.yaml early enough" 2021-11-02 17:54:37 +00:00
Jens Harbott
c2491bac9d Stop creating a keystone admin site
Keystone no longer has any special functionality hidden behind the admin
site. KEYSTONE_AUTH_URI which used to point to the admin site has long
ago been changed to be a copy of KEYSTONE_SERVICE_URI, which points to
the public site.

Drop all KEYSTONE_AUTH_* variables except KEYSTONE_AUTH_URI which may
still be in use in some plugins.

This also allows to finally drop the fixup_keystone() function.

Change-Id: I549f3cadc27d137e014241cdd47e90267859c848
2021-10-17 17:11:03 +02:00
Dr. Jens Harbott
84901f563e
Create clouds.yaml early enough
When using glance limits, the create_glance_accounts call needs access
to the devstack-system-admin cloud definition, so we need to create
the clouds.yaml file before that step.

Change-Id: Ie6d807c46b88b16b316aa166870a6a13f2bb346d
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
2021-10-15 15:07:15 +02:00
Jens Harbott
959a7c262a Enable running on Debian Bullseye
Some adaption in database handling is all that is missing. Also add a
platform job that tests this.

Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I6dd3e48444dd415d84df5e7f5c74540847cdd6db
2021-10-04 17:27:33 +02:00
Zuul
ebc7b0ed25 Merge "Simplify pip install" 2021-08-12 19:54:55 +00:00
Zuul
78a07d709c Merge "Add Fedora 34 support" 2021-08-12 14:29:59 +00:00
Ian Wienand
8dac135cb8 Simplify pip install
The uninstall here has been around since
Ibb4b42119dc2e51577c77bbbbffb110863e5324d.  At the time, there might
have been conflicts between packaged and installed pip.  We don't need
it today; get-pip.py keeps itself separate enough in /usr/local on all
platforms.  Thus we can also remove the suse/centos special-casing.

python3-pip is in the RPM list so we don't need to
re-install for Fedora.

Add a note on why we are over-installing pip.

Remove some old setuptools workarounds that are commented out.

Change-Id: Ie3cb81a8ff71cf4b81e23831c380f83b0381de71
2021-08-11 15:00:18 +10:00
Ian Wienand
156ccbad85 Add Fedora 34 support
* update the support distro filter

* don't install xinetd which doesn't exist in F34 any more.  I think
  there is probably a bit more to do with swift ring-server but that
  can be a problem for another time.

* remove old F31 workaround

Change-Id:If2f74f146a166b9721540aaf3f1f9fce3030525c
2021-08-11 14:07:21 +10:00
0456baaee5 Fix Usage of rdo-release rpm
rdo-release.el8.rpm rpm points to latest RDO release,
so use it for master, for stable releases use corresponding
release rpm.

Change-Id: I508eceb00d7501ffcfac73d7bc2272badb241494
2021-07-30 19:32:51 +05:30
Zuul
2e559c7dfb Merge "os_vif: Add support for configuring os_vif_ovs plugin" 2021-06-22 19:03:33 +00:00
Gregory Thiemonge
7befae663c Delay horizon startup
Move the 'Starting Horizon' task after the end of the wait for
create_flavors.
The start_horizon function restarts the httpd server, the openstack
services are unavailable during a short period of time, so the
"openstack flavor create" calls might fail randomly.

Closes-Bug: #1932580
Change-Id: I32ee7457586e3de8ba4dfce3b1a12025f9776542
2021-06-19 14:19:26 +02:00
Sean Mooney
5344885a61 os_vif: Add support for configuring os_vif_ovs plugin
This change add an os-vif lib that declares two new variables
OS_VIF_OVS_OVSDB_INTERFACE and OS_VIF_OVS_ISOLATE_VIF

The former is introduced to workaround bug #1929446 which cause the nova
and neutron agents to periodically block waiting for ovs to respond.

OS_VIF_OVS_ISOLATE_VIF is added to address bug #1734320 when using
ml2/ovs vif isolation should always be used to prevent cross tenant
traffic during a live migration.  This makes devstack more closely
mirror reality by enabling it when ml2/ovs is used and disabling it
otherwise.

Related-Bug: #1734320
Related-Bug: #1929446
Related-Bug: #1912310
Change-Id: I88254c6e22b52585506ee4907c1c03b8d4f2dac7
2021-06-18 14:05:39 +00:00
Zuul
c9abd1d6ba Merge "Drop Bionic support" 2021-05-18 07:54:47 +00:00
Clark Boylan
06b7352478 Fix async race updating nova configs
The configure_neutron_nova function updates nova configs. While that is
still running we separately update nova configs in stack.sh. This can
result in unexpected configs (that don't work). Fix this by waiting for
configure_neutron_nova to complete its work before we do nova config
updates directly in stack.sh.

For specifics we say that:

  [neutron]
  project_domain_name = Default

was missing from both nova.conf and nova-cpu.conf and instances could
not be created because keystone complained about not finding domain in
project. The strong suspicion here is that on some systems
configure_neutron_nova would write out project_domain_name while the
stack.sh inisets were running resulting in stack.sh overwriting the
project_domain_name content.

One theory is that disabling swift makes this problem more likely as
there is swift work in the middle of the async period. This is supported
by the fact that our job that hits this problem does indeed disable
swift.

Change-Id: I0961d882d555a21233c6b4fbfc077cfe33b88499
2021-04-29 11:54:50 -07:00
Ghanshyam Mann
7ad4cd07c8 Drop Bionic support
Since victoria cycle, we have moved upstream testing to
Ubuntu Focal (20.04) and so does no Bionic distro in
Xena cycle testing runtime[1]. Grenade jobs also started
running on Focal since victoria was released.

Only thing left was legacy jobs which were not migrated to
Ubuntu Focal in Victoria and as per another community-wide
goal[2], all the lgeacy jobs were suppsoed to be migrated
to zuulv3 native jobs in victoria cycle itself. One of the
pending job was in nova (nova-grenade-multinode) which is also
migrated to zuulv3 native now
- https://review.opendev.org/c/openstack/nova/+/778885

If there is any job running on bionic, we strongly recommend
to migrate it to Ubuntu Focal.

[1] https://governance.openstack.org/tc/reference/runtimes/xena.html
[2] https://governance.openstack.org/tc/goals/selected/victoria/native-zuulv3-jobs.html

Change-Id: I39e38e4a6c2e52dd3822c9fdea354258359a9f53
2021-04-29 09:33:59 -05:00
Stephen Finucane
970891a4ef Remove references to XenAPI driver
The XenAPI driver was removed during the Victoria release [1], while the
libvirt+xen driver has been removed in the Wallaby release [2]. Remove
references to Xen from DevStack since its all a no-op now.

[1] I42b302afbb1cfede7a0f7b16485a596cd70baf17
[2] I73305e82da5d8da548961b801a8e75fb0e8c4cf1

Change-Id: If7055feb88391f496a5e5e4c72008bf0050c5356
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-03-05 15:10:19 +00:00
Pierre Riteau
f361122798 Fix DevStack setup on CentOS 8.3
CentOS 8.3 changed the name of the PowerTools repository to powertools:
https://wiki.centos.org/Manuals/ReleaseNotes/CentOS8.2011#Yum_repo_file_and_repoid_changes

With this repository disabled, DevStack fails to install libyaml-devel,
which causes a failure to install many packages. In my environment
DevStack stopped with an error caused by a missing wget.

Keep the command using the old repository name, for compatibility with
older CentOS releases.

Change-Id: I5541a8aee8467abf10ce8a10d770618bdd693f02
2021-02-17 17:43:13 +01:00
Zuul
556f84aea9 Merge "Async task support" 2021-02-11 22:33:01 +00:00
Dan Smith
30d9bf9a6d Async task support
We have a *ton* of stuff in devstack that is very linear, specifically
the ten-ish minutes we spend loading osc to run a single API command
against something. We also generate configs, sync databases, and other
things that use one core of our worker and make our runtime longer
than it really needs to be.

The idea in this patch is to make it super simple to run some things
in the background and then wait for them to finish before proceeding
to something that will require them to be done. This avoids the
interleaving you would expect by redirecting the async tasks to a log
file, and then cat'ing that log file synchronously during the wait
operation. The per-task log file remains so it's easier to examine
it in isolation.

Multiple people have reported between 22-30% improvement in the
time it takes to stack with this. More can be done, but what is here
already makes a significant difference.

Change-Id: I270a910b531641b023c13f75dfedca057a1f1031
2021-02-09 15:57:04 -08:00
Vanou Ishii
eef2a0d751 Fix Early Use of die function in stack.sh
This commit fixes use of die function before it's defined.
die function can be used after sourcing $TOP_DIR/functions chain-
sources $TOP_DIR/functions-common.
Because fixed portion of stack.sh checks existence of $TOP_DIR/inc
and sourcing $TOP_DIR/function chain source $TOP_DIR/inc, this commit
uses echo & exit command instead of die function.

Closes-Bug: #1913021
Change-Id: I5ec174cf7b02269525b1bfd0bfa94ea889d16fce
2021-01-25 16:51:19 +09:00
Federico Ressi
7e3428b987 Install systemd-python from distribution package
CentOS 8 support start to fail because of an issue [1]
compiling systemd python binding modules.
Let install it from distribution packages as documented by
python-systemd project Web page [2]

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1862714
[2] https://github.com/systemd/python-systemd

Closes-Bug: #1908386
Change-Id: Ic7cfd72ce1b875e75b1cdbdd44a902b25d51abb8
2020-12-23 14:22:27 +00:00
Riccardo Pittau
8e74a617df Move supported distros to variable
Improve redability of the distro check using a variable to
store supported distributions.

Change-Id: Iffa1619065358d459bd04523b7eeae9049f8f2f0
2020-10-06 11:01:12 +02:00
Zuul
07bda155e9 Merge "Update opensuse version" 2020-09-28 19:29:34 +00:00
Zuul
0d14595b02 Merge "Remove Fedora 30 support" 2020-09-28 17:26:48 +00:00
Zuul
2c07049b40 Merge "Add Fedora 32 to supported list and use as fedora-latest" 2020-09-28 12:15:11 +00:00
Clark Boylan
471f1625bb Update opensuse version
Nova has newer qemu requirements which needs opensuse 15.2 to meet. The
infra opensuse 15 images are 15.2 now to accomodate that. Update
opensuse's supported version to match.

Change-Id: I6f3c5234920b185b2b0cd9c358371402f7a7b922
2020-09-28 09:45:39 +00:00
Lee Yarwood
f966e28739 Remove Fedora 30 support
Fedora 30 hit EOL earlier in the year and can be removed from devstack.

https://en.wikipedia.org/wiki/Fedora_version_history#Version_history

Change-Id: I47452700d520a544c93c0c187143ec763f026612
2020-09-25 11:34:03 +01:00
Lee Yarwood
fa5742f8e1 Add Fedora 32 to supported list and use as fedora-latest
This includes a workaround to a known dnsmasq >= 2.81 issue that
results in unanswered DHCP requests from instances as documented in the
following Neutron bug:

dnsmasq >= 2.81 not responding to DHCP requests with current q-dhcp configs
https://bugs.launchpad.net/neutron/+bug/1896945

For the time being we will attempt to downgrade to 2.80 to avoid this.

Related-Bug: #1896945
Change-Id: I3a760c43956221424926bd9dad0ebe9b28ae2b52
2020-09-25 11:33:57 +01:00
Abhishek Kekane
057aaa6ec9 Configure cinder store for glance
This patch will enable user to configure single cinder store as well as
multiple cinder stores for glance. Below are the parameters needs to be
added in local.conf.

A. For single store
USE_CINDER_FOR_GLANCE=True

B. For Multiple stores
USE_CINDER_FOR_GLANCE=True
GLANCE_ENABLE_MULTIPLE_STORES=True
CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-lvm:lvmdriver-1,lvm:lvmdriver-2,nfs:nfsdriver-1,ceph:cephdriver-1}
GLANCE_CINDER_DEFAULT_BACKEND=lvmdriver-1

enable_plugin devstack-plugin-nfs https://opendev.org/openstack/devstack-plugin-nfs
enable_plugin devstack-plugin-ceph https://opendev.org/openstack/devstack-plugin-ceph

NOTE:
GLANCE_CINDER_DEFAULT_BACKEND should be one of the value from CINDER_ENABLED_BACKENDS.
If you need to configure nfs and ceph backend for cinder then you need to add respective plugins in
local.conf file.
If GLANCE_ENABLE_MULTIPLE_STORES is True then it will not configure
swift store for glance even if it is enabled in local.conf file.

Needed-by: https://review.opendev.org/#/c/750018
Change-Id: Id0d63c4ea41cce389eee8dc9a96913a7d427f186
2020-09-23 15:23:15 +00:00
Lucas Alvares Gomes
1d468d45db [OVN] Move OVN module from Neutron to DevStack
As part of the Victoria PTG the Neutron team entertained the idea of
having the OVN driver as the default backend in DevStack (this hasn't
yet being decided by the community, this will be discussed within this
cycle).

For this to happen, we also would need to move the module that configures
OVN to the DevStack repository. This is what this patch is doing.

Note that we are updating the lib/neutron-legacy module instead of
lib/neutron in this patch, this is because as part of the PTG the
Neutron team has decided to un-deprecate the neutron-legacy module since
the "new" lib/neutron module is broken and nobody is current working on
it (also all services uses neutron-legacy).

Also, the ovsdbapp has been added to the ALL_LIBS list because a gate
job in the ovsdbapp project repository relies on installing the library
from source instead of pip to run.

Depends-On: https://review.opendev.org/#/c/740663/
Change-Id: Ib4194329474e8d68a90886d2a04f027eecd741df
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2020-08-11 15:47:01 +01:00
Zuul
3b37b95684 Merge "Allow plugins to override initial network creation" 2020-07-09 15:11:25 +00:00
Jens Harbott
32c00890ed Prepare for dropping keystone admin endpoint
Keystone no longer has any special functionality hidden behind the admin
endpoint. Stop referencing it in consumers, so it can later be dropped
completely.

Change-Id: I04a5d77908005268cc7c59e7e9ddeea70f6732e2
2020-06-26 15:26:22 +02:00
Jens Harbott
cd57449c9f Stop claiming to support Debian distros
We haven't had a working job for stretch or jessie in years, attempts
to get things fixed have been dropped, set let's be honest and drop
those from our support list.

Change-Id: Ia6152be79f8044f7ff039ec0911ad4938d6271f4
2020-06-23 08:08:13 +02:00
Dr. Jens Harbott
3480093b93 Make devstack run on focal (Ubuntu LTS 20.04)
- Add a nodeset and a platform job
- Drop uwsgi-py2 pkg that no longer exists
- Blacklist tests that are currently failing

Change-Id: Ib4416dc2f5e003fd770f5240a8f78213c56af8e6
2020-06-23 08:04:18 +02:00
Zuul
e0f051ced7 Merge "Clean up remainders of USE_SYSTEMD and logging setup" 2020-06-04 10:36:47 +00:00
Ian Wienand
67fd81a484 Cleanup yum things
We no longer support platforms with Yum on master.  Cleanup old
references and convert to dnf.

We don't need any of the failure wrapper stuff as dnf runs in strict
mode by default.

There seem to be a few callers out there, so we'll leave it called
yum_install for now.

Change-Id: Ie71a48fd85b00a97a14bf260cd013b18af4cce06
2020-04-30 16:05:15 +10:00