The mysql performance_schema method for counting per-database queries
is very heavyweight in that it requires full logging (in a table) of
every query. We do hundreds of thousands in the course of a tempest
run, which ends up creating its own performance problem.
This changes the approach we take, which is to bundle a very tiny
sqlalchemy plugin module which counts just what we care about in
a special database.
It is more complex than just enabling the features in mysql, but it
is a massively smaller runtime overhead. It also provides us the
opportunity to easily zero the counters just before a tempest run.
Change-Id: I361bc30bb970cdaf18b966951f217862d302f0b9
The new Ubuntu LTS release has been made last week, start running
devstack on it as a platform job.
Horizon has issues with py310, so gets disabled for now.
Run variants with OVS and OVN(default).
Co-Authored-By: yatinkarel <ykarel@redhat.com>
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I47696273d6b009f754335b44ef3356b4f5115cd8
osc is typicaly installed in /usr/local/bin
to avoid command not found errors when invoking osc
in devstack ensure that /usr/local/bin is included
in the PATH.
Change-Id: I605fbc4b131149bf5d1b6307b360fe365c680b1a
We always need the master branch of requirements in order to be able to
install tempest with it, so override GIT_DEPTH when cloning that repo.
Closes-Bug: 1956616
Change-Id: Id0b409bfadd73f2c30314724178d6e199121050b
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
When we need to read a DATABASE_PASSWORD from the user, make sure we
actually use it in our database URLs.
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I5ebf6b0280e82f2c87a63cbee7a9957c6bd26898
openEuler is an open-source Linux based operating system. The current
openEuler kernel is based on Linux and supports multi arch, such as X86_64
and aarch64. It fully unleashes the potential of computing chips. As an
efficient, stable, and secure open-source OS built by global open-source
contributors, openEuler applies to database, big data, cloud computing,
and AI scenarios. openEuler is using RPM for package management.
Note:
Currently there is no available package for uwsgi-plugin-python3 and ovn, so that
openEuler needs manually install them from source.
Website: https://www.openeuler.org/en/
Change-Id: I169a0017998054604a63ac6c177d0f43f8a32ba6
Co-Authored-By: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
This patch includes changes required to run devstack on CentOS Stream 9
which has been already published in official repos by CentOS team [1]:
- Add RDO deps repository for CS9.
- remove xinetd package from installation for swift. Note that
rsync-daemon is installed which should work fine.
- Replace genisoimage by xorriso in CS9.
- Use /etc/os-release to identify the distro in CS9 as it doesn't
provide lsb_release command.
- Use pip from rpm package instead of from get-pip.py as done in Fedora.
- Add non-voting job devstack-platform-centos-9-stream to the check
pipeline.
Change-Id: Ic67cddabd5069211dc0611994b8b8360bcd61bef
OS_CLOUD is used to communiate to CLI tools what cloud
credentials to utilize.
The change I86ffa9cd52454f1c1c72d29b3a0e0caa3e44b829
unfortunately set an explicit OS_CLOUD account which breaks
any jobs which are expecting a previosuly set OS_CLOUD which
may be different to work. For example, OS_CLOUD being set
as devstack-system-admin to facilitate Secure RBAC testing.
Change-Id: Iee900e552584dda622f57eea3508df48dff2e071
We can use the devstack-admin cloud configuration everywhere now
and don't need to set environment variables with cloud credentials
any longer.
Fix the swift setup, where some more options need to be explicitly
specified now and the default OS_CLOUD setting overridden.
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I86ffa9cd52454f1c1c72d29b3a0e0caa3e44b829
Keystone no longer has any special functionality hidden behind the admin
site. KEYSTONE_AUTH_URI which used to point to the admin site has long
ago been changed to be a copy of KEYSTONE_SERVICE_URI, which points to
the public site.
Drop all KEYSTONE_AUTH_* variables except KEYSTONE_AUTH_URI which may
still be in use in some plugins.
This also allows to finally drop the fixup_keystone() function.
Change-Id: I549f3cadc27d137e014241cdd47e90267859c848
When using glance limits, the create_glance_accounts call needs access
to the devstack-system-admin cloud definition, so we need to create
the clouds.yaml file before that step.
Change-Id: Ie6d807c46b88b16b316aa166870a6a13f2bb346d
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Some adaption in database handling is all that is missing. Also add a
platform job that tests this.
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I6dd3e48444dd415d84df5e7f5c74540847cdd6db
The uninstall here has been around since
Ibb4b42119dc2e51577c77bbbbffb110863e5324d. At the time, there might
have been conflicts between packaged and installed pip. We don't need
it today; get-pip.py keeps itself separate enough in /usr/local on all
platforms. Thus we can also remove the suse/centos special-casing.
python3-pip is in the RPM list so we don't need to
re-install for Fedora.
Add a note on why we are over-installing pip.
Remove some old setuptools workarounds that are commented out.
Change-Id: Ie3cb81a8ff71cf4b81e23831c380f83b0381de71
* update the support distro filter
* don't install xinetd which doesn't exist in F34 any more. I think
there is probably a bit more to do with swift ring-server but that
can be a problem for another time.
* remove old F31 workaround
Change-Id:If2f74f146a166b9721540aaf3f1f9fce3030525c
rdo-release.el8.rpm rpm points to latest RDO release,
so use it for master, for stable releases use corresponding
release rpm.
Change-Id: I508eceb00d7501ffcfac73d7bc2272badb241494
Move the 'Starting Horizon' task after the end of the wait for
create_flavors.
The start_horizon function restarts the httpd server, the openstack
services are unavailable during a short period of time, so the
"openstack flavor create" calls might fail randomly.
Closes-Bug: #1932580
Change-Id: I32ee7457586e3de8ba4dfce3b1a12025f9776542
This change add an os-vif lib that declares two new variables
OS_VIF_OVS_OVSDB_INTERFACE and OS_VIF_OVS_ISOLATE_VIF
The former is introduced to workaround bug #1929446 which cause the nova
and neutron agents to periodically block waiting for ovs to respond.
OS_VIF_OVS_ISOLATE_VIF is added to address bug #1734320 when using
ml2/ovs vif isolation should always be used to prevent cross tenant
traffic during a live migration. This makes devstack more closely
mirror reality by enabling it when ml2/ovs is used and disabling it
otherwise.
Related-Bug: #1734320
Related-Bug: #1929446
Related-Bug: #1912310
Change-Id: I88254c6e22b52585506ee4907c1c03b8d4f2dac7
The configure_neutron_nova function updates nova configs. While that is
still running we separately update nova configs in stack.sh. This can
result in unexpected configs (that don't work). Fix this by waiting for
configure_neutron_nova to complete its work before we do nova config
updates directly in stack.sh.
For specifics we say that:
[neutron]
project_domain_name = Default
was missing from both nova.conf and nova-cpu.conf and instances could
not be created because keystone complained about not finding domain in
project. The strong suspicion here is that on some systems
configure_neutron_nova would write out project_domain_name while the
stack.sh inisets were running resulting in stack.sh overwriting the
project_domain_name content.
One theory is that disabling swift makes this problem more likely as
there is swift work in the middle of the async period. This is supported
by the fact that our job that hits this problem does indeed disable
swift.
Change-Id: I0961d882d555a21233c6b4fbfc077cfe33b88499
Since victoria cycle, we have moved upstream testing to
Ubuntu Focal (20.04) and so does no Bionic distro in
Xena cycle testing runtime[1]. Grenade jobs also started
running on Focal since victoria was released.
Only thing left was legacy jobs which were not migrated to
Ubuntu Focal in Victoria and as per another community-wide
goal[2], all the lgeacy jobs were suppsoed to be migrated
to zuulv3 native jobs in victoria cycle itself. One of the
pending job was in nova (nova-grenade-multinode) which is also
migrated to zuulv3 native now
- https://review.opendev.org/c/openstack/nova/+/778885
If there is any job running on bionic, we strongly recommend
to migrate it to Ubuntu Focal.
[1] https://governance.openstack.org/tc/reference/runtimes/xena.html
[2] https://governance.openstack.org/tc/goals/selected/victoria/native-zuulv3-jobs.html
Change-Id: I39e38e4a6c2e52dd3822c9fdea354258359a9f53
The XenAPI driver was removed during the Victoria release [1], while the
libvirt+xen driver has been removed in the Wallaby release [2]. Remove
references to Xen from DevStack since its all a no-op now.
[1] I42b302afbb1cfede7a0f7b16485a596cd70baf17
[2] I73305e82da5d8da548961b801a8e75fb0e8c4cf1
Change-Id: If7055feb88391f496a5e5e4c72008bf0050c5356
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
CentOS 8.3 changed the name of the PowerTools repository to powertools:
https://wiki.centos.org/Manuals/ReleaseNotes/CentOS8.2011#Yum_repo_file_and_repoid_changes
With this repository disabled, DevStack fails to install libyaml-devel,
which causes a failure to install many packages. In my environment
DevStack stopped with an error caused by a missing wget.
Keep the command using the old repository name, for compatibility with
older CentOS releases.
Change-Id: I5541a8aee8467abf10ce8a10d770618bdd693f02
We have a *ton* of stuff in devstack that is very linear, specifically
the ten-ish minutes we spend loading osc to run a single API command
against something. We also generate configs, sync databases, and other
things that use one core of our worker and make our runtime longer
than it really needs to be.
The idea in this patch is to make it super simple to run some things
in the background and then wait for them to finish before proceeding
to something that will require them to be done. This avoids the
interleaving you would expect by redirecting the async tasks to a log
file, and then cat'ing that log file synchronously during the wait
operation. The per-task log file remains so it's easier to examine
it in isolation.
Multiple people have reported between 22-30% improvement in the
time it takes to stack with this. More can be done, but what is here
already makes a significant difference.
Change-Id: I270a910b531641b023c13f75dfedca057a1f1031
This commit fixes use of die function before it's defined.
die function can be used after sourcing $TOP_DIR/functions chain-
sources $TOP_DIR/functions-common.
Because fixed portion of stack.sh checks existence of $TOP_DIR/inc
and sourcing $TOP_DIR/function chain source $TOP_DIR/inc, this commit
uses echo & exit command instead of die function.
Closes-Bug: #1913021
Change-Id: I5ec174cf7b02269525b1bfd0bfa94ea889d16fce
Nova has newer qemu requirements which needs opensuse 15.2 to meet. The
infra opensuse 15 images are 15.2 now to accomodate that. Update
opensuse's supported version to match.
Change-Id: I6f3c5234920b185b2b0cd9c358371402f7a7b922
This includes a workaround to a known dnsmasq >= 2.81 issue that
results in unanswered DHCP requests from instances as documented in the
following Neutron bug:
dnsmasq >= 2.81 not responding to DHCP requests with current q-dhcp configs
https://bugs.launchpad.net/neutron/+bug/1896945
For the time being we will attempt to downgrade to 2.80 to avoid this.
Related-Bug: #1896945
Change-Id: I3a760c43956221424926bd9dad0ebe9b28ae2b52
This patch will enable user to configure single cinder store as well as
multiple cinder stores for glance. Below are the parameters needs to be
added in local.conf.
A. For single store
USE_CINDER_FOR_GLANCE=True
B. For Multiple stores
USE_CINDER_FOR_GLANCE=True
GLANCE_ENABLE_MULTIPLE_STORES=True
CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-lvm:lvmdriver-1,lvm:lvmdriver-2,nfs:nfsdriver-1,ceph:cephdriver-1}
GLANCE_CINDER_DEFAULT_BACKEND=lvmdriver-1
enable_plugin devstack-plugin-nfs https://opendev.org/openstack/devstack-plugin-nfs
enable_plugin devstack-plugin-ceph https://opendev.org/openstack/devstack-plugin-ceph
NOTE:
GLANCE_CINDER_DEFAULT_BACKEND should be one of the value from CINDER_ENABLED_BACKENDS.
If you need to configure nfs and ceph backend for cinder then you need to add respective plugins in
local.conf file.
If GLANCE_ENABLE_MULTIPLE_STORES is True then it will not configure
swift store for glance even if it is enabled in local.conf file.
Needed-by: https://review.opendev.org/#/c/750018
Change-Id: Id0d63c4ea41cce389eee8dc9a96913a7d427f186
As part of the Victoria PTG the Neutron team entertained the idea of
having the OVN driver as the default backend in DevStack (this hasn't
yet being decided by the community, this will be discussed within this
cycle).
For this to happen, we also would need to move the module that configures
OVN to the DevStack repository. This is what this patch is doing.
Note that we are updating the lib/neutron-legacy module instead of
lib/neutron in this patch, this is because as part of the PTG the
Neutron team has decided to un-deprecate the neutron-legacy module since
the "new" lib/neutron module is broken and nobody is current working on
it (also all services uses neutron-legacy).
Also, the ovsdbapp has been added to the ALL_LIBS list because a gate
job in the ovsdbapp project repository relies on installing the library
from source instead of pip to run.
Depends-On: https://review.opendev.org/#/c/740663/
Change-Id: Ib4194329474e8d68a90886d2a04f027eecd741df
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
Keystone no longer has any special functionality hidden behind the admin
endpoint. Stop referencing it in consumers, so it can later be dropped
completely.
Change-Id: I04a5d77908005268cc7c59e7e9ddeea70f6732e2
We haven't had a working job for stretch or jessie in years, attempts
to get things fixed have been dropped, set let's be honest and drop
those from our support list.
Change-Id: Ia6152be79f8044f7ff039ec0911ad4938d6271f4
- Add a nodeset and a platform job
- Drop uwsgi-py2 pkg that no longer exists
- Blacklist tests that are currently failing
Change-Id: Ib4416dc2f5e003fd770f5240a8f78213c56af8e6
We no longer support platforms with Yum on master. Cleanup old
references and convert to dnf.
We don't need any of the failure wrapper stuff as dnf runs in strict
mode by default.
There seem to be a few callers out there, so we'll leave it called
yum_install for now.
Change-Id: Ie71a48fd85b00a97a14bf260cd013b18af4cce06