openstack/neutron
Code Issues Proposed changes

Add tempest job which will run on the oldest Ubuntu release

Browse Source 
As was discussed in the TC meeting during last PTG [1] we should run
at least one tempest job on the previosly supported Ubuntu version to
make sure that current release can work on older operating system and
that upgrade process can be as smooth as possible for operators.

[1] https://etherpad.opendev.org/p/tc-2023-1-ptg#L428

Depends-On: https://review.opendev.org/c/openstack/tempest/+/861110
Depends-On: https://review.opendev.org/c/openstack/rally-openstack/+/862219
Depends-On: https://review.opendev.org/c/openstack/devstack/+/862479

Change-Id: I8164d76b28904cde88f4de7da08d647f6bd98a10
This commit is contained in:
Slawek Kaplonski 2022-10-24 14:58:52 +02:00
parent 0da328fbee
commit 01b11df1b5
3 changed files with 110 additions and 97 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

197
doc/source/contributor/testing/ci_scenario_jobs.rst
View File

@ -34,58 +34,61 @@ Each of those jobs runs on slightly different configuration of Neutron services.
Below is a summary of those jobs.
::
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
| Job name | Run tests | nodes | L2 agent | firewall | L3 agent | L3 HA | L3 DVR | enable_dvr | Run in gate |
| | | | | driver | mode | | | | queue |
+==============================================+==================================+=======+=============+=================+==========+=======+========+============+=============+
|neutron-tempest-plugin-designate-scenario |neutron_tempest_plugin.scenario.\ | 1 | openvswitch | openvswitch | legacy | False | False | True | No |
| |test_dns_integration | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-plugin-linuxbridge |neutron_tempest_plugin.api | 1 | linuxbridge | iptables | ha | False | False | False | Yes |
| |neutron_tempest_plugin.scenario | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | |
| | test_attach_interfaces | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | |
| | test_multiple_create | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-plugin-openvswitch |neutron_tempest_plugin.api | 1 | openvswitch | openvswitch | ha | False | False | False | Yes |
| |neutron_tempest_plugin.scenario | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | |
| | test_attach_interfaces | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | |
| | test_multiple_create | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-plugin-openvswitch-\ |neutron_tempest_plugin.api | 1 | openvswitch | iptables_hybrid | ha | False | False | False | Yes |
| iptables_hybrid |neutron_tempest_plugin.scenario | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | |
| | test_attach_interfaces | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | |
| | test_multiple_create | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-plugin-scenario-ovn | neutron_tempest_plugin.scenario | 1 | ovn | ovn | --- | False | False | False | Yes |
| |tempest.api.compute.servers.\ | | | | | | | | |
| | test_attach_interfaces | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | |
| | test_multiple_create | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovs-tempest-multinode-full |tempest.api (without slow tests) | 2 | openvswitch | openvswitch | legacy | False | False | True | Yes |
| |tempest.scenario | | | | | | | | |
| |(only tests related to | | | | | | | | |
| |Neutron and Nova) | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovs-tempest-dvr-ha-multinode-full |tempest.api (without slow tests) | 3 | openvswitch | openvswitch | dvr | True | True | True | No |
|(non-voting) |tempest.scenario | | | | dvr_snat | | | | |
| | | | | | dvr_snat | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovs-tempest-slow |tempest slow tests | 2 | openvswitch | openvswitch | legacy | False | False | True | Yes |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|tempest-integrated-networking |tempest.api (without slow tests) | 1 | ovn | ovn | --- | False | False | True | Yes |
| |tempest.scenario | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-ipv6-only-ovs-release |tempest.api (without slow tests) | 1 | ovn | ovn | --- | False | False | True | Yes |
| |(only tests related to | | | | | | | | |
| |Neutron and Nova) | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
| Job name | Run tests | nodes | Operating system | L2 agent | firewall | L3 agent | L3 HA | L3 DVR | enable_dvr | Run in gate |
| | | | | driver | mode | | | | | queue |
+==============================================+==================================+=======+==================+==============+================+==========+=======+========+============+=============+
|neutron-tempest-plugin-designate-scenario |neutron_tempest_plugin.scenario.\ | 1 | Ubuntu Jammy | openvswitch | openvswitch | legacy | False | False | True | No |
| |test_dns_integration | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-plugin-linuxbridge |neutron_tempest_plugin.api | 1 | Ubuntu Jammy | linuxbridge | iptables | ha | False | False | False | Yes |
| |neutron_tempest_plugin.scenario | | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | | |
| | test_attach_interfaces | | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | | |
| | test_multiple_create | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-plugin-openvswitch |neutron_tempest_plugin.api | 1 | Ubuntu Jammy | openvswitch | openvswitch | ha | False | False | False | Yes |
| |neutron_tempest_plugin.scenario | | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | | |
| | test_attach_interfaces | | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | | |
| | test_multiple_create | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-plugin-openvswitch-\ |neutron_tempest_plugin.api | 1 | Ubuntu Jammy | openvswitch | iptables_hybrid | ha | False | False | False | Yes |
| iptables_hybrid |neutron_tempest_plugin.scenario | | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | | |
| | test_attach_interfaces | | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | | |
| | test_multiple_create | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-plugin-scenario-ovn | neutron_tempest_plugin.scenario | 1 | Ubuntu Jammy | ovn | ovn | --- | False | False | False | Yes |
| |tempest.api.compute.servers.\ | | | | | | | | | |
| | test_attach_interfaces | | | | | | | | | |
| |tempest.api.compute.servers.\ | | | | | | | | | |
| | test_multiple_create | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovs-tempest-multinode-full |tempest.api (without slow tests) | 2 | Ubuntu Jammy | openvswitch | openvswitch | legacy | False | False | True | Yes |
| |tempest.scenario | | | | | | | | | |
| |(only tests related to | | | | | | | | | |
| |Neutron and Nova) | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovs-tempest-dvr-ha-multinode-full |tempest.api (without slow tests) | 3 | Ubuntu Jammy | openvswitch | openvswitch | dvr | True | True | True | No |
|(non-voting) |tempest.scenario | | | | | dvr_snat | | | | |
| | | | | | | dvr_snat | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovs-tempest-slow |tempest slow tests | 2 | Ubuntu Jammy | openvswitch | openvswitch | legacy | False | False | True | Yes |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|tempest-integrated-networking |tempest.api (without slow tests) | 1 | Ubuntu Jammy | ovn | ovn | --- | False | False | True | Yes |
| |tempest.scenario | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-ovs-release-ubuntu-old |tempest.api (without slow tests) | 1 | Ubuntu Focal | ovn | ovn | --- | False | False | True | Yes |
| |tempest.scenario | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-ipv6-only-ovs-release |tempest.api (without slow tests) | 1 | Ubuntu Jammy | ovn | ovn | --- | False | False | True | Yes |
| |(only tests related to | | | | | | | | | |
| |Neutron and Nova) | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
Grenade jobs running in Neutron CI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -95,15 +98,15 @@ Each of those jobs runs on slightly different configuration of Neutron services.
Below is summary of those jobs.
::
+------------------------------------+-------+-------------+-------------+----------+-------+--------+------------+-------------+
| Job name | nodes | L2 agent | firewall | L3 agent | L3 HA | L3 DVR | enable_dvr | Run in gate |
| | | | driver | mode | | | | queue |
+====================================+=======+=============+=============+==========+=======+========+============+=============+
| neutron-ovs-grenade-multinode | 2 | openvswitch | openvswitch | legacy | False | False | True | Yes |
+------------------------------------+-------+-------------+-------------+----------+-------+--------+------------+-------------+
| neutron-ovs-grenade-dvr-multinode | 2 | openvswitch | openvswitch | dvr | False | False | True | Yes |
| | | | | dvr_snat | | | | |
+------------------------------------+-------+-------------+-------------+----------+-------+--------+------------+-------------+
+------------------------------------+-------+------------------+-------------+-------------+----------+-------+--------+------------+-------------+
| Job name | nodes | Operating system | L2 agent | firewall | L3 agent | L3 HA | L3 DVR | enable_dvr | Run in gate |
| | | | | driver | mode | | | | queue |
+====================================+=======+==================+=============+=============+==========+=======+========+============+=============+
| neutron-ovs-grenade-multinode | 2 | Ubuntu Jammy | openvswitch | openvswitch | legacy | False | False | True | Yes |
+------------------------------------+-------+------------------+-------------+-------------+----------+-------+--------+------------+-------------+
| neutron-ovs-grenade-dvr-multinode | 2 | Ubuntu Jammy | openvswitch | openvswitch | dvr | False | False | True | Yes |
| | | | | | dvr_snat | | | | |
+------------------------------------+-------+------------------+-------------+-------------+----------+-------+--------+------------+-------------+
Tempest jobs running in Neutron experimental CI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -116,46 +119,46 @@ the patch in Gerrit.
Currently we have in that queue jobs like listed below.
::
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
| Job name | Run tests | nodes | L2 agent | firewall | L3 agent | L3 HA | L3 DVR | enable_dvr | Run in gate |
| | | | | driver | mode | | | | queue |
+==============================================+==================================+=======+=============+=================+==========+=======+========+============+=============+
|neutron-ovn-tempest-with-uwsgi-loki |tempest.api (without slow tests) | 1 | ovn | ovn | --- | --- | --- | --- | No |
|(non-voting) |tempest.scenario | | | | | | | | |
| |(only tests related to | | | | | | | | |
| |Neutron and Nova) | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-full-multinode-ovs-master |Various tempest api, scenario | 2 | ovn | ovn | --- | --- | --- | --- | No |
| |and neutron_tempest_plugi tests | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-ovs-master |Various tempest api, scenario | 1 | ovn | ovn | --- | --- | --- | --- | No |
| |and neutron_tempest_plugi tests | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-with-neutron-lib-master |tempest.api (without slow tests) | 1 | openvswitch | openvswitch | legacy | False | False | True | No |
| |tempest.scenario | | | | | | | | |
| |(only tests related to | | | | | | | | |
| |Neutron and Nova) | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-with-uwsgi-loki |tempest.api (without slow tests) | 1 | openvswitch | openvswitch | legacy | False | False | True | No |
|(non-voting) |tempest.scenario | | | | | | | | |
| |(only tests related to | | | | | | | | |
| |Neutron and Nova) | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-ipv6-only-ovs-master |tempest.api (without slow tests) | 1 | ovn | ovn | --- | False | False | True | Yes |
| |(only tests related to | | | | | | | | |
| |Neutron and Nova) | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+-------------+-----------------+----------+-------+--------+------------+-------------+
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
| Job name | Run tests | nodes | Operating system | L2 agent | firewall | L3 agent | L3 HA | L3 DVR | enable_dvr | Run in gate |
| | | | | | driver | mode | | | | queue |
+==============================================+==================================+=======+==================+=============+=================+==========+=======+========+============+=============+
|neutron-ovn-tempest-with-uwsgi-loki |tempest.api (without slow tests) | 1 | Ubuntu Jammy | ovn | ovn | --- | --- | --- | --- | No |
|(non-voting) |tempest.scenario | | | | | | | | | |
| |(only tests related to | | | | | | | | | |
| |Neutron and Nova) | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-full-multinode-ovs-master |Various tempest api, scenario | 2 | Ubuntu Jammy | ovn | ovn | --- | --- | --- | --- | No |
| |and neutron_tempest_plugi tests | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-ovs-master |Various tempest api, scenario | 1 | Ubuntu Jammy | ovn | ovn | --- | --- | --- | --- | No |
| |and neutron_tempest_plugi tests | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-with-neutron-lib-master |tempest.api (without slow tests) | 1 | Ubuntu Jammy | openvswitch | openvswitch | legacy | False | False | True | No |
| |tempest.scenario | | | | | | | | | |
| |(only tests related to | | | | | | | | | |
| |Neutron and Nova) | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-tempest-with-uwsgi-loki |tempest.api (without slow tests) | 1 | Ubuntu Jammy | openvswitch | openvswitch | legacy | False | False | True | No |
|(non-voting) |tempest.scenario | | | | | | | | | |
| |(only tests related to | | | | | | | | | |
| |Neutron and Nova) | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
|neutron-ovn-tempest-ipv6-only-ovs-master |tempest.api (without slow tests) | 1 | Ubuntu Jammy | ovn | ovn | --- | False | False | True | Yes |
| |(only tests related to | | | | | | | | | |
| |Neutron and Nova) | | | | | | | | | |
+----------------------------------------------+----------------------------------+-------+------------------+-------------+-----------------+----------+-------+--------+------------+-------------+
And we also have Grenade jobs in the experimental queue.
::
+--------------------------------+-------+-------------+-------------+----------+-------+--------+------------+-------------+
| Job name | nodes | L2 agent | firewall | L3 agent | L3 HA | L3 DVR | enable_dvr | Run in gate |
| | | | driver | mode | | | | queue |
+================================+=======+=============+=============+==========+=======+========+============+=============+
| neutron-ovn-grenade-multinode | 2 | OVN | --- | --- | --- | --- | False | no |
| (non-voting) | | | | | | | | |
+--------------------------------+-------+-------------+-------------+----------+-------+--------+------------+-------------+
+--------------------------------+-------+------------------+-------------+-------------+----------+--------+------------+------------+-------------+
| Job name | nodes | Operating system | L2 agent | firewall | L3 agent | L3 HA | L3 DVR | enable_dvr | Run in gate |
| | | | | driver | mode | | | | queue |
+================================+=======+=============+==================+=============+==========+========+============+============+=============+
| neutron-ovn-grenade-multinode | 2 | Ubuntu Jammy | OVN | --- | --- | --- | --- | False | no |
| (non-voting) | | | | | | | | | |
+--------------------------------+-------+------------------+-------------+-------------+----------+--------+------------+------------+-------------+
Columns description

4
zuul.d/project.yaml
View File

@ -66,6 +66,8 @@
- ^roles/.*functional.*$
- ^playbooks/.*functional.*$
- ^zuul.d/(?!(project)).*\.yaml
- neutron-ovn-tempest-ovs-release-ubuntu-old:
irrelevant-files: *ovn-irrelevant-files
gate:
jobs:
@ -77,3 +79,5 @@
- neutron-ovn-tempest-ipv6-only-ovs-release
- tempest-integrated-networking:
irrelevant-files: *ovn-irrelevant-files
- neutron-ovn-tempest-ovs-release-ubuntu-old:
irrelevant-files: *ovn-irrelevant-files

6
zuul.d/tempest-singlenode.yaml
View File

@ -561,6 +561,12 @@
description: Job testing for devstack/tempest testing Neutron with ovn driver and latest OVN version provided by the packages released in the operating system this job is deployed on
parent: neutron-ovn-base
- job:
name: neutron-ovn-tempest-ovs-release-ubuntu-old
description: Job testing for devstack/tempest testing Neutron with ovn driver and previously supported (old) version of the Ubuntu operating system
parent: neutron-ovn-base
nodeset: openstack-single-node-focal
- job:
name: neutron-ovn-tempest-ovs-master
description: Job testing for devstack/tempest testing Neutron with ovn driver and OVN master branch