glance/.zuul.yaml

550 lines
18 KiB
YAML
Raw Normal View History

- job:
name: glance-tox-oslo-tips-base
parent: tox
abstract: true
nodeset: ubuntu-noble
timeout: 2400
description: Abstract job for Glance vs. oslo libraries
# NOTE(rosmaita): we only need functional test jobs, oslo is
# already running periodic jobs using our unit tests. Those
# jobs are configured for glance in openstack/project-config/
# zuul.d/projects.yaml using the template 'periodic-jobs-with-oslo-master'
# which is defined in openstack/openstack-zuul-jobs/zuul.d/
# project-templates.yaml; the jobs the template refers to are
# defined in openstack/openstack-zuul-jobs/zuul.d/jobs.yaml
required-projects:
- name: openstack/debtcollector
- name: openstack/futurist
- name: openstack/oslo.concurrency
- name: openstack/oslo.config
- name: openstack/oslo.context
- name: openstack/oslo.db
- name: openstack/oslo.i18n
- name: openstack/oslo.log
- name: openstack/oslo.messaging
- name: openstack/oslo.middleware
- name: openstack/oslo.policy
- name: openstack/oslo.utils
- name: openstack/osprofiler
- name: openstack/stevedore
- name: openstack/taskflow
- job:
name: glance-tox-functional-py312-oslo-tips
parent: glance-tox-oslo-tips-base
description: |
Glance py312 functional tests vs. oslo libraries masters
nodeset: debian-bookworm
vars:
python_version: 3.12
tox_envlist: functional-py312
python_use_pyenv: True
- job:
name: glance-tox-functional-py311-oslo-tips
parent: glance-tox-oslo-tips-base
description: |
Glance py311 functional tests vs. oslo libraries masters
vars:
python_version: 3.11
tox_envlist: functional-py311
- job:
name: glance-tox-keystone-tips-base
parent: tox
abstract: true
nodeset: ubuntu-noble
timeout: 2400
description: Abstract job for Glance vs. keystone
required-projects:
- name: openstack/keystoneauth
- name: openstack/keystonemiddleware
- name: openstack/python-keystoneclient
- job:
name: glance-tox-py312-keystone-tips
parent: glance-tox-keystone-tips-base
description: |
Glance py312 unit tests vs. keystone masters
nodeset: debian-bookworm
vars:
python_version: 3.12
tox_envlist: py312
python_use_pyenv: True
- job:
name: glance-tox-py311-keystone-tips
parent: glance-tox-keystone-tips-base
description: |
Glance py311 unit tests vs. keystone masters
vars:
python_version: 3.11
tox_envlist: py311
- job:
name: glance-tox-functional-py312-keystone-tips
parent: glance-tox-keystone-tips-base
description: |
Glance py312 functional tests vs. keystone masters
nodeset: debian-bookworm
vars:
python_version: 3.12
tox_envlist: functional-py312
python_use_pyenv: True
- job:
name: glance-tox-functional-py311-keystone-tips
parent: glance-tox-keystone-tips-base
description: |
Glance py311 functional tests vs. keystone masters
vars:
python_version: 3.11
tox_envlist: functional-py311
- job:
name: glance-tox-glance_store-tips-base
parent: tox
abstract: true
nodeset: ubuntu-noble
timeout: 2400
description: Abstract job for Glance vs. glance_store
required-projects:
- name: openstack/glance_store
- job:
name: glance-tox-py312-glance_store-tips
parent: glance-tox-glance_store-tips-base
description: |
Glance py312 unit tests vs. glance_store master
nodeset: debian-bookworm
vars:
python_version: 3.12
tox_envlist: py312
python_use_pyenv: True
- job:
name: glance-tox-py311-glance_store-tips
parent: glance-tox-glance_store-tips-base
description: |
Glance py311 unit tests vs. glance_store master
vars:
python_version: 3.11
tox_envlist: py311
- job:
name: glance-tox-functional-py312-glance_store-tips
parent: glance-tox-glance_store-tips-base
description: |
Glance py312 functional tests vs. glance_store master
nodeset: debian-bookworm
vars:
python_version: 3.12
tox_envlist: functional-py312
python_use_pyenv: True
- job:
name: glance-tox-functional-py311-glance_store-tips
parent: glance-tox-glance_store-tips-base
description: |
Glance py311 functional tests vs. glance_store master
vars:
python_version: 3.11
tox_envlist: functional-py311
- job:
name: glance-tox-cursive-tips-base
parent: tox
abstract: true
nodeset: ubuntu-noble
timeout: 2400
description: Abstract job for Glance vs. cursive and related libs
required-projects:
- name: x/cursive
- name: openstack/python-barbicanclient
- name: openstack/castellan
- job:
name: glance-tox-py312-cursive-tips
parent: glance-tox-cursive-tips-base
description: |
Glance py312 unit tests vs. cursive (and related libs) master
nodeset: debian-bookworm
vars:
python_version: 3.12
tox_envlist: py312
python_use_pyenv: True
- job:
name: glance-tox-py311-cursive-tips
parent: glance-tox-cursive-tips-base
description: |
Glance py311 unit tests vs. cursive (and related libs) master
vars:
python_version: 3.11
tox_envlist: py311
- job:
name: glance-tox-functional-py312-cursive-tips
parent: glance-tox-cursive-tips-base
description: |
Glance py312 functional tests vs. cursive (and related libs) master
nodeset: debian-bookworm
vars:
python_version: 3.12
tox_envlist: functional-py312
python_use_pyenv: True
- job:
name: glance-tox-functional-py311-cursive-tips
parent: glance-tox-cursive-tips-base
description: |
Glance py311 functional tests vs. cursive (and related libs) master
vars:
python_version: 3.11
tox_envlist: functional-py311
- job:
name: tempest-integrated-storage-import
parent: tempest-integrated-storage
description: |
The regular tempest-integrated-storage job but with glance metadata injection
post-run: playbooks/post-check-metadata-injection.yaml
Increase timeout for tempest-integrated-storage-import job The tempest-integrated-storage-import has been timing out frequently around 2 hour 5 minutes mark and similarly one of the jobs inheriting from it, tempest-integrated-storage-import-standalone, also times out around the same time. The timeout issue has increased recently as can be seen in the job build history[1]. We had a similar issue with the glance-multistore-cinder-import job for which we increased the timeout to 3 hours[2] but the job completes successfully on average 2 hour 10 minutes and has been completing in that time for more than 8 months since the change to increase timout landed. The underlying cause could be due to a lot of reasons: 1. Lower concurrency (3) as compared to parent job (6) 2. More number of tests being added to tempest 3. Slow Devstack installation 4. Slow tempest test execution 5. memory consumption of OpenStack services increased making environment slow 6. no significant improvement in the hardware specifications on which we run the job This patch generalizes the timeout to the tempest-integrated-storage-import job so all of it's child jobs also don't face similar timeouts. Also we are updating timeout for tempest-integrated-storage-enforce-scope-new-defaults job as we see similar time out issues in that job. [1] https://zuul.opendev.org/t/openstack/builds?job_name=tempest-integrated-storage-import&skip=0 [2] https://github.com/openstack/glance/commit/64e6da6fe1c810befab1e5a59230ce8711dce709 Change-Id: I7165bb8c588659362b7b36394111c41772ac802d
2024-07-03 22:38:03 +05:30
timeout: 10800
vars:
configure_swap_size: 8192
tempest_concurrency: 3
zuul_copy_output:
/etc/glance-remote: logs
devstack_localrc:
GLANCE_STANDALONE: False
GLANCE_USE_IMPORT_WORKFLOW: True
devstack_services:
g-api-r: true
devstack_local_conf:
post-config:
$GLANCE_API_CONF:
DEFAULT:
enabled_import_methods: "[\"copy-image\", \"glance-direct\"]"
wsgi:
python_interpreter: /opt/stack/data/venv/bin/python
$GLANCE_IMAGE_IMPORT_CONF:
image_import_opts:
image_import_plugins: "['inject_image_metadata', 'image_conversion']"
inject_metadata_properties:
ignore_user_roles:
inject: |
"glance_devstack_test":"doyouseeme?"
image_conversion:
output_format: raw
test-config:
"$TEMPEST_CONFIG":
image:
image_caching_enabled: True
- job:
name: glance-centralized-cache
parent: tempest-integrated-storage-import
description: |
The regular job to test with glance centralized cache driver
vars:
devstack_local_conf:
post-config:
$GLANCE_API_CONF:
DEFAULT:
image_cache_driver: "centralized_db"
- job:
name: glance-grenade-centralized-cache
parent: grenade-multinode
description: |
Glance grenade multinode job where old glance will use
sqlite as cache driver and new glance will use centralized_db
as cache driver.
required-projects:
- opendev.org/openstack/grenade
- opendev.org/openstack/glance
vars:
# NOTE(abhishekk): We always want base devstack to install from
# stable/2023.2 where 'sqlite' is default cache driver, so that
# on upgrade we can verify that cache data is transferred from
# sqlite to central database. We will remove this job in 'E'
# cycle when 'sqlite' cache driver is removed from the glance.
grenade_from_branch: stable/2023.2
grenade_devstack_localrc:
shared:
GLANCE_STANDALONE: False
GLANCE_USE_IMPORT_WORKFLOW: True
devstack_local_conf:
test-config:
"$TEMPEST_CONFIG":
image:
image_caching_enabled: True
tox_envlist: all
tempest_test_regex: tempest.api.image.v2.admin.test_image_caching
- job:
name: glance-multistore-cinder-import
parent: tempest-integrated-storage-import
description: |
The regular import workflow job to test with multiple cinder stores
vars:
devstack_localrc:
USE_CINDER_FOR_GLANCE: True
GLANCE_ENABLE_MULTIPLE_STORES: True
CINDER_ENABLED_BACKENDS: lvm:lvmdriver-1,lvm:lvmdriver-2
GLANCE_CINDER_DEFAULT_BACKEND: lvmdriver-1
# Glance RBAC new defaults are tested by default so we need to test
# old defaults in some job.
GLANCE_ENFORCE_SCOPE: false
- job:
name: glance-s3-multistore
parent: tempest-integrated-storage-import
description: |
Just like the normal s3 job, but with glance multistore
timeout: 9000
vars:
devstack_localrc:
GLANCE_USE_S3: True
GLANCE_ENABLE_MULTIPLE_STORES: True
GLANCE_USE_IMPORT_WORKFLOW: True
GLANCE_LIMIT_IMAGE_SIZE_TOTAL: 2048
devstack_services:
c-bak: false
tls-proxy: false
s-account: true
s-container: true
s-object: true
s-proxy: true
s3api: true
openstack-cli-server: true
devstack_local_conf:
post-config:
$GLANCE_API_CONF:
DEFAULT:
enabled_backends: "cheap:file, s3_fast:s3, web:http"
cheap:
filesystem_store_datadir: /opt/stack/data/glance/images/
web:
https_insecure: false
os_glance_staging_store:
filesystem_store_datadir: /opt/stack/data/glance/os_glance_staging_store/
os_glance_tasks_store:
filesystem_store_datadir: /opt/stack/data/glance/os_glance_tasks_store/
- job:
name: glance-multistore-cinder-import-fips
parent: tempest-integrated-storage-import
description: |
The regular import workflow job to test with multiple cinder stores with
fips enabled
nodeset: devstack-single-node-centos-9-stream
pre-run: playbooks/enable-fips.yaml
vars:
configure_swap_size: 4096
nslookup_target: 'opendev.org'
devstack_local_conf:
post-config:
$GLANCE_API_CONF:
wsgi:
python_interpreter: $PYTHON
devstack_localrc:
USE_CINDER_FOR_GLANCE: True
GLANCE_ENABLE_MULTIPLE_STORES: True
CINDER_ENABLED_BACKENDS: lvm:lvmdriver-1,lvm:lvmdriver-2
GLANCE_CINDER_DEFAULT_BACKEND: lvmdriver-1
GLOBAL_VENV: false
- job:
name: tempest-integrated-storage-import-standalone
parent: tempest-integrated-storage-import
description: |
The regular tempest-integrated-storage-import-workflow job but with glance in
standalone mode
vars:
devstack_services:
g-api-r: false
devstack_localrc:
GLANCE_STANDALONE: True
- job:
name: glance-ceph-thin-provisioning
parent: devstack-plugin-ceph-tempest-py3
description: |
Just like devstack-plugin-ceph-tempest-py3, but with thin provisioning enabled
required-projects:
- name: openstack/glance_store
vars:
tempest_concurrency: 3
devstack_local_conf:
post-config:
$GLANCE_API_CONF:
glance_store:
rbd_thin_provisioning: True
# TODO(pdeore): Remove this jobs once Cinder enable the RBAC new defaults
# by default. All other services including glance enable new defaults by
# default.
- job:
name: tempest-integrated-storage-enforce-scope-new-defaults
parent: tempest-integrated-storage
nodeset: openstack-single-node-jammy
description: |
This job runs the Tempest tests with scope and new defaults enabled
Glance services.
Increase timeout for tempest-integrated-storage-import job The tempest-integrated-storage-import has been timing out frequently around 2 hour 5 minutes mark and similarly one of the jobs inheriting from it, tempest-integrated-storage-import-standalone, also times out around the same time. The timeout issue has increased recently as can be seen in the job build history[1]. We had a similar issue with the glance-multistore-cinder-import job for which we increased the timeout to 3 hours[2] but the job completes successfully on average 2 hour 10 minutes and has been completing in that time for more than 8 months since the change to increase timout landed. The underlying cause could be due to a lot of reasons: 1. Lower concurrency (3) as compared to parent job (6) 2. More number of tests being added to tempest 3. Slow Devstack installation 4. Slow tempest test execution 5. memory consumption of OpenStack services increased making environment slow 6. no significant improvement in the hardware specifications on which we run the job This patch generalizes the timeout to the tempest-integrated-storage-import job so all of it's child jobs also don't face similar timeouts. Also we are updating timeout for tempest-integrated-storage-enforce-scope-new-defaults job as we see similar time out issues in that job. [1] https://zuul.opendev.org/t/openstack/builds?job_name=tempest-integrated-storage-import&skip=0 [2] https://github.com/openstack/glance/commit/64e6da6fe1c810befab1e5a59230ce8711dce709 Change-Id: I7165bb8c588659362b7b36394111c41772ac802d
2024-07-03 22:38:03 +05:30
timeout: 10800
vars:
tempest_concurrency: 3
devstack_localrc:
# Nova and glance scope and new defaults are enabled by default in
# Devstack so we do not need to explicitly set that to True.
CINDER_ENFORCE_SCOPE: true
# NOTE(gmann): Remove this job once Glance remove the RBAC old defaults.
- job:
name: tempest-integrated-storage-rbac-old-defaults
parent: tempest-integrated-storage
description: |
This job runs the Tempest tests with glance RBAC old defaults.
timeout: 10800
vars:
tempest_concurrency: 3
devstack_localrc:
GLANCE_ENFORCE_SCOPE: false
- project:
templates:
- check-requirements
- integrated-gate-storage
- openstack-python3-jobs
- openstack-python3-jobs-arm64
- periodic-stable-jobs
- publish-openstack-docs-pti
- release-notes-jobs-python3
check:
jobs:
- openstack-tox-functional-py39
- openstack-tox-functional-py310
- openstack-tox-functional-py312
- glance-ceph-thin-provisioning:
voting: false
irrelevant-files: &tempest-irrelevant-files
- ^(test-|)requirements.txt$
- ^.*\.rst$
- ^api-ref/.*$
- ^glance/hacking/.*$
- ^glance/locale/.*$
- ^glance/tests/.*$
- ^doc/.*$
- ^releasenotes/.*$
- ^tools/.*$
- ^tox.ini$
- tempest-integrated-storage:
irrelevant-files: *tempest-irrelevant-files
- tempest-integrated-storage-enforce-scope-new-defaults:
irrelevant-files: *tempest-irrelevant-files
- tempest-integrated-storage-import:
irrelevant-files: *tempest-irrelevant-files
- tempest-integrated-storage-import-standalone:
irrelevant-files: *tempest-irrelevant-files
- glance-multistore-cinder-import:
irrelevant-files: *tempest-irrelevant-files
- grenade:
irrelevant-files: *tempest-irrelevant-files
- tempest-ipv6-only:
irrelevant-files: *tempest-irrelevant-files
- nova-ceph-multistore:
irrelevant-files: *tempest-irrelevant-files
- glance-centralized-cache:
voting: false
irrelevant-files: *tempest-irrelevant-files
- glance-grenade-centralized-cache:
voting: false
irrelevant-files: *tempest-irrelevant-files
- glance-s3-multistore:
voting: false
irrelevant-files: *tempest-irrelevant-files
- glance-secure-rbac-protection-functional
gate:
jobs:
- openstack-tox-functional-py39
- openstack-tox-functional-py310
- openstack-tox-functional-py312
- tempest-integrated-storage:
irrelevant-files: *tempest-irrelevant-files
- tempest-integrated-storage-enforce-scope-new-defaults:
irrelevant-files: *tempest-irrelevant-files
- tempest-integrated-storage-import:
irrelevant-files: *tempest-irrelevant-files
- tempest-integrated-storage-import-standalone:
irrelevant-files: *tempest-irrelevant-files
- grenade:
irrelevant-files: *tempest-irrelevant-files
- grenade-skip-level:
irrelevant-files: *tempest-irrelevant-files
- tempest-ipv6-only:
irrelevant-files: *tempest-irrelevant-files
- nova-ceph-multistore:
irrelevant-files: *tempest-irrelevant-files
- glance-secure-rbac-protection-functional
experimental:
jobs:
- glance-tox-py312-glance_store-tips
- glance-tox-py311-glance_store-tips
- glance-tox-functional-py312-glance_store-tips
- glance-tox-functional-py311-glance_store-tips
- barbican-tempest-plugin-simple-crypto
- grenade-multinode
- tempest-pg-full:
irrelevant-files: *tempest-irrelevant-files
- glance-multistore-cinder-import-fips:
irrelevant-files: *tempest-irrelevant-files
- tempest-integrated-storage-rbac-old-defaults
periodic:
jobs:
# NOTE(rosmaita): we only want the "tips" jobs to be run against
# master, hence the 'branches' qualifiers below. Without them, when
# a stable branch is cut, the tests would be run against the stable
# branch as well, which is pointless because these libraries are
# frozen (more or less) in the stable branches.
#
# The "tips" jobs can be removed from the stable branch .zuul.yaml
# files if someone is so inclined, but that would require manual
# maintenance, so we do not do it by default. Another option is
# to define these jobs in the openstack/project-config repo.
# That would make us less agile in adjusting these tests, so we
# aren't doing that either.
- glance-tox-functional-py312-oslo-tips:
branches: master
- glance-tox-functional-py311-oslo-tips:
branches: master
- glance-tox-py312-keystone-tips:
branches: master
- glance-tox-py311-keystone-tips:
branches: master
- glance-tox-functional-py312-keystone-tips:
branches: master
- glance-tox-functional-py311-keystone-tips:
branches: master
- glance-tox-py312-glance_store-tips:
branches: master
- glance-tox-py311-glance_store-tips:
branches: master
- glance-tox-functional-py312-glance_store-tips:
branches: master
- glance-tox-functional-py311-glance_store-tips:
branches: master
- glance-tox-py312-cursive-tips:
branches: master
- glance-tox-py311-cursive-tips:
branches: master
- glance-tox-functional-py312-cursive-tips:
branches: master
- glance-tox-functional-py311-cursive-tips:
branches: master
- glance-multistore-cinder-import-fips:
branches: master
- tempest-integrated-storage-rbac-old-defaults:
branches: master