437 Commits

Author SHA1 Message Date
Ivan Halomi
7a9f04573a Adding container engine to kolla_container_facts
Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/
in which was suggested to split patch into smaller ones.

This change adds container_engine variable to kolla_container_facts
module, this prepares module to be used with docker and podman as well
without further changes in roles.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I9e8fa30646844ab4a288555f3aafdda345b3a118
2022-11-02 13:44:45 +01:00
Radosław Piliszek
a4b4043308 Fix image from volume upload ERRORs and WARNINGs with Ceph RBD
By resetting image_upload_use_cinder_backend to upstream default.

When uploading volume to glance image, cinder looks at the backend's
image_upload_use_cinder_backend config knob to decide whether to try link
the glance image to a cloned volume made by cinder, i.e. by doing all work
locally and only updating glance's locations for the image (when the knob
is set to True). However, after all [1], [2] and [3], which happens since
Victoria, this option requires further config from user (using volume type
with image_service:store_id property (aka extra spec) set to the desired
glance store (even if there is only one cinder store configured).

Please read the bug report as to why the option removal is the
best option (TL;DR it is the most compatible approach).

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/708114
[2] https://review.opendev.org/c/openstack/glance_store/+/746556
[3] https://review.opendev.org/c/openstack/cinder/+/661676

Closes-Bug: #1991516
Change-Id: Ife87ee0241d907a0c407eb21811a354ed1734408
2022-10-07 15:05:12 +00:00
Radosław Piliszek
52ec5ef390 [glance][cinder] Stop owning RBD defaults
These are upstream defaults, no need to carry them around.

TrivialFix

Change-Id: I2907d5f38c6a74776961bd473553edf2d83f7257
2022-10-03 16:53:35 +02:00
Michal Nasiadka
9d5f9080da CI: rocky9 - add to multiple if base_distro statements
Change-Id: Ib068117237a199db380fcdfb757d5d0e5d34326b
2022-09-28 15:59:05 +02:00
Zuul
6bff120d2d Merge "Introduce variables for cinder backend names" 2022-09-27 17:13:40 +00:00
Michal Nasiadka
1aac65de0c Fix issues introduced by ansible-lint 6.6.0
mainly jinja spacing and jinja[invalid] related

Change-Id: I6f52f2b0c1ef76de626657d79486d31e0f47f384
2022-09-21 14:34:54 +00:00
Zuul
89c3a92066 Merge "Add api_workers for each service to defaults" 2022-08-22 15:30:33 +00:00
Will Szumski
0fe8010c88 Introduce variables for cinder backend names
This allows you to use a more descriptive name if you desire.
For example, when using cinder with multiple ceph backends, rbd-1,
doesn't convey much information. You could include location, disk
technology, etc. in the name.

Change-Id: Icfdc2e5726fec8b645d6c2c63391a13c31f2ce9a
2022-08-17 15:19:37 +01:00
Michal Arbet
4838591c6c Add loadbalancer-config role and wrap haproxy-config role inside
This patch adds loadbalancer-config role
which is "wrapper" around haproxy-config
and proxysql-config role which will be added
in follow-up patches.

Change-Id: I64d41507317081e1860a94b9481a85c8d400797d
2022-08-09 12:15:49 +02:00
Michal Arbet
baad47ac61 Edit services roles to support database sharding
Depends-On: https://review.opendev.org/c/openstack/kolla/+/769385
Depends-On: https://review.opendev.org/c/openstack/kolla/+/765781

Change-Id: I3c4182a6556dafd2c936eaab109a068674058fca
2022-08-09 12:15:26 +02:00
Zuul
0272805d5b Merge "Revert "Allow cinder-backup to be configured to use S3 backend."" 2022-07-27 09:12:02 +00:00
Michal Nasiadka
b7fe60fc4d Revert "Allow cinder-backup to be configured to use S3 backend."
This reverts commit 73fc230fe3f1d159b5bb9d62a6e15f93cecb6e7c.

Reason for revert: CI jobs failing with "msg": "{{ s3_url }}: 's3_url' is undefined"

Change-Id: Iba7099988cea0c0d8254b9e202309cd9c82a984d
2022-07-27 06:52:22 +00:00
Zuul
35a3aa3458 Merge "Allow cinder-backup to be configured to use S3 backend." 2022-07-26 09:04:00 +00:00
Michal Nasiadka
dcf5a8b65f Fix var-spacing
ansible-lint introduced var-spacing - let's fix our code.

Change-Id: I0d8aaf3c522a5a6a5495032f6dbed8a2be0251f0
2022-07-25 22:15:15 +02:00
Sergei Raiskii
73fc230fe3 Allow cinder-backup to be configured to use S3 backend.
Added options to configure S3 cinder backup driver, so cinder backup
can use S3 storage, for safekeeping backups.

Change-Id: Id6ff6206714581555baacecebfb6d8dd53bed8ac
2022-07-21 15:38:08 +03:00
Michal Arbet
3e8db91a1e Add api_workers for each service to defaults
Render {{ openstack_service_workers }} for workers
of each openstack service is not enough. There are
several services which has to have more workers because
there are more requests sent to them.

This patch is just adding default value for workers for
each service and sets {{ openstack_service_workers }} as
default, so value can be overrided in hostvars per server.
Nothing changed for normal user.

Change-Id: Ifa5863f8ec865bbf8e39c9b2add42c92abe40616
2022-07-12 20:09:16 +02:00
Will Szumski
49006e56d9 Add keystone_authtoken.service_type
Fixes an issue where access rules failed to validate:

    Cannot validate request with restricted access rules. Set
    service_type in [keystone_authtoken] to allow access rule validation

I've used the values from the endpoint. This was mostly a straight
forward copy and paste, except:

- versioned endpoints e.g cinderv3 where I stripped the version
- monasca has multiple endpoints associated with a single service. For
  this, I concatenated logging and monitoring to be logging-monitoring.

Closes-Bug: #1965111
Change-Id: Ic4b3ab60abad8c3dd96cd4923a67f2a8f9d195d7
2022-06-09 22:49:38 +02:00
Michal Arbet
eafd3a59af Remove configuration related to api_workers
This patch is removing api related configuration
from service's config files as we are using
apache mod_wsgi and this configuration is not
used.

Change-Id: I69a1542a6f24214fbf6e703782aefb566de4fb26
2022-06-02 12:09:37 +02:00
Radosław Piliszek
7ca9349b09 Do not use keystone_admin_url et al
Following up on [1].
The 3 variables are only introducing noise after we removed
the reliance on Keystone's admin port.

[1] I5099b08953789b280c915a6b7a22bdd4e3404076

Change-Id: I3f9dab93042799eda9174257e604fd1844684c1c
2022-05-28 18:19:01 +02:00
Radosław Piliszek
3e75a33ad4 Use the new image naming scheme
Change-Id: Ib4b15ed4feac82d8492b1c0f0238a752eac668e6
2022-05-23 06:37:25 +00:00
Zuul
4976bff593 Merge "Remove sanity checks" 2022-05-16 10:14:40 +00:00
Tim Beermann
591f366ed7 Remove sanity checks
"Smoke tests" for barbican, cinder, glance and keystone have been removed as discussed in PTG April 2022.

Signed-off-by: Tim Beermann <beermann@osism.tech>
Change-Id: I613287a31e0ea6aede070e7e9c519ab2f5f182bd
2022-05-13 17:41:34 +00:00
Simon Dodsley
f088b799a5 Add support for Pure Storage FlashArray backend to Cinder
Add an enable_cinder_backend_pure_iscsi and
enable_cinder_backend_pure_fc options to etc/kolla/globals.yml
to enable use of the FlashArray backend.
Update the documentation to include a section on configuring
Cinder with the FlashArray.

Implements: blueprint pure-cinder-driver
Change-Id: I464733f1322237321ed1ffff8636cf30bd1cbb38
2022-05-13 10:11:08 -04:00
Marcin Juszkiewicz
1620ab5be9 drop install_type from image names
We have only one value for install_type now and it gets removed from
image names.

Change-Id: I8bf95fd7aa9dd26b80d618ca0fcb097003b4cb0a
2022-04-20 12:29:12 +02:00
Zuul
ed148cd8dd Merge "[external-ceph] Use template instead of copy" 2022-03-19 00:04:33 +00:00
Imran Hussain
4c221be86e [external-ceph] Use template instead of copy
Consistently use template instead of copy. This has the added
advantage of allowing variables inside ceph conf files and keyrings.

Closes-Bug: 1959565

Signed-off-by: Imran Hussain <ih@imranh.co.uk>
Change-Id: Ibd0ff2641a54267ff06d3c89a26915a455dff1c1
2022-03-18 15:09:30 +00:00
Zuul
6c04445c7b Merge "cinder: restart services after upgrade" 2022-03-18 13:47:31 +00:00
Zuul
83fa907961 Merge "Add support for VMware First Class Disk (FCD)" 2022-02-21 11:07:00 +00:00
alecorps
812e03f75e Add support for VMware First Class Disk (FCD)
An FCD, also known as an Improved Virtual Disk (IVD) or
Managed Virtual Disk, is a named virtual disk independent of
a virtual machine. Using FCDs for Cinder volumes eliminates
the need for shadow virtual machines.
This patch adds Kolla support.

Change-Id: Ic0b66269e6d32762e786c95cf6da78cb201d2765
2022-02-18 11:15:14 +00:00
Zuul
55e79e7bc3 Merge "Remove obsolete opt from cinder.conf" 2022-01-07 10:14:13 +00:00
wu.chunyang
87a7425839 Remove obsolete opt from cinder.conf
glance_api_version and os_region_name are removed from cinder.
see: https://docs.openstack.org/cinder/xena/configuration/block-storage/samples/cinder.conf.html

Closes-Bug: #1830997
Change-Id: I751bfe64d47935f183ff2ca891ec56f61e618009
2022-01-04 22:11:51 +08:00
Pierre Riteau
56fc74f231 Move project_name and kolla_role_name to role vars
Role vars have a higher precedence than role defaults. This allows to
import default vars from another role via vars_files without overriding
project_name (see related bug for details).

Change-Id: I3d919736e53d6f3e1a70d1267cf42c8d2c0ad221
Related-Bug: #1951785
2021-12-31 09:26:25 +00:00
Dr. Jens Harbott
479a78706a Stop creating non-keystone admin endpoints
The admin interface for endpoints never had any real use, the
functionality was the same as for the public or internal endpoints,
except for Keystone. Even for Keystone with API v3 it would no longer
really be needed, but it is still being required by some libraries that
cannot be changed in order to stay backwards compatible.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: Icf3bf08deab2c445361f0a0124d87ad8b0e4e9d9
2021-12-21 13:09:36 +01:00
Mark Goddard
80a32c3c74 cinder: restart services after upgrade
This patch is roughly an adaptation of
Ia6fc9011ee6f5461f40a1307b72709d769814a79 for cinder.

During an upgrade, cinder pins the version of RPC calls to the minimum
seen across all services. This ensures that old services do not receive
data they cannot handle. After the upgrade is complete, all cinder
services are supposed to be reloaded to cause them to check again the
RPC versions of services and use the new latest version which should now
be supported by all running services.

There is a second issue in that it takes some time for the upgraded
services to update the cinder services database table with their new
version. We need to wait until all cinder services have done this
before the restart is performed, otherwise the RPC version cap will
remain in place. There is currently no interface in cinder available for
checking these versions, so as a workaround we use a configurable
delay with a default duration of 30 seconds, as we do for nova.

This change restarts all cinder services after an upgrade, after a 30
second delay.

Closes-Bug: #1954932
Related-Bug: #1833069

Change-Id: I9164dc589386d2c2d4daf1bf84061b806ba9988d
2021-12-18 09:42:24 +00:00
Radosław Piliszek
9ff2ecb031 Refactor and optimise image pulling
We get a nice optimisation by using a filtered loop instead
of task skipping per service with 'when'.

Partially-Implements: blueprint performance-improvements
Change-Id: I8f68100870ab90cb2d6b68a66a4c97df9ea4ff52
2021-08-10 11:57:54 +00:00
Mark Goddard
ade5bfa302 Use ansible_facts to reference facts
By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.

This change updates all references to Ansible facts within Kolla Ansible
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.

This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.

[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars

Change-Id: I7e9d5c9b8b9164d4aee3abb4e37c8f28d98ff5d1
Partially-Implements: blueprint performance-improvements
2021-06-23 10:38:06 +01:00
Zuul
46e4f5a33a Merge "Add missing region_name in keystoneauth sections" 2021-06-22 11:08:56 +00:00
Michal Arbet
7da770d290 Add missing region_name in keystoneauth sections
Closes-Bug: #1933025

Change-Id: Ib67d715ddfa986a5b70a55fdda39e6d0e3333162
2021-06-22 08:35:35 +02:00
Radosław Piliszek
0158221fd2 Drop support for Cinder ZFSSA backend
Following upstream which removed ZFSSA support in Ussuri [1].

[1] https://review.opendev.org/c/openstack/cinder/+/690137

Change-Id: Idb311e18b437fba696759ecb1cf2a6b4803aa5c5
2021-06-21 09:53:01 +00:00
Zuul
99e09537f2 Merge "cinder: fix condition to copy backend TLS certs" 2021-05-28 10:56:48 +00:00
wu.chunyang
f94c7beab1 cleanup no longer needed task for cinder
we don't need this task anymore.

Change-Id: I1ba60fa51ecc86c74d05898b897d7b84c70707ef
2021-05-10 19:36:11 +08:00
Zuul
a2eb9a6a21 Merge "Use Docker healthchecks for cinder services" 2021-03-26 19:48:32 +00:00
Mark Goddard
adf492bd9f cinder: fix condition to copy backend TLS certs
We need to import copy-certs.yml when either copying a CA file into
containers, or when a service has backend TLS enabled. Cinder only
included the former condition. This patch fixes it.

TrivialFix

Change-Id: I70aab86055cadad9abf28956c6d6e8a90a9668c0
2021-03-23 15:59:54 +00:00
wuchunyang
f332fe8bb5 Remove cinder v2 api
Cinder api v2 support removed in  wallaby.
refer to http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018697.html

Change-Id: I120101d6c30f9169f4c3811357b5ed06496c90ff
2021-03-10 11:24:40 +08:00
Zuul
4e7d1ab9cb Merge "Add variable for changing Apache HTTP timeout" 2021-03-07 10:41:37 +00:00
Doug Szumski
647ff667e6 Add variable for changing Apache HTTP timeout
In services which use the Apache HTTP server to service HTTP requests,
there exists a TimeOut directive [1] which defaults to 60 seconds. APIs
which come under heavy load, such as Cinder, can sometimes exceed this
which results in a HTTP 504 Gateway timeout, or similar. However, the
request can still be serviced without error. For example, if Nova calls
the Cinder API to detach a volume, and this operation takes longer
than the shortest of the two timeouts, Nova will emit a stack trace
with a 504 Gateway timeout. At some time later, the request to detach
the volume will succeed. The Nova and Cinder DBs then become
out-of-sync with each other, and frequently DB surgery is required.

Although strictly this category of bugs should be fixed in OpenStack
services, it is not realistic to expect this to happen in the short
term. Therefore, this change makes it easier to set the Apache HTTP
timeout via a new variable.

An example of a related bug is here:

https://bugs.launchpad.net/nova/+bug/1888665

Whilst this timeout can currently be set by overriding the WSGI
config for individual services, this change makes it much easier.

Change-Id: Ie452516655cbd40d63bdad3635fd66693e40ce34
Closes-Bug: #1917648
2021-03-04 11:25:06 +00:00
Doug Szumski
fa5511c75d Fix Cinder log parsing
The Cinder API log is currently written to a file called
cinder-wsgi.log, and the WSGI logs to cinder-api.log. Fluentd
then tries to parse the WSGI log as an OpenStack log which
results in 'got incomplete line' errors and prevents proper
ingestion of these logs.

Co-Authored-By: yaoning <yaoning@unitedstack.com>
Closes-Bug: 1916752
Change-Id: I3296dcc4780160cbf88bd18285571276f58bb249
2021-02-24 13:27:00 +00:00
Zuul
4d4dff0cb8 Merge "Use a better process name for httpd subprocess" 2021-02-20 05:07:48 +00:00
wu.chunyang
1e9506a45b Use Docker healthchecks for cinder services
This change enables the use of Docker healthchecks for cinder
services.

Change-Id: I9bef02a66aae2024052078d8354059ab28a71e1e
Implements: blueprint container-health-check
2021-01-09 18:51:06 +08:00
Mark Goddard
11a60a6568 Add support for using a tmpfs for Cinder image conversion
This can improve performance of image format conversion and encryption, if
sufficient memory is available on the cinder-volume host.

Closes-Bug: #1897276

Change-Id: I4ca1c4db7b66fdfc6bb873aad2570234f3882d81
2020-12-16 10:02:48 +00:00