584 Commits

Author SHA1 Message Date
Zuul
2aa4922bdd Merge "Add support for transferring encrypted volumes" 2022-08-30 11:02:51 +00:00
Alan Bishop
d59e41fb3c Add support for transferring encrypted volumes
A new microversion 3.70 adds the ability to transfer a volume's
encryption key when transferring a volume to another project.

When the volume transfer is initiated, the volume's encryption
secret is essentially transferred to the cinder service.
- The cinder service creates a new encryption_key_id that contains
  a copy of the volume's encryption secret.
- The volume (and its snapshots) is updated with the new
  encryption_key_id (the one owned by the cinder service).
- The volume's original encryption_key_id (owned by the volume's
  owner) is deleted.

When the transfer is accepted, the secret is transferred to the
user accepting the transfer.
- A new encryption_key_id is generated on behalf of the new user
  that contains a copy of the volume's encryption secret.
- The volume (and its snapshots) is updated with the new
  encryption_key_id (the one owned by the user).
- The intermediate encryption_key_id owned by the cinder service
  is deleted.

When a transfer is cancelled (deleted), the same process is used
to transfer ownship back to the user that cancelled the transfer.

Implements: blueprint transfer-encrypted-volume
Change-Id: I459f06504e90025c9c0b539981d3d56a2a9394c7
2022-08-26 06:32:41 -07:00
Zuul
7fa996bb66 Merge "Update volume delete api-ref" 2022-08-20 15:44:03 +00:00
tushargite96
dc6388b6fd Update volume delete api-ref
This patch update volume delete section in api-ref
to understand preconditions more correctly

Partial-Bug: #1918932
Change-Id: Icb5b4da3423555cd2ab0f5ee6d1dc881fc782484
2022-08-18 15:44:42 +05:30
Gorka Eguileor
ef741228d8 Report tri-state shared_targets for NVMe volumes
NVMe-oF drivers that share the subsystem have the same race condition
issue that iSCSI volumes that share targets do.

The race condition is caused by AER messages that trigger automatic
rescans on the connector host side in both cases.

For iSCSI we added a feature on the Open-iSCSI project that allowed
disabling these scans, and added support for it in os-brick.

Since manual scans is a new feature that may be missing in a host's
iSCSI client, cinder has a flag in volumes to indicate when they use
shared targets.  Using that flag os-brick consumers can use the
"guard_connection" context manager to ensure race conditions don't
happen.

The race condition is prevented by os-brick using manual scans if they
are available in the iSCSI client, or a file lock if not.

The problem we face now is that we also want to use the lock for NVMe-oF
volumes that share a subsystem for multiple namespaces (there is no way
to disable automatic scans), but cinder doesn't (and shouldn't) expose
the actual storage protocol on the volume resource, so we need to
leverage the "shared_targets" parameter.

So with a single boolean value we need to encode 3 possible options:

- Don't use locks because targets/subystems are not shared
- Use locks if iSCSI client doesn't support automatic connections
- Always use locks (for example for NVMe-oF)

The only option we have is using the "None" value as well. That way we
can encode 3 different cases.

But we have an additional restriction, "True" is already taken for the
iSCSI case, because there will exist volumes in the database that
already have that value stored.

And making guard_connection always lock when shared_targets is set to
True will introduce the bottleneck from bug (#1800515).

That leaves us with the "None" value to force the use of locks.

So we end up with the following tristate for "shared_targets":

- True to use lock if iSCSI initiator doesn't support manual scans
- False means that os-brick should never lock.
- None means that os-brick should always lock.

The alternative to this encoding would be to have an online data
migration for volumes to change "True" to "None", and accept that there
could be race conditions during the rolling upgrade (because os-brick on
computes will interpret "None" as "False").

Since "in theory" Cinder was only returning True or False for the
"shared_target", we add a new microversion with number 3.69 that returns
null when the value is internally set to None.

The patch also updates the database with a migration, though it looks
like it's not necessary since the DB already allows null values, but it
seems more correct to make sure that's always the case.

This patch doesn't close but #1961102 because the os-brick patch is
needed for that.

Related-Bug: #1961102
Change-Id: I8cda6d9830f39e27ac700b1d8796fe0489fd7c0a
2022-05-24 15:13:23 +02:00
James Palmer
0e6ba48347 Change key "cluster" to "cluster_name"
According to the request documentation above this sample,
the key "cluster" should be "cluster_name".

Closes-Bug: #1970135
Signed-off-by: James Palmer <jpalmer7698@gmail.com>
Change-Id: Ifd64ba39a46cdbf7d75bc17baa2862790a7a39c1
2022-04-25 19:18:45 +00:00
Zuul
41e315d63a Merge "Prohibit volume manage to an encrypted volume type" 2022-03-21 20:07:22 +00:00
Yadiel Xuan(轩艳东)
8088dc9580 Prohibit volume manage to an encrypted volume type
Managing a volume to an encrypted volume type should not be allowed.
One reason is that there is no way for an operator to specify an
encryption key ID for the volume. Another is that we already don't
allow a volume of an encrypted type to be un-managed, so this change
will be symmetric.

Also update and correct the api-ref for this call.

Co-authored-by: Yadiel Xuan(轩艳东) <xuanyandong@inspur.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Change-Id: Ic2da41f3962c1108f974aca952bce3da6d6ac277
Closes-bug: #1944577
2022-03-21 11:48:01 -04:00
Zuul
fb9e391476 Merge "api-ref: include links on backups list and details" 2022-03-18 20:02:21 +00:00
Yikun Jiang
d69e89ea3b Support volume re-image
This patch adds volume re-image API to enable the ability to
re-image a specific volume.

Implements: blueprint add-volume-re-image-api

Co-Authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>

Change-Id: I031aae50ee82198648f46c503bba04c6e231bbe5
2022-02-24 15:23:38 +05:30
Zuul
5b89039e9e Merge "Remove the need for project_id from API endpoints" 2022-02-23 15:40:46 +00:00
Alan Bishop
31b34e91e0 Remove the need for project_id from API endpoints
Inclusion of a project_id in API URLs is now optional, and no longer
required. Removing the project_id requirement facilitates supporting
Secure RBAC's notion of system scope, in which an API method is not
associated with a specific project.

The API v3 routing is enhanced to provide duplicate routes for API
methods that traditionally required a project_id in the URL:
- The existing route for which a project_id is in the URL
- A new route for when the URL does not include a project_id

To test both routes and ensure there are no regresssions, the "API
samples" functional tests include a project_id in the URLs, and the
rest of the functional tests do not include the project_id. This is
handled by changing the 'noauth' WSGI middleware to no longer add the
project_id, and adding a new 'noauth_include_project_id' middleware
filter that implements the legacy behavior.

A new microversion V3.67 is introduced, but it only serves to inform
clients whether the project_id is optional or required. When an API
node supports mv 3.67, the project_id is optional in all API requests,
even when the request specifies a earlier microversion. See the spec
Ia44f199243be8f862520d7923007e7182b32f67d for more details on this
behavior.

Note: Much of the groundwork for this is based on manila's patch
I5127e150e8a71e621890f30dba6720b3932cf583.

DocImpact
APIImpact

Implements: blueprint project-id-optional-in-urls
Change-Id: I3729cbe1902ab4dc335451d13ed921ec236fb8fd
2022-02-08 08:32:23 -08:00
Lucas Oliveira
1f67b918ac api-ref: include links on backups list and details
This patch adds backups_links optional on parameters.yaml and
include as response parameter on v3 backups list(/backups)
and details(/backups/detail).

Closes-Bug: 1944463
Change-Id: I29d2fb4b33f508638d703659a7717edf2891527d
2022-01-24 11:43:29 -03:00
Zuul
938463e36c Merge "Reject bad img formats for uploaded encrypted vols" 2022-01-11 17:19:38 +00:00
whoami-rajat
104e3e049d Add default types info in admin docs
This patch adds information about default volume types i.e. project
default and deployment default (set via default_volume_type in cinder.conf)
and also regarding __DEFAULT__ type.

Change-Id: I47151b621003aea04082d38bd1d5cc911afab0f4
2021-11-19 05:13:46 -05:00
whoami-rajat
ba28f29254 api-ref: Add info regarding default types
This patch adds information regarding the default volume types
(including __DEFAULT__) in the api-ref.
The sections modified are volume create, volume type list, show
default volume type and delete volume type.

Change-Id: I896c809bd4ff09e2f26398a1cdaa9a91dc0f8638
2021-11-18 04:19:51 -05:00
Eric Harney
9ed7c16ef0 Snapshot in-use volumes without force flag
Introduces API microversion 3.66, which allows
snapshot creation on in-use volumes without the
force flag being passed.

Co-authored-by: Eric Harney <eharney@redhat.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Implements: blueprint fix-snapshot-create-force
Change-Id: I6d45aeab065197a85ce62740fc95306bce9dfc45
2021-08-31 14:48:32 -04:00
Zuul
d5f22c6652 Merge "Expose volume & snapshot use_quota field" 2021-08-29 08:39:44 +00:00
Zuul
244a75fd6b Merge "Correct the rest api url of import_record of backup" 2021-08-28 15:56:40 +00:00
xuanyandong
400008194b Correct the rest api url of import_record of backup
Change-Id: I16cdd1bc19bf51b6012d84a7163fa5d2a710800a
2021-08-28 02:42:14 +00:00
Gorka Eguileor
ec44fc8999 Expose volume & snapshot use_quota field
This patch adds a new REST API microversion to expose the new use_quota
DB field present in volumes and snapshots.

The field will be reported when listing and showing resources and can be
used when filtering as well.

The field is exposed in the REST API as `consumes_quota` to prevent
confusion for users and admins, since exposing it as `use_quota` may
give them the wrong impression that they can set it up for their own
purposes.

For users we say what is happening with the quota for that resource -it
consumes or doesn't consume quota-, whereas internally we express
instruction to the core code -whether to use quota or not-, hence the
difference in the naming.

APIImpact
DocImpact
Implements: blueprint temp-resources
Change-Id: I655a47fc75ddc11caf1defe984d9a66a9ad5a2e7
2021-08-26 19:04:12 +02:00
Gorka Eguileor
d29f12117e Update snapshot api microversion docs and tests
Our docs and tests are not up to date with the changes introduced to the
snapshot details introduced by the different microversion.

This patch updates the docs, samples, and tests.

Change-Id: I46fa6e1f7a5d6ad0147f17c2bafbf4fb8fa89eb8
2021-08-25 15:59:56 +02:00
Zuul
3bb14f2a9c Merge "Doc: api-ref doc update" 2021-08-20 17:18:06 +00:00
Stephan Pampel
7b9143fae9 Doc: Remove backup_id from backup import url and parameter
In the documentation for importing backups the backup_id is removed
because the backup_id is generated after the upload and the user
does not need to provide it. Therefore the request parameter
backup_id has also been removed.

Closes-Bug: #1930526
Change-Id: I49e0adea8fbd97d49f4386b8240320ea2baf3661
2021-08-09 12:03:39 +02:00
Zuul
4896811eab Merge "Update volume api microversion doc and tests" 2021-08-03 17:19:57 +00:00
Zuul
776d56ff69 Merge "Update Block Storage API v2 api-ref" 2021-07-22 05:18:17 +00:00
Zuul
196a58dae1 Merge "Remove Block Storage API v2" 2021-07-22 05:18:14 +00:00
tushargite96
98de5e08bf Doc: api-ref doc update
Volume multi-attach support was added in cinder and
nova in queens release.

Change-Id: I485e87b9a80bb4acec7f9bfde188e601663becb7
2021-07-19 14:13:42 +05:30
Brian Rosmaita
de8b3b0b00 Reject bad img formats for uploaded encrypted vols
Cinder only supports uploading volumes of encrypted volume types as
images with disk format 'raw' and container format 'bare'.  Screen
for this at the REST API layer when the request is made.

Change-Id: Ibb77b8b1be6c35c5db3b07fdc4056afd51d48782
Closes-bug: #1935688
2021-07-09 14:51:51 -04:00
Zuul
7a854c11ed Merge "Fix: Schema validation for attachment create API" 2021-06-17 19:21:57 +00:00
Rajat Dhasmana
560318c82e Fix: Schema validation for attachment create API
Currently the schema validation for attachment create assumes that
instance UUID will always be present in the request but that is
not the case when glance calls cinder for attachment.
Also there isn't any schema validation for MV3.54 which accepts
attachment mode in the request hence failing requests passing mode.
This patch removes the instance_uuid from required parameters and
adds schema validation for MV3.54.

Change-Id: I5108fd51effa4d72581654ed450d191a13e0e964
2021-06-16 11:26:21 -04:00
Brian Rosmaita
dbf78b8c28 Update Block Storage API v2 api-ref
Add a note that the v2 API was removed in Xena.

Change-Id: I3dd81d81995913e30addf5bf4ee8a4406c36632f
2021-06-04 18:18:07 -04:00
Brian Rosmaita
e05b261af7 Remove Block Storage API v2
In this patch:
- adjusted VersionsController to return only v3
- removed cinder.api.v2.router
- adjustments to cinder.tests.unit.api.contrib to use /v3 only
- moved cinder.api.v2.snapshot_metadata (and tests) to cinder.api.v3
- moved cinder.api.v2.types (and view, tests) to cinder.api.v3
- updated versions response in api-ref
- removed unnecessary config option
- updated various sample config files
- removed experimental tempest-cinder-v2-api job
- updated some docs
- updated non-voting rally job config

Some cinder.api.v2 modules are left because the v3 classes depend on
them, but with the v2 router removed, these are unreachable via the
/v2 path.

Depends-on: https://review.opendev.org/c/openstack/rally-openstack/+/794891
(changes rally to use Block Storage API v3)
Depends-on: https://review.opendev.org/c/openstack/requirements/+/794894
(corrects regression in upper-constraint on Sphinx)

Change-Id: I2093d77db9beec7543c7524d2cd273e79dd5fd5d
2021-06-04 17:21:28 -04:00
Gorka Eguileor
dab5df1747 Update volume api microversion doc and tests
We have many microversions but we but we haven't kept the samples, docs,
and tests up to date with them.

Of the many microversions this patch will focus on the ones that have
updated the fields returned for volumes updating the docs, adding the
appropriate json samples, updating the tests, and updating the
contributor docs to highlight the need to keep these up to date.

Some convenience methods have been added to facilitate the testing:

- use_versions: class decorator that allows us to run the same tests
  with different microversions, each using their respective json and
  templates.

- override_mv: method decorator to change the microversion in a single
  test.

- common_api_sample: context manager to use the base sample instead of a
  microversion specific one.

Change-Id: Ide303685a4cd1477c319b36e67a3b05c8d129126
2021-05-31 12:41:46 +02:00
Gorka Eguileor
da2fbff247 Fix functional jobs
In Change-ID I8db34f31d35375dc9656d53ac7af644c28a3d3fe we updated
api-ref samples, but functional tests didn't run so we didn't detect
that those samples changes broke our functional tests.

This patch fixes the functional tests, updates remaining cases of
key_size, and enables functional tests for api-ref changes to prevent
this from happening again.

Change-Id: I2a1f3121458c2db5e389dbcf31b34d412ffcd561
2021-05-25 16:55:05 +02:00
Eric Harney
e8c7910453 api-ref: Use key_size 256 in API examples
As the api-ref doc notes, the key size is usually
256.  Use 256 in the examples instead of 128 to
make things more clear.

Change-Id: I8db34f31d35375dc9656d53ac7af644c28a3d3fe
2021-05-12 11:04:06 -04:00
Zuul
12aa750114 Merge "api-ref: Fix "name" parameter" 2021-04-16 22:19:38 +00:00
Eric Harney
70912ef15e api-ref: Fix "name" parameter
Prior to this change, the "name" field in
the response for "Update a snapshot's metadata"
referred to volume transfers instead of snapshots.

Make "name" a generic parameter in case it is reused
in other APIs, and make specific parameters for
snapshots and transfers.

(Similar to I0b0a692139)

Change-Id: Ifb93bd1b20c216fd29ea90c4ac70709bee569d1d
2021-04-16 20:31:08 +00:00
Zuul
9b388e1262 Merge "api-ref: Fix "id" parameter" 2021-04-16 19:45:41 +00:00
Brian Rosmaita
a166f7bb16 api-ref: add additional info for mv 3.64
Update the 'encryption_key_id' description in the volume detail
response.

Change-Id: I560aeeec568e59b9ac60ed91f967bc0c886cc20a
2021-03-31 16:40:13 -04:00
Eric Harney
c98f88208d api-ref: Fix "id" parameter
The parameter "id" was used for volume transfer
and other various APIs, resulting in things like
consistency group snapshot APIs indicating that
they return volume transfer ids.

Make "id" generic for those cases and make a
specific "transfer_id" parameter for volume
transfers to prevent this problem in other
places.

Change-Id: I0b0a69213910c33570453cd961e10e22941369e9
2021-03-25 09:14:32 -04:00
Zuul
98ef22a16f Merge "api-ref: Don't list HTTP 500 for backup create" 2021-03-19 22:09:11 +00:00
Zuul
a41344870e Merge "Fix in api-ref" 2021-03-19 20:58:09 +00:00
Eric Harney
8bb3f55dbd api-ref: Don't list HTTP 500 for backup create
HTTP 500 just means that something broke in
the service.  We don't need to list this as part
of our API because we don't intend to deliver
it as a response for this API.

Change-Id: I7a08926fbbc5d2753f7d9d0c6c496ad3f967ec10
2021-03-09 10:51:01 -05:00
Zuul
45217bfd9e Merge "Add encryption_key_id to volume and backup details" 2021-03-04 15:10:38 +00:00
tushargite96
2c99acd1f5 Fix in api-ref
Add a separate var for snapshot and use in:
https://github.com/openstack/cinder/blob/master/api-ref/source/v3/volumes-v3-snapshots.inc

Change-Id: Ib6a153562aec1c73ceb7ad9d4f6c4ee104d768e2
2021-02-24 15:31:43 +00:00
Zuul
6bc06f1dbd Merge "Remove NestedQuotaDriver" 2021-02-16 16:26:03 +00:00
Alan Bishop
f91aec5869 Add encryption_key_id to volume and backup details
Add microversion 3.64 for including the encryption_key_id in the
volume and backup details when the associated volume is encrypted.
This facilitates associating encryption keys (typically stored in
Barbican) with the volume or backup that uses it.

The encryption_key_id is included in the details only when the
associated volume is encrypted, and it isn't using the all-zeros
key ID used by the legacy fixed-key ConfKeyMgr.

APIImpact
DocImpact

Implements: blueprint include-encryption-key-id-in-details
Change-Id: I16f54e6722cdbcbad4af1eb0d30264b0039412fd
2021-02-04 07:23:08 -08:00
Zuul
1ab00fd2c5 Merge "Introduces MV to add volume type ID in volumes details" 2021-02-03 18:32:51 +00:00
Rajat Dhasmana
aa77afe310 Remove NestedQuotaDriver
NestedQuotaDriver was marked as deprecated in Train release and
was expect to be removed in Ussuri.
This patch removes the same in Wallaby release.

Change-Id: Ide2d53caf1bc5e3ba49f34b2f48de31abaf655d0
2021-01-19 17:43:29 +00:00