openstack/cinder
Code Issues Proposed changes

Support volume re-image

Browse Source 
This patch adds volume re-image API to enable the ability to
re-image a specific volume.

Implements: blueprint add-volume-re-image-api

Co-Authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>

Change-Id: I031aae50ee82198648f46c503bba04c6e231bbe5
This commit is contained in:
Yikun Jiang 2018-10-11 21:08:57 +08:00 committed by whoami-rajat
parent 55ea01c1d0
commit d69e89ea3b
19 changed files with 493 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
api-ref/source/v3/parameters.yaml
View File

@ -2134,6 +2134,13 @@ os-migrate_volume_completion:
in: body
required: true
type: object
os-reimage:
description: |
The ``os-reimage`` action.
in: body
required: true
type: object
min_version: 3.68
os-reserve:
description: |
The ``os-reserve`` action.
@ -2479,6 +2486,15 @@ reference:
in: body
required: true
type: object
reimage_reserved:
description: |
Normally, volumes to be re-imaged are in ``available`` or ``error`` status.
When ``true``, this parameter will allow a volume in the ``reserved`` status
to be re-imaged. The ability to re-image a volume in ``reserved`` status
may be restricted to administrators in some clouds. Default value is ``false``.
in: body
required: false
type: boolean
remove_project_access:
description: |
Removes volume type access from a project.

4
api-ref/source/v3/samples/versions/version-show-response.json
View File

@ -21,8 +21,8 @@
],
"min_version": "3.0",
"status": "CURRENT",
"updated": "2021-12-16T00:00:00Z",
"version": "3.67"
"updated": "2022-03-30T00:00:00Z",
"version": "3.68"
}
]
}

4
api-ref/source/v3/samples/versions/versions-response.json
View File

@ -21,8 +21,8 @@
],
"min_version": "3.0",
"status": "CURRENT",
"updated": "2021-12-16T00:00:00Z",
"version": "3.67"
"updated": "2022-03-30T00:00:00Z",
"version": "3.68"
}
]
}

6
api-ref/source/v3/samples/volume-os-reimage-request.json Normal file
View File

@ -0,0 +1,6 @@
{
"os-reimage": {
"image_id": "71543ced-a8af-45b6-a5c4-a46282108a90",
"reimage_reserved": false
}
}

41
api-ref/source/v3/volumes-v3-volumes-actions.inc
View File

@ -973,3 +973,44 @@ Request Example
.. literalinclude:: ./samples/volume-readonly-update-request.json
:language: javascript
Reimage a volume
~~~~~~~~~~~~~~~~
.. rest_method:: POST /v3/{project_id}/volumes/{volume_id}/action
Re-image a volume with a specific image. Specify the ``os-reimage`` action
in the request body.
A volume in ``available`` or ``error`` status can be re-imaged directly. To
re-image a volume in ``reserved`` status, you must include the
``reimage_reserved`` parameter set to ``true``.
.. note:: Image signature verification is currently unsupported when
re-imaging a volume.
Response codes
--------------
.. rest_status_code:: success ../status.yaml
- 202
Request
-------
.. rest_parameters:: parameters.yaml
- project_id: project_id_path
- volume_id: volume_id_path
- image_id: image_id
- reimage_reserved: reimage_reserved
- os-reimage: os-reimage
Request Example
---------------
.. literalinclude:: ./samples/volume-os-reimage-request.json
:language: javascript

20
cinder/api/contrib/volume_actions.py
View File

@ -326,6 +326,26 @@ class VolumeActionsController(wsgi.Controller):
self.volume_api.update(context, volume, update_dict)
@wsgi.Controller.api_version(mv.SUPPORT_REIMAGE_VOLUME)
@wsgi.response(HTTPStatus.ACCEPTED)
@wsgi.action('os-reimage')
@validation.schema(volume_action.reimage, mv.SUPPORT_REIMAGE_VOLUME)
def _reimage(self, req, id, body):
"""Re-image a volume with specific image."""
context = req.environ['cinder.context']
# Not found exception will be handled at the wsgi level
volume = self.volume_api.get(context, id)
params = body['os-reimage']
reimage_reserved = params.get('reimage_reserved', 'False')
reimage_reserved = strutils.bool_from_string(reimage_reserved,
strict=True)
image_id = params['image_id']
try:
self.volume_api.reimage(context, volume, image_id,
reimage_reserved)
except exception.InvalidVolume as error:
raise webob.exc.HTTPBadRequest(explanation=error.msg)
class Volume_actions(extensions.ExtensionDescriptor):
"""Enable volume actions."""

2
cinder/api/microversions.py
View File

@ -173,6 +173,8 @@ SNAPSHOT_IN_USE = '3.66'
PROJECT_ID_OPTIONAL_IN_URL = '3.67'
SUPPORT_REIMAGE_VOLUME = '3.68'
def get_mv_header(version):
"""Gets a formatted HTTP microversion header.

3
cinder/api/openstack/api_version_request.py
View File

@ -153,13 +153,14 @@ REST_API_VERSION_HISTORY = """
operation.
* 3.66 - Allow snapshotting in-use volumes without force flag.
* 3.67 - API URLs no longer need to include a project_id parameter.
* 3.68 - Support re-image volume
"""
# The minimum and maximum versions of the API supported
# The default api version request is defined to be the
# minimum version of the API supported.
_MIN_API_VERSION = "3.0"
_MAX_API_VERSION = "3.67"
_MAX_API_VERSION = "3.68"
UPDATED = "2021-11-02T00:00:00Z"

5
cinder/api/openstack/rest_api_version_history.rst
View File

@ -513,3 +513,8 @@ route: ``https://$(controller)s/volume/v3/$(project_id)s/volumes`` is
equivalent to ``https://$(controller)s/volume/v3/volumes``. When interacting
with the cinder service as system or domain scoped users, a project_id should
not be specified in the API path.
3.68
----
Support ability to re-image a volume with a specific image. Specify the
``os-reimage`` action in the request body.

17
cinder/api/schemas/volume_actions.py
View File

@ -202,3 +202,20 @@ volume_readonly_update = {
'required': ['os-update_readonly_flag'],
'additionalProperties': False,
}
reimage = {
'type': 'object',
'properties': {
'os-reimage': {
'type': 'object',
'properties': {
'image_id': parameter_types.uuid,
'reimage_reserved': parameter_types.boolean,
},
'required': ['image_id'],
'additionalProperties': False,
},
},
'required': ['os-reimage'],
'additionalProperties': False,
}

18
cinder/compute/nova.py
View File

@ -143,6 +143,11 @@ class API(base.Base):
'server_uuid': server_id,
'tag': volume_id}
def _get_volume_reimaged_event(self, server_id, volume_id):
return {'name': 'volume-reimaged',
'server_uuid': server_id,
'tag': volume_id}
def _send_events(self, context, events, api_version=None):
nova = novaclient(context, privileged_user=True,
api_version=api_version)
@ -219,3 +224,16 @@ class API(base.Base):
resource_uuid=volume_id,
detail=message_field.Detail.NOTIFY_COMPUTE_SERVICE_FAILED)
return result
def reimage_volume(self, context, server_ids, volume_id):
api_version = '2.91'
events = [self._get_volume_reimaged_event(server_id, volume_id)
for server_id in server_ids]
result = self._send_events(context, events, api_version=api_version)
if not result:
self.message_api.create(
context,
message_field.Action.REIMAGE_VOLUME,
resource_uuid=volume_id,
detail=message_field.Detail.NOTIFY_COMPUTE_SERVICE_FAILED)
return result

22
cinder/policies/volume_actions.py
View File

@ -39,6 +39,8 @@ RESERVE_POLICY = "volume_extension:volume_actions:reserve"
ROLL_DETACHING_POLICY = "volume_extension:volume_actions:roll_detaching"
TERMINATE_POLICY = "volume_extension:volume_actions:terminate_connection"
INITIALIZE_POLICY = "volume_extension:volume_actions:initialize_connection"
REIMAGE_POLICY = "volume:reimage"
REIMAGE_RESERVED_POLICY = "volume:reimage_reserved"
deprecated_extend_policy = base.CinderDeprecatedRule(
name=EXTEND_POLICY,
@ -323,6 +325,26 @@ volume_action_policies = [
],
deprecated_rule=deprecated_detach_policy,
),
policy.DocumentedRuleDefault(
name=REIMAGE_POLICY,
check_str=base.SYSTEM_ADMIN_OR_PROJECT_MEMBER,
description="Reimage a volume in 'available' or 'error' status.",
operations=[
{
'method': 'POST',
'path': '/volumes/{volume_id}/action (os-reimage)'
}
]),
policy.DocumentedRuleDefault(
name=REIMAGE_RESERVED_POLICY,
check_str=base.SYSTEM_ADMIN_OR_PROJECT_MEMBER,
description="Reimage a volume in 'reserved' status.",
operations=[
{
'method': 'POST',
'path': '/volumes/{volume_id}/action (os-reimage)'
}
]),
]

67
cinder/tests/unit/api/contrib/test_volume_actions.py
View File

@ -1552,3 +1552,70 @@ class VolumeImageActionsTest(test.TestCase):
vol_db = objects.Volume.get_by_id(self.context, volume.id)
self.assertEqual('uploading', vol_db.status)
self.assertEqual('available', vol_db.previous_status)
def _build_reimage_req(self, body, vol_id,
version=mv.SUPPORT_REIMAGE_VOLUME):
req = fakes.HTTPRequest.blank(
'/v3/%s/volumes/%s/action' % (fake.PROJECT_ID, id))
req.method = "POST"
req.body = jsonutils.dump_as_bytes(body)
req.environ['cinder.context'] = self.context
req.api_version_request = mv.get_api_version(version)
req.headers["content-type"] = "application/json"
return req
@ddt.data(None, False, True)
@mock.patch.object(volume_api.API, "reimage")
def test_volume_reimage(self, reimage_reserved, mock_image):
vol = utils.create_volume(self.context)
body = {"os-reimage": {"image_id": fake.IMAGE_ID}}
if reimage_reserved is not None:
body["os-reimage"]["reimage_reserved"] = reimage_reserved
req = self._build_reimage_req(body, vol.id)
self.controller._reimage(req, vol.id, body=body)
@mock.patch.object(volume_api.API, "reimage")
def test_volume_reimage_invaild_params(self, mock_image):
vol = utils.create_volume(self.context)
body = {"os-reimage": {"image_id": fake.IMAGE_ID,
"reimage_reserved": 'wrong'}}
req = self._build_reimage_req(body, vol)
self.assertRaises(exception.ValidationError,
self.controller._reimage, req,
vol.id, body=body)
def test_volume_reimage_before_3_68(self):
vol = utils.create_volume(self.context)
body = {"os-reimage": {"image_id": fake.IMAGE_ID}}
req = self._build_reimage_req(body, vol.id, version="3.67")
self.assertRaises(exception.VersionNotFoundForAPIMethod,
self.controller._reimage, req, vol.id, body=body)
def test_reimage_volume_invalid_status(self):
def fake_reimage_volume(*args, **kwargs):
msg = "Volume status must be available."
raise exception.InvalidVolume(reason=msg)
self.mock_object(volume.api.API, 'reimage',
fake_reimage_volume)
vol = utils.create_volume(self.context)
body = {"os-reimage": {"image_id": fake.IMAGE_ID}}
req = self._build_reimage_req(body, vol)
self.assertRaises(webob.exc.HTTPBadRequest,
self.controller._reimage, req,
vol.id, body=body)
@mock.patch('cinder.context.RequestContext.authorize')
def test_reimage_volume_attach_more_than_one_server(self, mock_authorize):
vol = utils.create_volume(self.context)
va_objs = [objects.VolumeAttachment(context=self.context, id=i)
for i in [fake.OBJECT_ID, fake.OBJECT2_ID, fake.OBJECT3_ID]]
va_list = objects.VolumeAttachmentList(context=self.context,
objects=va_objs)
vol.volume_attachment = va_list
self.mock_object(volume_api.API, 'get', return_value=vol)
body = {"os-reimage": {"image_id": fake.IMAGE_ID}}
req = self._build_reimage_req(body, vol)
self.assertRaises(webob.exc.HTTPConflict,
self.controller._reimage, req, vol.id, body=body)

9
cinder/tests/unit/volume/test_rpcapi.py
View File

@ -676,3 +676,12 @@ class VolumeRPCAPITestCase(test.RPCAPITestCase):
server=self.fake_group.host,
group=self.fake_group,
version='3.14')
def test_reimage(self):
self._test_rpc_api('reimage', rpc_method='cast',
server=self.fake_volume_obj.host,
volume=self.fake_volume_obj,
image_meta={'id': fake.IMAGE_ID,
'container_format': 'fake_type',
'disk_format': 'fake_format'},
version='3.18')

136
cinder/tests/unit/volume/test_volume_reimage.py Normal file
View File

@ -0,0 +1,136 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Tests for Volume reimage Code."""
from unittest import mock
import ddt
from oslo_concurrency import processutils
from cinder import exception
from cinder.tests.unit import fake_constants
from cinder.tests.unit.image import fake as fake_image
from cinder.tests.unit import utils as tests_utils
from cinder.tests.unit import volume as base
@ddt.ddt
class VolumeReimageTestCase(base.BaseVolumeTestCase):
def setUp(self):
super(VolumeReimageTestCase, self).setUp()
self.patch('cinder.volume.volume_utils.clear_volume', autospec=True)
fake_image.mock_image_service(self)
self.image_meta = fake_image.FakeImageService().show(
self.context, fake_constants.IMAGE_ID)
def test_volume_reimage(self):
volume = tests_utils.create_volume(self.context, status='downloading',
previous_status='available')
self.assertEqual(volume.status, 'downloading')
self.assertEqual(volume.previous_status, 'available')
self.volume.create_volume(self.context, volume)
with mock.patch.object(self.volume.driver, 'copy_image_to_volume'
) as mock_cp_img:
self.volume.reimage(self.context, volume, self.image_meta)
mock_cp_img.assert_called_once_with(self.context, volume,
fake_image.FakeImageService(),
self.image_meta['id'])
self.assertEqual(volume.status, 'available')
def test_volume_reimage_raise_exception(self):
volume = tests_utils.create_volume(self.context)
self.volume.create_volume(self.context, volume)
with mock.patch.object(self.volume.driver, 'copy_image_to_volume'
) as mock_cp_img:
mock_cp_img.side_effect = processutils.ProcessExecutionError
self.assertRaises(exception.ImageCopyFailure, self.volume.reimage,
self.context, volume, self.image_meta)
self.assertEqual(volume.previous_status, 'available')
self.assertEqual(volume.status, 'error')
mock_cp_img.side_effect = exception.ImageUnacceptable(
image_id=self.image_meta['id'], reason='')
self.assertRaises(exception.ImageUnacceptable, self.volume.reimage,
self.context, volume, self.image_meta)
mock_cp_img.side_effect = exception.ImageTooBig(
image_id=self.image_meta['id'], reason='')
self.assertRaises(exception.ImageTooBig, self.volume.reimage,
self.context, volume, self.image_meta)
mock_cp_img.side_effect = Exception
self.assertRaises(exception.ImageCopyFailure, self.volume.reimage,
self.context, volume, self.image_meta)
mock_cp_img.side_effect = exception.ImageCopyFailure(reason='')
self.assertRaises(exception.ImageCopyFailure, self.volume.reimage,
self.context, volume, self.image_meta)
@mock.patch('cinder.volume.volume_utils.check_image_metadata')
@mock.patch('cinder.volume.rpcapi.VolumeAPI.reimage')
@ddt.data('available', 'error')
def test_volume_reimage_api(self, status, mock_reimage, mock_check):
volume = tests_utils.create_volume(self.context)
volume.status = status
volume.save()
self.assertEqual(volume.status, status)
# The available or error volume can be reimage directly
self.volume_api.reimage(self.context, volume, self.image_meta['id'])
mock_check.assert_called_once_with(self.image_meta, volume.size)
mock_reimage.assert_called_once_with(self.context, volume,
self.image_meta)
@mock.patch('cinder.volume.volume_utils.check_image_metadata')
@mock.patch('cinder.volume.rpcapi.VolumeAPI.reimage')
def test_volume_reimage_api_with_reimage_reserved(self, mock_reimage,
mock_check):
volume = tests_utils.create_volume(self.context)
# The reserved volume can not be reimage directly, and only can
# be reimaged with reimage_reserved flag
volume.status = 'reserved'
volume.save()
self.assertEqual(volume.status, 'reserved')
self.volume_api.reimage(self.context, volume, self.image_meta['id'],
reimage_reserved=True)
mock_check.assert_called_once_with(self.image_meta, volume.size)
mock_reimage.assert_called_once_with(self.context, volume,
self.image_meta)
def test_volume_reimage_api_with_invaild_status(self):
volume = tests_utils.create_volume(self.context)
# The reserved volume can not be reimage directly, and only can
# be reimaged with reimage_reserved flag
volume.status = 'reserved'
volume.save()
self.assertEqual(volume.status, 'reserved')
ex = self.assertRaises(exception.InvalidVolume,
self.volume_api.reimage,
self.context, volume,
self.image_meta['id'],
reimage_reserved=False)
self.assertIn("status must be available or error",
str(ex))
# The other status volume can not be reimage
volume.status = 'in-use'
volume.save()
self.assertEqual(volume.status, 'in-use')
ex = self.assertRaises(exception.InvalidVolume,
self.volume_api.reimage,
self.context, volume, self.image_meta['id'],
reimage_reserved=True)
self.assertIn("status must be "
"available or error or reserved",
str(ex))

33
cinder/volume/api.py
View File

@ -29,6 +29,7 @@ from oslo_utils import excutils
from oslo_utils import strutils
from oslo_utils import timeutils
from oslo_utils import versionutils
import webob
from cinder.api import common
from cinder.common import constants
@ -2529,6 +2530,38 @@ class API(base.Base):
volume_utils.notify_about_volume_usage(ctxt, volume, "detach.end")
return volume.volume_attachment
def reimage(self, context, volume, image_id, reimage_reserved=False):
if volume.status in ['reserved']:
context.authorize(vol_action_policy.REIMAGE_RESERVED_POLICY,
target_obj=volume)
else:
context.authorize(vol_action_policy.REIMAGE_POLICY,
target_obj=volume)
if len(volume.volume_attachment) > 1:
msg = _("Cannot re-image a volume which is attached to more than "
"one server.")
raise webob.exc.HTTPConflict(explanation=msg)
# Build required conditions for conditional update
expected = {'status': ('available', 'error', 'reserved'
) if reimage_reserved else ('available',
'error')}
values = {'status': 'downloading',
'previous_status': volume.model.status}
result = volume.conditional_update(values, expected)
if not result:
msg = (_('Volume %(vol_id)s status must be %(statuses)s, but '
'current status is %(status)s.') %
{'vol_id': volume.id,
'statuses': utils.build_or_str(expected['status']),
'status': volume.status})
raise exception.InvalidVolume(reason=msg)
image_meta = self.image_service.show(context, image_id)
volume_utils.check_image_metadata(image_meta, volume['size'])
self.volume_rpcapi.reimage(context,
volume,
image_meta)
class HostAPI(base.Base):
"""Sub-set of the Volume Manager API for managing host operations."""

48
cinder/volume/manager.py
View File

@ -5305,3 +5305,51 @@ class VolumeManager(manager.CleanableManager,
raise exception.VolumeBackendAPIException(data=err_msg)
return {'replication_targets': replication_targets}
def _refresh_volume_glance_meta(self, context, volume, image_meta):
volume_utils.enable_bootable_flag(volume)
volume_meta = volume_utils.get_volume_image_metadata(
image_meta['id'], image_meta)
LOG.debug("Creating volume glance metadata for volume %(volume_id)s"
" backed by image %(image_id)s with: %(vol_metadata)s.",
{'volume_id': volume.id, 'image_id': image_meta['id'],
'vol_metadata': volume_meta})
self.db.volume_glance_metadata_delete_by_volume(context, volume.id)
self.db.volume_glance_metadata_bulk_create(context, volume.id,
volume_meta)
def reimage(self, context, volume, image_meta):
"""Reimage a volume with specific image."""
image_id = None
try:
image_id = image_meta['id']
image_service, _ = glance.get_remote_image_service(
context, image_meta['id'])
image_location = image_service.get_location(context, image_id)
volume_utils.copy_image_to_volume(self.driver, context, volume,
image_meta, image_location,
image_service)
self._refresh_volume_glance_meta(context, volume, image_meta)
volume.status = volume.previous_status
volume.save()
if volume.status in ['reserved']:
nova_api = compute.API()
attachments = volume.volume_attachment
instance_uuids = [attachment.instance_uuid
for attachment in attachments]
nova_api.reimage_volume(context, instance_uuids, volume.id)
LOG.debug("Re-image %(image_id)s"
" to volume %(volume_id)s successfully.",
{'image_id': image_id, 'volume_id': volume.id})
except Exception:
with excutils.save_and_reraise_exception():
LOG.error('Failed to re-image volume %(volume_id)s with '
'image %(image_id)s.',
{'image_id': image_id, 'volume_id': volume.id})
volume.previous_status = volume.status
volume.status = 'error'
volume.save()

8
cinder/volume/rpcapi.py
View File

@ -137,9 +137,10 @@ class VolumeAPI(rpc.RPCAPI):
3.15 - Add revert_to_snapshot method
3.16 - Add no_snapshots to accept_transfer method
3.17 - Make get_backup_device a cast (async)
3.18 - Add reimage method
"""
RPC_API_VERSION = '3.17'
RPC_API_VERSION = '3.18'
RPC_DEFAULT_VERSION = '3.0'
TOPIC = constants.VOLUME_TOPIC
BINARY = constants.VOLUME_BINARY
@ -533,3 +534,8 @@ class VolumeAPI(rpc.RPCAPI):
cctxt = self._get_cctxt(group.service_topic_queue, version='3.14')
return cctxt.call(ctxt, 'list_replication_targets',
group=group)
@rpc.assert_min_rpc_version('3.18')
def reimage(self, ctxt, volume, image_meta):
cctxt = self._get_cctxt(volume.service_topic_queue, version='3.18')
cctxt.cast(ctxt, 'reimage', volume=volume, image_meta=image_meta)

40
releasenotes/notes/add-volume-re-image-api-6f02dcefd4975a66.yaml Normal file
View File

@ -0,0 +1,40 @@
---
features:
- |
Add microversion 3.68 to support ability to re-image a volume with a
specific image. Specify the ``os-reimage`` action in the request body.
The 'available' and 'error' volume can be re-imaged directly, and the
'reserved' volume can only be re-imaged when the `reimage_reserved`
parameter is set to 'true'. When reimaging a volume, the volume state
will be changed to ``downloading`` first.
Note that this is a destructive action, that is, all data currently
contained in a volume is destroyed when the volume is re-imaged.
Two new policies are introduced to govern this functionality:
* ``REIMAGE_POLICY`` - users who satisfy this policy may re-image a volume
in status ``available`` or ``error``
* ``REIMAGE_RESERVED_POLICY`` - users who satisfy this policy may re-image
a volume in status ``reserved``
The default setting for both policies allow an administrator or the volume
owner to perform the associated action. See the `Policy configuration
<https://docs.openstack.org/cinder/yoga/configuration/block-storage/policy.html>`_
documentation in the `Cinder Service Configuration` guide for details.
upgrade:
- |
Two new policies are introduced to govern the volume reimage functionality
introduced with microversion 3.68:
* ``REIMAGE_POLICY`` - users who satisfy this policy may re-image a volume
in status ``available`` or ``error``
* ``REIMAGE_RESERVED_POLICY`` - users who satisfy this policy may re-image
a volume in status ``reserved``
The default setting for both policies allow an administrator or the volume
owner to perform the associated action. See the `Policy configuration
<https://docs.openstack.org/cinder/yoga/configuration/block-storage/policy.html>`_
documentation in the `Cinder Service Configuration` guide for details.