openstack/manila
Code Issues Proposed changes

Update micversion to 2.77, support share transfer between project

Browse Source 
user can create a transfer for a share. will return transfer id
and auth_key. another user can use transfer_id and auth_key to
accept this share. salt of transfer and auth_key compute crypt_hash
by sha1. then compare with crypt_hash in db.

APIImpact
DocImpact

Partially-Implements: blueprint transfer-share-between-project

Change-Id: I8facf9112a6b09e6b7aed9956c0a87fb5f1fc31f
This commit is contained in:
haixin 2022-05-30 17:56:14 +08:00
parent d5792fe218
commit b4a0fd9af0
40 changed files with 2607 additions and 23 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
api-ref/source/index.rst
View File

@ -52,6 +52,7 @@ shared file system storage resources.
.. include:: share-groups.inc .. include:: share-groups.inc
.. include:: share-group-types.inc .. include:: share-group-types.inc
.. include:: share-group-snapshots.inc .. include:: share-group-snapshots.inc
.. include:: share-transfers.inc
====================================== ======================================
Shared File Systems API (EXPERIMENTAL) Shared File Systems API (EXPERIMENTAL)

110
api-ref/source/parameters.yaml
View File

@ -159,6 +159,12 @@ snapshot_instance_id_path:
in: path in: path
required: true required: true
type: string type: string
transfer_id:
description: |
The unique identifier for a transfer.
in: path
required: true
type: string
# variables in query # variables in query
action_id: action_id:
@ -389,7 +395,7 @@ metadata_query:
name_inexact_query: name_inexact_query:
description: | description: |
The name pattern that can be used to filter shares, The name pattern that can be used to filter shares,
share snapshots, share networks or share groups. share snapshots, share networks, transfers or share groups.
in: query in: query
required: false required: false
type: string type: string
@ -463,6 +469,12 @@ resource_type:
in: query in: query
required: false required: false
type: string type: string
resource_type_query:
description: |
The type of the resource for which the transfer was created.
in: query
required: false
type: string
security_service_query: security_service_query:
description: | description: |
The security service ID to filter out share networks. The security service ID to filter out share networks.
@ -581,7 +593,7 @@ snapshot_id_query:
type: string type: string
sort_dir: sort_dir:
description: | description: |
The direction to sort a list of shares. A valid The direction to sort a list of resources. A valid
value is ``asc``, or ``desc``. value is ``asc``, or ``desc``.
in: query in: query
required: false required: false
@ -606,6 +618,15 @@ sort_key_messages:
in: query in: query
required: false required: false
type: string type: string
sort_key_transfer:
description: |
The key to sort a list of transfers. A valid value
is ``id``, ``name``, ``resource_type``, ``resource_id``,
``source_project_id``, ``destination_project_id``, ``created_at``,
``expires_at``.
in: query
required: false
type: string
source_share_group_snapshot_id_query: source_share_group_snapshot_id_query:
description: | description: |
The source share group snapshot ID to list the The source share group snapshot ID to list the
@ -639,6 +660,12 @@ with_count_query:
min_version: 2.42 min_version: 2.42
# variables in body # variables in body
accepted:
description: |
Whether the transfer has been accepted.
in: body
required: true
type: boolean
access: access:
description: | description: |
The ``access`` object. The ``access`` object.
@ -782,6 +809,12 @@ allow_access:
in: body in: body
required: true required: true
type: object type: object
auth_key:
description: |
The authentication key for the transfer.
in: body
required: true
type: string
availability_zone: availability_zone:
description: | description: |
The name of the availability zone the share exists within. The name of the availability zone the share exists within.
@ -952,6 +985,12 @@ cidr:
required: true required: true
type: string type: string
max_version: 2.50 max_version: 2.50
clear_access_rules:
description: |
Whether clear all access rules when accept share.
in: body
required: false
type: boolean
compatible: compatible:
description: | description: |
Whether the destination backend can or can't handle the share server Whether the destination backend can or can't handle the share server
@ -1045,6 +1084,12 @@ description_request:
in: body in: body
required: false required: false
type: string type: string
destination_project_id:
description: |
UUID of the destination project to accept transfer resource.
in: body
required: true
type: string
destination_share_server_id: destination_share_server_id:
description: | description: |
UUID of the share server that was created in the destination backend during UUID of the share server that was created in the destination backend during
@ -2757,6 +2802,12 @@ share_group_type_name_request:
in: body in: body
required: false required: false
type: string type: string
share_id_request:
description: |
The UUID of the share.
in: body
required: true
type: string
share_id_response: share_id_response:
description: | description: |
The UUID of the share. The UUID of the share.
@ -3593,6 +3644,61 @@ totalSnapshotGigabytesUsed:
in: body in: body
required: true required: true
type: integer type: integer
transfer:
description: |
The transfer object.
in: body
required: true
type: object
transfer_expires_at_body:
description: |
The date and time stamp when the resource transfer will expire.
After transfer expired, will be automatically deleted.
The date and time stamp format is `ISO 8601
<https://en.wikipedia.org/wiki/ISO_8601>`_:
::
CCYY-MM-DDThh:mm:ss±hh:mm
The ``±hh:mm`` value, if included, returns the time zone as an
offset from UTC.
For example, ``2016-12-31T13:14:15-05:00``.
in: body
required: true
type: string
transfer_id_in_body:
description: |
The transfer UUID.
in: body
required: true
type: string
transfer_name:
description: |
The transfer display name.
in: body
required: false
type: string
transfer_resource_id:
description: |
The UUID of the resource for the transfer.
in: body
required: true
type: string
transfer_resource_type:
description: |
The type of the resource for the transfer.
in: body
required: true
type: string
transfers:
description: |
List of transfers.
in: body
required: true
type: array
unit: unit:
description: | description: |
The time interval during which a number of API The time interval during which a number of API

6
api-ref/source/samples/share-transfer-accept-request.json Normal file
View File

@ -0,0 +1,6 @@
{
"accept": {
"auth_key": "d7ef426932068a33",
"clear_access_rules": true
}
}

6
api-ref/source/samples/share-transfer-create-request.json Normal file
View File

@ -0,0 +1,6 @@
{
"transfer": {
"share_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"name": "test_transfer"
}
}

24
api-ref/source/samples/share-transfer-create-response.json Normal file
View File

@ -0,0 +1,24 @@
{
"transfer": {
"id": "f21c72c4-2b77-445b-aa12-e8d1b44163a2",
"created_at": "2022-09-06T08:17:43.629495",
"name": "test_transfer",
"resource_type": "share",
"resource_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"auth_key": "406a2d67cdb09afe",
"source_project_id": "714198c7ac5e45a4b785de732ea4695d",
"destination_project_id": null,
"accepted": false,
"expires_at": "2022-09-06T08:22:43.629495",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/f21c72c4-2b77-445b-aa12-e8d1b44163a2"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/f21c72c4-2b77-445b-aa12-e8d1b44163a2"
}
]
}
}

23
api-ref/source/samples/share-transfer-show-response.json Normal file
View File

@ -0,0 +1,23 @@
{
"transfer": {
"id": "d2035732-d0c0-4380-a44c-f978a264ab1a",
"created_at": "2022-09-07T01:12:29.000000",
"name": "transfer1",
"resource_type": "share",
"resource_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"source_project_id": "714198c7ac5e45a4b785de732ea4695d",
"destination_project_id": null,
"accepted": false,
"expires_at": "2022-09-07T01:17:29.000000",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/d2035732-d0c0-4380-a44c-f978a264ab1a"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/d2035732-d0c0-4380-a44c-f978a264ab1a"
}
]
}
}

46
api-ref/source/samples/share-transfers-list-detailed-response.json Normal file
View File

@ -0,0 +1,46 @@
{
"transfers": [
{
"id": "42b0fab4-df77-4f25-a958-5370e1c95ed2",
"created_at": "2022-09-07T01:52:39.000000",
"name": "transfer2",
"resource_type": "share",
"resource_id": "0fe7cf64-b879-4902-9d86-f80aeff12b06",
"source_project_id": "714198c7ac5e45a4b785de732ea4695d",
"destination_project_id": null,
"accepted": false,
"expires_at": "2022-09-07T01:57:39.000000",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/42b0fab4-df77-4f25-a958-5370e1c95ed2"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/42b0fab4-df77-4f25-a958-5370e1c95ed2"
}
]
},
{
"id": "506a7e77-42e7-4f33-ac36-1d1dd7f2b9af",
"created_at": "2022-09-07T01:52:30.000000",
"name": "transfer1",
"resource_type": "share",
"resource_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"source_project_id": "714198c7ac5e45a4b785de732ea4695d",
"destination_project_id": null,
"accepted": false,
"expires_at": "2022-09-07T01:57:30.000000",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/506a7e77-42e7-4f33-ac36-1d1dd7f2b9af"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/506a7e77-42e7-4f33-ac36-1d1dd7f2b9af"
}
]
}
]
}

36
api-ref/source/samples/share-transfers-list-response.json Normal file
View File

@ -0,0 +1,36 @@
{
"transfers": [
{
"id": "02a948b4-671b-4c62-b13a-18d613cb4576",
"resource_type": "share",
"resource_id": "0fe7cf64-b879-4902-9d86-f80aeff12b06",
"name": "transfer2",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/02a948b4-671b-4c62-b13a-18d613cb4576"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/02a948b4-671b-4c62-b13a-18d613cb4576"
}
]
},
{
"id": "a10209ff-b55d-4fed-9f63-abea53b6f107",
"resource_type": "share",
"resource_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"name": "transfer1",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/a10209ff-b55d-4fed-9f63-abea53b6f107"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/a10209ff-b55d-4fed-9f63-abea53b6f107"
}
]
}
]
}

286
api-ref/source/share-transfers.inc Normal file
View File

@ -0,0 +1,286 @@
.. -*- rst -*-
Share transfer (since API v2.77)
================================
Transfers a share across projects.
Create a share transfer
~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: POST /v2/share-transfers
Initiates a share transfer from a source project namespace to a destination
project namespace.
**Preconditions**
* The share ``status`` must be ``available``
* If the share has snapshots, those snapshots must be ``available``
* The share can not belong to share group
Response codes
--------------
.. rest_status_code:: success status.yaml
- 202
.. rest_status_code:: error status.yaml
- 400
- 403
- 404
Request
-------
.. rest_parameters:: parameters.yaml
- transfer: transfer
- name: transfer_name
- share_id: share_id_request
Request Example
---------------
.. literalinclude:: ./samples/share-transfer-create-request.json
:language: javascript
Response Parameters
-------------------
.. rest_parameters:: parameters.yaml
- id: transfer_id_in_body
- created_at: created_at
- name: transfer_name
- resource_type: transfer_resource_type
- resource_id: transfer_resource_id
- auth_key: auth_key
- source_project_id: project_id
- destination_project_id: destination_project_id
- accepted: accepted
- expires_at: transfer_expires_at_body
- links: links
Response Example
----------------
.. literalinclude:: ./samples/share-transfer-create-response.json
:language: javascript
Accept a share transfer in the destination project namespace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: POST /v2/share-transfers/{transfer_id}/accept
Accepts a share transfer.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 202
.. rest_status_code:: error status.yaml
- 400
- 403
- 404
- 413
Request
-------
.. rest_parameters:: parameters.yaml
- transfer_id: transfer_id
- auth_key: auth_key
- clear_access_rules: clear_access_rules
Request Example
---------------
.. literalinclude:: ./samples/share-transfer-accept-request.json
:language: javascript
List share transfers for a project
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: GET /v2/share-transfers
Lists share transfers.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 200
Request
-------
.. rest_parameters:: parameters.yaml
- all_tenants: all_tenants_query
- limit: limit_query
- offset: offset
- sort_key: sort_key_transfer
- sort_dir: sort_dir
- name: name_query
- name~: name_inexact_query
- resource_type: resource_type_query
Response Parameters
-------------------
.. rest_parameters:: parameters.yaml
- transfers: transfers
- id: transfer_id_in_body
- resource_type: transfer_resource_type
- resource_id: transfer_resource_id
- name: transfer_name
- links: links
Response Example
----------------
.. literalinclude:: ./samples/share-transfers-list-response.json
:language: javascript
List share transfers and details
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: GET /v2/share-transfers/detail
Lists share transfers, with details.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 200
Request
-------
.. rest_parameters:: parameters.yaml
- all_tenants: all_tenants_query
- limit: limit_query
- offset: offset
- sort_key: sort_key_transfer
- sort_dir: sort_dir
Response Parameters
-------------------
.. rest_parameters:: parameters.yaml
- transfers: transfers
- id: transfer_id_in_body
- created_at: created_at
- name: transfer_name
- resource_type: transfer_resource_type
- resource_id: transfer_resource_id
- source_project_id: project_id
- destination_project_id: destination_project_id
- accepted: accepted
- expires_at: transfer_expires_at_body
- links: links
Response Example
----------------
.. literalinclude:: ./samples/share-transfers-list-detailed-response.json
:language: javascript
Show share transfer detail
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: GET /v2/share-transfers/{transfer_id}
Shows details for a share transfer.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 200
.. rest_status_code:: error status.yaml
- 404
Request
-------
.. rest_parameters:: parameters.yaml
- transfer_id: transfer_id
Response Parameters
-------------------
.. rest_parameters:: parameters.yaml
- id: transfer_id_in_body
- created_at: created_at
- name: transfer_name
- resource_type: transfer_resource_type
- resource_id: transfer_resource_id
- source_project_id: project_id
- destination_project_id: destination_project_id
- accepted: accepted
- expires_at: transfer_expires_at_body
- links: links
Response Example
----------------
.. literalinclude:: ./samples/share-transfer-show-response.json
:language: javascript
Delete a share transfer
~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: DELETE /v2/share-transfers/{transfer_id}
Deletes a share transfer.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 202
Request
-------
.. rest_parameters:: parameters.yaml
- transfer_id: transfer_id

3
api-ref/source/shares.inc
View File

@ -83,6 +83,9 @@ A share has one of these status values:
+----------------------------------------+--------------------------------------------------------+ +----------------------------------------+--------------------------------------------------------+
| ``reverting_error`` | Share revert to snapshot failed. | | ``reverting_error`` | Share revert to snapshot failed. |
+----------------------------------------+--------------------------------------------------------+ +----------------------------------------+--------------------------------------------------------+
| ``awaiting_transfer`` | Share is being transferred to a different project's |
| | namespace. |
+----------------------------------------+--------------------------------------------------------+
List shares List shares

3
manila/api/openstack/api_version_request.py
View File

@ -193,6 +193,7 @@ REST_API_VERSION_HISTORY = """
* 2.75 - Added option to specify quiesce wait time in share replica * 2.75 - Added option to specify quiesce wait time in share replica
promote API. promote API.
* 2.76 - Added 'default_ad_site' field in security service object. * 2.76 - Added 'default_ad_site' field in security service object.
* 2.77 - Added support for share transfer between different projects.
""" """
@ -200,7 +201,7 @@ REST_API_VERSION_HISTORY = """
# The default api version request is defined to be the # The default api version request is defined to be the
# minimum version of the API supported. # minimum version of the API supported.
_MIN_API_VERSION = "2.0" _MIN_API_VERSION = "2.0"
_MAX_API_VERSION = "2.76" _MAX_API_VERSION = "2.77"
DEFAULT_API_VERSION = _MIN_API_VERSION DEFAULT_API_VERSION = _MIN_API_VERSION

4
manila/api/openstack/rest_api_version_history.rst
View File

@ -418,3 +418,7 @@ ____
2.76 2.76
---- ----
Added 'default_ad_site' field in security service object. Added 'default_ad_site' field in security service object.
2.77
----
Added support for share transfer between different projects.

8
manila/api/v2/router.py
View File

@ -51,6 +51,7 @@ from manila.api.v2 import share_snapshot_export_locations
from manila.api.v2 import share_snapshot_instance_export_locations from manila.api.v2 import share_snapshot_instance_export_locations
from manila.api.v2 import share_snapshot_instances from manila.api.v2 import share_snapshot_instances
from manila.api.v2 import share_snapshots from manila.api.v2 import share_snapshots
from manila.api.v2 import share_transfer
from manila.api.v2 import share_types from manila.api.v2 import share_types
from manila.api.v2 import shares from manila.api.v2 import shares
from manila.api import versions from manila.api import versions
@ -544,6 +545,13 @@ class APIRouter(manila.api.openstack.APIRouter):
collection={'detail': 'GET'}, collection={'detail': 'GET'},
member={'action': 'POST'}) member={'action': 'POST'})
self.resources['share_transfers'] = (
share_transfer.create_resource())
mapper.resource("share-transfer", "share-transfers",
controller=self.resources['share_transfers'],
collection={'detail': 'GET'},
member={'accept': 'POST'})
self.resources["share-replica-export-locations"] = ( self.resources["share-replica-export-locations"] = (
share_replica_export_locations.create_resource()) share_replica_export_locations.create_resource())
for path_prefix in ['/{project_id}', '']: for path_prefix in ['/{project_id}', '']:

201
manila/api/v2/share_transfer.py Normal file
View File

@ -0,0 +1,201 @@
# Copyright (c) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The share transfer api."""
from http import client as http_client
from oslo_log import log as logging
from oslo_utils import strutils
from oslo_utils import uuidutils
import webob
from webob import exc
from manila.api import common
from manila.api.openstack import wsgi
from manila.api.views import transfers as transfer_view
from manila import exception
from manila.i18n import _
from manila.transfer import api as transfer_api
LOG = logging.getLogger(__name__)
SHARE_TRANSFER_VERSION = "2.77"
class ShareTransferController(wsgi.Controller):
"""The Share Transfer API controller for the OpenStack API."""
resource_name = 'share_transfer'
_view_builder_class = transfer_view.ViewBuilder
def __init__(self):
self.transfer_api = transfer_api.API()
super(ShareTransferController, self).__init__()
@wsgi.Controller.authorize('get')
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
def show(self, req, id):
"""Return data about active transfers."""
context = req.environ['manila.context']
# Not found exception will be handled at the wsgi level
transfer = self.transfer_api.get(context, transfer_id=id)
return self._view_builder.detail(req, transfer)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
def index(self, req):
"""Returns a summary list of transfers."""
return self._get_transfers(req, is_detail=False)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
def detail(self, req):
"""Returns a detailed list of transfers."""
return self._get_transfers(req, is_detail=True)
@wsgi.Controller.authorize('get_all')
def _get_transfers(self, req, is_detail):
"""Returns a list of transfers, transformed through view builder."""
context = req.environ['manila.context']
params = req.params.copy()
pagination_params = common.get_pagination_params(req)
limit, offset = [pagination_params.pop('limit', None),
pagination_params.pop('offset', None)]
sort_key, sort_dir = common.get_sort_params(params)
filters = params
key_map = {'name': 'display_name', 'name~': 'display_name~'}
for k in key_map:
if k in filters:
filters[key_map[k]] = filters.pop(k)
LOG.debug('Listing share transfers.')
transfers = self.transfer_api.get_all(context,
limit=limit,
sort_key=sort_key,
sort_dir=sort_dir,
filters=filters,
offset=offset)
if is_detail:
transfers = self._view_builder.detail_list(req, transfers)
else:
transfers = self._view_builder.summary_list(req, transfers)
return transfers
@wsgi.response(http_client.ACCEPTED)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
@wsgi.Controller.authorize('create')
def create(self, req, body):
"""Create a new share transfer."""
LOG.debug('Creating new share transfer %s', body)
context = req.environ['manila.context']
if not self.is_valid_body(body, 'transfer'):
msg = _("'transfer' is missing from the request body.")
raise exc.HTTPBadRequest(explanation=msg)
transfer = body.get('transfer', {})
share_id = transfer.get('share_id')
if not share_id:
msg = _("Must supply 'share_id' attribute.")
raise exc.HTTPBadRequest(explanation=msg)
if not uuidutils.is_uuid_like(share_id):
msg = _("The 'share_id' attribute must be a uuid.")
raise exc.HTTPBadRequest(explanation=msg)
transfer_name = transfer.get('name')
if transfer_name is not None:
transfer_name = transfer_name.strip()
LOG.debug("Creating transfer of share %s", share_id)
try:
new_transfer = self.transfer_api.create(context, share_id,
transfer_name)
except exception.Invalid as error:
raise exc.HTTPBadRequest(explanation=error.msg)
transfer = self._view_builder.create(req,
dict(new_transfer))
return transfer
@wsgi.response(http_client.ACCEPTED)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
@wsgi.Controller.authorize('accept')
def accept(self, req, id, body):
"""Accept a new share transfer."""
transfer_id = id
LOG.debug('Accepting share transfer %s', transfer_id)
context = req.environ['manila.context']
if not self.is_valid_body(body, 'accept'):
msg = _("'accept' is missing from the request body.")
raise exc.HTTPBadRequest(explanation=msg)
accept = body.get('accept', {})
auth_key = accept.get('auth_key')
if not auth_key:
msg = _("Must supply 'auth_key' while accepting a "
"share transfer.")
raise exc.HTTPBadRequest(explanation=msg)
clear_rules = accept.get('clear_access_rules', False)
if clear_rules:
try:
clear_rules = strutils.bool_from_string(clear_rules,
strict=True)
except (ValueError, TypeError):
msg = (_('Invalid boolean clear_access_rules : %(value)s') %
{'value': accept['clear_access_rules']})
raise exc.HTTPBadRequest(explanation=msg)
LOG.debug("Accepting transfer %s", transfer_id)
try:
self.transfer_api.accept(
context, transfer_id, auth_key, clear_rules=clear_rules)
except (exception.ShareSizeExceedsLimit,
exception.ShareLimitExceeded,
exception.ShareSizeExceedsAvailableQuota,
exception.ShareReplicasLimitExceeded,
exception.ShareReplicaSizeExceedsAvailableQuota,
exception.SnapshotSizeExceedsAvailableQuota,
exception.SnapshotLimitExceeded) as e:
raise exc.HTTPRequestEntityTooLarge(explanation=e.msg,
headers={'Retry-After': '0'})
except (exception.InvalidShare,
exception.InvalidSnapshot,
exception.InvalidAuthKey,
exception.TransferNotFound) as error:
raise exc.HTTPBadRequest(explanation=error.msg)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
@wsgi.Controller.authorize('delete')
def delete(self, req, id):
"""Delete a transfer."""
context = req.environ['manila.context']
LOG.debug("Delete transfer with id: %s", id)
# Not found exception will be handled at the wsgi level
self.transfer_api.delete(context, transfer_id=id)
return webob.Response(status_int=http_client.OK)
def create_resource():
return wsgi.Resource(ShareTransferController())

86
manila/api/views/transfers.py Normal file
View File

@ -0,0 +1,86 @@
# Copyright (C) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from manila.api import common
class ViewBuilder(common.ViewBuilder):
"""Model transfer API responses as a python dictionary."""
_collection_name = "share-transfer"
def __init__(self):
"""Initialize view builder."""
super(ViewBuilder, self).__init__()
def summary_list(self, request, transfers,):
"""Show a list of transfers without many details."""
return self._list_view(self.summary, request, transfers)
def detail_list(self, request, transfers):
"""Detailed view of a list of transfers ."""
return self._list_view(self.detail, request, transfers)
def summary(self, request, transfer):
"""Generic, non-detailed view of a transfer."""
return {
'transfer': {
'id': transfer['id'],
'name': transfer['display_name'],
'resource_type': transfer['resource_type'],
'resource_id': transfer['resource_id'],
'links': self._get_links(request,
transfer['id']),
},
}
def detail(self, request, transfer):
"""Detailed view of a single transfer."""
detail_body = {
'transfer': {
'id': transfer.get('id'),
'created_at': transfer.get('created_at'),
'name': transfer.get('display_name'),
'resource_type': transfer['resource_type'],
'resource_id': transfer['resource_id'],
'source_project_id': transfer['source_project_id'],
'destination_project_id': transfer.get(
'destination_project_id'),
'accepted': transfer['accepted'],
'expires_at': transfer.get('expires_at'),
'links': self._get_links(request, transfer['id']),
}
}
return detail_body
def create(self, request, transfer):
"""Detailed view of a single transfer when created."""
create_body = self.detail(request, transfer)
create_body['transfer']['auth_key'] = transfer.get('auth_key')
return create_body
def _list_view(self, func, request, transfers):
"""Provide a view for a list of transfers."""
transfers_list = [func(request, transfer)['transfer'] for transfer in
transfers]
transfers_links = self._get_collection_links(request,
transfers,
self._collection_name)
transfers_dict = dict(transfers=transfers_list)
if transfers_links:
transfers_dict['transfers_links'] = transfers_links
return transfers_dict

5
manila/common/config.py
View File

@ -132,6 +132,11 @@ global_opts = [
help='Maximum time (in seconds) to keep a share in the recycle ' help='Maximum time (in seconds) to keep a share in the recycle '
'bin, it will be deleted automatically after this amount ' 'bin, it will be deleted automatically after this amount '
'of time has elapsed.'), 'of time has elapsed.'),
cfg.IntOpt('transfer_retention_time',
default=300,
help='Maximum time (in seconds) to keep a share in '
'awaiting_transfer state, after timeout, the share will '
'automatically be rolled back to the available state'),
] ]
CONF.register_opts(global_opts) CONF.register_opts(global_opts)

4
manila/common/constants.py
View File

@ -44,6 +44,10 @@ STATUS_REPLICATION_CHANGE = 'replication_change'
STATUS_RESTORING = 'restoring' STATUS_RESTORING = 'restoring'
STATUS_REVERTING = 'reverting' STATUS_REVERTING = 'reverting'
STATUS_REVERTING_ERROR = 'reverting_error' STATUS_REVERTING_ERROR = 'reverting_error'
STATUS_AWAITING_TRANSFER = 'awaiting_transfer'
# Transfer resource type
SHARE_RESOURCE_TYPE = 'share'
# Access rule states # Access rule states
ACCESS_STATE_QUEUED_TO_APPLY = 'queued_to_apply' ACCESS_STATE_QUEUED_TO_APPLY = 'queued_to_apply'

57
manila/db/api.py
View File

@ -494,6 +494,58 @@ def share_restore(context, share_id):
################### ###################
def share_transfer_get(context, transfer_id):
"""Get a share transfer record or raise if it does not exist."""
return IMPL.share_transfer_get(context, transfer_id)
def transfer_get_all(context, limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
"""Get all share transfer records."""
return IMPL.transfer_get_all(context, limit=limit,
sort_key=sort_key, sort_dir=sort_dir,
filters=filters, offset=offset)
def transfer_get_all_by_project(context, project_id,
limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
"""Get all share transfer records for specified project."""
return IMPL.transfer_get_all_by_project(context, project_id,
limit=limit, sort_key=sort_key,
sort_dir=sort_dir,
filters=filters, offset=offset)
def transfer_create(context, values):
"""Create an entry in the transfers table."""
return IMPL.transfer_create(context, values)
def transfer_destroy(context, transfer_id, update_share_status=True):
"""Destroy a record in the share transfer table."""
return IMPL.transfer_destroy(context, transfer_id,
update_share_status=update_share_status)
def transfer_accept(context, transfer_id, user_id, project_id,
accept_snapshots=False):
"""Accept a share transfer."""
return IMPL.transfer_accept(context, transfer_id, user_id, project_id,
accept_snapshots=accept_snapshots)
def transfer_accept_rollback(context, transfer_id, user_id,
project_id, rollback_snap=False):
"""Rollback a share transfer."""
return IMPL.transfer_accept_rollback(context, transfer_id,
user_id, project_id,
rollback_snap=rollback_snap)
###################
def share_access_create(context, values): def share_access_create(context, values):
"""Allow access to share.""" """Allow access to share."""
return IMPL.share_access_create(context, values) return IMPL.share_access_create(context, values)
@ -1177,6 +1229,11 @@ def get_all_expired_shares(context):
return IMPL.get_all_expired_shares(context) return IMPL.get_all_expired_shares(context)
def get_all_expired_transfers(context):
"""Get all expired transfers DB records."""
return IMPL.get_all_expired_transfers(context)
def share_server_backend_details_set(context, share_server_id, server_details): def share_server_backend_details_set(context, share_server_id, server_details):
"""Create DB record with backend details.""" """Create DB record with backend details."""
return IMPL.share_server_backend_details_set(context, share_server_id, return IMPL.share_server_backend_details_set(context, share_server_id,

68
manila/db/migrations/alembic/versions/1e2d600bf972_add_transfers.py Normal file
View File

@ -0,0 +1,68 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""add_transfers
Revision ID: 1e2d600bf972
Revises: c476aeb186ec
Create Date: 2022-05-30 16:37:18.325464
"""
# revision identifiers, used by Alembic.
revision = '1e2d600bf972'
down_revision = 'c476aeb186ec'
from alembic import op
from oslo_log import log
import sqlalchemy as sa
LOG = log.getLogger(__name__)
def upgrade():
context = op.get_context()
mysql_dl = context.bind.dialect.name == 'mysql'
datetime_type = (sa.dialects.mysql.DATETIME(fsp=6)
if mysql_dl else sa.DateTime)
try:
op.create_table(
'transfers',
sa.Column('id', sa.String(36), primary_key=True, nullable=False),
sa.Column('created_at', datetime_type),
sa.Column('updated_at', datetime_type),
sa.Column('deleted_at', datetime_type),
sa.Column('deleted', sa.String(36), default='False'),
sa.Column('resource_id', sa.String(36), nullable=False),
sa.Column('resource_type', sa.String(255), nullable=False),
sa.Column('display_name', sa.String(255)),
sa.Column('salt', sa.String(255)),
sa.Column('crypt_hash', sa.String(255)),
sa.Column('expires_at', datetime_type),
sa.Column('source_project_id', sa.String(255), nullable=True),
sa.Column('destination_project_id', sa.String(255), nullable=True),
sa.Column('accepted', sa.Boolean, default=False),
mysql_engine='InnoDB',
mysql_charset='utf8',
)
except Exception:
LOG.error("Table |%s| not created!", 'transfers')
raise
def downgrade():
try:
op.drop_table('transfers')
except Exception:
LOG.error("transfers table not dropped")
raise

183
manila/db/sqlalchemy/api.py
View File

@ -2506,6 +2506,189 @@ def share_restore(context, share_id):
################### ###################
@context_manager.reader
def _transfer_get(context, transfer_id, resource_type='share',
session=None, read_deleted=False):
"""resource_type can be share or network(TODO network transfer)"""
query = model_query(context, models.Transfer,
session=session,
read_deleted=read_deleted).filter_by(id=transfer_id)
if not is_admin_context(context):
if resource_type == 'share':
share = models.Share
query = query.filter(models.Transfer.resource_id == share.id,
share.project_id == context.project_id)
result = query.first()
if not result:
raise exception.TransferNotFound(transfer_id=transfer_id)
return result
@context_manager.reader
def share_transfer_get(context, transfer_id, read_deleted=False):
return _transfer_get(context, transfer_id, read_deleted=read_deleted)
def _transfer_get_all(context, limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
session = get_session()
sort_key = sort_key or 'created_at'
sort_dir = sort_dir or 'desc'
with session.begin():
query = model_query(context, models.Transfer, session=session)
if filters:
legal_filter_keys = ('display_name', 'display_name~',
'id', 'resource_type', 'resource_id',
'source_project_id', 'destination_project_id')
query = exact_filter(query, models.Transfer,
filters, legal_filter_keys)
query = utils.paginate_query(query, models.Transfer, limit,
sort_key=sort_key,
sort_dir=sort_dir,
offset=offset)
return query.all()
@require_admin_context
def transfer_get_all(context, limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
return _transfer_get_all(context, limit=limit,
sort_key=sort_key, sort_dir=sort_dir,
filters=filters, offset=offset)
@require_context
def transfer_get_all_by_project(context, project_id,
limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
filters = filters.copy() if filters else {}
filters['source_project_id'] = project_id
return _transfer_get_all(context, limit=limit,
sort_key=sort_key, sort_dir=sort_dir,
filters=filters, offset=offset)
@require_context
@handle_db_data_error
def transfer_create(context, values):
if not values.get('id'):
values['id'] = uuidutils.generate_uuid()
resource_id = values['resource_id']
now_time = timeutils.utcnow()
time_delta = datetime.timedelta(
seconds=CONF.transfer_retention_time)
transfer_timeout = now_time + time_delta
values['expires_at'] = transfer_timeout
session = get_session()
with session.begin():
transfer = models.Transfer()
transfer.update(values)
transfer.save(session=session)
update = {'status': constants.STATUS_AWAITING_TRANSFER}
if values['resource_type'] == 'share':
share_update(context, resource_id, update)
return transfer
@require_context
@oslo_db_api.wrap_db_retry(max_retries=5, retry_on_deadlock=True)
def transfer_destroy(context, transfer_id,
update_share_status=True):
session = get_session()
with session.begin():
update = {'status': constants.STATUS_AVAILABLE}
transfer = share_transfer_get(context, transfer_id)
if transfer['resource_type'] == 'share':
if update_share_status:
share_update(context, transfer['resource_id'], update)
transfer_query = model_query(context, models.Transfer,
session=session).filter_by(id=transfer_id)
transfer_query.soft_delete()
@require_context
@oslo_db_api.wrap_db_retry(max_retries=5, retry_on_deadlock=True)
def transfer_accept(context, transfer_id, user_id, project_id,
accept_snapshots=False):
session = get_session()
with session.begin():
share_id = share_transfer_get(context, transfer_id)['resource_id']
update = {'status': constants.STATUS_AVAILABLE,
'user_id': user_id,
'project_id': project_id,
'updated_at': timeutils.utcnow()}
share_update(context, share_id, update)
# Update snapshots for transfer snapshots with share.
if accept_snapshots:
snapshots = share_snapshot_get_all_for_share(context, share_id)
for snapshot in snapshots:
LOG.debug('Begin to transfer snapshot: %s', snapshot['id'])
update = {'user_id': user_id,
'project_id': project_id,
'updated_at': timeutils.utcnow()}
share_snapshot_update(context, snapshot['id'], update)
query = session.query(models.Transfer).filter_by(id=transfer_id)
query.update({'deleted': True,
'deleted_at': timeutils.utcnow(),
'updated_at': timeutils.utcnow(),
'destination_project_id': project_id,
'accepted': True})
@require_context
def transfer_accept_rollback(context, transfer_id, user_id,
project_id, rollback_snap=False):
session = get_session()
with session.begin():
share_id = share_transfer_get(
context, transfer_id, read_deleted=True)['resource_id']
update = {'status': constants.STATUS_AWAITING_TRANSFER,
'user_id': user_id,
'project_id': project_id,
'updated_at': timeutils.utcnow()}
share_update(context, share_id, update)
# rollback snapshots for transfer snapshots with share.
if rollback_snap:
snapshots = share_snapshot_get_all_for_share(context, share_id)
for snapshot in snapshots:
LOG.debug('Begin to rollback snapshot: %s', snapshot['id'])
update = {'user_id': user_id,
'project_id': project_id,
'updated_at': timeutils.utcnow()}
share_snapshot_update(context, snapshot['id'], update)
query = session.query(models.Transfer).filter_by(id=transfer_id)
query.update({'deleted': 'False',
'deleted_at': None,
'updated_at': timeutils.utcnow(),
'destination_project_id': None,
'accepted': 0})
@require_admin_context
def get_all_expired_transfers(context):
session = get_session()
with session.begin():
query = model_query(context, models.Transfer, session=session)
expires_at_attr = getattr(models.Transfer, 'expires_at', None)
now_time = timeutils.utcnow()
query = query.filter(expires_at_attr.op('<=')(now_time))
result = query.all()
return result
###################
def _share_access_get_query(context, session, values, read_deleted='no'): def _share_access_get_query(context, session, values, read_deleted='no'):
"""Get access record.""" """Get access record."""
query = (model_query( query = (model_query(

20
manila/db/sqlalchemy/models.py
View File

@ -1170,6 +1170,26 @@ class ShareNetworkSecurityServiceAssociation(BASE, ManilaBase):
nullable=False) nullable=False)
class Transfer(BASE, ManilaBase):
"""Represents a share transfer request."""
__tablename__ = 'transfers'
id = Column(String(36), primary_key=True, nullable=False)
deleted = Column(String(36), default='False')
# resource type can be "share" or "share_network"
resource_type = Column(String(36), nullable=False)
# The uuid of the related resource.
resource_id = Column(String(36), nullable=False)
display_name = Column(String(255))
salt = Column(String(255))
crypt_hash = Column(String(255))
expires_at = Column(DateTime)
source_project_id = Column(String(255), nullable=True)
destination_project_id = Column(String(255), nullable=True)
accepted = Column(Boolean, default=False)
class NetworkAllocation(BASE, ManilaBase): class NetworkAllocation(BASE, ManilaBase):
"""Represents network allocation data.""" """Represents network allocation data."""
__tablename__ = 'network_allocations' __tablename__ = 'network_allocations'

16
manila/exception.py
View File

@ -329,6 +329,10 @@ class HostBinaryNotFound(NotFound):
message = _("Could not find binary %(binary)s on host %(host)s.") message = _("Could not find binary %(binary)s on host %(host)s.")
class TransferNotFound(NotFound):
message = _("Transfer %(transfer_id)s could not be found.")
class InvalidReservationExpiration(Invalid): class InvalidReservationExpiration(Invalid):
message = _("Invalid reservation expiration %(expire)s.") message = _("Invalid reservation expiration %(expire)s.")
@ -489,6 +493,10 @@ class InvalidShare(Invalid):
message = _("Invalid share: %(reason)s.") message = _("Invalid share: %(reason)s.")
class InvalidAuthKey(Invalid):
message = _("Invalid auth key: %(reason)s")
class ShareBusyException(Invalid): class ShareBusyException(Invalid):
message = _("Share is busy with an active task: %(reason)s.") message = _("Share is busy with an active task: %(reason)s.")
@ -527,6 +535,10 @@ class ShareSnapshotAccessExists(InvalidInput):
message = _("Share snapshot access %(access_type)s:%(access)s exists.") message = _("Share snapshot access %(access_type)s:%(access)s exists.")
class InvalidSnapshot(Invalid):
message = _("Invalid snapshot: %(reason)s")
class InvalidSnapshotAccess(Invalid): class InvalidSnapshotAccess(Invalid):
message = _("Invalid access rule: %(reason)s") message = _("Invalid access rule: %(reason)s")
@ -543,6 +555,10 @@ class InvalidShareAccessType(Invalid):
message = _("Invalid or unsupported share access type: %(type)s.") message = _("Invalid or unsupported share access type: %(type)s.")
class DriverCannotTransferShareWithRules(ManilaException):
message = _("Driver failed to transfer share with rules.")
class ShareBackendException(ManilaException): class ShareBackendException(ManilaException):
message = _("Share backend error: %(msg)s.") message = _("Share backend error: %(msg)s.")

6
manila/message/message_field.py
View File

@ -36,6 +36,7 @@ class Action(object):
SHRINK = ('009', _('shrink')) SHRINK = ('009', _('shrink'))
UPDATE_ACCESS_RULES = ('010', _('update access rules')) UPDATE_ACCESS_RULES = ('010', _('update access rules'))
ADD_UPDATE_SECURITY_SERVICE = ('011', _('add or update security service')) ADD_UPDATE_SECURITY_SERVICE = ('011', _('add or update security service'))
TRANSFER_ACCEPT = ('026', _('transfer accept'))
ALL = ( ALL = (
ALLOCATE_HOST, ALLOCATE_HOST,
CREATE, CREATE,
@ -48,6 +49,7 @@ class Action(object):
SHRINK, SHRINK,
UPDATE_ACCESS_RULES, UPDATE_ACCESS_RULES,
ADD_UPDATE_SECURITY_SERVICE, ADD_UPDATE_SECURITY_SERVICE,
TRANSFER_ACCEPT,
) )
@ -136,6 +138,9 @@ class Detail(object):
_("Share Driver failed to create share because a security service " _("Share Driver failed to create share because a security service "
"has not been added to the share network used. Please add a " "has not been added to the share network used. Please add a "
"security service to the share network.")) "security service to the share network."))
DRIVER_FAILED_TRANSFER_ACCEPT = (
'026',
_("Share transfer cannot be accepted without clearing access rules."))
ALL = ( ALL = (
UNKNOWN_ERROR, UNKNOWN_ERROR,
@ -163,6 +168,7 @@ class Detail(object):
SECURITY_SERVICE_FAILED_AUTH, SECURITY_SERVICE_FAILED_AUTH,
NO_DEFAULT_SHARE_TYPE, NO_DEFAULT_SHARE_TYPE,
MISSING_SECURITY_SERVICE, MISSING_SECURITY_SERVICE,
DRIVER_FAILED_TRANSFER_ACCEPT,
) )
# Exception and detail mappings # Exception and detail mappings

2
manila/policies/__init__.py
View File

@ -42,6 +42,7 @@ from manila.policies import share_snapshot
from manila.policies import share_snapshot_export_location from manila.policies import share_snapshot_export_location
from manila.policies import share_snapshot_instance from manila.policies import share_snapshot_instance
from manila.policies import share_snapshot_instance_export_location from manila.policies import share_snapshot_instance_export_location
from manila.policies import share_transfer
from manila.policies import share_type from manila.policies import share_type
from manila.policies import share_types_extra_spec from manila.policies import share_types_extra_spec
from manila.policies import shares from manila.policies import shares
@ -78,4 +79,5 @@ def list_rules():
message.list_rules(), message.list_rules(),
share_access.list_rules(), share_access.list_rules(),
share_access_metadata.list_rules(), share_access_metadata.list_rules(),
share_transfer.list_rules(),
) )

151
manila/policies/share_transfer.py Normal file
View File

@ -0,0 +1,151 @@
# Copyright (c) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo_policy import policy
from manila.policies import base
BASE_POLICY_NAME = 'share_transfer:%s'
DEPRECATED_REASON = """
The transfer API now supports system scope and default roles.
"""
deprecated_share_transfer_get_all = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'get_all',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_get_all_tenant = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'get_all_tenant',
check_str=base.RULE_ADMIN_API,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_create = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'create',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_get = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'get',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_accept = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'accept',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_delete = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'delete',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
share_transfer_policies = [
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'get_all',
check_str=base.ADMIN_OR_PROJECT_READER,
description="List share transfers.",
operations=[
{
'method': 'GET',
'path': '/share-transfers'
},
{
'method': 'GET',
'path': '/share-transfers/detail'
}
],
deprecated_rule=deprecated_share_transfer_get_all
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'get_all_tenant',
check_str=base.ADMIN,
scope_types=['project'],
description="List share transfers with all tenants.",
operations=[
{
'method': 'GET',
'path': '/share-transfers'
},
{
'method': 'GET',
'path': '/share-transfers/detail'
}
],
deprecated_rule=deprecated_share_transfer_get_all_tenant
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'create',
check_str=base.ADMIN_OR_PROJECT_MEMBER,
description="Create a share transfer.",
operations=[
{
'method': 'POST',
'path': '/share-transfers'
}
],
deprecated_rule=deprecated_share_transfer_create
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'get',
check_str=base.ADMIN_OR_PROJECT_READER,
description="Show one specified share transfer.",
operations=[
{
'method': 'GET',
'path': '/share-transfers/{transfer_id}'
}
],
deprecated_rule=deprecated_share_transfer_get
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'accept',
check_str=base.ADMIN_OR_PROJECT_MEMBER,
description="Accept a share transfer.",
operations=[
{
'method': 'POST',
'path': '/share-transfers/{transfer_id}/accept'
}
],
deprecated_rule=deprecated_share_transfer_accept
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'delete',
check_str=base.ADMIN_OR_PROJECT_MEMBER,
description="Delete share transfer.",
operations=[
{
'method': 'DELETE',
'path': '/share-transfers/{transfer_id}'
}
],
deprecated_rule=deprecated_share_transfer_delete
),
]
def list_rules():
return share_transfer_policies

41
manila/share/api.py
View File

@ -141,8 +141,9 @@ class API(base.Base):
compatible_azs_name.append(az['name']) compatible_azs_name.append(az['name'])
return compatible_azs_name, compatible_azs_multiple return compatible_azs_name, compatible_azs_multiple
def _check_if_share_quotas_exceeded(self, context, quota_exception, @staticmethod
share_size, operation='create'): def check_if_share_quotas_exceeded(context, quota_exception,
share_size, operation='create'):
overs = quota_exception.kwargs['overs'] overs = quota_exception.kwargs['overs']
usages = quota_exception.kwargs['usages'] usages = quota_exception.kwargs['usages']
quotas = quota_exception.kwargs['quotas'] quotas = quota_exception.kwargs['quotas']
@ -171,9 +172,10 @@ class API(base.Base):
'operation': operation}) 'operation': operation})
raise exception.ShareLimitExceeded(allowed=quotas['shares']) raise exception.ShareLimitExceeded(allowed=quotas['shares'])
def _check_if_replica_quotas_exceeded(self, context, quota_exception, @staticmethod
replica_size, def check_if_replica_quotas_exceeded(context, quota_exception,
resource_type='share_replica'): replica_size,
resource_type='share_replica'):
overs = quota_exception.kwargs['overs'] overs = quota_exception.kwargs['overs']
usages = quota_exception.kwargs['usages'] usages = quota_exception.kwargs['usages']
quotas = quota_exception.kwargs['quotas'] quotas = quota_exception.kwargs['quotas']
@ -291,7 +293,7 @@ class API(base.Base):
supported=CONF.enabled_share_protocols)) supported=CONF.enabled_share_protocols))
raise exception.InvalidInput(reason=msg) raise exception.InvalidInput(reason=msg)
self._check_is_share_size_within_per_share_quota_limit(context, size) self.check_is_share_size_within_per_share_quota_limit(context, size)
deltas = {'shares': 1, 'gigabytes': size} deltas = {'shares': 1, 'gigabytes': size}
share_type_attributes = self.get_share_attributes_from_share_type( share_type_attributes = self.get_share_attributes_from_share_type(
@ -306,10 +308,10 @@ class API(base.Base):
reservations = QUOTAS.reserve( reservations = QUOTAS.reserve(
context, share_type_id=share_type_id, **deltas) context, share_type_id=share_type_id, **deltas)
except exception.OverQuota as e: except exception.OverQuota as e:
self._check_if_share_quotas_exceeded(context, e, size) self.check_if_share_quotas_exceeded(context, e, size)
if share_type_supports_replication: if share_type_supports_replication:
self._check_if_replica_quotas_exceeded(context, e, size, self.check_if_replica_quotas_exceeded(context, e, size,
resource_type='share') resource_type='share')
share_group = None share_group = None
if share_group_id: if share_group_id:
@ -702,7 +704,7 @@ class API(base.Base):
share_type_id=share_type['id'] share_type_id=share_type['id']
) )
except exception.OverQuota as e: except exception.OverQuota as e:
self._check_if_replica_quotas_exceeded(context, e, share['size']) self.check_if_replica_quotas_exceeded(context, e, share['size'])
az_request_multiple_subnet_support_map = {} az_request_multiple_subnet_support_map = {}
if share_network_id: if share_network_id:
@ -1437,6 +1439,12 @@ class API(base.Base):
self.share_rpcapi.unmanage_share_server( self.share_rpcapi.unmanage_share_server(
context, share_server, force=force) context, share_server, force=force)
def transfer_accept(self, context, share, new_user,
new_project, clear_rules=False):
self.share_rpcapi.transfer_accept(context, share,
new_user, new_project,
clear_rules=clear_rules)
def create_snapshot(self, context, share, name, description, def create_snapshot(self, context, share, name, description,
force=False, metadata=None): force=False, metadata=None):
policy.check_policy(context, 'share', 'create_snapshot', share) policy.check_policy(context, 'share', 'create_snapshot', share)
@ -2390,7 +2398,8 @@ class API(base.Base):
} }
raise exception.ShareBusyException(reason=msg) raise exception.ShareBusyException(reason=msg)
def _check_is_share_size_within_per_share_quota_limit(self, context, size): @staticmethod
def check_is_share_size_within_per_share_quota_limit(context, size):
"""Raises an exception if share size above per share quota limit.""" """Raises an exception if share size above per share quota limit."""
try: try:
values = {'per_share_gigabytes': size} values = {'per_share_gigabytes': size}
@ -2442,8 +2451,8 @@ class API(base.Base):
'size': share['size']}) 'size': share['size']})
raise exception.InvalidInput(reason=msg) raise exception.InvalidInput(reason=msg)
self._check_is_share_size_within_per_share_quota_limit(context, self.check_is_share_size_within_per_share_quota_limit(context,
new_size) new_size)
# ensure we pass the share_type provisioning filter on size # ensure we pass the share_type provisioning filter on size
try: try:
@ -2479,12 +2488,12 @@ class API(base.Base):
reservations = QUOTAS.reserve(context, **deltas) reservations = QUOTAS.reserve(context, **deltas)
except exception.OverQuota as exc: except exception.OverQuota as exc:
# Check if the exceeded quota was 'gigabytes' # Check if the exceeded quota was 'gigabytes'
self._check_if_share_quotas_exceeded(context, exc, share['size'], self.check_if_share_quotas_exceeded(context, exc, share['size'],
operation='extend') operation='extend')
# NOTE(carloss): Check if the exceeded quota is # NOTE(carloss): Check if the exceeded quota is
# 'replica_gigabytes'. If so the failure could be caused due to # 'replica_gigabytes'. If so the failure could be caused due to
# lack of quotas to extend the share's replicas, then the # lack of quotas to extend the share's replicas, then the
# '_check_if_replica_quotas_exceeded' method can't be reused here # 'check_if_replica_quotas_exceeded' method can't be reused here
# since the error message must be different from the default one. # since the error message must be different from the default one.
if supports_replication: if supports_replication:
overs = exc.kwargs['overs'] overs = exc.kwargs['overs']

13
manila/share/driver.py
View File

@ -588,6 +588,19 @@ class ShareDriver(object):
""" """
raise NotImplementedError() raise NotImplementedError()
def transfer_accept(self, context, share, new_user, new_project,
access_rules=None, share_server=None):
"""Backend update project and user info if stored on the backend.
:param context: The 'context.RequestContext' object for the request.
:param share: Share instance model.
:param access_rules: A list of access rules for given share.
:param new_user: the share will be updated with the new user id .
:param new_project: the share will be updated with the new project id.
:param share_server: share server for given share.
"""
pass
def migration_get_progress( def migration_get_progress(
self, context, source_share, destination_share, source_snapshots, self, context, source_share, destination_share, source_snapshots,
snapshot_mappings, share_server=None, snapshot_mappings, share_server=None,

7
manila/share/drivers/cephfs/driver.py
View File

@ -764,6 +764,13 @@ class CephFSDriver(driver.ExecuteMixin, driver.GaneshaMixin,
def get_configured_ip_versions(self): def get_configured_ip_versions(self):
return self.protocol_helper.get_configured_ip_versions() return self.protocol_helper.get_configured_ip_versions()
def transfer_accept(self, context, share, new_user, new_project,
access_rules=None, share_server=None):
# CephFS driver cannot transfer shares by preserving access rules
same_project = share["project_id"] == new_project
if access_rules and not same_project:
raise exception.DriverCannotTransferShareWithRules()
class NativeProtocolHelper(ganesha.NASHelperBase): class NativeProtocolHelper(ganesha.NASHelperBase):
"""Helper class for native CephFS protocol""" """Helper class for native CephFS protocol"""

82
manila/share/manager.py
View File

@ -53,6 +53,7 @@ from manila.share import rpcapi as share_rpcapi
from manila.share import share_types from manila.share import share_types
from manila.share import snapshot_access from manila.share import snapshot_access
from manila.share import utils as share_utils from manila.share import utils as share_utils
from manila.transfer import api as transfer_api
from manila import utils from manila import utils
profiler = importutils.try_import('osprofiler.profiler') profiler = importutils.try_import('osprofiler.profiler')
@ -136,6 +137,11 @@ share_manager_opts = [
help='This value, specified in seconds, determines how often ' help='This value, specified in seconds, determines how often '
'the share manager will check for expired shares and ' 'the share manager will check for expired shares and '
'delete them from the Recycle bin.'), 'delete them from the Recycle bin.'),
cfg.IntOpt('check_for_expired_transfers',
default=300,
help='This value, specified in seconds, determines how often '
'the share manager will check for expired transfers and '
'destroy them and roll back share state.'),
] ]
CONF = cfg.CONF CONF = cfg.CONF
@ -243,7 +249,7 @@ def add_hooks(f):
class ShareManager(manager.SchedulerDependentManager): class ShareManager(manager.SchedulerDependentManager):
"""Manages NAS storages.""" """Manages NAS storages."""
RPC_API_VERSION = '1.24' RPC_API_VERSION = '1.25'
def __init__(self, share_driver=None, service_name=None, *args, **kwargs): def __init__(self, share_driver=None, service_name=None, *args, **kwargs):
"""Load the driver from args, or from flags.""" """Load the driver from args, or from flags."""
@ -286,6 +292,7 @@ class ShareManager(manager.SchedulerDependentManager):
self.message_api = message_api.API() self.message_api = message_api.API()
self.share_api = api.API() self.share_api = api.API()
self.transfer_api = transfer_api.API()
if CONF.profiler.enabled and profiler is not None: if CONF.profiler.enabled and profiler is not None:
self.driver = profiler.trace_cls("driver")(self.driver) self.driver = profiler.trace_cls("driver")(self.driver)
self.hooks = [] self.hooks = []
@ -3557,6 +3564,79 @@ class ShareManager(manager.SchedulerDependentManager):
LOG.info("share %s has expired, will be deleted", share['id']) LOG.info("share %s has expired, will be deleted", share['id'])
self.share_api.delete(ctxt, share) self.share_api.delete(ctxt, share)
@periodic_task.periodic_task(
spacing=CONF.check_for_expired_transfers)
def delete_expired_transfers(self, ctxt):
LOG.info("Checking for expired transfers.")
expired_transfers = self.db.get_all_expired_transfers(ctxt)
for transfer in expired_transfers:
LOG.debug("Transfer %s has expired, will be destroyed.",
transfer['id'])
self.transfer_api.delete(ctxt, transfer_id=transfer['id'])
@utils.require_driver_initialized
def transfer_accept(self, context, share_id, new_user,
new_project, clear_rules):
# need elevated context as we haven't "given" the share yet
elevated_context = context.elevated()
share_ref = self.db.share_get(elevated_context, share_id)
access_rules = self.db.share_access_get_all_for_share(
elevated_context, share_id)
share_instances = self.db.share_instances_get_all_by_share(
elevated_context, share_id)
share_server = self._get_share_server(context, share_ref)
for share_instance in share_instances:
share_instance = self.db.share_instance_get(context,
share_instance['id'],
with_share_data=True)
if clear_rules and access_rules:
try:
self.access_helper.update_access_rules(
context,
share_instance['id'],
delete_all_rules=True
)
access_rules = []
except Exception:
with excutils.save_and_reraise_exception():
msg = (
"Can not remove access rules for share "
"instance %(si)s belonging to share %(shr)s.")
msg_payload = {
'si': share_instance['id'],
'shr': share_id,
}
LOG.error(msg, msg_payload)
try:
self.driver.transfer_accept(context, share_instance,
new_user,
new_project,
access_rules=access_rules,
share_server=share_server)
except exception.DriverTransferShareWithRules as e:
with excutils.save_and_reraise_exception():
self.message_api.create(
context,
message_field.Action.TRANSFER_ACCEPT,
new_project,
resource_type=message_field.Resource.SHARE,
resource_id=share_id,
detail=(message_field.Detail.
DRIVER_FAILED_TRANSFER_ACCEPT))
msg = _("The backend failed to accept the share: %s.")
LOG.error(msg, e)
msg = ('Share %(share_id)s has transfer from %(old_project_id)s to '
'%(new_project_id)s completed successfully.')
msg_args = {
"share_id": share_id,
"old_project_id": share_ref['project_id'],
"new_project_id": context.project_id
}
LOG.info(msg, msg_args)
@add_hooks @add_hooks
@utils.require_driver_initialized @utils.require_driver_initialized
def create_snapshot(self, context, share_id, snapshot_id): def create_snapshot(self, context, share_id, snapshot_id):

15
manila/share/rpcapi.py
View File

@ -84,6 +84,7 @@ class ShareAPI(object):
1.23 - Add update_share_server_network_allocations() and 1.23 - Add update_share_server_network_allocations() and
check_update_share_server_network_allocations() check_update_share_server_network_allocations()
1.24 - Add quiesce_wait_time paramater to promote_share_replica() 1.24 - Add quiesce_wait_time paramater to promote_share_replica()
1.25 - Add transfer_accept()
""" """
BASE_RPC_API_VERSION = '1.0' BASE_RPC_API_VERSION = '1.0'
@ -92,7 +93,7 @@ class ShareAPI(object):
super(ShareAPI, self).__init__() super(ShareAPI, self).__init__()
target = messaging.Target(topic=CONF.share_topic, target = messaging.Target(topic=CONF.share_topic,
version=self.BASE_RPC_API_VERSION) version=self.BASE_RPC_API_VERSION)
self.client = rpc.get_client(target, version_cap='1.24') self.client = rpc.get_client(target, version_cap='1.25')
def create_share_instance(self, context, share_instance, host, def create_share_instance(self, context, share_instance, host,
request_spec, filter_properties, request_spec, filter_properties,
@ -311,6 +312,18 @@ class ShareAPI(object):
call_context = self.client.prepare(fanout=True, version='1.0') call_context = self.client.prepare(fanout=True, version='1.0')
call_context.cast(context, 'publish_service_capabilities') call_context.cast(context, 'publish_service_capabilities')
def transfer_accept(self, ctxt, share, new_user,
new_project, clear_rules=False):
msg_args = {
'share_id': share['id'],
'new_user': new_user,
'new_project': new_project,
'clear_rules': clear_rules
}
host = utils.extract_host(share['instance']['host'])
call_context = self.client.prepare(server=host, version='1.25')
call_context.call(ctxt, 'transfer_accept', **msg_args)
def extend_share(self, context, share, new_size, reservations): def extend_share(self, context, share, new_size, reservations):
host = utils.extract_host(share['instance']['host']) host = utils.extract_host(share['instance']['host'])
call_context = self.client.prepare(server=host, version='1.2') call_context = self.client.prepare(server=host, version='1.2')

440
manila/tests/api/v2/test_share_transfer.py Normal file
View File

@ -0,0 +1,440 @@
# Copyright (c) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import http.client as http_client
from unittest import mock
import ddt
from oslo_serialization import jsonutils
import webob
from manila.api.v2 import share_transfer
from manila import context
from manila import db
from manila import exception
from manila import quota
from manila.share import api as share_api
from manila.share import rpcapi as share_rpcapi
from manila.share import share_types
from manila import test
from manila.tests.api import fakes
from manila.tests import db_utils
from manila.transfer import api as transfer_api
SHARE_TRANSFER_VERSION = "2.77"
@ddt.ddt
class ShareTransferAPITestCase(test.TestCase):
"""Test Case for transfers V3 API."""
microversion = SHARE_TRANSFER_VERSION
def setUp(self):
super(ShareTransferAPITestCase, self).setUp()
self.share_transfer_api = transfer_api.API()
self.v2_controller = share_transfer.ShareTransferController()
self.ctxt = context.RequestContext(
'fake_user_id', 'fake_project_id', auth_token=True, is_admin=True)
def _create_transfer(self, share_id='fake_share_id',
display_name='test_transfer'):
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, display_name)
return transfer
def _create_share(self, display_name='test_share',
display_description='this is a test share',
status='available',
size=1,
project_id='fake_project_id',
user_id='fake_user_id',
share_type_id='fake_type_id',
share_network_id=None):
"""Create a share object."""
share = db_utils.create_share(display_name=display_name,
display_description=display_description,
status=status, size=size,
project_id=project_id,
user_id=user_id,
share_type_id=share_type_id,
share_network_id=share_network_id
)
share_id = share['id']
return share_id
def test_show_transfer(self):
share_id = self._create_share(size=5)
transfer = self._create_transfer(share_id)
path = '/v2/fake_project_id/share-transfers/%s' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.show(req, transfer['id'])
self.assertEqual('test_transfer', res_dict['transfer']['name'])
self.assertEqual(transfer['id'], res_dict['transfer']['id'])
self.assertEqual(share_id, res_dict['transfer']['resource_id'])
def test_list_transfers(self):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5)
transfer1 = self._create_transfer(share_id_1)
transfer2 = self._create_transfer(share_id_2)
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.index(req)
self.assertEqual(transfer1['id'], res_dict['transfers'][1]['id'])
self.assertEqual('test_transfer', res_dict['transfers'][1]['name'])
self.assertEqual(transfer2['id'], res_dict['transfers'][0]['id'])
self.assertEqual('test_transfer', res_dict['transfers'][0]['name'])
def test_list_transfers_with_all_tenants(self):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5, project_id='fake_project_id2',
user_id='fake_user_id2')
self._create_transfer(share_id_1)
self._create_transfer(share_id_2)
path = '/v2/fake_project_id/share-transfers?all_tenants=true'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = context.get_admin_context()
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.index(req)
self.assertEqual(2, len(res_dict['transfers']))
def test_list_transfers_with_limit(self):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5)
self._create_transfer(share_id_1)
self._create_transfer(share_id_2)
path = '/v2/fake_project_id/share-transfers?limit=1'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.index(req)
self.assertEqual(1, len(res_dict['transfers']))
@ddt.data("desc", "asc")
def test_list_transfers_with_sort(self, sort_dir):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5)
transfer1 = self._create_transfer(share_id_1)
transfer2 = self._create_transfer(share_id_2)
path = \
'/v2/fake_project_id/share-transfers?sort_key=id&sort_dir=%s' % (
sort_dir)
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.index(req)
self.assertEqual(2, len(res_dict['transfers']))
order_ids = sorted([transfer1['id'],
transfer2['id']])
expect_result = order_ids[1] if sort_dir == "desc" else order_ids[0]
self.assertEqual(expect_result,
res_dict['transfers'][0]['id'])
def test_list_transfers_detail(self):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5)
transfer1 = self._create_transfer(share_id_1)
transfer2 = self._create_transfer(share_id_2)
path = '/v2/fake_project_id/share-transfers/detail'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
req.headers['Accept'] = 'application/json'
res_dict = self.v2_controller.detail(req)
self.assertEqual('test_transfer',
res_dict['transfers'][1]['name'])
self.assertEqual(transfer1['id'], res_dict['transfers'][1]['id'])
self.assertEqual(share_id_1, res_dict['transfers'][1]['resource_id'])
self.assertEqual('test_transfer',
res_dict['transfers'][0]['name'])
self.assertEqual(transfer2['id'], res_dict['transfers'][0]['id'])
self.assertEqual(share_id_2, res_dict['transfers'][0]['resource_id'])
def test_create_transfer(self):
share_id = self._create_share(status='available', size=5)
body = {"transfer": {"name": "transfer1",
"share_id": share_id}}
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
res_dict = self.v2_controller.create(req, body)
self.assertIn('id', res_dict['transfer'])
self.assertIn('auth_key', res_dict['transfer'])
self.assertIn('created_at', res_dict['transfer'])
self.assertIn('name', res_dict['transfer'])
self.assertIn('resource_id', res_dict['transfer'])
@ddt.data({},
{"transfer": {"name": "transfer1"}},
{"transfer": {"name": "transfer1",
"share_id": "invalid_share_id"}})
def test_create_transfer_with_invalid_body(self, body):
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.create, req, body)
def test_create_transfer_with_invalid_share_status(self):
share_id = self._create_share()
body = {"transfer": {"name": "transfer1",
"share_id": share_id}}
db.share_update(context.get_admin_context(),
share_id, {'status': 'error'})
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.create, req, body)
def test_create_transfer_share_with_network_id(self):
share_id = self._create_share(share_network_id='fake_id')
body = {"transfer": {"name": "transfer1",
"share_id": share_id}}
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.create, req, body)
def test_create_transfer_share_with_invalid_snapshot(self):
share_id = self._create_share(share_network_id='fake_id')
db_utils.create_snapshot(share_id=share_id)
body = {"transfer": {"name": "transfer1",
"share_id": share_id}}
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.create, req, body)
def test_delete_transfer_awaiting_transfer(self):
share_id = self._create_share()
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
path = '/v2/fake_project_id/share-transfers/%s' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'DELETE'
req.headers['Content-Type'] = 'application/json'
self.v2_controller.delete(req, transfer['id'])
# verify transfer has been deleted
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res = req.get_response(fakes.app())
self.assertEqual(http_client.NOT_FOUND, res.status_int)
self.assertEqual(db.share_get(context.get_admin_context(),
share_id)['status'], 'available')
def test_delete_transfer_not_awaiting_transfer(self):
share_id = self._create_share()
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
db.share_update(context.get_admin_context(),
share_id, {'status': 'available'})
path = '/v2/fake_project_id/share-transfers/%s' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'DELETE'
req.headers['Content-Type'] = 'application/json'
self.assertRaises(exception.InvalidShare,
self.v2_controller.delete, req,
transfer['id'])
def test_transfer_accept_share_id_specified(self):
share_id = self._create_share()
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
self.mock_object(quota.QUOTAS, 'reserve', mock.Mock())
self.mock_object(quota.QUOTAS, 'commit', mock.Mock())
self.mock_object(share_api.API,
'check_is_share_size_within_per_share_quota_limit',
mock.Mock())
self.mock_object(share_rpcapi.ShareAPI,
'transfer_accept',
mock.Mock())
fake_share_type = {'id': 'fake_id',
'name': 'fake_name',
'is_public': True}
self.mock_object(share_types, 'get_share_type',
mock.Mock(return_value=fake_share_type))
self.mock_object(db, 'share_snapshot_get_all_for_share',
mock.Mock(return_value={}))
body = {"accept": {"auth_key": transfer['auth_key']}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.v2_controller.accept(req, transfer['id'], body)
def test_transfer_accept_with_not_public_share_type(self):
share_id = self._create_share()
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
fake_share_type = {'id': 'fake_id',
'name': 'fake_name',
'is_public': False,
'projects': ['project_id1', 'project_id2']}
self.mock_object(share_types, 'get_share_type',
mock.Mock(return_value=fake_share_type))
body = {"accept": {"auth_key": transfer['auth_key']}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.accept, req,
transfer['id'], body)
@ddt.data({},
{"accept": {}},
{"accept": {"auth_key": "fake_auth_key",
"clear_access_rules": "invalid_bool"}})
def test_transfer_accept_with_invalid_body(self, body):
path = '/v2/fake_project_id/share-transfers/fake_transfer_id/accept'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.accept, req,
'fake_transfer_id', body)
def test_transfer_accept_with_invalid_auth_key(self):
share_id = self._create_share(size=5)
transfer = self._create_transfer(share_id)
body = {"accept": {"auth_key": "invalid_auth_key"}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.accept, req,
transfer['id'], body)
def test_transfer_accept_with_invalid_share_status(self):
share_id = self._create_share(size=5)
transfer = self._create_transfer(share_id)
db.share_update(context.get_admin_context(),
share_id, {'status': 'error'})
body = {"accept": {"auth_key": transfer['auth_key']}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.accept, req,
transfer['id'], body)
@ddt.data({'overs': {'gigabytes': 'fake'}},
{'overs': {'shares': 'fake'}},
{'overs': {'snapshot_gigabytes': 'fake'}},
{'overs': {'snapshots': 'fake'}})
@ddt.unpack
def test_accept_share_over_quota(self, overs):
share_id = self._create_share()
db_utils.create_snapshot(share_id=share_id, status='available')
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
usages = {'gigabytes': {'reserved': 5, 'in_use': 5},
'shares': {'reserved': 10, 'in_use': 10},
'snapshot_gigabytes': {'reserved': 5, 'in_use': 5},
'snapshots': {'reserved': 10, 'in_use': 10}}
quotas = {'gigabytes': 5, 'shares': 10,
'snapshot_gigabytes': 5, 'snapshots': 10}
exc = exception.OverQuota(overs=overs, usages=usages, quotas=quotas)
self.mock_object(quota.QUOTAS, 'reserve', mock.Mock(side_effect=exc))
self.mock_object(quota.QUOTAS, 'commit', mock.Mock())
self.mock_object(share_api.API,
'check_is_share_size_within_per_share_quota_limit',
mock.Mock())
self.mock_object(share_rpcapi.ShareAPI,
'transfer_accept',
mock.Mock())
fake_share_type = {'id': 'fake_id',
'name': 'fake_name',
'is_public': True}
self.mock_object(share_types, 'get_share_type',
mock.Mock(return_value=fake_share_type))
body = {"accept": {"auth_key": transfer['auth_key']}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPRequestEntityTooLarge,
self.v2_controller.accept, req,
transfer['id'], body)

143
manila/tests/db/sqlalchemy/test_api.py
View File

@ -5083,3 +5083,146 @@ class AsyncOperationDatabaseAPITestCase(test.TestCase):
self.ctxt, test_id) self.ctxt, test_id)
self.assertEqual({}, actual_result) self.assertEqual({}, actual_result)
class TransfersTestCase(test.TestCase):
"""Test case for transfers."""
def setUp(self):
super(TransfersTestCase, self).setUp()
self.user_id = uuidutils.generate_uuid()
self.project_id = uuidutils.generate_uuid()
self.ctxt = context.RequestContext(user_id=self.user_id,
project_id=self.project_id)
@staticmethod
def _create_transfer(resource_type='share',
resource_id=None, source_project_id=None):
"""Create a transfer object."""
if resource_id and source_project_id:
transfer = db_utils.create_transfer(
resource_type=resource_type,
resource_id=resource_id,
source_project_id=source_project_id)
elif resource_id:
transfer = db_utils.create_transfer(
resource_type=resource_type,
resource_id=resource_id)
elif source_project_id:
transfer = db_utils.create_transfer(
resource_type=resource_type,
source_project_id=source_project_id)
else:
transfer = db_utils.create_transfer(
resource_type=resource_type)
return transfer['id']
def test_transfer_create(self):
# If the resource_id is Null a KeyError exception will be raised.
self.assertRaises(KeyError, self._create_transfer)
share = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)
share_id = share['id']
self._create_transfer(resource_id=share_id)
def test_share_transfer_get(self):
share_id = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
transfer_id = self._create_transfer(resource_id=share_id)
transfer = db_api.share_transfer_get(self.ctxt, transfer_id)
self.assertEqual(share_id, transfer['resource_id'])
new_ctxt = context.RequestContext(user_id='new_user_id',
project_id='new_project_id')
self.assertRaises(exception.TransferNotFound,
db_api.share_transfer_get, new_ctxt, transfer_id)
transfer = db_api.share_transfer_get(new_ctxt.elevated(), transfer_id)
self.assertEqual(share_id, transfer['resource_id'])
def test_transfer_get_all(self):
share_id1 = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
share_id2 = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
self._create_transfer(resource_id=share_id1,
source_project_id=self.project_id)
self._create_transfer(resource_id=share_id2,
source_project_id=self.project_id)
self.assertRaises(exception.NotAuthorized,
db_api.transfer_get_all,
self.ctxt)
transfers = db_api.transfer_get_all(context.get_admin_context())
self.assertEqual(2, len(transfers))
transfers = db_api.transfer_get_all_by_project(self.ctxt,
self.project_id)
self.assertEqual(2, len(transfers))
new_ctxt = context.RequestContext(user_id='new_user_id',
project_id='new_project_id')
transfers = db_api.transfer_get_all_by_project(new_ctxt,
'new_project_id')
self.assertEqual(0, len(transfers))
def test_transfer_destroy(self):
share_id1 = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
share_id2 = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
transfer_id1 = self._create_transfer(resource_id=share_id1,
source_project_id=self.project_id)
transfer_id2 = self._create_transfer(resource_id=share_id2,
source_project_id=self.project_id)
transfers = db_api.transfer_get_all(context.get_admin_context())
self.assertEqual(2, len(transfers))
db_api.transfer_destroy(self.ctxt, transfer_id1)
transfers = db_api.transfer_get_all(context.get_admin_context())
self.assertEqual(1, len(transfers))
db_api.transfer_destroy(self.ctxt, transfer_id2)
transfers = db_api.transfer_get_all(context.get_admin_context())
self.assertEqual(0, len(transfers))
def test_transfer_accept_then_rollback(self):
share = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)
transfer_id = self._create_transfer(resource_id=share['id'],
source_project_id=self.project_id)
new_ctxt = context.RequestContext(user_id='new_user_id',
project_id='new_project_id')
transfer = db_api.share_transfer_get(new_ctxt.elevated(), transfer_id)
self.assertEqual(share['project_id'], transfer['source_project_id'])
self.assertFalse(transfer['accepted'])
self.assertIsNone(transfer['destination_project_id'])
# accept the transfer
db_api.transfer_accept(new_ctxt.elevated(), transfer_id,
'new_user_id', 'new_project_id')
transfer = db_api.model_query(
new_ctxt.elevated(), models.Transfer,
read_deleted='yes').filter_by(id=transfer_id).first()
share = db_api.share_get(new_ctxt.elevated(), share['id'])
self.assertEqual(share['project_id'], 'new_project_id')
self.assertEqual(share['user_id'], 'new_user_id')
self.assertTrue(transfer['accepted'])
self.assertEqual('new_project_id', transfer['destination_project_id'])
# then test rollback the transfer
db_api.transfer_accept_rollback(new_ctxt.elevated(), transfer_id,
self.user_id, self.project_id)
transfer = db_api.model_query(
new_ctxt.elevated(),
models.Transfer).filter_by(id=transfer_id).first()
share = db_api.share_get(new_ctxt.elevated(), share['id'])
self.assertEqual(share['project_id'], self.project_id)
self.assertEqual(share['user_id'], self.user_id)
self.assertFalse(transfer['accepted'])

8
manila/tests/db_utils.py
View File

@ -299,3 +299,11 @@ def create_message(**kwargs):
'message_level': message_levels.ERROR, 'message_level': message_levels.ERROR,
} }
return _create_db_row(db.message_create, message_dict, kwargs) return _create_db_row(db.message_create, message_dict, kwargs)
def create_transfer(**kwargs):
transfer = {'display_name': 'display_name',
'salt': 'salt',
'crypt_hash': 'crypt_hash',
'resource_type': constants.SHARE_RESOURCE_TYPE}
return _create_db_row(db.transfer_create, transfer, kwargs)

20
manila/tests/share/drivers/cephfs/test_driver.py
View File

@ -588,6 +588,26 @@ class CephFSDriverTestCase(test.TestCase):
(self._driver.protocol_helper.get_configured_ip_versions. (self._driver.protocol_helper.get_configured_ip_versions.
assert_called_once_with()) assert_called_once_with())
@ddt.data(
([{'id': 'instance_mapping_id1', 'access_id': 'accessid1',
'access_level': 'rw', 'access_type': 'cephx', 'access_to': 'alice'
}], 'fake_project_uuid_1'),
([{'id': 'instance_mapping_id1', 'access_id': 'accessid1',
'access_level': 'rw', 'access_type': 'cephx', 'access_to': 'alice'
}], 'fake_project_uuid_2'),
([], 'fake_project_uuid_1'),
([], 'fake_project_uuid_2'),
)
@ddt.unpack
def test_transfer_accept(self, access_rules, new_project):
fake_share_1 = {"project_id": "fake_project_uuid_1"}
same_project = new_project == 'fake_project_uuid_1'
if access_rules and not same_project:
self.assertRaises(exception.DriverCannotTransferShareWithRules,
self._driver.transfer_accept,
self._context, fake_share_1,
'new_user', new_project, access_rules)
@ddt.ddt @ddt.ddt
class NativeProtocolHelperTestCase(test.TestCase): class NativeProtocolHelperTestCase(test.TestCase):

4
manila/tests/share/test_api.py
View File

@ -929,8 +929,8 @@ class ShareAPITestCase(test.TestCase):
if replication_type: if replication_type:
# Prevent the raising of an exception, to force the call to the # Prevent the raising of an exception, to force the call to the
# function _check_if_replica_quotas_exceeded # function check_if_replica_quotas_exceeded
self.mock_object(self.api, '_check_if_share_quotas_exceeded') self.mock_object(self.api, 'check_if_share_quotas_exceeded')
self.assertRaises( self.assertRaises(
expected_exception, expected_exception,

49
manila/tests/share/test_manager.py
View File

@ -49,6 +49,7 @@ from manila.tests import fake_notifier
from manila.tests import fake_share as fakes from manila.tests import fake_share as fakes
from manila.tests import fake_utils from manila.tests import fake_utils
from manila.tests import utils as test_utils from manila.tests import utils as test_utils
from manila.transfer import api as transfer_api
from manila import utils from manila import utils
@ -3994,6 +3995,54 @@ class ShareManagerTestCase(test.TestCase):
api.API.delete.assert_called_once_with( api.API.delete.assert_called_once_with(
self.context, share1) self.context, share1)
def test_delete_expired_transfers(self):
self.mock_object(db, 'get_all_expired_transfers',
mock.Mock(return_value=[{"id": "transfer1",
"name": "test_tr"}, ]))
self.mock_object(transfer_api.API, 'delete')
self.share_manager.delete_expired_transfers(self.context)
db.get_all_expired_transfers.assert_called_once_with(self.context)
transfer1 = {"id": "transfer1", "name": "test_tr"}
transfer_api.API.delete.assert_called_once_with(
self.context, transfer_id=transfer1["id"])
@ddt.data(True, False)
def test_transfer_accept(self, clear_rules):
share = db_utils.create_share(id="fake")
self.mock_object(db, 'share_get', mock.Mock(return_value=share))
update_access_rules_call = self.mock_object(
self.share_manager.access_helper,
'update_access_rules')
transfer_accept_call = self.mock_object(self.share_manager.driver,
'transfer_accept')
instances, rules = self._setup_init_mocks()
self.mock_object(self.share_manager.db,
'share_access_get_all_for_share',
mock.Mock(return_value=rules))
self.mock_object(self.share_manager.db,
'share_instances_get_all_by_share',
mock.Mock(return_value=instances))
self.mock_object(db, 'share_instance_get',
mock.Mock(return_value=instances[0]))
self.mock_object(self.share_manager,
'_get_share_server',
mock.Mock(return_value=None))
self.share_manager.transfer_accept(self.context, "fake_share_id",
"fake_user_id", "fake_project_id",
clear_rules)
if clear_rules:
update_access_rules_call.assert_called_with(
self.context, instances[0]['id'], delete_all_rules=True)
transfer_accept_call.assert_called_with(
self.context, instances[0], "fake_user_id",
"fake_project_id", access_rules=[],
share_server=None)
else:
transfer_accept_call.assert_called_with(
self.context, instances[0], "fake_user_id",
"fake_project_id", access_rules=rules,
share_server=None)
@mock.patch('manila.tests.fake_notifier.FakeNotifier._notify') @mock.patch('manila.tests.fake_notifier.FakeNotifier._notify')
def test_extend_share_invalid(self, mock_notify): def test_extend_share_invalid(self, mock_notify):
share = db_utils.create_share() share = db_utils.create_share()

9
manila/tests/share/test_rpcapi.py
View File

@ -235,6 +235,15 @@ class ShareRpcAPITestCase(test.TestCase):
rpc_method='cast', rpc_method='cast',
share_server=self.fake_share_server) share_server=self.fake_share_server)
def test_transfer_accept(self):
self._test_share_api('transfer_accept',
rpc_method='call',
version='1.25',
share=self.fake_share,
new_user='new_user',
new_project='new_project',
clear_rules=False)
def test_extend_share(self): def test_extend_share(self):
self._test_share_api('extend_share', self._test_share_api('extend_share',
rpc_method='cast', rpc_method='cast',

0
manila/transfer/__init__.py Normal file
View File

440
manila/transfer/api.py Normal file
View File

@ -0,0 +1,440 @@
# Copyright (C) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Handles all requests relating to transferring ownership of shares.
"""
import hashlib
import hmac
import os
from oslo_config import cfg
from oslo_log import log as logging
from oslo_utils import excutils
from oslo_utils import strutils
from manila.common import constants
from manila.db import base
from manila import exception
from manila.i18n import _
from manila import policy
from manila import quota
from manila.share import api as share_api
from manila.share import share_types
from manila.share import utils as share_utils
share_transfer_opts = [
cfg.IntOpt('share_transfer_salt_length',
default=8,
help='The number of characters in the salt.',
min=8,
max=255),
cfg.IntOpt('share_transfer_key_length',
default=16,
help='The number of characters in the autogenerated auth key.',
min=16,
max=255),
]
CONF = cfg.CONF
CONF.register_opts(share_transfer_opts)
LOG = logging.getLogger(__name__)
QUOTAS = quota.QUOTAS
class API(base.Base):
"""API for interacting share transfers."""
def __init__(self):
self.share_api = share_api.API()
super().__init__()
def get(self, context, transfer_id):
transfer = self.db.share_transfer_get(context, transfer_id)
return transfer
def delete(self, context, transfer_id):
"""Delete a share transfer."""
transfer = self.db.share_transfer_get(context, transfer_id)
policy.check_policy(context, 'share_transfer', 'delete', target_obj={
'project_id': transfer['source_project_id']})
update_share_status = True
share_ref = None
try:
share_ref = self.db.share_get(context, transfer.resource_id)
except exception.NotFound:
update_share_status = False
if update_share_status:
share_instance = share_ref['instance']
if share_ref['status'] != constants.STATUS_AWAITING_TRANSFER:
msg = (_('Transfer %(transfer_id)s: share id %(share_id)s '
'expected in awaiting_transfer state.'))
msg_payload = {'transfer_id': transfer_id,
'share_id': share_ref['id']}
LOG.error(msg, msg_payload)
raise exception.InvalidShare(reason=msg)
if update_share_status:
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.delete.start")
self.db.transfer_destroy(context, transfer_id,
update_share_status=update_share_status)
if update_share_status:
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.delete.end")
LOG.info('Transfer %s has been deleted successful.', transfer_id)
def get_all(self, context, limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
filters = filters or {}
all_tenants = strutils.bool_from_string(filters.pop('all_tenants',
'false'))
query_by_project = False
if all_tenants:
try:
policy.check_policy(context, 'share_transfer',
'get_all_tenant')
except exception.PolicyNotAuthorized:
query_by_project = True
else:
query_by_project = True
if query_by_project:
transfers = self.db.transfer_get_all_by_project(
context, context.project_id,
limit=limit, sort_key=sort_key, sort_dir=sort_dir,
filters=filters, offset=offset)
else:
transfers = self.db.transfer_get_all(context,
limit=limit,
sort_key=sort_key,
sort_dir=sort_dir,
filters=filters,
offset=offset)
return transfers
def _get_random_string(self, length):
"""Get a random hex string of the specified length."""
rndstr = ""
# Note that the string returned by this function must contain only
# characters that the recipient can enter on their keyboard. The
# function sha256().hexdigit() achieves this by generating a hash
# which will only contain hexadecimal digits.
while len(rndstr) < length:
rndstr += hashlib.sha256(os.urandom(255)).hexdigest()
return rndstr[0:length]
def _get_crypt_hash(self, salt, auth_key):
"""Generate a random hash based on the salt and the auth key."""
def _format_str(input_str):
if not isinstance(input_str, (bytes, str)):
input_str = str(input_str)
if isinstance(input_str, str):
input_str = input_str.encode('utf-8')
return input_str
salt = _format_str(salt)
auth_key = _format_str(auth_key)
return hmac.new(salt, auth_key, hashlib.sha256).hexdigest()
def create(self, context, share_id, display_name):
"""Creates an entry in the transfers table."""
LOG.debug("Generating transfer record for share %s", share_id)
try:
share_ref = self.share_api.get(context, share_id)
except exception.NotFound:
msg = _("Share specified was not found.")
raise exception.InvalidShare(reason=msg)
policy.check_policy(context, "share_transfer", "create",
target_obj=share_ref)
share_instance = share_ref['instance']
if share_ref['status'] != "available":
raise exception.InvalidShare(reason=_("Share's status must be "
"available"))
if share_ref['share_network_id']:
raise exception.InvalidShare(reason=_(
"Shares exported over share networks cannot be transferred."))
if share_ref['share_group_id']:
raise exception.InvalidShare(reason=_(
"Shares within share groups cannot be transferred."))
if share_ref.has_replicas:
raise exception.InvalidShare(reason=_(
"Shares with replicas cannot be transferred."))
snapshots = self.db.share_snapshot_get_all_for_share(context, share_id)
for snapshot in snapshots:
if snapshot['status'] != "available":
msg = _("Snapshot: %s status must be "
"available") % snapshot['id']
raise exception.InvalidSnapshot(reason=msg)
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.create.start")
# The salt is just a short random string.
salt = self._get_random_string(CONF.share_transfer_salt_length)
auth_key = self._get_random_string(CONF.share_transfer_key_length)
crypt_hash = self._get_crypt_hash(salt, auth_key)
transfer_rec = {'resource_type': constants.SHARE_RESOURCE_TYPE,
'resource_id': share_id,
'display_name': display_name,
'salt': salt,
'crypt_hash': crypt_hash,
'expires_at': None,
'source_project_id': share_ref['project_id']}
try:
transfer = self.db.transfer_create(context, transfer_rec)
except Exception:
with excutils.save_and_reraise_exception():
LOG.error("Failed to create transfer record for %s", share_id)
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.create.end")
return {'id': transfer['id'],
'resource_type': transfer['resource_type'],
'resource_id': transfer['resource_id'],
'display_name': transfer['display_name'],
'auth_key': auth_key,
'created_at': transfer['created_at'],
'source_project_id': transfer['source_project_id'],
'destination_project_id': transfer['destination_project_id'],
'accepted': transfer['accepted'],
'expires_at': transfer['expires_at']}
def _handle_snapshot_quota(self, context, snapshots, donor_id):
snapshots_num = len(snapshots)
share_snap_sizes = 0
for snapshot in snapshots:
share_snap_sizes += snapshot['size']
try:
reserve_opts = {'snapshots': snapshots_num,
'gigabytes': share_snap_sizes}
reservations = QUOTAS.reserve(context, **reserve_opts)
except exception.OverQuota as e:
reservations = None
overs = e.kwargs['overs']
usages = e.kwargs['usages']
quotas = e.kwargs['quotas']
def _consumed(name):
return (usages[name]['reserved'] + usages[name]['in_use'])
if 'snapshot_gigabytes' in overs:
msg = ("Quota exceeded for %(s_pid)s, tried to accept "
"%(s_size)sG snapshot (%(d_consumed)dG of "
"%(d_quota)dG already consumed).")
LOG.warning(msg, {
's_pid': context.project_id,
's_size': share_snap_sizes,
'd_consumed': _consumed('snapshot_gigabytes'),
'd_quota': quotas['snapshot_gigabytes']})
raise exception.SnapshotSizeExceedsAvailableQuota()
elif 'snapshots' in overs:
msg = ("Quota exceeded for %(s_pid)s, tried to accept "
"%(s_num)s snapshot (%(d_consumed)d of "
"%(d_quota)d already consumed).")
LOG.warning(msg, {'s_pid': context.project_id,
's_num': snapshots_num,
'd_consumed': _consumed('snapshots'),
'd_quota': quotas['snapshots']})
raise exception.SnapshotLimitExceeded(
allowed=quotas['snapshots'])
try:
reserve_opts = {'snapshots': -snapshots_num,
'gigabytes': -share_snap_sizes}
donor_reservations = QUOTAS.reserve(context,
project_id=donor_id,
**reserve_opts)
except exception.OverQuota:
donor_reservations = None
LOG.exception("Failed to update share providing snapshots quota:"
" Over quota.")
return reservations, donor_reservations
@staticmethod
def _check_share_type_access(context, share_type_id, share_id):
share_type = share_types.get_share_type(
context, share_type_id, expected_fields=['projects'])
if not share_type['is_public']:
if context.project_id not in share_type['projects']:
msg = _("Share type of share %(share_id)s is not public, "
"and current project can not access the share "
"type ") % {'share_id': share_id}
LOG.error(msg)
raise exception.InvalidShare(reason=msg)
def _check_transferred_project_quota(self, context, share_ref_size):
try:
reserve_opts = {'shares': 1, 'gigabytes': share_ref_size}
reservations = QUOTAS.reserve(context,
**reserve_opts)
except exception.OverQuota as exc:
reservations = None
self.share_api.check_if_share_quotas_exceeded(context, exc,
share_ref_size)
return reservations
@staticmethod
def _check_donor_project_quota(context, donor_id, share_ref_size,
transfer_id):
try:
reserve_opts = {'shares': -1, 'gigabytes': -share_ref_size}
donor_reservations = QUOTAS.reserve(context.elevated(),
project_id=donor_id,
**reserve_opts)
except Exception:
donor_reservations = None
LOG.exception("Failed to update quota donating share"
" transfer id %s", transfer_id)
return donor_reservations
@staticmethod
def _check_snapshot_status(snapshots, transfer_id):
for snapshot in snapshots:
# Only check snapshot with instances
if snapshot.get('status'):
if snapshot['status'] != 'available':
msg = (_('Transfer %(transfer_id)s: Snapshot '
'%(snapshot_id)s is not in the expected '
'available state.')
% {'transfer_id': transfer_id,
'snapshot_id': snapshot['id']})
LOG.error(msg)
raise exception.InvalidSnapshot(reason=msg)
def accept(self, context, transfer_id, auth_key, clear_rules=False):
"""Accept a share that has been offered for transfer."""
# We must use an elevated context to make sure we can find the
# transfer.
transfer = self.db.share_transfer_get(context.elevated(), transfer_id)
crypt_hash = self._get_crypt_hash(transfer['salt'], auth_key)
if crypt_hash != transfer['crypt_hash']:
msg = (_("Attempt to transfer %s with invalid auth key.") %
transfer_id)
LOG.error(msg)
raise exception.InvalidAuthKey(reason=msg)
share_id = transfer['resource_id']
try:
# We must use an elevated context to see the share that is still
# owned by the donor.
share_ref = self.share_api.get(context.elevated(), share_id)
except exception.NotFound:
msg = _("Share specified was not found.")
raise exception.InvalidShare(reason=msg)
share_instance = share_ref['instance']
if share_ref['status'] != constants.STATUS_AWAITING_TRANSFER:
msg = (_('Transfer %(transfer_id)s: share id %(share_id)s '
'expected in awaiting_transfer state.')
% {'transfer_id': transfer_id, 'share_id': share_id})
LOG.error(msg)
raise exception.InvalidShare(reason=msg)
share_ref_size = share_ref['size']
share_type_id = share_ref.get('share_type_id')
# check share type access
if share_type_id:
self._check_share_type_access(context, share_type_id, share_id)
# check per share quota limit
self.share_api.check_is_share_size_within_per_share_quota_limit(
context, share_ref_size)
# check accept transferred project quotas
reservations = self._check_transferred_project_quota(
context, share_ref_size)
# check donor project quotas
donor_id = share_ref['project_id']
donor_reservations = self._check_donor_project_quota(
context, donor_id, share_ref_size, transfer_id)
snap_res = None
snap_donor_res = None
accept_snapshots = False
snapshots = self.db.share_snapshot_get_all_for_share(
context.elevated(), share_id)
if snapshots:
self._check_snapshot_status(snapshots, transfer_id)
accept_snapshots = True
snap_res, snap_donor_res = self._handle_snapshot_quota(
context, snapshots, share_ref['project_id'])
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.accept.start")
try:
self.share_api.transfer_accept(context,
share_ref,
context.user_id,
context.project_id,
clear_rules=clear_rules)
# Transfer ownership of the share now, must use an elevated
# context.
self.db.transfer_accept(context.elevated(),
transfer_id,
context.user_id,
context.project_id,
accept_snapshots=accept_snapshots)
if reservations:
QUOTAS.commit(context, reservations)
if snap_res:
QUOTAS.commit(context, snap_res)
if donor_reservations:
QUOTAS.commit(context, donor_reservations, project_id=donor_id)
if snap_donor_res:
QUOTAS.commit(context, snap_donor_res, project_id=donor_id)
LOG.info("share %s has been transferred.", share_id)
except Exception:
with excutils.save_and_reraise_exception():
try:
# storage try to rollback
self.share_api.transfer_accept(context,
share_ref,
share_ref['user_id'],
share_ref['project_id'])
# db try to rollback
self.db.transfer_accept_rollback(
context.elevated(), transfer_id,
share_ref['user_id'], share_ref['project_id'],
rollback_snap=accept_snapshots)
finally:
if reservations:
QUOTAS.rollback(context, reservations)
if snap_res:
QUOTAS.rollback(context, snap_res)
if donor_reservations:
QUOTAS.rollback(context, donor_reservations,
project_id=donor_id)
if snap_donor_res:
QUOTAS.rollback(context, snap_donor_res,
project_id=donor_id)
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.accept.end")

4
releasenotes/notes/bp-share-transfer-between-project-5c2ba9944b17e26e.yaml Normal file
View File

@ -0,0 +1,4 @@
---
features:
- Share can be transferred between project with API version ``2.77``
and beyond.