52370e58fe
These are in the cinder doc tree so of course they're block storage-related. Change-Id: Ic1950ff89021a89de397619eef17f8100eb3d847 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
364 lines
15 KiB
ReStructuredText
364 lines
15 KiB
ReStructuredText
Generic Volume Groups
|
|
=====================
|
|
|
|
Introduction to generic volume groups
|
|
-------------------------------------
|
|
|
|
Generic volume group support was added in cinder in the Newton release.
|
|
There is support for creating group types and group specs, creating
|
|
groups of volumes, and creating snapshots of groups. Detailed information
|
|
on how to create a group type, a group, and a group snapshot can be found
|
|
in :doc:`block storage admin guide </admin/groups>`.
|
|
|
|
How is generic volume groups different from consistency groups in cinder?
|
|
The consistency group feature was introduced in cinder in Juno and are
|
|
supported by a few drivers. Currently consistency groups in cinder only
|
|
support consistent group snapshot. It cannot be extended easily to serve
|
|
other purposes. A tenant may want to put volumes used in the same application
|
|
together in a group so that it is easier to manage them together, and this
|
|
group of volumes may or may not support consistent group snapshot. Generic
|
|
volume group is introduced to solve this problem. By decoupling the tight
|
|
relationship between the group construct and the consistency concept,
|
|
generic volume groups can be extended to support other features in the future.
|
|
|
|
Action items for drivers supporting consistency groups
|
|
------------------------------------------------------
|
|
|
|
Drivers currently supporting consistency groups are in the following:
|
|
|
|
- Juno: EMC VNX
|
|
|
|
- Kilo: EMC VMAX, IBM (GPFS, Storwize, SVC, and XIV), ProphetStor, Pure
|
|
|
|
- Liberty: Dell Storage Center, EMC XtremIO, HPE 3Par and LeftHand
|
|
|
|
- Mitaka: EMC ScaleIO, NetApp Data ONTAP, SolidFire
|
|
|
|
- Newton: CoprHD, FalconStor, Huawei
|
|
|
|
Since the addition of generic volume groups, there is plan to migrate
|
|
consistency groups to generic volume groups. A migration command and
|
|
changes in CG APIs to support migrating CGs to groups are developed and
|
|
merged in Ocata [1][2]. In order to support rolling upgrade, it will take
|
|
a couple of releases before consistency groups can be deprecated.
|
|
|
|
For drivers planning to add consistency groups support, the new generic
|
|
volume group driver interfaces should be implemented instead of the CG
|
|
interfaces.
|
|
|
|
For drivers already supporting consistency groups, the new generic
|
|
volume group driver interfaces should be implemented to include the
|
|
CG support.
|
|
|
|
For drivers wanting generic volume groups but not consistent group
|
|
snapshot support, no code changes are necessary. By default, every
|
|
cinder volume driver already supports generic volume groups since
|
|
Newton because the support was added to the common code. Testing
|
|
should be done for every driver to make sure this feature works properly.
|
|
|
|
Drivers already supporting CG are expected to add CG support to
|
|
generic volume groups by Pike-1. This is a deadline discussed and
|
|
agreed upon at the Ocata summit in Barcelona.
|
|
|
|
Group Type and Group Specs / Volume Types and Extra Specs
|
|
---------------------------------------------------------
|
|
|
|
The driver interfaces for consistency groups and generic volume groups
|
|
are very similar. One new concept introduced for generic volume groups
|
|
is the group type. Group type is used to categorize a group just like a
|
|
volume type is used to describe a volume. Similar to extra specs for
|
|
a volume type, group specs are also introduced to be associated with a
|
|
group type. Group types allow a user to create different types of groups.
|
|
|
|
A group can support multiple volume types and volume types are required
|
|
as input parameters when creating a group. In addition to volume types,
|
|
a group type is also required when creating a group.
|
|
|
|
Group types and volume types are created by the Cloud Administrator.
|
|
A tenant uses the group types and volume types to create groups and
|
|
volumes.
|
|
|
|
A driver can support both consistent group snapshot and a group of
|
|
snapshots that do not maintain the write order consistency by using
|
|
different group types. In other words, a group supporting consistent
|
|
group snapshot is a special type of generic volume group.
|
|
|
|
For a group to support consistent group snapshot, the group specs in the
|
|
corresponding group type should have the following entry::
|
|
|
|
{'consistent_group_snapshot_enabled': <is> True}
|
|
|
|
Similarly, for a volume to be in a group that supports consistent group
|
|
snapshots, the volume type extra specs would also have the following entry::
|
|
|
|
{'consistent_group_snapshot_enabled': <is> True}
|
|
|
|
By requiring the above entry to be in both group specs and volume type
|
|
extra specs, we can make sure the scheduler will choose a backend that
|
|
supports the group type and volume types for a group. It is up to the driver
|
|
to parse the group type info when creating a group, parse the volume type
|
|
info when creating a volume, and set things up as requested.
|
|
|
|
Capabilities reporting
|
|
----------------------
|
|
The following entry is expected to be added to the stats/capabilities update
|
|
for drivers supporting consistent group snapshot::
|
|
|
|
stats["consistent_group_snapshot_enabled"] = True
|
|
|
|
Driver methods
|
|
--------------
|
|
The following driver methods should to be implemented for the driver to
|
|
support consistent group snapshot:
|
|
|
|
- create_group(context, group)
|
|
|
|
- delete_group(context, group, volumes)
|
|
|
|
- update_group(context, group, add_volumes=None, remove_volumes=None)
|
|
|
|
- create_group_from_src(context, group, volumes,
|
|
group_snapshot=None, snapshots=None,
|
|
source_group=None, source_vols=None)
|
|
|
|
- create_group_snapshot(context, group_snapshot, snapshots)
|
|
|
|
- delete_group_snapshot(context, group_snapshot, snapshots)
|
|
|
|
Here is an example that add CG capability to generic volume groups [3].
|
|
Details of driver interfaces are as follows.
|
|
|
|
**create_group**
|
|
|
|
This method creates a group. It has context and group object as input
|
|
parameters. A group object has volume_types and group_type_id that can be used
|
|
by the driver.
|
|
|
|
create_group returns model_update. model_update will be in this format:
|
|
{'status': xxx, ......}.
|
|
|
|
If the status in model_update is 'error', the manager will throw
|
|
an exception and it will be caught in the try-except block in the
|
|
manager. If the driver throws an exception, the manager will also
|
|
catch it in the try-except block. The group status in the db will
|
|
be changed to 'error'.
|
|
|
|
For a successful operation, the driver can either build the
|
|
model_update and return it or return None. The group status will
|
|
be set to 'available'.
|
|
|
|
**delete_group**
|
|
|
|
This method deletes a group. It has context, group object, and a list
|
|
of volume objects as input parameters. It returns model_update and
|
|
volumes_model_update.
|
|
|
|
volumes_model_update is a list of volume dictionaries. It has to be built
|
|
by the driver. An entry will be in this format: {'id': xxx, 'status': xxx,
|
|
......}. model_update will be in this format: {'status': xxx, ......}.
|
|
The driver should populate volumes_model_update and model_update
|
|
and return them.
|
|
|
|
The manager will check volumes_model_update and update db accordingly
|
|
for each volume. If the driver successfully deleted some volumes
|
|
but failed to delete others, it should set statuses of the volumes
|
|
accordingly so that the manager can update db correctly.
|
|
|
|
If the status in any entry of volumes_model_update is 'error_deleting'
|
|
or 'error', the status in model_update will be set to the same if it
|
|
is not already 'error_deleting' or 'error'.
|
|
|
|
If the status in model_update is 'error_deleting' or 'error', the
|
|
manager will raise an exception and the status of the group will be
|
|
set to 'error' in the db. If volumes_model_update is not returned by
|
|
the driver, the manager will set the status of every volume in the
|
|
group to 'error' in the except block.
|
|
|
|
If the driver raises an exception during the operation, it will be
|
|
caught by the try-except block in the manager. The statuses of the
|
|
group and all volumes in it will be set to 'error'.
|
|
|
|
For a successful operation, the driver can either build the
|
|
model_update and volumes_model_update and return them or
|
|
return None, None. The statuses of the group and all volumes
|
|
will be set to 'deleted' after the manager deletes them from db.
|
|
|
|
**update_group**
|
|
|
|
This method adds existing volumes to a group or removes volumes
|
|
from a group. It has context, group object, a list of volume objects
|
|
to be added to the group, and a list of a volume objects to be
|
|
removed from the group. It returns model_update, add_volumes_update,
|
|
and remove_volumes_update.
|
|
|
|
model_update is a dictionary that the driver wants the manager
|
|
to update upon a successful return. If None is returned, the manager
|
|
will set the status to 'available'.
|
|
|
|
add_volumes_update and remove_volumes_update are lists of dictionaries
|
|
that the driver wants the manager to update upon a successful return.
|
|
Note that each entry requires a {'id': xxx} so that the correct
|
|
volume entry can be updated. If None is returned, the volume will
|
|
remain its original status.
|
|
|
|
If the driver throws an exception, the status of the group as well as
|
|
those of the volumes to be added/removed will be set to 'error'.
|
|
|
|
**create_group_from_src**
|
|
|
|
This method creates a group from source. The source can be a
|
|
group_snapshot or a source group. create_group_from_src has context,
|
|
group object, a list of volume objects, group_snapshot object, a list
|
|
of snapshot objects, source group object, and a list of source volume
|
|
objects as input parameters. It returns model_update and
|
|
volumes_model_update.
|
|
|
|
volumes_model_update is a list of dictionaries. It has to be built by
|
|
the driver. An entry will be in this format: {'id': xxx, 'status': xxx,
|
|
......}. model_update will be in this format: {'status': xxx, ......}.
|
|
|
|
To be consistent with other volume operations, the manager will
|
|
assume the operation is successful if no exception is thrown by
|
|
the driver. For a successful operation, the driver can either build
|
|
the model_update and volumes_model_update and return them or
|
|
return None, None.
|
|
|
|
**create_group_snapshot**
|
|
|
|
This method creates a group_snapshot. It has context, group_snapshot
|
|
object, and a list of snapshot objects as input parameters. It returns
|
|
model_update and snapshots_model_update.
|
|
|
|
snapshots_model_update is a list of dictionaries. It has to be built by the
|
|
driver. An entry will be in this format: {'id': xxx, 'status': xxx, ......}.
|
|
model_update will be in this format: {'status': xxx, ......}. The driver
|
|
should populate snapshots_model_update and model_update and return them.
|
|
|
|
The manager will check snapshots_model_update and update db accordingly
|
|
for each snapshot. If the driver successfully created some snapshots
|
|
but failed to create others, it should set statuses of the snapshots
|
|
accordingly so that the manager can update db correctly.
|
|
|
|
If the status in any entry of snapshots_model_update is 'error', the
|
|
status in model_update will be set to the same if it is not already
|
|
'error'.
|
|
|
|
If the status in model_update is 'error', the manager will raise an
|
|
exception and the status of group_snapshot will be set to 'error' in
|
|
the db. If snapshots_model_update is not returned by the driver, the
|
|
manager will set the status of every snapshot to 'error' in the except
|
|
block.
|
|
|
|
If the driver raises an exception during the operation, it will be
|
|
caught by the try-except block in the manager and the statuses of
|
|
group_snapshot and all snapshots will be set to 'error'.
|
|
|
|
For a successful operation, the driver can either build the
|
|
model_update and snapshots_model_update and return them or
|
|
return None, None. The statuses of group_snapshot and all snapshots
|
|
will be set to 'available' at the end of the manager function.
|
|
|
|
**delete_group_snapshot**
|
|
|
|
This method deletes a group_snapshot. It has context, group_snapshot
|
|
object, and a list of snapshot objects. It returns model_update and
|
|
snapshots_model_update.
|
|
|
|
snapshots_model_update is a list of dictionaries. It has to be built by
|
|
the driver. An entry will be in this format: {'id': xxx, 'status': xxx,
|
|
......}. model_update will be in this format: {'status': xxx, ......}.
|
|
The driver should populate snapshots_model_update and model_update
|
|
and return them.
|
|
|
|
The manager will check snapshots_model_update and update db accordingly
|
|
for each snapshot. If the driver successfully deleted some snapshots
|
|
but failed to delete others, it should set statuses of the snapshots
|
|
accordingly so that the manager can update db correctly.
|
|
|
|
If the status in any entry of snapshots_model_update is
|
|
'error_deleting' or 'error', the status in model_update will be set to
|
|
the same if it is not already 'error_deleting' or 'error'.
|
|
|
|
If the status in model_update is 'error_deleting' or 'error', the
|
|
manager will raise an exception and the status of group_snapshot will
|
|
be set to 'error' in the db. If snapshots_model_update is not returned
|
|
by the driver, the manager will set the status of every snapshot to
|
|
'error' in the except block.
|
|
|
|
If the driver raises an exception during the operation, it will be
|
|
caught by the try-except block in the manager and the statuses of
|
|
group_snapshot and all snapshots will be set to 'error'.
|
|
|
|
For a successful operation, the driver can either build the
|
|
model_update and snapshots_model_update and return them or
|
|
return None, None. The statuses of group_snapshot and all snapshots
|
|
will be set to 'deleted' after the manager deletes them from db.
|
|
|
|
Migrate CGs to Generic Volume Groups
|
|
------------------------------------
|
|
|
|
This section only affects drivers already supporting CGs by the
|
|
Newton release. Drivers planning to add CG support after Newton are
|
|
not affected.
|
|
|
|
A group type named default_cgsnapshot_type will be created by the
|
|
migration script. The following command needs to be run to migrate
|
|
migrate data and copy data from consistency groups to groups and
|
|
from cgsnapshots to group_snapshots. Migrated consistency groups
|
|
and cgsnapshots will be removed from the database::
|
|
|
|
cinder-manage db online_data_migrations
|
|
--max_count <max>
|
|
|
|
max_count is optional. Default is 50.
|
|
|
|
After running the above migration command to migrate CGs to generic
|
|
volume groups, CG and group APIs work as follows:
|
|
|
|
* Create CG only creates in the groups table.
|
|
|
|
* Modify CG modifies in the CG table if the CG is in the
|
|
CG table, otherwise it modifies in the groups table.
|
|
|
|
* Delete CG deletes from the CG or the groups table
|
|
depending on where the CG is.
|
|
|
|
* List CG checks both CG and groups tables.
|
|
|
|
* List CG Snapshots checks both the CG and the groups
|
|
tables.
|
|
|
|
* Show CG checks both tables.
|
|
|
|
* Show CG Snapshot checks both tables.
|
|
|
|
* Create CG Snapshot creates either in the CG or the groups
|
|
table depending on where the CG is.
|
|
|
|
* Create CG from Source creates in either the CG or the
|
|
groups table depending on the source.
|
|
|
|
* Create Volume adds the volume either to the CG or the
|
|
group.
|
|
|
|
* default_cgsnapshot_type is reserved for migrating CGs.
|
|
|
|
* Group APIs will only write/read in/from the groups table.
|
|
|
|
* Group APIs will not work on groups with default_cgsnapshot_type.
|
|
|
|
* Groups with default_cgsnapshot_type can only be operated by
|
|
CG APIs.
|
|
|
|
* After CG tables are removed, we will allow default_cgsnapshot_type
|
|
to be used by group APIs.
|
|
|
|
References
|
|
----------
|
|
[1] Migration script
|
|
https://review.openstack.org/#/c/350350/
|
|
[2] CG APIs changes for migrating CGs
|
|
https://review.openstack.org/#/c/401839/
|
|
[3] Example adding CG capability to generic volume groups
|
|
https://review.openstack.org/#/c/413927/
|