cinder/doc/source/contributor/groups.rst

Generic Volume Groups

Introduction to generic volume groups

Generic volume group support was added in cinder in the Newton release. There is support for creating group types and group specs, creating groups of volumes, and creating snapshots of groups. Detailed information on how to create a group type, a group, and a group snapshot can be found in block storage admin guide.

How is generic volume groups different from consistency groups in cinder? The consistency group feature was introduced in cinder in Juno and are supported by a few drivers. Currently consistency groups in cinder only support consistent group snapshot. It cannot be extended easily to serve other purposes. A tenant may want to put volumes used in the same application together in a group so that it is easier to manage them together, and this group of volumes may or may not support consistent group snapshot. Generic volume group is introduced to solve this problem. By decoupling the tight relationship between the group construct and the consistency concept, generic volume groups can be extended to support other features in the future.

Action items for drivers supporting consistency groups

Drivers currently supporting consistency groups are in the following:

• Juno: EMC VNX
• Kilo: EMC VMAX, IBM (GPFS, Storwize, SVC, and XIV), ProphetStor, Pure
• Liberty: Dell Storage Center, EMC XtremIO, HPE 3Par and LeftHand
• Mitaka: EMC ScaleIO, NetApp Data ONTAP and E-Series, SolidFire
• Newton: CoprHD, FalconStor, Huawei

Since the addition of generic volume groups, there is plan to migrate consistency groups to generic volume groups. A migration command and changes in CG APIs to support migrating CGs to groups are developed and merged in Ocata [1][2]. In order to support rolling upgrade, it will take a couple of releases before consistency groups can be deprecated.

For drivers planning to add consistency groups support, the new generic volume group driver interfaces should be implemented instead of the CG interfaces.

For drivers already supporting consistency groups, the new generic volume group driver interfaces should be implemented to include the CG support.

For drivers wanting generic volume groups but not consistent group snapshot support, no code changes are necessary. By default, every cinder volume driver already supports generic volume groups since Newton because the support was added to the common code. Testing should be done for every driver to make sure this feature works properly.

Drivers already supporting CG are expected to add CG support to generic volume groups by Pike-1. This is a deadline discussed and agreed upon at the Ocata summit in Barcelona.

Group Type and Group Specs / Volume Types and Extra Specs

The driver interfaces for consistency groups and generic volume groups are very similar. One new concept introduced for generic volume groups is the group type. Group type is used to categorize a group just like a volume type is used to describe a volume. Similar to extra specs for a volume type, group specs are also introduced to be associated with a group type. Group types allow a user to create different types of groups.

A group can support multiple volume types and volume types are required as input parameters when creating a group. In addition to volume types, a group type is also required when creating a group.

Group types and volume types are created by the Cloud Administrator. A tenant uses the group types and volume types to create groups and volumes.

A driver can support both consistent group snapshot and a group of snapshots that do not maintain the write order consistency by using different group types. In other words, a group supporting consistent group snapshot is a special type of generic volume group.

For a group to support consistent group snapshot, the group specs in the corresponding group type should have the following entry:

{'consistent_group_snapshot_enabled': <is> True}

Similarly, for a volume to be in a group that supports consistent group snapshots, the volume type extra specs would also have the following entry:

{'consistent_group_snapshot_enabled': <is> True}

By requiring the above entry to be in both group specs and volume type extra specs, we can make sure the scheduler will choose a backend that supports the group type and volume types for a group. It is up to the driver to parse the group type info when creating a group, parse the volume type info when creating a volume, and set things up as requested.

Capabilities reporting

The following entry is expected to be added to the stats/capabilities update for drivers supporting consistent group snapshot:

stats["consistent_group_snapshot_enabled"] = True

Driver methods

The following driver methods should to be implemented for the driver to support consistent group snapshot:

• create_group(context, group)
• delete_group(context, group, volumes)
• update_group(context, group, add_volumes=None, remove_volumes=None)

• create_group_from_src(context, group, volumes,

  group_snapshot=None, snapshots=None, source_group=None, source_vols=None)

• create_group_snapshot(context, group_snapshot, snapshots)
• delete_group_snapshot(context, group_snapshot, snapshots)

Here is an example that add CG capability to generic volume groups [3]. Details of driver interfaces are as follows.

create_group

This method creates a group. It has context and group object as input parameters. A group object has volume_types and group_type_id that can be used by the driver.

create_group returns model_update. model_update will be in this format: {'status': xxx, ......}.

If the status in model_update is 'error', the manager will throw an exception and it will be caught in the try-except block in the manager. If the driver throws an exception, the manager will also catch it in the try-except block. The group status in the db will be changed to 'error'.

For a successful operation, the driver can either build the model_update and return it or return None. The group status will be set to 'available'.

delete_group

This method deletes a group. It has context, group object, and a list of volume objects as input parameters. It returns model_update and volumes_model_update.

volumes_model_update is a list of volume dictionaries. It has to be built by the driver. An entry will be in this format: {'id': xxx, 'status': xxx, ......}. model_update will be in this format: {'status': xxx, ......}. The driver should populate volumes_model_update and model_update and return them.

The manager will check volumes_model_update and update db accordingly for each volume. If the driver successfully deleted some volumes but failed to delete others, it should set statuses of the volumes accordingly so that the manager can update db correctly.

If the status in any entry of volumes_model_update is 'error_deleting' or 'error', the status in model_update will be set to the same if it is not already 'error_deleting' or 'error'.

If the status in model_update is 'error_deleting' or 'error', the manager will raise an exception and the status of the group will be set to 'error' in the db. If volumes_model_update is not returned by the driver, the manager will set the status of every volume in the group to 'error' in the except block.

If the driver raises an exception during the operation, it will be caught by the try-except block in the manager. The statuses of the group and all volumes in it will be set to 'error'.

For a successful operation, the driver can either build the model_update and volumes_model_update and return them or return None, None. The statuses of the group and all volumes will be set to 'deleted' after the manager deletes them from db.

update_group

This method adds existing volumes to a group or removes volumes from a group. It has context, group object, a list of volume objects to be added to the group, and a list of a volume objects to be removed from the group. It returns model_update, add_volumes_update, and remove_volumes_update.

model_update is a dictionary that the driver wants the manager to update upon a successful return. If None is returned, the manager will set the status to 'available'.

add_volumes_update and remove_volumes_update are lists of dictionaries that the driver wants the manager to update upon a successful return. Note that each entry requires a {'id': xxx} so that the correct volume entry can be updated. If None is returned, the volume will remain its original status.

If the driver throws an exception, the status of the group as well as those of the volumes to be added/removed will be set to 'error'.

create_group_from_src

This method creates a group from source. The source can be a group_snapshot or a source group. create_group_from_src has context, group object, a list of volume objects, group_snapshot object, a list of snapshot objects, source group object, and a list of source volume objects as input parameters. It returns model_update and volumes_model_update.

volumes_model_update is a list of dictionaries. It has to be built by the driver. An entry will be in this format: {'id': xxx, 'status': xxx, ......}. model_update will be in this format: {'status': xxx, ......}.

To be consistent with other volume operations, the manager will assume the operation is successful if no exception is thrown by the driver. For a successful operation, the driver can either build the model_update and volumes_model_update and return them or return None, None.

create_group_snapshot

This method creates a group_snapshot. It has context, group_snapshot object, and a list of snapshot objects as input parameters. It returns model_update and snapshots_model_update.

snapshots_model_update is a list of dictionaries. It has to be built by the driver. An entry will be in this format: {'id': xxx, 'status': xxx, ......}. model_update will be in this format: {'status': xxx, ......}. The driver should populate snapshots_model_update and model_update and return them.

The manager will check snapshots_model_update and update db accordingly for each snapshot. If the driver successfully created some snapshots but failed to create others, it should set statuses of the snapshots accordingly so that the manager can update db correctly.

If the status in any entry of snapshots_model_update is 'error', the status in model_update will be set to the same if it is not already 'error'.

If the status in model_update is 'error', the manager will raise an exception and the status of group_snapshot will be set to 'error' in the db. If snapshots_model_update is not returned by the driver, the manager will set the status of every snapshot to 'error' in the except block.

If the driver raises an exception during the operation, it will be caught by the try-except block in the manager and the statuses of group_snapshot and all snapshots will be set to 'error'.

For a successful operation, the driver can either build the model_update and snapshots_model_update and return them or return None, None. The statuses of group_snapshot and all snapshots will be set to 'available' at the end of the manager function.

delete_group_snapshot

This method deletes a group_snapshot. It has context, group_snapshot object, and a list of snapshot objects. It returns model_update and snapshots_model_update.

snapshots_model_update is a list of dictionaries. It has to be built by the driver. An entry will be in this format: {'id': xxx, 'status': xxx, ......}. model_update will be in this format: {'status': xxx, ......}. The driver should populate snapshots_model_update and model_update and return them.

The manager will check snapshots_model_update and update db accordingly for each snapshot. If the driver successfully deleted some snapshots but failed to delete others, it should set statuses of the snapshots accordingly so that the manager can update db correctly.

If the status in any entry of snapshots_model_update is 'error_deleting' or 'error', the status in model_update will be set to the same if it is not already 'error_deleting' or 'error'.

If the status in model_update is 'error_deleting' or 'error', the manager will raise an exception and the status of group_snapshot will be set to 'error' in the db. If snapshots_model_update is not returned by the driver, the manager will set the status of every snapshot to 'error' in the except block.

If the driver raises an exception during the operation, it will be caught by the try-except block in the manager and the statuses of group_snapshot and all snapshots will be set to 'error'.

For a successful operation, the driver can either build the model_update and snapshots_model_update and return them or return None, None. The statuses of group_snapshot and all snapshots will be set to 'deleted' after the manager deletes them from db.

Migrate CGs to Generic Volume Groups

This section only affects drivers already supporting CGs by the Newton release. Drivers planning to add CG support after Newton are not affected.

A group type named default_cgsnapshot_type will be created by the migration script. The following command needs to be run to migrate migrate data and copy data from consistency groups to groups and from cgsnapshots to group_snapshots. Migrated consistency groups and cgsnapshots will be removed from the database:

cinder-manage db online_data_migrations
--max_count <max>
--ignore_state

max_count is optional. Default is 50. ignore_state is optional. Default is False.

After running the above migration command to migrate CGs to generic volume groups, CG and group APIs work as follows:

• Create CG only creates in the groups table.
• Modify CG modifies in the CG table if the CG is in the CG table, otherwise it modifies in the groups table.
• Delete CG deletes from the CG or the groups table depending on where the CG is.
• List CG checks both CG and groups tables.
• List CG Snapshots checks both the CG and the groups tables.
• Show CG checks both tables.
• Show CG Snapshot checks both tables.
• Create CG Snapshot creates either in the CG or the groups table depending on where the CG is.
• Create CG from Source creates in either the CG or the groups table depending on the source.
• Create Volume adds the volume either to the CG or the group.
• default_cgsnapshot_type is reserved for migrating CGs.
• Group APIs will only write/read in/from the groups table.
• Group APIs will not work on groups with default_cgsnapshot_type.
• Groups with default_cgsnapshot_type can only be operated by CG APIs.
• After CG tables are removed, we will allow default_cgsnapshot_type to be used by group APIs.

References

[1] Migration script

https://review.openstack.org/#/c/350350/

[2] CG APIs changes for migrating CGs

https://review.openstack.org/#/c/401839/

[3] Example adding CG capability to generic volume groups

https://review.openstack.org/#/c/413927/