Use https where possible for links. Change-Id: I764d3cf2f135bbe10b5bc514c39e5b8808e52137
12 KiB
Block Storage
(For nick: Restructure the introduction to this chapter to provide context of what we're actually going to do.)
By default, data in OpenStack instances is stored on 'ephemeral' disks. These disks remain with the instance throughout its lifetime. When you terminate the instance, that storage and all the data stored on it disappears. Ephemeral storage is allocated to a single instance and cannot be moved to another instance.
This section introduces block storage, also known as volume storage, which provides access to persistent storage devices. You interact with block storage by attaching volumes to running instances just as you might attach a USB drive to a physical server. You can detach volumes from one instance and reattach them to another instance and the data remains intact. The OpenStack Block Storage (cinder) project implements block storage.
Though you might have configured Object Storage to store images, the Fractal application needs a database to track the location of, and parameters that were used to create, images in Object Storage. This database server cannot fail.
If you are an advanced user, think about how you might remove the
database from the architecture and replace it with Object Storage
metadata, and then contribute these steps to craziness
.
Otherwise, continue reading to learn how to work with, and move the Fractal application database server to use, block storage.
Basics
Later on, you will use a Block Storage volume to provide persistent storage for the database server for the Fractal application. But first, learn how to create and attach a Block Storage device.
dotnet
Warning
This section has not yet been completed for the .NET SDK.
fog
Warning
This section has not yet been completed for the fog SDK.
pkgcloud
Warning
This section has not yet been completed for the pkgcloud SDK.
openstacksdk
Warning
This section has not yet been completed for the OpenStack SDK.
phpopencloud
Warning
This section has not yet been completed for the PHP-OpenCloud SDK.
Connect to the API endpoint:
libcloud
../samples/libcloud/block_storage.py
jclouds
../samples/jclouds/BlockStorage.java
shade
../samples/shade/block_storage.py
To try it out, make a 1GB volume called 'test'.
libcloud
../samples/libcloud/block_storage.py
<StorageVolume id=755ab026-b5f2-4f53-b34a-6d082fb36689 size=1 driver=OpenStack>
jclouds
../samples/jclouds/BlockStorage.java
shade
../samples/shade/block_storage.py
Note
The parameter size
is in gigabytes.
To see if the volume creation was successful, list all volumes:
libcloud
../samples/libcloud/block_storage.py
[<StorageVolume id=755ab026-b5f2-4f53-b34a-6d082fb36689 size=1 driver=OpenStack>]
jclouds
../samples/jclouds/BlockStorage.java
shade
../samples/shade/block_storage.py
Attach the storage volume to a running instance.
Use Block Storage for the Fractal database server
You need a server for the dedicated database. Use the image, flavor,
and keypair that you used in /getting_started
to launch an
app-database
instance.
You also need a security group to permit access to the database server (for MySQL, port 3306) from the network:
libcloud
../samples/libcloud/block_storage.py
jclouds
../samples/jclouds/BlockStorage.java
shade
../samples/shade/block_storage.py
Create a volume object by using the unique identifier (UUID) for the
volume. Then, use the server object from the previous code snippet to
attach the volume to it at /dev/vdb
:
libcloud
../samples/libcloud/block_storage.py
jclouds
../samples/jclouds/BlockStorage.java
shade
../samples/shade/block_storage.py
Log in to the server to run the following steps.
Note
Replace IP_DATABASE
with the IP address of the database
instance and USERNAME to the appropriate user name.
Now prepare the empty block device.
$ ssh -i ~/.ssh/id_rsa USERNAME@IP_DATABASE
# fdisk -l
Disk /dev/vdb: 1073 MB, 1073741824 bytes
16 heads, 63 sectors/track, 2080 cylinders, total 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/vdb doesn't contain a valid partition table
# mke2fs /dev/vdb
mke2fs 1.42.9 (4-Feb-2014)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
# mkdir /mnt/database
# mount /dev/vdb /mnt/database
Stop the running MySQL database service and move the database files
from /var/lib/mysql
to
the new volume, which is temporarily mounted at /mnt/database
.
# systemctl stop mariadb
# mv /var/lib/mysql/* /mnt/database
Sync the file systems and mount the block device that contains the
database files to /var/lib/mysql
.
# sync
# umount /mnt/database
# rm -rf /mnt/database
# echo "/dev/vdb /var/lib/mysql ext4 defaults 1 2" >> /etc/fstab
# mount /var/lib/mysql
Finally, start the stopped MySQL database service and validate that everything works as expected.
# systemctl start mariadb
# mysql -ufaafo -ppassword -h localhost faafo -e 'show tables;'
Extras
You can detach the volume and reattach it elsewhere, or use the following steps to delete the volume.
Warning
The following operations are destructive and result in data loss.
To detach and delete a volume:
libcloud
../samples/libcloud/block_storage.py
True
Note
detach_volume
and destroy_volume
take a
volume object, not a name.
jclouds
../samples/jclouds/BlockStorage.java
shade
../samples/shade/block_storage.py
libcloud
Other features, such as creating volume snapshots, are useful for backups:
../samples/libcloud/block_storage.py
Do we need a note here to mention that 'test' is the volume name and not the volume object?
For information about these and other calls, see libcloud documentation.
jclouds
Other features, such as creating volume snapshots, are useful for backups:
../samples/jclouds/BlockStorage.java
The following file contains all of the code from this section of the tutorial. This comprehensive code sample lets you view and run the code as a single file.
../samples/jclouds/BlockStorage.java
Work with the OpenStack Database service
Previously, you manually created the database, which is useful for a
single database that you rarely update. However, the OpenStack
trove
component provides Database as a Service (DBaaS).
Note
This OpenStack Database service is not installed in many clouds right now, but if your cloud supports it, it can make your life a lot easier when working with databases.
SDKs do not generally support the service yet, but you can use the 'trove' command-line client to work with it instead.
To install the 'trove' command-line client, see Install the OpenStack command-line clients.
To set up environment variables for your cloud in an openrc.sh
file, see Set
environment variables using the OpenStack RC file.
Ensure you have an openrc.sh
file, source it, and validate that your
trove client works:
$ cat openrc.sh
export OS_USERNAME=your_auth_username
export OS_PASSWORD=your_auth_password
export OS_TENANT_NAME=your_project_name
export OS_AUTH_URL=http://controller:5000/v2.0
export OS_REGION_NAME=your_region_name
$ source openrc.sh
$ trove --version
1.0.9
For information about supported features and how to work with an existing database service installation, see Database as a Service in OpenStack.
Next steps
You should now be fairly confident working with Block Storage volumes. For information about other calls, see the volume documentation for your SDK. Or, try one of these tutorial steps:
/orchestration
: Automatically orchestrate your application./networking
: Learn about complex networking./advice
: Get advice about operations.