openstack/trove
Code Issues Proposed changes

Trove doc and CI

Browse Source 
Change-Id: I49bd6c824b837d22d9f3945a3e8c2343c831b352
This commit is contained in:
Lingxian Kong 2020-05-27 23:34:34 +12:00
parent 91a6e5fa8a
commit e2b26fdd7d
13 changed files with 92 additions and 742 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
.zuul.yaml
View File

@ -15,49 +15,26 @@
- trove-tox-bandit-baseline: - trove-tox-bandit-baseline:
voting: false voting: false
- trove-tempest - trove-tempest
- trove-functional-mysql:
voting: false
- trove-scenario-mysql-single:
voting: false
- trove-scenario-mysql-multi:
voting: false
- trove-scenario-mariadb-single:
voting: false
- trove-scenario-mariadb-multi:
voting: false
- trove-tempest-ipv6-only: - trove-tempest-ipv6-only:
voting: false voting: false
- trove-functional-mysql-nondev:
voting: false
gate: gate:
queue: trove queue: trove
jobs: jobs:
- openstack-tox-pylint - openstack-tox-pylint
- trove-functional-mysql: - trove-tempest
voting: false
- trove-scenario-mysql-single:
voting: false
- trove-scenario-mysql-multi:
voting: false
experimental: experimental:
jobs: jobs:
- trove-grenade - trove-functional-mysql
- trove-scenario-cassandra-single - trove-scenario-mysql-single
- trove-scenario-cassandra-multi - trove-scenario-mysql-multi
- trove-scenario-couchbase-single - trove-scenario-mariadb-single
- trove-scenario-couchdb-single - trove-scenario-mariadb-multi
- trove-scenario-percona-single
- trove-scenario-percona-multi
- trove-scenario-pxc-single
- trove-scenario-pxc-multi
- trove-scenario-redis-single
- trove-scenario-redis-multi
periodic: periodic:
jobs: jobs:
- publish-trove-guest-image-mysql-ubuntu-xenial: - publish-trove-guest-image-ubuntu-bionic:
branches: branches:
- master - master
- publish-trove-guest-image-mysql-ubuntu-xenial-dev: - publish-trove-guest-image-ubuntu-bionic-dev:
branches: branches:
- master - master
@ -399,28 +376,26 @@
- openstack/tripleo-image-elements - openstack/tripleo-image-elements
- job: - job:
name: publish-trove-guest-image-mysql-ubuntu-xenial name: publish-trove-guest-image-ubuntu-bionic
parent: publish-trove-guest-image parent: publish-trove-guest-image
description: | description: |
Build and publish Ubuntu Xenial based Trove guest image to Build and publish Ubuntu Bionic based Trove guest image to
tarballs.openstack.org. tarballs.openstack.org.
vars: vars:
datastore_type: mysql
guest_os: ubuntu guest_os: ubuntu
guest_os_release: xenial guest_os_release: bionic
guest_username: ubuntu guest_username: ubuntu
branch: master branch: master
dev_mode: false dev_mode: false
image_suffix: "" image_suffix: ""
- job: - job:
name: publish-trove-guest-image-mysql-ubuntu-xenial-dev name: publish-trove-guest-image-ubuntu-bionic-dev
parent: publish-trove-guest-image parent: publish-trove-guest-image
description: | description: |
Build and publish Ubuntu Xenial based Trove guest image to Build and publish Ubuntu Xenial based Trove guest image to
tarballs.openstack.org. tarballs.openstack.org.
vars: vars:
datastore_type: mysql
guest_os: ubuntu guest_os: ubuntu
guest_os_release: xenial guest_os_release: xenial
guest_username: ubuntu guest_username: ubuntu

410
doc/source/admin/database_module_usage.rst
View File

@ -1,410 +0,0 @@
.. _database_module_usage:
=====================================
Create and use modules for a database
=====================================
To continue with this document, we recommend that you have installed
the Database service and populated your data store with images for the
type and versions of databases that you want, and that you can create
and access a database.
This example shows you how to create and apply modules to a MySQL 5.6
database and redis 3.2.6 database cluster.
Create and apply a module to a mysql database
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#. **Create the module file and trove module**
If you wish to apply a module, you must create the module first
and register it with the trove service. A user can not directly
apply a module to a trove instance.
The module created here is a demo module called ping. It is the
basic type made for testing purposes. To create it, it is as
simple as the following :command: ``echo`` command:
.. code-block:: console
$ echo "message=Module.V1" > ping1.dat
You can create a test module and mysql database with the module
applied by doing the following:
.. code-block:: console
$ trove module-create mymod ping ping1.dat --live_update \
--datastore mysql
+----------------------+--------------------------------------+
| Property | Value |
+----------------------+--------------------------------------+
| apply_order | 5 |
| auto_apply | False |
| created | 2017-06-02T17:06:21 |
| datastore | all |
| datastore_id | None |
| datastore_version | all |
| datastore_version_id | None |
| description | None |
| id | 0065a8ed-0668-4db5-a4ad-d88d0a166388 |
| instance_count | 2 |
| is_admin | True |
| live_update | True |
| md5 | 7f700cc7b99606615f8b51946f6d3228 |
| name | mymod |
| priority_apply | False |
| tenant | eac1e46e5f7840e39012aff46a92073a |
| tenant_id | eac1e46e5f7840e39012aff46a92073a |
| type | ping |
| updated | 2017-06-02T17:06:21 |
| visible | True |
+----------------------+--------------------------------------+
$ trove create myinst 15 --size 1 --module mymod --datastore mysql
+-------------------------+--------------------------------------+
| Property | Value |
+-------------------------+--------------------------------------+
| created | 2017-06-02T17:22:24 |
| datastore | mysql |
| datastore_version | 5.6 |
| encrypted_rpc_messaging | True |
| flavor | 15 |
| id | 6221b30c-8292-4378-b624-c7e9b0f8ba9e |
| name | myinst |
| region | RegionOne |
| server_id | None |
| status | BUILD |
| tenant_id | eac1e46e5f7840e39012aff46a92073a |
| updated | 2017-06-02T17:22:24 |
| volume | 1 |
| volume_id | None |
+-------------------------+--------------------------------------+
.. _show_and_list_modules:
#. **Show and list modules**
You can view the modules on your instance by doing the following:
.. code-block:: console
$ trove module-query myinst
+-------+------+-----------+---------+--------+-----------+------------------------+------------------------+
| Name | Type | Datastore | Version | Status | Message | Created | Updated |
+-------+------+-----------+---------+--------+-----------+------------------------+------------------------+
| mymod | ping | all | all | OK | Module.V1 | 2017-06-02 17:23:40.50 | 2017-06-02 17:23:40.50 |
+-------+------+-----------+---------+--------+-----------+------------------------+------------------------+
You can count the instances each module is applied to by doing the
following:
.. code-block:: console
$ trove module-instance-count mymod
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T17:22:25 | 2017-06-02T17:22:25 | 7f700cc7b99606615f8b51946f6d3228 | True | 1 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
You can list the instances that have a particular module applied
by doing the following:
.. code-block:: console
$ trove module-instances mymod
+--------------------------------------+--------+-----------+-------------------+--------+-----------+------+-----------+----------------------------------+
| ID | Name | Datastore | Datastore Version | Status | Flavor ID | Size | Region | Tenant ID |
+--------------------------------------+--------+-----------+-------------------+--------+-----------+------+-----------+----------------------------------+
| 6221b30c-8292-4378-b624-c7e9b0f8ba9e | myinst | mysql | 5.6 | ACTIVE | 15 | 1 | RegionOne | eac1e46e5f7840e39012aff46a92073a |
+--------------------------------------+--------+-----------+-------------------+--------+-----------+------+-----------+----------------------------------+
Updating and creating a second module for a redis cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To update a module you should have another file ready to update the
module with:
.. code-block:: console
$ echo "message=Module.V2" > ping2.dat
$ trove module-update mymod --file ping2.dat
+----------------------+--------------------------------------+
| Property | Value |
+----------------------+--------------------------------------+
| apply_order | 5 |
| auto_apply | False |
| created | 2017-06-02T17:06:21 |
| datastore | all |
| datastore_id | None |
| datastore_version | all |
| datastore_version_id | None |
| description | None |
| id | 0065a8ed-0668-4db5-a4ad-d88d0a166388 |
| is_admin | True |
| live_update | True |
| md5 | ba7c204979c8de54be6efb70a17d40b9 |
| name | mymod |
| priority_apply | False |
| tenant | eac1e46e5f7840e39012aff46a92073a |
| tenant_id | eac1e46e5f7840e39012aff46a92073a |
| type | ping |
| updated | 2017-06-02T17:56:12 |
| visible | True |
+----------------------+--------------------------------------+
Now to show the usage with a redis cluster, create as follows:
.. code-block:: console
$ trove cluster-create myclust redis 3.2.6 \
--instance=flavor=15,volume=1,module=mymod \
--instance=flavor=15,volume=1,module=mymod \
--instance=flavor=15,volume=1,module=mymod
+-------------------+--------------------------------------+
| Property | Value |
+-------------------+--------------------------------------+
| created | 2017-06-02T18:00:17 |
| datastore | redis |
| datastore_version | 3.2.6 |
| id | e4d91ca6-5980-430c-94d0-bf7abc63f712 |
| instance_count | 3 |
| name | myclust |
| task_description | Building the initial cluster. |
| task_name | BUILDING |
| updated | 2017-06-02T18:00:17 |
+-------------------+--------------------------------------+
The original :command: ``count`` command will show the first instance,
unless the ``--include_clustered`` option is used. You can see the
MD5 from each applied module, and you know that the single instance
one is not current.
.. code-block:: console
$ trove module-instance-count mymod
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T17:22:25 | 2017-06-02T17:22:25 | 7f700cc7b99606615f8b51946f6d3228 | False | 1 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
$ trove module-instance-count mymod --include_clustered
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T17:22:25 | 2017-06-02T17:22:25 | 7f700cc7b99606615f8b51946f6d3228 | False | 1 |
| mymod | 2017-06-02T18:00:18 | 2017-06-02T18:00:18 | ba7c204979c8de54be6efb70a17d40b9 | True | 3 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
Update the module again. By doing this, it will cause the instances
to report their module is not current.
.. code-block:: console
$ echo "message=Module.V3" > ping3.dat
$ trove module-update mymod --file ping3.dat
+----------------------+--------------------------------------+
| Property | Value |
+----------------------+--------------------------------------+
| apply_order | 5 |
| auto_apply | False |
| created | 2017-06-02T17:06:21 |
| datastore | all |
| datastore_id | None |
| datastore_version | all |
| datastore_version_id | None |
| description | None |
| id | 0065a8ed-0668-4db5-a4ad-d88d0a166388 |
| is_admin | True |
| live_update | True |
| md5 | 869744bdd18e306a96c145df562065ab |
| name | mymod |
| priority_apply | False |
| tenant | eac1e46e5f7840e39012aff46a92073a |
| tenant_id | eac1e46e5f7840e39012aff46a92073a |
| type | ping |
| updated | 2017-06-02T18:06:53 |
| visible | True |
+----------------------+--------------------------------------+
$ trove module-instance-count mymod --include_clustered
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T17:22:25 | 2017-06-02T17:22:25 | 7f700cc7b99606615f8b51946f6d3228 | False | 1 |
| mymod | 2017-06-02T18:00:18 | 2017-06-02T18:00:18 | ba7c204979c8de54be6efb70a17d40b9 | False | 3 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
To update an instance in a cluster you can use the
:command:`trove module-apply` command:
.. code-block:: console
$ trove cluster-instances myclust
+--------------------------------------+------------------+-----------+------+--------+
| ID | Name | Flavor ID | Size | Status |
+--------------------------------------+------------------+-----------+------+--------+
| 393462d5-906d-4214-af0d-538b7f618b2d | myclust-member-2 | 15 | 1 | ACTIVE |
| a3fc5326-e1b6-456a-a8b1-08ad6bbb2278 | myclust-member-3 | 15 | 1 | ACTIVE |
| cba31d4b-d038-42c2-ab03-56c6c176b49d | myclust-member-1 | 15 | 1 | ACTIVE |
+--------------------------------------+------------------+-----------+------+--------+
$ trove module-apply 393462d5-906d-4214-af0d-538b7f618b2d mymod
+-------+------+-----------+---------+--------+-----------+
| Name | Type | Datastore | Version | Status | Message |
+-------+------+-----------+---------+--------+-----------+
| mymod | ping | all | all | OK | Module.V3 |
+-------+------+-----------+---------+--------+-----------+
$ trove module-instance-count mymod --include_clustered
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T17:22:25 | 2017-06-02T17:22:25 | 7f700cc7b99606615f8b51946f6d3228 | False | 1 |
| mymod | 2017-06-02T18:00:18 | 2017-06-02T18:00:18 | ba7c204979c8de54be6efb70a17d40b9 | False | 2 |
| mymod | 2017-06-02T18:18:37 | 2017-06-02T18:18:37 | 869744bdd18e306a96c145df562065ab | True | 1 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
For variety in this example, create one more instance and module:
.. code-block:: console
$ trove create myinst_2 15 --size 1 --module mymod
+-------------------------+--------------------------------------+
| Property | Value |
+-------------------------+--------------------------------------+
| created | 2017-06-02T18:21:56 |
| datastore | redis |
| datastore_version | 3.2.6 |
| encrypted_rpc_messaging | True |
| flavor | 15 |
| id | cdd85d94-13a0-4d90-89eb-9c05523d2ac6 |
| name | myinst_2 |
| region | RegionOne |
| server_id | None |
| status | BUILD |
| tenant_id | eac1e46e5f7840e39012aff46a92073a |
| updated | 2017-06-02T18:21:56 |
| volume | 1 |
| volume_id | None |
+-------------------------+--------------------------------------+
$ echo "message=Module.V4" > ping4.dat
$ trove module-update mymod --file ping4.dat
+----------------------+--------------------------------------+
| Property | Value |
+----------------------+--------------------------------------+
| apply_order | 5 |
| auto_apply | False |
| created | 2017-06-02T17:06:21 |
| datastore | all |
| datastore_id | None |
| datastore_version | all |
| datastore_version_id | None |
| description | None |
| id | 0065a8ed-0668-4db5-a4ad-d88d0a166388 |
| is_admin | True |
| live_update | True |
| md5 | 6e2c81c1547d640b4c6e7752ed0e33ab |
| name | mymod |
| priority_apply | False |
| tenant | eac1e46e5f7840e39012aff46a92073a |
| tenant_id | eac1e46e5f7840e39012aff46a92073a |
| type | ping |
| updated | 2017-06-02T18:26:22 |
| visible | True |
+----------------------+--------------------------------------+
Now we have 2 single instances, and 3 cluster instances on various
versions of the module, none current.
.. code-block:: console
$ trove list
+--------------------------------------+----------+-----------+-------------------+--------+-----------+------+-----------+
| ID | Name | Datastore | Datastore Version | Status | Flavor ID | Size | Region |
+--------------------------------------+----------+-----------+-------------------+--------+-----------+------+-----------+
| 6221b30c-8292-4378-b624-c7e9b0f8ba9e | myinst | mysql | 5.6 | ACTIVE | 15 | 1 | RegionOne |
| cdd85d94-13a0-4d90-89eb-9c05523d2ac6 | myinst_2 | redis | 3.2.6 | ACTIVE | 15 | 1 | RegionOne |
+--------------------------------------+----------+-----------+-------------------+--------+-----------+------+-----------+
$ trove module-instance-count mymod --include_clustered
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T17:22:25 | 2017-06-02T17:22:25 | 7f700cc7b99606615f8b51946f6d3228 | False | 1 |
| mymod | 2017-06-02T18:00:18 | 2017-06-02T18:00:18 | ba7c204979c8de54be6efb70a17d40b9 | False | 2 |
| mymod | 2017-06-02T18:18:37 | 2017-06-02T18:21:57 | 869744bdd18e306a96c145df562065ab | False | 2 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
When the latest module was created, the ``--include_clustered`` was
not used. Use the :command:`trove module-reapply` command:
.. code-block:: console
$ trove module-reapply mymod --md5=869744bdd18e306a96c145df562065ab --include_clustered
$ trove module-instance-count mymod --include_clustered
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T17:22:25 | 2017-06-02T17:22:25 | 7f700cc7b99606615f8b51946f6d3228 | False | 1 |
| mymod | 2017-06-02T18:00:18 | 2017-06-02T18:00:18 | ba7c204979c8de54be6efb70a17d40b9 | False | 2 |
| mymod | 2017-06-02T18:38:48 | 2017-06-02T18:38:48 | 6e2c81c1547d640b4c6e7752ed0e33ab | True | 2 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
Now they are both updated. If the ``--force`` flag is used, it can
reapply to already applied instances. Notice that the only thing that
changes is the minimum and maximum updated date fields.
.. code-block:: console
$ trove module-reapply mymod --md5=6e2c81c1547d640b4c6e7752ed0e33ab --include_clustered --force
$ trove module-instance-count mymod --include_clustered
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T17:22:25 | 2017-06-02T17:22:25 | 7f700cc7b99606615f8b51946f6d3228 | False | 1 |
| mymod | 2017-06-02T18:00:18 | 2017-06-02T18:00:18 | ba7c204979c8de54be6efb70a17d40b9 | False | 2 |
| mymod | 2017-06-02T18:40:45 | 2017-06-02T18:40:46 | 6e2c81c1547d640b4c6e7752ed0e33ab | True | 2 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
To bring every instance to the current version, use some of the
optional arguments to control how many instances are updated at the
same time. This is useful to avoid potential network issues, if the
module payload is large. Since we are not using the ``--force`` flag,
the minimum updated date will not change.
.. code-block:: console
$ trove module-reapply mymod --include_clustered --batch_size=1 --delay=3
$ trove module-instance-count mymod --include_clustered
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| Module Name | Min Updated Date | Max Updated Date | Module MD5 | Current | Count |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+
| mymod | 2017-06-02T18:40:45 | 2017-06-02T18:44:10 | 6e2c81c1547d640b4c6e7752ed0e33ab | True | 5 |
+-------------+---------------------+---------------------+----------------------------------+---------+-------+

286
doc/source/admin/datastore.rst
View File

@ -20,91 +20,44 @@ The Database service provides resource isolation at high performance
levels, and automates complex administrative tasks such as deployment, levels, and automates complex administrative tasks such as deployment,
configuration, patching, backups, restores, and monitoring. configuration, patching, backups, restores, and monitoring.
You can modify various cluster characteristics by editing the Create datastore
``/etc/trove/trove.conf`` file. A comprehensive list of the Database ~~~~~~~~~~~~~~~~
service configuration options is described in the `Database service
<https://docs.openstack.org/ocata/config-reference/database.html>`_
chapter in the *Configuration Reference*.
Create a data store An administrative user can create datastores for a variety of databases.
~~~~~~~~~~~~~~~~~~~
An administrative user can create data stores for a variety of
databases.
This section assumes you do not yet have a MySQL data store, and shows This section assumes you do not yet have a MySQL data store, and shows
you how to create a MySQL data store and populate it with a MySQL 5.5 you how to create a MySQL data store and populate it with a MySQL 5.5
data store version. data store version.
.. note::
From Victoria release, all the datastores can be configured with a same
Glance image but with different datastore name and version number.
**To create a data store** **To create a data store**
#. **Create a trove image** #. **Create a trove image**
Create an image for the type of database you want to use, for Refer to `Build images using trovestack
example, MySQL, MongoDB, Cassandra. <https://docs.openstack.org/trove/latest/admin/building_guest_images.html#build-images-using-trovestack>`_
This image must have the trove guest agent installed, and it must
have the ``trove-guestagent.conf`` file configured to connect to
your OpenStack environment. To configure ``trove-guestagent.conf``,
add the following lines to ``trove-guestagent.conf`` on the guest
instance you are using to build your image:
.. code-block:: ini
rabbit_host = controller
rabbit_password = RABBIT_PASS
trove_auth_url = http://controller:35357/v2.0
This example assumes you have created a MySQL 5.5 image called
``mysql-5.5.qcow2``.
.. important::
If you have a guest image that was created with an OpenStack version
before Kilo, modify the guest agent init script for the guest image to
read the configuration files from the directory ``/etc/trove/conf.d``.
For a backwards compatibility with pre-Kilo guest instances, set the
database service configuration options ``injected_config_location`` to
``/etc/trove`` and ``guest_info`` to ``/etc/guest_info``.
#. **Register image with Image service** #. **Register image with Image service**
You need to register your guest image with the Image service. You need to register your guest image with the Image service as cloud admin.
In this example, you use the :command:`openstack image create`
command to register a ``mysql-5.5.qcow2`` image.
.. code-block:: console .. code-block:: console
$ openstack image create mysql-5.5 --disk-format qcow2 --container-format bare --public < mysql-5.5.qcow2 openstack image create \
+------------------+------------------------------------------------------+ trove-guest-ubuntu-bionic \
| Field | Value | --private \
+------------------+------------------------------------------------------+ --disk-format qcow2 --container-format bare \
| checksum | 133eae9fb1c98f45894a4e60d8736619 | --file $image_file \
| container_format | bare | --property hw_rng_model='virtio' \
| created_at | 2016-12-21T12:10:02Z | --tag trove
| disk_format | qcow2 |
| file | /v2/images/d1afb4f0-2360-4400-8d97-846b1ab6af52/file |
| id | d1afb4f0-2360-4400-8d97-846b1ab6af52 |
| min_disk | 0 |
| min_ram | 0 |
| name | mysql-5.5 |
| owner | 5669caad86a04256994cdf755df4d3c1 |
| protected | False |
| schema | /v2/schemas/image |
| size | 13200896 |
| status | active |
| tags | |
| updated_at | 2016-12-21T12:10:03Z |
| virtual_size | None |
| visibility | public |
+------------------+------------------------------------------------------+
#. **Create the datastore** #. **Create the datastore**
Create the data store that will house the new image. To do this, use Create the data store that configured with the new image. To do this, use
the :command:`trove-manage` :command:`datastore_update` command. the :command:`trove-manage` :command:`datastore_update` command.
This example uses the following arguments: This example uses the following arguments:
@ -228,23 +181,14 @@ data store version.
#. **Load validation rules for configuration groups** #. **Load validation rules for configuration groups**
.. note::
**Applies only to MySQL and Percona data stores**
* If you just created a MySQL or Percona data store, then you need
to load the appropriate validation rules, as described in this
step.
* If you just created a different data store, skip this step.
**Background.** You can manage database configuration tasks by using **Background.** You can manage database configuration tasks by using
configuration groups. Configuration groups let you set configuration configuration groups. Configuration groups let you set configuration
parameters, in bulk, on one or more databases. parameters, in bulk, on one or more databases.
When you set up a configuration group using the trove When you set up a configuration group using the :command:`openstack database
:command:`configuration-create` command, this command compares the configuration configuration create` command, this command compares the configuration
values you are setting against a list of valid configuration values values you are setting against a list of valid configuration values that are
that are stored in the ``validation-rules.json`` file. stored in the ``validation-rules.json`` file.
.. list-table:: .. list-table::
:header-rows: 1 :header-rows: 1
@ -292,7 +236,7 @@ data store version.
.. code-block:: console .. code-block:: console
$ trove datastore-list $ openstack datastore list
+--------------------------------------+--------------+ +--------------------------------------+--------------+
| id | name | | id | name |
+--------------------------------------+--------------+ +--------------------------------------+--------------+
@ -300,193 +244,13 @@ data store version.
| e5dc1da3-f080-4589-a4c2-eff7928f969a | mysql | | e5dc1da3-f080-4589-a4c2-eff7928f969a | mysql |
+--------------------------------------+--------------+ +--------------------------------------+--------------+
Take the ID of the MySQL data store and pass it in with the Show the versions of a specific datastore:
:command:`datastore-version-list` command:
.. code-block:: console .. code-block:: console
$ trove datastore-version-list DATASTORE_ID $ openstack datastore version list mysql
+--------------------------------------+-----------+ +--------------------------------------+-----------+
| id | name | | id | name |
+--------------------------------------+-----------+ +--------------------------------------+-----------+
| 36a6306b-efd8-4d83-9b75-8b30dd756381 | mysql-5.5 | | 36a6306b-efd8-4d83-9b75-8b30dd756381 | mysql-5.5 |
+--------------------------------------+-----------+ +--------------------------------------+-----------+
Data store classifications
--------------------------
The Database service supports a variety of both relational and
non-relational database engines, but to a varying degree of support for
each *data store*. The Database service project has defined
several classifications that indicate the quality of support for each
data store. Data stores also implement different extensions.
An extension is called a *strategy* and is classified similar to
data stores.
Valid classifications for a data store and a strategy are:
* Experimental
* Technical preview
* Stable
Each classification builds on the previous one. This means that a data store
that meets the ``technical preview`` requirements must also meet all the
requirements for ``experimental``, and a data store that meets the ``stable``
requirements must also meet all the requirements for ``technical preview``.
**Requirements**
* Experimental
A data store is considered to be ``experimental`` if it meets these criteria:
* It implements a basic subset of the Database service API including
``create`` and ``delete``.
* It has guest agent elements that allow guest agent creation.
* It has a definition of supported operating systems.
* It meets the other
`Documented Technical Requirements <https://specs.openstack.org/openstack/trove-specs/specs/kilo/experimental-datastores.html#requirements>`_.
A strategy is considered ``experimental`` if:
* It meets the
`Documented Technical Requirements <https://specs.openstack.org/openstack/trove-specs/specs/kilo/experimental-datastores.html#requirements>`_.
* Technical preview
A data store is considered to be a ``technical preview`` if it meets the
requirements of ``experimental`` and further:
* It implements APIs required to plant and start the capabilities of the
data store as defined in the
`Datastore Compatibility Matrix <https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix>`_.
.. note::
It is not required that the data store implements all features like
resize, backup, replication, or clustering to meet this classification.
* It provides a mechanism for building a guest image that allows you to
exercise its capabilities.
* It meets the other
`Documented Technical Requirements <https://specs.openstack.org/openstack/trove-specs/specs/kilo/experimental-datastores.html#requirements>`_.
.. important::
A strategy is not normally considered to be ``technical
preview``.
* Stable
A data store or a strategy is considered ``stable`` if:
* It meets the requirements of ``technical preview``.
* It meets the other
`Documented Technical Requirements <https://specs.openstack.org/openstack/trove-specs/specs/kilo/experimental-datastores.html#requirements>`_.
**Initial Classifications**
The following table shows the current classification assignments for the
different data stores.
.. list-table::
:header-rows: 1
:widths: 30 30
* - Classification
- Data store
* - Stable
- MySQL
* - Technical Preview
- Cassandra, MongoDB
* - Experimental
- All others
Redis data store replication
----------------------------
Replication strategies are available for Redis with
several commands located in the Redis data store
manager:
- :command:`create`
- :command:`detach-replica`
- :command:`eject-replica-source`
- :command:`promote-to-replica-source`
Additional arguments for the :command:`create` command
include :command:`--replica_of` and
:command:`--replica_count`.
Redis integration and unit tests
--------------------------------
Unit tests and integration tests are also available for
Redis.
#. Install trovestack:
.. code-block:: console
$ ./trovestack install
.. note::
Trovestack is a development script used for integration
testing and Database service development installations.
Do not use Trovestack in a production environment. For
more information, see `the Database service
developer docs <https://docs.openstack.org/developer/trove/dev/install.html#running-trovestack-to-setup-trove>`_
#. Start Redis:
.. code-block:: console
$ ./trovestack kick-start redis
#. Run integration tests:
.. code-block:: console
$ ./trovestack int-tests --group=replication
You can run :command:`--group=redis_supported`
instead of :command:`--group=replication` if needed.
Configure a cluster
~~~~~~~~~~~~~~~~~~~
An administrative user can configure various characteristics of a
MongoDB cluster.
**Query routers and config servers**
**Background.** Each cluster includes at least one query router and
one config server. Query routers and config servers count against your
quota. When you delete a cluster, the system deletes the associated
query router(s) and config server(s).
**Configuration.** By default, the system creates one query router and
one config server per cluster. You can change this by editing
the ``/etc/trove/trove.conf`` file. These settings are in the
``mongodb`` section of the file:
.. list-table::
:header-rows: 1
:widths: 30 30
* - Setting
- Valid values are:
* - num_config_servers_per_cluster
- 1 or 3
* - num_query_routers_per_cluster
- 1 or 3

1
doc/source/admin/index.rst
View File

@ -9,5 +9,4 @@
datastore datastore
building_guest_images building_guest_images
secure_oslo_messaging secure_oslo_messaging
database_module_usage
troubleshooting troubleshooting

4
doc/source/admin/run_trove_in_production.rst
View File

@ -287,6 +287,10 @@ particular data store.
used in a production environment. The images are available for download and used in a production environment. The images are available for download and
are located at http://tarballs.openstack.org/trove/images/. are located at http://tarballs.openstack.org/trove/images/.
From Victoria release, Trove uses a single guest image for all the supported
datastores. Database service is running as docker container inside the trove
instance which simplifies the datastore management and maintenance.
For use with production systems, it is recommended to create and maintain your For use with production systems, it is recommended to create and maintain your
own images in order to conform to standards set by the company's security team. own images in order to conform to standards set by the company's security team.
In Trove community, we use `Disk Image Builder(DIB) In Trove community, we use `Disk Image Builder(DIB)

4
doc/source/admin/troubleshooting.rst
View File

@ -12,7 +12,7 @@ The possible reasons for this issue:
to the message queue, which is expected to be received and handled by the to the message queue, which is expected to be received and handled by the
trove-guestagent service which is running inside the instance. The instance trove-guestagent service which is running inside the instance. The instance
status should be updated by trove-guestagent service after handling. status should be updated by trove-guestagent service after handling.
Apparently, If the trove-taskmanager can't connect with RabbitMQ, the Apparently, If the trove-guestagent can't connect with RabbitMQ, the
instance status won't be updated. instance status won't be updated.
* Code bug in trove-guestagent. You should be able to see some error log in * Code bug in trove-guestagent. You should be able to see some error log in
trove-guestagent log file (by default, trove-guestagent log file (by default,
@ -55,6 +55,6 @@ openstack-discuss@lists.openstack.org if help needed.
.. note:: .. note::
The Trove instance creation time varies in different environments, the The Trove instance creation time varies in different environments, the
default value of ``usage_timeout`` option (1800 seconds) may not be applied default value of ``usage_timeout`` option (3600 seconds) may not be applied
to all, the cloud administrator should change that based on testing so that to all, the cloud administrator should change that based on testing so that
the instance creation should fail in a reasonable timely manner. the instance creation should fail in a reasonable timely manner.

15
doc/source/user/backup-db-incremental.rst
View File

@ -10,6 +10,16 @@ Restoring a database instance from an incremental backup is the same as
creating a database instance from a regular backup—the Database service creating a database instance from a regular backup—the Database service
handles the complexities of applying the chain of incremental backups. handles the complexities of applying the chain of incremental backups.
The artifacts created by backup are stored in OpenStack Swift, by default in a
container named 'database_backups'. As the end user, you are able to access all
the objects but make sure not to delete those objects manually. When a backup
is deleted in Trove, the related objects are automatically removed from Swift.
.. caution::
If the objects in 'database_backups' container are deleted manually, the
database can't be properly restored.
This example shows you how to use incremental backups with a MySQL This example shows you how to use incremental backups with a MySQL
database. database.
@ -44,7 +54,6 @@ Create and use incremental backups
.. code-block:: console .. code-block:: console
$ openstack database backup create INSTANCE_ID backup1.1 --parent BACKUP_ID $ openstack database backup create INSTANCE_ID backup1.1 --parent BACKUP_ID
+-------------+--------------------------------------+ +-------------+--------------------------------------+
| Property | Value | | Property | Value |
+-------------+--------------------------------------+ +-------------+--------------------------------------+
@ -74,7 +83,6 @@ Create and use incremental backups
.. code-block:: console .. code-block:: console
$ openstack database backup create INSTANCE_ID backup1.2 --parent BACKUP_ID $ openstack database backup create INSTANCE_ID backup1.2 --parent BACKUP_ID
+-------------+--------------------------------------+ +-------------+--------------------------------------+
| Property | Value | | Property | Value |
+-------------+--------------------------------------+ +-------------+--------------------------------------+
@ -105,8 +113,7 @@ Create and use incremental backups
.. code-block:: console .. code-block:: console
$ openstack database instance create guest2 10 --size 1 --backup BACKUP_ID $ openstack database instance create guest2 10 --size 1 --nic net-id=$network_id --backup BACKUP_ID
+-------------------+-----------------------------------------------------------+ +-------------------+-----------------------------------------------------------+
| Property | Value | | Property | Value |
+-------------------+-----------------------------------------------------------+ +-------------------+-----------------------------------------------------------+

19
doc/source/user/backup-db.rst
View File

@ -7,6 +7,16 @@ artifact in the Object Storage service. Later on, if the original
database is damaged, you can use the backup artifact to restore the database is damaged, you can use the backup artifact to restore the
database. The restore process creates a database instance. database. The restore process creates a database instance.
The artifacts created by backup are stored in OpenStack Swift, by default in a
container named 'database_backups'. As the end user, you are able to access all
the objects but make sure not to delete those objects manually. When a backup
is deleted in Trove, the related objects are automatically removed from Swift.
.. caution::
If the objects in 'database_backups' container are deleted manually, the
database can't be properly restored.
This example shows you how to back up and restore a MySQL database. This example shows you how to back up and restore a MySQL database.
#. **Backup the database instance** #. **Backup the database instance**
@ -31,7 +41,6 @@ This example shows you how to back up and restore a MySQL database.
.. code-block:: console .. code-block:: console
$ openstack database instance list $ openstack database instance list
+--------------------------------------+--------+-----------+-------------------+--------+-----------+------+ +--------------------------------------+--------+-----------+-------------------+--------+-----------+------+
| id | name | datastore | datastore_version | status | flavor_id | size | | id | name | datastore | datastore_version | status | flavor_id | size |
+--------------------------------------+--------+-----------+-------------------+--------+-----------+------+ +--------------------------------------+--------+-----------+-------------------+--------+-----------+------+
@ -52,7 +61,6 @@ This example shows you how to back up and restore a MySQL database.
.. code-block:: console .. code-block:: console
$ openstack database backup create INSTANCE_ID backup1 $ openstack database backup create INSTANCE_ID backup1
+-------------+--------------------------------------+ +-------------+--------------------------------------+
| Property | Value | | Property | Value |
+-------------+--------------------------------------+ +-------------+--------------------------------------+
@ -90,7 +98,6 @@ This example shows you how to back up and restore a MySQL database.
.. code-block:: console .. code-block:: console
$ openstack database backup show BACKUP_ID $ openstack database backup show BACKUP_ID
+-------------+----------------------------------------------------+ +-------------+----------------------------------------------------+
| Property | Value | | Property | Value |
+-------------+----------------------------------------------------+ +-------------+----------------------------------------------------+
@ -123,8 +130,7 @@ This example shows you how to back up and restore a MySQL database.
.. code-block:: console .. code-block:: console
$ openstack database instance create guest2 10 --size 2 --backup BACKUP_ID $ openstack database instance create guest2 10 --size 2 --nic net-id=$network_id --backup BACKUP_ID
+-------------------+----------------------------------------------+ +-------------------+----------------------------------------------+
| Property | Value | | Property | Value |
+-------------------+----------------------------------------------+ +-------------------+----------------------------------------------+
@ -164,7 +170,6 @@ This example shows you how to back up and restore a MySQL database.
.. code-block:: console .. code-block:: console
$ openstack database instance show INSTANCE_ID $ openstack database instance show INSTANCE_ID
+-------------------+--------------------------------------+ +-------------------+--------------------------------------+
| Property | Value | | Property | Value |
+-------------------+--------------------------------------+ +-------------------+--------------------------------------+
@ -190,7 +195,6 @@ This example shows you how to back up and restore a MySQL database.
.. code-block:: console .. code-block:: console
$ openstack database db list INSTANCE_ID $ openstack database db list INSTANCE_ID
+--------------------+ +--------------------+
| name | | name |
+--------------------+ +--------------------+
@ -206,7 +210,6 @@ This example shows you how to back up and restore a MySQL database.
.. code-block:: console .. code-block:: console
$ openstack database user list INSTANCE_ID $ openstack database user list INSTANCE_ID
+--------+------+-----------+ +--------+------+-----------+
| name | host | databases | | name | host | databases |
+--------+------+-----------+ +--------+------+-----------+

15
doc/source/user/manage-db-config.rst
View File

@ -28,7 +28,6 @@ Bulk-configure a database or databases
.. code-block:: console .. code-block:: console
$ openstack datastore version list mysql $ openstack datastore version list mysql
+--------------------------------------+-----------+ +--------------------------------------+-----------+
| id | name | | id | name |
+--------------------------------------+-----------+ +--------------------------------------+-----------+
@ -42,7 +41,6 @@ Bulk-configure a database or databases
.. code-block:: console .. code-block:: console
$ openstack database configuration parameter list DATASTORE_VERSION_ID $ openstack database configuration parameter list DATASTORE_VERSION_ID
+--------------------------------+---------+---------+----------------------+------------------+ +--------------------------------+---------+---------+----------------------+------------------+
| name | type | min | max | restart_required | | name | type | min | max | restart_required |
+--------------------------------+---------+---------+----------------------+------------------+ +--------------------------------+---------+---------+----------------------+------------------+
@ -100,16 +98,14 @@ Bulk-configure a database or databases
- *NAME*. The name you want to use for this group. - *NAME*. The name you want to use for this group.
- *VALUES*. The list of key-value pairs. - *VALUES*. The list of key-value pairs. Set *VALUES* as a JSON dictionary, for example:
- *DATASTORE_NAME*. The name of the associated data store.
Set *VALUES* as a JSON dictionary, for example:
.. code-block:: json .. code-block:: json
{"myFirstKey" : "someString", "mySecondKey" : 1} {"myFirstKey" : "someString", "mySecondKey" : 1}
- *DATASTORE_NAME*. The name of the associated data store.
This example creates a configuration group called ``group1``. This example creates a configuration group called ``group1``.
``group1`` contains just one key and value pair, and this pair sets ``group1`` contains just one key and value pair, and this pair sets
the ``sync_binlog`` option to ``1``. the ``sync_binlog`` option to ``1``.
@ -117,7 +113,6 @@ Bulk-configure a database or databases
.. code-block:: console .. code-block:: console
$ openstack database configuration create group1 '{"sync_binlog" : 1}' --datastore mysql $ openstack database configuration create group1 '{"sync_binlog" : 1}' --datastore mysql
+----------------------+--------------------------------------+ +----------------------+--------------------------------------+
| Property | Value | | Property | Value |
+----------------------+--------------------------------------+ +----------------------+--------------------------------------+
@ -162,7 +157,6 @@ Bulk-configure a database or databases
.. code-block:: console .. code-block:: console
$ openstack database instance list $ openstack database instance list
+-------------+------------------+-----------+-------------------+--------+-----------+------+ +-------------+------------------+-----------+-------------------+--------+-----------+------+
| id | name | datastore | datastore_version | status | flavor_id | size | | id | name | datastore | datastore_version | status | flavor_id | size |
+-------------+------------------+-----------+-------------------+--------+-----------+------+ +-------------+------------------+-----------+-------------------+--------+-----------+------+
@ -174,7 +168,6 @@ Bulk-configure a database or databases
.. code-block:: console .. code-block:: console
$ openstack database configuration list $ openstack database configuration list
+-------------+--------+-------------+---------------------+ +-------------+--------+-------------+---------------------+
| id | name | description |datastore_version_id | | id | name | description |datastore_version_id |
+-------------+--------+-------------+---------------------+ +-------------+--------+-------------+---------------------+
@ -228,7 +221,7 @@ features for working with configuration groups. You can:
the :command:`openstack database configuration detach` command. the :command:`openstack database configuration detach` command.
- Modify a configuration group on the fly, using the - Modify a configuration group on the fly, using the
:command:`trove configuration-patch` command. :command:`openstack database configuration parameter set` command.
- Find out what instances are using a configuration group, using the - Find out what instances are using a configuration group, using the
:command:`openstack database configuration instances` command. :command:`openstack database configuration instances` command.

2
doc/source/user/set-up-replication.rst
View File

@ -43,7 +43,7 @@ Set up replication
.. code-block:: console .. code-block:: console
$ openstack database instance create replica_1 6 --size=5 \ $ openstack database instance create replica_1 6 --size=5 --nic net-id=$netid \
--datastore_version mysql-5.5 \ --datastore_version mysql-5.5 \
--datastore mysql --replica_of ID_OF_ORIGINAL_INSTANCE --datastore mysql --replica_of ID_OF_ORIGINAL_INSTANCE

2
integration/scripts/trovestack
View File

@ -769,7 +769,7 @@ function cmd_build_and_upload_image() {
glance_imageid=$(openstack ${CLOUD_ADMIN_ARG} image list --name $name -f value -c ID) glance_imageid=$(openstack ${CLOUD_ADMIN_ARG} image list --name $name -f value -c ID)
if [[ -z ${glance_imageid} ]]; then if [[ -z ${glance_imageid} ]]; then
mkdir -p ${output_dir} mkdir -p ${output_dir}
output=${output_dir}/${name} output=${output_dir}/${name}.qcow2
cmd_build_image ${guest_os} ${guest_release} ${dev_mode} ${guest_username} ${output} cmd_build_image ${guest_os} ${guest_release} ${dev_mode} ${guest_username} ${output}
glance_imageid=$(openstack ${CLOUD_ADMIN_ARG} image create ${name} --public --disk-format qcow2 --container-format bare --file ${output} --property hw_rng_model='virtio' --tag trove -c id -f value) glance_imageid=$(openstack ${CLOUD_ADMIN_ARG} image create ${name} --public --disk-format qcow2 --container-format bare --file ${output} --property hw_rng_model='virtio' --tag trove -c id -f value)

15
releasenotes/notes/victoria-database-containerization.yaml Normal file
View File

@ -0,0 +1,15 @@
---
features:
- Database service (mysql and mariadb) is now running as docker container
inside the trove instance. The image is defined by ``docker_image`` config
option for each datastore.
- The database backup and restore operations are performed by docker
container inside the trove instance.
- Only one trove guest image is needed for all the datastores.
upgrade:
- Existing database services are not affected. However, in order for
Trove to communicate with trove guest agent, new guest image needs to be
built and existing trove instances need to be backed up and restored.
deprecations:
- Most of the options related to backup and restore are removed, e.g.
backup_namespace, restore_namespace, backup_incremental_strategy

6
trove/common/apischema.py
View File

@ -411,9 +411,9 @@ instance = {
"uniqueItems": True, "uniqueItems": True,
"items": { "items": {
"type": "string", "type": "string",
"pattern": "^([0-9]{1,3}\.){3}[0-9]{1,3}" "pattern": "^([0-9]{1,3}\\.){3}[0-9]{1,3}"
"(\/([0-9]|[1-2][0-9]|3[0-2]))?" "(\\/([0-9]|[1-2][0-9]|3[0-2]))"
"$" "?$"
} }
} }
} }