Use `management_networks` instead. `management_networks`will be used
as admin networks which will be attached to Trove instance
automatically.
Change-Id: I5c6004b568c3a428bc0f0a8b0e36665d3c5b3087
Support for the [oslo_messaging_rabbit] section has been removed in
oslo.messaging==9.0.0 package [1].
That's why integration tests failures on build devstack step.
This is the moment where we need to use the transport_url directive
in the [DEFAULT] section instead.
Moreover rpc_backend property (which was used in trove to enable fake
RPC backend) has been removed from DEFAULT section and API tests
fails on TIME_OUT as they starts on rabbit RPC backend.
Fake RPC can be now configured as described here [2]:
transport_url = 'fake:/'
[1] https://docs.openstack.org/releasenotes/oslo.messaging/unreleased.html
[2] https://docs.openstack.org/oslo.messaging/latest/configuration/conffixture.html
Change-Id: Id6c5a9198d5a213cb085407a1d8b534e7c755f69
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.
[1]https://review.openstack.org/#/c/508522/
Change-Id: I46bb0acaf1dec9bb4b91d4a2180b3e7ed66741c2
A new entrypoint in setup.cfg and a config file are added for
using olso.policy helper script to generate the sample file.
A new tox target also is added to simplify the environment
setting up. Now policy sample file can be generated
automatically, so the in-repo sample file is no longer needed.
Co-Authored-By: Andrew Laski <andrew@lascii.com>
Partial-Implements: blueprint policy-in-code
Change-Id: Ic336fa154ccc05b5e9db3a8e751a484b1cc5aa9c
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
Current Nova server volume support is broken. Nova also declared the
'os-volumes_boot' will be deprecated in the future. As creating volumes
by cinderclient has been supoorted for a long time, we could just drop
support of Nova server volume.
This patch also migrate to the new block_device_mapping_v2 parameter of
Nova servers creating API.
Closes-Bug: #1673408
Change-Id: I74d86241a5a0d0b1804b959313432168f68faf89
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
Sometime,huge notification messages will effect the rabbitMQ cluster's
performance.This commit adds functionality do define different Message
and Notification destination.
Closes-Bug: #1734823
Change-Id: I2ef4d008c9ca91a03d8c7e1380e0188bf6792595
We already had default rule in code, so we should not
still define all of them again in policy file.
Besides, we should you yaml format for now instead json.
Another thing, we don't need to config policy file in
Devstack enviroment.
Change-Id: I783ba51695271d358764557899fe91e84620556d
The option oslo_messaging_rabblt.rabbit_password has been
deprecated. Therefore the option is commented out in the
sample config files etc/trove/ directory.
Change-Id: I613087217c88b8a5dbf8dfc3f51e48b8fe4c84c8
Remove notifier_queue_* because the options are no longer needed and
should be removed.
Closes-bug: #1706528
Change-Id: Idfd205473f1756a69d0d9607e9f8c1dd9780553c
The exists_notification_ticks conf option was removed under
commit 08dc866fb241f535cce0609b02a34853882538f5 but was not removed from the
sample configuration file.
Change-Id: I3608929bd1ce4cd6ec767665b6f60fd5d86127b4
This is a point that is easily hidden. In the case of not open
port 16379, redis cluster can still be deployed successfully,
but this success is a false success, there is no data communication
between the various nodes. This is because 16379 is the port for
data port[1]. We set the default value in cfg, but in the
case of conf,it is easy to be covered. Adding 16379 to conf.sample
helps the user to reduce some problem caused by the redis
configuration.
[1]:https://redis.io/topics/cluster-tutorial
Change-Id: If517072c1c875df68106af14dac1802bb959d17e
This change enables behind mod-wsgi as part of the community pike goal
goal-deploy-api-in-wsgi.
The change includes:
- the wsgi script files to run trove api under apache
- updates to the devstack plugin
- a basic deploy doc which explains this change
Change-Id: Icdd39b47a1be426e87a5d09f9e9d567af1974a9c
Depends-On: I3d6f6649430ee40879de15fee0b215dc32e8b666
Closes-Bug: #1681478
Trove's code was setup to use :5000 and :35357 for keystone's auth end
points. Change that to reflect the new settings that are /identity/
and /identity_admin/. See also [1]
[1] https://review.openstack.org/#/c/456344/
Change-Id: I3d6f6649430ee40879de15fee0b215dc32e8b666
Related: I46294fb24e3c23fa19fcfd7d6c9ee8a932354702
Server side support for the new 'reapply' command.
This reapplies a given module to all instances that it had
previously been applied to.
Originally, a module designated live-update would automatically
be re-applied whenever it was updated. Adding a specific
command however, allows operators/users more control over
how the new payload would be distributed. Old 'modules'
could be left if desired, or updated with the new command.
Scenario tests were updated to test the new command.
DocImpact: update documentation to reflect module-reapply command
Change-Id: I4aea674ebe873a96ed22b5714263d0eea532a4ca
Depends-On: Ic4cc9e9085cb40f1afbec05caeb04886137027a4
Closes-Bug: #1554903
The Oslo Policy library provides support for RBAC policy
enforcement across all OpenStack services.
Update the devstack plugin to copy the default policy file
over to /etc/trove in the gate environments.
Note: Not adding a rule for 'reset-password' instance
action as that API was discontinued years ago
and is now just waiting for removal (Bug: 1645866).
DocImpact
Co-Authored-By: Ali Adil <aadil@tesora.com>
Change-Id: Ic443a4c663301840406cad537159eab7b0b5ed1c
Implements: blueprint trove-policy
Currently Trove supports full offline backups for DB2. In this
implementation, we have added support for full online backups for DB2
using archival logging.
Change-Id: I30b5b0b85120fd105cc3db57983b062fad5fab5a
Implements: blueprint db2-online-backup
A Mistral workbook adding a workflow for creating a backup.
This is used by the "trove schedule-create" command to schedule
a backup.
Change-Id: I68d997abf4ec7c32044dd18cf2a70e687c0fed9f
Implements: blueprint schedule-backups
With this email[0], you must migrate API reference docs into RST. The
conf.py and the tox environment are also cribbed from nova.
Still need to retain the install_command in tox.ini, otherwise the
api-ref job fails.
[0] http://lists.openstack.org/pipermail/openstack-dev/2016-May/093765.html
Co-Authored-By: Anne Gentle <agentle@cisco.com>
Co-Authored-By: Amrith Kumar <amrith@tesora.com>
Change-Id: I3315261aa18729fa7a6aa79d4a1d6c24de1e2c6b
This changes the default setting for use_nova_server_config_drive
from False to True. By default, nova does not use a config
drive for launched instances. Therefore trove must implicitly
request the config drive be used so that it can inject
guest_info.conf into the guest VM.
This commit also adds an error to guestagent when guest_id
is missing. If the guest_id configuration parameter is not
set at guestagent startup it will throw a RuntimeError. This
case typically occurs because guest_info.conf was
not injected into the guest, or was not included in the set
of configs that guestagent reads at startup.
And finally, this commit adds a section in the 'building guest
images' documentation describing how configuration is injected
into the guest agent.
Change-Id: I084c841472183893a63ca7b70d560f44a2f08901
Partial-Bug: 1609915
This change introduces new datastore option "icmp" to
configure whether to permit ICMP. It helps users to
check DB instance health in different way from access
DB ports.
DocImpact
Closes-Bug: #1485884
Change-Id: I61edeb38ded5543b7976a01363108a7b5b4fc5b5
This commit enables to handle HTTP_X_FORWARDED_PROTO by using
http_proxy_to_wsgi middleware of oslo.middleware.
Change-Id: I6a11c8470205ca78bdb027fa9a06fec3acda33ad
Closes-Bug: #1590608
As discussed in the Liberty Design Summit "Moving apps to Python 3"
cross-project workshop, the way forward in the near future is to
switch to the pure-python PyMySQL library as a default.
https://etherpad.openstack.org/p/liberty-cross-project-python3
BaseMySqlRootAccess.enable_root(): catch also InternalError because
the PyMySQL error is not wrapped into a SQLAlchemy OperationalError,
but a generic SQLAlchemy InternalError. Similar change is made in
026_datastore_versions_unique_fix.py.
This change requires a trove integration change to add the PyMySQL to
the guest image: Id4d013d174ba40a453819f900aaa316a93e59b48.
Partially implements: blueprint trove-python3
Co-Authored-By: Victor Stinner <vstinner@redhat.com>
Depends-On: Id4d013d174ba40a453819f900aaa316a93e59b48
Change-Id: I65e8a8d5dc251a8b00529cdfb1a6ada3d5720f68
The recommended method for doing full backups in CouchDB
has been a simple filesystem copy of the data files. This
is because CouchDB stores data in wholly contained append
only files. For example, when a user creates a database,
a corresponding <database-name>.couch file is created in
the database directory.
The backup functionality has been implemented by compressing
the database directory and then encrypting it and sending it
over to store in Swift. Similarly, the restore functionality
has been implemented by fetching the files from Swift and
uncompressing them into the database directory. After this,
the ownership of the directory needs to be updated.
To test the changes, follow the steps:
- Create a CouchDB instance
- Access the admin console called Futon using the following
url: http://10.0.0.5:5984/_utils/
- Create a database from there and create one or more documents
- Create a backup of this CouchDB instance
- Create another CouchDB instance from the backup created above
- Access the admin console for this new instance and verify that
the database created above is there
couchdb client library for the integration tests has been added
to global-requirements: https://review.openstack.org/#/c/285191/
Change-Id: Iad6d69bf60ace73825819081964a43ad53d6c6fc
Implements: blueprint couchdb-backup-restore
Implementation of backup and restore functionality for db2
databases. Backup occurs on instance and then it is
compressed and streamed to Swift. Restore works backwards.
Change-Id: I78dd67369a1670ca72a89cc111cae40ed091fe47
Implements: blueprint db2-backup-restore
The default values needed for trove's implementation of cors
middleware have been moved from paste.ini into a common
set_defaults method, invoked on load. Unlike similar patches
on other services, this patch does not include config-generation
hooks, as trove doesn't use them yet.
Change-Id: Id8e04249498f63e42dadcacbd2c08b525adc0958
Closes-Bug: 1551836
The configuration server port 27019 was missing from the Trove config
files. Add it, and replace the hardcoded version in the guestagent with
a ref to the conf.
Change-Id: Ibca4cbed7669a0fde82af4e3c402f80d24bd9fad
Closes-bug: #1555180
Implement backup and restore functionality for Cassandra datastore.
We implement full backup strategy using the Nodetool
(http://goo.gl/QtXVsM) utility.
Snapshots:
Nodetool can take a snapshot of one or more keyspace(s).
Snapshot(s) will be stored in the data directory tree:
'<data dir>/<keyspace>/<table>/snapshots/<snapshot name>'
A snapshot can be restored by moving all *.db files from a snapshot
directory to the respective keyspace overwriting any existing files.
NOTE: It is recommended to include the system keyspace in the backup.
Keeping the system keyspace will reduce the restore time
by avoiding need to rebuilding indexes.
The Backup Procedure:
1. Clear existing snapshots.
2. Take a snapshot of all keyspaces.
3. Collect all *.db files from the snapshot directories package them
into a single TAR archive.
Transform the paths such that the backup can be restored simply by
extracting the archive right to an existing data directory
(i.e. place the root into the <data dir> and
remove the 'snapshots/<snapshot name>' portion of the path).
The data directory itself is not included in the backup archive
(i.e. the archive is rooted inside the data directory).
This is to make sure we can always restore an old backup
even if the standard guest agent data directory changes.
Attempt to preserve access modifiers on the archived files.
Assert the backup is not empty as there should always be
at least the system keyspace. Fail if there is nothing to backup.
4. Compress and/or encrypt the archive as required.
5. This archive is streamed to the storage location.
The Restore Procedure:
1. Create a new data directory as it does not exist.
2. Unpack the backup to that directory.
3. Update ownership of the restored files to the Cassandra user.
Notes on 'cluster_name' property:
Cassandra has a concept of clusters. Clusters are composed of
nodes - instances. All nodes belonging to one cluster must all have the
same 'cluster_name' property. This prevents nodes from different logical
clusters from accidentally talking to each other.
The cluster name can be changed in the configuration file.
It is also stored in the system keyspace.
When the Cassandra service boots up it verifies that the cluster name
stored in the database matches the name in the configuration file and
fails if not. This is to prevent the operator from accidentally
launching a node with data from another cluster.
The operator has to update the configuration file.
Similarly, when a backup is restored it carries the original cluster
name with it. We have to update the configuration file to use the old
name.
When a node gets restored it will still belong to the original cluster.
Notes on superuser password reset:
Database is no longer wide open and requires password authentication.
The 'root' password stored in the system keyspace
needs to be reset before we can start up with restored data.
A general password reset procedure is:
- disable user authentication and remote access
- restart the service
- update the password in the 'system_auth.credentials' table
- re-enable authentication and make the host reachable
- restart the service
Note: The superuser-password-reset and related methods that
potentially expose the database contents are intentionally
decorated with '_' and '__' to discourage a caller from
using them unless absolutely necessary.
Additional changes:
- Adds backup/restore namespaces to the sample config
file 'trove-guestagent.conf.sample'.
We include the other datastores too
for the sake of consistency.
(Auston McReynolds, Jul 6, 2014)
Implements: blueprint cassandra-backup-restore
Co-Authored-By: Denis Makogon <dmakogon@mirantis.com>
Change-Id: I3671a737d3e71305982d8f4965215a73e785ea2d
CORS middleware's latent configuration feature, new in 3.0.0,
allows adding headers that apply to all valid origins.
This patch adds headers commonly used in openstack to trove's
paste pipeline, so that operators do not have to be aware of
additional configuration magic to ensure that browsers can talk
to the API.
For more information:
http://docs.openstack.org/developer/oslo.middleware/cors.html#configuration-for-pastedeploy
Change-Id: Idf2cd7a0d0d701002f2c1f178475da39ae1a9caf
oslo.messaging has deprecated the use of messaging config settings,
specifically rabbit_* settings, in the [DEFAULT] section. This commit
moves the rabbit settings to a [oslo_messaging_rabbit] section in
each of the relevant trove service sample config files.
Change-Id: Ia869768102a8a841313cd7e0fd8a9fdab257d3e3
Closes-Bug: #1528391
To properly support different storage strategies the taskmanager
needs to be able to access the proper storage strategy to determine
things like the container name.
The patch addresses moving the storage strategy from guestagent
to common.
Change-Id: If81100cc88c6b883492c9f7b1a5e2437ba155eda
Closes-Bug: 1525283