268 Commits

Author SHA1 Message Date
Lingxian Kong
44fbcfaaa6 Deprecate the config option default_neutron_networks
Use `management_networks` instead. `management_networks`will be used
as admin networks which will be attached to Trove instance
automatically.

Change-Id: I5c6004b568c3a428bc0f0a8b0e36665d3c5b3087
2019-03-26 14:14:31 +13:00
Marcin Piwowarczyk
dc7303a135 Be compilance with latest oslo.messaging
Support for the [oslo_messaging_rabbit] section has been removed in
oslo.messaging==9.0.0 package [1].
That's why integration tests failures on build devstack step.

This is the moment where we need to use the transport_url directive
in the [DEFAULT] section instead.

Moreover rpc_backend property (which was used in trove to enable fake
RPC backend) has been removed from DEFAULT section and API tests
fails on TIME_OUT as they starts on rabbit RPC backend.

Fake RPC can be now configured as described here [2]:
transport_url = 'fake:/'

[1] https://docs.openstack.org/releasenotes/oslo.messaging/unreleased.html
[2] https://docs.openstack.org/oslo.messaging/latest/configuration/conffixture.html

Change-Id: Id6c5a9198d5a213cb085407a1d8b534e7c755f69
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
2018-10-13 20:00:16 +02:00
caoyuan
38a5679dac Update auth_uri option to www_authenticate_uri
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.

[1]https://review.openstack.org/#/c/508522/

Change-Id: I46bb0acaf1dec9bb4b91d4a2180b3e7ed66741c2
2018-04-11 22:28:23 +08:00
melissaml
f83a247bd6 fix a typo in documentation
Change-Id: Ic53adfb74c628c8ca85b2fa2107999c1ae816833
2018-03-23 08:08:18 +08:00
Zuul
15c867672a Merge "Add functionality to define different Message and Notification destination" 2018-03-19 13:48:16 +00:00
Zhao Chao
71ebd353ca Generate policy sample file automatically.
A new entrypoint in setup.cfg and a config file are added for
using olso.policy helper script to generate the sample file.

A new tox target also is added to simplify the environment
setting up. Now policy sample file can be generated
automatically, so the in-repo sample file is no longer needed.

Co-Authored-By: Andrew Laski <andrew@lascii.com>
Partial-Implements: blueprint policy-in-code
Change-Id: Ic336fa154ccc05b5e9db3a8e751a484b1cc5aa9c
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-02-12 18:11:22 +08:00
Zhao Chao
29362a18a4 Remove support of creating volume from Nova.
Current Nova server volume support is broken. Nova also declared the
'os-volumes_boot' will be deprecated in the future. As creating volumes
by cinderclient has been supoorted for a long time, we could just drop
support of Nova server volume.

This patch also migrate to the new block_device_mapping_v2 parameter of
Nova servers creating API.

Closes-Bug: #1673408
Change-Id: I74d86241a5a0d0b1804b959313432168f68faf89
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-02-09 14:30:50 +08:00
ZhongShengping
6ff047e743 Remove the heat related documents
This patch[1] removed heat related support.

[1]https://review.openstack.org/#/c/385513/

Change-Id: Ib1b6f9f36097ae043d92fef3b32cf1f0ebac9672
2018-01-17 16:17:25 +08:00
wangjun
1dcdf38bc2 Add functionality to define different Message and Notification destination
Sometime,huge notification messages will effect the rabbitMQ cluster's
performance.This commit adds functionality do define different Message
and Notification destination.

Closes-Bug: #1734823
Change-Id: I2ef4d008c9ca91a03d8c7e1380e0188bf6792595
2018-01-05 10:31:49 +08:00
Dai Dang Van
dd6b22d47a Remove policy.json file
We already had default rule in code, so we should not
still define all of them again in policy file.
Besides, we should you yaml format for now instead json.

Another thing, we don't need to config policy file in
Devstack enviroment.

Change-Id: I783ba51695271d358764557899fe91e84620556d
2017-12-07 16:38:03 +07:00
Zuul
290cecb627 Merge "Comment out the option oslo_messaging_rabbit.rabbit_password" 2017-11-24 01:36:09 +00:00
Zuul
c3c6b7fdc7 Merge "Open the volume_support of redis" 2017-11-09 08:08:31 +00:00
jiansong
5b7930446f Open the volume_support of redis
In the past, we support volume_support when we support
redis-cluster[1], but we did not update to conf[2].
we are making the sample conf files match the 
default configuration in code.

[1]:https://review.openstack.org/#/c/203702/
[2]:https://review.openstack.org/#/c/107117/

Closes-Bug: #1701469
Change-Id: I428e274fdb85621b21453780e90a8a0ab401ffd9
2017-11-03 02:14:05 +00:00
Zuul
d48bb35cd1 Merge "Remove exists_notification_ticks from sample conf" 2017-10-23 17:43:05 +00:00
Stephen Ma
68e02bbdf7 Comment out the option oslo_messaging_rabbit.rabbit_password
The option oslo_messaging_rabblt.rabbit_password has been
deprecated. Therefore the option is commented out in the
sample config files etc/trove/ directory.

Change-Id: I613087217c88b8a5dbf8dfc3f51e48b8fe4c84c8
2017-08-21 11:17:16 -07:00
Fan Zhang
327eb48384 Remove inexistent option in install guide.
Remove notifier_queue_* because the options are no longer needed and
should be removed.

Closes-bug: #1706528

Change-Id: Idfd205473f1756a69d0d9607e9f8c1dd9780553c
2017-08-07 09:33:58 +08:00
Samuel Matzek
9f4d0f7f13 Remove exists_notification_ticks from sample conf
The exists_notification_ticks conf option was removed under
commit 08dc866fb241f535cce0609b02a34853882538f5 but was not removed from the
sample configuration file.

Change-Id: I3608929bd1ce4cd6ec767665b6f60fd5d86127b4
2017-07-26 10:05:46 -05:00
Jenkins
a42496e68d Merge "Wrong comment symbol in configuration file" 2017-06-16 05:26:11 +00:00
jiansong
dbeff81d50 Add port 16379 to conf.sample
This is a point that is easily hidden. In the case of not open
port 16379, redis cluster can still be deployed successfully,
but this success is a false success, there is no data communication
between the various nodes. This is because 16379 is the port for
data port[1]. We set the default value in cfg, but in the
case of conf,it is easy to be covered. Adding 16379 to conf.sample
helps the user to reduce some problem caused by the redis
configuration.

[1]:https://redis.io/topics/cluster-tutorial

Change-Id: If517072c1c875df68106af14dac1802bb959d17e
2017-04-26 03:08:28 +00:00
Amrith Kumar
20d51467c7 enable trove-api behind mod-wsgi
This change enables behind mod-wsgi as part of the community pike goal
goal-deploy-api-in-wsgi.

The change includes:

    - the wsgi script files to run trove api under apache
    - updates to the devstack plugin
    - a basic deploy doc which explains this change

Change-Id: Icdd39b47a1be426e87a5d09f9e9d567af1974a9c
Depends-On: I3d6f6649430ee40879de15fee0b215dc32e8b666
Closes-Bug: #1681478
2017-04-25 06:54:29 -04:00
Amrith Kumar
191e3036e1 fix-gate: change trove auth URL's to reflect new URL settings
Trove's code was setup to use :5000 and :35357 for keystone's auth end
points. Change that to reflect the new settings that are /identity/
and /identity_admin/. See also [1]

[1] https://review.openstack.org/#/c/456344/

Change-Id: I3d6f6649430ee40879de15fee0b215dc32e8b666
Related: I46294fb24e3c23fa19fcfd7d6c9ee8a932354702
2017-04-23 00:17:13 +00:00
Peter Stachowski
83089aa5cc Add support for module-reapply command
Server side support for the new 'reapply' command.
This reapplies a given module to all instances that it had
previously been applied to.

Originally, a module designated live-update would automatically
be re-applied whenever it was updated.  Adding a specific
command however, allows operators/users more control over
how the new payload would be distributed.  Old 'modules'
could be left if desired, or updated with the new command.

Scenario tests were updated to test the new command.

DocImpact: update documentation to reflect module-reapply command

Change-Id: I4aea674ebe873a96ed22b5714263d0eea532a4ca
Depends-On: Ic4cc9e9085cb40f1afbec05caeb04886137027a4
Closes-Bug: #1554903
2017-02-27 18:43:32 +00:00
LiuYang
7be9e623d6 Wrong comment symbol in configuration file
Change-Id: Icf6fa8296ae3bfa05a8e4f07c38ede1fb951d18d
2017-02-24 14:56:03 +08:00
Petr Malik
21250cf20c Add support for Oslo Policies to Trove
The Oslo Policy library provides support for RBAC policy
enforcement across all OpenStack services.

Update the devstack plugin to copy the default policy file
over to /etc/trove in the gate environments.

Note: Not adding a rule for 'reset-password' instance
action as that API was discontinued years ago
and is now just waiting for removal (Bug: 1645866).

DocImpact
Co-Authored-By: Ali Adil <aadil@tesora.com>

Change-Id: Ic443a4c663301840406cad537159eab7b0b5ed1c
Implements: blueprint trove-policy
2016-12-06 21:51:21 +00:00
Jenkins
39ec28436e Merge "Use http_proxy_to_wsgi middleware" 2016-11-21 11:03:52 +00:00
Mariam John
ff2a52038b Implement full online backups for DB2
Currently Trove supports full offline backups for DB2. In this
implementation, we have added support for full online backups for DB2
using archival logging.

Change-Id: I30b5b0b85120fd105cc3db57983b062fad5fab5a
Implements: blueprint db2-online-backup
2016-09-13 20:35:01 +00:00
Jenkins
4603d75b0a Merge "Trove workbook for scheduled backups" 2016-09-09 21:34:05 +00:00
Morgan Jones
caab14ecf1 Trove workbook for scheduled backups
A Mistral workbook adding a workflow for creating a backup.
This is used by the "trove schedule-create" command to schedule
a backup.

Change-Id: I68d997abf4ec7c32044dd18cf2a70e687c0fed9f
Implements: blueprint schedule-backups
2016-08-25 17:00:48 -04:00
Jenkins
40da584699 Merge "Add trove-guestagnet.conf sample for log rotation" 2016-08-23 01:59:36 +00:00
Amrith Kumar
3b0d1ea25d Adds the api-ref migrated RST + YAML files
With this email[0], you must migrate API reference docs into RST. The
conf.py and the tox environment are also cribbed from nova.

Still need to retain the install_command in tox.ini, otherwise the
api-ref job fails.

[0] http://lists.openstack.org/pipermail/openstack-dev/2016-May/093765.html

Co-Authored-By: Anne Gentle <agentle@cisco.com>
Co-Authored-By: Amrith Kumar <amrith@tesora.com>

Change-Id: I3315261aa18729fa7a6aa79d4a1d6c24de1e2c6b
2016-08-17 17:46:41 -04:00
Masaki Matsushita
f80dd43297 Add trove-guestagnet.conf sample for log rotation
This change adds a configuration sample of guestagent
to use log rotation.

Change-Id: I2f2fb0d42f2f9d674daf3adff8725a365cc1a5b3
Closes-Bug: #1269615
2016-08-09 13:24:11 +09:00
Greg Retkowski
f7b8805c3a guest_id missing err, switch config_drive default
This changes the default setting for use_nova_server_config_drive
from False to True. By default, nova does not use a config
drive for launched instances. Therefore trove must implicitly
request the config drive be used so that it can inject
guest_info.conf into the guest VM.

This commit also adds an error to guestagent when guest_id
is missing. If the guest_id configuration parameter is not
set at guestagent startup it will throw a RuntimeError. This
case typically occurs because guest_info.conf was
not injected into the guest, or was not included in the set
of configs that guestagent reads at startup.

And finally, this commit adds a section in the 'building guest
images' documentation describing how configuration is injected
into the guest agent.

Change-Id: I084c841472183893a63ca7b70d560f44a2f08901
Partial-Bug: 1609915
2016-08-06 15:16:20 -07:00
Masaki Matsushita
1584f198a5 Introduce "icmp" option for security group rule
This change introduces new datastore option "icmp" to
configure whether to permit ICMP. It helps users to
check DB instance health in different way from access
DB ports.

DocImpact
Closes-Bug: #1485884
Change-Id: I61edeb38ded5543b7976a01363108a7b5b4fc5b5
2016-08-04 13:58:56 +09:00
Masaki Matsushita
583d5cd428 Use http_proxy_to_wsgi middleware
This commit enables to handle HTTP_X_FORWARDED_PROTO by using
http_proxy_to_wsgi middleware of oslo.middleware.

Change-Id: I6a11c8470205ca78bdb027fa9a06fec3acda33ad
Closes-Bug: #1590608
2016-08-03 09:29:29 +09:00
Jeremy Stanley
42de1e7f7e Switch from MySQL-python to PyMySQL
As discussed in the Liberty Design Summit "Moving apps to Python 3"
cross-project workshop, the way forward in the near future is to
switch to the pure-python PyMySQL library as a default.

https://etherpad.openstack.org/p/liberty-cross-project-python3

BaseMySqlRootAccess.enable_root(): catch also InternalError because
the PyMySQL error is not wrapped into a SQLAlchemy OperationalError,
but a generic SQLAlchemy InternalError. Similar change is made in
026_datastore_versions_unique_fix.py.

This change requires a trove integration change to add the PyMySQL to
the guest image: Id4d013d174ba40a453819f900aaa316a93e59b48.

Partially implements: blueprint trove-python3
Co-Authored-By: Victor Stinner <vstinner@redhat.com>
Depends-On: Id4d013d174ba40a453819f900aaa316a93e59b48
Change-Id: I65e8a8d5dc251a8b00529cdfb1a6ada3d5720f68
2016-06-30 08:47:01 +02:00
Matt Fischer
bd360fb3c3 Switch test config file to use os_region_name
The old value of nova_region_name has not been used in some time.

Change-Id: I8fdd6d8ae4f0982b37cff7b5583429ccb6bd4aaf
2016-06-05 20:22:04 -06:00
Amrith Kumar
02d9f9e57a remove verbose option in Trove config files
This option is now deprecated and will be removed in Newton. The
default is True, time to get rid of it in Trove.

Partial-Bug: 1586066
Change-Id: I63e3a2832d0862254548be36fea77d5d4fa381b3
related: https://review.openstack.org/#/c/314573/
ref: http://lists.openstack.org/pipermail/openstack-dev/2016-May/095166.html
2016-05-26 14:46:05 +00:00
Jenkins
dfa7e86265 Merge "Add MongoDB config server port to cfg" 2016-03-18 13:59:47 +00:00
Mariam John
65917f3ec0 Implement Backup and Restore for CouchDB
The recommended method for doing full backups in CouchDB
has been a simple filesystem copy of the data files. This
is because CouchDB stores data in wholly contained append
only files. For example, when a user creates a database,
a corresponding <database-name>.couch file is created in
the database directory.

The backup functionality has been implemented by compressing
the database directory and then encrypting it and sending it
over to store in Swift. Similarly, the restore functionality
has been implemented by fetching the files from Swift and
uncompressing them into the database directory. After this,
the ownership of the directory needs to be updated.

To test the changes, follow the steps:
- Create a CouchDB instance
- Access the admin console called Futon using the following
  url: http://10.0.0.5:5984/_utils/
- Create a database from there and create one or more documents
- Create a backup of this CouchDB instance
- Create another CouchDB instance from the backup created above
- Access the admin console for this new instance and verify that
  the database created above is there

couchdb client library for the integration tests has been added
to global-requirements: https://review.openstack.org/#/c/285191/

Change-Id: Iad6d69bf60ace73825819081964a43ad53d6c6fc
Implements: blueprint couchdb-backup-restore
2016-03-15 14:37:35 -05:00
Jenkins
8d9af8c4c8 Merge "Addition of DB2 backup & restore functionality" 2016-03-15 18:43:02 +00:00
Ishita Mandhan
421689e14f Addition of DB2 backup & restore functionality
Implementation of backup and restore functionality for db2
databases. Backup occurs on instance and then it is
compressed and streamed to Swift. Restore works backwards.

Change-Id: I78dd67369a1670ca72a89cc111cae40ed091fe47
Implements: blueprint db2-backup-restore
2016-03-15 09:58:57 +00:00
Michael Krotscheck
552f1f2070 Moved CORS middleware configuration into set_defaults
The default values needed for trove's implementation of cors
middleware have been moved from paste.ini into a common
set_defaults method, invoked on load. Unlike similar patches
on other services, this patch does not include config-generation
hooks, as trove doesn't use them yet.

Change-Id: Id8e04249498f63e42dadcacbd2c08b525adc0958
Closes-Bug: 1551836
2016-03-14 10:48:40 -07:00
Matt Van Dijk
0790c349f0 Add MongoDB config server port to cfg
The configuration server port 27019 was missing from the Trove config
files. Add it, and replace the hardcoded version in the guestagent with
a ref to the conf.

Change-Id: Ibca4cbed7669a0fde82af4e3c402f80d24bd9fad
Closes-bug: #1555180
2016-03-09 10:44:37 -05:00
Petr Malik
e722342ce7 Add backup & restore for Cassandra
Implement backup and restore functionality for Cassandra datastore.

We implement full backup strategy using the Nodetool
(http://goo.gl/QtXVsM) utility.

Snapshots:

Nodetool can take a snapshot of one or more keyspace(s).
Snapshot(s) will be stored in the data directory tree:
'<data dir>/<keyspace>/<table>/snapshots/<snapshot name>'

A snapshot can be restored by moving all *.db files from a snapshot
directory to the respective keyspace overwriting any existing files.

NOTE: It is recommended to include the system keyspace in the backup.
      Keeping the system keyspace will reduce the restore time
      by avoiding need to rebuilding indexes.

The Backup Procedure:

1. Clear existing snapshots.

2. Take a snapshot of all keyspaces.

3. Collect all *.db files from the snapshot directories package them
into a single TAR archive.

Transform the paths such that the backup can be restored simply by
extracting the archive right to an existing data directory
(i.e. place the root into the <data dir> and
remove the 'snapshots/<snapshot name>' portion of the path).
The data directory itself is not included in the backup archive
(i.e. the archive is rooted inside the data directory).
This is to make sure we can always restore an old backup
even if the standard guest agent data directory changes.

Attempt to preserve access modifiers on the archived files.

Assert the backup is not empty as there should always be
at least the system keyspace. Fail if there is nothing to backup.

4. Compress and/or encrypt the archive as required.

5. This archive is streamed to the storage location.

The Restore Procedure:

1. Create a new data directory as it does not exist.

2. Unpack the backup to that directory.

3. Update ownership of the restored files to the Cassandra user.

Notes on 'cluster_name' property:

Cassandra has a concept of clusters. Clusters are composed of
nodes - instances. All nodes belonging to one cluster must all have the
same 'cluster_name' property. This prevents nodes from different logical
clusters from accidentally talking to each other.

The cluster name can be changed in the configuration file.
It is also stored in the system keyspace.
When the Cassandra service boots up it verifies that the cluster name
stored in the database matches the name in the configuration file and
fails if not. This is to prevent the operator from accidentally
launching a node with data from another cluster.
The operator has to update the configuration file.

Similarly, when a backup is restored it carries the original cluster
name with it. We have to update the configuration file to use the old
name.
When a node gets restored it will still belong to the original cluster.

Notes on superuser password reset:

Database is no longer wide open and requires password authentication.
The 'root' password stored in the system keyspace
needs to be reset before we can start up with restored data.

A general password reset procedure is:
- disable user authentication and remote access
- restart the service
- update the password in the 'system_auth.credentials' table
- re-enable authentication and make the host reachable
- restart the service

Note: The superuser-password-reset and related methods that
      potentially expose the database contents are intentionally
      decorated with '_' and '__' to discourage a caller from
      using them unless absolutely necessary.

Additional changes:

- Adds backup/restore namespaces to the sample config
  file 'trove-guestagent.conf.sample'.
  We include the other datastores too
  for the sake of consistency.
  (Auston McReynolds, Jul 6, 2014)

Implements: blueprint cassandra-backup-restore
Co-Authored-By: Denis Makogon <dmakogon@mirantis.com>
Change-Id: I3671a737d3e71305982d8f4965215a73e785ea2d
2016-02-13 03:29:28 +00:00
Jenkins
da67fc3c1f Merge "Move rabbit conf settings to separate section" 2016-02-01 23:49:20 +00:00
Michael Krotscheck
1fe8c0bac1 Added Keystone and RequestID headers to CORS middleware
CORS middleware's latent configuration feature, new in 3.0.0,
allows adding headers that apply to all valid origins.
This patch adds headers commonly used in openstack to trove's
paste pipeline, so that operators do not have to be aware of
additional configuration magic to ensure that browsers can talk
to the API.

For more information:
http://docs.openstack.org/developer/oslo.middleware/cors.html#configuration-for-pastedeploy

Change-Id: Idf2cd7a0d0d701002f2c1f178475da39ae1a9caf
2016-01-08 11:26:40 -08:00
Doug Shelley
4be983e42e Move rabbit conf settings to separate section
oslo.messaging has deprecated the use of messaging config settings,
specifically rabbit_* settings, in the [DEFAULT] section. This commit
moves the rabbit settings to a [oslo_messaging_rabbit] section in
each of the relevant trove service sample config files.

Change-Id: Ia869768102a8a841313cd7e0fd8a9fdab257d3e3
Closes-Bug: #1528391
2015-12-22 10:35:25 -05:00
Duk Loi
b70bb10c7f Move storage strategy code from guestagent to common
To properly support different storage strategies the taskmanager
needs to be able to access the proper storage strategy to determine
things like the container name.

The patch addresses moving the storage strategy from guestagent
to common.

Change-Id: If81100cc88c6b883492c9f7b1a5e2437ba155eda
Closes-Bug: 1525283
2015-12-11 13:42:18 -05:00
Jenkins
052138727e Merge "Do not use api-paste.ini osprofiler options" 2015-11-24 23:38:03 +00:00
Jenkins
68f76e701e Merge "Fix example value for notification_topics" 2015-11-23 18:26:51 +00:00