40 Commits

Author SHA1 Message Date
Peter Stachowski
85339a246c Add support for module ordering on apply
A method for specifying 'priority' modules plus a way to rank the order
in which modules are applied has been added.  Two new attributes
'priority_apply' and 'apply_order' are available in the payload
on create and update.  In addition, an is_admin flag was added as an
automatic attribute, to be set when someone with admin credentials
creates a module or updates an existing module with 'admin-only'
options.  This allows better control on the driver plugin
side with regards to security concerns, etc.  The attribute is
now passed in to the guest 'apply' interface for use by the driver.
All three of these attributes are stored in the Trove database.

An admin can create a 'non-admin' module by passing in --full_access
on the command line (or python interface).  This will cause an
error if any admin-only options are selected.

Scenario tests have been added to verify that the modules are
applied in the correct order.  The timestamp for the 'updated'
field on the guest was also enhanced to allow for fractional
seconds, since most applies take less than a second.

The issue where modules were allowed to be applied even if
they belonged to a different datastore has been fixed and
scenario tests added to check for this case.

Change-Id: I7fcd0cf12790564ba62e7d6451fff96f763e539d
Implements: blueprint module-management-ordering
2016-12-26 05:35:39 +00:00
Jenkins
436eb38774 Merge "when pylint has an error loading the config, it should fail" 2016-11-30 11:06:59 +00:00
Amrith Kumar
da2dc0afac when pylint has an error loading the config, it should fail
When someone has a pylint config file that has an error such as when a
merge goes bad, loading the file fails, and pylint motors along
happily with the default file (i.e. nothing ignored) and this is
confusing. Unless you happen to catch the little error that it prints
out.

Instead, just bomb out with an error.

Change-Id: Ie94dda35652f2a8f06502af72b68be73d61fc94a
Closes-Bug: 1639040
2016-11-14 13:12:15 -05:00
Morgan Jones
3f93ff110b Multi-Region Support
This is an initial attempt at supporting multiple regions.  It should
handle the mechanics of deploying an instance/volume to a remote
region.  Additional changes may be required to allow the guest
agent on the instance to connect back to the originating region.

Co-Authored-By: Petr Malik <pmalik@tesora.com>
Change-Id: I780de59dae5f90955139ab8393cf7d59ff3a21f6
2016-11-04 15:36:16 +00:00
Jenkins
8b98c51708 Merge "Remove downgrade" 2016-11-01 23:05:14 +00:00
Trevor McCasland
6593986b34 Remove downgrade
Removed downgrade from all existing migrations.
Modified a test that verifies that no migrations have a downgrade.
Modified test _walk_versions to only do upgrades.
Removed exceptions from pylint.config

Related cross-project spec: https://review.openstack.org/152337
Closes-Bug: #1434103

Change-Id: I9a7c87ae3f4e2eff3a4a6df881d086d52062dbba
2016-10-06 16:57:41 -05:00
Petr Malik
7fd8801d8d Merge Postgresql service modules
Postgresql has been implemented differently from
all other guest agents.
This has been causing difficulties with maintenance
and future development of Trove design.
It has been decided (Newton midcycle) to bring the guest agent
in line with others.

This patch set merges all Postgres service modules into
App/Admin classes.

This also resolves 20 pylint issues related to mixin usage.

Change-Id: I1b12a5296c59e9d3b08bcf34ac196d16189d525e
2016-09-28 12:51:21 -04:00
Amrith Kumar
a2d336de2a improve pylint; generate errors and config in sorted order
Since the config is not in a deterministic order makes it hard to
compare two config's and see what changed. Personally, I'm not
positive I understand this use-case; i.e. you have an existing config
file, you save it, and then rebuild and then diff the two files. I'd
have thought you'd just run check and the output of the tool was the
diff.

I however do see the value in sorting the file so that when someone
submits a change that includes a change to the config, reviewers can
see more easily what the change is doing.

Similarly, the output from pylint (errors) are generated one file at a
time and os.walk makes no guarantee of deterministic order. So we
should collect all errors (across all files) and then print an ordered
list for human consumption.

The intent is also to make pylint voting soon (in master). the changes
to contributing.rst and tox.ini are to make that easier. The config
file has also been sorted in place.

This change was motivated by an email exchange with Peter so I am
marking him as a co-conspirator.

The line numbers were removed from the tools/trove-pylint.config file
as these would change whenever the line numbers in the file changed
(since they are currently not being used in the comparison; they can
be re-added if deemed necessary at the cost of having every 'rebuild'
run create a different file).

The tools/trove-pylint.config was regenerated as well, since the
remaining two errors seem to be innocuous:

  ERROR: trove/taskmanager/manager.py 392: E1101 no-member,
  Manager.upgrade: Instance of 'BuiltInstance' has no 'upgrade' member
  (new method introduced by instance upgrade; other BuiltInstance
  member errors are already ignored.)

and

  ERROR: trove/guestagent/datastore/experimental/postgresql/service/
  access.py 80: E1101 no-member, PgSqlAccess.list_access: Instance ofi
  'PgSqlAccess' has no '_find_user' member
  (this is due to the fact that PostgreSQL is spread over multiple
  files and pylint should cease to complain once
  https://review.openstack.org/#/c/346082/ lands.)

Change-Id: I910c738d3845b7749e57910f76523150ec5a5bff
Closes-Bug: #1625158
Closes-Bug: #1625245
Co-Authored-By: Peter Stachowski <peter@tesora.com>
2016-09-19 21:04:14 +00:00
Amrith Kumar
a0bc0dcb7d initial chkin of pylint
this is a pylint wrapper for Trove's tox tests. This commit includes a
basic infrastructure for running pylint through tox.

It also fixes some very obvious import errors that are flagged by the
tool. One is to handle missing imports for _ and _LE.

There is one instance where an exception is being raised but
trove.common.exception isn't imported, and another where an exception
is being incorrectly thrown.

A short readme is also provided.

Change-Id: I0a38f5efde3cb491f1f6c27f6c6500ab29987968
Partial-Bug: #1621636
2016-09-14 13:56:24 -04:00
dineshbhor
43546ce905 Replace OpenStack LLC with OpenStack Foundation
Change-Id: I7b738715873eff90a49357dec5be4e179c04713c
Closes-Bug: #1214176
2016-07-22 15:07:03 +05:30
bhagyashris
fa08854a6d Replace print statment with print function
In PY3, print should used as a function instead of a statement.
This is PY27 compatible as well.

For example, instances of the first case were replaced by the
second case:

>>> print 'help'
  File "<stdin>", line 1
    print 'help'
               ^
SyntaxError: Missing parentheses in call to 'print'

>>> print('help')
help

Partially implements: blueprint trove-python3
Change-Id: I21de65bc874d0fcfb51714857f8eaa97acb2f51d
Closes-bug: #1594741
2016-06-22 11:42:34 +05:30
Julien Danjou
414507227e Remove Python 2.6 classifier
Trove does not support Python 2.6 anymore starting with Kilo and might
not work correctly with it, so remove the classifier.

Change-Id: Ida17ce3ce76258442222b591846e1e1717119eed
2014-12-02 09:57:52 +01:00
Ionuț Arțăriși
a5d745dd87 rename and fix the db_wipe command
* rename db_wipe to db_recreate
* make --repo-path an optional argument for db_wipe and use it
* add command line help strings for db_wipe

Closes-Bug: #1279734

Change-Id: I1a1d3ad56989026fb10f6c57bb479814dc51328e
2014-03-10 10:16:03 +01:00
He Yongli
52db0a385a Remove extraneous vim configuration comments
Remove vim setting, Remove line containing:
comment - # vim: tabstop=4 shiftwidth=4 softtabstop=4

at the top of source code files, except for files in
openstack/common.

Change-Id: I5e2aad6dca8407890fc3c2afbda541bd4f124ea8
Closes-Bug: #1229324
2014-02-27 15:05:21 +08:00
Michael Basnight
b06db4ab7f Migrating trove to entry points
PBR/Distribute generates binscripts for us. By moving the
binscript code into the source tree, we can unit test it
as well. Run setup.py develop to generate the binscripts for
development use.

implements blueprint entrypoints-for-binscripts
Change-Id: I9ad4bb4ddc55f64bcd806c588a795cd6e0847aa9
2014-02-01 01:19:58 +00:00
Zhenguo Niu
d4b324573f Enclose command args in with_venv.sh
It will support argument with blanks.

Change-Id: I83313290a964bf10fe46570f43cf213ceab8a326
Fixes: bug #1220937
2013-09-09 15:02:40 +08:00
Michael Basnight
7ab80b554d Rename from reddwarf to trove.
Implements Blueprint reddwarf-trove-rename

Change-Id: Ia9ee609bbc06a1d8b9d6917642529f30347541fd
2013-06-24 14:11:15 -07:00
Monty Taylor
b11db36f47 Updated to use normal requirements names.
Change-Id: I20545a3250c7db826b5c9133b8a94e3a8b2a3dbf
2013-06-03 18:44:20 -04:00
Monty Taylor
fc930eb91d Migrate to pbr.
Fixes bug 1179007

Change-Id: If7c187f3b4833bdd7f2c0632c061afdcacb9832f
2013-06-03 18:44:17 -04:00
Monty Taylor
98d42e50e3 Migrate to flake8.
Fixes bug 1172444

Change-Id: I09e1e2d03f41a175aa39760b5fc560ed0d9d30a5
2013-05-16 10:29:59 -07:00
justin-hopper
b3c32e3f87 Controller and API changes for backups.
-API controller for backups
-adding swift url to the config
-Fixing failing tests
-Renaming 'instance' param.
 Checking for NotFound models so that the returned error is friendly
-adding feature to list all backups from a specific instance
-Adding checks for creating/deleting/restoring backups when it is not completed
-Adding unit tests for backup controller
-adding check to see if file is in swift
-Adding skeleton code for delete backup in the task manager
-Fixed backups API to pass in the backup_id during create_backup.
-adding int tests for backup controller
-Adding backup list and delete int tests
-Adding list backups for instance test
-Adding quota for create backup

BP: https://blueprints.launchpad.net/reddwarf/+spec/consistent-snapshots

Change-Id: I35c2fefcce4b3009e76ba7232c52dabf502a3ac0
2013-05-09 09:26:06 -07:00
justin-hopper
770c0fd83b Refreshed Oslo Code
Change to auth_tok to auth_token
    Added dependency on oslo.config package
    Remove reddwarf.openstack.common.cfg

Change-Id: I61af7a54d09f3d1d6b6efe470ea6308be8ace2e7
Fixes: bug #1175757
2013-05-08 16:11:02 -07:00
Tim Simpson
8855ea83a8 Pinning proboscis to a specific version.
fixes bug #1152370
Change-Id: I9ef270a8c50fd9e3aedbfaf1545c31af2c051769
2013-03-08 13:05:42 -06:00
daniel-a-nguyen
cd6e9d7677 Update test-requires to point to rd cli tarball
Sync up the client code to server code for absolute/rate limits
Implements: blueprint rate-limits

Change-Id: I2d465216fa8432674e268eb3a18b5f4cbde2692c
2013-03-08 10:15:03 -08:00
ruiyuan-shen
0e929485b4 Add python-keystoneclient to deployment dependency for reddwarf
Bug #1150904

Change-Id: Ife20228c771dcaad201dc871ed3e2079d7fd6de1
2013-03-06 17:35:25 -08:00
Jenkins
9fc2231647 Merge "Use tarball for python-reddwarfclient dependency" 2013-03-06 23:37:47 +00:00
daniel-a-nguyen
0a71ef9e88 Rate limits implementation
added unittest for limits
    reverted changes to openstack/common
    removed commented code
    cleaned up unittest
    added int-tests
    updated reference to XMLNS
    removed 1.1 XMLMS in wsgi

Implements: blueprint rate-limits
Change-Id: I842de3a6cae1859cc246264a5836abfd97fb8074
2013-03-05 14:17:36 -08:00
Vipul Sabhaya
ddaf6decfa Use tarball for python-reddwarfclient dependency
* Openstack-ci creates a tarball at tarballs.o.o when tag matches:
  ref: ^refs/tags/([0-9]+\.)+[0-9]*
        (alpha|beta|candidate|rc|a|b|c|r|g)[0-9]*$

Fixes bug#1131509

Change-Id: I5e3d00c57dfe977f6e9a01dea25b97c8fcdf0d1d
2013-03-04 18:07:17 -08:00
Ed Cranford
90f8ca81d9 Modify-user features.
Grant, revoke, list user access.
Change user password and get single user.

Partially implements blueprint modify-users

Change-Id: I0001a7a9d1c527b88a1ed965f0f077c864e602cf
2013-02-19 20:40:26 -06:00
Ed Cranford
ce6b98e2d1 Checks guest status during migration.
During a migration, waits for the guest and service to start up
following the status change to VERIFY_RESIZE. Confirms if everything
is all right, or reverts if not.

Implements blueprint migration-check-guest-status

Change-Id: Ia7c7ed1fd0070429fed93323ca559d1c0742bd8f
2013-01-17 16:59:16 -06:00
Steve Leon
b1f061b2e8 Adding guest agent pkg unit tests and running them with testr
Added unit tests for reddwarf.guestagent.pkg module.
Added tests for package install, remove and version
Changing the test runner of unittests to be run by testr instead of probocis/nose
Remove probocis reference from the dbaas test
Moved dbaas.py to reddwarf/tests/unittests

tracked by bug #1090139

Change-Id: Ia25fbe996d04d39e83edef0369de0e7fccce579e
2013-01-11 15:06:25 -08:00
Michael Basnight
c8a5bc39dc Update oslo codebase within reddwarf.
* Updated logging,cfg,setup to new oslo
* Split out the paste ini from the conf files
* Modified reddwarf-api/server to use new modules
* Modified reddwarf-manage to use new cfg
* Added rpc helper for rpc services
* Modified reddwarf-taskmanager to use rpc helper
* Modified reddwarf-guestagent to use new rpc helper
* Fixed guestagent api to use rpc proxy
* Fixed taskmanager module to conform to new rpc
* Updated guestagent manager/pkg to use new rpc
* Updated api paste to use keystoneclient auth_token
* Updated managers to use periodic tasks

Implements: blueprint reddwarf/upgrade-oslo

Change-Id: I9ad1b441eca855a4304454014ae746ec51bef8f3
2012-12-18 21:18:28 -06:00
Steve Leon
732f4102b3 Adding some dbaas.py unittests
The tests added are createDB, deleteDB and deleteUser

tracked in Bug #1090139

Change-Id: I811a85bfe37ec4282864ec81328dc40c9b15f4ef
2012-12-14 10:38:47 -08:00
Nikhil Manchanda
5e7757faa6 Update tools/install_venv.py to work w/ latest pip
pip dropped the -E option as of 1.1; now to install in a virtualenv,
you just explicitly activate the virtualenv.  This commit fakes the
same behavior of explicitly specifying a virtualenv for installation
by altering tools/with_venv.sh to optionally take the -E flag

Fixes Bug #1081200

Change-Id: I470ba9563864337b8af2a79ae463e269ba66bf8f
2012-11-21 15:34:14 -05:00
Tim Simpson
c007356a78 Adding tests to Reddwarf.
The tests come from the Reddwarf Integration repository. wsgi_intercept
is used to allow the test code to hit the server code directly. It also
properly sets up the SqlLite database before each run.

* Adds an "event simulator" which queues up functions that would
  normally be spawned with eventlet. The various sleep functions are
  then swapped with a method that runs these faux-events.

* Adds many of the Reddwarf Integration tests. The idea is these
  could still run in a richer, real environment, but are running here
  enables us to quickly check feature breaks for each commit and
  lowers the learning curve necessary to test the API. The downside
  is some of these tests still have artifacts of their origins, such
  as (unused) classes to connect to MySQL databases. Some more work
  will be necessary to separate the "real mode" functionality of these
  tests further.

Implements: blueprint tox-tests
Change-Id: I9857f265c1cb46832906ef5e6a0c7bb4a092e637
2012-11-19 12:33:40 -06:00
Tim Simpson
11a3e531f7 Removes the vestigial tests.
This gets rid of the misleading tests in this repo which have not been
relied on for most of this project's life (for historical reasons, tests
are currently stored in the reddwarf_lite-integration repo).

Change-Id: Ic452396ac50ea252bbd34d41638c2f337da3ed2a
2012-11-02 16:19:55 -05:00
Monty Taylor
bdbb0a64a6 Aligned tox file with project.
Change-Id: Ib391c0d1025605fccda6405865bba5780ebb0c74
2012-10-26 13:21:45 -04:00
Monty Taylor
bc518af78b Added .gitreview file. 2012-10-26 13:12:07 -04:00
Tim Simpson
334a36ab68 Got the unit test venv environment working. 2012-03-07 08:47:10 -06:00
mbasnight
6a3f66f0e9 Adding the venv/unit test framework stuff
* run_tests stuff
* gerrit stuff
* test-requires for pip
2012-02-22 21:20:45 -06:00