Significant changes:
* Using docker image to install datastore.
* Datastore image is common to different datastores.
* Using backup docker image to do backup and restore.
* Support MariaDB replication
* Set most of the functional jobs as non-voting as nested
virtualization is not supported in CI.
Change-Id: Ia9c97a63a961eebc336b70d28dc77638144c1834
Include address type in getting instance response.
* Deprecate confip option network_label_regex as we don't reply on Nova
to get addresses, network names don't make any sense.
* Add 'addresses' in instance API response, keep 'ip' as is but mark
it deprecated in API doc, python-troveclient shouldn't break.
Story: 2007562
Task: 39445
Change-Id: Ia0458b5ddae8959ce29c17e444e1a51a026283cd
- Execute test groups in serial to make sure no more than 2 database
instance are created at the same time.
- Remove some unneccesary tests
- Remove unneeded datastore, e.g. 'Test_Datastore_1'
- Remove unsupported trovestack subcommands
- Move unsupported DIB elements to the 'deprecated-elements' folder
- Decrease default value of 'agent_call_high_timeout' to 5min
- Add initial_deplay for pooling task
- Use socket file to connect with database instead of using localhost
IP
Change-Id: Ie5030a671fbeb453eafa6cbe04e08da7b52e33c9
- 'HEALTHY' means the db service is responsive, 'ACTIVE' means the db
service is alive.
- Remove the CI job fakemodetests, but will add similar testing task in
the future.
- Fix the periodic CI job
- Remove MongoDB and related jobs
Change-Id: I5abe9091ba203297dc87db5fba139179166321f7
Deal with the situation when Trove fails to look for the Nova server
when waiting for the db instance ACTIVE
Change-Id: I484d45f24176c89d999864d3eb1c48860b3038bd
When building the guest image for dev_mode=true, the controller IP
address is injected to the image in order for the guest-agent to
download Trove code during initialization. As a result, we have to build
guest image each time in the Trove CI because the image build relies on
the devstack host IP address.
If we could remove the dependency, we can build the image(in
devmode=true) for a specific datastore once and use that image for all
related CI jobs, which could save some time for Trove CI.
Fix the current CI issue as well.
Change-Id: If23f4f179a6ab72cfb35e4c45d55142fedb76498
- Explicity specify branch name for building non-dev image
- Use branch name for building dev image in devstack
- Update the doc
Candidate backport for stable/train
Change-Id: I4d2fe38fac34c75b42234fa7a20aba51e6dc1c4e
DevStack is deploying Trove in service tenant model, for swift backed
instance backup, the swift container should be transparent to the end
users.
Story: 2006647
Task: 36883
Change-Id: I5859f4c9911fc2c129c8f23611c6607044fbc145
- Remove Designate v1 support. python-designateclient removed v1 support
in
093d8d7170
- Fix get port IP address error in devstack script.
Change-Id: I08916ddfba85c0218aeab5bfbad58ce3222521f3
- The users need to specify the network to create Trove instance, but
trove-taskmanager will create port in that network for Nova instance
creation. Using port gives Trove more capabilities to define how the
database service is exposed.
- Deprecate ICMP protocol for the instance.
- Restrict 'nics' parameter for creating instance.
- Add 'access' parameter for creating instance.
- Add 'public_network_id' option in order to create floating IP for the
instance.
- Do not create records for security groups, but Trove can still delete
existing instances for backward compatibility.
- Delete unreasonable Host, Account, Storage API.
Story: 2006500
Task: 36468
Task: 36466
Change-Id: I80827e1ad5e6b130cbf94c2bb7a909c44d5cf1e5
Allow the cloud admin to control the security groups on the management
port of Trove instance, a new config option `management_security_groups`
is introduced for that purpose.
Change-Id: I4b22b87d37792be700d4ec7f78a7ea479ddb5814
Story: 2006466
Task: 36395
Since Trove already supports to specify a Nova keypair when creating
instance for management convenience, devstack needs to be changed to
create the management keypair and add to Trove config file.
One extra change in this patch is to use a single config file for Trove
API, task-manager and conductor.
Change-Id: I1e6c4f4305104815bdf89b31776a4955de61bc89
Story: 2005429
Task: 30463
During debugging, the following changes are also included:
- Support to specify an image ID to run the integration test.
- Fix the reboot function bug.
- Remove the unsuccessful restart test.
How to run integration test with dev_mode=false:
ADMIN_PASSWORD=password \
SERVICE_PASSWORD=password \
DEV_MODE=false \
/opt/stack/trove/integration/scripts/trovestack gate-tests mysql mysql
Change-Id: I31d4ee579a554f4c98f9facb9fd4b7779665a3dd
Remove several variables local to plugin.sh so it is possible to
set a non default datastore by changing only the 3 most obvious ones:
TROVE_DATASTORE_TYPE
TROVE_DATASTORE_VERSION
TROVE_DATASTORE_PACKAGE
Change-Id: I782757647985bdfb859043fe68e2b4cd27569ab3
Changes to get jobs working
1) After [1] devstack no longer changes the ownership of the whole
/opt/stack tree to the stack user unconditionally. Switch to the
stack user when running integration test.
2) Add bindep.txt file[2]. The default fallback file is not installed
anymore and therefore a bindep.txt file is needed to add install
additional packages.
3) Use trovestack script rather than devstack to build image so many
global variables could be used for consistency. By default, devstack
won't build image.
4) Remove the tools/test-setup.sh as it is not used any more.
5) Instance upgrade test keeps failing in CI for some reason, although
it's always passed on my local environment. In order not to block
other patches, skip the instance upgrade tests temporarily.
[1] https://review.opendev.org/203698
[2] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006888.html
Change-Id: I35e17afb9e827b1fead9d28dbf32f11ce4034a9b
The service tenant deployment model means Trove creates most of the
resources(vm, volume, security group, etc.) relating to a database
instance in the Trove service tenant rather than the end user.
With this deployment model, most of the related resources behind the
scenes are invisible to the user, which is a more secure deployment
model for either private or public cloud provider.
DevStack should follow this model as it will be recommended for Trove
deployment in production.
Changes included in this patch that are necessary in order to make that
happen:
- Add 'admin' role to Trove service user(username: trove, project:
service) in DevStack.
- Create Trove management network resources for Trove service user in
DevStack.
- Enable Trove remote client configuration by default in DevStack.
- Mainly use alt_demo user in alt_demo project for integration tests,
config trove user as the admin role user in integration tests.
- Disable the module related tests(module_groups) for now because of no
use cases but need effort to fix all the failed tests in the service
tenant model.
Story: #2005445
Task: #30489
Change-Id: I2efb69d3d50344914a875b773f62a227dba2ccaf
- Add a new element 'guest-agent' for image building. This element is
used when dev_mode=false, so that the trove code is downloaded into
the image during the building phase rather than during the guest
agent initialization.
- Improve trovestack sub-command 'build-image'.
./trovestack build-image ${datastore_type} \
${guest_os} \
${guest_release} \
${dev_mode}
- Improve documentation.
Story: #2005387
Task: #30375
Change-Id: I9d7acbd6a97f8c01b48b0f2cf94398d549d89124
Trove mainly uses trovestack script to trigger the CI tests, including
the guest agent image building, so the image building in devstack could
be skipped to decrease the testing duration.
Change-Id: I26fa0adc1a5e39612be15b75d37d22ea1cca3be1
Story: #2005391
Task: #30380
Use `management_networks` instead. `management_networks`will be used
as admin networks which will be attached to Trove instance
automatically.
Change-Id: I5c6004b568c3a428bc0f0a8b0e36665d3c5b3087
In Ubuntu Bionic, the following command returns "uid" so that the
guest-agent service will fail because of invalid IP address.
$ ip route get 8.8.8.8 | head -1 | cut -d' ' -f8
uid
$ ip route get 8.8.8.8 | head -1
8.8.8.8 via 10.0.17.1 dev ens3 src 10.0.17.131 uid 0
In Ubuntu Xenial:
$ ip route get 8.8.8.8 | head -1 | cut -d' ' -f8
10.0.17.132
$ ip route get 8.8.8.8 | head -1
8.8.8.8 via 192.168.192.1 dev eth0 src 10.0.17.132
Change-Id: Iae24686fa65603b7d22b09da99b7590e4e117454
Trove implies strict network requirements. All Trove instances must be
launched in a network that is connected to the management network so
that Guest Agents residing in the data plane could communicate with
control plane services, e.g. RabbitMQ and Trove Conductor. Trove
Devstack plugin creates a management network bound to management physnet
(trove-mgmt), as well as tenant network (alt-private) connected to it
via a router. Consequently, Trove instances deployed in the tenant
network can easily access the management services mentioned above.
However, these networks are visible only from the perspective of the
tenant that created the networks.
Since Tempest does not provide an easy way to create a similar
management network, and as such does not enable creating tenant network
for Trove instances automatically, this change adds --share flag to
networks created by Devstack so that Tempest could reuse them.
Change-Id: I8c642ab6150858478c48291a51675ef9d15dafaa
Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>
In the devstack script, `[ -n $SUBNETPOOL_V4_ID ]` returns true even if
SUBNETPOOL_V4_ID variable is not defined, which is not expected.
I can see the error as the following:
```
2019-02-06 07:20:57.095 | Finalizing Neutron networking for Trove
2019-02-06 07:20:57.095 | Dumping current network parameters:
...
2019-02-06 07:20:57.096 | SUBNETPOOL_V4_ID:
...
2019-02-06 07:21:02.706 | openstack subnet create: error: argument --subnet-pool: expected one argument
```
This patch changes that to `[ -n "$SUBNETPOOL_V4_ID" ]`
Change-Id: Ib37b2ade3d7cf556340944a56944beb4bb47d7d9
When installing trove using devstack, an error occurs when rerun
devstack:
2019-02-06 06:22:57.450 | /opt/stack/.ssh/id_rsa already exists.
2019-02-06 06:22:57.453 | Overwrite (y/n)? Error on exit
Because the ${SSH_DIR} folder was created in the last run.
This patch fixes the issue by checking if the folder exists and only
generate the ssh key if not.
Change-Id: I2f0e3f99b03584a30571199f6be7a7ef28fbbd0d
Main goal is to have a working vm image registered in
trove after devstack installation.
It is a first step towards trovestack-redesign as described
in https://etherpad.openstack.org/p/trovestack-redesign
It is enabled by default during devstack installation with
trove enabled plugin, but it can be skipped by setting
DISABLE_TROVE_IMAGE_SETUP=TRUE
Implementation details:
* export environmental variables for diskimage-builder
* build an image with disk-image-create script
* register the newly created image in trove
* add tripleo-elements information to default settings
Change-Id: I6e57890c5b6fb65e9b8c63363f92e5a7c70e523e
Signed-off-by: Dariusz Krol <d.krol@samsung.com>
Tempest's service_availability config option includes all the service
availability which is further used by tests to take decision of skip
or run the test.
For example, [service_availability].trove is true then, trove test will run
or if [service_availability].trove is false then, all trove related tests either
in trove tempest plugin or any other plugins will be skipped.
Currently this setting for trove service[1] is in devstack lib/tempest
which is being removed by - https://review.openstack.org/#/c/619973/
For better maintenance, we are moving all tempest non-owned service setting
to service devstack plugin side.
This commit add the setting of trove service on trove devstack plugin.
Closes-Bug: #1743688
[1] https://git.openstack.org/cgit/openstack/trove-tempest-plugin/tree/trove_tempest_plugin/config.py#n17
Needed-By: https://review.openstack.org/#/c/619973/
Change-Id: Ie3c6bab6e0be9dfcfe29ec3f412e704f3ad79d61
Support for the [oslo_messaging_rabbit] section has been removed in
oslo.messaging==9.0.0 package [1].
That's why integration tests failures on build devstack step.
This is the moment where we need to use the transport_url directive
in the [DEFAULT] section instead.
Moreover rpc_backend property (which was used in trove to enable fake
RPC backend) has been removed from DEFAULT section and API tests
fails on TIME_OUT as they starts on rabbit RPC backend.
Fake RPC can be now configured as described here [2]:
transport_url = 'fake:/'
[1] https://docs.openstack.org/releasenotes/oslo.messaging/unreleased.html
[2] https://docs.openstack.org/oslo.messaging/latest/configuration/conffixture.html
Change-Id: Id6c5a9198d5a213cb085407a1d8b534e7c755f69
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
It seems that the current guestagent only needs to communicate
through amqp, and does not need to interact with nova, so the
related nova configuration can be removed from the guestagent,
which can reduce some invalid configuration.
Similarly, we removed the related configuration of condutor
and guestagent in devstack plugin.
Change-Id: I3f0b3b445c053351c1f3e966d77ea228df0e94a7
A previous commit attempted to set the guest agent command
process timeout value high but had a copy-paste error in
the line that actually sets the environment variable value
into the guest conf file.
This fixes the copy-paste issue.
Change-Id: Ib8aa566fb34eb52e2541d0ccb595234373e01bc2
We already had default rule in code, so we should not
still define all of them again in policy file.
Besides, we should you yaml format for now instead json.
Another thing, we don't need to config policy file in
Devstack enviroment.
Change-Id: I783ba51695271d358764557899fe91e84620556d
apache_site_config_for is a shell function, not a shell variable.
Shell syntax for functions is (). Variables use {}.
Change-Id: Ia9a35cdb0883f576efe584bc854de94072e13b87
Increase the timeout for out of process commands called
by the guest agent in the devstack plugin.
Change-Id: Ic8fec30a829ca828bd7fdef18e65fa2cb59f73c2
The devstack plugin sets a 3 hour Keystone token life
but Keystone is already running when this configuration change
is called so Keystone still runs with the default for the tests.
Add a restart of Keystone to enable the longer life.
Change-Id: Ie340b1c0b56652fc9ca4c0981468363c1be9fb25
Issue 1:
The os-testr 1.0.0 release had a couple of required config changes due
to it's internal usage of stestr.
This change to stestr changed the way tests were discovered by
os-testr and as a result the unit test run was picking up tempest
tests.
A regex is added to the py3base environment call of ostestr because
the use of --serial and --blacklist-file together is broken in stestr
and adding the regex parameter allows the blacklist-file to be
processed. The stestr issue is documented here [1].
Issue 2:
Cache dirs for PKI tokens have been removed for all services in
devstack under I5680376e70e74882e9fdb87ee1b95d5f40570ad7.
We must also remove the use here to pass the right parameters to
configure_auth_token_middleware.
Issue 3:
Keystone V2 APIs have been removed. When creating Nova and Glance
clients, the test code was either hard coding v2 Keystone or not
providing enough information for the V3 auth.
Issue 4:
Oslo context has deprecated parameters such as 'tenant', has removed
them from its constructor and is using a rename decorator to handle
them. As such, the code and test case to check for unrecognized
parameters to TroveContext and Context is erroneously removing the
tenant parameter. Oslo context has also changed the from_dict method
since the original code to remove parameters was introduced into
Trove. The new method signature and code should already provide most
or all of the protections against incompatibility the original code
was attempting to provide. The fix for this issue is to change
TroveContext's from_dict method to use the kwargs to handle its own
__init__ parameters and be more in line with what Nova is doing in
its RequestContext subclass.
Issue 5:
Jobs run as jenkins on Zuul v2 but run as user zuul on Zuul v3.
Issue 6:
Ignore one case of pylint E1101 in the Ceilometer notification
code base.
[1] https://github.com/mtreinish/stestr/issues/103
Change-Id: Ic55187b0d73d4c572d7f8332882b4f455a6177c8
This change enables behind mod-wsgi as part of the community pike goal
goal-deploy-api-in-wsgi.
The change includes:
- the wsgi script files to run trove api under apache
- updates to the devstack plugin
- a basic deploy doc which explains this change
Change-Id: Icdd39b47a1be426e87a5d09f9e9d567af1974a9c
Depends-On: I3d6f6649430ee40879de15fee0b215dc32e8b666
Closes-Bug: #1681478