Grenade is an OpenStack test harness to exercise the upgrade
process between releases. It uses DevStack to perform an initial
OpenStack install and as a reference for the final configuration.
Add support of it to Manila project, doing following changes:
- Add all required scripts for 'grenade' job, such as following:
-- settings
-- shutdown.sh
-- upgrade.sh
- Add optional 'resources.sh' script that performs functional
tests using bash and manilaclient. List of scenarios:
-- 1. create and delete share setting metadata and access rule
-- 2. attach security service to share network
-- 3. update quotas
-- 4. create private share type and add access for project
Note following:
- Tempest is not used by design.
- Manilaclient is used of stable version - version of 'base' devstack,
which is 'stable/mitaka'. It also means that used API microversion
is latest for that branch.
Change-Id: I7d7e0c50933776ca1a710bcf069abb0282b821ac
Depends-On: I4232382d3c944666bbf0916a3f285b84f6ff9d64
Implements blueprint grenade
Set opt "driver_handles_share_servers" to "False" value if
ZFSonLinux share driver is enabled.
Change-Id: Ia4b8abedd18e5002a3068beb3732d26ec21b35df
Closes-Bug: #1560126
'grenade' test job runs two versions of devstack - 'base' and 'target'.
And all devstack functions should be error-proof running more than one
time - 'grenade' needs twice. Therefore, make lots of devstack
functions check for results of previous runs such as presence of
resources that are going to be created.
Change-Id: I4232382d3c944666bbf0916a3f285b84f6ff9d64
Manila Devstack plugin creates admin network for Generic driver
using env var "TENANT_ID". And after its removal in Devstack
Manila fails to create Neutron network and fails on migration
tests that use results of this network creation.
Make Manila get ID of required project calling 'openstack client'
to avoid such error.
Change-Id: I43b18b2d2f2e69a150cd3e29ac84426550a6e40d
Closes-Bug: #1566826
Feature of Neutron DNS resolvers was disabled as
unsupported by manila-service-image.
Enable it, since, it is supported now [1].
[1] Icf7f189d621e9686c4bb449b5770b83e4bd072ee
Change-Id: I667c811c752864d6d7f239e2dfd4597e8ac98cb1
Use the install_package function from devstack instead of
calling the different package managers from distros.
Also support installation on SUSE based systems.
Change-Id: I575769aa972d7e123e79d0955d4618c09f1559d3
This fix basically ports nova fix https://review.openstack.org/#/c/136931/
plus additional bug fixing to Manila. It creates a common class to get
used for all clients (neutron, nova, cinder).
Idea is to create an auth session and pass this to the client object
instead of let the client do that. The auth session will be created by
a config group which means the configuration for the clients needed to be
heavily reworked. Patch is also backward compatible with old options but
flag them as deprecated.
DocImpact
Change-Id: Ic211a11308a3295409467efd88bff413482ee58d
Closes-bug: #1555093
This reverts commit 466a19f18f12cb77b45aa7c7a272fae7c45c4c93.
Also this reverts commit 4f74b224fad7bf9ca665d70a66c79c98cf8d5674.
Manila has decided to remove LXD from the Mitaka release.
http://lists.openstack.org/pipermail/openstack-dev/2016-March/089522.html
Change-Id: I3c4b15e8c9d92c874be754205b0b981151809cc1
Commit [1] changed link to service image and its name.
But tempest config option is not updated and refers to
old nonexistent image name.
It leads to impossibility to create Nova VMs in scenario CI job.
[1] I391119cf08949974efc469b8669d5e4120170c0d
Change-Id: I912ecd61556e8a0ff3a82f7b70a47cbfcc490d22
Closes-Bug: #1555189
Rather complex logic was added to ZFS driver to cope with
mount-namespace-related problems caused by neutron. A better
solution is to ensure mounts (and unmount) propagate to
namespaces.
Closes-Bug: 1552552
Change-Id: Id952cb110c0187021a46ed5de02308b10ddf1239
SSH is required for replication to work in the ZfsOnLinux
driver, but the gate defaults to disabling it. Make it
configurable and make sure to preinstall host key.
Change-Id: Iece2ad2e966fafe3ab4d63c82d062830ac9f3170
Closes-Bug: 1552244
A devstack run with a keystone v3 only setup (ENABLE_IDENTITY_V2=False)
fails currently.
Partial-bug: #1553082
Change-Id: Ie8885d2c2f4d82d9894f4ca233153184125c20f2
The current devstack plugin has a hidden dependency on
Cinder, even for drivers that don't require Cinder (such
as LVM). This patch tries to eliminate that dependency.
Also avoid editing cinder.conf when Cinder isn't present.
Closes-Bug: 1553351
Change-Id: Iea2a83fc54412c18401e589ea0706ae95214ab9d
Removed functionality of Share Migration relying on Manila Share
Service node, moved code to Data Service node for copy phase.
Added parameter 'notify' and share/api methods for future
implementation (see dependent patches).
Added new copy operation statuses, in order to implement future
API calls to obtain progress and cancel migration.
Added possibility of 2-phase migration for driver migration and
generic (fallback) migration.
Added admin export location support and removed approach of
replacing IP with config parameter.
Added Admin-only API entry points to:
- Migration Cancel (only during copying)
- Reset Task State field
- Migration Get Progress (only during copying)
- Migration Complete (2nd phase migration)
- Notify parameter on Migrate Share
APIImpact
DocImpact
Implements: blueprint data-service-migration
Change-Id: I1d65aac2f36942cd70eb214be561d59a15a4ba26
This driver exploits a container as a share server instead
of a virtual machine. The container acts as a NFS/CIFS server
and shares logical volume attached to it. The volume gets created
simultaneously to the container on a host in a volume group
specified by the user. Currently NFS is limited to v3 since the
only userspace NFS server that works in a container out of the box
seems to be unfs3. Later it will be replaced with nfs-ganesha.
Co-Authored-By: Alexey Ovchinnikov <aovchinnikov@mirantis.com>
Implements: blueprint lxc-driver
Depends-On: Ib1896997f2e7a505b5bf8ec0c9e5ee35942f79a6
Change-Id: Iea73bd34b94919d77ebe17cf054dde1f580384be
Added a new service defined as data, whose purpose is to
receive requests, process data operations such as copying,
migration, backup, and send back the response after operation
has been completed.
In this patch the service has no methods, it is empty, the
operation it should perform will be added in subsequent patches.
Implements: blueprint data-copy-service
Change-Id: I31365c8ac4197541af175c82f8f18a3d11740a99
Implemented admin network support in generic driver
by creating extra ports in service instance.
In order to implement admin export location, Admin must
specify admin_network_id and admin_subnet_id
in driver configuration in manila.conf to create ports
between host and network of choice.
Possible scenarios are covered by this change:
*1: service network.
*2: service network and tenant network.
*3: service network and admin network.
*4: tenant network and admin network.
Admin network and tenant network configurations have
higher priority than service network. If both are
present, service network is not used.
Only admin network export locations are
is_admin_only = True.
Included additional admin_only export location
when creating shares and creating shares from
snapshots.
Removed deprecated 'service_ip' property of
server backend details.
Implements: blueprint admin-network-generic-driver
Change-Id: I8d8694ac3d83aa12a756112dfefebd7e17e32383
Add support for ZFS (on Linux) filesystem.
Following features are supported:
- Create/delete share
- Create/delete snapshot
- Create share from snapshot
- Extend/shrink share
- Update NFS IP-based access rules using new interface.
(SMB support planned for future).
- create/delete/update/promote share replica
ZFS-related notes:
- Any amount of ZFS zpools can be used by share driver.
- Allowed to configure default options for ZFS datasets that are used
for share creation.
- Any amount of nested datasets is allowed to be used.
- All share replicas are read-only.
- All share replicas are synched periodically, not continuously.
So, status 'in_sync' means latest sync was successful.
Time range between syncs equals to value of
config global opt 'replica_state_update_interval'.
Driver-related notes:
- Able to use remote ZFSonLinux storage as well as local.
Other made changes:
- updated driver private data DB methods removing filtering by host
as redundant operation. Replication requires some common metadata
storage and filtering by host breaks it. It is safe to do so, because
if driver gets some ID of entity then it is allowed to read its info
too.
Implements bp zfsonlinux-driver
DocImpact
Change-Id: I3ddd3767184e4843037de0ac75ff18dce709b6dc
Many Manila drivers don't require Nova, but the devstack
plugin still depends on Nova. This change allows using
the devstack plugin without Nova.
Change-Id: Ie76300f5af605365e327e7528a6d5aefa85a74fe
Manila CI jobs create lots of shares and for Generic driver it means
lots of Cinder volumes. After merge of [1] to Cinder project, we started
having NoValidHostFound errors in Cinder logs, it changed default
oversubscription ratio to 1.0 from 20.0. But Manila CI requires it
be bigger than 1.0, so set it explicitly to Cinder's default value 20.0
that is used by all other driver but not LVM.
[1] I5a214462b573d00d26086512821b4f8cc1db4fc3
Change-Id: If0a6e535b92b1471572e1de98e3b44ab6a2c65c9
Closes-Bug: #1540947
Devstack always installs test-requirements, but we do not expect it for
Manila UI project. Moreover, such installation started failing with
PIP versions 8.x
Change that triggered bug is Devstack change at [1].
[1] I49658ce4056c773943321270defd461bbf3e9fb9
Change-Id: I841b56f65b50e9c5673cc02e93b4329ed29125ba
Closes-Bug: #1540188
Devstack function "recreate_database" takes only one argument.
UTF8 is activated by default, see:
https://review.openstack.org/#/c/126267/
Trivial-fix
Change-Id: Ie43fdaa6f60641bf6d5717f4e358434758dae8e6
Option "verbose" from group "DEFAULT" is deprecated for removal.
Its value may be silently ignored in the future.
If this option is not set explicitly, there is no such warning.
Furthermore, the default value of verbose is true, so there is no
need to set this value in config files.
And we use oslo_config.generator to generate a sample file.
We'd better fix the doc not encourage user to set verbose=True.
Change-Id: Ie9a68e511425d34cbce04e416f258db3064d0479
Reuse code of old LVM share driver.
LVM Driver is 1st party Manila driver with NFS and CIFS support
and no share server support.
LVM Driver doesn't rely on Neutron, Nova, or Cinder. As such, it
is ideal for testing purposes and, after a period of maturation,
it should be useful in production environments.
Move generic driver's helpers to manila/share/drivers/helpers.py
and reuse them in LVM driver.
Implement 'ro' access, user access, extend_share function for
LVM driver.
Implements bp lvm-driver
Change-Id: Ia46c51ed400dbb0f1d87a758fb165ca711ed3d7c
Modifications to neutron config files must be conditional
on neutron actually being enabled.
Change-Id: I89846cbc438210c71278a98f9ffa7fd02c6a718e
Closes-Bug: #1534592
Recent devstack change [1] made Horizon be installed at later point than it
was. It is inconsistent with our Devstack plugin that now installs Manila UI
earlier than Horizon. So, postpone installation of Manila UI until Horizon
exists and fix gates by this action.
[1] I8bf2ceaf7734c4f7cec68bc05d7cdbae81ef311e
Change-Id: Icb2fc77235cb7f6ec4349dacddf262c3e3be76c2
Closes-Bug: #1532300
After merge of change [1] to devstack project, preconfigured creds for
Tempest cannot be used anymore. But Manila requires such.
For the moment, latest Tempest interfaces cannot allow us to use
preconfigured creds because of bug #1524717.
So, until this bug fixed, we should workaround Devstack changes and
set legacy opts explicitly.
Also, redefine new Nutron option 'dnsmasq_local_resolv' to 'False' value,
that is 'True' by default and was added in change [2].
Default value causes Nova operations initiated by Manila be very slow.
That leads to breakage of CI.
[1] I65b56ff681d6c27094380693c953fbc3664eceb0
[2] I17a884f467d307432a06f67a9dd93ed2fa6081a3
Change-Id: I516a6c9ab4396b14e3984d5f810210e4fcf7ec85
Related-Bug: #1531049
Closes-Bug: #1530847
As per the plugin documentation, for reference
http://docs.openstack.org/developer/devstack/plugins.html
the config environment variables should be added to
"devstack/settings" file
Change-Id: I676fd091c22d2bef71cb988f1163a811bf4cd1de
With oslo-incubator going away, we need to pull those classes into
the Manila code base, along with their unit tests. This presents a
good opportunity to do some long-needed housecleaning. This commit
does the following:
1. Moves the scheduler classes from openstack.common to manila.
2. Adds the unit tests from olso-incubator into Manila.
3. Removes duplication among the combined scheduler modules.
4. Moves scheduler drivers into a sub-module.
5. Normalizes class and module naming throughout the scheduler.
6. Splits some unit test files so they match the names of the
modules that they test.
7. Converts usage of mox & oslotest to mock & unittest.
8. Adds a few unit tests to boost coverage levels.
Implements: blueprint reorganize-manila-scheduler
Change-Id: I7aa237e17787e89a95bb198093ea9bc9498279cd
Right now devstack plugin has old workaround for python-manilaclient
that sets usage of Keystone v2 API for Manila requests.
So, remove it as it is not needed anymore since
python-manilaclient release 1.5.0 that supports Keystone API v3.
Change-Id: I42ce87ed6414b9048e49e24e5368a3f78c79acc5
This patch addresses issues commented on
change I64b0a3ee77b27278cc294f72702408a27888e0e9
after it was merged, according to bug below.
Closes-bug: #1494000
Change-Id: I9079ea16e0edc359b380705bebba2a7e98446d5c
To prevent a microversioned client from managing a non-microversioned
Manila server, Manila must update its REST endpoints by adding /v2 for
all microversioned APIs.
This commit does the following:
* Add /v2 to the URL map, connected to all the same /v1 API methods
* Renumber the microversion sequence starting from 2.0
* Update the versions API to reflect v2
* Publish the new endpoint to Keystone in the DevStack plug-in
* Update relevant documentation
* Update Tempest tests for microversions
APIImpact
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Closes-Bug: 1488624
Change-Id: I56a516b5f81914557dd2465746629431cfd6deac
3rd party CI tests do not use the service image, and
its download can be slow and fail resulting in an
unneceassirly slow or false-fail CI job.
Add an option to skip it.
Closes-Bug: #1477772
Change-Id: Ifc6d0287c40e14265c4866a0558bc84807bad2ec
Use novaclient for creation of keypair instead of openstack client in
devstack plugin, because the latter is not compatible with
nova microversion 2.1
Also, specify explicitly Nova API version in manila wrapper over
novaclient that is used by Generic and Windows drivers.
Also, revert back amount of tempest threads for Ci jobs to avoid job
timeout failures.
Also, fix raise of SnapshotBuildErrorException in Tempest plugin.
Tempest tries to raise exception SnapshotBuildErrorException from wrong
place. It should use "share_exceptions" instead of "exceptions".
Change-Id: Ie9ea9f6d25a13f8cd8f0ccbc73f6fa75cca35e0b
Closes-Bug: #1493008
Closes-Bug: #1493072
