3rd party CIs cannot set this config by exporting an env var
or local.conf setting, as it is always being overriden in code.
Change-Id: Ia2a4790d3a0d03faa231b400c32f6c1928bdca1f
Closes-bug: #1649249
Recent change to devstack [1] broke our DHSS=False CI job running
generic driver that was depending on ip route from host to
private network. So, to avoid this error use floating ip address
for connection to service Nova VMs from host machine.
Also, fix generation of second export location that should be dedicated
for service needs such as mounting share doing host-assisted
migration.
[1] If45e3fc15c050cfbac11b57c1eaf137dd7ed816f
Change-Id: Ieea992293ae02898741c939da15f0dbb4609d8b0
Closes-Bug: #1644523
Closes-Bug: #1646097
Generic share driver has fake support of consistency groups. It was
implemented only for testing purpose. Now, it is redundant, because
we have 'dummy' share driver that already used to test this behavior.
So, drop fake CG support from Generic share driver and disable
appropriate tempest tests.
Change-Id: I6ce07fd3a11cd62a3a01ba4ee7c424b839a62757
Closes-Bug: #1638994
Make devstack grenade code use 'newton' as base branch for migration to
master (ocata).
After switch [1] to dummy driver we need to override defaults.
So, update pre_test_hook approach for saving configuration data properly.
[1] I2b698f69a29ef983b13534b2f4d393570be4dac8
Change-Id: Ie5fa0f78b0e3caa06d34a0822a8ed60b69406f9d
Closes-Bug: #1631412
Improve coverage by adding tests that validate the share-type
change while also changing the driver mode.
Closes-bug: #1620800
Change-Id: I924c34aa69591754b437d75f43db91d77e73fb07
Current implementation of share migration works only if we use
local shell executor. So, refactor approach for it to make work
with both shell executors - local and remote (SSH).
Change-Id: Ibe31bf52006040ea26badfa7236ed67559f7aded
Closes-Bug: #1620502
Add support of share migration feature to ZFsonLinux driver.
Implements BP zfsonlinux-share-migration
Change-Id: If9e1fec9a6d4f96d2bb5c176c6249fafc7e3e891
At Austin 2016 summit there were several improvements to
Share migration feature discussed. This patch implements
these changes.
Changes are:
- Added 'Writable' API parameter: user chooses whether share must
remain writable during migration.
- Added 'Preserve Metadata' API parameter: user chooses whether
share must preserve all file metadata on migration.
- Added 'Non-disruptive' API parameter: user chooses whether
migration of share must be performed non-disruptively.
- Removed existing 'Notify', thus removing 1-phase migration
possibility.
- Renamed existing 'Force Host Copy' parameter to 'Force
Host-assisted Migration'.
- Renamed all 'migration_info' and 'migration_get_info' entries to
'connection_info' and 'connection_get_info'.
- Updated driver interfaces with the new API parameters, drivers
must respect them.
- Changed share/api => scheduler RPCAPI back to asynchronous.
- Added optional SHA-256 validation to perform additional check if
bytes were corrupted during copying.
- Added mount options configuration to Data Service so CIFS shares
can be mounted.
- Driver may override _get_access_mapping if supports a different
access_type/protocol combination than what is defined by default.
- Added CIFS share protocol support and 'user' access type
support to Data Service.
- Reset Task State API now allows task_state to be unset using
'None' value.
- Added possibility to change share-network when migrating a share.
- Bumped microversion to 2.22.
- Removed support of all previous versions of Share Migration APIs.
APIImpact
DocImpact
Implements: blueprint newton-migration-improvements
Change-Id: Ief49a46c86ed3c22d3b31021aff86a9ce0ecbe3b
In Scenario job, tests consist of VMs mounting shares
provided by share servers.
Admin network is used by share migration tests and is
by default used in other tests as well. Previously it
used the same network address range as tenant networks
(used by VMs) and there was no conflict.
Recently some unknown change broke connectivity between
VMs and share server in scenario job. It seems as there
is now a conflict between those network addresses ranges.
This patch changes the admin network address range,
removing the existing conflict.
Closes-bug: #1616115
Change-Id: I5bdb70cae4eab2930f3f559a621643ad732909bb
This driver exploits a Docker container as a share server instead
of a virtual machine. The container acts as a CIFS server
and shares logical volume attached to it. The volume gets created
simultaneously to the container on a host in a volume group
specified by the user.
DocImpact
Change-Id: I711813a2022d765ab7a70ed670e34f68f9cf1399
Implements: blueprint manila-container-driver
Implement 'manage snapshot' feature in ZFSonLinux share driver.
Prerequisites:
- snapshot being managed should exist.
Details:
- snapshot being managed will be renamed by share driver based on
taken ID.
- size for managed snapshot will be taken either from
REST API (--driver_options size=5) or
from its "used" attribute.
Implements BP zfsonlinux-manage-snapshot
DocImpact
Change-Id: If240ecde5676c334d39faaccd5703e93544aaa06
Depends-On: Ifc762a882a78159adacfe168a4edbe824178301a
Added dummy driver for ease of testing and development of REST APIs
and internal manila interfaces.
It can be enabled as backend in the same way as all other share drivers.
But, it does not provide any resources.
Also, update tempest to be able to work with share-networks when
networks are defined via config and not API.
CI hooks are configured to run tempest twice using both available
driver modes. Where DHSS=True mode uses StandAloneNetworkPlugin as
network backend.
Change-Id: I9053dddfc643cb5a6602f15235b91bbaea727dbd
Implements BluePrint dummy-driver
Tempest 12.2.0 was recently released and manila tempest plugin runs
just fine with it.
Update tempest in our CI to use tempest at the commit corresponding to
this tag.
Change-Id: Ia00670a5cf0b52a41fa0d3cd9a82cd81f7802155
Neutron is using tempest plugins too and it's needed newer changes
in tempest we need to update our commit id.
Change-Id: Ic6621fbad9e5ba932643e0c0fb717b71a1fc5808
Closes-bug: #1599429
Implement 'manage share' feature in ZFSonLinux share driver.
Prerequisites:
- share being managed should exist.
- share being managed should not be part of any replication relations.
- share being managed should not have redefined mountpoint.
Details:
- share being managed will be renamed by share driver based on taken ID.
- share being managed will get "quota" attribute set based on provided
value in API (--driver_options size=5) or it will set quota
to nearest bigger rounded integer of already
used space (2,4 -> 3, 3.6 -> 4).
Implements blueprint zfsonlinux-manage-share
DocImpact
Change-Id: I03eab5e02835fe660dde0020601c02c0136bddeb
Fix small incompatibility that left to be done and bump Tempest version
to unblock our CI jobs.
Also, remove temporary fix [1] that was done for unblocking CI.
[1] I44655c83cc5588c5b94d0fad81162dd0c238b5b4
Change-Id: I12e2652638ba26833f9365f06fa84a98ca24765c
Currently, Manila CI Tempest jobs use deprecated options of Tempest.
Recently, latest version of Tempest dropped them [1].
Manila CI jobs work ok, because they check out specific version of
Tempest. But it will work until first Devstack update for some one
more Tempest feature and then we will be forced to use latest Tempest
again. Switch to supported approach of setting up users in Tempest,
to avoid such sudden blockers.
[1] I8c24cd17f643083dde71ab2bd2a38417c54aeccb
Change-Id: I5b3ebab52ea1401f6f7a116d1260268eb10ebe0c
Closes-Bug: #1586129
Recently merged commit to "neutron-fwaas" project [1] started using
one of Tempest's modules that appeared after version our plugin uses.
So, implement hotfix-workaround until we make all compatibility changes
to latest Tempest in our plugin.
For the moment, we cannot switch to latest Tempest as we already
have incompatibility for it, see bug [2].
[1] https://review.openstack.org/#/c/326150/
[2] https://bugs.launchpad.net/manila/+bug/1586129
Change-Id: I44655c83cc5588c5b94d0fad81162dd0c238b5b4
Closes-Bug: #1589964
Latest Devstack is not compatible with Tempest version
used by Manila after merge of commit [1].
Therefore, bump Tempest's version and fix incompatibilities
with plugin.
[1] I63658b8d8dfa999e0feb79f8f2968f2b32e3ff57
Change-Id: Ib6fb886f8a17ff6419fd75db74f290f821a3d302
Grenade is an OpenStack test harness to exercise the upgrade
process between releases. It uses DevStack to perform an initial
OpenStack install and as a reference for the final configuration.
Add support of it to Manila project, doing following changes:
- Add all required scripts for 'grenade' job, such as following:
-- settings
-- shutdown.sh
-- upgrade.sh
- Add optional 'resources.sh' script that performs functional
tests using bash and manilaclient. List of scenarios:
-- 1. create and delete share setting metadata and access rule
-- 2. attach security service to share network
-- 3. update quotas
-- 4. create private share type and add access for project
Note following:
- Tempest is not used by design.
- Manilaclient is used of stable version - version of 'base' devstack,
which is 'stable/mitaka'. It also means that used API microversion
is latest for that branch.
Change-Id: I7d7e0c50933776ca1a710bcf069abb0282b821ac
Depends-On: I4232382d3c944666bbf0916a3f285b84f6ff9d64
Implements blueprint grenade
We are forced to bump version of Tempest because of regular
introduced incompatibility.
Also, add compatibility fixes to Manila plugin for
latest Tempest version.
Change-Id: I13c838c39d99c008d53e544f54dadee01f36fdf2
Closes-Bug: #1568785
Generic driver fails from time to time on Cinder not being able
to schedule shares because of exceeded size limit.
Since we do not use size in API tests at all increase
Cinder oversubscription ratio from 20.0 to 100.0
to make it enough for sure.
Such error appears from time to time and
more often with addon of new Tempest tests that create shares.
TrivialFix
Change-Id: I8a14f806c0235f5a12930eee362a838404cd5c06
With merge of commit [1] all calls of clients in Manila
post_test_hook.sh script fail, because used Devstack's file 'openrc'
does not export env var that contains domain name. That became
required after switch to Keystone v3 API in CI jobs [1].
Note: manilaclient calls should have been worked, as default values are
automatically picked up by Keystone server. But such clients as nova
requires set of domain name explicitly.
[1] Ic211a11308a3295409467efd88bff413482ee58d
Change-Id: Ia1976af058164da8ef9e33296006750aab7b3f50
Closes-Bug: #1559456
This fix basically ports nova fix https://review.openstack.org/#/c/136931/
plus additional bug fixing to Manila. It creates a common class to get
used for all clients (neutron, nova, cinder).
Idea is to create an auth session and pass this to the client object
instead of let the client do that. The auth session will be created by
a config group which means the configuration for the clients needed to be
heavily reworked. Patch is also backward compatible with old options but
flag them as deprecated.
DocImpact
Change-Id: Ic211a11308a3295409467efd88bff413482ee58d
Closes-bug: #1555093
This reverts commit 466a19f18f12cb77b45aa7c7a272fae7c45c4c93.
Also this reverts commit 4f74b224fad7bf9ca665d70a66c79c98cf8d5674.
Manila has decided to remove LXD from the Mitaka release.
http://lists.openstack.org/pipermail/openstack-dev/2016-March/089522.html
Change-Id: I3c4b15e8c9d92c874be754205b0b981151809cc1
By default, these tests are disabled; 3rd party CI systems
should enable these tests through the
RUN_MANILA_REPLICATION_TESTS environment variable
if their driver supports share replication.
Partially implements: blueprint manila-share-replication
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Change-Id: Idaf3c61600337cc30568b982dfb23948209c1d6e
SSH is required for replication to work in the ZfsOnLinux
driver, but the gate defaults to disabling it. Make it
configurable and make sure to preinstall host key.
Change-Id: Iece2ad2e966fafe3ab4d63c82d062830ac9f3170
Closes-Bug: 1552244
Download of Generic/Windows image for Nova should be disabled for
all other share drivers in CI.
Change-Id: Ib2c952e85dd0a6eb36fe945e0a33950c38dfdfae
Closes-Bug: #1553343
Removed functionality of Share Migration relying on Manila Share
Service node, moved code to Data Service node for copy phase.
Added parameter 'notify' and share/api methods for future
implementation (see dependent patches).
Added new copy operation statuses, in order to implement future
API calls to obtain progress and cancel migration.
Added possibility of 2-phase migration for driver migration and
generic (fallback) migration.
Added admin export location support and removed approach of
replacing IP with config parameter.
Added Admin-only API entry points to:
- Migration Cancel (only during copying)
- Reset Task State field
- Migration Get Progress (only during copying)
- Migration Complete (2nd phase migration)
- Notify parameter on Migrate Share
APIImpact
DocImpact
Implements: blueprint data-service-migration
Change-Id: I1d65aac2f36942cd70eb214be561d59a15a4ba26
tempest_lib was renamed to tempest.lib which causes problems
when running tests because tempest imports plugins from other
projects and crashes.
Closes bug: 1552520
Change-Id: Id7d43dee256856062bd0dadad9c469c1500ee36e
This driver exploits a container as a share server instead
of a virtual machine. The container acts as a NFS/CIFS server
and shares logical volume attached to it. The volume gets created
simultaneously to the container on a host in a volume group
specified by the user. Currently NFS is limited to v3 since the
only userspace NFS server that works in a container out of the box
seems to be unfs3. Later it will be replaced with nfs-ganesha.
Co-Authored-By: Alexey Ovchinnikov <aovchinnikov@mirantis.com>
Implements: blueprint lxc-driver
Depends-On: Ib1896997f2e7a505b5bf8ec0c9e5ee35942f79a6
Change-Id: Iea73bd34b94919d77ebe17cf054dde1f580384be
Add support for ZFS (on Linux) filesystem.
Following features are supported:
- Create/delete share
- Create/delete snapshot
- Create share from snapshot
- Extend/shrink share
- Update NFS IP-based access rules using new interface.
(SMB support planned for future).
- create/delete/update/promote share replica
ZFS-related notes:
- Any amount of ZFS zpools can be used by share driver.
- Allowed to configure default options for ZFS datasets that are used
for share creation.
- Any amount of nested datasets is allowed to be used.
- All share replicas are read-only.
- All share replicas are synched periodically, not continuously.
So, status 'in_sync' means latest sync was successful.
Time range between syncs equals to value of
config global opt 'replica_state_update_interval'.
Driver-related notes:
- Able to use remote ZFSonLinux storage as well as local.
Other made changes:
- updated driver private data DB methods removing filtering by host
as redundant operation. Replication requires some common metadata
storage and filtering by host breaks it. It is safe to do so, because
if driver gets some ID of entity then it is allowed to read its info
too.
Implements bp zfsonlinux-driver
DocImpact
Change-Id: I3ddd3767184e4843037de0ac75ff18dce709b6dc
Fix several issues in scenario test to get them stable again:
- The SSH timeout was set to 80 seconds what is way too low for an
Nova instance to come up in the gate.
- Scenario test access rule cleanup is not necessary, access rules
are deleted when share instance is deleted.
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Closes-bug #1545715
Change-Id: I9f22f0416bb248666fe906e1ca653517054f21b5
Add APIs to support manage and unmanage share snapshots.
Also add support in the Generic driver.
This only supports for DHSS=False driver mode.
Add provider_location column to the share_snapshots table
to save data used to identify the snapshot on the backend.
Also need to bump microversion.
APIImpact
DocImpact
Change-Id: I87a066173c85d969607d132accd9f0e9bd49c235
Implements: blueprint manage-unmanage-snapshot
The post_test_hook.sh script was hardcoding the Tempest concurrency
setting to 8 for DHSS=False test runs. This patch has it honor the
MANILA_TEMPEST_CONCURRENCY environment variable, if it exists.
Change-Id: Ia2f0495560517484ba4d9290ced2c4dc3de6fb34
Closes-bug: 1546150
- Add update_access() method to driver interface
- Move all code related to access operations to ShareInstanceAccess
class
- Statuses from individual access rules are now mapped to
share_instance's access_rules_status
- Add 'access_rules_status' field to share instance, which indicates
current status of applying access rules
APIImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Tiago Pasqualini da Silva <tiago.pasqualini@gmail.com>
Implements: bp new-share-access-driver-interface
Change-Id: Iff1ec2e3176a46e9f6bd383b38ffc5d838aa8bb8
Manila CI jobs create lots of shares and for Generic driver it means
lots of Cinder volumes. After merge of [1] to Cinder project, we started
having NoValidHostFound errors in Cinder logs, it changed default
oversubscription ratio to 1.0 from 20.0. But Manila CI requires it
be bigger than 1.0, so set it explicitly to Cinder's default value 20.0
that is used by all other driver but not LVM.
[1] I5a214462b573d00d26086512821b4f8cc1db4fc3
Change-Id: If0a6e535b92b1471572e1de98e3b44ab6a2c65c9
Closes-Bug: #1540947
After merge of change [1] to Devstack project Manila Tempest plugin not
compatible anymore with latest Devstack.
So, bump Tempest version to latest and fix compatibility with it.
[1] If66096cde31fedda407e1e6ac6899b196569a454
Change-Id: Idf9a9eabf9c98d090e47207cf89d9e5a5fc0ea97
Closes-Bug: #1539007
Reuse code of old LVM share driver.
LVM Driver is 1st party Manila driver with NFS and CIFS support
and no share server support.
LVM Driver doesn't rely on Neutron, Nova, or Cinder. As such, it
is ideal for testing purposes and, after a period of maturation,
it should be useful in production environments.
Move generic driver's helpers to manila/share/drivers/helpers.py
and reuse them in LVM driver.
Implement 'ro' access, user access, extend_share function for
LVM driver.
Implements bp lvm-driver
Change-Id: Ia46c51ed400dbb0f1d87a758fb165ca711ed3d7c
Use arguments of Devstack hook scripts to configure Manila and Tempest in
CI jobs instead of dependency on names of CI jobs.
Change-Id: I96d8bca54320f4476789c00e1061d4e96b697d05
Depends-On: I521f0419e65e27f22acbc03aaa1f0f997492e8f8
After merge of change [1] to devstack project, preconfigured creds for
Tempest cannot be used anymore. But Manila requires such.
For the moment, latest Tempest interfaces cannot allow us to use
preconfigured creds because of bug #1524717.
So, until this bug fixed, we should workaround Devstack changes and
set legacy opts explicitly.
Also, redefine new Nutron option 'dnsmasq_local_resolv' to 'False' value,
that is 'True' by default and was added in change [2].
Default value causes Nova operations initiated by Manila be very slow.
That leads to breakage of CI.
[1] I65b56ff681d6c27094380693c953fbc3664eceb0
[2] I17a884f467d307432a06f67a9dd93ed2fa6081a3
Change-Id: I516a6c9ab4396b14e3984d5f810210e4fcf7ec85
Related-Bug: #1531049
Closes-Bug: #1530847
