4609 Commits

Author SHA1 Message Date
Zuul
49afbfc0fc Merge "Allow configuration of a back end specific availability zone" 2019-01-24 11:52:28 +00:00
Zuul
074876a1d5 Merge "Drop [DEFAULT]root_helper config option" 2019-01-24 00:52:46 +00:00
Goutham Pacha Ravi
a75fe3d7cc Allow configuration of a back end specific availability zone
"storage_availability_zone" in the [DEFAULT] section of
manila's configuration file has allowed deployers to configure
and manage both service (scheduler, share manager) and storage
system availability. However, quite often manila's services
(api, scheduler, share and data managers) are run on a dedicated
control plane that is a different failure domain from that of
the storage that manila manages.

Also, when using share replication, deployers would need to
run multiple manila share manager services with different
configuration files, each with their own
"storage_availability_zone".

To allow flexibility of separating service and storage
availability zones, we introduce a new configuration option
"backend_availability_zone" within the share driver/backend
section. When this option is used, it will override the value
of the "storage_availability_zone" from the [DEFAULT] section.

Change-Id: Ice99a880dd7be7af94dea86b31a6db88be3d7d9b
Implements: bp per-backend-availability-zones
2019-01-23 17:13:24 +00:00
zhufl
7e28b872af [Trivial fix] add missing ws seperator between words
This is to add missing ws seperator between words.

Change-Id: I314f8a2e8de4df2179504cfaee2c372c774d1f19
2019-01-23 14:36:00 +08:00
Thomas Bechtold
5ce6f3843e Drop [DEFAULT]root_helper config option
The option is not used anywhere in the codebase. Manila uses "sudo"
together with "rootwrap".

Change-Id: If276a1bac32f5273c433805263d507e2fc3b2f26
2019-01-23 07:10:55 +01:00
Yong Huang
e4156ddb08 [Unity] Revert to snapshot support
Implement revert to snapshot support for Dell EMC Unity driver.

Change-Id: I3a4da4a506ae4c2dbcd97207121ce19ecfb98043
Implements: unity-revert-to-snapshot
2019-01-16 15:20:47 -05:00
Tom Barron
8f88779778 Convert dummy job to py3
Install all python packages for python3 and
run tempest with python 3.

Change-Id: If25075933b0b07a12814fda44ef5139a40c78f67
2019-01-08 19:05:53 +00:00
Zuul
165e28ad84 Merge "Publish sample config file in the genconfig job" 2019-01-08 11:37:41 +00:00
Tom Barron
575d6ae621 Set paramiko logging to DEBUG level
Connections to backends via paramiko often fail for obscure
reasons.

If the service log level is DEBUG, increase paramiko logging
to DEBUG level as well so that we have more information when issues
of this sort occur.

Change-Id: I44d8902d58c2ad48b8a37dfbff96c5b9471651e4
2019-01-05 00:49:08 +00:00
Tom Barron
1814ad4113 Change ssh_utils parameter to correctly send keepalive packets
The current implementation of ssh_utils never sends keepalive packets.
In ssh_utils.SSHPool, the socket timeout parameter is set to None
intending to keep ssh connections open.
However, when the parameter is set to None, ssh_utils does not run the
code to compare idle duration and keepalive interval.

This patch reverts the socket timeout parameter to default (0.1 sec).
The ssh_utils compares them every 0.1 seconds, and sends a keepalive
packet if idle duration > keepalive interval (= self.conn_timeout).

See cinder change: I8234083107207b9ebc0849947e8de92b5cf3e36e

Change-Id: Ib13e5c6246412d667554cb0bd9c419b513af70c7
Related-Bug: #1673662
2019-01-05 00:48:23 +00:00
Zuul
a8daa9540c Merge "Improve service instance module debug logging" 2019-01-05 00:47:56 +00:00
Zuul
70b92620ff Merge "Move/Drop useless SQL related config options" 2019-01-04 21:06:56 +00:00
Zuul
d1b2ecfbbe Merge "devstack: Do a vgscan before checking if the VG is there" 2019-01-04 19:07:36 +00:00
Zuul
77fe6ae7d6 Merge "QNAP: Fix inconsistent cases while create/manage from snapshot" 2019-01-04 16:55:04 +00:00
Zuul
d88bbd2496 Merge "Fix the misspelling of "except"" 2019-01-04 15:26:32 +00:00
Zuul
e96e46d0d8 Merge "Add Ubuntu Bionic CephFS jobs" 2019-01-04 14:48:15 +00:00
Thomas Bechtold
49e68f8409 devstack: Do a vgscan before checking if the VG is there
There might be a rescan for volume groups needed before checking if
the volume group is already there. Otherwise, the check for the volume
group fails which means the code tries to create the volume, but that
fails then because the volume is already there.
Here is the devstack run log:

  [...]
  plugin.sh:configure_backing_file:538  sudo losetup -f --show /opt/stack/data/lvm-shares-backing-file
  plugin.sh:configure_backing_file:538  DEV=/dev/loop2
  plugin.sh:configure_backing_file:543  sudo vgs lvm-shares
    Volume group "lvm-shares" not found
    Cannot process volume group lvm-shares
  plugin.sh:configure_backing_file:543  sudo vgcreate lvm-shares /dev/loop2
    Physical volume '/dev/loop2' is already in volume group 'lvm-shares'
    Unable to add physical volume '/dev/loop2' to volume group 'lvm-shares'
    /dev/loop2: physical volume not initialized.
  plugin.sh:configure_backing_file:1  exit_trap
  [...]

Change-Id: I0189580ae40b180249fd5846aa986052e44ce2c2
2019-01-04 12:01:34 +01:00
Chris Yang
ad62e9dde3 QNAP: Fix inconsistent cases while create/manage from snapshot
There are two situation may cause the size of share/snapshot
managed by manila is inconsistent with the NAS backend.
One is to create a share from snapshot. While the other one
is to manage an existing snapshot.

Change-Id: Iaef8d8cb4be0d8872a2796c0fc69279c14f15a80
Closes-Bug: #1810476
2019-01-04 18:38:32 +08:00
lijunjie
9557516861 Fix the misspelling of "except"
Change-Id: I19a5b802faf88aeb522cf14f474d79d8e226ea34
2019-01-04 17:30:30 +08:00
Zuul
7665ff4404 Merge "Drop trycmd() from manila/utils.py" 2019-01-04 07:15:36 +00:00
Goutham Pacha Ravi
116becf381 Publish sample config file in the genconfig job
Add a post run to enable copying the sample file
generated to zuul logs.

Change-Id: Ifc1e71ce486ef9cefc07dbe9fb7c78ce150317bd
2019-01-04 04:01:51 +00:00
Zuul
7f8f4bf5c9 Merge "QNAP: driver should not manage snapshot which does not exist" 2019-01-03 20:07:09 +00:00
Tom Barron
2117632c50 Improve service instance module debug logging
Change-Id: Ib3a9bebe362609d7198e053afebc6004f3d94baf
2019-01-03 11:06:30 -06:00
Thomas Bechtold
941354d6cf Move/Drop useless SQL related config options
The following config option was moved to another place:

- [Default]sqlite_db : Only used for testing so move it to the test
  config options. That way, it is not visible in the
  manila.conf.sample file

The following config options were removed and are now handled via
oslo.db:

- [Default]sqlite_synchronous : already handled by oslo.db . The new
  place is "[database]sqlite_synchronous" but the deprecated
  section/names are handled
- [Default]sql_idle_timeout : already handled by oslo.db . The new
  place is "[database]connection_recycle_time" but the deprecated
  section/names are handled
- [Default]sql_max_retries : already handled by oslo.db . The new
  place is "[database]max_retries" but the deprecated
  section/names are handled
- [Default]sql_retry_interval : already handled by oslo.db . The new
  place is "[database]retry_interval" but the deprecated
  section/names are handled

Change-Id: I17e9b69d4ce4b8e624313d05f72d25b3fc2b458c
2019-01-03 15:55:29 +01:00
Thomas Bechtold
0c177b902a Drop param2id() from cmd/manage.py
The function is not used anywhere.

Change-Id: I799e73992faa387e56a793d60cc599039d268f81
2019-01-03 14:10:01 +01:00
Thomas Bechtold
51a37a7aff Drop trycmd() from manila/utils.py
The function is not used anywhere.

Change-Id: If3daf29df1e47974366b6e263827847683929489
2019-01-03 13:56:22 +01:00
Chris Yang
cb7b8e292a QNAP: driver should not manage snapshot which does not exist
The snapshot which does not exist in NAS shuold not be managed.

Change-Id: I06475ccc507ee62f3da8f8795694f6da14064036
Closes-Bug: #1810374
2019-01-03 17:03:21 +08:00
Tom Barron
859d5ed80d Add Ubuntu Bionic CephFS jobs
We need these for python3 support since the CentOS
jobs will not run correctly under python3.

Depends-on: https://review.openstack.org/#/c/627037/
Change-Id: I0e4baaaca5046f9c0ee32cf3de78133f743fd66d
2019-01-02 14:59:54 -06:00
Thomas Bechtold
2a27a433d3 Drop is_eventlet_bug105() from manila/utils.py
The function is not used anywhere.

Change-Id: I9dd7aedec35c6295e764301e7f5a81a126de85df
2019-01-02 20:29:27 +01:00
Zuul
d144bf7593 Merge "[DevRef] Add code review guideline" 2019-01-02 14:52:36 +00:00
Zuul
13d7fe9722 Merge "QNAP: Support QES FW on TDS series NAS" 2019-01-02 13:38:49 +00:00
Zuul
bf9f61b6b2 Merge "Adjust ssh timeouts" 2019-01-02 12:12:35 +00:00
Zuul
fda32b169b Merge "Add devstack instructions and local.conf samples" 2019-01-02 09:27:03 +00:00
Chris Yang
bc2a18eee5 QNAP: Support QES FW on TDS series NAS
Support both QTS and QES FW on TDS series NAS models.

Change-Id: I87129fae531b6d50b8490785badcb054a9dd9397
2019-01-01 14:20:40 +08:00
Tom Barron
7548706b09 Adjust ssh timeouts
Generic driver jobs are failing because of timeouts when
establishing the initial ssh connection from manila-share
to the service VM.

Bump up the default value of the connection timeout for paramiko
client and also set the banner timeout since the failure occurred
during banner exchange.  Set the two timeouts to the same value
for now.  This ensures that the connection timeout is at least as
long as the banner timeout and there is no current need in manila
to control these independently.

This is more of a workaround than a real fix since a real fix
would remove the delay during banner exchange.  I suspect that
the real fix will need to be in neutron/ovs though.

Change-Id: Ib5e59faaf9667b9cb5e7d4072531b7d6c3d4da39
Partial-bug: #1807216
2018-12-30 11:04:16 -06:00
Zuul
cf41d1867c Merge "[doc] Fix api sections in the contributor doc" 2018-12-28 14:35:26 +00:00
Zuul
f7b3448e0d Merge "[Manila Unity/VNX] add 'snapshot support' related Doc for Unity/VNX driver" 2018-12-27 12:38:55 +00:00
Goutham Pacha Ravi
e1619c4626 Add devstack instructions and local.conf samples
Restructure the development environment documentation
and include sample devstack config to get folks
started easy when they're coding manila changes.

Change-Id: If2df8bdf88e77eec32e8795020b8ef349b55b41b
2018-12-26 12:31:44 -08:00
Goutham Pacha Ravi
dd7a15c0ff [doc] Fix api sections in the contributor doc
Weed out outdated/unnecessary info and add link
to api ref.

Change-Id: Ia45c8ad6e2a697d5b76232e17e4df34539d81c12
2018-12-26 11:56:53 -08:00
Tom Barron
fc3109243c Set ram for manila service image to 256
At 192 with ubuntu bionic SVMs with this service image
crash at boot with OOM condition.

Change-Id: I75d0a39bf83c384f902d3e30e223fb0bd363d840
2018-12-25 21:49:41 -06:00
Zuul
86f71cb20d Merge "Deprecate old keystone session config opts" 2018-12-24 13:10:39 +00:00
dingd
734ffeac85 [Manila Unity/VNX] add 'snapshot support' related Doc for Unity/VNX driver
Change-Id: I76c8b31c890e5f8d7beeddc6b8ab1a3a0731bcfb
2018-12-24 10:07:58 +08:00
Goutham Pacha Ravi
198bea78ac Deprecate old keystone session config opts
In the past, the options ca_certificates_file,
nova_ca_certificates_file, cinder_ca_certificates_file,
api_insecure, nova_api_insecure, cinder_api_insecure
were supplied to instantiate nova, neutron and cinder
clients. These options have now been subsumed in a more
generic way into the Keystone session logic as 'cafile'
and 'insecure'. Deprecate the older options in Stein so
that we can remove them in a future release.

This deprecation began many releases ago when we switched
to using keystone sessions [1]. However, we were still
overriding the values of "insecure" and "cafile" if provided,
forcing users to continue using deprecated parameters
"api_insecure" and "ca_certificates_file". So despite
this fix originating in the Stein release, it would be
prudent to backport it to all maintained releases and
remove support for these older options in/beyond Train
release (9.0.0).

[1] Ic211a11308a3295409467efd88bff413482ee58d
Change-Id: I148e9079c7c1ab119f519f727d4ad97758473325
Related-Bug: #1802393
Closes-Bug: #1809318
2018-12-20 17:29:40 -08:00
Zuul
1a658eb187 Merge "Only run the needed services for CephFS jobs" 2018-12-20 11:52:33 +00:00
Tom Barron
6299ed5ed0 Fix image_name retrieval in custom-image jobs
And use a 'file://' url for the custom manila image.

See also https://review.openstack.org/#/c/623330.  These
are interdependent patches but no harm is done by merging
this one first and having 623330 depend on it since retrieval
of the image name is not working correctly anyways.

Partial-Bug: #1807969

Change-Id: Ib3a74d170f159dfbc38b5302f00800f3530f4921
2018-12-19 15:00:51 -05:00
Tom Barron
f096d1145c Only run the needed services for CephFS jobs
These jobs often fail due to timeouts so only enable the services
required for devstack tests with these back ends.

Change-Id: Ib620ec3c2a9999ea9908a7c23aa7172c40a4cbd9
2018-12-19 01:18:23 +00:00
Zuul
a724e3c276 Merge "NetApp ONTAP: cifs add AD security service server as preferred DC" 2018-12-18 07:57:48 +00:00
Luigi Toscano
e02a625e9c Use the canonical URL for Manila repositories
- When the URL refers to cloning or using git repositories, use the
  cloning URL (https://git.openstack.org/openstack/<project>)
- When the URL refers to the browsable version of the repository, use
  the cgit frontend (https://git.openstack.org/cgit/openstack/<project>)

Change-Id: I11a8e15c6d663786465c090b40db9e20b19194e6
2018-12-17 13:56:17 +01:00
Zuul
ef418b4e8d Merge "fix http link to https link" 2018-12-14 10:22:16 +00:00
chengebj5238
3a5ea7184a fix http link to https link
Change-Id: I07e1a7ed99a79b774a7a7ba58f407972f3b724c3
2018-12-14 08:33:03 +00:00