169 Commits

Author SHA1 Message Date
Goutham Pacha Ravi
ffb283a04a [devstack][ci] Stop requiring g-reg service
When using manila's devstack plugin, we pull down
the latest manila testing image created with the
manila-image-elements project, and upload it to the
stack's Glance/Image service.

Glance's V2 API no longer needs g-reg which was a
legacy database proxy service for the older glance
API. Our devstack plugin should not have a hard
dependency on this service.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-May/094773.html
[2] https://specs.openstack.org/openstack/glance-specs/specs/newton/approved/glance/deprecate-registry.html

Change-Id: I58603032fb215ac0be2f76c8cd9fbce775be41dd
Closes-Bug: #1859847
2020-04-28 16:44:59 -07:00
Goutham Pacha Ravi
52cbbcb04e [grenade] Switch base version
Grenade jobs need to test Ussuri-->Victoria (dev)
upgrades in the master branch.

Change-Id: I95d1efb35f1c3c311071448117406bc275281e43
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-04-26 23:50:29 +00:00
Goutham Pacha Ravi
e0da21fe8d [CI] Fix grenade share networks test
This job fails because it's expecting network subnet
details to be in a format that was revised in API version
2.51.

This change can be backported to stable/ussuri
and only impacts the CI.

Change-Id: I3731e6acffccd2ecb398f3b5cf81a5f294c51f45
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-04-26 23:50:23 +00:00
Zuul
10d5cbf639 Merge "Enable the use scheduler creating share from snapshot flag" 2020-04-14 19:01:00 +00:00
Tom Barron
c0ee00f2c7 Increase MANILA_SERVICE_VM_FLAVOR_DISK
The manila service image was recently rebuilt and is
now bigger than 2 Gibibytes so increase it to 3, which
should last us for a while.

Closes-bug: #1870412

Change-Id: Ia03e48a3f0ff352b94a0224f76f278ef45acad93
2020-04-02 21:03:27 +00:00
Goutham Pacha Ravi
6f79bb6c89 Fix URLs in code and documentation
TrivialFix

Change-Id: I21863f7fdad2bdd5dee226a3f6872d25b1c4f8ad
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-03-18 18:59:27 -07:00
andrebeltrami
5088177006 Enable the use scheduler creating share from snapshot flag
This patch enables the use scheduler creating share from
snapshot flag in order to run properly the manila tempest tests.

Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend

Change-Id: I15311089c45be1574857d46caa073e89424e716d
2020-03-11 17:38:21 +00:00
Tom Barron
0c22a61335 Skip NFS/Samba install for CephFS
Today we either expose the CephFS back end via the CephFS native
protocol, in which case kernel NFS and Samba are not needed, or
by NFS-Ganesha, in which case Samba is not needed and kernel NFS
is incompatible (we just turn it off again later).

In the future someone could develop support for exposing CephFS via
Samba so we'd add specific checks for that then and install Samba.

Change-Id: Id65ce2536a3deb32a04a70bd0c49081297741965
2019-09-22 20:53:11 +00:00
Zuul
77ecab5f36 Merge "Remove support for `data_node_access_ip`" 2019-09-13 16:40:15 +00:00
Goutham Pacha Ravi
9551623c22 Remove support for `data_node_access_ip`
It was deprecated in the Queens release, in
favor of ``data_node_access_ips``.

Change-Id: I01f24c2e0d66e1da4c30c579c02afc8f3930c8f8
Related-Bug: #1745436
2019-09-09 14:45:30 +00:00
Zuul
0ebd368b44 Merge "Fix usage of deprecated devstack function" 2019-08-14 15:35:15 +00:00
zhangbailin
25506de0e4 Add manila-ui config instructions
In Devstack, even though horizon and manila plugins are enabled, if the
manila ui plugin is not explicitly enabled it won't show up.

This change adds config instructions in the readme and in the manila
contributor docs to enable Manila UI when deploying with Devstack.

Change-Id: I421db5b8ed56fecc90ac7e6d32078d443a3eaa2e
2019-07-31 18:31:22 +08:00
Goutham Pacha Ravi
d0bb79b30a Fix usage of deprecated devstack function
configure_auth_token_middleware was replaced by
configure_keystone_authtoken_middleware in
Id0dec1ba72467cce5cacfcfdb2bc0af2bd3a3610. Manila's
DevStack plugin no longer needs to create a cache
directory to use as the signing_dir for keystone
authtoken.

Change-Id: I18fa04e6c3a832e17ade3f84d8b43b15686f06fe
2019-07-30 13:19:24 -07:00
Goutham Pacha Ravi
8d03a2a46e Make manila-tempest-plugin installation optional
manila-tempest-plugin can be installed with its
devstack plugin; Installing it via manila's plugin
is unnecessary. So, deprecate its installation
in the DevStack plugin.

Change-Id: I21c08069ff82b3bfb52ef7ac960183ddc866c2ee
2019-07-29 17:54:09 -07:00
Zuul
6a07960a70 Merge "Conditionally restore default route in setup_ipv6" 2019-07-26 14:27:29 +00:00
Tom Barron
70e908344d Conditionally restore default route in setup_ipv6
Restoring the default IPv6 route is not needed in CI and
fails when there are multiple defaults, but it is useful
in local devstacks where multiple default routes are not
typical.

Add a variable in settings and use it to make this behavior
conditional and set it to False for the lvm job.

Closes-bug: #1836788
Change-Id: Id73de8100509ec5935641f5f35f93f482d108bcd
2019-07-24 19:09:27 +00:00
Tom Barron
9be39cbef2 add IPv6 support for CephFS/NFS back end
Change-Id: Id2b4edfa02521f9c9d9c27635e9f4945af79a0b7
2019-07-22 17:13:13 -04:00
Goutham Pacha Ravi
8358b5f4a0 [grenade] Switch base version
Grenade jobs need to test Stein-->Train (dev)
upgrades in the master branch.

Change-Id: I266d23c8cf875ba18a3bdd8e950b0d6feb91cb9f
2019-03-29 09:10:51 -07:00
Ian Wienand
4c42246b2a Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I4df9f22a60ef50677867eaf4baf9e7e55a951dda
2019-03-24 20:34:07 +00:00
Victoria Martinez de la Cruz
bd4673100b Deploy manila with uwsgi on devstack
Switch manila to use devstack common functions
for deploying a wsgi app under uwsgi and apache.

This change aims to fulfill the community requirement
for all projects on OpenStack to switch devstack jobs
to deploy control-plane API services under uwsgi with
Apache acting as a front end proxy.

More details in https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html

We used to deploy with mod_wsgi, but mod_wsgi has many
issues when being used in devstack for development or testing.

Now we leave mod_wsgi as optional and we default to uwsgi.

Implements: bp wsgi-web-servers-support
Depends-On: https://review.openstack.org/#/c/643188/
Change-Id: I12d4c864f2ceb0f3555e32d2dc893e00dbef0d96
2019-03-14 06:25:24 +00:00
Goutham Pacha Ravi
8f1c7dc91f Fix tls-proxy issues with the devstack plugin
Enabling tls-proxy allows devstack to
set up a tls proxy server that front-ends
interactions with the manila-api and
terminates tls connections.

Also enable tls-proxy in dummy and lvm
jobs. The dummy driver job is configured
to run the in-built wsgi server, the lvm
job is configured to use mod-wsgi.

Closes-Bug: #1816836
Change-Id: I48b0ccc082604d78242ba61bee94a45efeb2467b
2019-02-21 23:16:45 +00:00
Tom Barron
68b79e2cea Fix service image boot issues
We are getting:

403 errors 'Only volume-backed servers are allowed for flavors
with zero disk' on scenario tests.

Appears to be due to this change [1] which just merged to
nova master branch.  Ubuntu bionic server doc says the root disk
needs 1.5 GB so we are setting the flavor definition to require
2GB.

[1] c8e65a5eb1

Closes-bug: #1816050
Change-Id: Iba0a15b78bf75a04c1ac0e64e70634772b2dca5c
2019-02-15 11:22:43 -05:00
Goutham Pacha Ravi
22d25e834d Configure per backend availability zones in devstack
Every enabled backend gets assigned to its own AZ.
Test cases in manila-tempest-plugin already exercise
creating shares (and replicas) across AZs when
multiple AZs are available.

This is done for all back-end drivers that are not
the Generic driver. Configuring backend AZ for each
Generic driver based back end will require configuring
Nova and Cinder in a similar fashion.

Partially-implements: bp per-backend-availability-zones
Depends-On: https://review.openstack.org/#/c/630886/
Depends-On: https://review.openstack.org/#/c/629958/
Change-Id: I1b6ff535a22f10e70d379377767c8ffac3ef5286
2019-01-23 14:30:18 -08:00
Thomas Bechtold
49e68f8409 devstack: Do a vgscan before checking if the VG is there
There might be a rescan for volume groups needed before checking if
the volume group is already there. Otherwise, the check for the volume
group fails which means the code tries to create the volume, but that
fails then because the volume is already there.
Here is the devstack run log:

  [...]
  plugin.sh:configure_backing_file:538  sudo losetup -f --show /opt/stack/data/lvm-shares-backing-file
  plugin.sh:configure_backing_file:538  DEV=/dev/loop2
  plugin.sh:configure_backing_file:543  sudo vgs lvm-shares
    Volume group "lvm-shares" not found
    Cannot process volume group lvm-shares
  plugin.sh:configure_backing_file:543  sudo vgcreate lvm-shares /dev/loop2
    Physical volume '/dev/loop2' is already in volume group 'lvm-shares'
    Unable to add physical volume '/dev/loop2' to volume group 'lvm-shares'
    /dev/loop2: physical volume not initialized.
  plugin.sh:configure_backing_file:1  exit_trap
  [...]

Change-Id: I0189580ae40b180249fd5846aa986052e44ce2c2
2019-01-04 12:01:34 +01:00
Tom Barron
7548706b09 Adjust ssh timeouts
Generic driver jobs are failing because of timeouts when
establishing the initial ssh connection from manila-share
to the service VM.

Bump up the default value of the connection timeout for paramiko
client and also set the banner timeout since the failure occurred
during banner exchange.  Set the two timeouts to the same value
for now.  This ensures that the connection timeout is at least as
long as the banner timeout and there is no current need in manila
to control these independently.

This is more of a workaround than a real fix since a real fix
would remove the delay during banner exchange.  I suspect that
the real fix will need to be in neutron/ovs though.

Change-Id: Ib5e59faaf9667b9cb5e7d4072531b7d6c3d4da39
Partial-bug: #1807216
2018-12-30 11:04:16 -06:00
Tom Barron
fc3109243c Set ram for manila service image to 256
At 192 with ubuntu bionic SVMs with this service image
crash at boot with OOM condition.

Change-Id: I75d0a39bf83c384f902d3e30e223fb0bd363d840
2018-12-25 21:49:41 -06:00
Luigi Toscano
e02a625e9c Use the canonical URL for Manila repositories
- When the URL refers to cloning or using git repositories, use the
  cloning URL (https://git.openstack.org/openstack/<project>)
- When the URL refers to the browsable version of the repository, use
  the cgit frontend (https://git.openstack.org/cgit/openstack/<project>)

Change-Id: I11a8e15c6d663786465c090b40db9e20b19194e6
2018-12-17 13:56:17 +01:00
Goutham Pacha Ravi
8c925198ae [LVM][IPv6] Quagga changes to support Bionic Beaver
Pre-create config and log directories and
update Quagga service configuration.

This patch does not add Bionic support to the gate
jobs. https://review.openstack.org/#/c/608761/ does
the job configuration changes to run with Bionic.

Needed-By: https://review.openstack.org/#/c/608761/
Change-Id: I146ab5a41c06df2b2615ebdb38aaeb972db4c1d2
2018-11-19 22:14:34 +00:00
Zuul
20a2baac13 Merge "Use OS CLI instead of the neutronclient" 2018-11-19 20:31:54 +00:00
Goutham Pacha Ravi
8a02f469c8 Use OS CLI instead of the neutronclient
In the DevStack plugin.

Change-Id: I338634ab5e6360f4399d1df66c9ba3c9ba6c5ddf
2018-11-16 19:11:43 +00:00
Goutham Pacha Ravi
b1bf39412b [CI] Switch Xenial tempest jobs to Bionic Beaver
This change specifically ignores the grenade
job (manila-grenade), which is pending conversion
to zuulv3 because we need a way to set the default
python3 interpreter when running on Bionic Beaver.

Depends-On: https://review.openstack.org/#/c/605291/
Depends-On: https://review.openstack.org/#/c/605242/
Depends-On: https://review.openstack.org/#/c/605810/
Change-Id: Ic49d4ac5ac36dd3b3d909941434b2d2f104a10b9
2018-11-01 14:13:35 -07:00
Goutham Pacha Ravi
e3e3af52b1 [grenade] Switch base version
Grenade jobs need to test Rocky-->Stien (dev)
in the master branch.

Change-Id: Id7664187e1d95920be072cd85ebbb817ef5b087d
2018-10-01 22:48:26 -07:00
Goutham Pacha Ravi
9f0b35b807 [ZFSOnLinux] Allow devstack bootstrap in Ubuntu > 16.04
The package seems to be present and maintained
in the ubuntu repositories for all releases
following Xenial Xerus.

[1] https://packages.ubuntu.com/search?keywords=zfsutils-linux

Change-Id: If212e0482b360f3c3888918242dba6a710ab2818
Needed-By: https://review.openstack.org/#/c/604929/
2018-09-25 12:43:42 -07:00
Zuul
3d8ae6ac4b Merge "Remove logging overrides from plugin.sh" 2018-08-17 01:24:16 +00:00
Goutham Pacha Ravi
9448c3360d Remove logging overrides from plugin.sh
Avoid overwriting the log formatters, reuse
the helper available in Devstack.

Closes-Bug: #1787310
Change-Id: I25eb16ea9a6e3e98de6a7ee349df17438847d601
2018-08-16 03:09:38 +00:00
Goutham Pacha Ravi
f9ae7aab05 Test share type per test suite changes
Make default share type and group type
creation optional in
manila-tempest-minimal-dsvm-dummy job.

Depends-On: https://review.openstack.org/#/c/540070/
Change-Id: I585c15d9eb4eefb843df62f7aad08838c83611e2
2018-08-06 17:05:04 +00:00
Tom Barron
093072536e [Grenade] Switch base to stable/queens
We should test two neighbour branches in grenade job.
So, after branching of stable/queens we should use it as
base for testing of manila migration to current master
branch (Rocky-dev).

Change-Id: I20a78d69b4adcedd2e1691c7ef672567826a6b79
2018-05-24 09:04:19 -04:00
Rishabh Dave
72df3cd79a Circumvent bug #1747721 to prevent CI failures
Change-Id: I0fb057657fdf568b591bc6a43cfd6eb355010523
Related-Bug: #1747721
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-03-28 11:15:05 +05:30
Rishabh Dave
72f79e5c57 Fetch and install manila-tempest-plugin system-wide
And don't enlist manila-tempest-plugin for installation in venv-tempest.

Change-Id: I0844cdb48b9d415bd341c4687620412178be871b
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-03-08 17:35:30 +05:30
Rodrigo Barbieri
dc93ae534c Update manila plugin to support IPv6
This patch makes necessary changes to devstack manila
plugin in order to be able to run IPv6 scenario tests.

Part of the changes included the dependency of
neutron-dynamic-routing plugin and the installation
of quagga, so routes in devstack host can be created
automatically for each router and private network
created by tempest during the tests.

Also, added a new config option "override_ip_for_nfs_access"
for manila tempest plugin that overrides the access rules used
for NFS scenario tests. This option can be set by exporting the
environment variable "OVERRIDE_IP_FOR_NFS_ACCESS" with the
intended value before running devstack.

This change is enabled by a following change and
tested alongside a change on manila-tempest-plugin
project.

Please note that we are temporarily disabling IPv6
functionality in Host-assisted Share Migration, as
the Data Service is not able to handle IPv4 + IPv6
scenarios.

Change-Id: I4ca727f92618998242af18908bbbda6bb5f86303
Needed-By: Id8b005cdd429d53a75624885fe7ca795746c3ede
2018-01-24 17:11:22 -02:00
Zuul
11bf61f5ba Merge "Remove in-tree tempest plugin" 2017-12-22 13:40:44 +00:00
Raissa Sarmento
797048c5ce Remove in-tree tempest plugin
Remove in-tree tempest plugin in favor of newly
created manila-tempest-plugin repository.

Change-Id: I1fb124598f38067fee469df124ee684f748c6f57
2017-12-22 11:11:38 +00:00
zhongjun
64eaeae6bd [policy in code] Add support for AZ, scheduler and message resource [9/10]
This patch adds policy in code support for availability_zone,
scheduler_stats and message resources.

Change-Id: I9a79b5ececc583e85149cc920321e461e832b245
Partial-Implements: blueprint policy-in-code
2017-12-20 06:22:02 +00:00
junboli
411855cda7 [Grenade] Switch base to stable/pike
We should test two neighbour branches in grenade job.
So, after branching of stable/pike we should use it as
base for testing of manila migration to current master
branch (Queens-dev).

Change-Id: I0794a7e699622051a7b094b81dcfc843e7b43880
2017-09-07 00:36:01 +00:00
Ben Swartzlander
3755978ef7 Allow 2 or more export IPs for LVM driver
Supporting dual IPv4/IPv6 backends will eventually require
multi-IP support in the first party backends. This change
enables multi-IP support (IPv4-only currently) in the LVM
driver.

Change-Id: Ib3594aa5d7751c829820fce830d87f6ceea6b049
Partial-Implements: blueprint support-ipv6-access
2017-07-18 16:05:27 -04:00
Yuval Brik
65f4a1c564 devstack: clone Manila client only if marked to
Currently, Manila's devstack plugin clone manilaclient. Some projects
depend on Manila but would like to install packaged manilaclient.
Clone the manilaclient only if it was marked to be used from git

Change-Id: Ia8e726b9358f6901c172a31f638a59eba523e0fa
Depends-On: I94d2f143159b60c4f8693e35fe844bc1e411e580
2017-05-03 11:21:01 +03:00
Valeriy Ponomaryov
16bfc82962 Add possibility to run 'manila-api' with wsgi web servers
One of the goals for Pike [1] is to make each API service be able to
run under web servers that support WSGI applications,
such as Apache (+mod-wsgi) and Nginx (+uWSGI).

Do following to address governance requirements:
- Split existing manila/wsgi.py module into 3 modules:
  First (manila/wsgi/eventlet_server.py) is used by
  eventlet-based WSGI application approach.
  Second (manila/wsgi/wsgi.py) is used for WSGI web servers.
  And third (manila/wsgi/common.py) is common code for both.
  All three are made in cinder-like way to have alike-approach.
- Reuse common code from "oslo_service/wsgi.py" module that
  allows us to remove code duplication.
- Delete config opts that are defined by newly reused common code.
- Register new entry point that will be manila wsgi app: "manila-wsgi".
- Fix "manila/api/openstack/wsgi.py" module to be compatible
  with str/bytes handling approach used by Apache mod-wsgi plugin using
  different python versions (2/3).
- Add web server config template "devstack/apache-manila.template"
- Add devstack support where usage of this feature can be
  enabled or disabled using "MANILA_USE_MOD_WSGI" env var.
  It is set to "True" by default, because it is requirement for Pike
  release - to have it running in all CI jobs.
  Disable it only for one CI job that uses dummy driver and tests
  various manila core features that are not covered by other CI jobs.

[1] https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html

Partially-Implements BluePrint wsgi-web-servers-support
DocImpact
Change-Id: Ibdef3c6810b65a5d6f3611e2d0079c635ee523ab
2017-04-10 12:49:20 +03:00
Jenkins
fddb11eb74 Merge "devstack: skip nfs kernel install if nfs-ganesha" 2017-04-08 04:07:20 +00:00
Hongbin Lu
c0c3aa035c Move create_manila_accounts to post-config
In before, Manila devstack plugin create the service endpoint in
"extra" stage. This caused issue of other services that depends
on the manila endpoints that are not yet created. This commit
move the endpoint creation to "post-config" stage, which is the
practice of other OpenStack services

Change-Id: I3adfbf78d0792471a29c7ff979aad1418e6137d4
2017-03-24 21:28:07 -05:00
Tom Barron
ab778b3124 devstack: skip nfs kernel install if nfs-ganesha
is running under systemd since the services are incompatible.
Skip the samba install and startup as well since there is
no current use case for running samba plus nfs-ganesha.

TrivialFix

Change-Id: I6d6fe3a3babe2f7fcd9ca9461785ac413ff4ab2c
2017-03-15 14:40:39 -04:00