5455 Commits

Author SHA1 Message Date
Lance Bragstad
9ef2156c04 Implement secure RBAC for messages
This commit updates the policies for messages to understand scope checking and
account for a read-only role. This is part of a broader series of changes
across OpenStack to provide a consistent RBAC experience and improve security.

Change-Id: I30ae2a1d34fb1dcb438880b1a5b46afea7db8d0d
2021-01-11 23:19:10 -08:00
Lance Bragstad
a43911d7de Implement secure RBAC for storage availability zones
This commit updates the policies for availability zones to understand scope
checking and account for a read-only role. This is part of a broader series of
changes across OpenStack to provide a consistent RBAC experience and improve
security.

Change-Id: I1bbb6dc900ef413189c20d41fe08d9263a0038c2
2021-01-11 23:18:16 -08:00
Zuul
11d3da58ea Merge "Always use new_size when extend" 2021-01-12 06:51:28 +00:00
shews
fbc4ecb640 Always use new_size when extend
Always use new_size when extend.
Closes-Bug: #1909951

Change-Id: Ic5faa887448f363bb4d91bccfe8e035a06936454
2021-01-11 06:23:16 +00:00
Victoria Martinez de la Cruz
d7d83a3f3b Uses local variable with ipv4 gateway config
We were using an env var with the IPv4 gateway config
that is not always present. This was causing devstack to fail
in developer environment. Use the local variable instead.

Closes-Bug: #1910760
Change-Id: Iede8a9e59b96d0f21c117ab1464a0a9e3477c24b
2021-01-08 14:34:57 +00:00
zhongjun
3fa5cf4ed2 [Doc] Add admin doc for share group
Partial-Bug: #1659025
Change-Id: Ia0cc534e0bfb2ca5e495e575237e9911c746691b
2021-01-06 09:20:26 -03:00
silvacarloss
256c18d6c2 Add api reference for share server migration
Add documentation to the share server migration APIs introduced
during Victoria release.

Partial-bug: #1897903

Change-Id: I13d13c38a3869929bbfdf8083529a597d7982a16
2021-01-06 09:04:23 -03:00
silvacarloss
281b7dd1cd Add developer reference to share server migration
Updates the developer reference with informations regarding the
share server migration feature implemented during Victoria release.

Change-Id: Ia72cf037d2b7dc9fb9d4f19ce141cc044206d6fc
Partial-bug: #1897903
2021-01-06 08:47:25 -03:00
Zuul
249904842c Merge "Delete unavailable py2 package" 2020-12-29 01:03:36 +00:00
linpeiwen
bab899068c Delete unavailable py2 package
The openstack Ussuri and Victoria versions no longer support the
Centos7 and pyrhon2 environment packages. Correct the missing
problems in the latest document

Change-Id: If139927730071448abc04e1ea7ebb615749e7e3d
2020-12-16 01:11:41 +00:00
Zuul
99d21fa8e5 Merge "Switch from base64 encodestring to encodebytes" 2020-12-15 03:25:36 +00:00
Zuul
f55482c6b2 Merge "[glusterfs] don't reinit volume list on deletion" 2020-12-15 00:51:29 +00:00
Ghanshyam Mann
0cc7cbc36d [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also convert manila/tests/policy.json to manila/tests/policy.yaml
using oslopolicy-convert-json-to-yaml tool and replace
policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I3748313912b2527c43c9b16a6ba3e3ccd4cf5221
2020-12-13 00:05:00 +00:00
Zuul
ae7a782a1f Merge "Stop manila when unstacking" 2020-12-12 04:29:49 +00:00
Tom Barron
39734a70c5 Stop manila when unstacking
When you run unstack.sh from devstack, other devstack
services are stopped and disabled to provide a clean environment
for a restack, but manila services are left running.

This doesn't matter for CI where a new VM is stood up for each
devstack but it's inconvenient for local devstack and if you
restack without restarting the services manually the results you
see may not actually match the environment you intended.

Change-Id: I6761619042e4bc36ec2f1cab4be33cb1b39d00d7
2020-12-11 09:52:30 -05:00
Goutham Pacha Ravi
c0ee884861 Update requirements
pip 20.3 brings in a strict dependency resolver which
is enabled by default. This causes our lower-constraints
tests to fail, because the requirement files were out
of date from reality - they had conflicting requirements
which previous versions of pip were ignoring. Let's catch
up package versions to newer ones that are supported in
the python runtimes that the Wallaby release will be
deployed to.

[1] http://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html
[2] https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020

Change-Id: I5a31b561654aa368bb85a56f4dd38276cfdbb91a
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-12-10 19:48:15 -08:00
Zuul
10b3dc744f Merge "Updates to support in Ceph local.conf sample" 2020-12-10 21:30:07 +00:00
Tom Barron
8d1dcb1e0c Updates to support in Ceph local.conf sample
to make user devstack deployment a bit easier.

Since snapshot_support is now on by default for Ceph
backends, enable it in the default share type extra
specs.

Also specify the SHARE_DRIVER so the manila devstack
plugin doesn't think we're running the generic driver
and try to install kernel NFS and samba servers.

Change-Id: I636a047cacba7c5960df15a99fc79a35a818f45d
2020-12-10 15:44:08 -05:00
Zuul
473e1cd43b Merge "Docs: NFS-Ganesha and dbus" 2020-12-10 07:10:48 +00:00
Corey Bryant
8bef5e25b7 Switch from base64 encodestring to encodebytes
encodestring is deprecated since python 3.1 and
removed in python 3.9.

https: //docs.python.org/3.8/library/base64.html
Closes-Bug: #1907494
Change-Id: I56d7622207165b6d875559597c62c31a637b3f26
2020-12-09 15:11:38 -05:00
Goutham Pacha Ravi
914d873774 [glusterfs] don't reinit volume list on deletion
We don't need to re-initialize the volumes list
on deletion, it still makes sense to add a missing
volume to the list, going by the reasoning defined
in I14835f6c54376737b41cbf78c94908ea1befde15

Related-Bug: #1894362
Change-Id: I96d49f84122a34701328909c929ede4d66746911
2020-11-30 19:47:43 -08:00
Zuul
d8dab2d077 Merge "Fix logic that determines a share exists before manage" 2020-11-24 05:30:20 +00:00
Lance Bragstad
2a8aaf43ad Introduce base personas for secure RBAC
This commit adds some really basic check strings that we can re-use in
the existing policies. These checks strings implement a few of the most
common and useful personas. We're also trying to implement them across
OpenStack consistently, which will improve user experience and security.

Change-Id: Ib46402414e8ed9b63f024313500aef85f0c47a41
2020-11-19 21:41:35 +00:00
Lance Bragstad
a10927c6be Bump oslo.log version to 4.3.0
This allows us to use the WALLABY marker in oslo.log's versionutils for
deprecated notices.

Change-Id: I2e6c482989095ea8e102e5afe2e2f57214fe6f06
2020-11-18 22:17:11 +00:00
Zuul
d7a647e164 Merge "[NetApp] Fix hard-coded CA cert path for SSL" 2020-11-18 01:42:08 +00:00
Tom Barron
c8087fa0a4 Docs: NFS-Ganesha and dbus
Added paragraph to the NFS-Ganesha docs with a brief
explanation of the role of ``dbus`` messaging when doing
dynamic updates of exports.  Key point is that ``dbus-send``
and NFS-Ganesha must be in the same namespace even if NFS-ganesha
runs in a container.

Close-bug: #1883961
Change-Id: I57b916b16d07a8373143f396b42ea34bf80330a6
2020-11-17 20:15:44 -05:00
Zuul
d36b5df6a2 Merge "[NetApp] Support NFS shares on windows" 2020-11-17 13:37:26 +00:00
Zuul
fb86233489 Merge "Retry unmount operation on the LVM driver" 2020-11-17 12:22:33 +00:00
Zuul
7b15796aa5 Merge "Retry unmount operation on the ZFSOnLinux driver" 2020-11-17 01:56:34 +00:00
Chuan Miao
5e01d9e292
[NetApp] Support NFS shares on windows
Set following 4 attributes when creating nfs shares:
    showmount:          enabled
    v3-ms-dos-client:   support
    v3-connection-drop: disable
    ejukebox-errors:    disable

Change-Id: I86c3de32d36940e85545fcb337723e4b3edce216
Closes-bug: 1901937
Co-authored-by: Maurice Escher <maurice.escher@sap.com>
2020-11-16 21:04:09 +01:00
Felipe Rodrigues
70f7859f98 [NetApp] Fix hard-coded CA cert path for SSL
NetApp driver is hard-coding the location of CA certificates for SSL
verification during HTTPS requests. This location may change depending
on the environment or/and backend.

This patch adds the `netapp_ssl_cert_path` configuration, enabling
each backend to choose the directory with certificates of trusted CA
or the CA bundle. If set to a directory, it must have been processed
using the c_rehash utility supplied with OpenSSL. If not informed,
it will use the Mozilla's carefully curated collection of Root
Certificates for validating the trustworthiness of SSL certificates.

Closes-Bug: #1900191
Change-Id: Idbed4745104de26af99bb16e07c6890637dfcfd1
2020-11-16 16:04:26 -03:00
Goutham Pacha Ravi
8a691d8631 Retry unmount operation on the LVM driver
When a share is mounted on the same host as the manila-share
process, the kernel prevents us from destroying the
mount directory until the share has been cleanly unmounted
from the host. Kernel mounts can take a few seconds to get
unmounted fully especially when there are a lot of
linux namespaces that the mountpoint has been shared to.

Add a retry on these operations to harden the deletion
process and prevent spurious failures.

Change-Id: I3c1a2ec19d6bc18638db0875519ce60f2c89f33a
Closes-Bug: #1903773
Related-Bug: #1896672
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-11-11 23:27:16 -08:00
Zuul
2d29368bd0 Merge "Replace deprecated UPPER_CONSTRAINTS_FILE variable" 2020-11-12 02:33:56 +00:00
wu.shiming
5e8b2a475d Replace deprecated UPPER_CONSTRAINTS_FILE variable
Change-Id: I1237998bfaf258cf645d34ad8b8609aa5f03381c
2020-11-12 09:35:10 +08:00
Goutham Pacha Ravi
e3fea14788 Retry unmount operation on the ZFSOnLinux driver
When a share is mounted on the same host as the manila-share
process, zfs prevents us from destroying the underlying
dataset until the share has been cleanly unmounted from
the host. Kernel mounts can take a few seconds to get
unmounted fully especially when there are a lot of
linux namespaces that the mountpoint has been shared to.

Add a retry on these operations to harden the deletion
process and prevent spurious failures.

Change-Id: I4aba76b72df274d0a8cb90fe0ab8799523c260ef
Closes-Bug: #1903773
Related-Bug: #1896672
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-11-10 13:35:21 -08:00
silvacarloss
136a89937c Update share replicas export location API ref
Moves the share replica export location API reference out of the
experimental features documentation.

Change-Id: I16cbad169d0c4f89428f17a317840750966d240f
2020-11-06 09:05:36 -03:00
Zuul
abfbfd5781 Merge "Remove the unused coding style modules" 2020-11-04 00:17:30 +00:00
Maurice Escher
074b083597
fix reno file location and indention
Change-Id: Ibdb7f5fb04c9f53cd8c156e87a1bd95376d80839
2020-11-03 09:17:28 +01:00
wu.shiming
55ef4999f9 Remove the unused coding style modules
Python modules related to coding style checks (listed in blacklist.txt
in openstack/requirements repo) are dropped from lower-constraints.txt
as they are not actually used in tests

Change-Id: Ia64180303af54c512610add2f55987efea97017a
2020-10-30 11:43:07 +08:00
Zuul
e5c165dc4f Merge "disable M325" 2020-10-29 20:36:18 +00:00
silvacarloss
21c731e733 Move shares filtering to database layer
Moves the manila shares filtering to the database in order to have
the queries performance improved.

Change-Id: I031a3b9775c50e78b6b86752ff8d1a4871a91c0c
Co-Authored-By: MaAoyu <maaoyu@inspur.com>
2020-10-23 20:37:58 +00:00
haixin
1b5771ef15 Fix logic that determines a share exists before manage
The share "manage" API checks whether
an existing/known share is being imported by
matching the export path provided to existing
shares.

This lookup does not consider the
fact that shares may have multiple export
locations, because it relies on an old/deprecated
"export_location" property on shares which
was added to provide backwards compatibility
to the API that presented only one export
location per share.

Further, it's possible to get a
"ERROR: Invalid share: Share already exists"
exception even when no such share exists in the
database.

Fix the lookup by using the "export_location_path"
based lookup which is faster, since it performs
a meaningful join on the export locations table;
and remove the parameters "protocol"
and "share_type_id" - these things make no
difference when there's a duplicated export
location. We'll consider "host" as a lookup
parameter since we can't be sure that export
locations are unique in a deployment - but they
ought to be unique for a given host.

Closes-Bug: #1848608
Closes-Bug: #1893718
Change-Id: I1d1aef0c2b48764789b43b91b258def6464b389f
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-10-19 15:25:07 +08:00
Zuul
179b26f5dc Merge "Add share server migration admin documentation" 2020-10-14 12:04:56 +00:00
haixin
cc33a4b23e disable M325
Since we don't support python2.x anymore, we don't
need to use six.text_type.

Change-Id: Id1647b55af476ab4559759a27498ede353dd10ba
2020-10-14 14:05:30 +08:00
Zuul
05a6c74800 Merge "Set Victoria max version in the api doc" 2020-10-13 19:27:23 +00:00
Zuul
e6de3ff813 Merge "[NetApp] Fix access rules for CIFS promoted replica" 2020-10-13 19:27:21 +00:00
Goutham Pacha Ravi
6d8c99a7fb Set Victoria max version in the api doc
Change-Id: I02c1062e543986a6d6eaa3187dbd90797899021e
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-10-09 11:41:58 -07:00
Douglas Viroel
2e27c71877 Fix 'update_share_replica' doesn't provide share server model to drivers
This patch fixes the 'update_share_replica' operation in the share manager
that wasn't getting the share-server model from the share replica model.
When the operation is executed, the current share replica object may
not have the 'share-server-id' information and need to be updated before
anything else.

Closes-Bug: #1898924
Change-Id: I1d9d69bbdaa27a68a425d959fa8c5da83a157548
Signed-off-by: Douglas Viroel <viroel@gmail.com>
2020-10-08 20:31:55 -03:00
Douglas Viroel
3ca9c7678e Add share server migration admin documentation
Partial-Bug: #1897903
Change-Id: Ia6dd260e6d2f9e2a7d9d381d345d952a3069f8c5
Signed-off-by: Douglas Viroel <viroel@gmail.com>
2020-10-08 09:43:09 -03:00
Felipe Rodrigues
0c4d1e3398 [NetApp] Fix access rules for CIFS promoted replica
This patch fixes the access rules for NetApp promote replica when
using CIFS protocol. When promoting a replica, the NetApp ONTAP
driver updates the access rules for the promoted CIFS share entity
before actually creating it, failing on having those rules
applied.

The bug is fixed by switching the order of updating the access
and creating the promoted CIFS share entity.

Change-Id: I60e4057dc962d96cff57dea88587a28c2043b499
Closes-Bug: #1896949
2020-10-03 12:32:43 -03:00