5488 Commits

Author SHA1 Message Date
Zuul
8c85cf31a5 Merge "[devstack] Setup a "shared-file-system" service" 2021-02-08 23:42:02 +00:00
Zuul
e450aca33b Merge "Enable healthcheck middleware" 2021-02-08 23:41:57 +00:00
Zuul
9c1fb887f9 Merge "[devstack] create endpoint without project_id" 2021-02-06 01:06:03 +00:00
Zuul
be262aaa99 Merge "Advertise v2 API routes without project_id" 2021-02-05 22:26:55 +00:00
Goutham Pacha Ravi
27f01aaa76 [devstack] Setup a "shared-file-system" service
A few releases ago, service types were standardized
across OpenStack [1] and manila's service type was
recommended as "shared-file-system" [2]. Tools were
meant to look at the service-types-authority [3]
and os-service-types repository [4] to find a list
of standard service type names for various first
party OpenStack services.

The openstacksdk currently relies on the service
type to be "shared-file-system" [5], so let's set
one up in devstack, to aid contributors working on
openstacksdk.

It is desirable to have all clients to initially
look for the standard service type name in the
cloud's service catalog, and fall back to the
legacy name ("sharev2") in environments that
call the v2 API that.

[1] https://specs.openstack.org/openstack/service-types-authority/
[2] https://service-types.openstack.org/service-types.json
[3] https://opendev.org/openstack/service-types-authority
[4] https://opendev.org/openstack/os-service-types
[5] f4dd6fe5fd/openstack/_services_mixin.py (L71-L72)

Change-Id: I6a1ada102a40f3e83fe8e5287856d01dd137ea95
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2021-02-04 23:44:11 -08:00
Goutham Pacha Ravi
9efcb475cf [devstack] create endpoint without project_id
As of API version 2.60, a project_id is no
longer needed in the API URLs. We can stop
devstack from setting up an endpoint with
project_id in it.

Create a "sharev2_legacy" endpoint that
contains the project_id for testing the
compatibility with the older style of URLs.

Change-Id: I25aeb1b6dd1a4150c6e542e29b7d43e18d9ad94c
Implements: bp remove-project-id-from-urls
Depends-On: I5127e150e8a71e621890f30dba6720b3932cf583
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2021-02-05 07:25:31 +00:00
Goutham Pacha Ravi
263d5438f0 Advertise v2 API routes without project_id
Manila APIs have had the requirement to include
project_id in the URLs since the very beginning.
This comes from an old assumption that our APIs
would be differentiated per-tenant on the cloud,
and we would allow different kinds of API endpoints
(public, admin, internal, etc). While it is possible
to set up different endpoints against the API
service, the same and complete API is exposed at
each of these endpoints.

We don't _need_ the project_id information that
we receive in the URL for any of our APIs to
function. We rather authorize tenants by gathering
information from the Identity service (Keystone)
and wrapping that into a RequestContext object
that we then rely on to ensure namespace isolation.

Removing the requirement for "project_id" simplifies
our API endpoint structure in the service catalog
as well as provides a way for system scoped users
to interact with manila without having to declare
their project.

In order to make project_id optional in urls, the
possible values of project_id have to be constrained.
This change introduces a new configuration option
so deployers may control that. This configuration
option defaults to accepting UUIDs with and without
dashes.

Since manila can be used in standalone deployments
without the need for Keystone, this change introduces
a noauth middleware that can work without project_id
in the URL paths.

The API version has been incremented to signal this
change to end users. When 2.60 is available, deployments
may drop "project_id" in the service catalog endpoint
for Manila and end users applications can stop needing
it as well (if they don't already rely on the service
catalog for this data).

APIImpact
Implements: bp remove-project-id-from-urls
Change-Id: I5127e150e8a71e621890f30dba6720b3932cf583
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2021-02-04 23:20:19 -08:00
Zuul
2c46aec727 Merge "remove usage of six library from unit tests" 2021-02-03 23:18:40 +00:00
Takashi Kajinami
1c17ca4382 Enable healthcheck middleware
This change makes healtcheck middleware enabled by default, so that
operators can use the middleware to monitor service availability for
various purpose like healthcheck by load balancers.

Change-Id: I47e65d23ebfe310643896e1de3396cd627a3def3
2021-02-03 14:51:38 -08:00
Zuul
f855abd374 Merge "add additional driver data to migration get progress view" 2021-02-01 21:17:52 +00:00
Zuul
ac108a0a7b Merge "[Native CephFS] Add messages for async ACL ops" 2021-02-01 20:50:46 +00:00
Zuul
fdfed2830d Merge "remove usage of six library" 2021-01-30 06:47:53 +00:00
Zuul
db3d47116a Merge "[NetApp] Fix kerberos security service issues" 2021-01-30 00:04:26 +00:00
Goutham Pacha Ravi
da3ab2cf45 [Native CephFS] Add messages for async ACL ops
Access rules added to CephFS shares can fail
at the driver, or by the ceph volume client library.
Since the share manager can supply rule changes to
the driver in batches, the driver has to gracefully
handle individual rule failures.

Further some of the causes of the access rule
failures can be remedied by end users, therefore
asynchronous user messages would be a good vehicle
to register user faults that can be examined and
corrected.

Related-Bug: #1904015
[1] https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-27781

Change-Id: I3882fe5b1ad4a6cc71c13ea70fd6aea10430c42e
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2021-01-27 19:46:51 -08:00
haixin
3831e5500f remove usage of six library from unit tests
Replace six with Python 3 style code.

Change-Id: I5077e71663f6b60bd774f30fcf64b36d4078cf8e
2021-01-28 10:47:40 +08:00
haixin
6fb2e8510e remove usage of six library
remove usage of six library from the following directory:
1:common
2:data
3:db
4:message
5:network
6:scheduler

Change-Id: I9db0abf2b0847157074ca6ba84b5451bfe3f20d0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2021-01-27 13:46:31 -08:00
liviacavalcanti
be794fb168 Remove backend deprecated variables
MANILA_MULTI_BACKEND has been deprecated for five years now, we should remove
it from our code base.

This variable was removed from the settings scripts along with:
MANILA_BACKEND1_CONFIG_GROUP_NAME and MANILA_SHARE_BACKEND1_NAME;
MANILA_BACKEND2_CONFIG_GROUP_NAME and MANILA_SHARE_BACKEND2_NAME.
Because they work in the same context.

Instead of them, the already implemented and in use,
MANILA_ENABLED_BACKENDS variable was placed to garantee the successful
back-end setup. The same replacement was made in the contribution
samples scripts.

Apart from this, we avoid configuring generic1 and generic2 if
another backend/s are selected.

Closes-Bug: #1898791
Closes-Bug: #1878477

Change-Id: I67036a65da9255694a00a9c8d56cfdefbdf23c05
2021-01-27 10:14:42 +00:00
Maurice Escher
aa62a8090e
add additional driver data to migration get progress view
new field `details` is introduced with microversion 2.59

Change-Id: I91d455c585ff546cc65e57e8560f126a280eac8d
Closes-Bug: #1902854
2021-01-25 09:50:35 +01:00
Zuul
42b709e446 Merge "Adds missing caps for mgr in cephfs configuration" 2021-01-22 21:15:14 +00:00
Jose Castro Leon
ba3275e9ba Adds missing caps for mgr in cephfs configuration
Adds a missing cap to access mgr and be able to list subvolumegroups
and subvolumes in a cephfs volume

Change-Id: Ie9849ed875b996cf33aeb296f258ec84c8b720ef
Closes-Bug: #1907467
2021-01-22 20:07:13 +00:00
Felipe Rodrigues
68ad590c37 [NetApp] Fix kerberos security service issues
- Fixes the zapi calls for setting up a kerberos, which have
changed since ONTAP 8.3.
- Fixes kerberos configuration cleanup when deleting a
share server.
- Fixes access rules authentication methods for NFS when a
share server is configured for Kerberos.

Change-Id: I60b4f92979045b1fdb90ad8df4f65c1dfe463ae8
Closes-Bug: #1901189
Closes-Bug: #1904746
Closes-Bug: #1907669
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Signed-off-by: Douglas Viroel <viroel@gmail.com>
2021-01-22 09:29:29 -02:00
Tom Barron
dcdc1a98bb Update doc for manila-service-image download
It currently refers to the old openstack repository and as written
downloads redirect information rather than the image itself.

Add a '-L' argument to the ``curl`` command to handle redirects and
update the URL to use the opendev repository.

Related-bug: #1908838
Change-Id: I48509e5c9be41c04d00aa86efd4d657b067d9521
2021-01-21 20:00:59 +00:00
Zuul
4495069118 Merge "Fix missing group and group_snapshots in absolute limits" 2021-01-21 01:09:26 +00:00
Zuul
084ed5cb46 Merge "Add developer reference to share server migration" 2021-01-21 00:00:12 +00:00
Zuul
78c6460e89 Merge "[Doc] Add admin doc for share group" 2021-01-20 19:12:30 +00:00
Zuul
8159232534 Merge "Update manila and Ceph matrix support doc" 2021-01-18 15:45:02 +00:00
Zuul
ebef6408c3 Merge "[goal] Deprecate the JSON formatted policy file" 2021-01-15 23:43:15 +00:00
Zuul
a6b8d5b170 Merge "Move shares filtering to database layer" 2021-01-14 17:39:25 +00:00
Victoria Martinez de la Cruz
d7407afd55 Update manila and Ceph matrix support doc
Include Wallaby (master) support matrix and
some extra considerations to consider on manila
with cephfs backends.

Change-Id: Iad7cb229151b8dd707a59553cb6e966090d0eeae
2021-01-13 20:41:58 +00:00
Cloud User
6bb021375c Fix missing group and group_snapshots in absolute limits
Modified limits API to make it return the max number of
share_groups and share_group_snapshots, as well as the total number
of resources used

Change-Id: Ia4e69219b107fc0630cb9e97401b9a8bda5b1adc
Closes-Bug: #1868644
2021-01-13 14:52:41 +00:00
Lance Bragstad
7f72147f17 Implement secure RBAC for storage pool statistics
This commit updates the policies for storage pool statistics to understand
scope checking and account for a read-only role. This is part of a broader
series of changes across OpenStack to provide a consistent RBAC experience and
improve security.

Change-Id: I99af5f4f9bfc7e0718c618c95206b7051b724c73
2021-01-11 23:21:05 -08:00
Lance Bragstad
46f65eafa9 Implement secure RBAC for quotas
This commit updates the policies for quotas to understand scope checking and
account for a read-only role. This is part of a broader series of changes
across OpenStack to provide a consistent RBAC experience and improve security.

Change-Id: Id3f3192fb9948f71b0b9cb03d2ef8ffd0b770620
2021-01-11 23:20:36 -08:00
Lance Bragstad
5011b6719c Implement secure RBAC for quota classes
This commit updates the policies for quota classes to understand scope checking
and account for a read-only role. This is part of a broader series of changes
across OpenStack to provide a consistent RBAC experience and improve security.

Change-Id: I5c8ebcf72ab158021deab772693b869c35962a2b
2021-01-11 23:20:00 -08:00
Lance Bragstad
9ef2156c04 Implement secure RBAC for messages
This commit updates the policies for messages to understand scope checking and
account for a read-only role. This is part of a broader series of changes
across OpenStack to provide a consistent RBAC experience and improve security.

Change-Id: I30ae2a1d34fb1dcb438880b1a5b46afea7db8d0d
2021-01-11 23:19:10 -08:00
Lance Bragstad
a43911d7de Implement secure RBAC for storage availability zones
This commit updates the policies for availability zones to understand scope
checking and account for a read-only role. This is part of a broader series of
changes across OpenStack to provide a consistent RBAC experience and improve
security.

Change-Id: I1bbb6dc900ef413189c20d41fe08d9263a0038c2
2021-01-11 23:18:16 -08:00
Zuul
11d3da58ea Merge "Always use new_size when extend" 2021-01-12 06:51:28 +00:00
shews
fbc4ecb640 Always use new_size when extend
Always use new_size when extend.
Closes-Bug: #1909951

Change-Id: Ic5faa887448f363bb4d91bccfe8e035a06936454
2021-01-11 06:23:16 +00:00
Victoria Martinez de la Cruz
d7d83a3f3b Uses local variable with ipv4 gateway config
We were using an env var with the IPv4 gateway config
that is not always present. This was causing devstack to fail
in developer environment. Use the local variable instead.

Closes-Bug: #1910760
Change-Id: Iede8a9e59b96d0f21c117ab1464a0a9e3477c24b
2021-01-08 14:34:57 +00:00
zhongjun
3fa5cf4ed2 [Doc] Add admin doc for share group
Partial-Bug: #1659025
Change-Id: Ia0cc534e0bfb2ca5e495e575237e9911c746691b
2021-01-06 09:20:26 -03:00
silvacarloss
256c18d6c2 Add api reference for share server migration
Add documentation to the share server migration APIs introduced
during Victoria release.

Partial-bug: #1897903

Change-Id: I13d13c38a3869929bbfdf8083529a597d7982a16
2021-01-06 09:04:23 -03:00
silvacarloss
281b7dd1cd Add developer reference to share server migration
Updates the developer reference with informations regarding the
share server migration feature implemented during Victoria release.

Change-Id: Ia72cf037d2b7dc9fb9d4f19ce141cc044206d6fc
Partial-bug: #1897903
2021-01-06 08:47:25 -03:00
Zuul
249904842c Merge "Delete unavailable py2 package" 2020-12-29 01:03:36 +00:00
linpeiwen
bab899068c Delete unavailable py2 package
The openstack Ussuri and Victoria versions no longer support the
Centos7 and pyrhon2 environment packages. Correct the missing
problems in the latest document

Change-Id: If139927730071448abc04e1ea7ebb615749e7e3d
2020-12-16 01:11:41 +00:00
Zuul
99d21fa8e5 Merge "Switch from base64 encodestring to encodebytes" 2020-12-15 03:25:36 +00:00
Zuul
f55482c6b2 Merge "[glusterfs] don't reinit volume list on deletion" 2020-12-15 00:51:29 +00:00
Ghanshyam Mann
0cc7cbc36d [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also convert manila/tests/policy.json to manila/tests/policy.yaml
using oslopolicy-convert-json-to-yaml tool and replace
policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I3748313912b2527c43c9b16a6ba3e3ccd4cf5221
2020-12-13 00:05:00 +00:00
Zuul
ae7a782a1f Merge "Stop manila when unstacking" 2020-12-12 04:29:49 +00:00
Tom Barron
39734a70c5 Stop manila when unstacking
When you run unstack.sh from devstack, other devstack
services are stopped and disabled to provide a clean environment
for a restack, but manila services are left running.

This doesn't matter for CI where a new VM is stood up for each
devstack but it's inconvenient for local devstack and if you
restack without restarting the services manually the results you
see may not actually match the environment you intended.

Change-Id: I6761619042e4bc36ec2f1cab4be33cb1b39d00d7
2020-12-11 09:52:30 -05:00
Goutham Pacha Ravi
c0ee884861 Update requirements
pip 20.3 brings in a strict dependency resolver which
is enabled by default. This causes our lower-constraints
tests to fail, because the requirement files were out
of date from reality - they had conflicting requirements
which previous versions of pip were ignoring. Let's catch
up package versions to newer ones that are supported in
the python runtimes that the Wallaby release will be
deployed to.

[1] http://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html
[2] https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020

Change-Id: I5a31b561654aa368bb85a56f4dd38276cfdbb91a
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2020-12-10 19:48:15 -08:00
Zuul
10b3dc744f Merge "Updates to support in Ceph local.conf sample" 2020-12-10 21:30:07 +00:00