7166 Commits

Author SHA1 Message Date
6cabdf93fc Add Python3 xena unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for xena.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I5208077a9f700e8104dab697b8ac58fd85575f7a
2021-03-23 17:09:16 +00:00
99a5a0f1e8 Update master for stable/wallaby
Add file to the reno documentation build to show release notes for
stable/wallaby.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/wallaby.

Sem-Ver: feature
Change-Id: I1a6a99bb19181cd92ca5e328eba846c603425654
2021-03-23 17:09:12 +00:00
Zuul
922e544ca2 Merge "Tox.ini: add py39" 2021-03-18 17:37:51 +00:00
Zuul
2f6e3bc578 Merge "Make some metadef operations admin-only" 2021-03-17 18:08:04 +00:00
Dan Smith
cf94c9aab2 Fix a typo in contributor docs
This just makes a trivial typo fix in the minor-code-changes doc.

Change-Id: If0093316c393b09ed4d936d2625b2d27024bfdbc
Co-Authored-By: Abhishek Kekane <akekane@redhat.com>
2021-03-15 11:26:44 -07:00
Abhishek Kekane
f8551de8c9 Make some metadef operations admin-only
This restricts all metadef resource manipulation to admin-only, but
still allow users to see everything. There are multiple low-grade
security issues with the metadef API, detailed in the related bug.
Restricting resource manipulation to admin-only solves most of these
concerns.

SecurityImpact
Depends-On: https://review.opendev.org/c/openstack/tempest/+/780108
Change-Id: I333c58e73c202c1f523030e54e03f2868459b595
Related-Bug: #1916926
2021-03-15 07:59:05 -07:00
Zuul
50cd037bcd Merge "Refresh Glance example configs for Wallaby milestone 3" 2021-03-12 00:36:28 +00:00
Zuul
4dae619b81 Merge "Wallaby milestone 3 release notes" 2021-03-10 20:13:47 +00:00
Zuul
66f5c8d6e1 Merge "trivial: remove unnecessary grouping in base policies" 2021-03-10 18:20:29 +00:00
Zuul
751e5ed812 Merge "Enable second glance worker for import testing" 2021-03-10 02:27:07 +00:00
Zuul
32ce011bac Merge "trivial: Fix minor grammatical issues in cache middleware" 2021-03-10 01:34:52 +00:00
Zuul
9afbf466ab Merge "Add a release note for secure RBAC personas" 2021-03-09 21:50:50 +00:00
Zuul
3e7bf199ca Merge "Update the task policies" 2021-03-09 20:26:37 +00:00
Zuul
1010805bfe Merge "Implement project personas for image actions" 2021-03-09 20:25:38 +00:00
Lance Bragstad
aec2de7ffd Add a release note for secure RBAC personas
Provide some literature on what we introduced for operators in wallaby,
how they can configure it, and actions we recommend they take. Since
this marks the point at which we consider the feature implemented,
this also removes the legacy-rbac job and makes the secure-rbac job
voting.

Implements: blueprint secure-rbac
Change-Id: I8f980cf7731d26b92b5392fdada21e5be0f541c4
2021-03-09 09:51:47 -08:00
Zuul
eecb2f057e Merge "Fix erroneous exit from copy wait loop" 2021-03-09 12:49:24 +00:00
Abhishek Kekane
9c691b5560 Wallaby milestone 3 release notes
Change-Id: I509f042648cc0e437b26cf9bf5008c43038ad047
2021-03-09 10:02:42 +00:00
Zuul
1352661ec2 Merge "Add housekeeping module and staging cleaner" 2021-03-09 01:14:08 +00:00
Dan Smith
80b84d4e97 Fix erroneous exit from copy wait loop
The wait_for_copying() helper will exit the loop if the *last*
store is found in the list, instead of *all* of them. This technically
works if the stores are processed in the same order we are checking,
but it's fragile and likely to fail in confusing ways.

This makes us only exit if all of them are present.

Change-Id: I8d9ba50f46e22b6740fdbdec6f8ef7c61dddbcf1
2021-03-08 13:30:34 -08:00
Lance Bragstad
165cce6d6e Update the task policies
At one point, these policies were used to protect actual task API
endpoints. Since then, they have also been used internally within
glance when spawning a task on behalf of the user for long-running
operations (like import).

These policies should not apply to the internal usage, as doing so
prevents the operator from setting them to restrictive values in order
to provide granular access to some roles. In the future we will fix
that by moving those checks out of "the onion" and into the task API
operations themselves, thus decoupling the internal and external uses.

This adds documentation and scope definitions for these policies, as
well as deprecates the "modify_task" policy which is never used and
will be removed in the future. Control over the actual tasks API
remains coarse with the "tasks_api_access" policy until a future
release completes the above decoupling.

Implements: blueprint secure-rbac
Change-Id: I70a58acd78053b54187dba8e35273366f14c47a4
2021-03-08 09:32:06 -08:00
Lance Bragstad
2b498e61f4 trivial: remove unnecessary grouping in base policies
We've broken basic policies into granular checks with simple names and
we use them to construct more complex checks. In that process we
accidentally added some additional nesting to two of the check strings,
which isn't necessary.

This commit updates the check strings to remove an extra set of
parenthesis.

Change-Id: Iafa37d64a9779a3b646c34f328c62dfd6cd3e7f3
2021-03-08 13:56:02 +00:00
Lance Bragstad
4063d215a1 trivial: Fix minor grammatical issues in cache middleware
Change-Id: Id3a08decd65b1222c0e9d7908ecd07587d2455c9
2021-03-08 13:45:44 +00:00
Abhishek Kekane
dbab664039 Refresh Glance example configs for Wallaby milestone 3
Change-Id: Ifc957de2bcf4d22c1b7cba31ce2b467df9d5aa85
2021-03-08 08:59:50 +00:00
Abhishek Kekane
31414b9f61 Implement project personas for image actions
This commit updates the policies for image actions to use default roles
available from keystone. Specifically, we're updating the defaults to
user project-member and project-reader personas. The project-admin
persona is still reserved for administrative APIs access for system
administrators/operators. This will remain the case until we can
refactor portions of glance to make it easier to implement system-scope.

NOTE:
  Glance is implementing Secure RBAC as EXPERIMENTAL in Wallaby, so to
  enable it operator needs to set ``glance-api.conf [oslo_policy]
  enforce_new_defaults=True`` and ``glance-api.conf
  enforce_secure_rbac=True``

Implements: blueprint secure-rbac

Change-Id: If0c456617a9e17c006a6ffe2a83f4a73b53da3d0
2021-03-08 05:51:33 +00:00
Zuul
23ed884c4b Merge "Fix test_cache_middleware ImageStub" 2021-03-07 22:14:43 +00:00
Zuul
d154cb1058 Merge "Make copy_image plugin use action wrapper" 2021-03-07 21:58:56 +00:00
Dan Smith
81fea796d9 Fix test_cache_middleware ImageStub
This fixes the ImageStub in the test_cache_middleware module, which
does not implement enough of a real image to be usable in the next
patch where we need to dict() it. This does that refactor ahead of
time so that patch is smaller.

Change-Id: Ie86e0ae16c81fb7aa353dd350f0a4f3cf852d893
2021-03-07 08:13:02 -08:00
Zuul
74313e9c76 Merge "Make inject_image_metadata use action wrapper" 2021-03-07 16:05:36 +00:00
Zuul
ec9afd69b6 Merge "Fix nonsensical test mocks and assertions" 2021-03-07 10:32:12 +00:00
Zuul
fc0ee38b8b Merge "Allow plugins to mutate image extra_properties" 2021-03-07 09:48:29 +00:00
Zuul
f8d42b388d Merge "Make image_conversion use action wrapper" 2021-03-07 03:26:05 +00:00
Zuul
78b9de9b21 Merge "Bump Images API version to 2.12" 2021-03-06 21:53:41 +00:00
Zuul
f4af2f273f Merge "Add glance functional protection tests to check and gate" 2021-03-06 19:32:49 +00:00
Zuul
c825120158 Merge "Add missing fail case tests for image_conversion" 2021-03-06 17:35:00 +00:00
Zuul
cf67d36efe Merge "Make action wrapper support arbitrary properties" 2021-03-06 17:34:06 +00:00
Zuul
111f1acffc Merge "Make web-download revert all stores on fail" 2021-03-06 17:33:58 +00:00
Zuul
1bef6a32af Merge "Pass ImageActionWrapper to internal plugins" 2021-03-06 17:33:44 +00:00
Lance Bragstad
8d694786d3 Add glance functional protection tests to check and gate
This commit updates glance's zuul configuration to tack on a job
dedicated to protecting API authroization. The tests for this job live
in glance-tempest-plugin and they currently test full support for
project-reader and project-admin against the images API.

Future changes will update the policies in glance to consume
system-scope and additional test coverage will be added to
glance-tempest-plugin. But, until that happens, having protection
testing as part of the check and gate jobs is vital to ensuring we don't
inadvertently expose sensitive information or APIs to users.

This level of testing will also be useful in the future if we decide to
refactor authorization logic out of various parts of glance and into a
consistent layer.

Depends-On: https://review.opendev.org/c/openstack/glance-tempest-plugin/+/775742
Change-Id: Iddee8144fb21b7ac2dec4e7fbc62c132c186fa89
2021-03-04 21:53:30 +00:00
Dan Smith
662607f496 Enable second glance worker for import testing
This enables the g-api-r service in devstack, which allows tempest
to run the remote import test, causing it to stage and import an
image across two different workers. Note we disable it for the
standalone mode, since devstack does not support starting another
standalone glance.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/770487
Depends-On: https://review.opendev.org/c/openstack/tempest/+/770520
Change-Id: Ica715fc1922f4b36dd0bb008ef6706b86115ec05
2021-03-04 12:13:23 -08:00
Zuul
66281f0dbf Merge "Cleanup remaining tenant terminology in glance API docs" 2021-03-04 19:43:32 +00:00
Zuul
fe8240562c Merge "Add administrator docs for distributed-import" 2021-03-04 00:46:37 +00:00
Zuul
878d7f49d8 Merge "Distributed image import" 2021-03-03 23:58:57 +00:00
Dan Smith
232177e68c Add housekeeping module and staging cleaner
As noted in previous discussions, glance should clean its staging
directory on startup. This is important for scenarios where we
started an import operation, but failed in the middle. If, when we
recover, the image has already been deleted from the database, then
we will never remove the (potentially very large) residue from disk
in our staging directory.

This is currently a problem with web-download, but will also occur
with glance-direct once we have the non-shared distributed import
functionality merged.

Closes-Bug: #1913625
Change-Id: Ib80e9cfb58680f9e8ead5993dc206f4da882dd09
2021-03-03 14:36:46 -08:00
Dan Smith
d8a6309893 Add administrator docs for distributed-import
This adds some text to the documentation about configuring the import
mechanism, including details about shared vs. local staging
directories. It also clarifies that *all* import methods require the
staging directory to be configured, as well as cleans up some
single-store-specific wording in this area.

Related to blueprint distributed-image-import

Change-Id: I726abe5d1104510e8da0e94f90f2b36d43b82cbe
2021-03-03 06:37:29 -08:00
Dan Smith
41e1cecbe6 Distributed image import
This implements distributed image import support, which addresses
the problem when one API worker has staged the image and another
receives the import request.

The general approach is that when a worker stages the image, it
records its self-reference URL in the image's extra_properties.  When
the import request comes in, any other host will proxy that HTTP
request direct to the original host instead of trying to do the import
itself.

Implements: blueprint distributed-image-import

Change-Id: I12daccb43c535b579c22f9d0742039b2ab42e929
2021-03-02 11:52:12 -08:00
Dan Smith
e9852fb625 Make functional tests set node_staging_uri
Currently it is not possible to configure the staging directory
URI of the functional workers. We need to be able to do that in order
to enable the stage cleaning behavior. Right now, they're all sharing
/tmp/staging, which will cause workers to conflict while running in
parallel. This causes them to use their private test directory,
which may also help some other spurious failures due to interaction.

Related-Bug: #1913625
Change-Id: Ic2ac2a528206c50c38e948a096daf9eb8e5eb715
2021-03-02 11:49:47 -08:00
Zuul
144cdf90be Merge "Add get_ksa_client() helper" 2021-03-02 19:41:57 +00:00
Abhishek Kekane
bf838242ac Fail to start if authorization and policy is misconfigured
This informs operators of glance's support status for secure RBAC as of
the Wallaby release. Eventually, this message will be removed when
glance adopts more support for secure RBAC personas.

This also forces glance to fail if it's configured improperly. This is
done to explicitly prevent ambiguity with authoritative decisions.

Related: blueprint secure-rbac
Change-Id: I06293de08dd3fdfbd60b9a65501d1198f40ff434
2021-03-02 14:52:21 +00:00
Dan Smith
782ff60cbf Add get_ksa_client() helper
This adds a method in glance.context that will give us a
keystoneauth1 client, authorized with the user's token, suitable for
calling directly to other services.

Related to blueprint distributed-image-import

Change-Id: I71ed8c80939b4cfab6a081c2f8cde63299fc7893
2021-03-01 10:51:29 -08:00
Zuul
bdbad59dc9 Merge "Pass oslo.context RequestContext objects directly to policy enforcement" 2021-02-26 04:59:18 +00:00