This makes us enforce a quota on the total number of upload-related
image operations owned by a user.
Partially-implements: blueprint glance-unified-quotas
Change-Id: I2a28750aaf968e6a6324eb194d4280a640bfa5aa
This adds an efficient query method for getting a count of the user's
images that are in any state of uploading. This includes images
in the process of saving from upload, staging, importing, or being
copied to another store.
Partially-implements: blueprint glance-unified-quotas
Change-Id: Iebba15771000e86a0701d20ed9620eb578c2cb85
This adds a new tox target called functional-py38-rbac which enables
the new secure RBAC policy defaults for all functional tests. To do
this, the functional tests needed a little bit of extra work to
actually set those, and a new non-voting job is added to run these
in CI.
Related to blueprint policy-refactor
Change-Id: Id376193521671bdb0ebc08ea8e563578bbaa541f
Made changes to use default policies in our unit and functional tests
rather than referring it from glance/tests/etc/policy.yaml file. Existing
function 'set_policy_rules' can be used to test custom policy rules in
functional testing.
Improvements needed in followup patch:
1. Property protection related unit tests still reads the special policy
from the policy.yaml file, need to make provision to override it instead.
2. Need to remove 'set_policy' function which actually responsible for
above behavior.
Related to blueprint policy-refactor
Change-Id: I0de9b9f9a7de499574101e3366ced730b0cb5fd1
This makes us enforce a quota on the total number of (non-deleted)
images owned by a user.
Partially-implements: blueprint glance-unified-quotas
Change-Id: I8af124d9307263cd8289d0701fb9a745d13b1d56
This adds an efficient query method for getting a count of a user's
non-deleted images. It can be used by subsequent patches to enforce
the image count quota.
Partially-implements: blueprint glance-unified-quotas
Change-Id: I4d6e071b8d48862032f36c80ee19632a3fe85009
This makes us enforce a quota on the amount of data a user has in
staging.
Partially-implements: blueprint glance-unified-quotas
Change-Id: I3cca4e589adc0aec138e5933c311aefd69ccee51
This adds a method that counts the sizes of images that are in the
process of doing an import, indicating that they are using staging
storage resources. We gather images that are in obvious states of
being created via import, as well as any images that appear to be
active but in the middle of a copy, since those could be using staging
resources as well.
This is somewhat mirrored from user_get_storage_usage(). Since there
was no test for that method, this adds a test case for both that and
the new method.
Partially-implements: blueprint glance-unified-quotas
Change-Id: Ic2b045549ac0d6112b0c565d2e993576a5626799
This adds enforcement of the image_size_total keystone limit for
image upload and import. We simply check the quota before either of
these operations and refuse to proceed further if the user is over
their quota.
Note that this disables checking of the global size quota if keystone
quotas are enabled.
Note this includes another fix to couple unit tests that do not
properly pass context to the get_flow() method.
Partially-implements: blueprint glance-unified-quotas
Change-Id: Idf5f004b72436df1f9c77bb32d60b9be5ae77a68
This fixes the test to behave the way we expect. It was failing to
do the update because it was using an image the requester did not
own, and asserting the found behavior of 403. However, the intent
here is to allow it to be updated. So, this uses the proper image and
asserts the proper behavior.
Change-Id: I71afe6a877485c8f92e67dcf32bb475c1a1a42a3
Closes-Bug: #1933360
This adds some infrastructure to be able to query and honor limits
declared in keystone. It adds a single initial quota value for the
total size of all active images for bootstrapping the tests.
Checking these values is controlled by a new configuration option
that globally enables and disables the checking, defaulting to
False.
Related to blueprint glance-unified-quotas
Change-Id: I8d8f4aaed465486e80be85bc9a5d2c2be7f1ecad
When we stage an image, we know the image size, so we should set it.
Since the user may be streaming an image and did not declare the
expected size, this lets them confirm before they import. It also
provides us a value to count for the staging quota coming in a later
patch.
The bulk of this change is adjusting existing tests to validate the
size for all of our stage/import tests.
This follows the change to set the image size during conversion,
ensuring that we can set it during stage for non-conversion jobs,
and that the conversion code that re-sets it after changing the
image data and size continues to work.
Related to blueprint glance-unified-quotas
Change-Id: I93a9145df27594a0cc59828619a7d0573e58d4fc
When we convert an image to a specified format during import, we
update the disk_format to match. At that point, we also know the (new)
image.size, so we should set it.
This is somewhat related to setting image size on stage, in that once
it is set we will validate that it does not change in later steps.
Since this one comes between stage and the actual store upload, this
patch makes conversion set it and confirms that the later steps are
happy with that. A later patch sets it during stage, confirming that
we can change it here during conversion when we are changing the
actual image file itself.
Related to blueprint glance-unified-quotas
Change-Id: I795c52f606f85955e39efc29b75f2941be1264b4
This simply adds some functionality and flexibility to this test base
so that subsequent patches can do some more things.
Related to blueprint glance-unified-quotas
Change-Id: Ic9359aac5dba2b4d7d0d2c7fa92a5b67440e22e0
This reverts commit 810417df865c7b4eb71cee4dfee843d65972852b.
The Swift driver not being able to use Trusts had nothing to do
about the store driver itself nor that keystoneauth1 would have
broken the feature, but rather it not having the functionality
in the first place and us not catching that on reviews.
We should figure out how to test this before we try to replace
this code again.
Change-Id: If12a013404296486dc387b099477d1608b24ba63
Closes-Bug: #1916052
TaskExecutor._get_flow() passes a context argument to the task
initialization routine, but a couple of tests that short-circuit that
do not. This makes those tests mimic the real behavior separately
from the later patch that needs it to make it clear that it is not
actually requiring a new argument at runtime.
Related to blueprint glance-unified-quotas
Change-Id: Ifc2adcb7f8eaa7da2e7b063a8b79175025582500
Currently an exception raised in a get_flow() task builder will be
silently swallowed with no logging. This patch makes us log it and
then re-raise it so that it will be visible.
Change-Id: I4315bfd68e5a184305f384705e35c4049f75b906
This makes ImageTarget integrate the aliasing of project_id to owner,
avoiding the need to do it every time we use ImageTarget. This will
simplify subsequent patches.
Change-Id: Ia9998f638546ac36cd200f0c9002eadd18b71094
This gives us a consistent context with roles and project affiliation
for later policy check patches.
Change-Id: I241a81aabe0ca6f8c2b2abacedd86aed9e68c79c
This fixes the cache tests (and the 'caching' deployment flavor) to
send authorization headers so that they can be checked by later policy
patches.
Change-Id: I6099c7da24e06595e08fd292a5083327b9f0cc64
The scrubber tests do not pass suitable HTTP headers to pass any
of the policy checks we are going to start adding. This just imports
the base header template from other functional tests so that the
context that is constructed has a proper tenant member arrangement.
Change-Id: I350499b4b3f326ade2c64a2d50e1562c413e449a
Within simple/db/api.py, DATA['members'] was being statically
initialized as a dict.
Inspection of the code revealed a reset function that initialized the
same as a list. Further code inspection seemed to confirm that it was
always intended to be a list.
Updated static initialization to be a list.
Change-Id: Ia53890bcd1161b66a1b365bb3bcc7fc9fbcf6e40
Closes-Bug: #1620833
update tests for cinder's new attachment API code in glance
cinder store. This leaves a couple of mocks in place for the old
way of doing things so that we can tolerate the old and proposed
changes to glance_store. We can remove those once we require the
newer glance_store version.
Change-Id: If4db20affbba37806f8419b6b30050a32bcc2435
A slight error in the tasks_get_by_image() DB API method resulted in
our excluding in-progress tasks from the returned list. This is
because those tasks have expires_at=NULL, and we were comparing
the expires_at>=$NOW to find unexpired tasks. This makes us check for
"NULL or not expired" instead.
We did have a test asserting the wrong behavior, but it was done to
increase coverage and thus was asserting the behavior of the code and
not the *desired* behavior. This fixes that as well.
Closes-Bug: #1922928
Change-Id: I1b6971888673b64ef60bed8fbcc97bbcbcf5c2ac