100 Commits

Author SHA1 Message Date
melanie witt
43683f543e Add caching of limits in Enforcer
This adds caching of resource limits for an Enforcer in order to
improve performance when repeated limit checks are needed. The cache
lasts the lifetime of an Enforcer and is enabled by default. It can be
disabled by passing cache=False when instantiating an Enforcer.

One usage pattern for a caching Enforcer would be to create an Enforcer
per service request so that the caching lives only as long as the
request.

Change-Id: I8e43dceec76aecd2b2ae23a137e56519efe29777
2021-10-18 21:43:57 +00:00
Dan Smith
caa75c1bab Add a test fixture
This adds a fixture that can be used by consuming projects to
simulate a set of limits in keystone, without requiring actual
keystone. Currently, consumers have to mock oslo.limit internals (at
least) in order to do testing.

Change-Id: If72050e90ca8b03e26d128c7bbcef6bbea92b501
1.4.0 xena-em
2021-06-09 08:04:01 -07:00
Dan Smith
1175b0f7c1 Add Enforcer.calculate_usage()
In multiple situations, it is necessary to be able to probe the limits
set for a project without actually enforcing. Examples:

1. Exposing a usage API where we want to not only report the current
   usage, but the limit as well. Otherwise clients have to do their
   own calls to keystone and correlation to get a single integer
   limit value, which we should be able to expose for them.
2. When checking quota as part of a long-running process of consuming
   an unbounded data stream, we need to be able to determine how much
   quota remains so that we can stop the transfer if we exceed the
   limit. Without this, we have to periodically call to keystone
   during the transfer, which is expensive and could fail.

This patch adds a calculate_usage() method to the Enforcer which
calculates the usage using the enforcement model and returns a
mapping of resource names to namedtuples that contain limit and usage
information.

Change-Id: Ic0632cc5ec52aefb85a04f879651963bfa54dcbe
2021-06-07 08:47:38 -07:00
wu.shiming
efc26ae724 Upgrade the pre-commit-hooks version
[1] 9136088a24

Change-Id: I45f42b47cb3ba45416dc7e4325b4efd635df4a92
2021-05-18 11:11:24 +08:00
Zuul
0820b8c38f Merge "add py38 matedata" 2021-04-16 13:16:45 +00:00
Zuul
dba81a4ebb Merge "Use TOX_CONSTRAINTS_FILE" 2021-04-16 13:10:56 +00:00
Zuul
1bfa1280c0 Merge "remove unicode from code" 2021-04-16 10:43:42 +00:00
52a7eaacfe Add Python3 xena unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for xena.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I8a12f0dcdec43f3a5738858bafa210fdb6f1bac1
2021-04-15 18:18:02 +01:00
9ebe3c67e5 Update master for stable/wallaby
Add file to the reno documentation build to show release notes for
stable/wallaby.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/wallaby.

Sem-Ver: feature
Change-Id: I0ee8832bb010b0929ba0e7a0dd00fa9a399004e7
2021-04-15 18:17:47 +01:00
Hervé Beraud
7f167721ab Fix requirements issues
This patch is the merge of 2 current fixes that need to be solved in the
same time. Indeed each issue lock our gates independently.

Dropping lower constraints testing

We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].

I propose to drop this test because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019390.html

Move flake8 as a pre-commit local target.

The goal here is to avoid conflicts between flake8 and hacking version each
2 days.

Inspired from nova's approach[1].

The flake8 version to install will be determined by hacking and
requirements[2] will stay aligned instead of relying on different versions.

[1] https://opendev.org/openstack/nova/src/branch/master/.pre-commit-config.yaml#L26-L35
[2] https://opendev.org/openstack/hacking/src/branch/master/requirements.txt#L1

Co-authored-by: Daniel Bengtsson <dbengt@redhat.com>
Change-Id: I04a845fd98f0d21f7a21dae9f184117263390dd1
2021-03-23 14:15:59 +01:00
Hervé Beraud
fc270afcc5 Use TOX_CONSTRAINTS_FILE
UPPER_CONSTRAINTS_FILE is old name and deprecated
This allows to use upper-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.

[1] https://review.opendev.org/#/c/722814/
[2] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file

Change-Id: I1a006cf4368cc18c34dd6f236ecfe966e6b09388
2021-01-14 10:30:43 +01:00
xuanyandong
6115a3ea87 remove unicode from code
Change-Id: I09d641c27372bba638aa8be207d27d80c8ec01ab
2021-01-03 16:06:59 +08:00
likui
e4583b63d1 add py38 matedata
Change-Id: I86ffd71d9c5cdda410d654fe9b0bb46ad12eeb05
2020-12-02 13:17:44 +08:00
Hervé Beraud
a5c678ce8b Use py3 as the default runtime for tox
Moving on py3 as the default runtime for tox to avoid to update this at
each new cycle.

Wallaby support officially the following runtimes [1]:
- Python 3.6
- Python 3.8

During Victoria Python 3.7 was used as the default runtime [2] however this
version isn't longer officially supported.

[1] https://governance.openstack.org/tc/reference/runtimes/wallaby.html#python-runtimes-for-wallaby
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: Idb0f7ff31a8fa788c940d70eddd1e7218691a31d
1.3.0
2020-11-03 16:12:44 +01:00
zhoulinhui
592a0d09dd Fix hacking min version to 3.0.1
flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.

Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.

To avoid similar gate break in future, we need to bump the hacking min
version.

- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html

Change-Id: I70ca30dd0a5bf0f1aa44f627d92038c826b67174
2020-10-16 01:19:54 +00:00
maaoyu
9558a46c94 Remove install unnecessary packages
The docs requirements migrated to doc/requirements.txt
we need not install things from requirements.txt.

Change-Id: Ia65109bbb683489ba9a880d3a41baca5ef560d77
2020-09-23 21:20:57 +08:00
Zuul
55675ba447 Merge "Adding pre-commit" 2020-09-22 12:37:24 +00:00
65316ed5ca Add Python3 wallaby unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for wallaby.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I202c5e4bc869e3ac4f03acfca13d59f5e8e02c76
2020-09-11 20:58:01 +00:00
a66a3697c2 Update master for stable/victoria
Add file to the reno documentation build to show release notes for
stable/victoria.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/victoria.

Change-Id: I73e3c0ae23f57c57cb3d55157df3cd8024914100
Sem-Ver: feature
2020-09-11 20:57:59 +00:00
Hervé Beraud
00b4875144 Adding pre-commit
Introduced changes:
- pre-commit config and rules.
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
  (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
  (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks

Change-Id: Ic00da1340d695c7a109f41a09929b654baf995a7
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2020-09-08 13:48:34 +02:00
Moisés Guimarães de Medeiros
ddb62b9fd4 Bump bandit version
This patch bumps bandit allowed version to >=1.6.0,<1.7.0 in order to
avoid the errors detailed here https://github.com/PyCQA/bandit/pull/393

Change-Id: I568123c93c32cb224628c4a8371a9a0e7c19cf67
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
victoria-em 1.2.1
2020-07-24 17:33:41 +02:00
Zuul
8fa46cdf50 Merge "Align contributing doc with oslo's policy" 2020-07-20 15:41:34 +00:00
Zuul
625573ef41 Merge "Add user guide about how to add a new service" 2020-06-22 16:04:19 +00:00
Zuul
0b35c1614c Merge "Add debug tox environment" 1.2.0 2020-06-05 22:08:08 +00:00
Zuul
31d874bcb0 Merge "Add release notes links to doc index" 1.1.0 2020-06-03 09:05:51 +00:00
Colleen Murphy
bce0b7d435 Add debug tox environment
Change-Id: I2e8599555571539e2befb5842e83ca81454fb150
2020-06-01 14:40:20 -07:00
Victor Coutellier
7f93f46070 Add user guide about how to add a new service
This commit add a new user guide about how a service
developer can easily integrate oslo_limit and use
Keystone unified limit system.

Change-Id: I415fdef757e5f78f1c201c32489bc00a1f3cebd0
2020-06-01 16:39:57 +00:00
Zuul
1a64aa8dd5 Merge "Remove translation sections from setup.cfg" 2020-05-26 14:27:40 +00:00
Zuul
f33bf3e035 Merge "Remove the unused coding style modules" 2020-05-26 13:28:35 +00:00
Andreas Jaeger
16f9098312 Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Remove doc requirments from lower-constraints, they are not used for
install. Remove also hacking, the version is ancient and not needed
there either.

openstackdocstheme renames some variables, so follow the renames. A
couple of variables are also not needed anymore, remove them.

Set openstackdocs_auto_name to use project as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I5ec9dda8582f55fc0b287c422d5c43ad2e23b9b8
2020-05-21 13:42:35 +00:00
jacky06
ee778fa233 Remove the unused coding style modules
Python modules related to coding style checks (listed in blacklist.txt in
openstack/requirements repo) are dropped from lower-constraints.txt
as they are not actually used in tests (other than pep8).

more info: https://github.com/openstack/requirements/blob/master/blacklist.txt

Change-Id: I9360028e96dd2b99fee1e31ebf9645e44e845d5f
2020-05-15 17:19:14 +00:00
melissaml
4fb7dd3b40 Remove translation sections from setup.cfg
These translation sections are not needed anymore, Babel can
generate translation files without them.

Change-Id: I18696d80bad13d2f3e31ed1ad8f046fb9a202e16
2020-05-15 18:09:36 +08:00
Hervé Beraud
1443a6ec11 Align contributing doc with oslo's policy
Related to:
- https://review.opendev.org/#/c/723044/
- https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-contrib-docs.html

Change-Id: I6e02e68b689176a1a5e6adff0f32adf79ec473cf
2020-05-04 18:24:13 +02:00
Hervé Beraud
06123b8cb5 Add release notes links to doc index
Change-Id: Ie3794ae02083f5b1fc2026d9189e7ded399b9bda
2020-04-16 17:22:36 +02:00
3b5a820db3 Add Python3 victoria unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for victoria.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I5b9f15bb2a8f90938f682cc9589357619799bfd0
2020-04-14 10:14:43 +00:00
b2685099f2 Update master for stable/ussuri
Add file to the reno documentation build to show release notes for
stable/ussuri.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.

Change-Id: I28d813d853e7961adcf617e452769b64f9e7eccc
Sem-Ver: feature
2020-04-14 10:14:41 +00:00
Andreas Jaeger
485c770abf Update to hacking 3.0
Update to hacking 3.0.

Change-Id: I688ac4aea53d783ff8ae585d42efd8acab067f85
1.0.2 ussuri-em
2020-03-30 16:57:29 +02:00
Andreas Jaeger
009fd6c703 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 2.0 which
supports Python 3.

Blacklist: W504 line break after binary operator

Change-Id: I7ca780a2cff32031c562482b804888e5d49712c1
2020-03-27 19:14:46 +01:00
Zuul
84ed5bc4d9 Merge "drop use of six" 2020-03-17 16:01:19 +00:00
Sean McGinnis
2c1ccf6263
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: I0d2b84e2ed627481118151b36386014ed0930452
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-03-13 11:47:56 -05:00
Zuul
d7c3c5c4f2 Merge "ignore reno generated artifacts" 2020-03-11 16:14:01 +00:00
Daniel Bengtsson
4a1c93a513 Update the minversion parameter.
Update the minversion parameter to use the python -m pip to install
python packages:

https://tox.readthedocs.io/en/latest/changelog.html#id185

It's recommend to use this. Remove the useless install_command
parameter.

Change-Id: Ic5c0f747c8d579ce3369635b67cdbf2d134c175d
2020-03-03 15:25:51 +01:00
Hervé Beraud
f831302d61 ignore reno generated artifacts
Change-Id: Ifde690d7e5f59fc95533bc97d513d6aa6c432599
2020-03-03 14:52:45 +01:00
Hervé Beraud
2245235ece drop use of six
Change-Id: I40bb06d1d445d8eb8281f421a94d7dead2a89872
2020-03-02 14:58:28 +01:00
Hervé Beraud
64b78b7878 oslo.limit mistakenly released as 1.0.0 (release note)
For further reading:
http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012606.html

Change-Id: I44cb645bd118c17e1f067cb617dbce845dacd5c6
1.0.1
2020-02-21 10:34:02 +01:00
Hervé Beraud
eba8d80357 remove outdated header
Change-Id: I1bda5a8ccc9df040bf88df6bf8f428b6502f4542
2020-02-11 18:53:20 +01:00
Hervé Beraud
188ee76177 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in Ussuri cycle.

Complete discussion & schedule can be found in
-
http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: Ie48c796b6c83b42c85555d13a09f3b3ed96428f8
Sem-Ver: api-break
1.0.0
2020-02-06 10:24:15 +01:00
caoyuan
09eadf955e Trivial cleanup for tox
move 'basepython' to the top-level 'testenv'

Change-Id: I197d6486a3b126f616ffe2f7e3e669d922a1e337
2019-12-23 13:04:09 +00:00
Zuul
51431f19fc Merge "Add flat enforcer" 0.3.0 2019-12-05 20:21:07 +00:00
Zuul
54f4a893b3 Merge "Fetch unified limits from keystone" 2019-12-04 23:45:44 +00:00