Following up on comments in [1], clarify and unify the docs for the
following lockutils methods:
- lock_with_prefix
- synchronized_with_prefix
- remove_external_lock_file_with_prefix
[1] I4e723ee3be1e57c543684390b607c84388c6e930
Change-Id: I4179e8732dba7367bd0c835cbd11939ca7b8cc42
There's a convenience wrapper around the lockutils.synchronized
decorator that lets oslo.concurrency consumers set up a prefix to use
for all lock files (presumably to obviate collisions with other
consumers when using jejune lock names, like 'lock'). This commit adds
an equivalent wrapper around lockutils.lock, the context manager
counterpart to the lockutils.synchronized decorator.
Note that the unit test added for lock_with_prefix is pretty bare; but
it follows the precedent set by the existing tests. Future commits
should make all these tests more thorough/robust.
Change-Id: I4e723ee3be1e57c543684390b607c84388c6e930
This adds support for a "fair" variant of the lock. When there are
multiple entities within a single process that are blocked waiting
for the lock the fair lock will ensure that they acquire the lock
in FIFO order.
For now at least, when fair locks are in use we don't support
the "semaphores" argument.
If external locks are enabled, the inter-process ordering will be
determined by the underlying OS lock ordering and process scheduling.
Change-Id: I37577becff4978bf643c65fa9bc2d78d342ea35a
According to Openstack summit session [1],
stestr is maintained project to which all Openstack projects should migrate.
Let's switch to stestr as other projects have already moved to it.
[1] https://etherpad.openstack.org/p/YVR-python-pti
Change-Id: I753f32ecf3275cf49d8c93bf648a6a26bc6da8e7
In a case when a number of threads are concurrently accessing a
resource that has a external lock there is no indication when
one of the threads has taken the external lock.
This patch adds in a debug log to try and add some clarity.
Change-Id: Ibd7cfd3b39370b1a54f2ab50c75c6b92a876ae1a
This reverts commit 5021ef82fd8f0323b82d6d010bff9dab8a0cbcec.
That commit set default location for lock files to a world-writable
directory (like /tmp) which violates security precaution stated right in
the help string of lock_path config variable: lock_path should be
writable by the user of the process using it only.
There is some history behind this:
- when we decided to not do tempfile.mkdtemp if lock_path were not set
(because it was broken), it broke upgrades for all components
(starting with Cinder;
- it was suggested to provide the same default value:
https://review.openstack.org/60274
but it was not accepted because it weakens security;
- there was a thread about this in ML, please read it for more history:
http://lists.openstack.org/pipermail/openstack-dev/2013-December/021055.html
it ended up with understanding that only a project using the library
can provide sane default value for lock_path, not the library itself.
Change-Id: I801004ad3c83862849696f00c6e7ca9877fd496b
It seems we provide (and projects use) the provided decorator
'synchronized_with_prefix' but we don't provide an equivalent function
to complement that function to clean up its created lock files.
Providing a complementary method seems pretty useful for projects
that actually clean up these lock files (if any actually do in the
first place).
Change-Id: I601ce3992411e6a2ddded13aba4ac068cf8f14e2
Document in which version new types and functions were added using
".. versionadded:: x.y". Document changes using
".. versionchanged:: x.y."
For watchdog module, add the versionadded tag in the module top
docstring, not on each type/function.
Add docstring to ssh_execute().
I used "git blame" + "git tag --contains=SHA1" to find these version,
and then I checked manually each version.
Change-Id: I56a7d8a4335833e9b49eedab026d20adfeedf942
Using the utility function gets a better name.
For example:
$ python
>>> from oslo_utils import reflection
>>> class A(object):
... def m(self):
... pass
...
>>> z = A()
>>> reflection.get_callable_name(z.m)
'__main__.A.m'
Versus:
>>> z.m.__name__
'm'
Change-Id: I2daadd969383aaf49ad87876ba108dd80dd56f08
To ensure we show accurate timing information about
lock acqusition and release times use a timing mechanism
which can not/should not move backwards.
Change-Id: I9559b20cf7de67fc474e6e17eda23791ecc4122e
The fasteners library (extracted from this library and a couple other
variations) provides the interprocess lock logic and the reader writer
lock logic so we can remove the local version and we can just use it
from that library instead.
The tests that were ensuring the internals of this file lock have
now moved to the repo where that library is (for the time being),
currently travis is testing that repo against py2.6, py2.7 and py3.4.
https://github.com/harlowja/fasteners/tree/master/fasteners/tests
Docs also exist at:
http://fasteners.readthedocs.org/en/latest/
Change-Id: I98565b22e68358efe28fea62f74f8ebfcc438ff7
Provide an API to discover the path being used to store external lock
files. Tempest will use this to set up some additional locking and to
test its own behavior. Other projects might use it for error reporting
or other purposes.
Change-Id: Iad40c67072333cc25a6d3e39d7535ff14b573504
Taskflow has a reader/writer lock that is likely useful
to other projects; and it seems better at home in this
module.
The class provides a way to create reader/writer locks
where there may be many readers at the same time (but
only one writer). It does not allow (currently) for privilege
escalation (but this could be added with limited support
in the future).
Change-Id: Ie763ef92f31c34869d83a533bc8761b0fbd77217
In Neutron, we usually specify lock_path as:
lock_path = $state_path/lock
It turned out that ConfigFilter does not work correctly with
substitution feature, so its usage blocks Neutron adoption of the
library.
The issue is also not specific to Neutron: any deployer should be able
to use substitutions ($smth) in their configuration files without
determining whether those configuration options he's interested in are
defined as ConfigOpts or ConfigFilter.
So the best short term approach to push forward the adoption of the
library is to refrain from using ConfigFilter at least until there is
consensus on how to support substitution feature with the class.
Once oslo.config sorts the issue out, we may get back to using
ConfigFilter for lockutils.
Change-Id: I7b0b5c840381b2d9afc00ff116fee1d970743183
Closes-Bug: #1399897
Sometime in a future retrying version there will
likely/hopefully exist a ability to more easily
filter on specific exceptions; this commit adds a
note to the upstream pull request that may add this
feature.
Change-Id: I44f1c9fe4fbbd5f77032ca7bfff4e2e6b7ff7622
When a lock can't be acquired there is currently a hard coded
delay (0.01) that is used before trying again, instead of having
a hard coded delay we should allow this delay to be configured
since having it set at a hard coded value can limit concurrency (if
the delay is actually way to high) or cause to much contention (if
the delay is actually way to low).
This review adds on that logic and also uses the retrying library
to perform the acquisition attempts (and associated failures when/if
they occur); as well as shows logs after a given amount of time has
elapsed with the logs being output at a given periodicity.
Change-Id: Ideeefba1439ddd677c608d01becb4f6a0d4bc83d
It can be undesirable to have a globally shared sempahore
container, especially since oslo.concurrency can now be shared
among many disjoint applications and libraries.
When a single container is used it is now possible to have those
disjoint applications/libraries collide on the same sempahore names.
This is not a good pattern to continue with, so in order to move away
from it allow a custom container to be provided (which defaults to the
existing global one) so that users of oslo.concurrency may provide there
own container if they so desire.
Change-Id: I9aab42e21ba0f52997de3e7c9b0fea51db5c7289
Move the public API out of oslo.concurrency to oslo_concurrency. Retain
the ability to import from the old namespace package for backwards
compatibility for this release cycle.
bp/drop-namespace-packages
Change-Id: I20d1647b1c3ef8cab3b69eccfe168eeb01703b72