229065d781
Currently, the object-expirer is a little delete-happy -- after a
container gets processed, an attempt is always made to delete it,
even if we know it will fail with a 409 because either
* we found items that this worker won't process (see the
process/processes config options) or
* we encountered failures processing items and thus didn't
pop all the records off the queue that we *could* process.
The failed DELETE has downsides such as
* adding unnecessary load to the container servers (though
the 409 can hopefully be serviced fairly quickly) and
* evicting container info and listing shard ranges from
memcache, which
* can further load the container-servers (if the expiry queues
have gotten large enough to shard), as all the expirers
doing listings now have to go fetch shard ranges directly
from the DB.
Now, only attempt to delete an expiry container if it appears to be
empty. In any reasonably-sized cluster, we expect dozens to hundreds of
concurrent expirers to be processing the expiry queues. Seeing the
container empty is the most reliable signal that the delete is expected
to succeed.
Change-Id: I8c3f78d1d1850ab501180f884f81f84552617fb7
|
||
|---|---|---|
| .. | ||
| cors | ||
| functional | ||
| probe | ||
| s3api | ||
| unit | ||
| __init__.py | ||
| debug_logger.py | ||
| sample.conf | ||