Samuel Merritt d5c532a94e object-updater: add concurrent updates
The object updater now supports two configuration settings:
"concurrency" and "updater_workers". The latter controls how many
worker processes are spawned, while the former controls how many
concurrent container updates are performed by each worker
process. This should speed the processing of async_pendings.

There is a change to the semantics of the configuration
options. Previously, "concurrency" controlled the number of worker
processes spawned, and "updater_workers" did not exist. I switched the
meanings for consistency with other configuration options. In the
object reconstructor, object replicator, object server, object
expirer, container replicator, container server, account replicator,
account server, and account reaper, "concurrency" refers to the number
of concurrent tasks performed within one process (for reference, the
container updater and object auditor use "concurrency" to mean number
of processes).

On upgrade, a node configured with concurrency=N will still handle
async updates N-at-a-time, but will do so using only one process
instead of N.

UpgradeImpact:

If you have a config file like this:

    [object-updater]
    concurrency = <N>

and you want to take advantage of faster updates, then do this:

    [object-updater]
    concurrency = 8  # the default; you can omit this line
    updater_workers = <N>

If you want updates to be processed exactly as before, do this:

    [object-updater]
    concurrency = 1
    updater_workers = <N>

Change-Id: I17e18088e61f664e1b9942d66423666d0cae1689
2018-06-13 17:39:34 -07:00
..
2018-04-18 02:07:11 +00:00
2017-12-14 20:12:55 +00:00
2014-02-20 16:15:48 +08:00
2010-07-12 17:03:45 -05:00
2018-01-06 20:48:10 +00:00
2018-05-18 18:48:22 +01:00
2018-05-18 18:48:22 +01:00
2014-06-19 10:18:34 -07:00
2018-04-18 02:07:11 +00:00
2018-05-23 10:34:16 +08:00
2017-06-13 09:23:23 -07:00
2017-05-15 16:42:00 -07:00