swift/test/unit/obj
Samuel Merritt c4751d0d55 Make reconstructor go faster with --override-devices
The object reconstructor will now fork all available worker processes
when operating on a subset of local devices.

Example:
  A system has 24 disks, named "d1" through "d24"
  reconstructor_workers = 8
  invoked with --override-devices=d1,d2,d3,d4,d5,d6

In this case, the reconstructor will now use 6 worker processes, one
per disk. The old behavior was to use 2 worker processes, one for d1,
d3, and d5 and the other for d2, d4, and d6 (because 24 / 8 = 3, so we
assigned 3 disks per worker before creating another).

I think the new behavior better matches operators' expectations. If I
give a concurrent program six tasks to do and tell it to operate on up
to eight at a time, I'd expect it to do all six tasks at once, not run
two concurrent batches of three tasks apiece.

This has no effect when --override-devices is not specified. When
operating on all local devices instead of a subset, the new and old
code produce the same result.

The reconstructor's behavior now matches the object replicator's
behavior.

Change-Id: Ib308c156c77b9b92541a12dd7e9b1a8ea8307a30
2018-04-25 11:18:35 -07:00
..
__init__.py Initial commit of Swift code 2010-07-12 17:03:45 -05:00
common.py Merge "Add Timestamp.now() helper" 2017-07-18 03:27:50 +00:00
test_auditor.py Change object_audit_location_generator() to yield for a single policy. 2018-03-22 03:02:35 +00:00
test_diskfile.py Change object_audit_location_generator() to yield for a single policy. 2018-03-22 03:02:35 +00:00
test_expirer.py Unify expirer unit test task queue 2018-03-08 08:07:24 +00:00
test_reconstructor.py Make reconstructor go faster with --override-devices 2018-04-25 11:18:35 -07:00
test_replicator.py Multiprocess object replicator 2018-04-24 04:05:08 +00:00
test_server.py Fix object-server to not 400 all expirer DELETEs 2018-03-02 15:25:38 +00:00
test_ssync_receiver.py Add checksum to object extended attributes 2017-11-03 13:30:05 -04:00
test_ssync_sender.py Multiprocess object replicator 2018-04-24 04:05:08 +00:00
test_ssync.py Add checksum to object extended attributes 2017-11-03 13:30:05 -04:00
test_updater.py Keep object-updater stats logging consistent 2018-01-18 12:09:07 -08:00