swift/test/probe
Matthew Oliver 03b66c94f4 Proxy: Use namespaces when getting listing/updating shards
With the Related-Change, container servers can return a list Namespace
objects in response to a GET request.  This patch modifies the proxy
to take advantage of this when fetching namespaces. Specifically,
the proxy only needs Namespaces when caching 'updating' or 'listing'
shard range metadata.

In order to allow upgrades to clusters we can't just send
'X-Backend-Record-Type = namespace', as old container servers won't
know how to respond. Instead, proxies send a new header
'X-Backend-Record-Shard-Format = namespace' along with the existing
'X-Backend-Record-Type = shard' header. Newer container servers will
return namespaces, old container servers continue to return full
shard ranges and they are parsed as Namespaces by the new proxy.

This patch refactors _get_from_shards to clarify that it does not
require ShardRange objects. The method is now passed a list of
namespaces, which is parsed from the response body before the method
is called. Some unit tests are also refactored to be more realistic
when mocking _get_from_shards.

Also refactor the test_container tests to better test shard-range and
namespace responses from legacy and modern container servers.

Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Co-Authored-By: Jianjian Huo <jhuo@nvidia.com>
Related-Change: If152942c168d127de13e11e8da00a5760de5ae0d
Change-Id: I7169fb767525753554a40e28b8c8c2e265d08ecd
2024-01-11 10:46:53 +00:00
..
__init__.py probe tests: Work when fronted by a TLS terminator 2020-05-04 10:54:01 -07:00
brain.py Fix test.probe.brain CLI 2022-09-12 13:54:20 -07:00
common.py swift-manage-shard-ranges repair: check for parent-child overlaps. 2022-09-09 11:04:43 -07:00
test_account_failures.py Remove executable flag from some test modules 2016-10-31 21:22:10 +00:00
test_account_get_fake_responses_match.py probe tests: Work when fronted by a TLS terminator 2020-05-04 10:54:01 -07:00
test_account_reaper.py Update hacking for Python3 2020-04-03 21:21:07 +02:00
test_container_failures.py Allow internal container POSTs to not update put_timestamp 2023-03-20 11:41:27 +00:00
test_container_merge_policy_index.py replace md5 with swift utils version 2020-12-15 09:52:55 -05:00
test_container_sync.py Get rid of pipeline_property 2022-01-27 14:40:27 -08:00
test_dark_data.py tests: Make dark data probe tests pass with sync_method = ssync 2023-05-17 15:25:22 -07:00
test_db_replicator.py Apply remote metadata in _handle_sync_response 2018-03-06 19:52:59 +00:00
test_empty_device_handoff.py pep8: Turn on E305 2020-04-03 21:22:38 +02:00
test_mixed_policy_mpu.py s3api: Stop propagating storage policy to sub-requests 2023-11-14 15:09:18 +00:00
test_object_async_update.py func tests: work with etag-quoter on by default 2020-06-01 18:38:23 -05:00
test_object_conditional_requests.py py3: (mostly) port probe tests 2019-09-04 10:17:45 -07:00
test_object_expirer.py Add a new URL parameter to allow for async cleanup of SLO segments 2020-11-10 18:22:01 +00:00
test_object_failures.py Get rid of port to node assumptions and their modulo kludges 2021-09-30 00:42:24 -05:00
test_object_handoff.py Get rid of port to node assumptions and their modulo kludges 2021-09-30 00:42:24 -05:00
test_object_metadata_replication.py func tests: work with etag-quoter on by default 2020-06-01 18:38:23 -05:00
test_object_partpower_increase.py Drop the unnecessary sudo from probe tests 2021-05-10 17:25:09 -05:00
test_object_versioning.py tests: Fix config numbers in test_versioning_with_metadata_replication 2023-04-27 15:20:07 -07:00
test_orphan_container.py Band-aid and test the crash of the account server 2021-08-12 16:26:48 -05:00
test_reconstructor_rebuild.py reconstructor: make quarantine delay configurable 2021-07-06 16:41:08 +01:00
test_reconstructor_revert.py Add non-ascii meta values to ssync probe test 2023-08-03 12:33:56 -07:00
test_replication_servers_working.py Have REPLICATE with suffixes just append to hashes.invalid 2020-11-16 08:21:16 -06:00
test_reserved_name.py Allow internal clients to use reserved namespace 2019-11-27 11:22:00 -06:00
test_sharder.py Proxy: Use namespaces when getting listing/updating shards 2024-01-11 10:46:53 +00:00
test_signals.py Merge "probe-tests: Avoid a DuplicateSectionError on py3" 2020-10-07 00:11:43 +00:00