10278 Commits

Author SHA1 Message Date
Zuul
a57479a023 Merge "Add FakeStatsdClient to unit tests" 2023-08-08 20:40:33 +00:00
Zuul
54f8dd0ff8 Merge "proxy-server: de-duplicate implementation of is_good_source" 2023-08-08 13:11:30 +00:00
Zuul
3bb949bce0 Merge "proxy-server: clarify method naming in GET path" 2023-08-08 13:11:27 +00:00
Zuul
e81f0657db Merge "Save some effort when discovering used_source_etag" 2023-08-08 13:11:23 +00:00
Alistair Coles
6251539dce proxy-server: de-duplicate implementation of is_good_source
Both the EC and replicated GET paths have is_good_source() methods
that are similar. Refactor to have just one implementation.

Change-Id: I661d3704a76e3d92bfcfeaed1fff4ed5e28c79b4
2023-08-08 10:17:30 +01:00
Alistair Coles
629cd57514 proxy-server: clarify method naming in GET path
This patch tries to make the proxy-server GET path a little easier to
read. The changes are primarily to the GetOrHeadHandler and
ECFragGetter classes.

  - Make generator method names more uniform.

  - Use _<method> to make it more obvious which methods are internal
    to the class and which methods are the interface to the class.

  - Move the interface method to the end of the class.

  - Add some commentary and docstrings.

No intended behavioral change.

Change-Id: I3d00b1071669a42526a31588a2643f91c58ea5a8
2023-08-08 10:17:30 +01:00
Zuul
7c63fe476a Merge "proxy: encapsulate Getter resp, node and parts_iter" 2023-08-08 01:33:44 +00:00
Zuul
e71102c5bd Merge "docs: Clean up proxy logging docs" 2023-08-07 13:00:59 +00:00
Tim Burke
fe09ef2581 Save some effort when discovering used_source_etag
Change-Id: I35d682e58419f141429877ac6a6964132f7e658b
2023-08-07 12:16:32 +01:00
Alistair Coles
f8a6dee622 proxy: encapsulate Getter resp, node and parts_iter
The proxy GetterBase manages a set of attributes related to the
backend source for a GET response: source (a response object), node
and source_parts_iter. These attributes are always updated together so
benefit from encapsulation, along with some helper methods to simplify
the GET paths.

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: I76ea042ef4b3f5cc1caa4774e35d6191f4ca548e
2023-08-07 11:17:17 +01:00
Matthew Oliver
00bfc425ce Add FakeStatsdClient to unit tests
Currently we simply mock calls in the FakeLogger for calls statsd calls,
and there are also some helper methods for counting and collating
metrics that were called. This Fakelogger is overloaded and doesn't
simulate the real world.
In real life we use a Statsdclient that is attached to the logger.

We've been in the situation where unit tests pass but the statsd client
stacktraces because we don't actually fake the statsdclient based off
the real one and let it's use its internal logic.

This patch creates a new FakeStatsdClient that is based off the real
one, this can then be used (like the real statsd client) and attached to
the FakeLogger.
There is quite a bit of churn in tests to make this work, because we now
have to looking into the fake statsd client to check the faked calls
made.
The FakeStatsdClient does everything the real one does, except overrides
the _send method and socket creation so no actual statsd metrics are
emitted.

Change-Id: I9cdf395e85ab559c2b67b0617f898ad2d6a870d4
2023-08-07 10:10:45 +01:00
Zuul
b24a34f7fa Merge "Add non-ascii meta values to ssync probe test" 2023-08-04 21:20:29 +00:00
Tim Burke
0c9b545ea7 docs: Clean up proxy logging docs
Change-Id: I6ef909e826d3901f24d3c42a78d2ab1e4e47bb64
2023-08-04 11:30:42 -07:00
Zuul
95a2c71f23 Merge "container-server: use LIMIT in get_own_shard_range() query" 2023-08-04 10:40:44 +00:00
Zuul
1f9937b245 Merge "docs/SAIO: Make reconciler use the SAIO convention for logs" 2023-08-04 04:00:43 +00:00
Zuul
7b268fcdc1 Merge "doc/SAIO: Update the Fedora section" 2023-08-04 03:07:06 +00:00
Pete Zaitcev
27120b0523 docs/SAIO: Make reconciler use the SAIO convention for logs
Personally I'm not a big fan of how we arrange logs for SAIO,
but it is a historic standard. The reconciler has to conform.

Change-Id: I45a25ff406b31b6b1b403e213554aaabfebc6eb5
2023-08-03 15:01:38 -05:00
Pete Zaitcev
dd664c47db doc/SAIO: Update the Fedora section
The never ending changes in package names march on.
This is for Fedora 38.

Change-Id: Idffd1df7e2f23681eb7d7bf1717471b594612dbc
2023-08-03 15:00:06 -05:00
Tim Burke
cb468840b9 Add non-ascii meta values to ssync probe test
Change-Id: I61c7780a84a1f0cee6975da67d08417cf6aa4ea2
2023-08-03 12:33:56 -07:00
Alistair Coles
6e8d82c97e container-server: use LIMIT in get_own_shard_range() query
The get_own_shard_range() method is called during every container GET
or HEAD request (as well as at other times). The method delegates to
get_shard_ranges() which queries the DB for shard ranges. Although the
query has conditions to only select a single shard range name, and the
method will return only the first matching shard range, the query had
no LIMIT. Adding a LIMIT of 1 significantly reduces execution time
when the DB has many shard range rows.

On the author's machine, using a DB with ~12000 shard ranges this
patch reduces the get_own_shard_range() execution time by a factor of
approximately 100.

Change-Id: I43f79de3f0603b9fab8bf6f7b4c3b1892a8919b3
2023-08-03 11:48:31 -07:00
Zuul
55cd675f8b Merge "proxy: add new metrics to account/container_info cache for skip/miss" 2023-08-03 01:23:31 +00:00
Zuul
6c1d3535b0 Merge "container-server: do [end_]marker filtering in SQL query" 2023-08-03 00:03:49 +00:00
Jianjian Huo
bc300a516b proxy: add new metrics to account/container_info cache for skip/miss
This patch will add more granularity to metrics of account_info or
container_info cache and related backend lookups.

Before this patch, related metrics are:
  1.account.info.cache.[hit|miss|skip]
  2.container.info.cache.[hit|miss|skip]

With this patch, they are going to become:
  1.account/container.info.infocache.hit
    cache hits with infocache.
  2.account/container.info.cache.hit
    cache hits with memcache.
  3.account/container.info.cache.[miss|skip|disabled]
                    .<status_int>
    Those are operations made to backend due to below reasons.
      miss: cache misses.
      skip: the selective skips per skip percentage config.
      disabled: memcache is disabled.
    For each kind of operation metrics, suffix <status_int> will
    count operations with different status. Then a sum of all
    status sub-metrics will the total metrics of that operation.

UpgradeImpact
=============
Metrics dashboard will need updates to display those changed metrics
correctly, also some infocache metrics are newly added, please see
above message for all changes needed.

Change-Id: I60a9f1c349b4bc78ecb850fb26ae56eb20fa39c6
2023-08-02 11:12:01 -07:00
Tim Burke
c51e81f640 proxy: Bring back logging/metrics for get_*_info requests
A while back, we changed get_account_info and get_container_info to
call the proxy-server app directly, rather than whatever was right
of the current middleware. This reduced backend request amplification
on cache misses.

However, it *also* meant that we stopped emitting logs or metrics in
the proxy for these backend requests. This was an unfortunate and
unintended break from earlier behavior.

Now, extend the middleware decorating we're doing in loadapp() to
include a "logged app", i.e., the app wrapped by it's right-most
proxy-logging middleware. If there is not logging middleware (such
as would happen for the backend servers), the "logged app" will be
the main app. Make account and container info requests through
*that* app, so we get logging and metrics again.

Closes-Bug: #2027726
Related-Change: I49447c62abf9375541f396f984c91e128b8a05d5
Change-Id: I3f531f904340e4c8407185ed64b41d7d614a308a
2023-08-01 15:58:58 -07:00
OpenStack Proposal Bot
8a5baf8e13 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I7b9907645d7357824e197fa213f0ec68283ee72b
2023-07-28 04:11:58 +00:00
Zuul
167aff4b84 Merge "versioning: Only list versions container if it seems to exist" 2023-07-26 04:14:08 +00:00
Zuul
8b900becd5 Merge "proxy: remove client_chunk_size and skip_bytes from GetOrHeadHandler" 2023-07-24 22:04:28 +00:00
Alistair Coles
369a72c4cf proxy: remove client_chunk_size and skip_bytes from GetOrHeadHandler
The client_chunk_size attribute was introduced into GetOrHeadHandler
for EC support [1]. It was only ever not None for an
ECObjectController. The ECObjectController stopped using
GetOrHeadHandler for Object GET when the ECFragGetter class was
introduced [2], but the EC specific code was not expunged from
GetOrHeadHandler. In [3] the ECFragGetter client_chunk_size was renamed
to fragment_size to better reflect what it represented.

The skip_bytes attribute was similarly introduced for EC support. It
is only ever non-zero if client_chunk_size is an int. For EC,
skip_bytes is used to undo the effect of expanding the backend
range(s) to fetch whole fragments: the range(s) of decoded bytes
returned to the client may need to be narrower than the backend
ranges. There is no equivalent requirement for replicated GETs.

The elimination of client_chunk_size and skip_bytes simplifies the
yielding of chunks from the GetOrHeadHandler response iter.

Related-Change:
[1] I9c13c03616489f8eab7dcd7c5f21237ed4cb6fd2
[2] I0dc5644a84ededee753e449e053e6b1786fdcf32
[3] Ie1efaab3bd0510275d534b5c023cb73c98bec90d

Change-Id: I31ed36d32682469e3c5ca8bf9a2b383568d63c72
2023-07-24 09:15:12 -05:00
Zuul
740636b8e4 Merge "Add more unit tests for ranged GETs" 2023-07-24 06:12:11 +00:00
Zuul
1e352accec Merge "proxy: Pop ignore-range backend header on resume" 2023-07-24 06:12:08 +00:00
Zuul
1f007fd86b Merge "Add a docstring for utils.Namespace" 2023-07-21 15:02:25 +00:00
Zuul
427f575c4d Merge "docs: Fix link to sphinx project page" 2023-07-21 14:14:33 +00:00
Zuul
f6eae81584 Merge "Container-server: only check for shard range existence for DB state queries" 2023-07-21 11:21:40 +00:00
Alistair Coles
8e051c5b8e docs: Fix link to sphinx project page
Change-Id: I5e75a359660315a34ec36f5cb748808257f4185a
2023-07-21 10:19:49 +01:00
Alistair Coles
46eeea11d4 Add a docstring for utils.Namespace
The Related-Change extracted some attributes of the ShardRange class
into a new Namespace superclass. This patch adds a docstring for
Namespace.

Change-Id: I5a79c4a6da6e62698403bb0a5ef566355b5c850e
Related-Change: If98af569f99aa1ac79b9485ce9028fdd8d22576b
2023-07-21 09:10:33 +01:00
Jianjian Huo
bfbe8f909f Container-server: only check for shard range existence for DB state queries
Within ContainerBroker, at various places, for example get_db_state()
and sharding_initiated(), they query the number of shard ranges of
this container by pulling out all shard ranges from shard range table,
instantiating ShardRange objects and then counting how many they are.
Those operations are very expensive, and causing HEAD/GET into large
containers to be very slow.

Instead, this patch only checks if there is any qualified shard range
exists in the shard table with optimized SQL query, which can be very
fast. On a container server setup which serves a container with ~12000
shard ranges, this patch alone improves HTTP HEAD request rate by ~10X,
and improves HTTP GET request rate by ~2X; and together with other
optimizations (patch 888310 & 888575) targeting to fix similar problems,
strong synergistic effects are seen to bring total ~22X improvement to
HTTP HEAD and ~27X to HTTP GET request rates.

Change-Id: I01fd4f3e395c8846280f44e17a56935fc6210444
2023-07-20 10:44:31 -07:00
Alistair Coles
f7d6d5806e container-server: do [end_]marker filtering in SQL query
Each time the sharder audits a shard container it makes a GET request
to the root container for shard ranges, typically specifying
[end_]marker constraints to select a single shard range. When a
container DB has a large number of shard ranges, the execution time of
the get_shard_ranges() method was dominated by the time taken to
instantiate ShardRange objects for every shard range before then
selecting only those that satisfied the [end_]marker constraints. The
get_shard_ranges() call can be much faster if the [end_]marker
filtering is performed by the SQL query *before* instantiating
ShardRange objects for the selected shard ranges.

On the author's machine, using a DB with ~12000 shard ranges and
selecting a single shard range with marker and end_marker constraints,
this patch reduces the get_shard_ranges() execution time from
approximately 140ms to approximately 6ms.

A similar change was previously made to optimise calls with the
'includes' constraint [1].

[1] Related-Change: Ie40b62432dd9ccfd314861655f7972a2123938fe
Change-Id: Ic6e436f9e8fdff6a525466757f57f9be3a81c5b6
2023-07-20 14:04:11 +01:00
Alistair Coles
bdbe8ce9f8 s3api: fix statsd prefix mutation
The 'log_route' argument of utils.get_logger() determines which global
Logger instance is wrapped by the returned LogAdapter. Most middlewares
(s3api being the exception) explicity set 'log_route' to equal the
middleware 'brief' name e.g. 'bulk', 'tempauth' etc. However, the
s3api middleware sets 'log_route' to be the config 'log_name', if that
key is found in config.

When a proxy pipeline is instantiated via wsgi.run_wsgi(), all
middlewares and the proxy app are passed a default conf with
'"log_name": "proxy-server"'. As a result, the s3api middleware calls
get_logger() with log_route='proxy-server' and its LogAdapter
therefore shares the same Logger instance used by proxy-server app
(and any other middleware that similarly fails to explicitly
differentiate 'log_route)'.

Each Logger instance has a StatsdClient instance bound to it by
get_logger(). The Related-Change added statsd metrics to the s3api
middleware and sets 's3api' as the 'statsd_tail_prefix' when calling
get_logger(). This had the unintended effect of replacing the shared
Logger instance's StatsdClient with one that has prefix 's3api', such
that stats emitted by the proxy app (e.g. memcache shard range
hit/miss stats) would be erroneously prefixed with 's3api'.

This patch modifies the s3api middleware logger instantiation to
explictly set log_route='s3api', so that the s3api middleware
LogAdapter now wraps a unique global Logger instance, with a unique
StatsdClient instance bound to it.

The 'server' attribute of the middleware's LogAdapter, which may be
included in log lines by the "%(server)s" format element, is not
affected by this change. Its value is derived from the config
'log_name' or the 'name' argument passed to get_logger().

Change-Id: Ia89485bae8f92f4f3d9f5375cab8ff08f70a11a7
Related-Change: I4976b3ee24e4ec498c66359f391813261d42c495
2023-07-20 09:56:09 +01:00
Alistair Coles
771d605d03 Add more unit tests for ranged GETs
Change-Id: Icdc74856f7beb318bfd84b39b5776d0d531a74b9
2023-07-12 12:08:30 +01:00
Zuul
7cc8a01729 Merge "Object-server: keep SLO manifest files in page cache." 2023-07-10 19:50:47 +00:00
Zuul
338908c830 Merge "Encode header in latin-1 with wsgi_to_bytes" 2023-07-10 17:44:30 +00:00
Romain de Joux
365c0ef005 Encode header in latin-1 with wsgi_to_bytes
Prevent encoding corruption in client's metadata during ssync

Closes-Bug: #2020667
Change-Id: I0ea464bcda16678997865667287aa11ea89cdcde
2023-07-10 15:09:04 +01:00
Jianjian Huo
cb1e584e64 Object-server: keep SLO manifest files in page cache.
Currently, SLO manifest files will be evicted from page cache
after reading it, which cause hard drives very busy when user
requests a lot of parallel byte range GETs for a particular
SLO object.

This patch will add a new config 'keep_cache_slo_manifest', and
try keeping the manifest files in page cache by not evicting them
after reading if config settings allow so.

Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Change-Id: I557bd01643375d7ad68c3031430899b85908a54f
2023-07-07 12:48:24 -07:00
Jianjian Huo
1b7cf29476 Object-server: backfill unit test coverage for keep_cache_private
Change-Id: I035ef2dcc0d0e09337bd2e742baeff1fb62bf018
2023-07-06 22:36:10 -07:00
Tim Burke
14a227f305 versioning: Only list versions container if it seems to exist
It costs us an extra HEAD sometimes, but those at least get cached
some. Seems better than doing real GETs and going out to handoffs
every time when versioning isn't enabled.

Change-Id: Ic1b3a8c6c9b1aaead25070e49f514785c2d52c98
2023-07-06 11:24:06 -07:00
Zuul
671dbc8862 Merge "docs: Format metrics in fixed-width font, not italics" 2023-07-05 22:48:46 +00:00
Zuul
e7897f86c2 Merge "s3api: emit metrics for error responses" 2023-07-05 22:48:43 +00:00
Zuul
211152ae40 Merge "tests: Stop requiring <1ms test runtime" 2023-07-05 22:48:15 +00:00
Zuul
42f76159ac Merge "Fix handling of non-ASCII accounts" 2023-07-05 22:48:12 +00:00
Zuul
1b0d871792 Merge "CI: test under py311" 2023-07-04 01:29:04 +00:00