For various reasons, an operator might want to use specifics nameservers
instead of the systems ones to resolve CNAME in cname_lookup. This patch
creates a new configuration variable nameservers which accepts a list of
nameservers separated by commas. If not specified or empty, systems
namservers are used as previously.
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Change-Id: I34219e6ab7e45678c1a80ff76a1ac0730c64ddde
test_object_metadata_replication requires a replicated policy. We even
have it subclass ReplProbeTest, but then we hardcoded the policy index!
Stop doing that.
Change-Id: I8871cc0beceb0909abaf59babe40c3cafbcd0cc9
Warnings-as-errors breaks our docs gate currently, as the gate is using
an old liberasurecode. Stop treating warnings as errors until the gate
can be updated with a more-recent version.
Related-Change: I3985d027f0ac2119ceaeb4daba5964f937de6cea
Change-Id: I4869ec370f08e338be78d2c0f930106081f21a0a
The description of storage policy config options was
unstructured and repetitive. This patch attempts to
improve the doc by gathering the notes for each option
into a structured list.
Change-Id: I57090b35a70f365e82fb0e29ab42e533d6359a7b
- add proxy server per policy config as an optional
step in the configuration of a policy, with link to
the deployment guide
- add reverse link from deployment guide per-policy
config doc section to storage policies docs
Drive-by fix an incorrect test comment
Change-Id: Ib95310193270a63c9d1e321c6e7de240e00b387f
Related-Change: I3f718f425f525baa80045ba067950c752bcaaefc
Instead, link to the middleware list and auth overview, as well as
referring readers to proxy-server.conf-sample
TempAuth-related content that was previously in the deployment guide has
been moved to TempAuth's own docs, which have been cleaned up a bit.
Change-Id: I00070bb09294362c069f7ee9426ac570bc1b3ddb
This is an alternative approach to that proposed in [1]
Adds support for optional per-policy config sections
to be added in proxy-server.conf. This is highly desirable
to allow per-policy affinity options to be set for use with
duplicated EC policies [2] and composite rings [3].
Certain options found in per-policy conf sections will
override their equivalents that may be set in the
[app:proxy-server] section. Currently the options
handled that way are:
sorting_method
read_affinity
write_affinity
write_affinity_node_count
For example:
[proxy-server:policy:0]
sorting_method = affinity
read_affinity = r1=100
write_affinity = r1
write_affinity_node_count = 1 * replicas
The corresponding attributes of the proxy-server Application
are now available from instances of an OverrideConf object
that is obtained from Application.get_policy_options(policy).
[1] Related-Change: I9104fc789ba85ab3ab5ccd34096125b482821389
[2] Related-Change: Idd155401982a2c48110c30b480966a863f6bd305
[3] Related-Change: I0d8928b55020592f8e75321d1f7678688301d797
Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Change-Id: I3f718f425f525baa80045ba067950c752bcaaefc
We've got these lovely __enter__ and __exit__ methods; let's use them!
Note that this also changes how we patch classes' setUp methods so we
don't set self._orig_POLICIES when the class is already patched. I
hope this may fix some sporadic failures that include tracebacks
that look like
proxy ERROR: ERROR 500 Traceback (most recent call last):
File ".../swift/obj/server.py", line 1105, in __call__
res = getattr(self, req.method)(req)
File ".../swift/common/utils.py", line 1626, in _timing_stats
resp = func(ctrl, *args, **kwargs)
File ".../swift/obj/server.py", line 880, in GET
policy=policy, frag_prefs=frag_prefs)
File ".../swift/obj/server.py", line 211, in get_diskfile
return self._diskfile_router[policy].get_diskfile(
File ".../swift/obj/diskfile.py", line 555, in __getitem__
return self.policy_to_manager[policy]
KeyError: ECStoragePolicy(...)
... and try to unpatch more gracefully with TestCase.addCleanup
Change-Id: Iaa3d42ec21758b0707155878a645e665aa36696c
Add entries for these options in the deployment guide and
make the text in proxy-server.conf-sample and man page
consistent.
Change-Id: I5854ddb3e5864ddbeaf9ac2c930bfafdb47517c3
This is a follow up for https://review.openstack.org/#/c/441921
all of this patch is for minor fixes, I found in my self-review.
Change-Id: Ib3a1dc983c3da69dea592114e25a5047ec91a2b9
Previously, we were testing that a 254 (!?) character name would be valid
when the maximum configured is 500. Now we'll test that 500 character
names are valid.
While we're at it, stop patching self.test_check. It was unnecessary,
and we were doing it badly.
Change-Id: Ia604fa7b809a97fbce176c82606af73cdb92828c
* Adds a composite_builder module which provides the functionality to
build a composite ring from a number of component ring builders.
* Add id to RingBuilder to differentiate rings in composite.
A RingBuilder now gets a UUID when it is saved to file if
it does not already have one. A RingBuilder loaded from
file does NOT get a UUID assigned unless it was previously persisted in
the file. This forces users to explicitly assign an id to
existing ring builders by saving the state back to file.
The UUID is included in first line of the output from:
swift-ring-builder <builder-file>
Background:
This is another implementation for Composite Ring [1]
to enable better dispersion for global erasure coded cluster.
The most significant difference from the related-change [1] is that this
solution attempts to solve the problem as an offline tool rather than
dynamic compositing on the running servers. Due to the change, we gain
advantages such as:
- Less code and being simple
- No complex state validation on the running server
- Easy deployments with an offline tool
This patch does not provide a command line utility for managing
composite rings. The interface for such a tool is still under
discussion; this patch provides the enabling functionality first.
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
[1] Related-Change: I80ef36d3ac4d4b7c97a1d034b7fc8e0dc2214d16
Change-Id: I0d8928b55020592f8e75321d1f7678688301d797
The patch that introduced this (with my own help, no less),
also pasted these in tests, but someone else has already
cleaned that up.
Change-Id: I87c0a110831b5b5f75cb6fde1badec338f2150a9
cname_lookup and domain_remap are currently not catching redirections
(eg: staticweb). This behavior makes the domain to change when a call
through cname_lookup and domain_remap end up being redirected. Example:
commit fixes it.
- original request: http://mysite.com/subdir
- redirected: http://cont.acct.storage.domain.com/v1/AUTH_acct/cont/subdir/
- expected: http://mysite.com/subdir/
This patch is fixing this.
Closes-Bug: #1190625
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Change-Id: I67f642b8b070bc21e7760477d0a1e3b902ba7896
domain_remap strip all starting/ending slashes. This behavior does not
allow to access objects starting or ending with slash. It is also
impacting staticweb middleware as staticweb tries to redirect
pseudo-directory to an URL with an ending slash, but as domain_remap
strip it, it goes to an infinite loop.
With this commit, the path of the request is passed as-is when
reconstructing the new request path. Example
http://www.example.com//obj/ was previously rewritten to
http://storage.example.com/v1/AUTH_abc/container/obj. It is now
rewritten to http://storage.example.com/v1/AUTH_abc/container//obj/
Closes-Bug: #1682293
Co-Authored-By: Christian Schwede <cschwede@redhat.com>
Change-Id: I1ef6b8752183d27103a3b0e720edcb4ce06fb837
Recently out gate started blowing up intermittently with a strange
case of ports mixed up. Sometimes a functional tests tries to
authorize on a port that's clearly an object server port, and
the like. As it turns out, eventlet developers added an unavoidable
SO_REUSEPORT into listen(), which makes listen(("localhost",0)
to reuse ports.
There's an issue about it:
https://github.com/eventlet/eventlet/issues/411
This patch is working around the problem while eventlet people
consider the issue.
Change-Id: I67522909f96495a6a30e1acdb79835dce2189549
We've seen some failures in the gate like
==============================
Failed 1 tests - output below:
==============================
setUpModule (test.functional.test_account)
------------------------------------------
Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):
File "test/functional/test_account.py", line 33, in setUpModule
tf.setup_package()
File "test/functional/__init__.py", line 716, in setup_package
mem_object_server if in_mem_obj else object_server))
File "test/functional/__init__.py", line 621, in in_process_setup
create_account(AUTH_test)
File "test/functional/__init__.py", line 619, in create_account
assert(resp.status == 201)
AssertionError
...which aren't terribly useful in figuring out what went wrong.
Change-Id: I3cd31bb480dc8508828fe21416bfae33bc0985b7
