This change cleans up test/unit/obj/test_replicator.py's imports
to use only 1 version of multiline import syntaxes (' \' vs '()').
I don't really mind which, but we should be consistant, at least
in the same file.
This is a follow up for patch 215857.
Change-Id: Ie2d328c25865b19092c493981a803ee246a9d7a5
Previously, account listings that used the delimiter query param could
omit some containers if they ended with the character that follows the
delimiter.
See If196e3075612b121ef8da4a9128167d00a248c27 for the corresponding fix
for container listings.
Change-Id: I57fcb97e51f653f5f4e306a632fcb3a0fb148c4e
Under some concurrency the object-replicator could potentially send the
wrong X-Backed-Storage-Policy-Index header to it's partner nodes during
replication if there were multiple storage policies on the same node
because of a race where multiple jobs being processed concurrently would
mutate some shared state on the ObjectReplicator instance.
Instead of using shared stated on the ObjectReplicator instance when
mutating the default headers send with REPLICATION requests each job
will copy them into a local where they can safely be updated.
Change-Id: I5522db57af7e308b1f9d4181f14ea14e386a71fd
When using fast-post and POST (i.e. metadata update) is requested to
a SLO manifest files, current Swift drops the 'X-Static-Large-Object'
header from the existing metadata. It results in breaking the SLO
state because the manifest missing the 'X-Static-Large-Object' metadata
will be maintained as a normal files.
This patch fixes object-server to keep the existing
'X-Static-Large-Object' flag and then keep the SLO state.
Change-Id: Ib1eb569071372c322dd105c52baeeb094003291e
Closes-bug: #1453807
SAIO Configuration and documentation changes enabling to run the
container sync probe test by default
Change-Id: Iccf59533d0d4fe72549d318339ab125d04dde006
Related-Bug: #1476623
When the object-replicator encounters handoffs_first and
handoff_delete options as enabled it should emit a log
warning indicating that it should be changed back to the
default before the next "normal" rebalance.
Closes-Bug: #1457262
Change-Id: If9dc2796c18ed3cf13da920831e2d5c2ae9f12a0
The proxy was trying to pop a byterange off a Range header that didn't
contain syntactically-valid byteranges. This worked about as well as
you'd expect. Now we detect the bogus value and remove the header
entirely.
Change-Id: I24b92f900d33ec79880c7db2870378489d5a6810
Fix the pep8 warning H702 "Formatting operation should be outside of
localization method call".
For the logger, pass parameters as indexed parameters instead of using
the string str%args operator, the logger is more reliable in case of
formatting error.
Change-Id: If418bc155f6a6c0a00f63e3d87ebe4addf4aae55
The TestCase.assert_() has been deprecated in Python 2.7. Replace it
with assertTrue() or even better methods (assertIn, assertNotIn,
assertIsInstance) which provide better error messages.
Change-Id: I21c730351470031a2dabe5238693095eabdb8964
Keep HTTP_X_USER_ID and HTTP_X_PROJECT_ID to be available as
user_id and project_id in storage.objects.outgoing.bytes in
ceilometer when downloading a multipart object.
Change-Id: I0f4734f021e5d6e84d48ed9bebeb321d7a9590ad
Closes-Bug: #1477283
Improve the 'first_contribution_swift' by adding instructions for how to
rebuild swift following a rebase.
Change-Id: If5c91dc4e1e8d1712bbd8b326c675967fb4b8c15
This patch makes the count of object replication failure in recon.
And "failure_nodes" is added to Account Replicator and
Container Replicator.
Recon shows the count of object repliction failure as follows:
$ curl http://<ip>:<port>/recon/replication/object
{
"replication_last": 1416334368.60865,
"replication_stats": {
"attempted": 13346,
"failure": 870,
"failure_nodes": {
"192.168.0.1": {"sdb1": 3},
"192.168.0.2": {"sdb1": 851,
"sdc1": 1,
"sdd1": 8},
"192.168.0.3": {"sdb1": 3,
"sdc1": 4}
},
"hashmatch": 0,
"remove": 0,
"rsync": 0,
"start": 1416354240.9761429,
"success": 1908
},
"replication_time": 2316.5563162644703,
"object_replication_last": 1416334368.60865,
"object_replication_time": 2316.5563162644703
}
Note that 'object_replication_last' and 'object_replication_time' are
considered to be transitional and will be removed in the subsequent
releases. Use 'replication_last' and 'replication_time' instead.
Additionaly this patch adds the count in swift-recon and it will be
showed as follows:
$ swift-recon object -r
========================================================================
=======
--> Starting reconnaissance on 4 hosts
========================================================================
=======
[2014-11-27 16:14:09] Checking on replication
[replication_failure] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%,
no_result: 0, reported: 4
[replication_success] low: 3, high: 3, avg: 3.0, total: 12,
Failed: 0.0%, no_result: 0, reported: 4
[replication_time] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%,
no_result: 0, reported: 4
[replication_attempted] low: 1, high: 1, avg: 1.0, total: 4,
Failed: 0.0%, no_result: 0, reported: 4
Oldest completion was 2014-11-27 16:09:45 (4 minutes ago) by
192.168.0.4:6002.
Most recent completion was 2014-11-27 16:14:19 (-10 seconds ago) by
192.168.0.1:6002.
========================================================================
=======
In case there is a cluster which has servers, a server runs with this
patch and the other servers run without this patch. If swift-recon
executes on the server which runs with this patch, there are unnecessary
information on the output such as [failure], [success] and [attempted].
Because other servers which run without this patch are not able to
send a response with information that this patch needs.
Therefore once you apply this patch, you also apply this patch to other
servers before you execute swift-recon.
DocImpact
Change-Id: Iecd33655ae2568482833131f422679996c374d78
Co-Authored-By: Kenichiro Matsuda <matsuda_kenichi@jp.fujitsu.com>
Co-Authored-By: Brian Cline <bcline@softlayer.com>
Implements: blueprint enable-object-replication-failure-in-recon
Sometimes the given argument is internally altered and another filename is used
without a note to the operator. Even worse, a given .ring.gz filename is
sometimes written out as builder file, without updating the corresponding
.builder file.
There is already a method to parse the given argv and return the name of the
builder and ring file. However, it's rarely used and no warning is given to the
user if it is altered. This patch uses the already parsed builder and ring file
name instead of argv[1], and also adds a note to the user if the used filename
is differently to the one given as argument.
Closes-Bug: 1482096
Change-Id: I2f8ef23aeab8b07caaa799f7dcd57e684b4b2ad2
Increase the .durable quorum from 2 to "parity + 1" to guarantee
that we will never fail to rebuild an object. Otherwise, with
low durable responses back (< parity + 1), the putter objects
return with failed attribute set to true, thereby failing the
rebuild of fragments for an object.
Change-Id: I80d666f61273e589d0990baa78fd657b3470785d
Closes-Bug: 1484565
When this code lived in the proxy, it was protected by an "except
HTTPException" clause in proxy.Application.handle_request(). When it
moved to middleware, it lost that, and then things like
constraints.check_name_format that raised HTTPException would cause
500s. The HTTPException would make it all the way out to catch_errors
and get translated to a 500.
This commit just wraps a couple try/excepts around the bits in
versioned writes that can raise HTTPException. I tried to make it use
wsgify so I could get that for free, but that wound up being a real
pain because env/start_response are plumbed through pretty much the
whole versioned-writes middleware.
Closes-Bug: 1483705
Change-Id: Ife165bf709e64f313ed07c779b21914045e51f25