Swift recon introduced a new key for storage policies, and the CLI expected this
key in the server response. However, if one updates the CLI but not yet the
server an exception will be raised, because there is no default value and no
check if the key is included in the response.
This change checks if the policies key is included in the response and updates
one test to ensure backward compability.
Closes-Bug: 1453599
Change-Id: I7c7a90f9933bec2ab45595df9dc600a6cba65666
This is a follow-on from a previous commit which added recon info
for swift-drive-audit (https://review.openstack.org/#/c/122468/).
Here, the "--drievaudit" option is added to swift-recon tool. This
feature gives the statistics for the system-wide drive errors flagged
by swift-drive-audit. An example of the output is as follows:
(verbose mode)
swift-recon --driveaudit -v
===============================================================================
--> Starting reconnaissance on 5 hosts
===============================================================================
[2015-03-11 17:13:39] Checking drive-audit errors
-> http://1.2.3.4:6000/recon/driveaudit: {'drive_audit_errors': 14}
-> http://1.2.3.5:6000/recon/driveaudit: {'drive_audit_errors': 0}
-> http://1.2.3.6:6000/recon/driveaudit: {'drive_audit_errors': 37}
-> http://1.2.3.7:6000/recon/driveaudit: {'drive_audit_errors': 101}
-> http://1.2.3.8:6000/recon/driveaudit: {'drive_audit_errors': 0}
[drive_audit_errors] low: 0, high: 101, avg: 30.4, total: 152, Failed: 0.0%, no_result: 0, reported: 5
===============================================================================
Change-Id: Ia16c52a9d613eeb3de1a5a428d88dd1233631912
This patch validates the server end points on the ring. And also generates
a report on issues found.
Change-Id: I913799a35d5c9178164021cfb7fcb448141b058b
After the release of Swift ver. 2.0.0, some recon responses do not
show each policy's information yet. To make things worse, some recon
results only count on policy-0's score, therefore the total is not
shown in the recon results.
This patch makes the count of quarantined files policy-aware for recon
requests. Suppose a number of quarantined objects for policy-0 is 2
and a number for policy-1 is 3, recon sums up every policy's amount
and shows information for each policy as follows.
$ curl http://<host>:<port>/recon/quarantined
{"accounts": 0, "containers": 0, "objects": 5, "policies": {"0":
{"objects": 2}, "1": {"objects": 3}}}
Moreover, this patch adds stats for each policy in CLI output.
Change-Id: I07217c635f6fc4ea809ddbc3d859c4e81c4fde37
Related-Bug: 1375327
Related-Bug: 1375332
This patch fixes the unit tests to remove the temporary directories
created during run of unit tests. Some of unit tests did not tear down
correctly, whatever it had set it up for running. This would over period
of time bloat up the tmp directory. As on date, there were around 49 tmp
directories left uncleared per round of unit tests. This patch fixes it.
Change-Id: If591375ca9cc87d52c7c9c6dc16c9fb4b49e99fc
1) Added comment for H231, which we were already enforcing. H231
is for Python 3.x compatible except statements.
2) Added check for H201, which we were enforcing in reviews
but waiting on hacking checks to be updated. H201 is for bare
except statements, and the update in upstream hacking is to
support the " # noqa" flag on it.
The H201 check catches some existing bare excepts that are fixed.
Change-Id: I68638aa9ea925ef62f9035a426548c2c804911a8
The option "-r" is already used, thus only "--region" is used to specify
filter by region.
Change-Id: If769f2f3191c202933b03b48fe0f22b7c94a4dd6
Closes-Bug: 1369583
Recon middleware returns object ring file MD5 sums; this patch
updates it to include other object files that may be present
because of Storage Policies. Also adds unit test coverage for
the MD5 reporting function which previously had none.
The recon script will now check all rings the server responds with
match the on-disk md5's regardless of server-type; including any
storage policy object rings.
Note the small change to the ring save method, needed to
stimulate the right code paths in 2.6 and 2.7 versions of
gzip to enable testing of ring MD5 sums.
DocImpact
Implements: blueprint storage-policies
Change-Id: I01efd2999d6d9c57ee8693ac3a6236ace17c5566
In test_ptime() 2 uses of the former failed to assert with bogus
values used in the assert. Using assert_called_with() instead
correctly performs the assertion.
Change-Id: Idbd918f8e8d3ec5a4110725b949710fb54b4ba9a
I've seen several folks recently have problems with their Swift
clusters because they had different hash prefixes on different
nodes. Let's help them out by having recon check that.
Note that MD5-equality is stronger than what we need (which is
ConfigParser-equality for a particular set of keys), but this way we
don't expose the secret hash prefix and suffix across the internal
network, just the MD5 checksum of the file containing them.
Change-Id: I3af984ee45947345891b3c596a88e3464f178cc7
Fix also minor bug in zone filtering when zone set to 0.
Moved bin/swift-recon to swift/cli/recon.py, which makes
it possible to import it without using some scary hacks.
bin/swift-recon is now created by setup.py install.
Closes-Bug: #1261692
Change-Id: Id0729991c8ece73604467480dbf93fec7d8eb196