16 Commits

Author SHA1 Message Date
Christian Schwede
b4c1d73ad5 Make swift-recon compatible for servers without storage policies
Swift recon introduced a new key for storage policies, and the CLI expected this
key in the server response. However, if one updates the CLI but not yet the
server an exception will be raised, because there is no default value and no
check if the key is included in the response.

This change checks if the policies key is included in the response and updates
one test to ensure backward compability.

Closes-Bug: 1453599

Change-Id: I7c7a90f9933bec2ab45595df9dc600a6cba65666
2015-06-01 07:00:39 +00:00
Christian Schwede
663ccd4e7a More tests for swift recon
Change-Id: I8d568c0f6fbe1c01d97491740aebf299deb63732
2015-04-15 06:36:06 +00:00
Christian Schwede
e16df14a73 Add test for swift_recon.disk_usage
Change-Id: I4cab7aa6df3f0e1933e52ee5dbbb829f30604f10
2015-04-14 16:00:37 -07:00
Lorcan
0a46793662 Add swift-recon feature to track swift-drive-audit error count
This is a follow-on from a previous commit which added recon info
for swift-drive-audit (https://review.openstack.org/#/c/122468/).

Here, the "--drievaudit" option is added to swift-recon tool. This
feature gives the statistics for the system-wide drive errors flagged
by swift-drive-audit. An example of the output is as follows:
(verbose mode)

swift-recon --driveaudit -v
===============================================================================
--> Starting reconnaissance on 5 hosts
===============================================================================
[2015-03-11 17:13:39] Checking drive-audit errors
-> http://1.2.3.4:6000/recon/driveaudit: {'drive_audit_errors': 14}
-> http://1.2.3.5:6000/recon/driveaudit: {'drive_audit_errors': 0}
-> http://1.2.3.6:6000/recon/driveaudit: {'drive_audit_errors': 37}
-> http://1.2.3.7:6000/recon/driveaudit: {'drive_audit_errors': 101}
-> http://1.2.3.8:6000/recon/driveaudit: {'drive_audit_errors': 0}
[drive_audit_errors] low: 0, high: 101, avg: 30.4, total: 152, Failed: 0.0%, no_result: 0, reported: 5
===============================================================================

Change-Id: Ia16c52a9d613eeb3de1a5a428d88dd1233631912
2015-03-23 11:38:32 +00:00
Mahati Chamarthy
a248a5c09e Ring checker in swift-recon
This patch validates the server end points on the ring. And also generates
a report on issues found.

Change-Id: I913799a35d5c9178164021cfb7fcb448141b058b
2015-02-26 01:26:02 +05:30
Daisuke Morita
f8fa1a9234 Show each policy's information on quarantined files in recon
After the release of Swift ver. 2.0.0, some recon responses do not
show each policy's information yet. To make things worse, some recon
results only count on policy-0's score, therefore the total is not
shown in the recon results.

This patch makes the count of quarantined files policy-aware for recon
requests. Suppose a number of quarantined objects for policy-0 is 2
and a number for policy-1 is 3, recon sums up every policy's amount
and shows information for each policy as follows.

$ curl http://<host>:<port>/recon/quarantined
{"accounts": 0, "containers": 0, "objects": 5, "policies": {"0":
{"objects": 2}, "1": {"objects": 3}}}

Moreover, this patch adds stats for each policy in CLI output.

Change-Id: I07217c635f6fc4ea809ddbc3d859c4e81c4fde37
Related-Bug: 1375327
Related-Bug: 1375332
2015-01-20 18:42:20 +09:00
Jenkins
513eeb80d7 Merge "updated hacking rules" 2014-10-03 01:47:40 +00:00
Keshava Bharadwaj
0f93fff46a Fixes unit tests to clean up temporary directories
This patch fixes the unit tests to remove the temporary directories
created during run of unit tests. Some of unit tests did not tear down
correctly, whatever it had set it up for running. This would over period
of time bloat up the tmp directory. As on date, there were around 49 tmp
directories left uncleared per round of unit tests. This patch fixes it.

Change-Id: If591375ca9cc87d52c7c9c6dc16c9fb4b49e99fc
2014-09-26 22:39:48 +05:30
John Dickinson
e567722c4e updated hacking rules
1) Added comment for H231, which we were already enforcing. H231
is for Python 3.x compatible except statements.

2) Added check for H201, which we were enforcing in reviews
but waiting on hacking checks to be updated. H201 is for bare
except statements, and the update in upstream hacking is to
support the "  # noqa" flag on it.

The H201 check catches some existing bare excepts that are fixed.

Change-Id: I68638aa9ea925ef62f9035a426548c2c804911a8
2014-09-25 11:04:31 -07:00
Christian Schwede
fc5cee5f05 Allow filtering by region in swift-recon
The option "-r" is already used, thus only "--region" is used to specify
filter by region.

Change-Id: If769f2f3191c202933b03b48fe0f22b7c94a4dd6
Closes-Bug: 1369583
2014-09-15 17:31:16 +00:00
Christian Schwede
e21703ff7b Add test for swift-recon --auditor
Related-Bug: 1329785
Change-Id: I47cecd8a4cd55ca75c2a51153be7bb61c27d0ea0
2014-06-25 14:07:52 +00:00
Jenkins
570e50fe22 Merge "Change assertCalledWith to assert_called_with" 2014-06-24 05:04:06 +00:00
Paul Luse
8326dc9f2a Add Storage Policy Support to Recon Middleware
Recon middleware returns object ring file MD5 sums; this patch
updates it to include other object files that may be present
because of Storage Policies.  Also adds unit test coverage for
the MD5 reporting function which previously had none.

The recon script will now check all rings the server responds with
match the on-disk md5's regardless of server-type; including any
storage policy object rings.

Note the small change to the ring save method, needed to
stimulate the right code paths in 2.6 and 2.7 versions of
gzip to enable testing of ring MD5 sums.

DocImpact
Implements: blueprint storage-policies
Change-Id: I01efd2999d6d9c57ee8693ac3a6236ace17c5566
2014-06-18 21:09:54 -07:00
Paul Luse
d32169c9c8 Change assertCalledWith to assert_called_with
In test_ptime() 2 uses of the former failed to assert with bogus
values used in the assert.  Using assert_called_with() instead
correctly performs the assertion.

Change-Id: Idbd918f8e8d3ec5a4110725b949710fb54b4ba9a
2014-06-10 08:08:13 -07:00
Samuel Merritt
31dac18625 Check swift.conf MD5 with recon
I've seen several folks recently have problems with their Swift
clusters because they had different hash prefixes on different
nodes. Let's help them out by having recon check that.

Note that MD5-equality is stronger than what we need (which is
ConfigParser-equality for a particular set of keys), but this way we
don't expose the secret hash prefix and suffix across the internal
network, just the MD5 checksum of the file containing them.

Change-Id: I3af984ee45947345891b3c596a88e3464f178cc7
2014-04-10 14:08:27 -07:00
Christian Schwede
cd4b4da8b6 Add some tests for bin/swift-recon
Fix also minor bug in zone filtering when zone set to 0.

Moved bin/swift-recon to swift/cli/recon.py, which makes
it possible to import it without using some scary hacks.
bin/swift-recon is now created by setup.py install.

Closes-Bug: #1261692
Change-Id: Id0729991c8ece73604467480dbf93fec7d8eb196
2014-01-31 15:34:37 +00:00