24 Commits

Author SHA1 Message Date
Jenkins
067335a6e7 Merge "Add dispersion report flags to limit reports" 2012-12-10 16:49:33 +00:00
Florian Hines
e474dfb720 Add dispersion report flags to limit reports
- Add two optional flags that let you limit swift-dispersion-report to only
reporting on containers OR objects.
- Also make dispersion.conf and swift-dispersion-report manpages
  current.

DocImpact

Change-Id: Iad56133cad261241db27d0e2103098e3c2f3c245
2012-12-09 18:20:08 -06:00
Darrell Bishop
b8e3e9e1c2 Allow optional, temporary healthcheck failure.
A deployer may want to remove a Swift node from a load balancer for
maintenance or upgrade.  This patch provides an optional mechanism for
this.  The healthcheck filter config can specify "disable_path" which is
a filesystem path.  If a file is present at that location, the
healthcheck middleware returns a 503 with a body of "DISABLED BY FILE".

So a deployer can configure "disable_path" and then touch that
filesystem path, wait for the proxy to be removed from the load balancer
pool, perform maintenance/upgrade, and then remove the "disable_path"
file.

Also cleaned up the conf file man pages a bit.

Change-Id: I1759c78c74910a54c720f298d4d8e6fa57a4dab4
2012-12-04 09:14:27 -08:00
gholt
dcc89e3ad6 dispersion report option to output missing parts
Adds a -p option to swift-dispersion-report that will output the
partitions missing copies to standard error. Another thing we've been
meaning to add for forever. It's useful when you want to do some
further research on whether a partition truly has fewer copies or if
they're just somewhere else in the cluster due to a ring change, for
instance.

DocImpact

Change-Id: I5e47aa5818483ecc34b39ef6f8cd83ad312ed9a0
2012-10-30 17:40:42 +00:00
Vincent Untz
e1ff51c045 Do not use pickle for serialization in memcache, but JSON
We don't want to use pickle as it can execute arbitrary code. JSON is
safer. However, note that it supports serialization for only some
specific subset of object types; this should be enough for what we need,
though.

To avoid issues on upgrades (unability to read pickled values, and cache
poisoning for old servers not understanding JSON), we add a
memcache_serialization_support configuration option, with the following
values:

 0 = older, insecure pickle serialization
 1 = json serialization but pickles can still be read (still insecure)
 2 = json serialization only (secure and the default)

To avoid an instant full cache flush, existing installations should
upgrade with 0, then set to 1 and reload, then after some time (24
hours) set to 2 and reload. Support for 0 and 1 will be removed in
future versions.

Part of bug 1006414.

Change-Id: Id7d6d547b103b4f23ebf5be98b88f09ec6027ce4
2012-08-03 16:22:21 +02:00
Vincent Untz
faff4ae769 Forbid substrings based on a regexp in name_filter middleware
In comments from https://review.openstack.org/8798 it was raised that it
might make sense to forbid some substrings in the name_filter
middleware.

There is now a new forbidden_regexp option for the name_filter
middleware to specify which substrings to forbid. The default is
"/\./|/\.\./|/\.$|/\.\.$" (or in a non-regexp language: the /./ and /../
substrings as well as strings ending with /. or /..).

This can be useful for extra paranoia to avoid directory traversals
(bug 1005908), or for more general filtering.

Change-Id: I39bf2de45b9dc7d3ca4d350d24b3f2276e958a62
DocImpact: new forbidden_regexp option for the name_filter middleware
2012-07-19 14:13:47 +02:00
Jenkins
501a3e4929 Merge "Remove ambiguity in memcache_servers documentation" 2012-07-03 16:37:24 +00:00
Jenkins
c16b70bb3d Merge "Remove hard coded man page paths." 2012-06-28 15:31:21 +00:00
Vincent Untz
1125368624 Remove ambiguity in memcache_servers documentation
The documentation could be understood like the following:
memcache_servers from memcache.conf is always used if set, even if
memcache_servers in proxy-server.conf is set.

This is clearly not the case, as proxy-server.conf has a higher priority
if it memcache_servers is set there.

Change-Id: I967c7e80796a0e296c5c65bd097df1669d16203e
2012-06-28 16:09:10 +02:00
Jenkins
57008e553a Merge "Patch for Swift Solaris (Illumos) compability." 2012-06-27 16:41:43 +00:00
Dan Prince
69fd05f439 Remove hard coded man page paths.
Fixes LP Bug #1016656.

Change-Id: I22455d27cb82830dabd4fb0677da26915a60eb85
2012-06-22 13:05:53 -04:00
Victor Rodionov
13e4de1899 Patch for Swift Solaris (Illumos) compability.
* Add new configuration option log_address.

Change-Id: I636bd4116687629c997b70a0d804b7ed4bc46032
2012-06-19 15:38:56 +04:00
Greg Lange
6987d222de Added config options for rate limiting of large object downloads.
bug 786197

Change-Id: Idd73986ca4550b20dff340df4acd12f3bfc01075
2012-06-07 20:19:31 +00:00
Florian Hines
ccb6334c17 Expand recon middleware support
Expand recon middleware to include support for account and container
servers in addition to the existing object servers. Also add support
for retrieving recent information from auditors, replicators, and
updaters. In the case of certain checks (such as container auditors)
the stats returned are only for the most recent path processed.

The middleware has also been refactored and should now also handle
errors better in cases where stats are unavailable.

While new check's have been added the output from pre-existing
check's has not changed. This should allow existing 3rd party
utilities such as the Swift ZenPack to continue to function.

Change-Id: Ib9893a77b9b8a2f03179f2a73639bc4a6e264df7
2012-05-24 14:50:00 -05:00
John Dickinson
1e90b61076 Re-add cname lookup and domain remap middleware
Revert "removed cname lookup middleware"

This reverts commit b47bcf19e41e862ca84d77a7b8843f836e084b6a.

Revert "removed domain remap middleware"

This reverts commit 317cf868bdf66dbc17c02d4ca383efafa5e2f229.

Change-Id: I260498d555c93b28896ace48a6f0e96701cbcc38
2012-05-21 20:02:04 -05:00
gholt
1c3b75c291 Reverted the pulling out of various middleware:
RateLimit
StaticWeb
TempURL/FormPOST

Change-Id: I988e93e6f4aacb817a2e354d43a04e47516fdf88
2012-05-16 21:25:10 +00:00
John Dickinson
b47bcf19e4 removed cname lookup middleware
The code has moved to https://github.com/notmyname/swift-cnamelookup.

For current users of cname lookup, this will require installing the new
package and changing the "use" line of the cname lookup conf section's
to:

[filter:cname_lookup]
use = egg:swift_cnamelookup#swift_cnamelookup

And then 'swift-init proxy reload'.

Change-Id: If622486ddb04a53251244c9840aa3cfe72168fc5
2012-05-09 14:00:51 -05:00
gholt
3f00c1a630 Pulled out Rate Limit middleware
Rate Limit middleware is now at
http://dpgoetz.github.com/swift-ratelimit/

For current users of Rate Limit, this will require installing the new
package and changing the "use" line of the ratelimit conf section to:

[filter:ratelimit]
use = egg:swiftratelimit#middleware

And then 'swift-init proxy reload'.

Change-Id: I2ab774e9cee9fba4103c1be4bea6d52d1adb29f7
2012-05-09 18:44:45 +00:00
John Dickinson
7dfbd785b0 removed domain remap middleware
The code has moved to https://github.com/notmyname/swift-domainremap.

For current users of domain remap, this will require installing the new
package and changing the "use" line of the domain remap conf section's
to:

[filter:domain_remap]
use = egg:swift_domainremap#swift_domainremap

And then 'swift-init proxy reload'.

Change-Id: I710caf9b991f9d37df36b826ae4338086d0ec36d
2012-05-08 21:30:35 -05:00
gholt
c0532a6ef2 Pulled out TempURL/FormPOST
TempURL/FormPOST is now at http://gholt.github.com/swift-tempurl/

For current users of TempURL/FormPOST, this will require installing the new
package and changing the "use" line of the tempurl and formpost conf section's
to:

[filter:tempurl]
use = egg:swifttempurl#tempurl

[filter:formpost]
use = egg:swifttempurl#formpost

And then 'swift-init proxy reload'.

Change-Id: I5bddf7f9e09ee07815530a41c46ff901fc21b447
2012-05-08 20:20:29 +00:00
gholt
7dde909621 Pulled StaticWeb out to separate project
StaticWeb is now at http://gholt.github.com/swift-staticweb/

For current users of StaticWeb, this will require installing the new
package and changing the "use" line of the staticweb filter conf
section to:

use = egg:swiftstaticweb#middleware

And then 'swift-init proxy reload'.

Change-Id: Iab32adb5927698a667c5c6d6a572c44ca23414eb
2012-05-05 00:40:33 +00:00
Marcelo Martins
6a9d5f921d Appending some new information to proxy, account and container config manpages. Adding object-expirer files and swift-orphans as well. Also making some minor modification on the other files to stop lithian from complaining during package creation checks. Adding name to Authors file.
Change-Id: Id68231ced070b27d1f7fb5a230d345694632e7dd
2012-04-10 14:52:52 -05:00
Florian Hines
5e4127ae2a Add json output option to swift-dispersion-report
Add's the configuration file option "dump_json" or command line
options [-j|--dump-json] to have swift-dispersion-report output
the report in json format. This allows the dispersion report to
be more easily consumed elsewhere.

There's also a few pep8 fixes and removal of unused imports.

Change-Id: I2374311ccbef43e6bbae24665c9584e60f3da173
2012-02-29 04:24:54 +00:00
Marcelo Martins
0b0785e984 changing config files to be on section 5 and also running spellchecks against all files
"Also adding the new swift-recon and swift-ring-builder manpages to this set"
"Adding new manpages for configuration files and also making changes according to previous review suggestions"
"removing the Author line from the manpages according to suggestions"

Change-Id: I256d2b2851b55a379b59011894f214bf55ba7da9
2011-11-08 21:09:42 -06:00