Used groff to recreate the errors. I believe all the issues
except `binary-without-manpage` are solved. Would like
confirmation from someone using Lintian.
Closes-Bug: #1210114
Change-Id: I533205c53efdb7cdf3645cc3e3dc487f9ee5640a
The swift-dispersion-populate and swift-dispersion-report tools now
accept a --insecure option.
Also, dispersion.conf now has a keystone_api_insecure option.
Default is obviously to use the secure path.
DocImpact
Change-Id: I4000352e547d9ce5b08ade54e0c886281caff891
Change the default value of wsgi workers from 1 to auto. The new default
value for workers in the proxy, container, account & object wsgi servers will
spawn as many workers per process as you have cpu cores.
This will not be ideal for some configurations, but it's much more likely to
produce a successful out of the box deployment.
Inspect the number of cpu_cores using python's multiprocessing when available.
Multiprocessing was added in python 2.6, but I know I've compiled python
without it before on accident. The cpu_count method seems to be pretty system
agnostic, but it says it can raise NotImplementedError or sometimes return 0.
Add a new utility method 'config_auto_int_value' to pull an integer out of the
config which has a dynamic default.
* drive by s/container/proxy/ in proxy-server.conf.5
* fix misplaced max_clients in *-server.conf-sample
* update doc/development_saio to force workers = 1
DocImpact
Change-Id: Ifa563d22952c902ab8cbe1d339ba385413c54e95
For systems with very large numbers of partitions, 1% dispersion
coverage may simply be too much/take too long. This fix allows <1
values to be used for dispersion_coverage.
DocImpact
Change-Id: I5ed35b69754d55a410e66e658b3854de57c7666b
The old format is still present and works just like it did before, so
your existing scripts won't break.
New format pros:
* it's readable even for Swift newcomers
* it's easy to extend
* it's familiar to anyone who's used a Unix command line
* we don't have to maintain the parser
New format cons:
* you can't add multiple devices in one go
Old format pros:
* you can add many devices with one command
* it's compact
Old format cons:
* it confuses newcomers
* "wait, is that zone dash IP colon port slash device, or zone slash
IP dash port colon meta underscore device?" Just try walking
someone through adding a device over voice chat.
* it's annoying to add new fields
Note that this only affects the command "swift-ring-builder
<builderfile> add". Other swift-ring-builder commands are unchanged.
DocImpact
Change-Id: I034b7f79eb6f4d81a5c4da193e1358741441c5b5
The new max_clients parameter allows one full control over the maximum
number of client requests that will be handled by a given worker for
any of the proxy, account, container or object servers.
Lowering the number of clients handled per worker, and raising the
number of workers can lessen the impact that a CPU intensive, or
blocking, request can have on other requests served by the same
worker.
If the maximum number of clients is set to one, then a given worker
will not perform another accept(2) call while processing, allowing
other workers a chance to process it.
DocImpact
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Change-Id: Ic01430f7a6c5ff48d7aa349dc86a5f8ac463a420
The trans_id_suffix (default is empty) would be appended to the swift transaction
id allowing one to easily figure out from which cluster that X-Trans-Id
belongs to. This is very useful when one is managing more than one swift
cluster. Also updated sample and manpage to reflect the changes.
Change-Id: Icdf63643e9c1bde36a9ef5e3f41ee9fb20e55f5d
When someone split away python-swiftclient, he left behind the
client manpage.
Apparently all we need now is to delete it from swift and
add it to python-swiftclient. Except that of course the releases
have to be synchronized somehow or else files conflict.
Change-Id: I8b5f7b5557c28818048d8941df36473dacfb1d57
Fix bug 1129760
Without speed limit, DB auditor will likely consume high CPU% on
storage node. That will highly impact the cluster's performance.
This patch adds two options for account/container auditor:
- containers_per_second: Maximum containers audited per second
- accounts_per_second: Maximum accounts audited per second
DocImpact
Change-Id: I9faa506438185a83ca77db4906969328624d015f
These are mostly cosmetic fixes for irritating imperfections:
- "separated with commas" was duplicated, leave just one
- extra whitespace here and there, man pages are not PEP8, drop
- weird extra commas, drop
- Fedora logs to /var/log/messages
- "drive is has failed", drop "is"
Change-Id: I5ceba2e61b16db4855d76c92cbc83663b9b2a0da
Replaced GA code for cross-domain tracking.
Patchset addresses reviewer's comments
and follows new guidance from Foundation:
http://wiki.openstack.org/Documentation/Copyright
Adds current year to each Sphinx-built page.
Addresses only the docs copyright attribution, not code files.
Change-Id: Ib90fd1c92c8fafce2db821bc2b17cef1377cfc1e
Seems we missed these references when committing
357b12dc2ba7b19c66196a573ccb2489d2104b93
DocImpact
Change-Id: Ia226ce1d63e52769bc067d50ec4704cea4e11c5c
If invoked as 'swift-ring-builder-safe' the directory containing the builder
file provided will be locked (via lock_parent_directory()). This provides a
small safe guard against multiple instances of the swift-ring-builder (or
other utilities that observe this lock) from attempting to write to or read
the builder/ring files while operations are in progress.
This is particularly useful in environments where ring management has been
automated (via Chef or custom solutions) but the operator still occasionally
needs to manually interact with the ring.
DocImpact
Change-Id: Ia362744a8151a91bfb586d01da582906726852e6
You can now give swift-init a -k N (or --kill-wait N) option to
override the default 15 second wait for a process to die after
sending it the die signal. This is useful for boxes that are awfully
slow for whatever reason.
DocImpact
Change-Id: I328ec254f6e0ee1cd423c1d062ba4c5331bd8337
- Add two optional flags that let you limit swift-dispersion-report to only
reporting on containers OR objects.
- Also make dispersion.conf and swift-dispersion-report manpages
current.
DocImpact
Change-Id: Iad56133cad261241db27d0e2103098e3c2f3c245
A deployer may want to remove a Swift node from a load balancer for
maintenance or upgrade. This patch provides an optional mechanism for
this. The healthcheck filter config can specify "disable_path" which is
a filesystem path. If a file is present at that location, the
healthcheck middleware returns a 503 with a body of "DISABLED BY FILE".
So a deployer can configure "disable_path" and then touch that
filesystem path, wait for the proxy to be removed from the load balancer
pool, perform maintenance/upgrade, and then remove the "disable_path"
file.
Also cleaned up the conf file man pages a bit.
Change-Id: I1759c78c74910a54c720f298d4d8e6fa57a4dab4
Adds a -p option to swift-dispersion-report that will output the
partitions missing copies to standard error. Another thing we've been
meaning to add for forever. It's useful when you want to do some
further research on whether a partition truly has fewer copies or if
they're just somewhere else in the cluster due to a ring change, for
instance.
DocImpact
Change-Id: I5e47aa5818483ecc34b39ef6f8cd83ad312ed9a0
We don't want to use pickle as it can execute arbitrary code. JSON is
safer. However, note that it supports serialization for only some
specific subset of object types; this should be enough for what we need,
though.
To avoid issues on upgrades (unability to read pickled values, and cache
poisoning for old servers not understanding JSON), we add a
memcache_serialization_support configuration option, with the following
values:
0 = older, insecure pickle serialization
1 = json serialization but pickles can still be read (still insecure)
2 = json serialization only (secure and the default)
To avoid an instant full cache flush, existing installations should
upgrade with 0, then set to 1 and reload, then after some time (24
hours) set to 2 and reload. Support for 0 and 1 will be removed in
future versions.
Part of bug 1006414.
Change-Id: Id7d6d547b103b4f23ebf5be98b88f09ec6027ce4
In comments from https://review.openstack.org/8798 it was raised that it
might make sense to forbid some substrings in the name_filter
middleware.
There is now a new forbidden_regexp option for the name_filter
middleware to specify which substrings to forbid. The default is
"/\./|/\.\./|/\.$|/\.\.$" (or in a non-regexp language: the /./ and /../
substrings as well as strings ending with /. or /..).
This can be useful for extra paranoia to avoid directory traversals
(bug 1005908), or for more general filtering.
Change-Id: I39bf2de45b9dc7d3ca4d350d24b3f2276e958a62
DocImpact: new forbidden_regexp option for the name_filter middleware
The documentation could be understood like the following:
memcache_servers from memcache.conf is always used if set, even if
memcache_servers in proxy-server.conf is set.
This is clearly not the case, as proxy-server.conf has a higher priority
if it memcache_servers is set there.
Change-Id: I967c7e80796a0e296c5c65bd097df1669d16203e
Expand recon middleware to include support for account and container
servers in addition to the existing object servers. Also add support
for retrieving recent information from auditors, replicators, and
updaters. In the case of certain checks (such as container auditors)
the stats returned are only for the most recent path processed.
The middleware has also been refactored and should now also handle
errors better in cases where stats are unavailable.
While new check's have been added the output from pre-existing
check's has not changed. This should allow existing 3rd party
utilities such as the Swift ZenPack to continue to function.
Change-Id: Ib9893a77b9b8a2f03179f2a73639bc4a6e264df7
The code has moved to https://github.com/notmyname/swift-cnamelookup.
For current users of cname lookup, this will require installing the new
package and changing the "use" line of the cname lookup conf section's
to:
[filter:cname_lookup]
use = egg:swift_cnamelookup#swift_cnamelookup
And then 'swift-init proxy reload'.
Change-Id: If622486ddb04a53251244c9840aa3cfe72168fc5
Rate Limit middleware is now at
http://dpgoetz.github.com/swift-ratelimit/
For current users of Rate Limit, this will require installing the new
package and changing the "use" line of the ratelimit conf section to:
[filter:ratelimit]
use = egg:swiftratelimit#middleware
And then 'swift-init proxy reload'.
Change-Id: I2ab774e9cee9fba4103c1be4bea6d52d1adb29f7
The code has moved to https://github.com/notmyname/swift-domainremap.
For current users of domain remap, this will require installing the new
package and changing the "use" line of the domain remap conf section's
to:
[filter:domain_remap]
use = egg:swift_domainremap#swift_domainremap
And then 'swift-init proxy reload'.
Change-Id: I710caf9b991f9d37df36b826ae4338086d0ec36d
TempURL/FormPOST is now at http://gholt.github.com/swift-tempurl/
For current users of TempURL/FormPOST, this will require installing the new
package and changing the "use" line of the tempurl and formpost conf section's
to:
[filter:tempurl]
use = egg:swifttempurl#tempurl
[filter:formpost]
use = egg:swifttempurl#formpost
And then 'swift-init proxy reload'.
Change-Id: I5bddf7f9e09ee07815530a41c46ff901fc21b447
StaticWeb is now at http://gholt.github.com/swift-staticweb/
For current users of StaticWeb, this will require installing the new
package and changing the "use" line of the staticweb filter conf
section to:
use = egg:swiftstaticweb#middleware
And then 'swift-init proxy reload'.
Change-Id: Iab32adb5927698a667c5c6d6a572c44ca23414eb
Add's the configuration file option "dump_json" or command line
options [-j|--dump-json] to have swift-dispersion-report output
the report in json format. This allows the dispersion report to
be more easily consumed elsewhere.
There's also a few pep8 fixes and removal of unused imports.
Change-Id: I2374311ccbef43e6bbae24665c9584e60f3da173
"Also adding the new swift-recon and swift-ring-builder manpages to this set"
"Adding new manpages for configuration files and also making changes according to previous review suggestions"
"removing the Author line from the manpages according to suggestions"
Change-Id: I256d2b2851b55a379b59011894f214bf55ba7da9