31 Commits

Author SHA1 Message Date
Jenkins
8b770aa55e Merge "Add config option to turn eventlet debug on/off" 2012-12-10 20:37:31 +00:00
Chuck Thier
4c6a354483 Add config option to turn eventlet debug on/off
By default, this will be turned off.  This will cause eventlet to not
print stack traces to stderr which can be very annoying on production
systems.  It is still recommended to turn it on for development or
debuging purposes.

DocImpact
Change-Id: I5e5b902d3d9ed85f784549e53f2ee2fc87cbe2e5
2012-12-10 10:22:09 -06:00
clayg
3a70112d03 Add config of server start timeouts for probetests
Currently the timeout for a wsgi server successfully binding to a port
and for a probetest background service to finish starting are hard coded
to 30 seconds.  While a reasonable default for most configurations, a
small virtualized environment may need a little more time in order for
probe tests to complete successfully.

This patch adds a 'bind_timeout' option to the DEFAULT section of the
main wsgi servers' config.  Also a new [probe_test] section and
'check_server_timeout' option to test.conf

DocImpact

Change-Id: Ibcaff153c7633bbf32e460fd9dbf04932eddb56f
2012-12-07 14:47:08 -08:00
Darrell Bishop
b8e3e9e1c2 Allow optional, temporary healthcheck failure.
A deployer may want to remove a Swift node from a load balancer for
maintenance or upgrade.  This patch provides an optional mechanism for
this.  The healthcheck filter config can specify "disable_path" which is
a filesystem path.  If a file is present at that location, the
healthcheck middleware returns a 503 with a body of "DISABLED BY FILE".

So a deployer can configure "disable_path" and then touch that
filesystem path, wait for the proxy to be removed from the load balancer
pool, perform maintenance/upgrade, and then remove the "disable_path"
file.

Also cleaned up the conf file man pages a bit.

Change-Id: I1759c78c74910a54c720f298d4d8e6fa57a4dab4
2012-12-04 09:14:27 -08:00
Florian Hines
92826d0602 add support for custom log handlers
Add a hook to get_logger to run custom functions to add custom log
handlers or the like.

Change-Id: Ib04b12939dcac7e4ad6453dea9795682044c6ae0
2012-10-05 15:56:34 -05:00
Darrell Bishop
4a2ae2b460 Upating proxy-server StatsD logging.
Removed many StatsD logging calls in proxy-server and added
swift-informant-style catch-all logging in the proxy-logger middleware.
Many errors previously rolled into the "proxy-server.<type>.errors"
counter will now appear broken down by response code and with timing
data at: "proxy-server.<type>.<verb>.<status>.timing".  Also, bytes
transferred (sum of in + out) will be at:
"proxy-server.<type>.<verb>.<status>.xfer".  The proxy-logging
middleware can get its StatsD config from standard vars in [DEFAULT] or
from access_log_statsd_* config vars in its config section.

Similarly to Swift Informant, request methods ("verbs") are filtered
using the new proxy-logging config var, "log_statsd_valid_http_methods"
which defaults to GET, HEAD, POST, PUT, DELETE, and COPY.  Requests with
methods not in this list use "BAD_METHOD" for <verb> in the metric name.
To avoid user error, access_log_statsd_valid_http_methods is also
accepted.

Previously, proxy-server metrics used "Account", "Container", and
"Object" for the <type>, but these are now all lowercase.

Updated the admin guide's StatsD docs to reflect the above changes and
also include the "proxy-server.<type>.handoff_count" and
"proxy-server.<type>.handoff_all_count" metrics.

The proxy server now saves off the original req.method and proxy_logging
will use this if it can (both for request logging and as the "<verb>" in
the statsd timing metric).  This fixes bug 1025433.

Removed some stale access_log_* related code in proxy/server.py.  Also
removed the BaseApplication/Application distinction as it's no longer
necessary.

Fixed up the sample config files a bit (logging lines, mostly).

Fixed typo in SAIO development guide.

Got proxy_logging.py test coverage to 100%.

Fixed proxy_logging.py for PEP8 v1.3.2.

Enhanced test.unit.FakeLogger to track more calls to enable testing
StatsD metric calls.

Change-Id: I45d94cb76450be96d66fcfab56359bdfdc3a2576
2012-08-29 16:08:30 -07:00
gholt
c509ac2371 Added ability to disable fallocate
Change-Id: Id8872c581ed23378a8e14cbf3bf049b5c0d21577
2012-08-29 19:57:26 +00:00
Victor Rodionov
13e4de1899 Patch for Swift Solaris (Illumos) compability.
* Add new configuration option log_address.

Change-Id: I636bd4116687629c997b70a0d804b7ed4bc46032
2012-06-19 15:38:56 +04:00
Florian Hines
ccb6334c17 Expand recon middleware support
Expand recon middleware to include support for account and container
servers in addition to the existing object servers. Also add support
for retrieving recent information from auditors, replicators, and
updaters. In the case of certain checks (such as container auditors)
the stats returned are only for the most recent path processed.

The middleware has also been refactored and should now also handle
errors better in cases where stats are unavailable.

While new check's have been added the output from pre-existing
check's has not changed. This should allow existing 3rd party
utilities such as the Swift ZenPack to continue to function.

Change-Id: Ib9893a77b9b8a2f03179f2a73639bc4a6e264df7
2012-05-24 14:50:00 -05:00
gholt
9eb797b099 !! Changed db_preallocation to False
Long explanation, but hopefully answers any questions.

We don't like changing the default behavior of Swift unless there's a
really good reason and, up until now, I've tried doing this with this
new db_preallocation setting.

For clusters with dedicated account/container servers that usually
have fewer disks overall but SSD for speed, having db_preallocation
on will gobble up disk space quite quickly and the fragmentation it's
designed to fight isn't that big a speed impact to SSDs anyway.

For clusters with account/container servers spread across all servers
along with object servers usually having standard disks for cost,
having db_preallocation off will cause very fragmented database files
impacting speed, sometimes dramatically.

Weighing these two negatives, it seems the second is the lesser evil.
The first can cause disks to fill up and disable the cluster. The
second will cause performance degradation, but the cluster will still
function.

Furthermore, if just one piece of code that touches all databases
runs with db_preallocation on, it's effectively on for the whole
cluster. We discovered this most recently when we finally configured
everything within the Swift codebase to have db_preallocation off,
only to find out Slogging didn't know about the new setting and so
ran with it on and starting filling up SSDs.

So that's why I'm proposing this change to the default behavior.

We will definitely need to post a prominent notice of this change
with the next release.

Change-Id: I48a43439264cff5d03c14ec8787f718ee44e78ea
2012-05-22 00:30:47 +00:00
Darrell Bishop
3d3ed34f44 Adding StatsD logging to Swift.
Documentation, including a list of metrics reported and their semantics,
is in the Admin Guide in a new section, "Reporting Metrics to StatsD".
An optional "metric prefix" may be configured which will be prepended to
every metric name sent to StatsD.

Here is the rationale for doing a deep integration like this versus only
sending metrics to StatsD in middleware.  It's the only way to report
some internal activities of Swift in a real-time manner. So to have one
way of reporting to StatsD and one place/style of configuration, even
some things (like, say, timing of PUT requests into the proxy-server)
which could be logged via middleware are consistently logged the same
way (deep integration via the logger delegate methods).

When log_statsd_host is configured, get_logger() injects a
swift.common.utils.StatsdClient object into the logger as
logger.statsd_client.  Then a set of delegate methods on LogAdapter
either pass through to the StatsdClient object or become no-ops. This
allows StatsD logging to look like:
    self.logger.increment('some.metric.here')
and do the right thing in all cases and with no messy conditional logic.

I wanted to use the pystatsd module for the StatsD client, but the
version on PyPi is lagging the git repo (and is missing both the prefix
functionality and timing_since() method).  So I wrote my
swift.common.utils.StatsdClient.  The interface is the same as
pystatsd.Client, but the code was written from scratch.  It's pretty
simple, and the tests I added cover it.  This also frees Swift from an
optional dependency on the pystatsd module, making this feature easier
to enable.

There's test coverage for the new code and all existing tests continue
to pass.

Refactored out _one_audit_pass() method in swift/account/auditor.py and
swift/container/auditor.py.

Fixed some misc. PEP8 violations.

Misc test cleanups and refactorings (particularly the way "fake logging"
is handled).

Change-Id: Ie968a9ae8771f59ee7591e2ae11999c44bfe33b2
2012-05-11 15:25:38 -07:00
Tom Fifield
9920aeb7d4 bug 661267 adding config eastereggs, fixing defaults
Change-Id: I41356ee250c9088a2387b0d493586dd990a04ac3
2012-04-30 08:22:42 +10:00
gholt
0becfab629 Added option to disable SQLite db preallocation
Added option to disable SQLite db preallocation. This can be very
useful on pure ssd account/container servers where the extra space is
worth more than the lesser fragmentation.

Change-Id: I8fbb028a9b6143775b25b343e97896497a8b63a9
2012-03-28 21:45:18 +00:00
John Dickinson
156f27c921 object versioning
Object versioning in swift is implemented by setting a flag on the container
to tell swift to version all objects in the container. The flag is the
``X-Versions-Location`` header on the container, and its value is the
container where the versions are stored.

When data is ``PUT`` into a versioned container (a container with the
versioning flag turned on), the existing data in the file is redirected to a
new object and the data in the ``PUT`` request is saved as the data for the
versioned object. The new object name (for the previous version) is
``<versions_container>/<object_name>/<timestamp>``, where the timestamp is
generated by converting the ``Last-Modified`` header value of the current
version to a unix timestamp.

A ``GET`` to a versioned object will return the current version of the object
without having to do any request redirects or metadata lookups.

Change-Id: I4fcd723145e02bbb2ec1d3ad356713f5dea43b8b
2012-03-28 15:48:43 -05:00
gholt
52ba08d67d Improvements to database replication.
Note: I'd like to get this released as soon as possible as it is a
data durability issue.

1) Orders nodes so that none get starved (see code and footnote).

2) New max_diffs setting that caps how long the replicator will spend
   trying to sync a given database per pass so the other databases
   don't get starved.

3) Replaces run_pause with the more standard "interval", which means
   the replicator won't pause unless it takes less than the interval
   set.

Change-Id: I986742229e65031df88f5251ca61746b7c8d2bde
2012-03-06 06:08:18 +00:00
gholt
552fa38ed1 Doc updates 2011-06-22 15:05:23 +00:00
gholt
145651e04f Updated container-server.conf-sample 2011-06-06 16:12:54 +00:00
gholt
7ac6baf6d1 container-sync: Support HTTP proxy. 2011-06-03 02:28:21 +00:00
gholt
adb45bc871 Restrict hosts that can be targets/sources of container syncing 2011-02-24 10:50:00 -08:00
gholt
305e4b41f5 Initial work on container syncing 2011-02-23 23:26:05 -08:00
gholt
a734be95d3 Merge from trunk 2011-01-24 16:28:22 -08:00
gholt
4905c71669 More doc updates for logger stuff 2011-01-23 13:18:28 -08:00
gholt
217198b83b container-updater: temporrar account update suppression on errors 2011-01-21 12:43:50 -08:00
John Dickinson
c53f49ce98 merged with trunk 2010-10-18 15:47:59 +00:00
Jay Payne
66c8b412c8 Moved backlog setting into the [Default] section of the sample-conf files 2010-10-13 21:24:30 +00:00
Jay Payne
223c2e9011 add default backlog setting to sample configs 2010-10-12 19:46:01 +00:00
John Dickinson
2579cf54c2 updated container auditor to only do local work and updated auditor configs 2010-10-12 10:47:56 -05:00
Chuck Thier
04a5ccb4b1 Added vm_test_mode to the sample configs 2010-08-24 14:10:36 +00:00
Chuck Thier
c62707ae72 Refactored logging configuration so that it has sane defaults 2010-08-24 13:41:58 +00:00
Chuck Thier
2c596c0a0f Initial commit of middleware refactor 2010-08-20 00:42:38 +00:00
Chuck Thier
001407b969 Initial commit of Swift code 2010-07-12 17:03:45 -05:00