2096 Commits

Author SHA1 Message Date
Jenkins
1ef17d8284 Merge "add support for custom log handlers" 2012-10-05 21:34:48 +00:00
Florian Hines
92826d0602 add support for custom log handlers
Add a hook to get_logger to run custom functions to add custom log
handlers or the like.

Change-Id: Ib04b12939dcac7e4ad6453dea9795682044c6ae0
2012-10-05 15:56:34 -05:00
Victor Rodionov
5b02fded4e cleanup double repeated response.headers['X-Timestamp'] = file.metadata['X-Timestamp']
in swift.obj.server.HEAD

Change-Id: Ie7cf853319589db7c48836dd5d7fd5e68cb009cd
2012-10-04 02:47:22 +04:00
Jenkins
6801920785 Merge "close_swift_conn is a method of swift.proxy.controllers.base.Controller, not function" 2012-10-03 21:25:21 +00:00
Jenkins
026846c7b5 Merge "local WSGI Request and Response classes" 2012-10-03 20:49:05 +00:00
Victor Rodionov
0b756d75c4 close_swift_conn is a method of swift.proxy.controllers.base.Controller, not function
Change-Id: I84dd7bc2fc5ebd1190a7184f42fc5ecf5fd7b4d3
2012-10-04 00:48:25 +04:00
Darrell Bishop
57ebd17910 Warn when killing fails with EPERM
Have swift-init warn when the running user doesn't have permissions to
signal processes.  Fixes bug 1017494.

Change-Id: Icb9048ab36f1ca73bb93b11c9c2aed882d99dfa7
2012-10-03 08:28:36 -07:00
Jenkins
550e876869 Merge "Allow underscores in account/user in tempauth via encoding." 2012-10-02 23:42:59 +00:00
Jenkins
a0d96b184d Merge "catch no such file in hash_suffix" 2012-10-02 18:49:02 +00:00
David Goetz
6f9473d31d catch no such file in hash_suffix
Change-Id: I70ee55d12b1ca91fdfe4891c3f82ab0f832f15d3
2012-10-02 10:42:28 -07:00
Darrell Bishop
5582f166c2 Allow underscores in account/user in tempauth via encoding.
This patch introduces a new user format to tempauth which starts with
"user64_" and in which the account and user are base64 encoded (sans
padding equal signs):

user64_<account_b64>_<user_b64> = <key> [group] [group] [...] [storage_url]

This allows unambiguous parsing and adds the ability to have a user or
account name containing underscores.

The handling of tempauth configuration is fully backward compatible and
this only adds new, optional functionality.

Also beefed up unit tests to get full coverage of the user parsing code
in tempauth's __init__().

Change-Id: Id06304934ab8055a28921f2e1aa3c58a2036d8f8
2012-10-02 09:05:22 -07:00
Michael Barton
5e3e9a882d local WSGI Request and Response classes
This change replaces WebOb with a mostly compatible local library,
swift.common.swob.  Subtle changes to WebOb's API over the years have been a
huge headache.  Swift doesn't even run on the current version.

There are a few incompatibilities to simplify the implementation/interface:
 * It only implements the header properties we use.  More can be easily added.
 * Casts header values to str on assignment.
 * Response classes ("HTTPNotFound") are no longer subclasses, but partials
   on Response, so things like isinstance no longer work on them.
 * Unlike newer webob versions, will never return unicode objects.

Change-Id: I76617a0903ee2286b25a821b3c935c86ff95233f
2012-09-28 14:48:48 -07:00
Jenkins
f0bd91dd14 Merge "allow replicator run_once to check specific devices/partitions" 2012-09-28 21:24:36 +00:00
Jenkins
e6d720cde5 Merge "Fix errors with fdatasync on BSD platforms" 2012-09-28 21:13:36 +00:00
David Goetz
a6c44d2764 allow replicator run_once to check specific devices/partitions
Change-Id: If45f77fda269ae6e251579542e70eb71bd11fe2a
2012-09-28 12:24:15 -07:00
Brian Cline
f91198d09b Fix errors with fdatasync on BSD platforms
Abstracts fdatasync/fsync/fcntl into an FsyncWrapper to determine, the
first time it is used, which fsync-like call is supported on the host
platform:

 - Linux: use fdatasync(fd) as Swift already does today.
 - Darwin: use fcntl(fd, F_FULLFSYNC), since Darwin provides the
   F_FULLFSYNC flag when you wish to fsync data you care about, ensuring
   that it is written to disk and not simply a buffer.
 - BSD/Other: use fsync for non-Darwin BSDs and as a fallback for other
   platforms.

Change-Id: Ibcfdeef7843b2f10cf8a17a6b989038ea2364376
2012-09-28 10:01:03 -05:00
Michael Barton
bc60cc1160 base controller GETorHEAD refactor
simplify, reduce the amount of code

Change-Id: I443a43030291f35a0b283cdd036cb09ec2ef9444
2012-09-27 15:19:27 -07:00
Jenkins
be8777965f Merge "replace a value set with utils.TRUE_VALUES" 2012-09-27 17:40:06 +00:00
John Dickinson
b530e62313 updated with 1.7.5 version bump and changelog update
Change-Id: I04be779539b2bff76e2f614f50ddba2592fe4a36
2012-09-25 14:37:45 -07:00
gholt
7e0c4eb152 Fixed leak on 499s
This fixes an issue where Request objects (and related objects) were
not getting garbage collected when a 499 (client disconnect) occurred
for responses that still would have had more than the proxy server's
client chunk size left to send.

Fixed bug #1055834

Change-Id: Ib7b5fffe970dcdec60b90a9bdcd67c49b68fb914
2012-09-25 19:23:31 +00:00
Darrell Bishop
57b10cd38c Restore old SIGPIPE handler in a proxy server test.
I think not doing this was setting the SIGPIPE handler to signal.SIG_DFL
instead of the Python default of signal.SIG_IGN.  This could cause other
tests which make a client stop reading before all data "chunks" are read
to fail harder than they should (i.e. the SIGPIPE there is benign and
even expected--the other side of the socket really did get closed
early).

Change-Id: I946a1913714a34ec677b701a2b5724bc8b715f70
2012-09-25 11:22:11 -07:00
Jenkins
4b858c1860 Merge "Gracefully handle stdin flush failure on BSD" 2012-09-23 05:12:16 +00:00
Brian Cline
87c1c5dce9 Gracefully handle stdin flush failure on BSD
Catches an IOError that is thrown when starting Swift on BSD-based
platforms, as the underlying system call to flush returns errno 9 (bad
file descriptor), whereas on Linux it succeeds.

Change-Id: Ic143d2fe6c3e1e0b39794958b40b0f5efdc17c06
2012-09-21 14:17:59 -05:00
Brian Cline
da99c3390b Only set TCP_KEEPIDLE where supported
Resolves an issue preventing Swift from starting on BSD-based platforms
that do not have this TCP flag (FreeBSD, OSX/Darwin, OpenBSD); identical
to Glance bug 879195, change I2f062a34, Glance commit efcdac36
(originally reported and patched by Brian Waldon).

Change-Id: Ifd696fe1080464d060e608c075ad0c113baab5ab
2012-09-21 11:06:18 -05:00
John Dickinson
8a7afa535f 1.7.2. changelog and version bump to 1.7.3-dev
Change-Id: Id398b4c0764ca0995cec03c5f0ea6198ef877ade
2012-09-18 15:45:40 -07:00
Jenkins
73aa3cbda1 Merge "Fix bug where serialization_format is ignored" 2012-09-18 18:54:11 +00:00
Jenkins
b0a2f795f4 Merge "builder.add_devs gets next id if not provided" 2012-09-18 18:53:43 +00:00
gholt
c97fb37e10 Fix bug where serialization_format is ignored
Change-Id: I5a5ac8b5f18e077105ab12e9b1f0ccafac3983f7
2012-09-18 18:24:47 +00:00
dk647
568b61af22 replace a value set with utils.TRUE_VALUES
replace a value set with utils.TRUE_VALUES, and fix some pep8 issues

Change-Id: If4db41d0d538afddee0d78449d229ce319c5efba
2012-09-18 16:16:24 +08:00
Dan Prince
411c7cb8ff Add README.md to the tarball.
Fixes LP Bug #1051125

Change-Id: If1f96e1d759276cafd5378b4e74b0dbcb1fa2358
2012-09-14 20:42:05 -04:00
Florian Hines
c4f5761101 builder.add_devs gets next id if not provided
Have builder.add_devs get the next id to use when adding a new device
if its not specified in the dict.

Change-Id: I5a0defab43f5cfc5d997080bfd8563bfe72368ad
2012-09-14 16:11:50 -05:00
Jenkins
30337a8875 Merge "added errno.EINVAL check to fallocate" 2012-09-14 20:50:20 +00:00
John Dickinson
bfc114bc1d new more helpful README
Change-Id: Ie5e88178e061955aea8759e5451a2d33ccc76511
2012-09-13 20:59:41 -07:00
John Dickinson
343968b2d3 added disable_fallocate info to docs
Change-Id: I8130e258d77f47bfa28ecf79a51a6e34671ada22
2012-09-12 11:15:25 -07:00
John Dickinson
2ca379d40f added errno.EINVAL check to fallocate
On some systems (eg Illumos), posix_fallocate() returns
EINVAL when the underlying filesystem doesn't support the
operation. On Linux, the fallocate() call returns
EOPNOTSUPP.

The need for this patch was revealed by Victor Rodionov
<victor.rodionov@nexenta.com>.

Change-Id: I06fa9d49e7ec4084135843b7e0c91948dc098d27
2012-09-12 09:00:10 -07:00
Jenkins
a7e6d44706 Merge "Adds Error Handling to swift-drive-audit for missing or unreadable /var/log/kern.log" 2012-09-11 20:54:46 +00:00
Jenkins
5c67f1a695 Merge "container_info returns a dict" 2012-09-11 20:24:13 +00:00
Andy McCrae
463da7e170 Adds Error Handling to swift-drive-audit for missing or unreadable /var/log/kern.log
Fixes Bug 1049081

Change-Id: If977080350cc5cdb6bc633b6af7d3c490ed23d46
2012-09-11 16:23:32 +00:00
David Goetz
d24e280bf4 obj replicator speed up
Change-Id: If02b573353dedea9c2368ce4733fe97599229b2e
2012-09-10 15:12:39 -07:00
Michael Barton
063789b14e container_info returns a dict
This change refactors container_info to return a dict instead of a tuple
with 6 (and counting...) values.

Also adds container size to cached info.

Change-Id: If68a06ab170504fbf07323354f3d0e073850493d
2012-09-09 03:18:26 -07:00
Jenkins
3482eb26f9 Merge "swift constraints are now settable via config" 2012-09-07 23:21:47 +00:00
Jenkins
67f21f5e4a Merge "Breakout search_devs & add get_builder() for reuse" 2012-09-07 22:09:58 +00:00
Jenkins
d96a7a89f4 Merge "format some code according to pep8" 2012-09-07 22:03:03 +00:00
Jenkins
aee707b78d Merge "Fix pep8 issues" 2012-09-07 22:01:48 +00:00
Jenkins
6643fb1473 Merge "Fix pep8 issues" 2012-09-07 21:55:17 +00:00
Jenkins
fc58cb2c91 Merge "Obj replicator cleans up files where part dirs should be." 2012-09-07 21:54:05 +00:00
John Dickinson
a2ac5efaa6 swift constraints are now settable via config
Change previously hard-coded constants into config variables. This
allows deployers to tune their cluster more specifically based on
their needs. For example, a deployment that uses direct swift access
for public content may need to set a larger header value constraint to
allow for the full object name to be represented in the Content-
Disposition header (which browsers check to determine the name of a
downloaded object).

The new settings are set in the [swift-constraints] section of
/etc/swift/swift.conf. Comments were also added to this config file.

Cleaned up swift/common/constraints.py to pass pep8 1.3.3

Funtional tests now require constraints to be defined in /etc/test.conf or in
/etc/swift/swift.conf (in the case of running the functional tests against a
local swift cluster). To have any hope of tests passing, the defined
constraints must match the constraints on the tested cluster.

Removed a ton of "magic numbers" in both unit and functional tests.

Change-Id: Ie4588e052fd158314ddca6cd8fca9bc793311465
2012-09-07 11:18:42 -07:00
Jenkins
4bf5ee3850 Merge "Fix pep8 issues in test_memcached.py." 2012-09-07 17:17:54 +00:00
dk647
3139760218 Fix pep8 issues
Fix pep8 issues in swift/obj/expirer.py

Change-Id: Iba80aa5010f0c23e7fe1e5abc230a1c7f76687a8
2012-09-07 12:20:20 +08:00
dk647
cde6d597a7 Fix pep8 issues
Fix pep8 issues in swift/obj/auditor.py

Change-Id: Iae6fe2c7399fd52455d66ba57360e2f7d9e017f3
2012-09-07 12:11:35 +08:00