Commit Graph

2770 Commits

Author SHA1 Message Date
a24c89e5bd adding copyright notice
Change-Id: Ic2c79d52770208c22b39cb2b00f2f8a6a86e8aa7
2013-07-01 12:04:26 -07:00
4a90414fc7 Merge "docfix apache2 now supports client chunked encodin" 2013-06-29 01:21:10 +00:00
1f7c86ec91 Merge "Update docs to use default XFS inode size" 2013-06-29 00:51:24 +00:00
581f7f5517 Update docs to use default XFS inode size
In past couple of years, the XFS team has greatly improved inode use in
xfs.  With more recent kernels, there is no performance penalty for
using the default inode size, and a smaller inode size gives us
improvements in other areas where disk access is involved.

DocImpact

Change-Id: Ie9da53a6e8bf43d1d02881befbb52595462c9f2e
2013-06-28 19:52:17 +00:00
e759dad84f Merge "Add test for swift.common.utils.lock_file" 2013-06-28 17:53:23 +00:00
68cb91097b docfix apache2 now supports client chunked encodin
As reported in the documentation bug, the apache deployment guide's
reference to apache2 mod_wsgi not supporting client chunked encoding
has become outdated. It now supports this feature, using an optional
parameter.

Updated the paragraph in question to reflect this

patchset 2 mentions the WSGIChunkedRequest variable and adds it
to the sample configs - On by default. Feedback welcome

fixes bug 1194935

Change-Id: I07c5c8506ac34e1e0e08fa6d961babde2f9b7367
2013-06-28 15:27:33 +10:00
0e602979c9 Add test for swift.common.utils.lock_file
Change-Id: Id03f130ad7016b11b740050b7aadf879773d5a5f
2013-06-28 00:28:26 -04:00
c6e53721e0 Merge "Make formpost work with multiple tempurl keys." 2013-06-27 20:45:53 +00:00
2a91768911 Merge "TempAuth: don't return 401/403 if .r without .rlistings" 2013-06-26 22:25:46 +00:00
53908a1147 Merge "Allow SLOs to be made up of other SLOs" 2013-06-26 21:51:07 +00:00
bd00f0c8c9 Make formpost work with multiple tempurl keys.
Formpost wouldn't look at X-Account-Meta-Temp-URL-Key-2 for signature
validation, which was confusing to users. Now it does, and there's a
utility function that formpost and tempurl both use to help prevent
divergence in the future.

Also removed formpost's custom cache-lookup logic. Formpost was
looking for tempurl keys in memcache, but it's been broken since
commit 21343ab (2013-05-02) when tempurl changed its memcache keys, so
the lookup would always miss. (Since tempurl started storing *lists*
of keys, formpost would have choked on the new data, so that oversight
was the only thing keeping formpost working at all.)

Now formpost just uses the same get_account_info() call as everyone
else does, so it gets all the benefits of the new caching framework.

Change-Id: I426d8f2882ebe3ce373de604b9ff18846f5999e3
2013-06-26 11:44:48 -07:00
9f942b1256 Allow SLOs to be made up of other SLOs
We've gone back and forth about this. In the initial commit, it couldn't
possibly work because you wouldn't be able to get the Etags to match. Then it
was expressly disallowed with a custom error message, and now its allowed. The
reason we're allowing it is that 1,000 segments isn't enough for some use cases
and we decided its better than just upping the number of allowed segments. The
code to make it work isn't all that complicated and it allows for virtually
unlimited SLO object size. There is also a new configurable limit on the
maximum connection time for both SLOs and DLOs defaulting to 1 day. This will
hopefully alleviate worries about infinite requests. Think I'll leave the
python-swift client support for nested SLOs to somebody else though :).

DocImpact

Change-Id: Id16187481b37e716d2bd09bdbab8cc87537e3ddd
2013-06-26 09:44:33 -07:00
b012fd998c Change ring partition size for SAIO
Making this smaller (10 instead of 18) can make some of the tests run
faster and makes rebuilding of the rings faster.

Change-Id: Ibe46011d8e6a6482d39b3a20ac9c091d9fbc6ef7
2013-06-26 15:52:18 +00:00
1620f4e99e Merge "Adding unit tests for direct client" 2013-06-26 15:42:40 +00:00
8ff374de97 TempAuth: don't return 401/403 if .r without .rlistings
When checking referer (.r, .rlistings), tempauth return 401/403 if '.r'
exists without '.rlistings' or obj. Actually it should skip this kind of
authorizing and check the following authorizing.

And remove some unnecessary lines

Change-Id: Ib60e59b7c80331a699cb5c3496f948cf32135d0b
2013-06-26 23:35:16 +08:00
66a0817e99 Merge "Remove dead code and tests for staticweb." 2013-06-26 14:32:01 +00:00
882078dcf1 Merge "version bump to 1.9.1" 2013-06-26 06:46:12 +00:00
fefdf5f90b Merge "updated changelog and authors update for 1.9.0 release" 2013-06-26 06:43:13 +00:00
8b4520a483 Remove dead code and tests for staticweb.
Dead since e499b91.

Change-Id: I3c571ae471eb4a0b95772b19eac90183cfc7fe22
2013-06-25 23:18:57 -07:00
68fc163fdd version bump to 1.9.1
Change-Id: If04c88f7e686f4b4709c5b68351ebbbea97d12bd
2013-06-25 23:00:31 -07:00
8226761889 Deleted account respond as non existing accounts
Currently clients can not distinguish between non existing accounts
(which can be created) and accounts marked for deletion, which has
not yet been reaped and therefore cannot be re-created until reaped.

Following this patch, if an account is marked as deleted but hasn't
been reaped and is still on disk, responses will include a status
header:
    'X-Account-Status' = 'Deleted'

Fixes:Bug #1188609
Change-Id: Ibd39965ae3f5d45fd78f130e0e31f5a0141a8633
2013-06-26 08:33:23 +03:00
abda080366 updated changelog and authors update for 1.9.0 release
Change-Id: I61ffbaba01aba86d61e4f3d988b5abeee8f07813
2013-06-25 21:41:28 -07:00
edf4068c8b Merge "Local write affinity for object PUT requests." 2013-06-26 04:39:53 +00:00
83bdd0b2e1 Merge "Refactor DiskFile instantiation in object server." 2013-06-25 20:29:25 +00:00
a42e87a464 Merge "Add sample rsyslog.conf." 2013-06-25 19:58:01 +00:00
9ac0b14a05 Merge "Fix swift-dispersion-report." 2013-06-25 19:43:59 +00:00
9fc55ee2b4 Add sample rsyslog.conf.
Give users a sample rsyslog.conf to manage their logs easily based on
LOCAL0 swift used. In this patch I offer some choices to output logs
via commnet lines or uncomment.

Change-Id: I2fe150a6e3d164a989c3520c0b7f032897a71f18
2013-06-25 10:24:26 +08:00
6768d5b4be Use object name from request in bulk Errors
This will allow users to more easily match the failed objects in Errors
for bulk delete requests to the object names they provided in the request.
For extract Errors, it reports the failed path from the tar archive.

DocImpact

Change-Id: I084057810fc4fb7fdac05494cc6fec2cbf81bb9d
2013-06-24 16:44:32 -04:00
57d4e6e718 Fix swift-dispersion-report.
Fixes bug 1180680.

Change-Id: I1c0eaa6f934f73853e832a8e468e1b8369a21117
2013-06-24 10:58:22 -07:00
d9f2a76973 Local write affinity for object PUT requests.
The proxy can now be configured to prefer local object servers for PUT
requests, where "local" is governed by the "write_affinity". The
"write_affinity_node_count" setting controls how many local object
servers to try before giving up and going on to remote ones.

I chose to simply re-order the object servers instead of filtering out
nonlocal ones so that, if all of the local ones are down, clients can
still get successful responses (just slower).

The goal is to trade availability for throughput. By writing to local
object servers across fast LAN links, clients get better throughput
than if the object servers were far away over slow WAN links. The
downside, of course, is that data availability (not durability) may
suffer when drives fail.

The default configuration has no write affinity in it, so the default
behavior is unchanged.

Added some words about these settings to the admin guide.

DocImpact

Change-Id: I09a0bd00524544ff627a3bccdcdc48f40720a86e
2013-06-23 22:04:56 -07:00
13347af64c Merge "Improve swift's keystoneauth ACL support" 2013-06-24 04:57:13 +00:00
054ffbe4a6 Merge "Stop getting useless bytes on manifest Range requests." 2013-06-23 17:48:49 +00:00
60388c9978 Adding unit tests for direct client
- Added simple unit tests for direct client
 - Implemented chunked put in direct_put_object

Change-Id: Icec1019b5157fae33e2fd36a7fc3b9c0cc15afec
2013-06-22 16:15:29 +08:00
e8e6bffc95 Clean up some remnants of StaticWeb's and TempURL's custom caching.
As of e499b91, these config values and functions are unused in StaticWeb.

As of 2e155e5, the comment in TempURL is false.

Change-Id: I75b631ece8a9a6075c406765361629c549c449f6
2013-06-21 17:37:06 -07:00
75660a1e9e Merge "bulk response Content-Type does not match request Accept header" 2013-06-21 17:21:03 +00:00
461b88b7bd Merge "Added utils.FileLikeIter" 2013-06-21 15:34:41 +00:00
2412f6c737 Merge "Add "normal", optparse-style options to swift-ring-builder add." 2013-06-21 15:22:51 +00:00
4eda025a58 Merge "Add example Apache config files" 2013-06-21 08:20:29 +00:00
d5736ec00b Merge "python3: Introduce py33 to tox.ini" 2013-06-21 01:45:45 +00:00
cc660921dc Merge "use get_container_info for staticweb" 2013-06-21 01:32:22 +00:00
e872e54c2a Add example Apache config files
This patch is needed in 1.9 as a dependency for adding swift/apache
to devstack.
zhang-hare is working in parallel on a devstack patch that depend
on this patch.

As part of this patch we add examples dir.
In this dir we add template config files that can be used either by
a script (as will be done by devstack) or serve a an example to the
admin.

Apache2 vhost files were added under examples/apache2
Wsgi files were added under examples/wsgi

Fixes: Bug #1191388

Change-Id: If90e38a0ba36f6b59016d4a1374f4100f0bbee7b
2013-06-20 21:56:21 +03:00
ff02537d97 Merge "update .gitignore" 2013-06-20 15:00:44 +00:00
c19daab149 Merge "Add max_header_size to swift.conf-sample and relative UT" 2013-06-20 14:04:26 +00:00
2a9da16de6 update .gitignore
Ignore build/ directory, which is auto-created when running `python
setup.py bdist` like dist/ directory.

Change-Id: Ie58936320d995e4e81f8929128475e538665bde8
2013-06-20 17:58:50 +08:00
e499b9100b use get_container_info for staticweb
Update staticweb to use get_container_info instead of its own memcache entry.

Change-Id: I59bc0d94b9e45f3d5776a5462f53b82ab78101d4
2013-06-20 01:04:53 -07:00
1b930dd669 version bump to 1.9.0 to reflect current dev target
Change-Id: Ia0a1b36ae90ed895dcc9876c016aaf261ad56674
2013-06-19 15:30:48 -07:00
527dc236c5 Merge "Make TempURL more cache-efficient." 2013-06-19 21:43:25 +00:00
9d0a0e850e Merge "Swift tests for unicode headers" 2013-06-19 20:23:33 +00:00
440f508ebe Merge "Optional TCP_KEEPIDLE in unit tests" 2013-06-19 19:32:05 +00:00
2e155e5603 Make TempURL more cache-efficient.
So, there's two cases here. In one case, we have memcache enabled;
account info gets cached in the usual place, and tempurl keys get
cached separately. In the other case, we have no cache. In neither
case is anything gained by having TempURL cache keys separately from
the account info.

This commit removes TempURL's custom caching logic and makes it rely
on get_account_info() instead.

Benefits include:
 * immediate visibility of new keys on account POST
 * less data in the cache per account --> more stuff fits in cache
 * less code for bugs to hide in

Change-Id: Idb0b6c165da14196b4c79149c546f0159b54edcb
2013-06-19 11:44:47 -07:00