2550 Commits

Author SHA1 Message Date
gholt
f63dc07b9d Extra safety on account-level DELETE
I just noticed tonight when adding a bunch of stuff to Swiftly that
the Bulk Delete middleware uses an account-level DELETE request,
albeit with a query parameter of bulk-delete. But, one typo and,
assuming the cluster supports it and you have access, whoops, you
just marked the account for deletion!

I put a bit of extra safety on the account deletion by requiring it
to have an empty query string.

Change-Id: Ib5df11193b04eff69d14185bd9d0607169131e7f
2013-04-19 09:43:31 +00:00
Peter Portante
dec517e349 Drop cache after fsync
Drop the cache (fadvise) after calling fsync to avoid redundant work
in the kernel visiting pages that are dirty. By dropping cache after
fsync(), the pages will be clean so can be easily removed from the
cache.

Change-Id: I3013d2ac1e27bbe43e58a81638d04805f5e2ae6e
Signed-off-by: Peter Portante <peter.portante@redhat.com>
2013-04-18 14:36:52 -04:00
Jenkins
a186c53b24 Merge "fix slo docs" 2013-04-17 19:39:43 +00:00
Jenkins
47eef53a35 Merge "slobjects can not be made of other slobjects" 2013-04-17 19:06:34 +00:00
Jenkins
dfcfb767d3 Merge "Add x-remove-versions-location for feature: disable versioning" 2013-04-17 17:57:53 +00:00
David Goetz
d2a4f75a95 fix slo docs
Change-Id: I49f8c272457547d8aae853986573549828d2503a
2013-04-17 07:03:46 -07:00
Jenkins
6a06b59347 Merge "Expirer now quotes names when deleting" 2013-04-16 16:02:53 +00:00
Jenkins
8ddfa5e533 Merge "Allow all headers requested for CORS." 2013-04-16 15:56:58 +00:00
Jenkins
c87576bb94 Merge "Refactored lists of nodes to contact for requests" 2013-04-15 23:05:47 +00:00
Jenkins
e3ccb3f52d Merge "Correcting output of swift-ring-builder" 2013-04-15 22:49:48 +00:00
gholt
c354db2158 Expirer now quotes names when deleting
Change-Id: I5c615c6f32967510f09b783b1ba7089119f1d8bd
2013-04-15 22:12:52 +00:00
Jenkins
9826c173da Merge "fix probe tests to work in a VM using mount_check" 2013-04-15 18:47:31 +00:00
Jenkins
59f6779603 Merge "Refactoring format_device function" 2013-04-12 22:54:03 +00:00
David Goetz
3f21e14398 slobjects can not be made of other slobjects
Change-Id: Ib500f5fc2e890c3d23641350815a910168bcd543
2013-04-12 14:36:45 -07:00
Sergey Kraynev
5afd0c3bb6 Refactoring format_device function
* Algorithm format_device was changed for simplicity extension new
  ip addresses parameters.
* Some prints outputs was replacement by function format_device.

Change-Id: I8565d42fcdb62eeb398c4432bb6f499c27c05cf6
2013-04-12 12:40:27 -04:00
John Dickinson
08ce7e1ded fix probe tests to work in a VM using mount_check
If mount_check is true (ie an SAIO with "real "devices, not loopback),
then the servers will correctly return 507 when given a nonsense path.
The first element is treated as a drive path, and that path isn't
mounted. This patch adds 507 as a valid status response to the server
check.

Change-Id: I1d1bb0ab78fd9ea17323635da7e686182fbdbf13
2013-04-11 22:55:52 -07:00
Jenkins
4ae0feb60d Merge "Account Quota Correction" 2013-04-11 19:53:28 +00:00
Jenkins
d31a8e2a93 Merge "fix email address" 2013-04-11 18:26:30 +00:00
John Dickinson
18ef0a7bc5 fix email address
Change-Id: I784a9a5b45fd39420543ff0fb56b0f6d04684f3a
2013-04-11 10:42:31 -07:00
Sergey Kraynev
d7873b5ce3 Correcting output of swift-ring-builder
Indent swift-ring-builder output was changed according
with old style (before implementation of region tier).

Change-Id: I0d1cc7acdc5baf86f343745aea6fc2120838fd36
2013-04-11 09:32:56 -04:00
Jenkins
c496ba1878 Merge "Bug in SLO with multipart-manifest=get content type." 2013-04-10 22:03:49 +00:00
Jenkins
5140c0d5da Merge "Adding a new optional variable called trans_id_suffix" 2013-04-10 20:05:03 +00:00
David Hadas
537af54970 Account Quota Correction
account_quota middleware fails to check if account_info exists
after retrieving it.

Change-Id: I1aa492d3bd713a6bf27e18ef4ec2e29f4cfb694d
Fixes: Bug #1167389
2013-04-10 21:41:59 +03:00
Marcelo Martins
1126e59c12 Adding a new optional variable called trans_id_suffix
The trans_id_suffix (default is empty) would be appended to the swift transaction
id allowing one to easily figure out from which cluster that X-Trans-Id
belongs to. This is very useful when one is managing more than one swift
cluster. Also updated sample and manpage to reflect the changes.

Change-Id: Icdf63643e9c1bde36a9ef5e3f41ee9fb20e55f5d
2013-04-10 06:37:32 -05:00
David Hadas
caa01cd81e objects md5-collisions
This patch identifies md5 collisions on objects and sends a 403
from the object server.

Credits for originating this fix are to Michael Factor.

Change-Id: I4f1b32183e2be6bbea56eaff86b9a4c7f440804a
Fix: Bug #1157454
2013-04-09 23:20:33 +03:00
Jenkins
b93b1327f7 Merge "add unit tests for db_replicator" 2013-04-08 20:51:57 +00:00
gholt
d79a67ebf6 Refactored lists of nodes to contact for requests
Extensive refactor here to consolidate what nodes are contacted for
any request. This consolidation means reads will contact the same set
of nodes that writes would, giving a very good chance that
read-your-write behavior will succeed. This also means that writes
will not necessarily try all nodes in the cluster as it would
previously, which really wasn't desirable anyway. (If you really want
that, you can set request_node_count to a really big number, but
understand that also means reads will contact every node looking for
something that might not exist.)

* Added a request_node_count proxy-server conf value that allows
  control of how many nodes are contacted for a normal request.

In proxy.controllers.base.Controller:

* Got rid of error_increment since it was only used in one spot by
  another method and just served to confuse.

* Made error_occurred also log the device name.

* Made error_limit require an error message and also documented a bit
  better.

* Changed iter_nodes to just take a ring and a partition and yield
  all the nodes itself so it could control the number of nodes used
  in a given request. Also happens to consolidate where sort_nodes is
  called.

* Updated account_info and container_info to use all nodes from
  iter_nodes and to call error_occurred appropriately.

* Updated GETorHEAD_base to not track attempts on its own and just
  stop when iter_nodes tells it to stop. Also, it doesn't take the
  nodes to contact anymore; instead it takes the ring and gets the
  nodes from iter_nodes itself.

Elsewhere:

* Ring now has a get_part method.

* Made changes to reflect all of the above.

Change-Id: I37f76c99286b6456311abf25167cd0485bfcafac
2013-04-08 20:48:32 +00:00
David Goetz
35d2cec919 Bug in SLO with multipart-manifest=get content type.
In trying to override the content-type for the convenience
call I broke COPYs and POSTs for SLO. yaaa...

Change-Id: Ifbcda6d2dd0ee43e43d62e58a90301e7afd05e27
2013-04-08 11:22:20 -07:00
Chmouel Boudjnah
c687f6956c Allow all headers requested for CORS.
- We allow all headers requested in preflight request. The CORS
  specification does leave the door open for this, as mentioned in
  http://www.w3.org/TR/cors/#resource-preflight-requests
  Note: Since the list of headers can be unbounded
  simply returning headers can be enough.
- This is a followup to review:
  https://review.openstack.org/#/c/24415/.
- Fixes bug 1155034.

Change-Id: If7b8f2f3a581c5209892d1ccc9f06ddb8fac92dd
2013-04-05 21:12:15 +01:00
Jenkins
48a94f39dd Merge "Allow a configurable set of TempURL methods" 2013-04-04 22:17:15 +00:00
tong li
f1aefcbc13 directory /var/cache/swift /srv/node should be created.
For multiple node swift cluster setup, these directories should be
created in the setup steps. Otherwise, when start up storage servers
there will be errors. The error messages won't tell if it is because
of missing the directory.

Change-Id: I67d74abb7743b76739b5e747d6dcbd3214b00774
Fixes: bug #1158519
2013-04-04 20:39:01 +00:00
gholt
1cb952a958 Allow a configurable set of TempURL methods
Folks have actually been asking for this. I think they're sending a
DELETE TempURL to someone way ahead of time and the someone issues it
when they're ready. Honestly, I'm not entirely sure of the use case,
but having the set of methods configurable wouldn't hurt.

Change-Id: Ibdb48f8a72077b045eeedddfae4c0a1f56098d7a
2013-04-04 20:37:23 +00:00
Greg Lange
30e88fd676 add unit tests for db_replicator
Change-Id: I9002fa193a51f40523e7936e3117a2f3f2b2f7f8
2013-04-04 18:45:24 +00:00
Jenkins
2927dcc967 Merge "Update documentation for account_quota middleware" 2013-04-03 23:34:05 +00:00
Jenkins
ace8cd5aef Merge "Fix container-server sorting for manifest listings." 2013-04-03 23:27:36 +00:00
Jenkins
ab355e349a Merge "Fix reading xattrs in object-server's unittests." 2013-04-03 20:55:06 +00:00
Jenkins
c5408926c6 Merge "SLO multipart-manifest=get returns incorrect Content-Type" 2013-04-03 18:47:18 +00:00
Jenkins
caaf5f24ce Merge "Change setup.cfg style." 2013-04-03 16:50:10 +00:00
Christian Schwede
e5cd78d902 Update documentation for account_quota middleware
Change-Id: I3dc7930ee2a1eb26b1f60e83fce2bc1bea0a8b0f
2013-04-03 15:10:42 +02:00
Jenkins
ba5a0b0fe1 Merge "Change logger name to module itself instead of root" 2013-04-02 14:45:12 +00:00
Ukov Dmitry
02beacf0da Change setup.cfg style.
Change-Id: I466470237ddd317740eb99bd55af7e7a50270ae3
Fixes: bug #1163258
2013-04-02 15:55:56 +04:00
Jenkins
01a5a47610 Merge "Dynamic Large Object sleeps too much" 2013-04-02 07:27:10 +00:00
David Goetz
3a8f314fdc SLO multipart-manifest=get returns incorrect Content-Type
Change-Id: I657c9f7a7555d90ccfa7e6ff0f5f3fcbd7eada1d
2013-04-01 13:16:46 -07:00
David Goetz
7c0b9a7424 Dynamic Large Object sleeps too much
DLO is supposed to sleep 1 second per segment after 10 (default) initial
segments are loaded to prevent abuse of the system. Currently, if the user
did a range request to the 11th segment the 1 sec delay would start immediately
on the 11th segment because the skipped segments were counted as initial
segments. This path fixes that.

Change-Id: I53bbc24af54921ddc04d9cb3b704366b51b3f740
2013-04-01 10:19:35 -07:00
Jenkins
b524a90bd9 Merge "fixes some problems pyflakes complains about" 2013-03-29 10:12:10 +00:00
John Dickinson
602bf524b8 updated AUTHORS and CHANGELOG
These updates are to reflect the changes that have been
backported into the 1.8.0 release candidate.

Change-Id: I9b2abb9809fa6715a427dc0dbe0474aba140a596
2013-03-28 21:01:49 -07:00
Samuel Merritt
aa716dd9b8 Fix container-server sorting for manifest listings.
The current code will always contact the container server for the
first replica of the container; this fixes it to use the same
backend-sort as all the other places in the proxy that talk to
backends.

Change-Id: If1c168b95bfc5cd120036869f3066549b7a1000a
2013-03-28 12:03:58 -07:00
Greg Lange
7920cbd80a fixes some problems pyflakes complains about
Change-Id: I1eb16fa595b0b1d97ae14279b8aa25bd2740f511
2013-03-28 18:49:28 +00:00
Greg Lange
44f00a23c1 fixed some minor things in tests that pyflakes complained about
Change-Id: Ifeab56a964630bcf941e932fcbe39e6572e62975
2013-03-26 20:42:26 +00:00
David Goetz
d2d9a15ddf Fixing SLO range requests
Change-Id: I355e99c971cfe8f7f8dedcd8f9f08c97079863cc
2013-03-26 09:40:03 -07:00