954 Commits

Author SHA1 Message Date
gholt
f63dc07b9d Extra safety on account-level DELETE
I just noticed tonight when adding a bunch of stuff to Swiftly that
the Bulk Delete middleware uses an account-level DELETE request,
albeit with a query parameter of bulk-delete. But, one typo and,
assuming the cluster supports it and you have access, whoops, you
just marked the account for deletion!

I put a bit of extra safety on the account deletion by requiring it
to have an empty query string.

Change-Id: Ib5df11193b04eff69d14185bd9d0607169131e7f
2013-04-19 09:43:31 +00:00
Jenkins
47eef53a35 Merge "slobjects can not be made of other slobjects" 2013-04-17 19:06:34 +00:00
Jenkins
dfcfb767d3 Merge "Add x-remove-versions-location for feature: disable versioning" 2013-04-17 17:57:53 +00:00
Jenkins
6a06b59347 Merge "Expirer now quotes names when deleting" 2013-04-16 16:02:53 +00:00
Jenkins
8ddfa5e533 Merge "Allow all headers requested for CORS." 2013-04-16 15:56:58 +00:00
Jenkins
c87576bb94 Merge "Refactored lists of nodes to contact for requests" 2013-04-15 23:05:47 +00:00
gholt
c354db2158 Expirer now quotes names when deleting
Change-Id: I5c615c6f32967510f09b783b1ba7089119f1d8bd
2013-04-15 22:12:52 +00:00
David Goetz
3f21e14398 slobjects can not be made of other slobjects
Change-Id: Ib500f5fc2e890c3d23641350815a910168bcd543
2013-04-12 14:36:45 -07:00
John Dickinson
08ce7e1ded fix probe tests to work in a VM using mount_check
If mount_check is true (ie an SAIO with "real "devices, not loopback),
then the servers will correctly return 507 when given a nonsense path.
The first element is treated as a drive path, and that path isn't
mounted. This patch adds 507 as a valid status response to the server
check.

Change-Id: I1d1bb0ab78fd9ea17323635da7e686182fbdbf13
2013-04-11 22:55:52 -07:00
Jenkins
4ae0feb60d Merge "Account Quota Correction" 2013-04-11 19:53:28 +00:00
Jenkins
c496ba1878 Merge "Bug in SLO with multipart-manifest=get content type." 2013-04-10 22:03:49 +00:00
Jenkins
5140c0d5da Merge "Adding a new optional variable called trans_id_suffix" 2013-04-10 20:05:03 +00:00
David Hadas
537af54970 Account Quota Correction
account_quota middleware fails to check if account_info exists
after retrieving it.

Change-Id: I1aa492d3bd713a6bf27e18ef4ec2e29f4cfb694d
Fixes: Bug #1167389
2013-04-10 21:41:59 +03:00
Marcelo Martins
1126e59c12 Adding a new optional variable called trans_id_suffix
The trans_id_suffix (default is empty) would be appended to the swift transaction
id allowing one to easily figure out from which cluster that X-Trans-Id
belongs to. This is very useful when one is managing more than one swift
cluster. Also updated sample and manpage to reflect the changes.

Change-Id: Icdf63643e9c1bde36a9ef5e3f41ee9fb20e55f5d
2013-04-10 06:37:32 -05:00
David Hadas
caa01cd81e objects md5-collisions
This patch identifies md5 collisions on objects and sends a 403
from the object server.

Credits for originating this fix are to Michael Factor.

Change-Id: I4f1b32183e2be6bbea56eaff86b9a4c7f440804a
Fix: Bug #1157454
2013-04-09 23:20:33 +03:00
Jenkins
b93b1327f7 Merge "add unit tests for db_replicator" 2013-04-08 20:51:57 +00:00
gholt
d79a67ebf6 Refactored lists of nodes to contact for requests
Extensive refactor here to consolidate what nodes are contacted for
any request. This consolidation means reads will contact the same set
of nodes that writes would, giving a very good chance that
read-your-write behavior will succeed. This also means that writes
will not necessarily try all nodes in the cluster as it would
previously, which really wasn't desirable anyway. (If you really want
that, you can set request_node_count to a really big number, but
understand that also means reads will contact every node looking for
something that might not exist.)

* Added a request_node_count proxy-server conf value that allows
  control of how many nodes are contacted for a normal request.

In proxy.controllers.base.Controller:

* Got rid of error_increment since it was only used in one spot by
  another method and just served to confuse.

* Made error_occurred also log the device name.

* Made error_limit require an error message and also documented a bit
  better.

* Changed iter_nodes to just take a ring and a partition and yield
  all the nodes itself so it could control the number of nodes used
  in a given request. Also happens to consolidate where sort_nodes is
  called.

* Updated account_info and container_info to use all nodes from
  iter_nodes and to call error_occurred appropriately.

* Updated GETorHEAD_base to not track attempts on its own and just
  stop when iter_nodes tells it to stop. Also, it doesn't take the
  nodes to contact anymore; instead it takes the ring and gets the
  nodes from iter_nodes itself.

Elsewhere:

* Ring now has a get_part method.

* Made changes to reflect all of the above.

Change-Id: I37f76c99286b6456311abf25167cd0485bfcafac
2013-04-08 20:48:32 +00:00
David Goetz
35d2cec919 Bug in SLO with multipart-manifest=get content type.
In trying to override the content-type for the convenience
call I broke COPYs and POSTs for SLO. yaaa...

Change-Id: Ifbcda6d2dd0ee43e43d62e58a90301e7afd05e27
2013-04-08 11:22:20 -07:00
Chmouel Boudjnah
c687f6956c Allow all headers requested for CORS.
- We allow all headers requested in preflight request. The CORS
  specification does leave the door open for this, as mentioned in
  http://www.w3.org/TR/cors/#resource-preflight-requests
  Note: Since the list of headers can be unbounded
  simply returning headers can be enough.
- This is a followup to review:
  https://review.openstack.org/#/c/24415/.
- Fixes bug 1155034.

Change-Id: If7b8f2f3a581c5209892d1ccc9f06ddb8fac92dd
2013-04-05 21:12:15 +01:00
gholt
1cb952a958 Allow a configurable set of TempURL methods
Folks have actually been asking for this. I think they're sending a
DELETE TempURL to someone way ahead of time and the someone issues it
when they're ready. Honestly, I'm not entirely sure of the use case,
but having the set of methods configurable wouldn't hurt.

Change-Id: Ibdb48f8a72077b045eeedddfae4c0a1f56098d7a
2013-04-04 20:37:23 +00:00
Greg Lange
30e88fd676 add unit tests for db_replicator
Change-Id: I9002fa193a51f40523e7936e3117a2f3f2b2f7f8
2013-04-04 18:45:24 +00:00
Jenkins
ab355e349a Merge "Fix reading xattrs in object-server's unittests." 2013-04-03 20:55:06 +00:00
Jenkins
c5408926c6 Merge "SLO multipart-manifest=get returns incorrect Content-Type" 2013-04-03 18:47:18 +00:00
Jenkins
01a5a47610 Merge "Dynamic Large Object sleeps too much" 2013-04-02 07:27:10 +00:00
David Goetz
3a8f314fdc SLO multipart-manifest=get returns incorrect Content-Type
Change-Id: I657c9f7a7555d90ccfa7e6ff0f5f3fcbd7eada1d
2013-04-01 13:16:46 -07:00
David Goetz
7c0b9a7424 Dynamic Large Object sleeps too much
DLO is supposed to sleep 1 second per segment after 10 (default) initial
segments are loaded to prevent abuse of the system. Currently, if the user
did a range request to the 11th segment the 1 sec delay would start immediately
on the 11th segment because the skipped segments were counted as initial
segments. This path fixes that.

Change-Id: I53bbc24af54921ddc04d9cb3b704366b51b3f740
2013-04-01 10:19:35 -07:00
Greg Lange
44f00a23c1 fixed some minor things in tests that pyflakes complained about
Change-Id: Ifeab56a964630bcf941e932fcbe39e6572e62975
2013-03-26 20:42:26 +00:00
Leah Klearman
60489cdedd additional unit test coverage for container/server.py
Change-Id: Id510cceb88f062f921450a5d25d7c97b7ca5943b
2013-03-22 19:49:47 -07:00
David Hadas
a979c8007b Add support for Hash Prefix
A new configuration parameter is added to /etc/swift/swift.conf
[swift-hash]
swift_hash_path_prefix = 'random unique string'

New installations are advised to set this parameter to a random secret,
which would not be disclosed ouside the organization.
The same secret needs to be used by all swift servers of the same cluster.

Existing installations should set this parameter to an empty string
(the default)

DocImpact

Fixes: Bug #1157454

Change-Id: I63b10d0b7d6dd3f74e0f10bb41b5f240fa03578a
2013-03-22 19:41:55 +02:00
Vladimir Vechkanov
9e3d2f6ea8 Fix reading xattrs in object-server's unittests.
Use for reading metadata in unit tests function from object-server.

Change-Id: I2bfeb76fdd775442a0e614fef740b0987fba4a22
Fixes: bug #1079131
2013-03-22 17:02:13 +04:00
Jenkins
37e96663ec Merge "Fix for unicode issues in staticweb" 2013-03-21 22:48:30 +00:00
Jenkins
1eb08bfd29 Merge "Fix bugs in bulk and slo and small doc change." 2013-03-21 18:52:59 +00:00
David Goetz
92e877ae51 Fix bugs in bulk and slo and small doc change.
Added 411 responses in bulk and slo when needed.

Make X-Static-Large-Object an illegal header with slo installed- somehow
that got lost with some refactor I did.

Change-Id: I986c1656658f874172860469624118cc63bff9bc
2013-03-21 09:56:54 -07:00
Donagh McCabe
7df4323745 Fix for attempted COPY of objects gt MAX_FILE_SIZE
If X-Copy-From is a manifest object, it can be bigger than MAX_FILE_SIZE.
Currently this fails with 503. It should be 413.

Bug #1158279

Change-Id: I7ec16088046c84e42d5be9c865e1338eb07845b6
2013-03-21 13:05:31 +00:00
Eohyung Lee
05c8d61aeb Fix for unicode issues in staticweb
When prefix or subdir is unicode, it fails.
Fixes: bug #1158092

Change-Id: I966a7a198a5ca5ea152872c94240e8c2ebe57d36
2013-03-21 11:35:11 +09:00
Jenkins
18c03d9aa7 Merge "increase unit test coverage on account server" 2013-03-20 18:39:20 +00:00
Jenkins
fc27698fd4 Merge "fix stupid bug with bulk delete max items" 2013-03-20 16:42:13 +00:00
David Goetz
1b150749f7 fix stupid bug with bulk delete max items
Change-Id: I52160b5e7480112d536c45dd4b0b10a808adcdcd
2013-03-19 11:48:14 -07:00
Kun Huang
7dd966192a Use role in insensitive case in keystoneauth.
Using insensitive case could tolerate human error. For example,
user maybe set like this "operator_roles = Admin, swiftoperator"

- also fix a mistake in test, ['admin'] is correct value for roles, not
  'admin' (it will be looped as ['a', 'd', 'm', 'i', 'n'])
- add test for insensitive cases

Fixes: bug #1013120
Change-Id: I56d71da8bc503e48e92dd743692ba6fc237f029e
2013-03-19 16:26:22 +08:00
Jenkins
85b7346808 Merge "Improved container-sync resiliency" 2013-03-19 07:20:57 +00:00
Leah Klearman
a5ba0d5edc increase unit test coverage on account server
Change-Id: Id728a17180f4ee2e31edaef0af6d26446b6014c2
2013-03-18 18:01:16 -07:00
Jenkins
157d5c7d49 Merge "Basic ring builder validation." 2013-03-19 00:27:31 +00:00
Jenkins
a0d5d7f725 Merge "Permit functional tests against Swift with Keystone" 2013-03-18 22:03:22 +00:00
Jenkins
7b03ca15b2 Merge "Fix crash in swift-ring-builder's list_parts command." 2013-03-18 18:25:51 +00:00
Samuel Merritt
d42a78a3aa Basic ring builder validation.
This prevents people from creating bogus ring builder files.

Example: "swift-ring-builder object.builder create 33 0.9 -4".

Fixes bug 924577.

Change-Id: I7bfc04f7fa5f55f70a4eaae96c414f6b2872e283
2013-03-18 09:50:45 -07:00
Kun Huang
5c3f75ba6a Add x-remove-versions-location for feature: disable versioning
We can set x-versions-location empty to remove this header in API, but
to keep consistent, we should allow x-remove-versions-location too. The
usage is post "x-remove-versions-locaion: x", just like ACL remove
headers.

Fixed: bug #1107592

Change-Id: I1271eec6401d4a0e8c1a7c2d63aeb8dfef00bf6d
2013-03-17 07:30:00 +08:00
Jenkins
94defcbc53 Merge "Add tests and doc entry for request.environ[reseller_request]" 2013-03-16 19:29:47 +00:00
Samuel Merritt
229ba53a19 Fix crash in swift-ring-builder's list_parts command.
If you run list_parts against a builder that has never been
rebalanced, you'd get a crash. Now you don't.

To reproduce:

$ swift-ring-builder foo.builder create 8 3 1
$ swift-ring-builder foo.builder add r1z1-1.2.3.4:6000/sda 100
$ swift-ring-builder foo.builder list_parts z1

Change-Id: Ic3edffab0c5c2e9551a2f89ddb881153f0b07db7
2013-03-14 18:49:30 -07:00
Jenkins
b115356af6 Merge "Remove check for valid Origin for the "actual request"." 2013-03-14 11:29:17 +00:00
David Goetz
f6e29b81d0 Remove check for valid Origin for the "actual request".
The only place in the spec that I could see the Origin being checked was
during the pre-flight OPTIONS request. If it gets to the actual request
let auth decide. Please correct me if this is wrong.

Change-Id: Ic31b71746ec056091c7778ebff3db7becc32bd9c
2013-03-13 13:43:19 -07:00