4365 Commits

Author SHA1 Message Date
Alistair Coles
4ca08cc395 Update guest VM OS recommendation in SAIO doc
The target development platform has changed to Ubuntu 14.04 [1].
This patch makes the suggested SAIO platform the same.

Also, remove pointer to wiki page for other platform install
instructions that either redirects back to this SAIO doc or
to another wike page and then a dead link.

[1] I0a96bcf692bb240f3ab5aab7fefd294a07735a83

DocImpact

Change-Id: I9f96104b5437c1f1f28f924c048ef83cf03338f4
2015-02-19 17:46:10 +00:00
Jenkins
a6091c0f39 Merge "Promote some of the best developers I know to CORE Emeritus" 2015-02-18 19:55:33 +00:00
Jenkins
7572b00abd Merge "update the getting started doc" 2015-02-18 11:30:25 +00:00
Jenkins
65d6b93b70 Merge "more probe test refactoring" 2015-02-18 00:55:45 +00:00
John Dickinson
949804eda4 update the getting started doc
Change-Id: I0a96bcf692bb240f3ab5aab7fefd294a07735a83
2015-02-17 13:24:33 -08:00
Jenkins
bf0e01c488 Merge "Add functional tests for container TempURLs" 2015-02-16 09:53:32 +00:00
OpenStack Proposal Bot
dd1a05f527 Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I013976c6192a8bff891c9050f829ae7a1e2fec59
2015-02-16 06:30:54 +00:00
John Dickinson
7acc291129 added swift_source to ratelimit info calls
Change-Id: I2b4ccb809c0f4505098e261f546f53c27440dd43
2015-02-15 17:14:31 -08:00
Richard Hawkins
023e29fa1e Add functional tests for container TempURLs
Change-Id: I7f54594df1522d72dd1d13556be0b9d33e811b30
2015-02-14 10:05:19 -06:00
Leah Klearman
ca0fce8542 more probe test refactoring
* move get_to_final_state into ProbeTest
* get rid of kill_servers
* add replicators manager and updaters manager to ProbeTest

(this is all going someplace, i promise)

Change-Id: I8393a2ebc0d04051cae48cc3c49580f70818dbf2
2015-02-13 16:55:45 -08:00
Clay Gerrard
2655465d82 Promote some of the best developers I know to CORE Emeritus
Change-Id: I4095bdbb2c137615bf9492e373a837daadea8920
2015-02-13 13:11:40 -08:00
Jenkins
9196c31f64 Merge "Imported Translations from Transifex" 2015-02-13 19:16:05 +00:00
Jenkins
af2ec5c634 Merge "Make container GET call authorize when account not found" 2015-02-13 19:15:57 +00:00
Jenkins
af9555be4b Merge "Minor cleanup post efficent multi-region replication" 2015-02-13 18:53:52 +00:00
Jenkins
6d7b068a95 Merge "move test comments around" 2015-02-13 18:53:44 +00:00
Jenkins
a10e42d910 Merge "Fix logger test coupling" 2015-02-13 18:53:35 +00:00
Jenkins
3c37c7637b Merge "Fix account-reaper" 2015-02-13 18:11:44 +00:00
Clay Gerrard
596042b9c7 Minor cleanup post efficent multi-region replication
One log line had a typo, and I refactored the per object cleanup code out of
update_deleted into the per object hashdir cleanup method.

Change-Id: I19d03d0706a75bd8ec2fe327a1eb1b5ec36de6d2
2015-02-13 08:04:56 -08:00
OpenStack Proposal Bot
b8e85a42ec Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I7655ef8e64b6700be9b8aaa6387c36de32a2d7b8
2015-02-13 06:10:44 +00:00
Leah Klearman
a4ffd1d1c6 move test comments around
should make later refactoring easier

Change-Id: I7af399a14c8bc78fcfc438e4440d2f023c8aa5db
2015-02-12 20:36:12 -08:00
Jenkins
923e17b2b8 Merge "Logging a policy index when container PUT request conflicts" 2015-02-13 01:50:53 +00:00
Jenkins
28c99763e9 Merge "Fix ssync send_delete" 2015-02-13 00:18:38 +00:00
Jenkins
8ff2f4378b Merge "Efficient Replication for Distributed Regions" 2015-02-13 00:18:12 +00:00
Jenkins
677811cb3d Merge "refactor probe tests" 2015-02-13 00:18:03 +00:00
Jenkins
947f979dee Merge "Show each policy's information on quarantined files in recon" 2015-02-13 00:17:53 +00:00
Jenkins
f2e2326b5f Merge "Update TempURL docs to include containers" 2015-02-12 22:32:07 +00:00
Alistair Coles
82e5090848 Fix ssync send_delete
The ssync_sender send_delete method treats its
timestamp argument as a string when in fact it is
passed a Timestamp object. As a result the method
always raises an exception and deletes are never
replicated.

This patch fixes bug and adds unit and probe tests
to verify expected behavior.

Closes-Bug: 1421425

Change-Id: I664fb8d5dfea7362313037a67927ea90021c3f62
2015-02-12 21:44:36 +00:00
Samuel Merritt
a9b5982d52 Fix account-reaper
As part of commit efb39a5, the account reaper grew a bind_port
attribute, but it wasn't being converted to int, so naturally "6002"
!= 6002, and it wouldn't reap anything.

The bind_port was only used for determining the local devices. Rather
than fix the code to call int(), this commit removes the need for
bind_port entirely by skipping the port check. If your rings have IPs,
this is the same behavior as pre-efb39a5, and if your rings have
hostnames, this still works.

Change-Id: I7bd18e9952f7b9e0d7ce2dce230ee54c5e23709a
2015-02-12 13:28:29 -08:00
Leah Klearman
2c1b5af062 refactor probe tests
* refactor probe tests to use probe.common.ProbeTest
* move reset_environment functionality to ProbeTest.setUp()
* choose rings and policies that meet the criteria - raise SkipTest if
nothing matches
* replace all AssertionErrors in setup with SkipTest

Change-Id: Id56c497d58083f5fd55f5283cdd346840df039d3
2015-02-12 11:30:21 -08:00
Richard Hawkins
f5b533fb2f Update TempURL docs to include containers
Change-Id: Ifb7c26b23ec81e3bde96b6e3bad0e950c0ca9408
2015-02-12 11:27:31 -06:00
Alistair Coles
f5ab30574e Fix logger test coupling
Two tests fail if test/unit/common/test_daemon.py is excluded
from the test set:

test.unit.common.test_utils.TestUtils.test_swift_log_formatter
test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals

They fail because they assert that logger.txn_id is None,
when previous tests cause txn_id to be set. This coupling is masked
when test_daemon.py is included because it reloads the common/utils
module, and is executed just before test.unit.common.test_utils.

The coupling can be observed by running, for example:

nosetests ./test/unit/common/middleware/test_except.py ./test/unit/common/test_utils.py
or
nosetests ./test/unit/proxy ./test/unit/common/test_utils.py

The failing tests should reset logger thread_locals before making
assertions. test_utils.reset_loggers() attempts this but is broken
because it sets thread_local on the wrong object.

Changes in this patch:
 - fix reset_loggers() to reset the LogAdapter thread local state
 - add a reset_logger_state decorator to call reset_loggers
   before and after a decorated method
 - use the decorator for increased isolation of tests that previously
   called reset_loggers only on exit

Change-Id: If9aa781a2dd1929a47ef69322ec8c53263d47660
2015-02-12 15:32:25 +00:00
Clay Gerrard
ca90be414e Add developer tools section to associated projects
vagrant-swift-all-in-one is being used and maintained by a number of swift
developers, it has an open source license.

The ansible playbook project serves a similar goal but it's based on a Fedora
distribution and includes Swift-on-File support.

Drive-by fix for the Swift-on-File link which has migrated to stackforge.

Change-Id: Id7478d58adcead57cf56ac4e1d05c6556c8c9b7b
2015-02-11 16:43:44 -08:00
Daisuke Morita
f5d509471b Logging a policy index when container PUT request conflicts
This is a continuing work of patch afdbf73. This patch enables
proxy-server to log a policy index when container PUT request
conflicts with existing container's policy index.

Change-Id: I6d40044c510632a0f61b817a9af2f6c13a721d39
Implements: blueprint logging-policy-number
2015-02-12 05:12:56 +09:00
David Goetz
69797efcbb small bug with account to account copy
Change-Id: I7fdf432666f7640082e9839b9dc3d7af94a0715f
2015-02-11 11:37:31 -08:00
Donagh McCabe
843ce7e301 API document update for container-level tempurl/formpost
Tempurl and Formpost now support secret keys at the container
level [1]. Adds these headers:

    X-Container-Meta-Temp-Url-Key
    X-Container-Meta-Temp-Url-Key-2

DocImpact -- because headers need to be added to [2]

[1] https://review.openstack.org/#/c/154293
[2] http://developer.openstack.org/api-ref-objectstorage-v1.html

Change-Id: I157cd4a540dcc4a9b45bc3d806df152ab56e2354
2015-02-11 12:02:38 +00:00
Jenkins
6e898aa3a7 Merge "Add support for container TempURL Keys" 2015-02-11 01:29:20 +00:00
Jenkins
0186194e96 Merge "Output logs of policy index" 2015-02-10 22:21:13 +00:00
Richard Hawkins
489dd5ff5d Add support for container TempURL Keys
Change-Id: Ic22b0b84b657e6cac7e0062fa410eefb09bc0f4d
Co-Authored-By: Christian Schwede <christian.schwede@enovance.com>
2015-02-10 21:37:10 +00:00
Kota Tsuyuzaki
20ca279d74 Efficient Replication for Distributed Regions
This change provides a efficient way of replication
between regions of a global distributed cluster.

This approach makes object-replicator to push replicas
to a primary node in a remote region, then, to skip
pushing them to next primary node in the region with
expecting asynchronous replication.

This implementation includes a couple of changes on
ssync_sender to allow object-replicator to delete local
handoff objects correctly. One is to return a list of existing
objects in remote region. The list includes local paths of the
objects which exist both on the local device and the remote device.
The other is supporting existence check for specified objects.
It requires the object list build by the first change. When
the object list is given, ssync_sender does only missing_check
based on the list. These changes are needed because current
swift can not handle the existence check in object-level.

Note that this feature will work partially (i.e. only when
primary-to-primary) with rsync.

Implements: blueprint efficient-replication
Change-Id: I5d990444d7977f4127bb37f9256212c893438df1
2015-02-10 12:52:15 -08:00
Jenkins
00d059c9be Merge "Included sysmeta in the object info" 2015-02-10 04:51:39 +00:00
Jenkins
bc7c496f71 Merge "Allow hostnames for nodes in Rings" 2015-02-10 04:32:38 +00:00
Janie Richling
f96b8e412d Included sysmeta in the object info
The cached info object dict did not include
the sysmeta.  This patch fixes that, and adds
a unit test.

Change-Id: I092200e76586af322ed4ff7d194a1034b1ca0433
2015-02-09 18:28:22 -06:00
Alistair Coles
a82bfb25ba Make container GET call authorize when account not found
When an account was not found, ContainerController would
return 404 unconditionally for a container GET or HEAD request,
without checking that the request was authorized.

This patch modifies the GETorHEAD method to first call any
callback method registered under 'swift.authorize' in the
request environ and prefer any response from that over the 404.

Closes-Bug: 1415957

Change-Id: I4f41fd9e445238e14af74b6208885d83698cc08d
2015-02-09 18:08:15 +00:00
OpenStack Proposal Bot
b54532ca05 Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I4b8411f84784a2825a24946c51f4c08ad01febc2
2015-02-06 06:10:34 +00:00
John Dickinson
b45b83fb00 Correct the config default for delay_auth_decision
Updated proxy-server.conf-sample with the correct default. Also
updated the note on the overview-auth doc page.

Change-Id: I5cd62a7a118a28f7b58f47b8d8d4d963f6bc7347
2015-02-05 11:52:41 -08:00
Jenkins
e7ef1dd5a8 Merge "Remove the X-Newest pre-flight request on X-Timestamp" 2015-02-05 16:40:55 +00:00
Jenkins
a919578fae Merge "Tests for the base class of storage nodes" 2015-02-05 16:39:52 +00:00
Jenkins
e7e49f6b37 Merge "Update auth_token section in documentation" 2015-02-05 15:30:51 +00:00
Mahati Chamarthy
74563ece47 Tests for the base class of storage nodes
Change-Id: I8866e2360e30a239d5f6b4a5ed92344291184c2a
2015-02-05 00:29:36 +05:30
Nicolas Trangez
b5f8c594f9 Add swift-scality-backend to associated projects
Change-Id: I7fd56c7cf5b7634224b8a2876258cf1f6be447f1
2015-02-04 16:54:46 +01:00