5224 Commits

Author SHA1 Message Date
John Dickinson
999479f9b1 Bump eventlet min version to 0.17.4
IPv6 support in Swift is dependent on IPv6 support in eventlet.
eventlet itself only claims support for IPv6 post v0.17
(https://github.com/eventlet/eventlet/issues/8). This update matches
the OpenStack global requirements version.

Change-Id: I9d8433cdd3bf7d7a93b8f50b991cc21721a80d22
2016-01-19 10:30:30 -08:00
Jenkins
b9fd530657 Merge "Sleep enough for trampoline" 2016-01-19 17:01:43 +00:00
Jenkins
6a473e3d7b Merge "Imported Translations from Zanata" 2016-01-18 16:46:30 +00:00
Jenkins
b46209e7a0 Merge "Allow to change auditor sleep interval in config" 2016-01-18 12:19:08 +00:00
OpenStack Proposal Bot
47e226418b Imported Translations from Zanata
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: Ic416c9afc8a1c76f552803a7c70fc905eda7b3cb
2016-01-18 06:20:14 +00:00
Jenkins
4db7e2e2e4 Merge "Speed up get_more_nodes() when there is an empty zone" 2016-01-16 12:30:15 +00:00
Jenkins
5b35235200 Merge "Drop python 2.6 support" 2016-01-15 23:48:36 +00:00
Jenkins
f138535854 Merge "Remove some Python 2.6 leftovers" 2016-01-15 23:34:42 +00:00
Jenkins
7c94b6bfe1 Merge "Show UTC time in swift-recon." 2016-01-15 06:03:21 +00:00
Samuel Merritt
5d449471b1 Remove some Python 2.6 leftovers
Change-Id: I798d08722c90327c66759aa0bb4526851ba38d41
2016-01-14 17:26:01 -08:00
Samuel Merritt
3c0cf549f1 Speed up get_more_nodes() when there is an empty zone
The ring has some optimizations in get_more_nodes() so that it can
find handoffs that span all the regions/zones/et cetera and then stop
looking. The stopping is the important part.

Previously, it would quickly find a handoff in each unused region,
then spend way too long looking for more unused regions; the same was
true for zones, IPs, and so on. Thus, in commit 9cd7c6c, we started
counting regions and zones, then stopping when we found them all.

This count included all regions and zones in the ring, regardless of
whether or not there were actually any parts assigned or not. In rings
with an empty region, i.e. a region for which there are only
zero-weight devices, get_more_nodes() would be very slow.

This commit ignores devices with no assigned partitions when counting
regions, zones, and so forth, thus greatly speeding things up.

The output of get_more_nodes() is unchanged. This is purely an
optimization.

Closes-Bug: 1534303

Change-Id: I4a5c57205e87e1205d40fd5d9458d4114e524332
2016-01-14 13:50:19 -08:00
keliang
70047709fc Drop python 2.6 support
Change-Id: Id6329c863dacb189fccfc304453ed7b6f9607c14
2016-01-15 00:31:51 +08:00
Jenkins
8460ddd607 Merge "Make object-auditor storage-policy-aware" 2016-01-14 12:57:57 +00:00
Ondřej Nový
a4c2fe95ab Allow to change auditor sleep interval in config
Change-Id: Ic451c5e0b686509f8982ed1bf65a223a2d77b9a0
2016-01-14 12:52:52 +01:00
Ondřej Nový
edc823e803 Show UTC time in swift-recon.
It's not consistent now for example local time in replication part and
UTC time at begging of line. Use _ptime in swift-recon for all time
printing and this function returns UTC now.
Change-Id: I732d9851db157130a08e825e8093b7e244b63e9c
2016-01-14 12:49:09 +01:00
Tim Burke
fa5b32d279 Make object-auditor storage-policy-aware
Previously, the object-auditor would always use a (replication)
DiskFileManager when walking through AuditLocations, which would cause
it to skip EC fragment archives with a warning like:

    Unexpected file <hash_path>/1452557187.03610#3.data:
    Invalid Timestamp value in filename '1452557187.03610#3.data'

Now, the AuditLocation's policy will be used to find an appropriate
manager to get the diskfile. Additionally, several .commit()s were added
to the auditor tests so the .durable will be written, which is required
when auditing EC fragment archives.

Change-Id: I960e7d696fd9ad704ca1872b4ac821f9078838c7
Closes-Bug: 1533002
2016-01-13 12:48:59 -08:00
Jenkins
f7c8a1834e Merge "Functional tests for if-match with multiple etags" 2016-01-13 18:01:27 +00:00
Jenkins
dd405d9dd4 Merge "Fix ClientException handling in Container Sync" 2016-01-13 14:44:00 +00:00
Jenkins
5961c5f7fb Merge "fixups for ipv6 memcache_servers docs" 2016-01-13 13:41:34 +00:00
Jenkins
00a4e2ce4b Merge "Install liberasurecode packages in SAIO." 2016-01-13 13:38:29 +00:00
Clay Gerrard
3347646023 fixups for ipv6 memcache_servers docs
Change-Id: I20d91c1e276014eaf210fa9eb43788bc17f4e8df
2016-01-12 21:08:58 -08:00
Jenkins
a66e5733dd Merge "Print min_part_hours lockout time remaining" 2016-01-13 03:07:26 +00:00
Jenkins
3077639d3f Merge "swift-init: New option kill-after-timeout" 2016-01-13 02:53:04 +00:00
Samuel Merritt
d5ff5447be Install liberasurecode packages in SAIO.
Change-Id: If673afa2b61a3e388612debf4860d561960963a3
2016-01-12 16:42:06 -08:00
Jonathan Hinson
4ffc4ba411 Functional tests for if-match with multiple etags
Multiple etags can be provided on an if-match or if-none-match
request. This is currently being tested in the unit tests, but not
in the functional tests. Since these etags can be modified by
middleware, we need functional tests to assert multiple-etag
requests are handled correctly.

Change-Id: Idc409c85e8aa82b59dc2bc28af6ca2617de82699
2016-01-12 14:26:38 -06:00
Jenkins
cf94d36bbf Merge "Add note COPY with conditional headers" 2016-01-12 13:54:31 +00:00
Ben Martin
1f3304c515 Print min_part_hours lockout time remaining
swift-ring-builder currently only displays min_part_hours and
not the amount of time remaining before a rebalance can occur.
This information is readily available and has been displayed
as a quality of life improvement.

Additionally, a bug where the time since the last rebalance
was always updated when rebalance was called regardless of
if any partitions were reassigned. This can lead to partitions
being unable to be reassigned as they never age according to
the time since last rebalance.

Change-Id: Ie0e2b5e25140cbac7465f31a26a4998beb3892e9
Closes-Bug: #1526017
2016-01-11 10:58:38 -06:00
Timur Alperovich
167bb5eeb8 Fix IPv6 handling in MemcacheConnPool.
The patch removes the assumption of IPv4-only addresses in the
MemcacheConnPool. The changes are around address handling.
Namely, if a server is specified with an address
[<address>]:port (port is optional), it is assumed to be an IPv6
address [1]. If an IPv6 address is specified without "[]", an exception
is raised as it is impossible to parse such addresses correctly.

For testing, memcache can be configured to listen on the link-local,
unique-local, or ::1 (equivalent to 127.0.0.1) addresses. Link-local
addresses are assigned by default to each interface and are of the form
"fe80::dead:beef". These addresses require a scope ID, which would look
like "fe80::dead:beef%eth0" (replacing eth0 with the correct interface).

Unique-local addresses are any addresses in the fc00::/7 subnet. To add
a ULA to an interface use the "ip" utility. For example:
"ip -6 address add fc01::dead:beef dev eth0". Lastly, and probably
simplest, memcache can be configured to listen on "::1". The same
address would be used in the swift configuration, e.g. "[::1]:11211".

Note: only memcached version 1.4.25 or greater supports binding to an
IPv6 address.

Fixes #1526570

[1] IPv6 host literals:
https://tools.ietf.org/html/rfc3986#section-3.2.2

Change-Id: I8408143c1d47d24e70df56a08167c529825276a2
2016-01-08 17:15:05 -08:00
Kota Tsuyuzaki
b35f3c90bd Add note COPY with conditional headers
Swift now supports Range header for COPY (or PUT with X-Copy-From)
to make a partial copied object. This patch adds the note to show
it obviously supported in Swift community.

Change-Id: I6bf28f0932c90e7b305cd61aabce4ed028ae691e
Partial-Bug: #1532126
2016-01-08 01:29:11 -08:00
Hisashi Osanai
23c7a58f8f Fix ClientException handling in Container Sync
swift/container/sync.py uses swift.common.internal_client.delete_object
and put_object and expected these methods raise ClientException.
But delete_object and put_object never raise the exception so this patch
raises ClientException when urllib2 library raises HTTPError.

Co-Authored-By: Eran Rom <eranr@il.ibm.com>
Closes-Bug: #1419901
Change-Id: I58cbf77988979a07998a46d9d81be84d29b0d9bf
2016-01-07 10:17:20 +00:00
Harshada Mangesh Kakad
6786cdf036 Fixing the deprecated library function.
os.popen() is deprecated since version 2.6. Resolved with use of
subprocess module.

Change-Id: I4409cdd9edbc1a26d6f99c125c9100fadda5d758
Partial-Bug: #1529836
2016-01-06 09:26:38 -08:00
Ondřej Nový
f53cf1043d Fixed few misspellings in comments
Change-Id: I8479c85cb8821c48b5da197cac37c80e5c1c7f05
2016-01-05 20:20:15 +01:00
Ondřej Nový
3b1591f235 swift-init: New option kill-after-timeout
This option send SIGKILL to daemon after kill_wait period.
When daemon hangs and doesn't respond to SIGTERM/SIGHUP
there is no way to stop it using swift-init now. Classic
init scripts in Linux kills hanged process after grace
period and this patch add same behaviour. This is most
usefull when using "restart" on hanged daemon.

Change-Id: I8c932b673a0f51e52132df87ea2f4396f4bba9d8
2016-01-05 11:34:03 +01:00
Jenkins
02ecd51b90 Merge "Fix AttributeError for LogAdapter" 2016-01-05 09:15:42 +00:00
Jenkins
c38ca6329f Merge "Port swift.common.utils.StatsdClient to Python 3" 2016-01-05 09:14:51 +00:00
ChangBo Guo(gcb)
79222e327f Fix AttributeError for LogAdapter
LogAdapter object has no attribute 'warn' but has attribute
'warning'.

Closes-Bug: #1529321
Change-Id: I0e0bd0a3dbc4bb5c1f0b343a8809e53491a1da5f
2015-12-26 13:17:00 +08:00
Jenkins
53c4735b02 Merge "Python 3 deprecated the logger.warn method in favor of warning" 2015-12-23 17:26:18 +00:00
Jenkins
d446035ec4 Merge "Add support for storage policies to have more than one name" 2015-12-23 00:45:59 +00:00
janonymous
684c4c0459 Python 3 deprecated the logger.warn method in favor of warning
DeprecationWarning: The 'warn' method is deprecated, use 'warning'
instead

Change-Id: I35df44374c4521b1f06be7a96c0b873e8c3674d8
2015-12-22 22:11:29 +05:30
Jenkins
2087dedc95 Merge "Fix duplication for headers in Access-Control-Expose-Headers" 2015-12-22 05:13:14 +00:00
Hisashi Osanai
d0a026fcb8 Fix duplication for headers in Access-Control-Expose-Headers
There are following problems with Access-Control-Expose-Headers.

* If headers in X-Container-Meta-Access-Control-Expose-Headers are
  configured, the headers are kept with case-sensitive string.
  Then a CORS request comes, the headers are merged into
  Access-Control-Expose-Headers as case-sensitive string even if
  there is a same header which is not case-sensitive string.

* Access-Control-Expose-Headers is handled by a list.
  If X-Container/Object-Meta-XXX is configured in container/object
  and X-Container-Meta-Access-Control-Expose-Headers, same header
  is listed in Access-Control-Expose-Headers.

This patch provides a fix for the problems.

Change-Id: Ifc1c14eb3833ec6a851631cfc23008648463bd81
2015-12-21 16:16:59 +09:00
Jenkins
cb8962fbdd Merge "Make ECDiskFile report all fragments found on disk" 2015-12-19 17:15:47 +00:00
Jenkins
dafeea6322 Merge "Fix incorrect kwarg in auth middleware example" 2015-12-18 22:04:47 +00:00
Jenkins
9b4bab707c Merge "Deprecated param timeout removed from memcached" 2015-12-18 22:04:39 +00:00
Jenkins
a5ce170c80 Merge "Update Erasure Coding Overview doc to remove Beta version" 2015-12-18 22:02:57 +00:00
Jenkins
e2bb995fec Merge "Fix rst errors so that html docs are complete" 2015-12-18 18:10:01 +00:00
Bill Huber
0bcd7fd50e Update Erasure Coding Overview doc to remove Beta version
The major functionality of EC has been released for Liberty and
the beta version of the code has been removed since it is now
in production.

Change-Id: If60712045fb1af803093d6753fcd60434e637772
2015-12-18 11:43:12 -06:00
Jenkins
184f797f58 Merge "Not used parameter" 2015-12-18 17:26:43 +00:00
Alistair Coles
84ba24a756 Fix rst errors so that html docs are complete
rst table format errors don't break the gate job
but do cause sections of the documents to go missing
from the html output.

Change-Id: Ic8c9953c93d03dcdafd8f47b271d276c7b356dc3
2015-12-18 11:24:34 +00:00
Kota Tsuyuzaki
9fe0e25604 Sleep enough for trampoline
When running unite test suite in local poor resource environment,
sometimes test/unit/proxy/test_server.py fails due to a lack of
waiting time to trampoline of eventlet thread.

This patch enables to sleep 1 more second when it doesn't seem
to have enough time to tranpoline.

Change-Id: I0bbc8fc245919d3c0a071ff87ff6e20b8d58f9b8
2015-12-18 09:56:03 +00:00