openstack/swift
Code Issues Proposed changes

tempurl: Make the digest algorithm configurable

Browse Source 
... and add support for SHA-256 and SHA-512 by default. This allows us
to start moving toward replacing SHA-1-based signatures. We've known
this would eventually be necessary for a while [1], and earlier this
year we've seen SHA-1 collisions [2].

Additionally, allow signatures to be base64-encoded, provided they start
with a digest name followed by a colon. Trailing padding is optional for
base64-encoded signatures, and both normal and "url-safe" modes are
supported. For example, all of the following SHA-1 signatures are
equivalent:

   da39a3ee5e6b4b0d3255bfef95601890afd80709
   sha1:2jmj7l5rSw0yVb/vlWAYkK/YBwk=
   sha1:2jmj7l5rSw0yVb/vlWAYkK/YBwk
   sha1:2jmj7l5rSw0yVb_vlWAYkK_YBwk=
   sha1:2jmj7l5rSw0yVb_vlWAYkK_YBwk

(Note that "normal" base64 encodings will require that you url encode
all "+" characters as "%2B" so they aren't misinterpretted as spaces.)

This was done for two reasons:

   1. A hex-encoded SHA-512 is rather lengthy at 128 characters -- 88
      isn't *that* much better, but it's something.
   2. This will allow us to more-easily add support for different
      digests with the same bit length in the future.

Base64-encoding is required for SHA-512 signatures; hex-encoding is
supported for SHA-256 signatures so we aren't needlessly breaking from
what Rackspace is doing.

[1] https://www.schneier.com/blog/archives/2012/10/when_will_we_se.html
[2] https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html

Change-Id: Ia9dd1a91cc3c9c946f5f029cdefc9e66bcf01046
Related-Bug: #1733634
This commit is contained in:
Tim Burke 2017-12-05 20:19:37 +00:00
parent 61fe6aae81
commit 5a4d3bdfc4
6 changed files with 328 additions and 82 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
etc/proxy-server.conf-sample
View File

@ -646,6 +646,10 @@ use = egg:swift#tempurl
# whitespace delimited list of header names and names can optionally end with # whitespace delimited list of header names and names can optionally end with
# '*' to indicate a prefix match. # '*' to indicate a prefix match.
# outgoing_allow_headers = x-object-meta-public-* # outgoing_allow_headers = x-object-meta-public-*
#
# The digest algorithm(s) supported for generating signatures;
# whitespace-delimited.
# allowed_digests = sha1 sha256 sha512
# Note: Put formpost just before your auth filter(s) in the pipeline # Note: Put formpost just before your auth filter(s) in the pipeline
[filter:formpost] [filter:formpost]

115
swift/common/middleware/tempurl.py
View File

@ -54,11 +54,19 @@ Client Usage
------------ ------------
To create temporary URLs, first an ``X-Account-Meta-Temp-URL-Key`` To create temporary URLs, first an ``X-Account-Meta-Temp-URL-Key``
header must be set on the Swift account. Then, an HMAC-SHA1 (RFC 2104) header must be set on the Swift account. Then, an HMAC (RFC 2104)
signature is generated using the HTTP method to allow (``GET``, ``PUT``, signature is generated using the HTTP method to allow (``GET``, ``PUT``,
``DELETE``, etc.), the Unix timestamp the access should be allowed until, ``DELETE``, etc.), the Unix timestamp until which the access should be allowed,
the full path to the object, and the key set on the account. the full path to the object, and the key set on the account.
The digest algorithm to be used may be configured by the operator. By default,
HMAC-SHA1, HMAC-SHA256, and HMAC-SHA512 are supported. Check the
``tempurl.allowed_digests`` entry in the cluster's capabilities response to
see which algorithms are supported by your deployment; see
:doc:`api/discoverability` for more information. On older clusters,
the ``tempurl`` key may be present while the ``allowed_digests`` subkey
is not; in this case, only HMAC-SHA1 is supported.
For example, here is code generating the signature for a ``GET`` for 60 For example, here is code generating the signature for a ``GET`` for 60
seconds on ``/v1/AUTH_account/container/object``:: seconds on ``/v1/AUTH_account/container/object``::
@ -82,10 +90,37 @@ Let's say ``sig`` ends up equaling
temp_url_sig=da39a3ee5e6b4b0d3255bfef95601890afd80709& temp_url_sig=da39a3ee5e6b4b0d3255bfef95601890afd80709&
temp_url_expires=1323479485 temp_url_expires=1323479485
For longer hashes, a hex encoding becomes unwieldy. Base64 encoding is also
supported, and indicated by prefixing the signature with ``"<digest name>:"``.
This is *required* for HMAC-SHA512 signatures. For example, comparable code
for generating a HMAC-SHA512 signature would be::
import base64
import hmac
from hashlib import sha512
from time import time
method = 'GET'
expires = int(time() + 60)
path = '/v1/AUTH_account/container/object'
key = 'mykey'
hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = 'sha512:' + base64.urlsafe_b64encode(hmac.new(
key, hmac_body, sha512).digest())
Supposing that ``sig`` ends up equaling
``sha512:ZrSijn0GyDhsv1ltIj9hWUTrbAeE45NcKXyBaz7aPbSMvROQ4jtYH4nRAmm
5ErY2X11Yc1Yhy2OMCyN3yueeXg==`` and ``expires`` ends up
``1516741234``, then the website could provide a link to::
https://swift-cluster.example.com/v1/AUTH_account/container/object?
temp_url_sig=sha512:ZrSijn0GyDhsv1ltIj9hWUTrbAeE45NcKXyBaz7aPbSMvRO
Q4jtYH4nRAmm5ErY2X11Yc1Yhy2OMCyN3yueeXg==&
temp_url_expires=1516741234
You may also use ISO 8601 UTC timestamps with the format You may also use ISO 8601 UTC timestamps with the format
``"%Y-%m-%dT%H:%M:%SZ"`` instead of UNIX timestamps in the URL ``"%Y-%m-%dT%H:%M:%SZ"`` instead of UNIX timestamps in the URL
(but NOT in the code above for generating the signature!). (but NOT in the code above for generating the signature!).
So, the latter URL could also be formulated as:: So, the above HMAC-SHA1 URL could also be formulated as::
https://swift-cluster.example.com/v1/AUTH_account/container/object? https://swift-cluster.example.com/v1/AUTH_account/container/object?
temp_url_sig=da39a3ee5e6b4b0d3255bfef95601890afd80709& temp_url_sig=da39a3ee5e6b4b0d3255bfef95601890afd80709&
@ -199,6 +234,12 @@ This middleware understands the following configuration settings:
Default: ``GET HEAD PUT POST DELETE`` Default: ``GET HEAD PUT POST DELETE``
``allowed_digests``
A whitespace delimited list of digest algorithms that are allowed
to be used when calculating the signature for a temporary URL.
Default: ``sha1 sha256 sha512``
""" """
__all__ = ['TempURL', 'filter_factory', __all__ = ['TempURL', 'filter_factory',
@ -207,7 +248,10 @@ __all__ = ['TempURL', 'filter_factory',
'DEFAULT_OUTGOING_REMOVE_HEADERS', 'DEFAULT_OUTGOING_REMOVE_HEADERS',
'DEFAULT_OUTGOING_ALLOW_HEADERS'] 'DEFAULT_OUTGOING_ALLOW_HEADERS']
import binascii
from calendar import timegm from calendar import timegm
import functools
import hashlib
from os.path import basename from os.path import basename
from time import time, strftime, strptime, gmtime from time import time, strftime, strptime, gmtime
@ -219,7 +263,8 @@ from swift.common.header_key_dict import HeaderKeyDict
from swift.common.swob import header_to_environ_key, HTTPUnauthorized, \ from swift.common.swob import header_to_environ_key, HTTPUnauthorized, \
HTTPBadRequest HTTPBadRequest
from swift.common.utils import split_path, get_valid_utf8_str, \ from swift.common.utils import split_path, get_valid_utf8_str, \
register_swift_info, get_hmac, streq_const_time, quote register_swift_info, get_hmac, streq_const_time, quote, get_logger, \
strict_b64decode
DISALLOWED_INCOMING_HEADERS = 'x-object-manifest x-symlink-target' DISALLOWED_INCOMING_HEADERS = 'x-object-manifest x-symlink-target'
@ -246,6 +291,8 @@ DEFAULT_OUTGOING_REMOVE_HEADERS = 'x-object-meta-*'
#: '*' to indicate a prefix match. #: '*' to indicate a prefix match.
DEFAULT_OUTGOING_ALLOW_HEADERS = 'x-object-meta-public-*' DEFAULT_OUTGOING_ALLOW_HEADERS = 'x-object-meta-public-*'
DEFAULT_ALLOWED_DIGESTS = 'sha1 sha256 sha512'
SUPPORTED_DIGESTS = set(DEFAULT_ALLOWED_DIGESTS.split())
CONTAINER_SCOPE = 'container' CONTAINER_SCOPE = 'container'
ACCOUNT_SCOPE = 'account' ACCOUNT_SCOPE = 'account'
@ -330,6 +377,9 @@ class TempURL(object):
#: The filter configuration dict. #: The filter configuration dict.
self.conf = conf self.conf = conf
self.allowed_digests = conf.get(
'allowed_digests', DEFAULT_ALLOWED_DIGESTS.split())
self.disallowed_headers = set( self.disallowed_headers = set(
header_to_environ_key(h) header_to_environ_key(h)
for h in DISALLOWED_INCOMING_HEADERS.split()) for h in DISALLOWED_INCOMING_HEADERS.split())
@ -401,6 +451,26 @@ class TempURL(object):
return self.app(env, start_response) return self.app(env, start_response)
if not temp_url_sig or not temp_url_expires: if not temp_url_sig or not temp_url_expires:
return self._invalid(env, start_response) return self._invalid(env, start_response)
if ':' in temp_url_sig:
hash_algorithm, temp_url_sig = temp_url_sig.split(':', 1)
if ('-' in temp_url_sig or '_' in temp_url_sig) and not (
'+' in temp_url_sig or '/' in temp_url_sig):
temp_url_sig = temp_url_sig.replace('-', '+').replace('_', '/')
try:
temp_url_sig = binascii.hexlify(strict_b64decode(
temp_url_sig + '=='))
except ValueError:
return self._invalid(env, start_response)
elif len(temp_url_sig) == 40:
hash_algorithm = 'sha1'
elif len(temp_url_sig) == 64:
hash_algorithm = 'sha256'
else:
return self._invalid(env, start_response)
if hash_algorithm not in self.allowed_digests:
return self._invalid(env, start_response)
account, container, obj = self._get_path_parts(env) account, container, obj = self._get_path_parts(env)
if not account: if not account:
return self._invalid(env, start_response) return self._invalid(env, start_response)
@ -415,16 +485,14 @@ class TempURL(object):
path = 'prefix:/v1/%s/%s/%s' % (account, container, path = 'prefix:/v1/%s/%s/%s' % (account, container,
temp_url_prefix) temp_url_prefix)
if env['REQUEST_METHOD'] == 'HEAD': if env['REQUEST_METHOD'] == 'HEAD':
hmac_vals = ( hmac_vals = [
self._get_hmacs(env, temp_url_expires, path, keys) + hmac for method in ('HEAD', 'GET', 'POST', 'PUT')
self._get_hmacs(env, temp_url_expires, path, keys, for hmac in self._get_hmacs(
request_method='GET') + env, temp_url_expires, path, keys, hash_algorithm,
self._get_hmacs(env, temp_url_expires, path, keys, request_method=method)]
request_method='POST') +
self._get_hmacs(env, temp_url_expires, path, keys,
request_method='PUT'))
else: else:
hmac_vals = self._get_hmacs(env, temp_url_expires, path, keys) hmac_vals = self._get_hmacs(
env, temp_url_expires, path, keys, hash_algorithm)
is_valid_hmac = False is_valid_hmac = False
hmac_scope = None hmac_scope = None
@ -589,7 +657,7 @@ class TempURL(object):
return ([(ak, ACCOUNT_SCOPE) for ak in account_keys] + return ([(ak, ACCOUNT_SCOPE) for ak in account_keys] +
[(ck, CONTAINER_SCOPE) for ck in container_keys]) [(ck, CONTAINER_SCOPE) for ck in container_keys])
def _get_hmacs(self, env, expires, path, scoped_keys, def _get_hmacs(self, env, expires, path, scoped_keys, hash_algorithm,
request_method=None): request_method=None):
""" """
:param env: The WSGI environment for the request. :param env: The WSGI environment for the request.
@ -597,6 +665,7 @@ class TempURL(object):
expires. expires.
:param path: The path which is used for hashing. :param path: The path which is used for hashing.
:param scoped_keys: (key, scope) tuples like _get_keys() returns :param scoped_keys: (key, scope) tuples like _get_keys() returns
:param hash_algorithm: The hash algorithm to use.
:param request_method: Optional override of the request in :param request_method: Optional override of the request in
the WSGI env. For example, if a HEAD the WSGI env. For example, if a HEAD
does not match, you may wish to does not match, you may wish to
@ -608,8 +677,9 @@ class TempURL(object):
if not request_method: if not request_method:
request_method = env['REQUEST_METHOD'] request_method = env['REQUEST_METHOD']
digest = functools.partial(hashlib.new, hash_algorithm)
return [ return [
(get_hmac(request_method, path, expires, key), scope) (get_hmac(request_method, path, expires, key, digest), scope)
for (key, scope) in scoped_keys] for (key, scope) in scoped_keys]
def _invalid(self, env, start_response): def _invalid(self, env, start_response):
@ -706,8 +776,23 @@ def filter_factory(global_conf, **local_conf):
'incoming_allow_headers': DEFAULT_INCOMING_ALLOW_HEADERS, 'incoming_allow_headers': DEFAULT_INCOMING_ALLOW_HEADERS,
'outgoing_remove_headers': DEFAULT_OUTGOING_REMOVE_HEADERS, 'outgoing_remove_headers': DEFAULT_OUTGOING_REMOVE_HEADERS,
'outgoing_allow_headers': DEFAULT_OUTGOING_ALLOW_HEADERS, 'outgoing_allow_headers': DEFAULT_OUTGOING_ALLOW_HEADERS,
'allowed_digests': DEFAULT_ALLOWED_DIGESTS,
} }
info_conf = {k: conf.get(k, v).split() for k, v in defaults.items()} info_conf = {k: conf.get(k, v).split() for k, v in defaults.items()}
allowed_digests = set(digest.lower()
for digest in info_conf['allowed_digests'])
not_supported = allowed_digests - SUPPORTED_DIGESTS
if not_supported:
logger = get_logger(conf, log_route='tempurl')
logger.warning('The following digest algorithms are configured but '
'not supported: %s', ', '.join(not_supported))
allowed_digests -= not_supported
if not allowed_digests:
raise ValueError('No valid digest algorithms are configured '
'for tempurls')
info_conf['allowed_digests'] = sorted(allowed_digests)
register_swift_info('tempurl', **info_conf) register_swift_info('tempurl', **info_conf)
conf.update(info_conf) conf.update(info_conf)

13
swift/common/utils.py
View File

@ -237,9 +237,9 @@ except InvalidHashPathConfigError:
pass pass
def get_hmac(request_method, path, expires, key): def get_hmac(request_method, path, expires, key, digest=sha1):
""" """
Returns the hexdigest string of the HMAC-SHA1 (RFC 2104) for Returns the hexdigest string of the HMAC (see RFC 2104) for
the request. the request.
:param request_method: Request method to allow. :param request_method: Request method to allow.
@ -247,11 +247,16 @@ def get_hmac(request_method, path, expires, key):
:param expires: Unix timestamp as an int for when the URL :param expires: Unix timestamp as an int for when the URL
expires. expires.
:param key: HMAC shared secret. :param key: HMAC shared secret.
:param digest: constructor for the digest to use in calculating the HMAC
Defaults to SHA1
:returns: hexdigest str of the HMAC-SHA1 for the request. :returns: hexdigest str of the HMAC for the request using the specified
digest algorithm.
""" """
return hmac.new( return hmac.new(
key, '%s\n%s\n%s' % (request_method, expires, path), sha1).hexdigest() key,
'%s\n%s\n%s' % (request_method, expires, path),
digest).hexdigest()
# Used by get_swift_info and register_swift_info to store information about # Used by get_swift_info and register_swift_info to store information about

18
test/functional/test_symlink.py
View File

@ -1581,6 +1581,7 @@ class TestSymlinkComparison(TestSymlinkTargetObjectComparison):
class TestSymlinkAccountTempurl(Base): class TestSymlinkAccountTempurl(Base):
env = TestTempurlEnv env = TestTempurlEnv
digest_name = 'sha1'
def setUp(self): def setUp(self):
super(TestSymlinkAccountTempurl, self).setUp() super(TestSymlinkAccountTempurl, self).setUp()
@ -1592,6 +1593,12 @@ class TestSymlinkAccountTempurl(Base):
"Expected tempurl_enabled to be True/False, got %r" % "Expected tempurl_enabled to be True/False, got %r" %
(self.env.tempurl_enabled,)) (self.env.tempurl_enabled,))
if self.digest_name not in cluster_info['tempurl'].get(
'allowed_digests', ['sha1']):
raise SkipTest("tempurl does not support %s signatures" %
self.digest_name)
self.digest = getattr(hashlib, self.digest_name)
self.expires = int(time.time()) + 86400 self.expires = int(time.time()) + 86400
self.obj_tempurl_parms = self.tempurl_parms( self.obj_tempurl_parms = self.tempurl_parms(
'GET', self.expires, self.env.conn.make_path(self.env.obj.path), 'GET', self.expires, self.env.conn.make_path(self.env.obj.path),
@ -1601,7 +1608,7 @@ class TestSymlinkAccountTempurl(Base):
sig = hmac.new( sig = hmac.new(
key, key,
'%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)), '%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)),
hashlib.sha1).hexdigest() self.digest).hexdigest()
return {'temp_url_sig': sig, 'temp_url_expires': str(expires)} return {'temp_url_sig': sig, 'temp_url_expires': str(expires)}
def test_PUT_symlink(self): def test_PUT_symlink(self):
@ -1665,6 +1672,7 @@ class TestSymlinkAccountTempurl(Base):
class TestSymlinkContainerTempurl(Base): class TestSymlinkContainerTempurl(Base):
env = TestContainerTempurlEnv env = TestContainerTempurlEnv
digest_name = 'sha1'
def setUp(self): def setUp(self):
super(TestSymlinkContainerTempurl, self).setUp() super(TestSymlinkContainerTempurl, self).setUp()
@ -1676,6 +1684,12 @@ class TestSymlinkContainerTempurl(Base):
"Expected tempurl_enabled to be True/False, got %r" % "Expected tempurl_enabled to be True/False, got %r" %
(self.env.tempurl_enabled,)) (self.env.tempurl_enabled,))
if self.digest_name not in cluster_info['tempurl'].get(
'allowed_digests', ['sha1']):
raise SkipTest("tempurl does not support %s signatures" %
self.digest_name)
self.digest = getattr(hashlib, self.digest_name)
expires = int(time.time()) + 86400 expires = int(time.time()) + 86400
sig = self.tempurl_sig( sig = self.tempurl_sig(
'GET', expires, self.env.conn.make_path(self.env.obj.path), 'GET', expires, self.env.conn.make_path(self.env.obj.path),
@ -1687,7 +1701,7 @@ class TestSymlinkContainerTempurl(Base):
return hmac.new( return hmac.new(
key, key,
'%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)), '%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)),
hashlib.sha1).hexdigest() self.digest).hexdigest()
def test_PUT_symlink(self): def test_PUT_symlink(self):
new_sym = self.env.container.file(Utils.create_name()) new_sym = self.env.container.file(Utils.create_name())

123
test/functional/test_tempurl.py
View File

@ -14,6 +14,8 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import base64
import functools
import hmac import hmac
import hashlib import hashlib
import json import json
@ -87,6 +89,7 @@ class TestTempurlEnv(TestTempurlBaseEnv):
class TestTempurl(Base): class TestTempurl(Base):
env = TestTempurlEnv env = TestTempurlEnv
digest_name = 'sha1'
def setUp(self): def setUp(self):
super(TestTempurl, self).setUp() super(TestTempurl, self).setUp()
@ -98,6 +101,12 @@ class TestTempurl(Base):
"Expected tempurl_enabled to be True/False, got %r" % "Expected tempurl_enabled to be True/False, got %r" %
(self.env.tempurl_enabled,)) (self.env.tempurl_enabled,))
if self.digest_name not in cluster_info['tempurl'].get(
'allowed_digests', ['sha1']):
raise SkipTest("tempurl does not support %s signatures" %
self.digest_name)
self.digest = getattr(hashlib, self.digest_name)
self.expires = int(time()) + 86400 self.expires = int(time()) + 86400
self.expires_8601 = strftime( self.expires_8601 = strftime(
tempurl.EXPIRES_ISO8601_FORMAT, gmtime(self.expires)) tempurl.EXPIRES_ISO8601_FORMAT, gmtime(self.expires))
@ -109,7 +118,7 @@ class TestTempurl(Base):
sig = hmac.new( sig = hmac.new(
key, key,
'%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)), '%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)),
hashlib.sha1).hexdigest() self.digest).hexdigest()
return {'temp_url_sig': sig, 'temp_url_expires': str(expires)} return {'temp_url_sig': sig, 'temp_url_expires': str(expires)}
def test_GET(self): def test_GET(self):
@ -332,7 +341,7 @@ class TestTempURLPrefix(TestTempurl):
key, key,
'%s\n%s\nprefix:%s' % (method, expires, '%s\n%s\nprefix:%s' % (method, expires,
'/'.join(path_parts[0:4]) + '/' + prefix), '/'.join(path_parts[0:4]) + '/' + prefix),
hashlib.sha1).hexdigest() self.digest).hexdigest()
return { return {
'temp_url_sig': sig, 'temp_url_expires': str(expires), 'temp_url_sig': sig, 'temp_url_expires': str(expires),
'temp_url_prefix': prefix} 'temp_url_prefix': prefix}
@ -428,6 +437,7 @@ class TestContainerTempurlEnv(BaseEnv):
class TestContainerTempurl(Base): class TestContainerTempurl(Base):
env = TestContainerTempurlEnv env = TestContainerTempurlEnv
digest_name = 'sha1'
def setUp(self): def setUp(self):
super(TestContainerTempurl, self).setUp() super(TestContainerTempurl, self).setUp()
@ -439,6 +449,12 @@ class TestContainerTempurl(Base):
"Expected tempurl_enabled to be True/False, got %r" % "Expected tempurl_enabled to be True/False, got %r" %
(self.env.tempurl_enabled,)) (self.env.tempurl_enabled,))
if self.digest_name not in cluster_info['tempurl'].get(
'allowed_digests', ['sha1']):
raise SkipTest("tempurl does not support %s signatures" %
self.digest_name)
self.digest = getattr(hashlib, self.digest_name)
expires = int(time()) + 86400 expires = int(time()) + 86400
sig = self.tempurl_sig( sig = self.tempurl_sig(
'GET', expires, self.env.conn.make_path(self.env.obj.path), 'GET', expires, self.env.conn.make_path(self.env.obj.path),
@ -450,7 +466,7 @@ class TestContainerTempurl(Base):
return hmac.new( return hmac.new(
key, key,
'%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)), '%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)),
hashlib.sha1).hexdigest() self.digest).hexdigest()
def test_GET(self): def test_GET(self):
contents = self.env.obj.read( contents = self.env.obj.read(
@ -694,6 +710,7 @@ class TestSloTempurlEnv(TestTempurlBaseEnv):
class TestSloTempurl(Base): class TestSloTempurl(Base):
env = TestSloTempurlEnv env = TestSloTempurlEnv
digest_name = 'sha1'
def setUp(self): def setUp(self):
super(TestSloTempurl, self).setUp() super(TestSloTempurl, self).setUp()
@ -705,11 +722,17 @@ class TestSloTempurl(Base):
"Expected enabled to be True/False, got %r" % "Expected enabled to be True/False, got %r" %
(self.env.enabled,)) (self.env.enabled,))
if self.digest_name not in cluster_info['tempurl'].get(
'allowed_digests', ['sha1']):
raise SkipTest("tempurl does not support %s signatures" %
self.digest_name)
self.digest = getattr(hashlib, self.digest_name)
def tempurl_sig(self, method, expires, path, key): def tempurl_sig(self, method, expires, path, key):
return hmac.new( return hmac.new(
key, key,
'%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)), '%s\n%s\n%s' % (method, expires, urllib.parse.unquote(path)),
hashlib.sha1).hexdigest() self.digest).hexdigest()
def test_GET(self): def test_GET(self):
expires = int(time()) + 86400 expires = int(time()) + 86400
@ -730,3 +753,95 @@ class TestSloTempurl(Base):
class TestSloTempurlUTF8(Base2, TestSloTempurl): class TestSloTempurlUTF8(Base2, TestSloTempurl):
pass pass
def requires_digest(digest):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
if digest not in cluster_info['tempurl'].get(
'allowed_digests', ['sha1']):
raise SkipTest("tempurl does not support %s signatures" %
digest)
return func(*args, **kwargs)
return wrapper
return decorator
class TestTempurlAlgorithms(Base):
env = TestTempurlEnv
def get_sig(self, expires, digest, encoding):
path = self.env.conn.make_path(self.env.obj.path)
sig = hmac.new(
self.env.tempurl_key,
'%s\n%s\n%s' % ('GET', expires,
urllib.parse.unquote(path)),
getattr(hashlib, digest))
if encoding == 'hex':
return sig.hexdigest()
elif encoding == 'base64':
return digest + ':' + base64.b64encode(sig.digest())
elif encoding == 'base64-no-padding':
return digest + ':' + base64.b64encode(sig.digest()).strip('=')
elif encoding == 'url-safe-base64':
return digest + ':' + base64.urlsafe_b64encode(sig.digest())
else:
raise ValueError('Unrecognized encoding: %r' % encoding)
def _do_test(self, digest, encoding, expect_failure=False):
expires = int(time()) + 86400
sig = self.get_sig(expires, digest, encoding)
if encoding == 'url-safe-base64':
# Make sure that we're actually testing url-safe-ness
while '-' not in sig and '_' not in sig:
expires += 1
sig = self.get_sig(expires, digest, encoding)
parms = {'temp_url_sig': sig, 'temp_url_expires': str(expires)}
if expect_failure:
with self.assertRaises(ResponseError):
self.env.obj.read(parms=parms, cfg={'no_auth_token': True})
self.assert_status([401])
# ditto for HEADs
with self.assertRaises(ResponseError):
self.env.obj.info(parms=parms, cfg={'no_auth_token': True})
self.assert_status([401])
else:
contents = self.env.obj.read(
parms=parms,
cfg={'no_auth_token': True})
self.assertEqual(contents, "obj contents")
# GET tempurls also allow HEAD requests
self.assertTrue(self.env.obj.info(
parms=parms, cfg={'no_auth_token': True}))
@requires_digest('sha1')
def test_sha1(self):
self._do_test('sha1', 'hex')
self._do_test('sha1', 'base64')
self._do_test('sha1', 'base64-no-padding')
self._do_test('sha1', 'url-safe-base64')
@requires_digest('sha256')
def test_sha256(self):
# apparently Cloud Files supports hex-encoded SHA-256
# let's not break that just for the sake of being different
self._do_test('sha256', 'hex')
self._do_test('sha256', 'base64')
self._do_test('sha256', 'base64-no-padding')
self._do_test('sha256', 'url-safe-base64')
@requires_digest('sha512')
def test_sha512(self):
# 128 chars seems awfully long for a signature -- let's require base64
self._do_test('sha512', 'hex', expect_failure=True)
self._do_test('sha512', 'base64')
self._do_test('sha512', 'base64-no-padding')
self._do_test('sha512', 'url-safe-base64')

137
test/unit/common/middleware/test_tempurl.py
View File

@ -28,11 +28,12 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import base64
import hmac import hmac
import itertools import itertools
import mock import mock
import unittest import unittest
from hashlib import sha1 import hashlib
from time import time, strftime, gmtime from time import time, strftime, gmtime
from swift.common.middleware import tempauth, tempurl from swift.common.middleware import tempauth, tempurl
@ -130,7 +131,7 @@ class TestTempURL(unittest.TestCase):
if not environ: if not environ:
environ = {} environ = {}
environ['QUERY_STRING'] = 'temp_url_sig=%s&temp_url_expires=%s' % ( environ['QUERY_STRING'] = 'temp_url_sig=%s&temp_url_expires=%s' % (
sig, expires) sig.replace('+', '%2B'), expires)
if prefix is not None: if prefix is not None:
environ['QUERY_STRING'] += '&temp_url_prefix=%s' % prefix environ['QUERY_STRING'] += '&temp_url_prefix=%s' % prefix
req = self._make_request(path, keys=keys, environ=environ) req = self._make_request(path, keys=keys, environ=environ)
@ -151,9 +152,20 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
self.assert_valid_sig(expires, path, [key], sig) self.assert_valid_sig(expires, path, [key], sig)
sig = hmac.new(key, hmac_body, hashlib.sha256).hexdigest()
self.assert_valid_sig(expires, path, [key], sig)
sig = base64.b64encode(hmac.new(
key, hmac_body, hashlib.sha256).digest())
self.assert_valid_sig(expires, path, [key], 'sha256:' + sig)
sig = base64.b64encode(hmac.new(
key, hmac_body, hashlib.sha512).digest())
self.assert_valid_sig(expires, path, [key], 'sha512:' + sig)
def test_get_valid_key2(self): def test_get_valid_key2(self):
method = 'GET' method = 'GET'
expires = int(time() + 86400) expires = int(time() + 86400)
@ -161,8 +173,8 @@ class TestTempURL(unittest.TestCase):
key1 = 'abc123' key1 = 'abc123'
key2 = 'def456' key2 = 'def456'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig1 = hmac.new(key1, hmac_body, sha1).hexdigest() sig1 = hmac.new(key1, hmac_body, hashlib.sha1).hexdigest()
sig2 = hmac.new(key2, hmac_body, sha1).hexdigest() sig2 = hmac.new(key2, hmac_body, hashlib.sha1).hexdigest()
for sig in (sig1, sig2): for sig in (sig1, sig2):
self.assert_valid_sig(expires, path, [key1, key2], sig) self.assert_valid_sig(expires, path, [key1, key2], sig)
@ -184,8 +196,8 @@ class TestTempURL(unittest.TestCase):
key1 = 'me' key1 = 'me'
key2 = 'other' key2 = 'other'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig1 = hmac.new(key1, hmac_body, sha1).hexdigest() sig1 = hmac.new(key1, hmac_body, hashlib.sha1).hexdigest()
sig2 = hmac.new(key2, hmac_body, sha1).hexdigest() sig2 = hmac.new(key2, hmac_body, hashlib.sha1).hexdigest()
account_keys = [] account_keys = []
for sig in (sig1, sig2): for sig in (sig1, sig2):
self.assert_valid_sig(expires, path, account_keys, sig, environ) self.assert_valid_sig(expires, path, account_keys, sig, environ)
@ -197,7 +209,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&' 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&'
'filename=bob%%20%%22killer%%22.txt' % (sig, expires)}) 'filename=bob%%20%%22killer%%22.txt' % (sig, expires)})
@ -219,7 +231,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'REQUEST_METHOD': 'HEAD', 'REQUEST_METHOD': 'HEAD',
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&' 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&'
@ -237,7 +249,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'REQUEST_METHOD': 'HEAD', 'REQUEST_METHOD': 'HEAD',
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s'
@ -247,7 +259,7 @@ class TestTempURL(unittest.TestCase):
get_method = 'GET' get_method = 'GET'
get_hmac_body = '%s\n%s\n%s' % (get_method, expires, path) get_hmac_body = '%s\n%s\n%s' % (get_method, expires, path)
get_sig = hmac.new(key, get_hmac_body, sha1).hexdigest() get_sig = hmac.new(key, get_hmac_body, hashlib.sha1).hexdigest()
get_req = self._make_request(path, keys=[key], environ={ get_req = self._make_request(path, keys=[key], environ={
'REQUEST_METHOD': 'GET', 'REQUEST_METHOD': 'GET',
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s'
@ -263,7 +275,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&' 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&'
'filename=bob%%20%%22killer%%22.txt&inline=' % (sig, expires)}) 'filename=bob%%20%%22killer%%22.txt&inline=' % (sig, expires)})
@ -285,7 +297,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&' 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&'
'inline=' % (sig, expires)}) 'inline=' % (sig, expires)})
@ -305,13 +317,13 @@ class TestTempURL(unittest.TestCase):
query_path = '/v1/a/c/' + prefix + 'o' query_path = '/v1/a/c/' + prefix + 'o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, sig_path) hmac_body = '%s\n%s\n%s' % (method, expires, sig_path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
self.assert_valid_sig(expires, query_path, [key], sig, prefix=prefix) self.assert_valid_sig(expires, query_path, [key], sig, prefix=prefix)
query_path = query_path[:-1] + 'p3/o' query_path = query_path[:-1] + 'p3/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, sig_path) hmac_body = '%s\n%s\n%s' % (method, expires, sig_path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
self.assert_valid_sig(expires, query_path, [key], sig, prefix=prefix) self.assert_valid_sig(expires, query_path, [key], sig, prefix=prefix)
def test_get_valid_with_prefix_empty(self): def test_get_valid_with_prefix_empty(self):
@ -321,7 +333,7 @@ class TestTempURL(unittest.TestCase):
query_path = '/v1/a/c/o' query_path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, sig_path) hmac_body = '%s\n%s\n%s' % (method, expires, sig_path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
self.assert_valid_sig(expires, query_path, [key], sig, prefix='') self.assert_valid_sig(expires, query_path, [key], sig, prefix='')
def test_obj_odd_chars(self): def test_obj_odd_chars(self):
@ -330,7 +342,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/a\r\nb' path = '/v1/a/c/a\r\nb'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
sig, expires)}) sig, expires)})
@ -350,7 +362,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
sig, expires)}) sig, expires)})
@ -370,7 +382,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o/' path = '/v1/a/c/o/'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
sig, expires)}) sig, expires)})
@ -390,7 +402,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request(path, keys=[key], environ={ req = self._make_request(path, keys=[key], environ={
'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&' 'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s&'
'filename=/i/want/this/just/as/it/is/' % (sig, expires)}) 'filename=/i/want/this/just/as/it/is/' % (sig, expires)})
@ -411,7 +423,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -429,7 +441,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'PUT', environ={'REQUEST_METHOD': 'PUT',
@ -446,7 +458,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'PUT', environ={'REQUEST_METHOD': 'PUT',
@ -463,7 +475,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -479,7 +491,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
hmac.new(key, hmac_body, sha1).hexdigest() hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': 'temp_url_expires=%s' % expires}) environ={'QUERY_STRING': 'temp_url_expires=%s' % expires})
@ -494,7 +506,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': 'temp_url_sig=%s' % sig}) environ={'QUERY_STRING': 'temp_url_sig=%s' % sig})
@ -509,7 +521,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/' path = '/v1/a/c/'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -525,7 +537,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[], path, keys=[],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -541,7 +553,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'HEAD', environ={'REQUEST_METHOD': 'HEAD',
@ -558,7 +570,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'HEAD', environ={'REQUEST_METHOD': 'HEAD',
@ -575,7 +587,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'HEAD', environ={'REQUEST_METHOD': 'HEAD',
@ -592,7 +604,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
# Deliberately fudge expires to show HEADs aren't just automatically # Deliberately fudge expires to show HEADs aren't just automatically
# allowed. # allowed.
expires += 1 expires += 1
@ -612,7 +624,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'POST', environ={'REQUEST_METHOD': 'POST',
@ -630,7 +642,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'DELETE', environ={'REQUEST_METHOD': 'DELETE',
@ -647,7 +659,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'DELETE', environ={'REQUEST_METHOD': 'DELETE',
@ -662,7 +674,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'REQUEST_METHOD': 'UNKNOWN', environ={'REQUEST_METHOD': 'UNKNOWN',
@ -690,7 +702,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new('account-key', hmac_body, sha1).hexdigest() sig = hmac.new('account-key', hmac_body, hashlib.sha1).hexdigest()
qs = '?temp_url_sig=%s&temp_url_expires=%s' % (sig, expires) qs = '?temp_url_sig=%s&temp_url_expires=%s' % (sig, expires)
# make request will setup the environ cache for us # make request will setup the environ cache for us
@ -712,7 +724,7 @@ class TestTempURL(unittest.TestCase):
# the container level; a different container in the same account is # the container level; a different container in the same account is
# out of scope and thus forbidden. # out of scope and thus forbidden.
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new('container-key', hmac_body, sha1).hexdigest() sig = hmac.new('container-key', hmac_body, hashlib.sha1).hexdigest()
qs = '?temp_url_sig=%s&temp_url_expires=%s' % (sig, expires) qs = '?temp_url_sig=%s&temp_url_expires=%s' % (sig, expires)
req = self._make_request(path + qs, **key_kwargs) req = self._make_request(path + qs, **key_kwargs)
@ -733,7 +745,7 @@ class TestTempURL(unittest.TestCase):
# account-level tempurls by reusing one of the account's keys on a # account-level tempurls by reusing one of the account's keys on a
# container. # container.
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new('shared-key', hmac_body, sha1).hexdigest() sig = hmac.new('shared-key', hmac_body, hashlib.sha1).hexdigest()
qs = '?temp_url_sig=%s&temp_url_expires=%s' % (sig, expires) qs = '?temp_url_sig=%s&temp_url_expires=%s' % (sig, expires)
req = self._make_request(path + qs, **key_kwargs) req = self._make_request(path + qs, **key_kwargs)
@ -754,7 +766,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path + '2', keys=[key], path + '2', keys=[key],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -770,7 +782,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
if sig[-1] != '0': if sig[-1] != '0':
sig = sig[:-1] + '0' sig = sig[:-1] + '0'
else: else:
@ -790,7 +802,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -806,7 +818,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key + '2'], path, keys=[key + '2'],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -823,7 +835,7 @@ class TestTempURL(unittest.TestCase):
query_path = '/v1/a/c/o' query_path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, sig_path) hmac_body = '%s\n%s\n%s' % (method, expires, sig_path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
query_path, keys=[key], query_path, keys=[key],
environ={'QUERY_STRING': environ={'QUERY_STRING':
@ -840,7 +852,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': environ={'QUERY_STRING':
@ -860,7 +872,7 @@ class TestTempURL(unittest.TestCase):
for hdr, value in [('X-Object-Manifest', 'private/secret'), for hdr, value in [('X-Object-Manifest', 'private/secret'),
('X-Symlink-Target', 'cont/symlink')]: ('X-Symlink-Target', 'cont/symlink')]:
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, method=method, keys=[key], path, method=method, keys=[key],
headers={hdr: value}, headers={hdr: value},
@ -881,7 +893,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
headers={'x-remove-this': 'value'}, headers={'x-remove-this': 'value'},
@ -900,7 +912,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
headers={'x-remove-this-one': 'value1', headers={'x-remove-this-one': 'value1',
@ -922,7 +934,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
headers={'x-conflict-header': 'value'}, headers={'x-conflict-header': 'value'},
@ -941,7 +953,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
headers={'x-conflict-header-test': 'value'}, headers={'x-conflict-header-test': 'value'},
@ -959,7 +971,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -978,7 +990,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % ( environ={'QUERY_STRING': 'temp_url_sig=%s&temp_url_expires=%s' % (
@ -998,7 +1010,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
headers={}, headers={},
@ -1020,7 +1032,7 @@ class TestTempURL(unittest.TestCase):
path = '/v1/a/c/o' path = '/v1/a/c/o'
key = 'abc' key = 'abc'
hmac_body = '%s\n%s\n%s' % (method, expires, path) hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest() sig = hmac.new(key, hmac_body, hashlib.sha1).hexdigest()
req = self._make_request( req = self._make_request(
path, keys=[key], path, keys=[key],
headers={}, headers={},
@ -1144,13 +1156,15 @@ class TestTempURL(unittest.TestCase):
self.assertEqual( self.assertEqual(
self.tempurl._get_hmacs( self.tempurl._get_hmacs(
{'REQUEST_METHOD': 'GET'}, 1, '/v1/a/c/o', {'REQUEST_METHOD': 'GET'}, 1, '/v1/a/c/o',
[('abc', 'account')]), [('abc', 'account')], 'sha1'),
[('026d7f7cc25256450423c7ad03fc9f5ffc1dab6d', 'account')]) [('026d7f7cc25256450423c7ad03fc9f5ffc1dab6d', 'account')])
self.assertEqual( self.assertEqual(
self.tempurl._get_hmacs( self.tempurl._get_hmacs(
{'REQUEST_METHOD': 'HEAD'}, 1, '/v1/a/c/o', {'REQUEST_METHOD': 'HEAD'}, 1, '/v1/a/c/o',
[('abc', 'account')], request_method='GET'), [('abc', 'account')], 'sha512', request_method='GET'),
[('026d7f7cc25256450423c7ad03fc9f5ffc1dab6d', 'account')]) [('240866478d94bbe683ab1d25fba52c7d0df21a60951'
'4fe6a493dc30f951d2748abc51da0cbc633cd1e0acf'
'6fadd3af3aedff00ee3d3434dc6a4c423e74adfc4a', 'account')])
def test_invalid(self): def test_invalid(self):
@ -1327,6 +1341,7 @@ class TestSwiftInfo(unittest.TestCase):
set(('x-object-meta-*',))) set(('x-object-meta-*',)))
self.assertEqual(set(info['outgoing_allow_headers']), self.assertEqual(set(info['outgoing_allow_headers']),
set(('x-object-meta-public-*',))) set(('x-object-meta-public-*',)))
self.assertEqual(info['allowed_digests'], ['sha1', 'sha256', 'sha512'])
def test_non_default_methods(self): def test_non_default_methods(self):
tempurl.filter_factory({ tempurl.filter_factory({
@ -1335,6 +1350,7 @@ class TestSwiftInfo(unittest.TestCase):
'incoming_allow_headers': 'x-timestamp x-versions-location', 'incoming_allow_headers': 'x-timestamp x-versions-location',
'outgoing_remove_headers': 'x-*', 'outgoing_remove_headers': 'x-*',
'outgoing_allow_headers': 'x-object-meta-* content-type', 'outgoing_allow_headers': 'x-object-meta-* content-type',
'allowed_digests': 'sha512 md5 not-a-valid-digest',
}) })
swift_info = utils.get_swift_info() swift_info = utils.get_swift_info()
self.assertIn('tempurl', swift_info) self.assertIn('tempurl', swift_info)
@ -1347,6 +1363,13 @@ class TestSwiftInfo(unittest.TestCase):
self.assertEqual(set(info['outgoing_remove_headers']), set(('x-*', ))) self.assertEqual(set(info['outgoing_remove_headers']), set(('x-*', )))
self.assertEqual(set(info['outgoing_allow_headers']), self.assertEqual(set(info['outgoing_allow_headers']),
set(('x-object-meta-*', 'content-type'))) set(('x-object-meta-*', 'content-type')))
self.assertEqual(info['allowed_digests'], ['sha512'])
def test_bad_config(self):
with self.assertRaises(ValueError):
tempurl.filter_factory({
'allowed_digests': 'md4',
})
if __name__ == '__main__': if __name__ == '__main__':