75 Commits

Author SHA1 Message Date
Tim Burke
f871591baa tests: swiftclient supports insecure
We already require swiftclient>=3.2.0, and have for years.
We can stop checking whether it's 1.x

Related-Change: I9842c9975821bda5c7d8bf2fc214480c0c0a5e96
Change-Id: I798904ab66ca10e21b4999ed7f2be74d1b63584c
2023-09-25 15:51:46 -07:00
Tim Burke
be16d6c4fd tests: Get rid of test.unit.SkipTest
unittest.SkipTest suffices.

Change-Id: I11eb73f7dc4a8598fae85d1efca721f69067fb4f
2023-02-16 23:59:53 -08:00
Tim Burke
597887dedc Extract SwiftHttpProtocol to its own module
Change-Id: I35cade2c46eb6acb66c064cde75d78173f46864c
2022-12-06 11:15:53 -08:00
Clay Gerrard
2a312d1cd5 Cleanup tests' import of debug_logger
Change-Id: I19ca860deaa6dbf388bdcd1f0b0f77f72ff19689
2021-04-27 12:04:41 +01:00
Tim Burke
918ab8543e Use socket_timeout kwarg instead of useless eventlet.wsgi.WRITE_TIMEOUT
No version of eventlet that I'm aware of hasany sort of support for
eventlet.wsgi.WRITE_TIMEOUT; I don't know why we've been setting that.
On the other hand, the socket_timeout argument for eventlet.wsgi.Server
has been supported for a while -- since 0.14 in 2013.

Drive-by: Fix up handling of sub-second client_timeouts.

Change-Id: I1dca3c3a51a83c9d5212ee5a0ad2ba1343c68cf9
Related-Change: I1d4d028ac5e864084a9b7537b140229cb235c7a3
Related-Change: I433c97df99193ec31c863038b9b6fd20bb3705b8
2020-11-11 14:23:40 -08:00
Tim Burke
05d8b8fba1 Increase container_update_timeout in in-process func tests
This should make us less likely to fail in the gate because of listing
consistency issues.

Drive-by: Remove the allowed_headers override; that hasn't been necessary
since we updated the default value a while back.

Change-Id: Id280bc93ed6e899a62b0115fdf144a564ef0cd8d
Related-Change: Ib82e175096716e42aecdab48f01f079e09da6a1d
2020-11-05 17:42:05 -08:00
Zuul
ba01f93ad9 Merge "func tests: work with etag-quoter on by default" 2020-06-05 20:52:30 +00:00
Thiago da Silva
20c6bdb71c Enable s3api and staticweb tests across all func tests
This patch removed the separate s3api, staticweb functional tests
gate jobs and added them across all other functional test jobs.

Change-Id: Ie1c606132a054defc2b3cc14a66031090e7b8449
2020-06-01 19:16:48 -07:00
Tim Burke
99947150dd func tests: work with etag-quoter on by default
Also, run the in-process encryption func tests like that.

Change-Id: I984ab8d1304d23b89589973950b10dda4aea0db3
2020-06-01 18:38:23 -05:00
Thiago da Silva
f4bc951508 fix s3api functional tests
Connection was hard coded to use saio config

Change-Id: I9c11162de89fa3aa2a78aea093b187d0309860f5
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2020-05-15 22:38:04 -07:00
Tim Burke
9581254e66 swift-dsvm: Enable more middlewares
Change-Id: I3cbbcd2ea9ced0923bee4a6b0783e4cf5e82e95b
2020-04-23 13:46:16 -07:00
Tim Burke
668242c422 pep8: Turn on E305
Change-Id: Ia968ec7375ab346a2155769a46e74ce694a57fc2
2020-04-03 21:22:38 +02:00
Andreas Jaeger
96b56519bf Update hacking for Python3
The repo is Python using both Python 2 and 3 now, so update hacking to
version 2.0 which supports Python 2 and 3. Note that latest hacking
release 3.0 only supports version 3.

Fix problems found.

Remove hacking and friends from lower-constraints, they are not needed
for installation.

Change-Id: I9bd913ee1b32ba1566c420973723296766d1812f
2020-04-03 21:21:07 +02:00
karen chan
6097660f0c s3api: Implement object versioning API
Translate AWS S3 Object Versioning API requests to native Swift Object
Versioning API, speficially:

 * bucket versioning status
 * bucket versioned objects listing params
 * object GETorHEAD & DELETE versionId
 * multi_delete versionId

Change-Id: I8296681b61996e073b3ba12ad46f99042dc15c37
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
2020-01-28 14:00:08 -08:00
Clay Gerrard
2759d5d51c New Object Versioning mode
This patch adds a new object versioning mode. This new mode provides
a new set of APIs for users to interact with older versions of an
object. It also changes the naming scheme of older versions and adds
a version-id to each object.

This new mode is not backwards compatible or interchangeable with the
other two modes (i.e., stack and history), especially due to the changes
in the namimg scheme of older versions. This new mode will also serve
as a foundation for adding S3 versioning compatibility in the s3api
middleware.

Note that this does not (yet) support using a versioned container as
a source in container-sync. Container sync should be enhanced to sync
previous versions of objects.

Change-Id: Ic7d39ba425ca324eeb4543a2ce8d03428e2225a1
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Co-Authored-By: Thiago da Silva <thiagodasilva@gmail.com>
2020-01-24 17:39:56 -08:00
Zuul
1c29bd678c Merge "Allow non-default domain to be used in func tests" 2019-09-20 01:00:02 +00:00
Tim Burke
74db367060 py3: Finish porting func tests
We were (indirectly) importing swiftclient (and therefore requests and
urllib3) before doing our eventlet monkey-patching. This would lead
boto3 (which digs an SSLContext out of urllib3) to trip RecursionErrors
on py3 similar to

   >>> from ssl import SSLContext, PROTOCOL_SSLv23
   >>> import eventlet
   >>> eventlet.monkey_patch(socket=True)
   >>> SSLContext(PROTOCOL_SSLv23).options |= 0
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/usr/lib/python3.6/ssl.py", line 465, in options
       super(SSLContext, SSLContext).options.__set__(self, value)
     File "/usr/lib/python3.6/ssl.py", line 465, in options
       super(SSLContext, SSLContext).options.__set__(self, value)
     File "/usr/lib/python3.6/ssl.py", line 465, in options
       super(SSLContext, SSLContext).options.__set__(self, value)
     [Previous line repeated 330 more times]
   RecursionError: maximum recursion depth exceeded while calling a Python object

Change-Id: I4bb59edd87336597791416c4f2a096efe0e72fe3
2019-08-16 20:34:16 -07:00
Tim Burke
2c727f65ff py3: Add swift-tox-func-encryption-py37 gate job
Change-Id: Ia9ae0fc226dfc9b40157faebac100c10a9180c62
2019-08-06 13:56:47 -07:00
Tim Burke
a48dd1950d Allow non-default domain to be used in func tests
Change-Id: I7afa7e367103bb9caaf74788a49cd055eca53cf6
2019-08-05 15:37:19 -07:00
Tim Burke
9d1b749740 py3: port staticweb and domain_remap func tests
Drive-by: Tighten domain_remap assertions on listings, which required
that we fix proxy pipeline placement. Add a note about it to the sample
config.

Change-Id: I41835148051294088a2c0fb4ed4e7a7b61273e5f
2019-07-10 09:51:38 -07:00
Tim Burke
506279235d Get functional/tests.py running under py3
Note that you need a pretty recent eventlet to pick up
https://github.com/eventlet/eventlet/commit/f0bc79e

Change-Id: I6b006b972e7431c406039f4e0f6890a8f74a4432
2019-05-08 17:44:03 -07:00
Tim Burke
359d136c29 Unify auth_host/port/ssl/prefix func test options into auth_uri
Retain support for the old way, though (at least for now).

Change-Id: Id3c2ed87230c5918c18e2c01d086df8157f036b1
2019-05-03 20:55:52 +00:00
John Dickinson
adc568c97f Fix bulk responses when using xml and Expect 100-continue
When we fixed bulk response heartbeating in https://review.openstack.org/#/c/510715/,
code review raised the issue of moving the xml header down to after the
early-exit clauses. At the time, it didn't seem to break anything, so
it was left in place. However, that insight was correct.

The purpose of the earlier patch was to force eventlet to use chunked
transfer encoding on the response in order to prevent eventlet from
buffering the whole response, thus defeating the purpose of the
heartbeat responses.

Moving the first line of the body lower (ie after the early exit
checks), allows other headers in a chunked transfer encoding response
to be appropriately processed before sending the headers. Sending the
xml declaration early causes it to get intermingled in the 100-continue
protocol, thus breaking the chunked transfer encoding semantics.

Closes-Bug: #1819252

Change-Id: I072f4dab21cd7cdb81b9e41072eb504131411dc8
2019-03-18 20:41:29 +00:00
Tim Burke
e5eb673ccb Stop monkey-patching mimetools
You could *try* doing something similar to what we were doing
there over in email.message for py3, but you would end up
breaking pkg_resources (and therefor entrypoints) in the
process.

Drive-by: have mem_diskfile implement more of the diskfile API.

Change-Id: I1ece4b4500ce37408799ee634ed6d7832fb7b721
2019-03-13 21:51:36 -07:00
Tim Burke
8b519d1abc Clean up func tests ahead of py3
- ConfigParser.set() requires that the value be a string
- The stdlib HTTP client responses don't have a body property
- We might raise a ResponseError with response=None
- Bodies should be bytes
- Headers should be strings
- Make containers()/files() return native strings
- file() isn't a thing on py3
- format should be a parm, not a header
- Switch sorted() to use key instead of cmp
- Use integer division explicitly

Change-Id: I99d3eebc9d7ec4e8b295352294b831492135c568
2019-03-01 16:10:53 -08:00
Tim Burke
333ae3086f Fix in-process func tests to work with newer PasteDeploy
Apparently they changed the order of how default, global, and local config
get applied? See also: https://github.com/Pylons/pastedeploy/commit/556611a

Change-Id: Ie7322ef362b0b22898c14db7d0fcd3dafc40d473
2018-12-03 21:44:01 +00:00
Tim Burke
9017db4519 Only print "SKIPPING FUNCTIONAL TESTS ..." messages once
Previously, these would get printed on every import of test.functional

Change-Id: I61da04f6b8296943ec8282c1f8ddacdcaf5196fd
2018-11-29 17:39:16 -08:00
zhufl
3465d639e3 Add missing ws seperator between words
This is to add missing ws seperator between words, usually
in log messages.

Change-Id: I6b88fab428b93def77632f9aed29fc2de4380b0d
2018-11-19 15:23:25 +08:00
Tim Burke
5a8cfd6e06 Add another user for s3api func tests
Previously we'd use two users, one admin and one unprivileged.

Ceph's s3-tests, however, assume that both users should have access to
create buckets. Further, there are different errors that may be returned
depending on whether you are the *bucket* owner or not when using
s3_acl. So now we've got:

  test:tester1  (admin)
  test:tester2  (also admin)
  test:tester3  (unprivileged)

Change-Id: I0b67c53de3bcadc2c656d86131fca5f2c3114f14
2018-09-14 13:33:51 +00:00
Alistair Coles
8b44c9df29 Allow domain remap storage_domain to be configured for func tests
Previously some tests in /test/functional/test_staticweb.py assumed
that domain_remap middleware was configured with the default
storage_domain 'example.com'. As a result those tests would fail if
run against a cluster with domain_remap middleware installed but not
having an 'example.com' storage_domain.

This patch allows a storage_domain to be configured in the test.conf
file so that the tests may be successfully run against any cluster. If
the option is not set then the tests will be skipped.

To ensure that the in process test job func-domain-remap-staticweb
still runs all the tests in test_staticweb.py, its config loader is
modified to add the required storage_domain option to the test config
with the same value used by the domain_remap middleware.

Related-Change: I6bc70039c1b99b8a5bf5a30e9c6877d4f0aa638d
Co-Authored-By: Tim Burke <tim.burke@gmail.com>

Change-Id: Ifab84fc3ef66b9790e96ebb8e5c8061a12636045
2018-06-07 11:59:08 +01:00
Darrell Bishop
661838d968 Add support for PROXY protocol v1 (only)
...to the proxy-server.

The point is to allow the Swift proxy server to log accurate
client IP addresses when there is a proxy or SSL-terminator between the
client and the Swift proxy server.  Example servers supporting this
PROXY protocol:
  stud (v1 only)
  stunnel
  haproxy
  hitch (v2 only)
  varnish

See http://www.haproxy.org/download/1.7/doc/proxy-protocol.txt

The feature is enabled by adding this to your proxy config file:

  [app:proxy-server]
  use = egg:swift#proxy
  ...
  require_proxy_protocol = true

The protocol specification states:

  The receiver MUST be configured to only receive the protocol
  described in this specification and MUST not try to guess
  whether the protocol header is present or not.

so valid deployments are:

  1) require_proxy_protocol = false  (or missing; default is false)
     and NOT behind a proxy that adds or proxies existing PROXY lines.
  2) require_proxy_protocol = true
     and IS behind a proxy that adds or proxies existing PROXY lines.

Specifically, in the default configuration, one cannot send the swift
proxy PROXY lines (no change from before this patch).  When this
feature is enabled, one _must_ send PROXY lines.

Change-Id: Icb88902f0a89b8d980c860be032d5e822845d03a
2018-05-23 18:10:40 -07:00
Kota Tsuyuzaki
636b922f3b Import swift3 into swift repo as s3api middleware
This attempts to import openstack/swift3 package into swift upstream
repository, namespace. This is almost simple porting except following items.

1. Rename swift3 namespace to swift.common.middleware.s3api
1.1 Rename also some conflicted class names (e.g. Request/Response)

2. Port unittests to test/unit/s3api dir to be able to run on the gate.

3. Port functests to test/functional/s3api and setup in-process testing

4. Port docs to doc dir, then address the namespace change.

5. Use get_logger() instead of global logger instance

6. Avoid global conf instance

Ex. fix various minor issue on those steps (e.g. packages, dependencies,
  deprecated things)

The details and patch references in the work on feature/s3api are listed
at https://trello.com/b/ZloaZ23t/s3api (completed board)

Note that, because this is just a porting, no new feature is developed since
the last swift3 release, and in the future work, Swift upstream may continue
to work on remaining items for further improvements and the best compatibility
of Amazon S3. Please read the new docs for your deployment and keep track to
know what would be changed in the future releases.

Change-Id: Ib803ea89cfee9a53c429606149159dd136c036fd
Co-Authored-By: Thiago da Silva <thiago@redhat.com>
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
2018-04-27 15:53:57 +09:00
Romain LE DISEZ
335899f89f Functional tests for staticweb middleware
Change-Id: I6bc70039c1b99b8a5bf5a30e9c6877d4f0aa638d
2018-03-07 12:36:20 -08:00
Steve Kowalik
5a06e3da3b No longer import nose
Since Python 2.7, unittest in the standard library has included mulitple
facilities for skipping tests by decorators as well as an exception.
Switch to that directly, rather than importing nose.

Change-Id: I4009033473ea24f0d0faed3670db844f40051f30
2017-11-07 15:39:25 +11:00
Samuel Merritt
728b4ba140 Add checksum to object extended attributes
Currently, our integrity checking for objects is pretty weak when it
comes to object metadata. If the extended attributes on a .data or
.meta file get corrupted in such a way that we can still unpickle it,
we don't have anything that detects that.

This could be especially bad with encrypted etags; if the encrypted
etag (X-Object-Sysmeta-Crypto-Etag or whatever it is) gets some bits
flipped, then we'll cheerfully decrypt the cipherjunk into plainjunk,
then send it to the client. Net effect is that the client sees a GET
response with an ETag that doesn't match the MD5 of the object *and*
Swift has no way of detecting and quarantining this object.

Note that, with an unencrypted object, if the ETag metadatum gets
mangled, then the object will be quarantined by the object server or
auditor, whichever notices first.

As part of this commit, I also ripped out some mocking of
getxattr/setxattr in tests. It appears to be there to allow unit tests
to run on systems where /tmp doesn't support xattrs. However, since
the mock is keyed off of inode number and inode numbers get re-used,
there's lots of leakage between different test runs. On a real FS,
unlinking a file and then creating a new one of the same name will
also reset the xattrs; this isn't the case with the mock.

The mock was pretty old; Ubuntu 12.04 and up all support xattrs in
/tmp, and recent Red Hat / CentOS releases do too. The xattr mock was
added in 2011; maybe it was to support Ubuntu Lucid Lynx?

Bonus: now you can pause a test with the debugger, inspect its files
in /tmp, and actually see the xattrs along with the data.

Since this patch now uses a real filesystem for testing filesystem
operations, tests are skipped if the underlying filesystem does not
support setting xattrs (eg tmpfs or more than 4k of xattrs on ext4).

References to "/tmp" have been replaced with calls to
tempfile.gettempdir(). This will allow setting the TMPDIR envvar in
test setup and getting an XFS filesystem instead of ext4 or tmpfs.

THIS PATCH SIGNIFICANTLY CHANGES TESTING ENVIRONMENTS

With this patch, every test environment will require TMPDIR to be
using a filesystem that supports at least 4k of extended attributes.
Neither ext4 nor tempfs support this. XFS is recommended.

So why all the SkipTests? Why not simply raise an error? We still need
the tests to run on the base image for OpenStack's CI system. Since
we were previously mocking out xattr, there wasn't a problem, but we
also weren't actually testing anything. This patch adds functionality
to validate xattr data, so we need to drop the mock.

`test.unit.skip_if_no_xattrs()` is also imported into `test.functional`
so that functional tests can import it from the functional test
namespace.

The related OpenStack CI infrastructure changes are made in
https://review.openstack.org/#/c/394600/.

Co-Authored-By: John Dickinson <me@not.mn>

Change-Id: I98a37c0d451f4960b7a12f648e4405c6c6716808
2017-11-03 13:30:05 -04:00
Kota Tsuyuzaki
1e79f828ad Remove all post_as_copy related code and configes
It was deprecated and we discussed on this topic in Denver PTG
for Queen cycle. Main motivation for this work is that deprecated
post_as_copy option and its gate blocks future symlink work.

Change-Id: I411893db1565864ed5beb6ae75c38b982a574476
2017-09-16 05:50:41 +00:00
Christian Schwede
2410b616bb Fix swiftdir option and usage of storage policy aliases
If swift-recon/swift-get-nodes/swift-object-info is used with the
swiftdir option they will read rings from the given directory; however
they are still using /etc/swift/swift.conf to find the policies on the
current node.

This makes it impossible to maintain a local swift.conf copy (if you
don't have write access to /etc/swift) or check multiple clusters from
the same node.

Until now swift-recon was also not usable with storage policy aliases,
this patch fixes this as well.

Closes-Bug: 1577582
Closes-Bug: 1604707
Closes-Bug: 1617951
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Co-Authored-By: Thiago da Silva <thiago@redhat.com>
Change-Id: I13188d42ec19e32e4420739eacd1e5b454af2ae3
2017-07-12 16:23:04 -04:00
Pete Zaitcev
5dfc3a75fb Open-code eventlet.listen()
Recently out gate started blowing up intermittently with a strange
case of ports mixed up. Sometimes a functional tests tries to
authorize on a port that's clearly an object server port, and
the like. As it turns out, eventlet developers added an unavoidable
SO_REUSEPORT into listen(), which makes listen(("localhost",0)
to reuse ports.

There's an issue about it:
 https://github.com/eventlet/eventlet/issues/411

This patch is working around the problem while eventlet people
consider the issue.

Change-Id: I67522909f96495a6a30e1acdb79835dce2189549
2017-05-11 01:39:14 -06:00
Tim Burke
4c8239503a Add message to account-creation assertion
We've seen some failures in the gate like

    ==============================
    Failed 1 tests - output below:
    ==============================

    setUpModule (test.functional.test_account)
    ------------------------------------------

    Captured traceback:
    ~~~~~~~~~~~~~~~~~~~
        Traceback (most recent call last):
          File "test/functional/test_account.py", line 33, in setUpModule
            tf.setup_package()
          File "test/functional/__init__.py", line 716, in setup_package
            mem_object_server if in_mem_obj else object_server))
          File "test/functional/__init__.py", line 621, in in_process_setup
            create_account(AUTH_test)
          File "test/functional/__init__.py", line 619, in create_account
            assert(resp.status == 201)
        AssertionError

...which aren't terribly useful in figuring out what went wrong.

Change-Id: I3cd31bb480dc8508828fe21416bfae33bc0985b7
2017-05-04 17:48:26 +00:00
Alistair Coles
5f610c76bd Support EC policy for in process functional tests
Add support for a 2+1 EC policy to be optionally used as default
policy when running in process functional tests.

The EC policy may be selected by setting the env var:

  SWIFT_TEST_IN_PROCESS_CONF_LOADER=ec tox

when running .functests, or by using the new tox test env:

  tox -e func-ec

Change-Id: I02e3553a74a024efdab91dcd609ac1cf4e4f3208
2017-03-09 10:42:34 +00:00
Kota Tsuyuzaki
0c8c764547 Add functional tests for new versioned_write mode
This patch is follow up for [1] and [2] to add new functional
tests for versioned_writes middlware 'history' mode.
(i.e. using X-History-Location header to a container).

The new test class, TestObjectHistoryModeVersioning, will use obvious
setting the mode via new X-History-Location header, since the change [2],
the setting X-Versions-Mode header added since [1] for incomming request has
been deprecated. Hence, since [2], the syntax for stack mode is back to
be same with older Swift than [1] so that the only thing we need now is
just adding a test suite for the new X-History-location.

It means the API has been changing like:
---------------
For stack mode:
---------------
Older than [1]:
X-Versions-Location

[1]~[2]:
X-Vesions-Location (and X-Versions-Mode: 'stack' for obvious)

Newer than [2]:
X-Vesions-Location

-----------------
For history mode:
-----------------
Older than [1]:
(Not supported)

[1]~[2]:
X-Vesions-Location and X-Versions-Mode: 'history'

Newer than [2]:
X-History-Location

Note that this functional tests work on newer swift than [2].

And then, this patch also sets allow_versioned_writes=True
for in-process testing (the container server allow_versions
option was already set, so this is just enabling in the middleware
too). That means that in-process functional tests (such as run by
the tox envs func-in-process-*) because history mode requires the
middleware allow_versioned_writes option to be explicity set to True.

1: https://review.openstack.org/#/c/214922/
2: https://review.openstack.org/#/c/373537/

Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>

Related-Change: I555dc17fefd0aa9ade681aa156da24e018ebe74b
Related-Change: Icfd0f481d4e40dd5375c737190aea7ee8dbc3bf9
Change-Id: Ifebc1c3ce558b1df9e576a58a4100f2219dfc7e7
2016-09-25 19:45:38 -07:00
Janie Richling
66520146cf Enable in-process func tests to optionally use encryption
Running functional tests in the in-process mode uses
the default value for the pipeline.  This patch adds support
to specify the SWIFT_TEST_IN_PROCESS_CONF_LOADER variable
to point to a labeled function that changes the proxy
configuration for the functional test.

The patch also adds a new tox environment
func-in-process-encryption
which runs with the environment variable
SWIFT_TEST_IN_PROCESS_CONF_LOADER=encryption

The motivation for this change is to put support in place for an
upstream CI job that will functionally test using encryption
middleware in the pipeline.  The gate job is proposed at:
https://review.openstack.org/#/c/348292/

Change-Id: I15c4b20f1d2be57ae21c69c614f6a9579145bee9
2016-07-28 14:19:48 -05:00
Alistair Coles
54e53ae87b Silence SkipTest noise in in-process test teardown
setup_package and teardown_package get called twice when
running functional tests using nose. With in-process mode
the first call to teardown_package clears global config
which causes the test swiftclient to raise a SkipTest error
during the second call to teardown_package.

To reproduce:
SWIFT_TEST_IN_PROCESS=1 nosetests ./test/functional/test_account.py

This patch simply tests for config existence before attempting
to create a test swiftclient in the teardown_package function.

Also fix a related issue whereby in_process flag would be reset to
False during second call to setup_package, thus causing some of the
in process cleanup in teardown_package to never be executed.

Change-Id: I074dcd3d39aa46b262632024b047556ca471e8b8
2016-02-24 14:41:28 +00:00
Jenkins
a2405e99af Merge "Add functional test for access control (RBAC) with Keystone" 2016-02-11 15:07:39 +00:00
Alistair Coles
c7e5e12500 Enable in-process func tests to optionally use fast-post
Running functional tests in the in-process mode uses
the default value for proxy-server object_post_as_copy,
which is True. This patch adds support for an environment
variable to change this value to False so that the fast-post
mode is tested when running in-process functional tests.

The patch also adds a new tox environment func-in-process-fast-post
which forces in-process functional testing with
object_post_as_copy=False.

The motivation for this change, apart from enabling configurable
local testing, is to put support in place for an upstream CI job
that will actually functionally test the fast-post mode, which is
otherwise only covered by unit tests.

There are currently two gate jobs that run the functional tests:

- gate-swift-dsvm-functional runs the tests *twice* against a
  devstack swift service, once using tempauth and once using
  keystoneauth. The devstack swift service uses the default
  object_post_as_copy=True.

- gate-swift-tox-func runs the func tests in in-process mode
  which also uses tempauth and object_post_as_copy=True. This
  duplicates one of the config scenarios above.

With this change either the gate-swift-tox-func job or a new job
could run the functional tests using object_post_as_copy=False.

Change-Id: Ia37f6df1dc38e44ef7404fbf0a52f6fc22fae0c2
2016-02-01 12:39:39 +00:00
Hisashi Osanai
09b188f03c Add functional test for access control (RBAC) with Keystone
This patch adds test cases for PUT, DELETE, GET, HEAD, POST and OPTIONS
requests to accounts, containers and objects using various combinations
of users/projects, roles and/or service tokens.

Change-Id: Iea8141ac74ad949a3ae7fa47fda3135d0f2612f6
2016-01-19 01:08:36 +00:00
Alistair Coles
169a7c7f9e Fix func test --until-failure and --no-discover options
This patch changes functional test classes to subclass
unittest2.TestCase rather than unittest.TestCase.
This fixes errors when attempting to use

tox -e func -- -n <test_path_including_test_method>

and

tox -e func -- --until-failure

Also migrate from using nose.SkipTest to unittest2.SkipTest

Change-Id: I903033f5e01833550b2f2b945894edca4233c4a2
Closes-Bug: 1526725
Co-Authored-By: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2015-12-16 15:28:25 +00:00
Richard Hawkins
9d7f71d575 Modify functional tests to use ostestr/testr
Defcore uses Tempest, which uses Test Repository.
This change makes it easier for Defcore to pull functional
tests from Swift and run them.  Additionally, using testr
allows tests to be run in parallel.

Concurrency set to 1 for now, >1 causes failures for
reasons that are still TBD.

With switch to ostestr all the server logs are being sent to stdout
which makes it completely unreadable. Suppressing the logs by default
now with a flag to enable it if desired.

Co-Authored-By: John Dickinson <me@not.mn>
Co-Authored-By: Robert Collins <rbtcollins@hpe.com>
Co-Authored-By: Matthew Oliver <matt@oliver.net.au>
Co-Authored-By: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>

Change-Id: I53ef4a116996a772cf1f3abc2eb0ad60047322d5
Related-Bug: 1177924
2015-12-15 22:30:44 +00:00
Victor Stinner
c0af385173 py3: Replace urllib imports with six.moves.urllib
The urllib, urllib2 and urlparse modules of Python 2 were reorganized
into a new urllib namespace on Python 3. Replace urllib, urllib2 and
urlparse imports with six.moves.urllib to make the modified code
compatible with Python 2 and Python 3.

The initial patch was generated by the urllib operation of the sixer
tool on: bin/* swift/ test/.

Change-Id: I61a8c7fb7972eabc7da8dad3b3d34bceee5c5d93
2015-10-08 15:24:13 +02:00
janonymous
923238aa1b test/(functional/probe):Replace python print operator with print function (pep H233, py33)
'print' function is compatible with 2.x and 3.x python versions
Link : https://www.python.org/dev/peps/pep-3105/

Python 2.6 has a __future__ import that removes print as language syntax,
letting you use the functional form instead

Change-Id: I416c6ac21ccbfb91ec328ffb1ed21e492ef52d58
2015-08-20 11:42:58 +09:00