OpenStack Storage (Swift)
Go to file
Samuel Merritt 3530708619 Stop mutating PATH_INFO in proxy server
The proxy server was calling swob.Request.path_info_pop() prior to
instantiating a controller so that req.path_info was just /a/c/o (sans
/v1). The version got moved over into SCRIPT_NAME.

This lead to some unfortunate behavior when trying to re-use a request
from middleware. Something like this:

    # Imagine we're a WSGIContext object here.
    #
    # To start, SCRIPT_NAME = '' and PATH_INFO='/v1/a/c/o'

    resp_iter = self._app_call(env, start_response)
    # Now SCRIPT_NAME='/v1' and PATH_INFO ='/a/c/o'

    if something_special in self._response_headers:
        env['REQUEST_METHOD'] = 'GET'
        env.pop('HTTP_RANGE', None)

	# 404 SURPRISE! The proxy calls path_info_pop() again,
        # and now SCRIPT_NAME='/v1/a' and PATH_INFO='/c/o', so this
        # gets treated as a container request. Yikes.
        resp_iter = self._app_call(env, start_response)

Now we just leave SCRIPT_NAME and PATH_INFO alone. To make life easy
for everyone who does want just /a/c/o, I defined
swob.Request.swift_entity_path, which just strips off the /v1.

Note that there's still one call to path_info_pop() in tempauth, but
that's only for requests going to /auth, so it won't affect Swift API
requests. It might be a good idea to remove that one later, but let's
do one thing at a time.

Change-Id: I87557a11c01f3f3889b610578cda6ba7d3933e7a
2013-12-06 10:57:37 -08:00
bin Merge "Remove swift-bench" 2013-12-06 00:00:54 +00:00
doc Merge "Added discoverable capabilities." 2013-11-28 00:11:35 +00:00
etc Merge "Remove swift-bench" 2013-12-06 00:00:54 +00:00
examples Add a user variable to templates 2013-09-17 11:46:04 +10:00
locale Reverted the pulling out of various middleware: 2012-05-16 21:25:10 +00:00
swift Stop mutating PATH_INFO in proxy server 2013-12-06 10:57:37 -08:00
test Stop mutating PATH_INFO in proxy server 2013-12-06 10:57:37 -08:00
.coveragerc Align tox.ini and fix coverage jobs in jenkins. 2012-06-08 20:05:14 -04:00
.functests Allow dot test runners from any dir 2012-12-07 14:08:49 -08:00
.gitignore fix(gitignore) : ignore *.egg and *.egg-info 2013-07-30 15:11:00 -04:00
.gitreview Add .gitreview config file for gerrit. 2011-10-24 15:05:49 -04:00
.mailmap Update my mailmap 2013-10-25 16:29:16 +08:00
.probetests Allow dot test runners from any dir 2012-12-07 14:08:49 -08:00
.unittests Add HTML reporting for test branch coverage. 2013-11-27 12:07:42 -05:00
AUTHORS CHANGELOG and AUTHORS updates for 1.10.0 release 2013-10-08 23:58:13 -07:00
babel.cfg add pybabel setup.py commands and initial .pot 2011-01-27 00:01:24 +00:00
CHANGELOG CHANGELOG and AUTHORS updates for 1.10.0 release 2013-10-08 23:58:13 -07:00
CONTRIBUTING.md HEAD on account returns 410 if account was deleted and not yet reaped 2013-10-29 13:49:38 -07:00
LICENSE Convert LICENSE to use unix style line endings. 2012-12-19 12:48:27 -05:00
MANIFEST.in Add requirements files to the source distribution 2013-06-03 19:26:20 +04:00
README.md Correct URL in readme 2013-10-07 22:27:34 -07:00
requirements.txt Make pbr a build-time only dependency 2013-10-29 12:29:49 -07:00
setup.cfg Remove swift-bench 2013-12-02 12:46:41 -08:00
setup.py Migrate to pbr for build 2013-08-14 19:10:07 -03:00
test-requirements.txt Start using Hacking 2013-07-15 11:41:58 +02:00
tox.ini Remove Python 3.3 from list of tox environments 2013-12-02 13:54:40 -05:00

Swift

A distributed object storage system designed to scale from a single machine to thousands of servers. Swift is optimized for multi-tenancy and high concurrency. Swift is ideal for backups, web and mobile content, and any other unstructured data that can grow without bound.

Swift provides a simple, REST-based API fully documented at http://docs.openstack.org/.

Swift was originally developed as the basis for Rackspace's Cloud Files and was open-sourced in 2010 as part of the OpenStack project. It has since grown to include contributions from many companies and has spawned a thriving ecosystem of 3rd party tools. Swift's contributors are listed in the AUTHORS file.

Docs

To build documentation install sphinx (pip install sphinx), run python setup.py build_sphinx, and then browse to /doc/build/html/index.html. These docs are auto-generated after every commit and available online at http://docs.openstack.org/developer/swift/.

For Developers

The best place to get started is the "SAIO - Swift All In One". This document will walk you through setting up a development cluster of Swift in a VM. The SAIO environment is ideal for running small-scale tests against swift and trying out new features and bug fixes.

You can run unit tests with .unittests and functional tests with .functests.

Code Organization

  • bin/: Executable scripts that are the processes run by the deployer
  • doc/: Documentation
  • etc/: Sample config files
  • swift/: Core code
    • account/: account server
    • common/: code shared by different modules
      • middleware/: "standard", officially-supported middleware
      • ring/: code implementing Swift's ring
    • container/: container server
    • obj/: object server
    • proxy/: proxy server
  • test/: Unit and functional tests

Data Flow

Swift is a WSGI application and uses eventlet's WSGI server. After the processes are running, the entry point for new requests is the Application class in swift/proxy/server.py. From there, a controller is chosen, and the request is processed. The proxy may choose to forward the request to a back- end server. For example, the entry point for requests to the object server is the ObjectController class in swift/obj/server.py.

For Deployers

Deployer docs are also available at http://docs.openstack.org/developer/swift/. A good starting point is at http://docs.openstack.org/developer/swift/deployment_guide.html

You can run functional tests against a swift cluster with .functests. These functional tests require /etc/swift/test.conf to run. A sample config file can be found in this source tree in test/sample.conf.

For Client Apps

For client applications, official Python language bindings are provided at http://github.com/openstack/python-swiftclient.

Complete API documentation at http://docs.openstack.org/api/openstack-object-storage/1.0/content/


For more information come hang out in #openstack-swift on freenode.

Thanks,

The Swift Development Team