44 Commits

Author SHA1 Message Date
Paul Luse
e52e8bc917 Add Storage Policy Documentation
Add overview and example information for using Storage Policies.

DocImpact
Implements: blueprint storage-policies
Change-Id: I6f11f7a1bdaa6f3defb3baa56a820050e5f727f1
2014-06-19 10:18:34 -07:00
Matt Kassawara
05854c5134 Alphabetized items under "Misc" in source documentation
I alphabetized the items under "Misc" in the source documentation to
make them easier to locate.

Change-Id: I1badca7a97e7d887fefac0be2e2208eec52cec60
2014-02-20 19:24:33 -07:00
anc
b1742169e1 Remove duplicate doc entry for swob
Fixes swob module being referenced twice in misc.rst
resulting in duplicate sections in the doc.

Also fixes build_sphinx warning for section underline
too short in middleware.rst.

Change-Id: Ibe44895f933a6503ca04ccd3a084bc0cfd913213
2014-02-12 11:18:19 +00:00
David Goetz
89deaf5f3c Add flag to stop swob from always using absolute location.
This is needed for SOS (along with patch
https://github.com/dpgoetz/sos/pull/37)
to work with swift 1.12 . By spec you should always use the absolute
location but this causes a problem with staticweb over a cdn using a
cname. Basically you want to be able to forward the browser to a
relative location instead of whatever full url the proxy server
thinks you are using.

Change-Id: I3fa1d415bf9b566be069458b838f7e65db0c4f39
2014-01-24 13:37:45 -08:00
Clay Gerrard
63f8c2284a Add documentation for GateKeeper
The purpose of GateKeeper mostly relates to the development of new swift code,
so I threw together a guide for development_middleware that covers some basics
with a eye towards metadata handling in-particular.

I also fixed up some missing autodoc's, split out middleware autodoc and added
some ref's here and about so I could link to them from the
development_middleware guide.

DocImpact
Change-Id: I20dd942ea8df9e33c3e794cb49669ffa1332c63e
2014-01-23 15:04:23 -05:00
gholt
f60d05686f New container sync configuration option
Summary of the new configuration option:

The cluster operators add the container_sync middleware to their
proxy pipeline and create a container-sync-realms.conf for their
cluster and copy this out to all their proxy and container servers.
This file specifies the available container sync "realms".

A container sync realm is a group of clusters with a shared key that
have agreed to provide container syncing to one another.

The end user can then set the X-Container-Sync-To value on a
container to //realm/cluster/account/container instead of the
previously required URL.

The allowed hosts list is not used with this configuration and
instead every container sync request sent is signed using the realm
key and user key.

This offers better security as source hosts can be faked much more
easily than faking per request signatures. Replaying signed requests,
assuming it could easily be done, shouldn't be an issue as the
X-Timestamp is part of the signature and so would just short-circuit
as already current or as superceded.

This also makes configuration easier for the end user, especially
with difficult networking situations where a different host might
need to be used for the container sync daemon since it's connecting
from within a cluster. With this new configuration option, the end
user just specifies the realm and cluster names and that is resolved
to the proper endpoint configured by the operator. If the operator
changes their configuration (key or endpoint), the end user does not
need to change theirs.

DocImpact

Change-Id: Ie1704990b66d0434e4991e26ed1da8b08cb05a37
2014-01-10 23:48:00 +00:00
Richard (Rick) Hawkins
2c4bf81464 Added discoverable capabilities.
Swift can now optionally be configured to allow requests to '/info',
providing information about the swift cluster.  Additionally a HMAC
signed requests to
'/info?swiftinfo_sig=<sign>&swiftinfo_expires=<expires>' can be
configured allowing privileged access to more sensitive information
not meant to be public.

DocImpact
Change-Id: I2379360fbfe3d9e9e8b25f1dc34517d199574495
Implements: blueprint capabilities
Closes-Bug: #1245694
2013-11-22 15:54:13 -06:00
Peter Portante
9411a24ba7 Revert "Refactor common/utils methods to common/ondisk"
This reverts commit 7760f41c3ce436cb23b4b8425db3749a3da33d32

Change-Id: I95e57a2563784a8cd5e995cc826afeac0eadbe62
Signed-off-by: Peter Portante <peter.portante@redhat.com>
2013-10-07 17:18:09 -04:00
Peter Portante
7760f41c3c Refactor common/utils methods to common/ondisk
Place all the methods related to on-disk layout and / or configuration
into a new common module that can be shared by the various modules
using the same on-disk layout.

Change-Id: I27ffd4665d5115ffdde649c48a4d18e12017e6a9
Signed-off-by: Peter Portante <peter.portante@redhat.com>
2013-09-17 17:32:04 -04:00
Christian Schwede
e5cd78d902 Update documentation for account_quota middleware
Change-Id: I3dc7930ee2a1eb26b1f60e83fce2bc1bea0a8b0f
2013-04-03 15:10:42 +02:00
Christian Schwede
28c75db0e7 Account quotas
Add a new middleware implementing account quotas.

This middleware blocks write requests (PUT, POST) if a given quota (in bytes)
is exceeded while DELETE requests are still allowed.

Quotas are stored in the x-account-meta-quota-bytes metadata entry.
Write requests to this metadata setting are only allowed for resellers.

Change-Id: I57fd7c6209f34cc79d4bab72d500d43ba2a62083
2013-03-08 14:31:35 +01:00
Sergey Lukjanov
7d5095c122 Support listing endpoints for an object.
Implements blueprint list-endpoints.

DocImpact: new middleware list_endpoints.

Change-Id: I0c4911ff726abd4cb8ce2b6245c99786ad46b410
2013-03-07 01:38:21 +04:00
David Goetz
5d73da158b Static Large Object Support
DocImpact

Change-Id: I7edaa5e44208ab451f7f7566b64bb571b8eea1f9
2013-03-01 16:46:10 -08:00
Chmouel Boudjnah
1b86380e2e Add generated keystone documentation to misc.rst.
- Change a bit the formatting of the documention as well.
- Fix WARNING: Title underline too short. in misc.rst.

Change-Id: I2f4e36bcb5e01e984f0af0152bc5b3b9f7e942ce
2013-02-11 21:44:53 +01:00
Michael Barton
24ef12027c Basic container quotas
Add a new middleware implementing some basic container quotas.

Quotas are subject to several limitations: eventual consistency, the timeliness
of the cached container_info (60 second ttl by default), and it’s unable to
reject chunked transfer uploads that exceed the quota (though once the quota
is exceeded, new chunked transfers will be refused).

However, they get most of the way to container quotas fairly inexpensively.

Quotas are set by adding meta values to the container, and are validated when
set:

  X-Container-Meta-Quota-Bytes: Maximum size of the container, in bytes.
  X-Container-Meta-Quota-Count: Maximum object count of the container.

DocImpact

Change-Id: I77cfbf6dc231a2e522bd67328e4c082424a93eee
2013-02-05 06:03:38 -08:00
David Goetz
2f663ff9a0 Bulk Requests: auto extract archive and bulk delete middleware.
Fix small problem in ratelimiting middleware.

DocImpact

Change-Id: Ide3e0b9f4887626c30cae0b97eb7e2237b1df3ed
2013-01-24 12:34:56 -08:00
Adrian Smith
89ee10bd92 Add handler for CORS "actual requests"
Fix for bug 1095130

* Added a wrapper function around public methods to handle
  CORS actual requests. These requests need to return some
  extra headers to be valid responses to a CORS request.
  Access-Control-Expose-Headers and Access-Control-Allow-Origin.

* Added support for the CORS header Access-Control-Expose-Headers.

* Some refactoring of the OPTIONS method so the
  "is_origin_allowed" logic can be reused.

* Added a little extra detail to the CORS documentation.

DocImpact

Change-Id: I68538e472a900775427f21a8a59e738a83dcc8bc
2013-01-23 19:31:18 +00:00
Scott Simpson
74b27d504d Adding CORS support
Change-Id: I894473994cdfea0996ad16e7619aff421f604abc
2012-10-23 14:48:59 -05:00
Chmouel Boudjnah
22572b506a Remove swift.common.client from here.
- It has been to its own gerrit project.
- direct_client should follow next.
- Implements blueprint clientbindings.

Change-Id: I3bb50c95eba81302bfec71cb7ce5288b85a41dc0
2012-06-07 16:36:49 +02:00
Michael Barton
7c98e7a625 Move proxy server logging to middleware.
Change-Id: I771c87207d4e1821e32c3424b341d182cc7ea7c0
2012-05-24 21:15:51 -07:00
Chmouel Boudjnah
d02a73f4a9 Remove swift3 from here.
- Reference https://github.com/fujita/swift3 in associated_projects.
- Implements blueprint add-associated-projects-docs.

Change-Id: I48ef4c03449edf6ef4fda1a391228cacac7d2ac6
2012-05-24 10:21:42 +02:00
gholt
1c3b75c291 Reverted the pulling out of various middleware:
RateLimit
StaticWeb
TempURL/FormPOST

Change-Id: I988e93e6f4aacb817a2e354d43a04e47516fdf88
2012-05-16 21:25:10 +00:00
John Dickinson
b47bcf19e4 removed cname lookup middleware
The code has moved to https://github.com/notmyname/swift-cnamelookup.

For current users of cname lookup, this will require installing the new
package and changing the "use" line of the cname lookup conf section's
to:

[filter:cname_lookup]
use = egg:swift_cnamelookup#swift_cnamelookup

And then 'swift-init proxy reload'.

Change-Id: If622486ddb04a53251244c9840aa3cfe72168fc5
2012-05-09 14:00:51 -05:00
gholt
3f00c1a630 Pulled out Rate Limit middleware
Rate Limit middleware is now at
http://dpgoetz.github.com/swift-ratelimit/

For current users of Rate Limit, this will require installing the new
package and changing the "use" line of the ratelimit conf section to:

[filter:ratelimit]
use = egg:swiftratelimit#middleware

And then 'swift-init proxy reload'.

Change-Id: I2ab774e9cee9fba4103c1be4bea6d52d1adb29f7
2012-05-09 18:44:45 +00:00
John Dickinson
7dfbd785b0 removed domain remap middleware
The code has moved to https://github.com/notmyname/swift-domainremap.

For current users of domain remap, this will require installing the new
package and changing the "use" line of the domain remap conf section's
to:

[filter:domain_remap]
use = egg:swift_domainremap#swift_domainremap

And then 'swift-init proxy reload'.

Change-Id: I710caf9b991f9d37df36b826ae4338086d0ec36d
2012-05-08 21:30:35 -05:00
gholt
c0532a6ef2 Pulled out TempURL/FormPOST
TempURL/FormPOST is now at http://gholt.github.com/swift-tempurl/

For current users of TempURL/FormPOST, this will require installing the new
package and changing the "use" line of the tempurl and formpost conf section's
to:

[filter:tempurl]
use = egg:swifttempurl#tempurl

[filter:formpost]
use = egg:swifttempurl#formpost

And then 'swift-init proxy reload'.

Change-Id: I5bddf7f9e09ee07815530a41c46ff901fc21b447
2012-05-08 20:20:29 +00:00
Jenkins
8d2e7bd112 Merge "Pulled StaticWeb out to separate project" 2012-05-08 18:43:38 +00:00
gholt
7dde909621 Pulled StaticWeb out to separate project
StaticWeb is now at http://gholt.github.com/swift-staticweb/

For current users of StaticWeb, this will require installing the new
package and changing the "use" line of the staticweb filter conf
section to:

use = egg:swiftstaticweb#middleware

And then 'swift-init proxy reload'.

Change-Id: Iab32adb5927698a667c5c6d6a572c44ca23414eb
2012-05-05 00:40:33 +00:00
Greg Lange
8d2fe89a7d Added an internal client.
Refactored object expirer to use this client.

Change-Id: Ibeca6dba873f8b4a558ecf3ba6e8d23d36f545b0
2012-05-04 18:07:54 +00:00
John Dickinson
edf2637026 updated docs for domain remap and cname lookup middleware
Change-Id: I1c571951f25a6e724cdd4699eb94baad5e47eb95
2012-04-05 15:02:15 -05:00
gholt
7fc1721d7d TempURL and FormPost Middleware
Change-Id: I8d2ce2abdfe3a44605c9441ad7b1abc6c77e282d
2012-01-10 21:57:52 +00:00
Florian Hines
bb8c4eab41 Add documentation for Swift Recon.
Change-Id: I37f4fb624bdc5b8bbf2e691d29aa6b15cd648aa8
2011-10-21 00:17:10 +00:00
gholt
f68b6354e2 Renaming TestAuth to TempAuth because nose hates anything with the word test in it. 2011-05-26 02:24:12 +00:00
gholt
3ee4a01100 Remove swauth; update references from swauth to testauth. 2011-05-26 02:17:42 +00:00
gholt
d872d94411 Update from feedback; docs 2011-03-24 03:37:07 +00:00
gholt
34f82afc62 Fixes with removing DevAuth 2011-03-14 03:08:23 +00:00
Clay Gerrard
22a45b3550 review cleanup 2011-02-14 14:52:49 -06:00
Chuck Thier
e2c0a23839 Added doc strings and pointers to docs for swift3 2011-01-14 13:49:05 -06:00
gholt
35f3487879 Incorporated Swauth into Swift as an optional DevAuth replacement. 2010-12-01 17:08:49 -08:00
David Goetz
a6251e8c87 changing source docs 2010-10-13 08:50:11 -07:00
David Goetz
f7c7120798 adding source docs 2010-10-13 08:43:37 -07:00
gholt
65eb19f103 Documentation of the new auth and acls middleware modules and bugfixes 2010-09-03 22:33:41 -07:00
Chuck Thier
386eb29b1f Updated docs (including SAIO) to refelct the paste.deploy config changes 2010-08-20 02:19:50 +00:00
Chuck Thier
001407b969 Initial commit of Swift code 2010-07-12 17:03:45 -05:00