105 Commits

Author SHA1 Message Date
John Dickinson
85c03d8b1d taking the global reqs that we can
Change-Id: Icfdeeab14e90728aa8167ad6587d51e9b10cb846
2014-05-21 09:37:22 -07:00
John Dickinson
d47a582e74 update setup.py with pbr version
This file is normally updated in the standard "update from
global requirements" patch, but since those contain other
changes we don't want, the setup.py file never gets updated.

This patch updates setup.py to bring it in line with the
standard global requirements way of doing things without also
updating [test-]requirements.txt

Change-Id: Id0e4f8e17dd4c714b4d2dd75985431a9e12e6a4f
2014-04-08 10:26:40 -07:00
Monty Taylor
4332bff3f5 Migrate to pbr for build
pbr is the libification of what was openstack.common.setup. If provides
the build information in a delcarative form, instead of as executable python
code, which works around the chicken and egg problem of needing setup
libraries present to run setup, but needing to run setup to tell if you
need setup libraries.

One of the features that comes along with this is versioning based on
git tags. If the current revision is a signed git tag, then that is the
version of the package. If it is not, the version is equal to the most
recent git tag, plus a commit count, plus a git sha (similar to git
describe, but scrubbed for python version rules compliance)

pbr updates are also part of the upcoming automation around ensuring
global requirements stay in sync.

Closes-Bug: #1179007
Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-08-14 19:10:07 -03:00
niu-zglinux
627d0ba52f Rename requires files to standard names.
Rename tools/pip-requires to requirements.txt and tools/test-requires
to test-requirements.txt. These are standard files, and tools in the
general world are growing intelligence about them.

Change-Id: Ib3e50a811868e2969923d978ee00c4f92682aa1c
Fixes: bug #1179008
2013-05-30 22:14:01 +00:00
Clay Gerrard
34f5085c3e conf.d support
Allow Swift daemons and servers to optionally accept a directory as the
configuration parameter.  Directory based configuration leverages
ConfigParser's native multi-file support.  Files ending in '.conf' in the
given directory are parsed in lexicographical order.  Filenames starting with
'.' are ignored.  A mixture of file and directory configuration paths is not
supported - if the configuration path is a file behavior is unchanged.

 * update swift-init to search for conf.d paths when building servers
    (e.g. /etc/swift/proxy-server.conf.d/)
 * new script swift-config can be used to inspect the cumulative configuration
 * pull a little bit of code out of run_wsgi and test separately
 * fix example config bug for the proxy servers client_disconnect option
 * added section on directory based configuration to deployment guide

DocImpact

Implements: blueprint confd
Change-Id: I89b0f48e538117f28590cf6698401f74ef58003b
2013-04-30 00:17:46 -07:00
Donagh McCabe
eb4b29d243 Add crossdomain.xml middleware
Allows client-side technologies such as Flash, Java and Silverlight running
on web pages served elsewhere to interact with the Swift API.

Bug #1159960

Change-Id: I7d0533a0aaf189ac452abbd983469acb064fdca4
2013-04-23 13:52:34 +01:00
Christian Schwede
28c75db0e7 Account quotas
Add a new middleware implementing account quotas.

This middleware blocks write requests (PUT, POST) if a given quota (in bytes)
is exceeded while DELETE requests are still allowed.

Quotas are stored in the x-account-meta-quota-bytes metadata entry.
Write requests to this metadata setting are only allowed for resellers.

Change-Id: I57fd7c6209f34cc79d4bab72d500d43ba2a62083
2013-03-08 14:31:35 +01:00
Sergey Lukjanov
7d5095c122 Support listing endpoints for an object.
Implements blueprint list-endpoints.

DocImpact: new middleware list_endpoints.

Change-Id: I0c4911ff726abd4cb8ce2b6245c99786ad46b410
2013-03-07 01:38:21 +04:00
David Goetz
5d73da158b Static Large Object Support
DocImpact

Change-Id: I7edaa5e44208ab451f7f7566b64bb571b8eea1f9
2013-03-01 16:46:10 -08:00
Chuck Thier
5d2b70b624 Updating classifiers status to stable and clean up
Change-Id: I72943a9575b873cef1bf193f197b390b544ed6bf
2013-02-27 15:08:46 -06:00
Michael Barton
24ef12027c Basic container quotas
Add a new middleware implementing some basic container quotas.

Quotas are subject to several limitations: eventual consistency, the timeliness
of the cached container_info (60 second ttl by default), and it’s unable to
reject chunked transfer uploads that exceed the quota (though once the quota
is exceeded, new chunked transfers will be refused).

However, they get most of the way to container quotas fairly inexpensively.

Quotas are set by adding meta values to the container, and are validated when
set:

  X-Container-Meta-Quota-Bytes: Maximum size of the container, in bytes.
  X-Container-Meta-Quota-Count: Maximum object count of the container.

DocImpact

Change-Id: I77cfbf6dc231a2e522bd67328e4c082424a93eee
2013-02-05 06:03:38 -08:00
David Goetz
2f663ff9a0 Bulk Requests: auto extract archive and bulk delete middleware.
Fix small problem in ratelimiting middleware.

DocImpact

Change-Id: Ide3e0b9f4887626c30cae0b97eb7e2237b1df3ed
2013-01-24 12:34:56 -08:00
Julien Danjou
dcff5ff516 Use install_requires in setup.py
Using install_requires makes sure that if some code is using Swift internal
and wants to require Swift, all its dependencies have been already pulled.
This is really useful on test automation for example, otherwise, code might
be failing because on of the module listed in pip-requires is not instaled.

Also, we change the pip-requires files to use >= rather than ==, so the
requirements are easily fulfiled with distribution provided packages.

Change-Id: I65814bcd8ce798da21a5c17b4d5916a23f59e962
Signed-off-by: Julien Danjou <julien@danjou.info>
2012-12-12 11:18:45 +01:00
Doug Hellmann
529f410605 Add OpenStack trove classifier for PyPI
Add trove classifier to have swift listed among the
other OpenStack-related projets on PyPI to reserve
the name, even though we won't release packages there.

Change-Id: I2c6cd2e37ea55d2bc2ddbba813ee2a190b1c90a9
Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>
2012-10-22 18:55:15 -04:00
Tom Fifield
9344a4a582 Random pep8 fixes!
This patch merely fixes a selection of files to the point where
pep8 1.3.3 is happy. Most of the errors are indentation related to
continued lines (E126, E127, E128), bracket positions (E124) and the
use of backslash (E502).

Patch 2 fixes David's comments regarding backslash and an odd comment
 - thanks David!

Change-Id: I4fbd77ecf5395743cb96acb95fa946c322c16560
2012-10-13 12:04:25 +11:00
Darrell Bishop
ed3b12d05c Can run swift-bench across multiple cores/servers.
You run one or more swift-bench-client processes like this:
    $ swift-bench-client 127.0.0.1 20001
    $ swift-bench-client 127.0.0.1 20002

Then you run swift-bench with a new option, --bench-clients (-b), which is
specified once for each swift-bench-client:
    $ swift-bench -b 127.0.0.1:20001 -b 127.0.0.1:20002

You get log lines from each client (interleaved) along with a final report
for all clients:
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:44:06,148 INFO Auth version: 1.0
    127.0.0.1:20001 swift-bench-server 2012-08-25 22:44:06,148 INFO Auth version: 1.0
    127.0.0.1:20001 swift-bench-server 2012-08-25 22:44:12,249 INFO 83 PUTS [0 failures], 41.5/s
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:44:14,430 INFO 74 PUTS [0 failures], 34.3/s
    ...
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:45:18,942 INFO Auth version: 1.0
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:45:20,946 INFO 238 DEL [2 failures], 118.9/s
    swift-bench 2012-08-25 22:45:27,549 INFO 2000 PUTS **FINAL** [0 failures], 56.8/s
    swift-bench 2012-08-25 22:45:27,550 INFO 30000 GETS **FINAL** [50 failures], 974.6/s
    swift-bench 2012-08-25 22:45:27,550 INFO 2000 DEL **FINAL** [20 failures], 237.1/s

The concurrency, PUT count, and GET count config settings are divided by
the number of bench_clients.  In other words, the same volume of work is
attempted (vs. not specifying --bench-clients), but it can now span
servers and CPU cores.

Benchmark containers are created (if use_proxy = yes) and deleted (if
delete = yes), with appropriate concurrency, in the initiating
swift-bench process, not any of the swift-bench-client processes.

Change-Id: Idbf31a23093244ab357a9bf77e6031257774f24a
2012-08-30 18:22:49 +00:00
Chmouel Boudjnah
afa4f70024 Move swift_auth middleware from keystone to swift.
- Rename it to keystoneauth for consistenties.
- Implements blueprint keystone-middleware.

Change-Id: I208fecdf3ee991694b4239f065032324d297fd35
2012-07-05 14:36:14 -04:00
Chmouel Boudjnah
22572b506a Remove swift.common.client from here.
- It has been to its own gerrit project.
- direct_client should follow next.
- Implements blueprint clientbindings.

Change-Id: I3bb50c95eba81302bfec71cb7ce5288b85a41dc0
2012-06-07 16:36:49 +02:00
Michael Barton
7c98e7a625 Move proxy server logging to middleware.
Change-Id: I771c87207d4e1821e32c3424b341d182cc7ea7c0
2012-05-24 21:15:51 -07:00
Chmouel Boudjnah
d02a73f4a9 Remove swift3 from here.
- Reference https://github.com/fujita/swift3 in associated_projects.
- Implements blueprint add-associated-projects-docs.

Change-Id: I48ef4c03449edf6ef4fda1a391228cacac7d2ac6
2012-05-24 10:21:42 +02:00
John Dickinson
1e90b61076 Re-add cname lookup and domain remap middleware
Revert "removed cname lookup middleware"

This reverts commit b47bcf19e41e862ca84d77a7b8843f836e084b6a.

Revert "removed domain remap middleware"

This reverts commit 317cf868bdf66dbc17c02d4ca383efafa5e2f229.

Change-Id: I260498d555c93b28896ace48a6f0e96701cbcc38
2012-05-21 20:02:04 -05:00
gholt
1c3b75c291 Reverted the pulling out of various middleware:
RateLimit
StaticWeb
TempURL/FormPOST

Change-Id: I988e93e6f4aacb817a2e354d43a04e47516fdf88
2012-05-16 21:25:10 +00:00
John Dickinson
b47bcf19e4 removed cname lookup middleware
The code has moved to https://github.com/notmyname/swift-cnamelookup.

For current users of cname lookup, this will require installing the new
package and changing the "use" line of the cname lookup conf section's
to:

[filter:cname_lookup]
use = egg:swift_cnamelookup#swift_cnamelookup

And then 'swift-init proxy reload'.

Change-Id: If622486ddb04a53251244c9840aa3cfe72168fc5
2012-05-09 14:00:51 -05:00
gholt
3f00c1a630 Pulled out Rate Limit middleware
Rate Limit middleware is now at
http://dpgoetz.github.com/swift-ratelimit/

For current users of Rate Limit, this will require installing the new
package and changing the "use" line of the ratelimit conf section to:

[filter:ratelimit]
use = egg:swiftratelimit#middleware

And then 'swift-init proxy reload'.

Change-Id: I2ab774e9cee9fba4103c1be4bea6d52d1adb29f7
2012-05-09 18:44:45 +00:00
John Dickinson
7dfbd785b0 removed domain remap middleware
The code has moved to https://github.com/notmyname/swift-domainremap.

For current users of domain remap, this will require installing the new
package and changing the "use" line of the domain remap conf section's
to:

[filter:domain_remap]
use = egg:swift_domainremap#swift_domainremap

And then 'swift-init proxy reload'.

Change-Id: I710caf9b991f9d37df36b826ae4338086d0ec36d
2012-05-08 21:30:35 -05:00
gholt
c0532a6ef2 Pulled out TempURL/FormPOST
TempURL/FormPOST is now at http://gholt.github.com/swift-tempurl/

For current users of TempURL/FormPOST, this will require installing the new
package and changing the "use" line of the tempurl and formpost conf section's
to:

[filter:tempurl]
use = egg:swifttempurl#tempurl

[filter:formpost]
use = egg:swifttempurl#formpost

And then 'swift-init proxy reload'.

Change-Id: I5bddf7f9e09ee07815530a41c46ff901fc21b447
2012-05-08 20:20:29 +00:00
gholt
7dde909621 Pulled StaticWeb out to separate project
StaticWeb is now at http://gholt.github.com/swift-staticweb/

For current users of StaticWeb, this will require installing the new
package and changing the "use" line of the staticweb filter conf
section to:

use = egg:swiftstaticweb#middleware

And then 'swift-init proxy reload'.

Change-Id: Iab32adb5927698a667c5c6d6a572c44ca23414eb
2012-05-05 00:40:33 +00:00
John Dickinson
1ecf5ebba1 updated copyright date for all files
Change-Id: Ifd909d3561c2647770a7e0caa3cd91acd1b4f298
2012-03-19 13:45:34 -05:00
Eamonn O'Toole
cf1aa3c309 Adds name_check filter
Bug 926048.

Filter checks path for user-defined forbidden characters, and for
user-defined maximum length.

Includes changes to reflect gholt's latest comments to Patch Set 4
Also includes a change to a unit-test, renames another unit-test,
and removes one superfluous unit-test.

Added section to the example proxy config

Fixed-up unit test pep8 warnings

Changed error response code to 400 (Bad Request)

Change-Id: Iace719d6a3d00fb3dda1b9d0bc185b8c4cbc00ca
2012-03-08 17:23:58 +00:00
gholt
7141c52bd6 Remove old stats tools.
https://blueprints.launchpad.net/swift/+spec/remove-old-swift-stats-tools

Change-Id: Ibb57b3d71c7f18060d2d48a1d3754a7f1444624d
2012-03-06 17:57:16 +00:00
gholt
7fc1721d7d TempURL and FormPost Middleware
Change-Id: I8d2ce2abdfe3a44605c9441ad7b1abc6c77e282d
2012-01-10 21:57:52 +00:00
gholt
65dba1a7aa Added swift-orphans and swift-oldies.
Change-Id: I95210098556a22d7bd05f245ae387ee13041fa61
2011-12-29 19:19:41 +00:00
gholt
872420efdb Expiring Objects Support
Please see the doc/source/overview_expiring_objects.rst for
more detail.

Change-Id: I4ab49e731248cf62ce10001016e0c819cc531738
2011-11-01 15:49:00 +00:00
John Dickinson
e76bfa67d4 simplified setup.py 2011-08-19 10:58:18 -05:00
Florian Hines
4d0e4622de add binary's/recon.py to setup.py 2011-08-11 15:17:28 -05:00
gholt
8c24a70139 Merged from trunk 2011-06-30 21:52:09 +00:00
John Dickinson
65d79d18b5 merged with trunk 2011-06-15 14:05:28 -05:00
gholt
ba4f74d4fe Merge from trunk 2011-06-14 23:13:47 +00:00
John Dickinson
0eeeae664e renamed st to swift 2011-06-14 11:04:06 -05:00
John Dickinson
5949217ce1 removed basic slogging code 2011-06-13 15:57:32 -05:00
gholt
b4221114c5 Merged with deswauth 2011-06-03 00:11:32 +00:00
gholt
ce710a54a5 Merged with trunk 2011-06-01 20:43:10 +00:00
gholt
288ccb1703 Merged from trunk 2011-05-31 18:09:53 +00:00
Soren Hansen
2d1274c497 Add a __canonical_version__ attribute to the swift module. This is only the numbered part of the version string, no "-dev" or similar suffixes.
Add a couple of unit tests to make sure __version__ and __canonical_version__ are generated correctly and to ensure __canonical_version__ never accidentally has anything other than numbers and points in it.
2011-05-26 11:25:39 +02:00
gholt
f68b6354e2 Renaming TestAuth to TempAuth because nose hates anything with the word test in it. 2011-05-26 02:24:12 +00:00
gholt
3ee4a01100 Remove swauth; update references from swauth to testauth. 2011-05-26 02:17:42 +00:00
gholt
b24b063552 Created testauth WSGI middleware to replace Swauth as development auth service. 2011-05-26 01:19:03 +00:00
David Goetz
bc894f9efb update with trunk 2011-05-12 14:24:51 -07:00
Jay Payne
56d00c2f29 Added bin/swift-container-stats-logger 2011-05-02 23:49:44 +00:00
gholt
0546c8d15f Merged from trunk 2011-04-15 23:11:34 +00:00