420 Commits

Author SHA1 Message Date
tong li
68355d7ea5 Instruction for multi node setup misses /var/run/swift creation
Multiple Server Swift Installation (Ubuntu) instruction does not
indicate that the directory /var/run/swift needs to be created.
That directory actually needs to be created and the ownship needs
to be changed to the user/group which swift service runs under.

This patch will fix the document and gives the steps how to create
the directory and set the ownership right. It also gives instruction
on how the script can be added so that swift services can be resarted
after system reboots.

Change-Id: Id61aa67cc0d6f66d749701e6ea824b1ff3b2c478
Fixes: bug #1156631
2013-03-18 13:57:40 -04:00
Jenkins
94defcbc53 Merge "Add tests and doc entry for request.environ[reseller_request]" 2013-03-16 19:29:47 +00:00
Samuel Merritt
ebcd60f7d9 Add a region tier to Swift's ring.
The region is one level above the zone; it is intended to represent a
chunk of machines that is distant from others with respect to
bandwidth and latency.

Old rings will default to having all their devices in region 1. Since
everything is in the same region by default, the ring builder will
simply distribute across zones as it did before, so your partition
assignment won't move because of this change. If you start adding
devices in other regions, of course, the assignment will change to
take that into account.

swift-ring-builder still accepts the same syntax as before, but will
default added devices to region 1 if no region is specified.

Examples:

$ swift-ring-builder foo.builder add r2z1-1.2.3.4:555/sda

$ swift-ring-builder foo.builder add r1z3-1.2.3.4:555/sda

$ swift-ring-builder foo.builder add z3-1.2.3.4:555/sda

Also, some updates to ring-overview doc.

Change-Id: Ifefbb839cdcf033e6c9201fadca95224c7303a29
2013-03-13 10:00:58 -07:00
Christian Schwede
157c3c91ee Add tests and doc entry for request.environ[reseller_request]
The recent account_quotas (https://review.openstack.org/23434)
patch added a new setting request.environ[reseller_request].
This patch adds tests for tempauth and keystoneauth as well as
an updated overview_auth.rst.

Change-Id: Icdb7ec9948ae7424b0721fc51a143782b2fdc5a6
2013-03-10 19:21:13 +01:00
Christian Schwede
28c75db0e7 Account quotas
Add a new middleware implementing account quotas.

This middleware blocks write requests (PUT, POST) if a given quota (in bytes)
is exceeded while DELETE requests are still allowed.

Quotas are stored in the x-account-meta-quota-bytes metadata entry.
Write requests to this metadata setting are only allowed for resellers.

Change-Id: I57fd7c6209f34cc79d4bab72d500d43ba2a62083
2013-03-08 14:31:35 +01:00
Jenkins
09f1387af5 Merge "Support tests for Apache" 2013-03-07 13:33:48 +00:00
David Hadas
40782ed20c Support tests for Apache
Add support for functional tests that work with Apache web front end

Change-Id: I72358a12016eeccc842d834461dbebaa188aa117
Implements: blueprint wsgi-application-interface
2013-03-07 01:32:16 +02:00
Sergey Lukjanov
7d5095c122 Support listing endpoints for an object.
Implements blueprint list-endpoints.

DocImpact: new middleware list_endpoints.

Change-Id: I0c4911ff726abd4cb8ce2b6245c99786ad46b410
2013-03-07 01:38:21 +04:00
Jenkins
28ac46ded3 Merge "Added per disk PUT timing monitoring support." 2013-03-06 01:59:23 +00:00
David Goetz
5d73da158b Static Large Object Support
DocImpact

Change-Id: I7edaa5e44208ab451f7f7566b64bb571b8eea1f9
2013-03-01 16:46:10 -08:00
Jian Zhang
1d8a02f25c Added per disk PUT timing monitoring support.
Fixes bug 1104708

There could be severe performance drop for swift is one disk of one
storage node is problematic due to the tragic state of async disk I/O.

This patch provided PUT timing per kB transfered (ms/kB) monitoring
support for each non-zero-byte request of each disk and report to
statsD for alert.
-adding "object-server.PUT.<device>.timing" metrics for object-server.

DocImpact.

Change-Id: Ie94bddad28e8be52e71683bf6c9db988664abe47
2013-02-28 02:52:06 -08:00
Jenkins
2a119b650e Merge "Move the swift.1 manpage to python-swiftclient" 2013-02-27 16:50:17 +00:00
Jenkins
a06c71c624 Merge "Add cache=swift.cache for authtoken example." 2013-02-27 16:10:57 +00:00
Pete Zaitcev
9a3e0f41bd Move the swift.1 manpage to python-swiftclient
When someone split away python-swiftclient, he left behind the
client manpage.

Apparently all we need now is to delete it from swift and
add it to python-swiftclient. Except that of course the releases
have to be synchronized somehow or else files conflict.

Change-Id: I8b5f7b5557c28818048d8941df36473dacfb1d57
2013-02-26 12:41:27 -07:00
Jenkins
249a65461e Merge "Adding speed limit options for DB auditor" 2013-02-26 06:22:25 +00:00
Jenkins
d1381ba86b Merge "Adds first-byte latency timings for GET requests." 2013-02-23 01:10:58 +00:00
Chmouel Boudjnah
e69f3bef8f Add cache=swift.cache for authtoken example.
- Things swill go badly with swift if we leave the default to authtoken
  to use its own memcache cache connection based python-memcache c based
  binding.

Change-Id: I293b875acdcb06e5a7a0cfa9a9bb5d7678675da0
2013-02-21 22:58:27 +01:00
yuan-zhou
09370862ca Adding speed limit options for DB auditor
Fix bug 1129760

Without speed limit, DB auditor will likely consume high CPU% on
storage node. That will highly impact the cluster's performance.

This patch adds two options for account/container auditor:
 - containers_per_second: Maximum containers audited per second
 - accounts_per_second: Maximum accounts audited per second

DocImpact

Change-Id: I9faa506438185a83ca77db4906969328624d015f
2013-02-20 13:54:59 +08:00
Pete Zaitcev
93ea7c63b1 Documentation fixups
These are mostly cosmetic fixes for irritating imperfections:
- "separated with commas" was duplicated, leave just one
- extra whitespace here and there, man pages are not PEP8, drop
- weird extra commas, drop
- Fedora logs to /var/log/messages
- "drive is has failed", drop "is"

Change-Id: I5ceba2e61b16db4855d76c92cbc83663b9b2a0da
2013-02-18 10:54:27 -07:00
Chuck Thier
e88ff34685 Cleanup of file permissions
Mostly removed execute on non-executable files

Change-Id: Ibfbe7e0cf0fbeabef602d70b20f75e1dd3bdf9c9
2013-02-15 11:06:11 -06:00
Darrell Bishop
bce8443c9e Adds first-byte latency timings for GET requests.
This was an oustanding TODO for StatsD Swift metrics.  A new timing
metric is tracked for (only) GET requests for accounts, containers,
and objects:

  proxy-server.<req_type>.GET.<status_int>.first-byte.timing

Also updated StatsD documentation in the Admin Guide to clarify that
timing metrics are sent in units of milliseconds.

Change-Id: I5bb781c06cefcb5280f4fb1112a526c029fe0c20
2013-02-13 15:58:57 -08:00
Jenkins
23f33b2069 Merge "Make statsd sample rate behave better." 2013-02-13 08:19:46 +00:00
Joe Gordon
45f0502b52 Fix spelling mistakes
git ls-files | misspellings -f -
Source: https://github.com/lyda/misspell-check

Change-Id: I4132e6a276e44e2a8985238358533d315ee8d9c4
2013-02-12 16:39:40 -08:00
Jenkins
af3bd46679 Merge "Add generated keystone documentation to misc.rst." 2013-02-12 12:03:21 +00:00
Jenkins
e4e55588fb Merge "Replaces Copyright statements for LLC with Foundation, removes date." 2013-02-12 06:25:45 +00:00
annegentle
72428434f7 Replaces Copyright statements for LLC with Foundation, removes date.
Replaced GA code for cross-domain tracking.

Patchset addresses reviewer's comments
and follows new guidance from Foundation:
http://wiki.openstack.org/Documentation/Copyright

Adds current year to each Sphinx-built page.

Addresses only the docs copyright attribution, not code files.

Change-Id: Ib90fd1c92c8fafce2db821bc2b17cef1377cfc1e
2013-02-11 16:32:33 -06:00
Chmouel Boudjnah
1b86380e2e Add generated keystone documentation to misc.rst.
- Change a bit the formatting of the documention as well.
- Fix WARNING: Title underline too short. in misc.rst.

Change-Id: I2f4e36bcb5e01e984f0af0152bc5b3b9f7e942ce
2013-02-11 21:44:53 +01:00
David Hadas
83d89228a9 minor development doc changes
Reviewed development saio and guidelines while installing
a new dev and test node.

Change-Id: I59238a4d43fa08a89be14f43681ad4693f1261dd
2013-02-09 11:11:05 +02:00
Jenkins
c0e8ad609b Merge "Allow change the endpoint_type when use swift-dispersion tools" 2013-02-08 23:55:03 +00:00
Michael Barton
24ef12027c Basic container quotas
Add a new middleware implementing some basic container quotas.

Quotas are subject to several limitations: eventual consistency, the timeliness
of the cached container_info (60 second ttl by default), and it’s unable to
reject chunked transfer uploads that exceed the quota (though once the quota
is exceeded, new chunked transfers will be refused).

However, they get most of the way to container quotas fairly inexpensively.

Quotas are set by adding meta values to the container, and are validated when
set:

  X-Container-Meta-Quota-Bytes: Maximum size of the container, in bytes.
  X-Container-Meta-Quota-Count: Maximum object count of the container.

DocImpact

Change-Id: I77cfbf6dc231a2e522bd67328e4c082424a93eee
2013-02-05 06:03:38 -08:00
gholt
85529531d6 Remove tempauth allowed_sync_hosts conf option
Seems we missed these references when committing
357b12dc2ba7b19c66196a573ccb2489d2104b93

DocImpact

Change-Id: Ia226ce1d63e52769bc067d50ec4704cea4e11c5c
2013-01-31 18:30:10 +00:00
Mehdi Abaakouk
a1395ec672 Allow change the endpoint_type when use swift-dispersion tools
Fixes bug 1102319
DocImpact

Change-Id: I8fb0417ab9468e97ed01a6cb1e262630905e7f29
2013-01-31 16:10:37 +01:00
Jenkins
0dfd705b98 Merge "Added fallocate_reserve option" 2013-01-30 23:45:36 +00:00
gholt
87a42ab9ca Added fallocate_reserve option
Some systems behave badly when they completely run out of space. To
alleviate this problem, you can set the fallocate_reserve conf value
to a number of bytes to "reserve" on each disk. When the disk free
space falls at or below this amount, fallocate calls will fail, even
if the underlying OS fallocate call would succeed. For example, a
fallocate_reserve of 5368709120 (5G) would make all fallocate calls
fail, even for zero-byte files, when the disk free space falls under
5G.

The default fallocate_reserve is 0, meaning "no reserve", and so the
software behaves exactly as it always has unless you set this conf
value to something non-zero.

Also fixed ring builder's search_devs doc bugs.

Related: To get rsync to do the same, see
https://github.com/rackspace/cloudfiles-rsync
Specifically, see this patch:
https://github.com/rackspace/cloudfiles-rsync/blob/master/debian/patches/limit-fs-fullness.diff

DocImpact

Change-Id: I8db176ae0ca5b41c9bcfeb7cb8abb31c2e614527
2013-01-29 20:07:26 +00:00
Florian Hines
00dbad0825 Add optional locking to swift-ring-builder
If invoked as 'swift-ring-builder-safe' the directory containing the builder
file provided will be locked (via lock_parent_directory()). This provides a
small safe guard against multiple instances of the swift-ring-builder (or
other utilities that observe this lock) from attempting to write to or read
the builder/ring files while operations are in progress.

This is particularly useful in environments where ring management has been
automated (via Chef or custom solutions) but the operator still occasionally
needs to manually interact with the ring.

DocImpact

Change-Id: Ia362744a8151a91bfb586d01da582906726852e6
2013-01-25 08:00:33 -08:00
David Goetz
2f663ff9a0 Bulk Requests: auto extract archive and bulk delete middleware.
Fix small problem in ratelimiting middleware.

DocImpact

Change-Id: Ide3e0b9f4887626c30cae0b97eb7e2237b1df3ed
2013-01-24 12:34:56 -08:00
Jenkins
77a562dbf1 Merge "Add handler for CORS "actual requests"" 2013-01-23 22:18:11 +00:00
Adrian Smith
89ee10bd92 Add handler for CORS "actual requests"
Fix for bug 1095130

* Added a wrapper function around public methods to handle
  CORS actual requests. These requests need to return some
  extra headers to be valid responses to a CORS request.
  Access-Control-Expose-Headers and Access-Control-Allow-Origin.

* Added support for the CORS header Access-Control-Expose-Headers.

* Some refactoring of the OPTIONS method so the
  "is_origin_allowed" logic can be reused.

* Added a little extra detail to the CORS documentation.

DocImpact

Change-Id: I68538e472a900775427f21a8a59e738a83dcc8bc
2013-01-23 19:31:18 +00:00
John Dickinson
ee98288345 Update auto-generated docs
Removed sidebar with broken (static) links referencing out-of-date docs.

Added an external link to the Swift API docs

fixes bug #1025099

Change-Id: I7f3106175b84b1063f74aa6c5693ab1e422cdb59
2013-01-22 15:24:06 -08:00
Jenkins
3814f9fcdd Merge "Add --run-dir switch to swift-init" 2013-01-21 19:10:13 +00:00
Chmouel Boudjnah
3282e3885c Add --run-dir switch to swift-init
- Add ability to specify an alternative run_dir than the default
  /var/run/swift.
- DocImpact

Change-Id: I17677588f2c8da563b7fec2dc4fdc52da87126ed
2013-01-21 18:12:17 +01:00
Darrell Bishop
8801b74090 Make statsd sample rate behave better.
As Dieter pointed out in bug 1090495
(https://bugs.launchpad.net/swift/+bug/1090495), the volume of metrics
can vary wildly between StatsD metrics.

This patch implements a partial solution by reducing the sample_rate
used for known high-volume metrics (operational experience will need to
inform this over time) and introducing a new tunable,
log_statsd_sample_rate_factor which is multiplied by the sample_rate for
every statsd stat.  This tunable can be used to reduce StatsD traffic
proportionally for all metrics and is intended to replace
log_statsd_default_sample_rate, which is left alone for
backward-compatibility, should anyone be using it.

This patch also includes a drive-by fix for log_udp_port which wasn't
being converted to an int (I didn't verify that actually causes trouble
in SysLogHandler(), but it's definitely an improvement regardles).

Change-Id: Id404636e3629f6431cf1c4e64a143959750a3c23
2013-01-19 15:25:27 -08:00
Jenkins
eff1862b73 Merge "consistency and grammatical tweaks" 2013-01-14 23:02:29 +00:00
Chmouel Boudjnah
0df6fe817d Add container-sync to container-server.conf doc.
- In howto install multi nodes container-sync was not included in the
  config example.

Change-Id: I2682d47c5cae28ff321ba03d70dae2fea00c276f
2013-01-12 23:37:25 +01:00
Cory Wright
c69e130df5 consistency and grammatical tweaks
Change-Id: I0a1dc9e07f246db3e294f5c62f4ffbd3560b0afd
2013-01-09 09:58:36 -08:00
Michael Barton
c37954d935 saio truncate instead of dd (for reals)
Changing that complex dd to a functionally equivalent truncate that's easier
to read and modify.

DocImpact

Change-Id: I64404318364608e62a1d80f6a0550271eb4cd03a
2012-12-20 20:12:50 -08:00
Michael Barton
5d5b262be8 Revert "suggest fallocate instead of dd in saio"
Change-Id: I3742c4580d920e09de434e86b75b134c62f1732e
2012-12-20 13:35:45 -08:00
Michael Barton
e81ca8daf3 suggest fallocate instead of dd in saio
Change-Id: I4d5ba8e58216749c2858dc1c0b8225b3619a7287
2012-12-19 14:10:07 -08:00
gholt
4fcbeff22f Added override for swift-init's KILL_WAIT
You can now give swift-init a -k N (or --kill-wait N) option to
override the default 15 second wait for a process to die after
sending it the die signal. This is useful for boxes that are awfully
slow for whatever reason.

DocImpact

Change-Id: I328ec254f6e0ee1cd423c1d062ba4c5331bd8337
2012-12-17 23:09:07 +00:00
Jenkins
8b770aa55e Merge "Add config option to turn eventlet debug on/off" 2012-12-10 20:37:31 +00:00