Change supports kern.log rotation in order to avoid loss
of significant information.
There is a year change functionality added as kern.log
does not keep record of year.
There is also backwards function added which allows
reading logs from the back to the front, speeding up the
execution along with the unit test for it
Fixes Bug 1080682
Change-Id: I93436c405aff5625396514000cab774b66022dd0
differently (if at all)
Adding a swift.source to wsgi pre_auth funcs and all middleware that makes
subrequests to proxy server.
NOTE: This change will result in a change in the number of proxy logs made for
staticweb, formpost, tempurl, and any other middleware that performs sub
requests (including swauth and SOS).
Please see docs for details.
DocImpact
Change-Id: I80cf2806add1c3d34054147e2515944be340455b
header to a query parameter.
This is needed because query parameters show up in proxy logs and headers do
not. With this change it will be easy to determine from any log line that gets
created from the original request (of which there is currently none) that the
request was a bulk action.
Note: This is not backwards compatible with the previous method of setting a
header. Because the bulk middleware has not been included in an openstack swift
release this should be fine.
Change-Id: I0297fa2de9e491bf0b8c430c0781e2e12316ed4b
Add a new middleware implementing some basic container quotas.
Quotas are subject to several limitations: eventual consistency, the timeliness
of the cached container_info (60 second ttl by default), and it’s unable to
reject chunked transfer uploads that exceed the quota (though once the quota
is exceeded, new chunked transfers will be refused).
However, they get most of the way to container quotas fairly inexpensively.
Quotas are set by adding meta values to the container, and are validated when
set:
X-Container-Meta-Quota-Bytes: Maximum size of the container, in bytes.
X-Container-Meta-Quota-Count: Maximum object count of the container.
DocImpact
Change-Id: I77cfbf6dc231a2e522bd67328e4c082424a93eee
The (account|container|object).ring.gz files contain, among other
things, a JSON-encoded dictionary. This change simply makes the JSON
serializer sort the keys of that dictionary so that two
Python-identical rings will result in two bytewise-identical ring
files. Also, to get repeatable compression, we lock down the timestamp
in the gzip output stream to a fixed value. (There's a timestamp value
in a gzip stream header; by default, gzip.GzipFile sticks time.time()
in there.)
This only works on Python 2.7; on 2.6, the 'mtime' argument to
gzip.GzipFile() is unsupported. Don't worry, serialization still works
on 2.6. It just doesn't always produce the same bytes for the same
ring.
Change-Id: Ide446413d0aeb78536883933fd0caf440b8f54ad
pyflakes itself can't be used in any automated gating way, because there are
two sets of false errors it raises. However, as an exercise, cleaning up the
'valid' ones uncovered three actual bugs. The other changes (mostly unused
variables) are included here for fun.
Command run: pyflakes swift | grep -v "undefined name '_'"
Change-Id: I18696bf047dedad1a9fdbde3463e214fba95f7c6
Passing a seed into rebalance makes the rebalance deterministic
which allows us to generate identical rings across disparate
nodes without having to copy the ring files around.
Change-Id: Ie5ae46ac030e61284bc501fdef9d77eeb5243afd
Some systems behave badly when they completely run out of space. To
alleviate this problem, you can set the fallocate_reserve conf value
to a number of bytes to "reserve" on each disk. When the disk free
space falls at or below this amount, fallocate calls will fail, even
if the underlying OS fallocate call would succeed. For example, a
fallocate_reserve of 5368709120 (5G) would make all fallocate calls
fail, even for zero-byte files, when the disk free space falls under
5G.
The default fallocate_reserve is 0, meaning "no reserve", and so the
software behaves exactly as it always has unless you set this conf
value to something non-zero.
Also fixed ring builder's search_devs doc bugs.
Related: To get rsync to do the same, see
https://github.com/rackspace/cloudfiles-rsync
Specifically, see this patch:
https://github.com/rackspace/cloudfiles-rsync/blob/master/debian/patches/limit-fs-fullness.diff
DocImpact
Change-Id: I8db176ae0ca5b41c9bcfeb7cb8abb31c2e614527
Fix for bug 1095130
* Added a wrapper function around public methods to handle
CORS actual requests. These requests need to return some
extra headers to be valid responses to a CORS request.
Access-Control-Expose-Headers and Access-Control-Allow-Origin.
* Added support for the CORS header Access-Control-Expose-Headers.
* Some refactoring of the OPTIONS method so the
"is_origin_allowed" logic can be reused.
* Added a little extra detail to the CORS documentation.
DocImpact
Change-Id: I68538e472a900775427f21a8a59e738a83dcc8bc
You can't really goof this up using bin/swift-ring-builder, but if you
have code that uses swift.common.ring.RingBuilder directly, you can
stuff e.g. "6002" in where you mean 6002, resulting in some fairly
baffling failures. (Yes, I have done this.)
Change-Id: I87b7b7066b9ea2ce6f82255605da99cf0d283689
If domain_remap and cname_lookup received request which has no host header,
then returns 500 error. This fixes that problem.
Change-Id: Ibb457e9b4cb21181d8243858c04ce255365690da
Fixes: bug #1100632
Tempauth handles a authorization request by request path.
If a request's path start with auth_prefix, tempauth middleware
handles that request to authorization method.
Therefore, when configuring auth_prefix to '/', all requests
handle to authorization method.
This change enables tempauth to prevent invalid auth_prefix
config '/' and similar empty auth_prefix in initialization method.
Fixes bug #1096538.
Change-Id: I20b157e2a0809c17409fc65a8eff0858fe4aff29
When the number of account/container or container/object replicas are
different, Swift had a few misbehaviors. This commit fixes them.
* On an object PUT/POST/DELETE, if there were 3 object replicas and
only 2 container replicas, then only 2 requests would be made to
object servers. Now, 3 requests will be made, but the third won't
have any X-Container-* headers in it.
* On an object PUT/POST/DELETE, if there were 3 object replicas and 4
container replicas, then only 3/4 container servers would receive
immediate updates; the fourth would be ignored. Now one of the
object servers will receive multiple (comma-separated) values in the
X-Container-* headers and it will attempt to contact both of them.
One side effect is that multiple async_pendings may be written for
updates to the same object. They'll have differing timestamps,
though, so all but the newest will be deleted unread. To trigger
this behavior, you have to have more container replicas than object
replicas, 2 or more of the container servers must be down, and the
headers sent to one object server must reference 2 or more down
container servers; it's unlikely enough and the consequences are so
minor that it didn't seem worth fixing.
The situation with account/containers is analogous, only without the
async_pendings.
Change-Id: I98bc2de93fb6b2346d6de1d764213d7563653e8d
This commit makes that AuditorWorker gets a logger from ObjectAuditor class
(instead of creating new one) so the auditor uses minimum unix-sockets.
Fixes: bug #1089215
Change-Id: Ia47d862cbe7505db821784b01fcce6f22196e79f
If the object replicator couldn't create a device's object directory
(due to permissions or whatever) it wouldn't do any work at all. This
fixes that.
Change-Id: I6a30439d036b29c9cfdb660428d13668e0dc8632
Unfortunately, SQLite truncates strings with null characters.
Additionally, XML pretty much hates them too.
Change-Id: Id9a8eaa27b841db6350d6959c202d3e3d6462b35
I've been doing this with cluster-wide log searches for far too long.
This adds support for reporting the oldest replication pass
completion as well as the most recent. This is quite useful for
finding those odd replicators that have hung up for some reason and
need intervention.
Change-Id: I7fd7260eca162d6b085f3e82aaa3cf90670f2d53
If a middleware (swift3, I'm looking at you), doesn't pass a Request
object into the Response constructor, Response._response_iter cannot
know to send zero bytes in the body of the HEAD response.
This patch fixes this usage of swob by making Response.__call__
helpfully reify self.request from env if it wasn't already set by the
Response object's constructor.
This fixes a bug in swift3 + swob-enabled-Swift where HEAD requests to
swift3 resulted in a response with a body in violation of the relevant
RFC and confusing clients.
Thanks to kostecky for finding the bug and describing it accurately.
Change-Id: I2bdb098052b161e1cddf1e4e482ab4dfafeb18c0
This change allows other auth systems to test against the functests but not
require them to support x-storage-token (which was deprecated before swift
was even open-sourced). All other places in the code where x-storage-token is
used is only to populate x-auth-token if x-auth-token doesn't already exist.
Change-Id: Ie903d0f7135fc643353b74a61c14becf7dfc4d9d
