7692 Commits

Author SHA1 Message Date
Thiago da Silva
0887f0985b Update saio sample config files
Added healthcheck middleware to account, container, object servers
Added the s3api, keymaster, encryption config to the proxy config
file to make it easy to enable it.

Change-Id: I96f120c5bc416e9aba388cbfa6c30b648d6ade2f
2018-08-13 15:33:09 -04:00
Zuul
b15111abe7 Merge "Remove some unnecessary SkipTests" 2018-07-27 00:49:11 +00:00
Zuul
cb011fe58b Merge "make probe tests voting in the gate" 2018-07-27 00:42:10 +00:00
Zuul
9065d86e9d Merge "Remove contentdir hack" 2018-07-27 00:17:04 +00:00
Thiago da Silva
84758d948c make probe tests voting in the gate
Change-Id: If8a77bb9de48890858b3de3b9297f6182c1cce05
2018-07-26 06:35:58 +00:00
Zuul
a254d42963 Merge "Lower the required version of ipaddress" 2018-07-26 06:31:56 +00:00
Tim Burke
c3d83bae38 Remove some unnecessary SkipTests
The @requires_acls decorator already checks for cluster_info["tempauth"]

Change-Id: I2c40fbea1fd799059dca6bfb82d5fed833991826
2018-07-25 16:15:38 -07:00
Thiago da Silva
6a039903c6 Remove contentdir hack
Latest version of CentOS should not need this anymore

Change-Id: Ief83c39420cfb4509f16d4e357d704025b116134
2018-07-25 18:08:10 -04:00
Zuul
00373dad61 Merge "Add keymaster to fetch root secret from KMIP service" 2018-07-25 03:49:50 +00:00
Tim Burke
60f1d34048 Lower the required version of ipaddress
CentOS 7 distributes 1.0.16, and there is no real reason to avoid it.
Plus, in our probetests we apparently cannot avoid the system package.

Change-Id: I108ec2438dbc02f3e85f8da43e11ff4cdcfd1e4b
2018-07-24 15:51:01 -07:00
Zuul
d398d60e5a Merge "py3: fix up some NameErrors" 2018-07-24 19:22:43 +00:00
Zuul
89854250c3 Merge "Add fallocate_reserve to account and container servers." 2018-07-20 08:42:51 +00:00
Samuel Merritt
8e651a2d3d Add fallocate_reserve to account and container servers.
The object server can be configured to leave a certain amount of disk
space free; default is 1%. This is useful in avoiding 100%-full
filesystems, as those can get Swift in a state where the filesystem is
too full to write tombstones, so you can't delete objects to free up
space.

When a cluster has accounts/containers and objects on the same disks,
then you can wind up with a 100%-full disk since account and container
servers don't respect fallocate_reserve. This commit makes account and
container servers respect fallocate_reserve so that disks shared
between account/container and object rings won't get 100% full.

When a disk's free space falls below the configured reserve, account
and container PUT, POST, and REPLICATE requests will fail with a 507
status code. These are the operations that can significantly increase
the disk space used by a given database.

I called the parameter "fallocate_reserve" for consistency with the
object server. No actual fallocate() call happens under Swift's
control in the account or container servers (sqlite3 might make such a
call, but it's out of our hands).

Change-Id: I083442eef14bf83c0ea717b1decb3e6b56dbf1d0
2018-07-18 17:27:11 +10:00
Zuul
791009a6ff Merge "PUT+POST: break out putter specific test classes" 2018-07-17 02:38:12 +00:00
Zuul
a21730348e Merge "Check other params preserved when slo_etag is extracted" 2018-07-12 21:35:16 +00:00
Zuul
e47fd206e5 Merge "Add unittest for slo_etag" 2018-07-12 18:48:52 +00:00
Alistair Coles
0cd42a2d33 Check other params preserved when slo_etag is extracted
Change-Id: Ie34ce2a33f2a642b97986fa28cf9db9e6da964d5
Related-Change: I67478923619b00ec1a37d56b6fec6a218453dafc
Related-Change: Ibaa630b5b4251cc4f821c01d3c09a8b8a6be342c
2018-07-12 10:01:58 -05:00
Clay Gerrard
f8b9c24a1c Add unittest for slo_etag
Related-Change-Id: I67478923619b00ec1a37d56b6fec6a218453dafc

Change-Id: Ibaa630b5b4251cc4f821c01d3c09a8b8a6be342c
2018-07-12 10:01:54 -05:00
Zuul
c991076a6e Merge "Include SLO ETag in container updates" 2018-07-12 05:30:56 +00:00
Tim Burke
fa4c30de28 Include s3api schemas in sdists
Change-Id: I5b6b5c283a9e2533a2e753fe81ba429792e590cd
Closes-Bug: 1781297
2018-07-11 16:56:28 -07:00
Tim Burke
c4c98eb64d Include SLO ETag in container updates
Container servers will store an etag like

   <MD5 of manifest on disk>; slo_etag=<MD5 on concatenated ETags>

which the SLO middleware will break out into separate

   "hash": "<MD5 of manifest on disk",
   "slo_etag": "\"<MD5 of concatenated ETags\"",

keys for JSON listings. Text and XML listings are unaffected.

If a middleware left of SLO already specified a container update
override, the slo_etag parameter will be appended. If the base header
value was blank, the MD5 of the manifest will be inserted.

SLOs that were created on previous versions of Swift will continue to
just have the MD5 of the manifest in container listings.

Closes-Bug: 1618573
Change-Id: I67478923619b00ec1a37d56b6fec6a218453dafc
2018-07-10 15:41:29 -07:00
Zuul
1ab691f637 Merge "IP Range restrictions in temp urls" 2018-07-10 04:28:16 +00:00
Nguyen Hai
4409da697f Update doc building instructions in Development Guidelines
Change-Id: Ifd2f468ad745d19ef474ae7503a8bd79d429fc1b
2018-07-06 07:40:52 +00:00
Zuul
ec7d60ebd2 Merge "Add Alpine Linux suuprt to bindep" 2018-07-06 03:10:40 +00:00
Ehud Kaldor
2f4f2dd78c Add Alpine Linux suuprt to bindep
This patch adds Alpine Linux (apk) entries to bindep.txt.

closes-bug: #1780163
Change-Id: Iff24a2b02b8927113d616a5af06331f1a1e31170
2018-07-05 23:55:38 +00:00
Alistair Coles
9a7b46e1e3 swift-ring-builder shows hint about composite builder file
If swift-ring-builder is erroneously given a composite builder
file, which it will fail to load, it will now print a hint
that the file is a composite builder file.

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: If4517f3b61977a7f6ca3e08ed5deb182aa87a366
2018-07-05 15:57:05 +01:00
Zuul
a5569ddd6b Merge "Be consistent about capitalizing Swift in the README" 2018-07-05 09:53:18 +00:00
Zuul
6c2016d830 Merge "Make docs link consistent" 2018-07-05 09:51:03 +00:00
Zuul
44f60d9245 Merge "Address some review comments" 2018-07-05 09:40:47 +00:00
Tim Burke
7bacd53062 Be consistent about capitalizing Swift in the README
Change-Id: If6441d61ee0cb11641c38d46763db6244e05e275
2018-07-04 23:06:10 -07:00
Tim Burke
f27f1ae649 Make docs link consistent
We already link to Swift's docs in particular (as opposed to OpenStack's
in general) later in the README, may as well do it earlier, too.

Change-Id: Ib11bfb551e44ccc6bbbad9d78e1e20d6486a3103
2018-07-04 23:02:42 -07:00
Zuul
cb3692f8db Merge "swob.Match: remove quotes when checking __contains__" 2018-07-04 06:34:17 +00:00
mmcardle
26b20ee729 IP Range restrictions in temp urls
This patch adds an additional optional parameter to tempurl
which restricts the ip's from which a temp url can be used from.

Change-Id: I23fe998a980960d4a32df042b3f6a21f096c36af
2018-07-03 12:25:28 +01:00
Alistair Coles
1951dc7e9a Add keymaster to fetch root secret from KMIP service
Add a new middleware that can be used to fetch an encryption root
secret from a KMIP service. The middleware uses a PyKMIP client
to interact with a KMIP endpoint. The middleware is configured with
a unique identifier for the key to be fetched and options required
for the PyKMIP client.

Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Change-Id: Ib0943fb934b347060fc66c091673a33bcfac0a6d
2018-07-03 09:00:21 +01:00
Pete Zaitcev
91a8cd2952 PUT+POST: break out putter specific test classes
In test_obj.py there are PUT tests that are specific to the Putter
type and others that apply to any Putter type. This patch refactors
them into separate classes to provide greater clarity and to allow
common tests to be applied to each Putter type.

Taking this infrastracture out and ahead of PUT+POST itself allows
it to be reviewed much easier, especially when you look at the diff.

Related-Change-Id: I64b0d8fdb2ffce786f56665a74ed7eb2603abfda
Change-Id: Ibb09b5a28098fb51e25ab5a7134b518cc68eaf89
2018-07-02 23:02:13 -05:00
Zuul
3378a48733 Merge "py3: port proxy/controllers/info.py" 2018-07-02 23:11:05 +00:00
Zuul
9dffe8b88d Merge "functests: Clean up objects better" 2018-07-02 11:04:40 +00:00
Zuul
3a8b43dc39 Merge "Make COPY work with unicode account names" 2018-07-02 10:58:20 +00:00
Zuul
d3a2553615 Merge "Make ACLs work with Unicode in user/account names" 2018-07-02 02:52:35 +00:00
Zuul
e93c6187f6 Merge "swob: Stop auto-encoding unicode bodies" 2018-07-02 02:41:05 +00:00
Zuul
60b530e03a Merge "Describe separate keymaster config file in docs" 2018-07-02 00:43:02 +00:00
Zuul
a4b986fd78 Merge "tempauth: Send back url-encoded account names" 2018-06-30 04:56:11 +00:00
Zuul
9f24b27019 Merge "Content-Length enforcement fixups" 2018-06-29 05:43:39 +00:00
Zuul
554de672c1 Merge "Improve README doc building instructions" 2018-06-29 01:40:26 +00:00
Tim Burke
8bfb97a3a8 Content-Length enforcement fixups
Change-Id: Ice8144700cf447ecf5c6175dc64aa7662013fba5
Related-Change: I74d8c13eba2a4917b5a116875b51a781b33a7abf
2018-06-28 10:31:16 -07:00
Tim Burke
783c7f6117 py3: port proxy/controllers/info.py
Yes, this means you can now start a proxy-server process on py3! And it
will give you something useful on /info!

Apparently you can even get auth tokens from tempauth, although the
tempauth unit tests don't pass yet.

Change-Id: I86ead2989b5934a7584cdd75719ce239826e01ec
2018-06-28 09:58:44 -07:00
Tim Burke
d03fc9bc54 swob: Stop auto-encoding unicode bodies
Instead, require that callers provide an encoding.

Related-Change: I31408f525ba9836f634a35581d4aee6fa2c9428f
Change-Id: I3e5ed9e4401eea76c375bb43ad4afc58b1d8006a
2018-06-28 09:58:44 -07:00
Zuul
bea74a3260 Merge "Update README related to documentation building" 2018-06-28 14:43:20 +00:00
Zuul
729fe6f79d Merge "swob: Fix up some WSGI string business" 2018-06-28 12:23:58 +00:00
Zuul
eb43d30320 Merge "Enforce Content-Length in catch_errors" 2018-06-28 12:23:45 +00:00