openstack/swift
Code Issues Proposed changes

AUTHORS/CHANGELOG for 2.27.0

Browse Source 
Change-Id: I1f4b17eb6e7ccd11cbea6d53f0f4b0112825ede4
This commit is contained in:
Tim Burke 2021-02-25 17:57:16 -08:00
parent 310298a948
commit c6a6403665
3 changed files with 444 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
AUTHORS
View File

@ -36,6 +36,7 @@ Samuel Merritt (sam@swiftstack.com)
Contributors
------------
Aaron Rosen (arosen@nicira.com)
Ade Lee (alee@redhat.com)
Adrian Smith (adrian_f_smith@dell.com)
Adrien Pensart (adrien.pensart@corp.ovh.com)
Akihiro Motoki (amotoki@gmail.com)
@ -57,6 +58,7 @@ Andy McCrae (andy.mccrae@gmail.com)
Anh Tran (anhtt@vn.fujitsu.com)
Ankur Gupta (ankur.gupta@intel.com)
Anne Gentle (anne@openstack.org)
aolivo (aolivo@blizzard.com)
Arnaud JOST (arnaud.jost@ovh.net)
arzhna (arzhna@gmail.com)
Atsushi Sakai (sakaia@jp.fujitsu.com)
@ -183,6 +185,7 @@ Gleb Samsonov (sams-gleb@yandex.ru)
Gonéri Le Bouder (goneri.lebouder@enovance.com)
Graham Hayes (graham.hayes@hpe.com)
Gregory Haynes (greg@greghaynes.net)
Grzegorz Grasza (xek@redhat.com)
Guang Yee (guang.yee@hpe.com)
guotao (guotao.bj@inspur.com)
Gábor Antal (antal@inf.u-szeged.hu)
@ -204,6 +207,7 @@ Hugo Kuo (tonytkdk@gmail.com)
Ilya Kharin (ikharin@mirantis.com)
Ionuț Arțăriși (iartarisi@suse.cz)
Iryoung Jeong (iryoung@gmail.com)
its-not-a-bug-its-a-feature (david.cole@sohonet.com)
Jaivish Kothari (jaivish.kothari@nectechnologies.in)
James E. Blair (jeblair@openstack.org)
James Page (james.page@ubuntu.com)
@ -252,6 +256,7 @@ Leah Klearman (lklrmn@gmail.com)
Li Riqiang (lrqrun@gmail.com)
Liang Jingtao (liang.jingtao@zte.com.cn)
lijunbo (lijunbo@fiberhome.com)
likui (likui@yovole.com)
Lin Yang (lin.a.yang@intel.com)
Lingxian Kong (anlin.kong@gmail.com)
lingyongxu (lyxu@fiberhome.com)
@ -344,6 +349,7 @@ Rainer Toebbicke (Rainer.Toebbicke@cern.ch)
rajat29 (rajat.sharma@nectechnologies.in)
Ray Chen (oldsharp@163.com)
Rebecca Finn (rebeccax.finn@intel.com)
Renich Bon Ćirić (renich@cloudsigma.com)
Ricardo Ferreira (ricardo.sff@gmail.com)
Richard Hawkins (richard.hawkins@rackspace.com)
Robert Francis (robefran@ca.ibm.com)
@ -417,6 +423,7 @@ wanghongtaozz (wanghongtaozz@inspur.com)
wanghui (wang_hui@inspur.com)
wangqi (wang.qi@99cloud.net)
whoami-rajat (rajatdhasmana@gmail.com)
wu.shiming (wushiming@yovole.com)
Wu Wenxiang (wu.wenxiang@99cloud.net)
Wyllys Ingersoll (wyllys.ingersoll@evault.com)
xhancar (pavel.hancar@gmail.com)

203
CHANGELOG
View File

@ -1,3 +1,204 @@
swift (2.27.0, OpenStack Wallaby)
* Added "audit watcher" hooks to allow operators to run arbitrary code
against every diskfile in a cluster. For more information, see
https://docs.openstack.org/swift/latest/development_watchers.html
* Added support for system-scoped "reader" roles when authenticating using
Keystone. Operators may configure this using the `system_reader_roles`
option in the `[filter:keystoneauth]` section of their proxy-server.conf.
A comparable group, `.reseller_reader`, is now available for development
purposes when authenticating using tempauth.
* Allow static large object segments to be deleted asynchronously.
Operators may opt into this new behavior by enabling the new
`allow_async_delete` option in the `[filter:slo]` section
in their proxy-server.conf. For more information, see
https://docs.openstack.org/swift/latest/overview_large_objects.html#deleting-a-large-object
* Added the ability to connect to memcached over TLS. See the
`tls_*` options in etc/memcache.conf-sample
* The proxy-server now caches 'listing' shards, improving listing
performance for sharded containers. A new config option,
`recheck_listing_shard_ranges`, controls the cache time and defaults to
10 minutes; set it to 0 to disable caching (the previous behavior).
* Added a new optional proxy-logging field `{wire_status_int}` for the
status code returned to the client. For more information, see
https://docs.openstack.org/swift/latest/logs.html#proxy-logs
* Errors downloading a Static Large Object that cause a shorter-than-expected
response are now logged as 500s.
* Memcache client error-limiting is now configurable. See the
`error_suppression_*` options in etc/memcache.conf-sample
* Added `tasks_per_second` option to rate-limit the object-expirer.
* Added `usedforsecurity` annotations for use on FIPS-compliant systems.
* Added an option to write EC fragments with legacy CRC to ensure a smooth
upgrade from liberasurecode<=1.5.0 to >=1.6.2. For more information, see
https://bugs.launchpad.net/liberasurecode/+bug/1886088
* **Known Issue**: Operators should verify that encryption is not enabled
in their reconciler pipelines; having it enabled there may harm data
durability. For more information, see https://launchpad.net/bugs/1910804
* S3 API improvements:
* Fixed a bug that prevented the s3api pipeline validation described in
proxy-server.conf-sample from being performed. As documented, operators
can disable this via the `auth_pipeline_check` option if proxy startup
fails with validation errors.
* Make allowable clock skew configurable, with a default value of
15 minutes to match AWS. Note that this was previously hardcoded at
5 minutes; operators may want to preserve the prior behavior by setting
`allowable_clock_skew = 300` in the `[filter:s3api]` section of their
proxy-server.conf.
* Fixed an issue where SHA mismatches in client XML payloads would cause
a server error. Swift now correctly responds with a client error about
the bad digest.
* Fixed an issue where non-base64 signatures would cause a server error.
Swift now correctly responds with a client error about the invalid
digest.
* Container ACLs are now cloned to the `+segments` container when it is
created.
* The correct storage policy is now logged for S3 requests.
* Added the ability to configure auth region in s3token middleware.
* CORS-related headers are now passed through appropriately when using
the S3 API. Note that allowed origins and other container metadata
must still be configured through the Swift API as documented at
https://docs.openstack.org/swift/latest/cors.html
Preflight requests do not contain enough information to map a
bucket to an account/container pair; a new cluster-wide option
`cors_preflight_allow_origin` may be configured for such OPTIONS
requests. The default (blank) rejects all S3 preflight requests.
* Sharding improvements:
* Prevent shard databases from losing track of their root database when
deleted.
* Prevent sharded root databases from being reclaimed to ensure that
shards can detect that they have been deleted.
* A `--no-auto-shard` option has been added to `swift-container-sharder`.
* The sharder daemon has been enhanced to better support the shrinking
of shards that are no longer required. Shard containers will now
discover from their root container if they should be shrinking. They
will also discover the shards into which they should shrink, which may
include the root container itself.
* A 'compact' command has been added to `swift-manage-shard-ranges` that
enables sequences of contiguous shards with low object counts to be
compacted into another existing shard, or into the root container.
* `swift-manage-shard-ranges` can now accept a config file; this
may be used to ensure consistency of threshold values with the
container-sharder config.
* Overlapping shrinking shards no longer generate audit warnings; these
are expected to sometimes overlap.
* The sharding progress reports in recon cache now continue to be included
for a period of time after sharding has completed. The time period
may be configured using the `recon_sharded_timeout` option in the
`[container-sharder]` section of container-server.conf, and defaults
to 12 hours.
* Add root containers with compactible ranges to recon cache.
* Expose sharding statistics in the backend recon middleware.
* Replication improvements:
* Fixed a race condition in ssync that could lead to a loss of data
durability (or even loss of data, for two-replica policies) when some
object servers have outdated rings. Replication via rsync is likely
still affected by a similar bug.
* Non-durable fragments can now be reverted from handoffs.
* The post-rsync REPLICATE call no longer recalculates hashes immediately.
* Hashes are no longer invalidated after a successful ssync; they were
already invalidated during the data transfer.
* Reduced log noise for common ssync errors.
* Python 3 fixes:
* Added support for Python 3.9.
* Staticweb correctly handles listings when paths include non-ASCII
characters.
* S3 API now allows multipart uploads with non-ASCII characters in the
object name.
* Fixed an import-ordering issue in `swift-dispersion-populate`.
* Partition power increase improvements:
* Fixed a bug where stale state files would cause misplaced data during
multiple partition power increases.
* Removed a race condition that could cause newly-written data to not be
linked into the new partition for the new partition power.
* Improved safety during cleanup to ensure files have been relinked
appropriately before unlinking.
* Added an option to drop privileges when running the relinker as root.
* Added an option to rate-limit how quickly data files are relinked or
cleaned up. This may be used to reduce I/O load during partition power
increases, improving end-user performance.
* Rehash partitions during the partition power increase. Previously, we
relied on the replication engine to perform the rehash, which could
cause an unexpected I/O spike after a partition power increase.
* Warn when relinking/cleaning up and any disks are unmounted.
* Log progress per partition when relinking/cleaning up.
* During clean-up, stop warning about tombstones that got reaped from
the new location but not the old.
* Added the ability to read options from object-server.conf, similar to
background daemons.
* Turned off thread-logging when monkey-patching with eventlet. This
addresses a potential hang in the proxy-server while logging client
disconnects.
* Fixed a bug that could cause EC GET responses to return a server error.
* Fixed an issue with `swift-drive-audit` when run around New Year's.
* Server errors encountered when validating the first segment of a Static or
Dynamic Large Object now return a 503 to the client, rather than a 409.
* Errors when setting keys in memcached are now logged. This helps
operators detect when shard ranges for caching have gotten too large to
be stored, for example.
* Various other minor bug fixes and improvements.
swift (2.26.0, OpenStack Victoria)
* Extend concurrent reads to erasure coded policies. Previously, the
@ -1142,7 +1343,7 @@ swift (2.15.0)
* Add support to increase object ring partition power transparently
to end users and with no cluster downtime. Increasing the ring
part power allows for incremental adjustment to the upper bound
partition power allows for incremental adjustment to the upper bound
of the cluster size. Please review the full docs at
<https://docs.openstack.org/swift/latest/ring_partpower.html>.

235
releasenotes/notes/2_27_0_release-a9ae967d6d271342.yaml Normal file
View File

@ -0,0 +1,235 @@
---
features:
- |
Added "audit watcher" hooks to allow operators to run arbitrary code
against every diskfile in a cluster. For more information, see `the documentation
<https://docs.openstack.org/swift/latest/development_watchers.html>`__.
- |
Added support for system-scoped "reader" roles when authenticating using
Keystone. Operators may configure this using the ``system_reader_roles``
option in the ``[filter:keystoneauth]`` section of their proxy-server.conf.
A comparable group, ``.reseller_reader``, is now available for development
purposes when authenticating using tempauth.
- |
Allow static large object segments to be deleted asynchronously.
Operators may opt into this new behavior by enabling the new
``allow_async_delete`` option in the ``[filter:slo]`` section
in their proxy-server.conf. For more information, see `the documentation
<https://docs.openstack.org/swift/latest/overview_large_objects.html#deleting-a-large-object>`__.
- |
Added the ability to connect to memcached over TLS. See the
``tls_*`` options in etc/memcache.conf-sample
- |
The proxy-server now caches 'listing' shards, improving listing
performance for sharded containers. A new config option,
``recheck_listing_shard_ranges``, controls the cache time and defaults to
10 minutes; set it to 0 to disable caching (the previous behavior).
- |
Added a new optional proxy-logging field ``{wire_status_int}`` for the
status code returned to the client. For more information, see `the documentation
<https://docs.openstack.org/swift/latest/logs.html#proxy-logs>`__.
- |
Memcache client error-limiting is now configurable. See the
``error_suppression_*`` options in etc/memcache.conf-sample
- |
Added ``tasks_per_second`` option to rate-limit the object-expirer.
- |
Added ``usedforsecurity`` annotations for use on FIPS-compliant systems.
- |
S3 API improvements:
* Make allowable clock skew configurable, with a default value of
15 minutes to match AWS. Note that this was previously hardcoded at
5 minutes; operators may want to preserve the prior behavior by setting
``allowable_clock_skew = 300`` in the ``[filter:s3api]`` section of their
proxy-server.conf.
* Container ACLs are now cloned to the ``+segments`` container when it is
created.
* Added the ability to configure auth region in s3token middleware.
* CORS-related headers are now passed through appropriately when using
the S3 API. Note that allowed origins and other container metadata
must still be `configured through the Swift API
<https://docs.openstack.org/swift/latest/cors.html>`__.
Preflight requests do not contain enough information to map a
bucket to an account/container pair; a new cluster-wide option
``cors_preflight_allow_origin`` may be configured for such OPTIONS
requests. The default (blank) rejects all S3 preflight requests.
- |
Sharding improvements:
* A ``--no-auto-shard`` option has been added to ``swift-container-sharder``.
* The sharder daemon has been enhanced to better support the shrinking
of shards that are no longer required. Shard containers will now
discover from their root container if they should be shrinking. They
will also discover the shards into which they should shrink, which may
include the root container itself.
* A 'compact' command has been added to ``swift-manage-shard-ranges`` that
enables sequences of contiguous shards with low object counts to be
compacted into another existing shard, or into the root container.
* ``swift-manage-shard-ranges`` can now accept a config file; this
may be used to ensure consistency of threshold values with the
container-sharder config.
* The sharding progress reports in recon cache now continue to be included
for a period of time after sharding has completed. The time period
may be configured using the ``recon_sharded_timeout`` option in the
``[container-sharder]`` section of container-server.conf, and defaults
to 12 hours.
* Add root containers with compactible ranges to recon cache.
* Expose sharding statistics in the backend recon middleware.
- |
Replication improvements:
* The post-rsync REPLICATE call no longer recalculates hashes immediately.
* Hashes are no longer invalidated after a successful ssync; they were
already invalidated during the data transfer.
- |
Added support for Python 3.9.
- |
Partition power increase improvements:
* Fixed a bug where stale state files would cause misplaced data during
multiple partition power increases.
* Removed a race condition that could cause newly-written data to not be
linked into the new partition for the new partition power.
* Improved safety during cleanup to ensure files have been relinked
appropriately before unlinking.
* Added an option to drop privileges when running the relinker as root.
* Added an option to rate-limit how quickly data files are relinked or
cleaned up. This may be used to reduce I/O load during partition power
increases, improving end-user performance.
* Rehash partitions during the partition power increase. Previously, we
relied on the replication engine to perform the rehash, which could
cause an unexpected I/O spike after a partition power increase.
* Warn when relinking/cleaning up and any disks are unmounted.
* Log progress per partition when relinking/cleaning up.
* During clean-up, stop warning about tombstones that got reaped from
the new location but not the old.
* Added the ability to read options from object-server.conf, similar to
background daemons.
issues:
- |
Operators should verify that encryption is not enabled in their reconciler
pipelines; having it enabled there may harm data durability. For more
information, see `bug 1910804 <https://launchpad.net/bugs/1910804>`__.
upgrade:
- |
Added an option to write EC fragments with legacy CRC to ensure a smooth
upgrade from liberasurecode<=1.5.0 to >=1.6.2. For more information, see
`bug 1886088 <https://bugs.launchpad.net/liberasurecode/+bug/1886088>`__.
fixes:
- |
Errors downloading a Static Large Object that cause a shorter-than-expected
response are now logged as 500s.
- |
S3 API fixes:
* Fixed a bug that prevented the s3api pipeline validation described in
proxy-server.conf-sample from being performed. As documented, operators
can disable this via the ``auth_pipeline_check`` option if proxy startup
fails with validation errors.
* Fixed an issue where SHA mismatches in client XML payloads would cause
a server error. Swift now correctly responds with a client error about
the bad digest.
* Fixed an issue where non-base64 signatures would cause a server error.
Swift now correctly responds with a client error about the invalid
digest.
* The correct storage policy is now logged for S3 requests.
- |
Sharding fixes:
* Prevent shard databases from losing track of their root database when
deleted.
* Prevent sharded root databases from being reclaimed to ensure that
shards can detect that they have been deleted.
* Overlapping shrinking shards no longer generate audit warnings; these
are expected to sometimes overlap.
- |
Replication fixes:
* Fixed a race condition in ssync that could lead to a loss of data
durability (or even loss of data, for two-replica policies) when some
object servers have outdated rings. Replication via rsync is likely
still affected by a similar bug.
* Non-durable fragments can now be reverted from handoffs.
* Reduced log noise for common ssync errors.
- |
Python 3 fixes:
* Staticweb correctly handles listings when paths include non-ASCII
characters.
* S3 API now allows multipart uploads with non-ASCII characters in the
object name.
* Fixed an import-ordering issue in ``swift-dispersion-populate``.
- |
Turned off thread-logging when monkey-patching with eventlet. This
addresses a potential hang in the proxy-server while logging client
disconnects.
- |
Fixed a bug that could cause EC GET responses to return a server error.
- |
Fixed an issue with ``swift-drive-audit`` when run around New Year's.
- |
Server errors encountered when validating the first segment of a Static or
Dynamic Large Object now return a 503 to the client, rather than a 409.
- |
Errors when setting keys in memcached are now logged. This helps
operators detect when shard ranges for caching have gotten too large to
be stored, for example.
- |
Various other minor bug fixes and improvements.