openstack/swift
Code Issues Proposed changes
ebcc95a96b
swift/setup.cfg

153 lines
5.1 KiB
INI
Raw Normal View History

Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
[metadata]
name = swift
summary = OpenStack Object Storage
description-file =
Convert README.md to README.rst Change-Id: I223890bd4ffe469becc2127f9362243cdb52bc08 Closes-Bug: #1567026
2016-04-08 18:17:44 -05:00
README.rst
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
author = OpenStack
author-email = openstack-dev@lists.openstack.org
Update URL home-page in documents according to document migration Change-Id: I9590d129e7d5e72d955983f3ce38b4e2d44c95b1
2017-07-14 16:35:41 +07:00
home-page = https://docs.openstack.org/swift/latest/
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
classifier =
Development Status :: 5 - Production/Stable
Environment :: OpenStack
Intended Audience :: Information Technology
Intended Audience :: System Administrators
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
[pbr]
skip_authors = True
skip_changelog = True
[files]
packages =
swift
scripts =
bin/swift-account-audit
bin/swift-account-auditor
Added swift-account-info tool. This is a very simple swift tool to retrieve information of an account that is located on the storage node. One can call the tool with a given account db file as it is stored on the storage node system. It will then return several information about that account. Change-Id: Ibfeee790adc000fc177b4b3c03d22ff785fda325
2014-03-30 21:52:53 +05:30
bin/swift-account-info
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
bin/swift-account-reaper
bin/swift-account-replicator
bin/swift-account-server
bin/swift-config
bin/swift-container-auditor
Added swift-container-info tool. This is a very simple swift tool to retrieve information of a container that is located on the storage node. One can call the tool with a given container db file as it is stored on the storage node system. It will then return several information about that container. Change-Id: Ifebaed6c51a9ed5fbc0e7572bb43ef05d7dd254b
2014-03-21 07:47:21 +05:30
bin/swift-container-info
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
bin/swift-container-replicator
bin/swift-container-server
Add sharder daemon, manage_shard_ranges tool and probe tests The sharder daemon visits container dbs and when necessary executes the sharding workflow on the db. The workflow is, in overview: - perform an audit of the container for sharding purposes. - move any misplaced objects that do not belong in the container to their correct shard. - move shard ranges from FOUND state to CREATED state by creating shard containers. - move shard ranges from CREATED to CLEAVED state by cleaving objects to shard dbs and replicating those dbs. By default this is done in batches of 2 shard ranges per visit. Additionally, when the auto_shard option is True (NOT yet recommeneded in production), the sharder will identify shard ranges for containers that have exceeded the threshold for sharding, and will also manage the sharding and shrinking of shard containers. The manage_shard_ranges tool provides a means to manually identify shard ranges and merge them to a container in order to trigger sharding. This is currently the recommended way to shard a container. Co-Authored-By: Alistair Coles <alistairncoles@gmail.com> Co-Authored-By: Tim Burke <tim.burke@gmail.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Change-Id: I7f192209d4d5580f5a0aa6838f9f04e436cf6b1f
2018-05-02 10:47:51 +01:00
bin/swift-container-sharder
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
bin/swift-container-sync
bin/swift-container-updater
Add container-reconciler daemon This daemon will take objects that are in the wrong storage policy and move them to the right ones, or delete requests that went to the wrong storage policy and apply them to the right ones. It operates on a queue similar to the object-expirer's queue. Discovering that the object is in the wrong policy will be done in subsequent commits by the container replicator; this is the daemon that handles them once they happen. Like the object expirer, you only need to run one of these per cluster see etc/container-reconciler.conf. DocImpact Implements: blueprint storage-policies Change-Id: I5ea62eb77ddcbc7cfebf903429f2ee4c098771c9
2014-04-28 20:03:48 -07:00
bin/swift-container-reconciler
bin/swift-reconciler-enqueue
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
bin/swift-dispersion-populate
Add bin/swift-dispersion-report Change-Id: I81736080fc478c2b69d5b71edd0cada39aad9400
2017-06-23 17:39:08 +02:00
bin/swift-dispersion-report
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
bin/swift-drive-audit
bin/swift-form-signature
bin/swift-get-nodes
bin/swift-init
bin/swift-object-auditor
bin/swift-object-expirer
bin/swift-object-info
bin/swift-object-replicator
Erasure Code Reconstructor This patch adds the erasure code reconstructor. It follows the design of the replicator but: - There is no notion of update() or update_deleted(). - There is a single job processor - Jobs are processed partition by partition. - At the end of processing a rebalanced or handoff partition, the reconstructor will remove successfully reverted objects if any. And various ssync changes such as the addition of reconstruct_fa() function called from ssync_sender which performs the actual reconstruction while sending the object to the receiver Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: John Dickinson <me@not.mn> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Tushar Gohad <tushar.gohad@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> blueprint ec-reconstructor Change-Id: I7d15620dc66ee646b223bb9fff700796cd6bef51
2014-10-28 09:51:06 -07:00
bin/swift-object-reconstructor
Add support to increase object ring partition power This patch adds methods to increase the partition power of an existing object ring without downtime for the users using a 3-step process. Data won't be moved to other nodes; objects using the new increased partition power will be located on the same device and are hardlinked to avoid data movement. 1. A new setting "next_part_power" will be added to the rings, and once the proxy server reloaded the rings it will send this value to the object servers on any write operation. Object servers will now create a hard-link in the new location to the original DiskFile object. Already existing data will be relinked using a new tool in the new locations using hardlinks. 2. The actual partition power itself will be increased. Servers will now use the new partition power to read from and write to. No longer required hard links in the old object location have to be removed now by the relinker tool; the relinker tool reads the next_part_power setting to find object locations that need to be cleaned up. 3. The "next_part_power" flag will be removed. This mostly implements the spec in [1]; however it's not using an "epoch" as described there. The idea of the epoch was to store data using different partition powers in their own namespace to avoid conflicts with auditors and replicators as well as being able to abort such an operation and just remove the new tree. This would require some heavy change of the on-disk data layout, and other object-server implementations would be required to adopt this scheme too. Instead the object-replicator is now aware that there is a partition power increase in progress and will skip replication of data in that storage policy; the relinker tool should be simply run and afterwards the partition power will be increased. This shouldn't take that much time (it's only walking the filesystem and hardlinking); impact should be low therefore. The relinker should be run on all storage nodes at the same time in parallel to decrease the required time (though this is not mandatory). Failures during relinking should not affect cluster operations - relinking can be even aborted manually and restarted later. Auditors are not quarantining objects written to a path with a different partition power and therefore working as before (though they are reading each object twice in the worst case before the no longer needed hard links are removed). Co-Authored-By: Alistair Coles <alistair.coles@hpe.com> Co-Authored-By: Matthew Oliver <matt@oliver.net.au> Co-Authored-By: Tim Burke <tim.burke@gmail.com> [1] https://specs.openstack.org/openstack/swift-specs/specs/in_progress/ increasing_partition_power.html Change-Id: I7d6371a04f5c1c4adbb8733a71f3c177ee5448bb
2016-07-04 18:21:54 +02:00
bin/swift-object-relinker
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
bin/swift-object-server
bin/swift-object-updater
bin/swift-oldies
bin/swift-orphans
bin/swift-proxy-server
Don't create bin/* files magically Just use import to make scripts available in bin/ instead of creating these during setup.py install. Change-Id: I7318bbb77f6564ed58736887e711e1c497873471
2014-02-12 20:26:29 +00:00
bin/swift-recon
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
bin/swift-recon-cron
Don't create bin/* files magically Just use import to make scripts available in bin/ instead of creating these during setup.py install. Change-Id: I7318bbb77f6564ed58736887e711e1c497873471
2014-02-12 20:26:29 +00:00
bin/swift-ring-builder
Add ring-builder analyzer. This is a tool to help developers quantify changes to the ring builder. It takes a scenario (JSON file) describing the builder's basic parameters (part_power, replicas, etc.) and a number of "rounds", where each round is a set of operations to perform on the builder. For each round, the operations are applied, and then the builder is rebalanced until it reaches a steady state. The idea is that a developer observes the ring builder behaving suboptimally, writes a scenario to reproduce the behavior, modifies the ring builder to fix it, and references the scenario with the commit so that others can see that things have improved. I decided to write this after writing my fourth or fifth hacky one-off script to reproduce some bad behavior in the ring builder. Change-Id: I114242748368f142304aab90a6d99c1337bced4c
2015-06-11 15:40:28 -07:00
bin/swift-ring-builder-analyzer
Experimental swift-ring-composer CLI to build composite rings Provides a simple, experimental, CLI tool to generate a composite ring from a list of component builder files. For example: swift-ring-composer <composite-file> compose \ <builder-file> <builder-file> --output <ring-file> Commands available: - compose: compose a list of builder file to a composite ring - show: show the metadata for a composite ring Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp> Co-Authored-By: Matthew Oliver <matt@oliver.net.au> Change-Id: I25a79e71c13af352e19e4358f60545265b51584f
2017-06-19 14:46:14 +01:00
bin/swift-ring-composer
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
Retrieve encryption root secret from Barbican This patch adds support for retrieving the encryption root secret from an external key management system. In practice, this is currently limited to Barbican. Change-Id: I1700e997f4ae6fa1a7e68be6b97539a24046e80b
2016-09-02 12:19:49 +00:00
[extras]
kms_keymaster =
oslo.config>=4.0.0,!=4.3.0,!=4.4.0 # Apache-2.0
fix barbican integration Added auth_url to the context we pass to castellan library. In a change [1] intended to deprecate the use of auth_endpoint passed as the oslo config, it actually completely removed the use of it[2], so this change became necessary or the integration is broken. [1] - https://review.openstack.org/#/c/483457 [2] - https://review.openstack.org/#/c/483457/6/castellan/key_manager/barbican_key_manager.py@143 Change-Id: I933367fa46aa0a3dc9aedf078b1be715bfa8c054
2018-01-11 07:15:48 -05:00
castellan>=0.13.0 # Apache-2.0
Retrieve encryption root secret from Barbican This patch adds support for retrieving the encryption root secret from an external key management system. In practice, this is currently limited to Barbican. Change-Id: I1700e997f4ae6fa1a7e68be6b97539a24046e80b
2016-09-02 12:19:49 +00:00
Add keymaster to fetch root secret from KMIP service Add a new middleware that can be used to fetch an encryption root secret from a KMIP service. The middleware uses a PyKMIP client to interact with a KMIP endpoint. The middleware is configured with a unique identifier for the key to be fetched and options required for the PyKMIP client. Co-Authored-By: Tim Burke <tim.burke@gmail.com> Change-Id: Ib0943fb934b347060fc66c091673a33bcfac0a6d
2018-06-26 12:48:58 +01:00
kmip_keymaster =
pykmip>=0.7.0 # Apache-2.0
add keystonemiddleware to extras area Change-Id: I805cf220aee4af69bf4984cc148d69520e958073
2018-02-13 14:38:22 -08:00
keystone =
keystonemiddleware>=4.17.0
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
[entry_points]
Add sharder daemon, manage_shard_ranges tool and probe tests The sharder daemon visits container dbs and when necessary executes the sharding workflow on the db. The workflow is, in overview: - perform an audit of the container for sharding purposes. - move any misplaced objects that do not belong in the container to their correct shard. - move shard ranges from FOUND state to CREATED state by creating shard containers. - move shard ranges from CREATED to CLEAVED state by cleaving objects to shard dbs and replicating those dbs. By default this is done in batches of 2 shard ranges per visit. Additionally, when the auto_shard option is True (NOT yet recommeneded in production), the sharder will identify shard ranges for containers that have exceeded the threshold for sharding, and will also manage the sharding and shrinking of shard containers. The manage_shard_ranges tool provides a means to manually identify shard ranges and merge them to a container in order to trigger sharding. This is currently the recommended way to shard a container. Co-Authored-By: Alistair Coles <alistairncoles@gmail.com> Co-Authored-By: Tim Burke <tim.burke@gmail.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Change-Id: I7f192209d4d5580f5a0aa6838f9f04e436cf6b1f
2018-05-02 10:47:51 +01:00
console_scripts =
swift-manage-shard-ranges = swift.cli.manage_shard_ranges:main
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
paste.app_factory =
proxy = swift.proxy.server:app_factory
object = swift.obj.server:app_factory
Allow access to the in-memory object server This will allow functional tests to be used against it, and can then be used for the storage-policy work as an example diskfile implementation associated with a storage policy. Change-Id: I47a88e70cee99225779baaed379b0c5d4c73611a
2013-12-04 16:05:56 -05:00
mem_object = swift.obj.mem_server:app_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
container = swift.container.server:app_factory
account = swift.account.server:app_factory
paste.filter_factory =
healthcheck = swift.common.middleware.healthcheck:filter_factory
crossdomain = swift.common.middleware.crossdomain:filter_factory
memcache = swift.common.middleware.memcache:filter_factory
Adds read_only middleware This patch adds a read_only middleware to swift. It gives the ability to make an entire cluster or individual accounts read only. When a cluster or an account is in read only mode, requests that would result in writes to the cluser are not allowed. DocImpact Change-Id: I7e0743aecd60b171bbcefcc8b6e1f3fd4cef2478
2015-10-12 15:59:26 +00:00
read_only = swift.common.middleware.read_only:filter_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
ratelimit = swift.common.middleware.ratelimit:filter_factory
cname_lookup = swift.common.middleware.cname_lookup:filter_factory
catch_errors = swift.common.middleware.catch_errors:filter_factory
domain_remap = swift.common.middleware.domain_remap:filter_factory
staticweb = swift.common.middleware.staticweb:filter_factory
tempauth = swift.common.middleware.tempauth:filter_factory
keystoneauth = swift.common.middleware.keystoneauth:filter_factory
recon = swift.common.middleware.recon:filter_factory
tempurl = swift.common.middleware.tempurl:filter_factory
formpost = swift.common.middleware.formpost:filter_factory
name_check = swift.common.middleware.name_check:filter_factory
bulk = swift.common.middleware.bulk:filter_factory
container_quotas = swift.common.middleware.container_quotas:filter_factory
account_quotas = swift.common.middleware.account_quotas:filter_factory
proxy_logging = swift.common.middleware.proxy_logging:filter_factory
Move all DLO functionality to middleware This is for the same reason that SLO got pulled into middleware, which includes stuff like automatic retry of GETs on broken connection and the multi-ring storage policy stuff. The proxy will automatically insert the dlo middleware at an appropriate place in the pipeline the same way it does with the gatekeeper middleware. Clusters will still support DLOs after upgrade even with an old config file that doesn't mention dlo at all. Includes support for reading config values from the proxy server's config section so that upgraded clusters continue to work as before. Bonus fix: resolve 'after' vs. 'after_fn' in proxy's required filters list. Having two was confusing, so I kept the more-general one. DocImpact blueprint multi-ring-large-objects Change-Id: Ib3b3830c246816dd549fc74be98b4bc651e7bace
2013-11-21 17:31:16 -08:00
dlo = swift.common.middleware.dlo:filter_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
slo = swift.common.middleware.slo:filter_factory
list_endpoints = swift.common.middleware.list_endpoints:filter_factory
Generic means for persisting system metadata. Middleware or core features may need to store metadata against accounts or containers. This patch adds a generic mechanism for system metadata to be persisted in backend databases, without polluting the user metadata namespace, by using the reserved header namespace x-<server_type>-sysmeta-*. Modifications are firstly that backend servers persist system metadata headers alongside user metadata and other system state. For accounts and containers, system metadata in PUT and POST requests is treated in a similar way to user metadata. System metadata is not yet supported for object requests. Secondly, changes in the proxy controllers ensure that headers in the system metadata namespace will pass through in requests to backend servers. Thirdly, system metadata returned from backend servers in GET or HEAD responses is added to the cached info dict, which middleware can access. Finally, a gatekeeper middleware module is provided which filters all system metadata headers from requests and responses by removing headers with names starting x-account-sysmeta-, x-container-sysmeta-. The gatekeeper also removes headers starting x-object-sysmeta- in anticipation of future support for system metadata being set for objects. This prevents clients from writing or reading system metadata. The required_filters list in swift/proxy/server.py is modified to include the gatekeeper middleware so that if the gatekeeper has not been configured in the pipeline then it will be automatically inserted close to the start of the pipeline. blueprint cluster-federation Change-Id: I80b8b14243cc59505f8c584920f8f527646b5f45
2013-12-03 22:02:39 +00:00
gatekeeper = swift.common.middleware.gatekeeper:filter_factory
New container sync configuration option Summary of the new configuration option: The cluster operators add the container_sync middleware to their proxy pipeline and create a container-sync-realms.conf for their cluster and copy this out to all their proxy and container servers. This file specifies the available container sync "realms". A container sync realm is a group of clusters with a shared key that have agreed to provide container syncing to one another. The end user can then set the X-Container-Sync-To value on a container to //realm/cluster/account/container instead of the previously required URL. The allowed hosts list is not used with this configuration and instead every container sync request sent is signed using the realm key and user key. This offers better security as source hosts can be faked much more easily than faking per request signatures. Replaying signed requests, assuming it could easily be done, shouldn't be an issue as the X-Timestamp is part of the signature and so would just short-circuit as already current or as superceded. This also makes configuration easier for the end user, especially with difficult networking situations where a different host might need to be used for the container sync daemon since it's connecting from within a cluster. With this new configuration option, the end user just specifies the realm and cluster names and that is resolved to the proper endpoint configured by the operator. If the operator changes their configuration (key or endpoint), the end user does not need to change theirs. DocImpact Change-Id: Ie1704990b66d0434e4991e26ed1da8b08cb05a37
2013-12-08 09:13:59 +00:00
container_sync = swift.common.middleware.container_sync:filter_factory
Add profiling middleware in Swift The profile middleware provide a tool to profile Swift code on the fly and collect statistic data for performance analysis. An native simple Web UI is also provided to help query and visualize the data. Change-Id: I6a1554b2f8dc22e9c8cd20cff6743513eb9acc05 Implements: blueprint profiling-middleware
2013-10-24 03:40:06 +08:00
xprofile = swift.common.middleware.xprofile:filter_factory
versioned writes middleware Rewrite object versioning as middleware to simplify the PUT method in the object controller. The functionality remains basically the same with the only major difference being the ability to now version slo manifest files. dlo manifests are still not supported as part of this patch. Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> DocImpact Change-Id: Ie899290b3312e201979eafefb253d1a60b65b837 Signed-off-by: Thiago da Silva <thiago@redhat.com> Signed-off-by: Prashanth Pai <ppai@redhat.com>
2014-11-09 13:13:27 -05:00
versioned_writes = swift.common.middleware.versioned_writes:filter_factory
Refactor server side copy as middleware Rewrite server side copy and 'object post as copy' feature as middleware to simplify the PUT method in the object controller code. COPY is no longer a verb implemented as public method in Proxy application. The server side copy middleware is inserted to the left of dlo, slo and versioned_writes middlewares in the proxy server pipeline. As a result, dlo and slo copy_hooks are no longer required. SLO manifests are now validated when copied so when copying a manifest to another account the referenced segments must be readable in that account for the manifest copy to succeed (previously this validation was not made, meaning the manifest was copied but could be unusable if the segments were not readable). With this change, there should be no change in functionality or existing behavior. This is asserted with (almost) no changes required to existing functional tests. Some notes (for operators): * Middleware required to be auto-inserted before slo and dlo and versioned_writes * Turning off server side copy is not configurable. * object_post_as_copy is no longer a configurable option of proxy server but of this middleware. However, for smooth upgrade, config option set in proxy server app is also read. DocImpact: Introducing server side copy as middleware Co-Authored-By: Alistair Coles <alistair.coles@hpe.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Change-Id: Ic96a92e938589a2f6add35a40741fd062f1c29eb Signed-off-by: Prashanth Pai <ppai@redhat.com> Signed-off-by: Thiago da Silva <thiago@redhat.com>
2015-02-18 11:59:31 +05:30
copy = swift.common.middleware.copy:filter_factory
Enable object body and metadata encryption Adds encryption middlewares. All object servers and proxy servers should be upgraded before introducing encryption middleware. Encryption middleware should be first introduced with the encryption middleware disable_encryption option set to True. Once all proxies have encryption middleware installed this option may be set to False (the default). Increases constraints.py:MAX_HEADER_COUNT by 4 to allow for headers generated by encryption-related middleware. Co-Authored-By: Tim Burke <tim.burke@gmail.com> Co-Authored-By: Christian Cachin <cca@zurich.ibm.com> Co-Authored-By: Mahati Chamarthy <mahati.chamarthy@gmail.com> Co-Authored-By: Peter Chng <pchng@ca.ibm.com> Co-Authored-By: Alistair Coles <alistair.coles@hpe.com> Co-Authored-By: Jonathan Hinson <jlhinson@us.ibm.com> Co-Authored-By: Hamdi Roumani <roumani@ca.ibm.com> UpgradeImpact Change-Id: Ie6db22697ceb1021baaa6bddcf8e41ae3acb5376
2016-06-07 15:01:32 +01:00
keymaster = swift.common.middleware.crypto.keymaster:filter_factory
encryption = swift.common.middleware.crypto:filter_factory
Retrieve encryption root secret from Barbican This patch adds support for retrieving the encryption root secret from an external key management system. In practice, this is currently limited to Barbican. Change-Id: I1700e997f4ae6fa1a7e68be6b97539a24046e80b
2016-09-02 12:19:49 +00:00
kms_keymaster = swift.common.middleware.crypto.kms_keymaster:filter_factory
Add keymaster to fetch root secret from KMIP service Add a new middleware that can be used to fetch an encryption root secret from a KMIP service. The middleware uses a PyKMIP client to interact with a KMIP endpoint. The middleware is configured with a unique identifier for the key to be fetched and options required for the PyKMIP client. Co-Authored-By: Tim Burke <tim.burke@gmail.com> Change-Id: Ib0943fb934b347060fc66c091673a33bcfac0a6d
2018-06-26 12:48:58 +01:00
kmip_keymaster = swift.common.middleware.crypto.kmip_keymaster:filter_factory
Move listing formatting out to proxy middleware Make some json -> (text, xml) stuff in a common module, reference that in account/container servers so we don't break existing clients (including out-of-date proxies), but have the proxy controllers always force a json listing. This simplifies operations on listings (such as the ones already happening in decrypter, or the ones planned for symlink and sharding) by only needing to consider a single response type. There is a downside of larger backend requests for text/plain listings, but it seems like a net win? Change-Id: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d
2017-03-23 18:26:21 -07:00
listing_formats = swift.common.middleware.listing_formats:filter_factory
Symlink implementation. Add a symbolic link ("symlink") object support to Swift. This object will reference another object. GET and HEAD requests for a symlink object will operate on the referenced object. DELETE and PUT requests for a symlink object will operate on the symlink object, not the referenced object, and will delete or overwrite it, respectively. POST requests are *not* forwarded to the referenced object and should be sent directly. POST requests sent to a symlink object will result in a 307 Error. Historical information on symlink design can be found here: https://github.com/openstack/swift-specs/blob/master/specs/in_progress/symlinks.rst. https://etherpad.openstack.org/p/swift_symlinks Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Janie Richling <jrichli@us.ibm.com> Co-Authored-By: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp> Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp> Change-Id: I838ed71bacb3e33916db8dd42c7880d5bb9f8e18 Signed-off-by: Thiago da Silva <thiago@redhat.com>
2015-10-07 15:14:58 -04:00
symlink = swift.common.middleware.symlink:filter_factory
Import swift3 into swift repo as s3api middleware This attempts to import openstack/swift3 package into swift upstream repository, namespace. This is almost simple porting except following items. 1. Rename swift3 namespace to swift.common.middleware.s3api 1.1 Rename also some conflicted class names (e.g. Request/Response) 2. Port unittests to test/unit/s3api dir to be able to run on the gate. 3. Port functests to test/functional/s3api and setup in-process testing 4. Port docs to doc dir, then address the namespace change. 5. Use get_logger() instead of global logger instance 6. Avoid global conf instance Ex. fix various minor issue on those steps (e.g. packages, dependencies, deprecated things) The details and patch references in the work on feature/s3api are listed at https://trello.com/b/ZloaZ23t/s3api (completed board) Note that, because this is just a porting, no new feature is developed since the last swift3 release, and in the future work, Swift upstream may continue to work on remaining items for further improvements and the best compatibility of Amazon S3. Please read the new docs for your deployment and keep track to know what would be changed in the future releases. Change-Id: Ib803ea89cfee9a53c429606149159dd136c036fd Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Tim Burke <tim.burke@gmail.com>
2017-10-16 21:39:12 +09:00
s3api = swift.common.middleware.s3api.s3api:filter_factory
s3token = swift.common.middleware.s3api.s3token:filter_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013-05-11 15:35:37 -04:00
Import upstream version 1.0.1
2010-07-19 11:22:40 -05:00
[egg_info]
Allow access to the in-memory object server This will allow functional tests to be used against it, and can then be used for the storage-policy work as an example diskfile implementation associated with a storage policy. Change-Id: I47a88e70cee99225779baaed379b0c5d4c73611a
2013-12-04 16:05:56 -05:00
tag_build =
Import upstream version 1.0.1
2010-07-19 11:22:40 -05:00
tag_date = 0
tag_svn_revision = 0
add pybabel setup.py commands and initial .pot
2011-01-27 00:01:24 +00:00
[compile_catalog]
Fix directory value for compile_catalog Commit 7a192987c0a5edb9e239ffb4aba3b1dac083b41e sets up swift for translation but the compile_catalog directory option is pointing at the wrong location to scan for po files. Change-Id: Id4dd24ddfde735ef8ef064882bea045361b5db90 Closes-Bug: #1367086
2014-09-08 21:00:49 -07:00
directory = swift/locale
add pybabel setup.py commands and initial .pot
2011-01-27 00:01:24 +00:00
domain = swift
[update_catalog]
domain = swift
Fix translation setup Fix the output directory, it should be swift/locale. This fixes the importing of translations. Change-Id: I48311773c9d200c3b1739dc796618849416096ed
2014-11-19 09:11:55 -05:00
output_dir = swift/locale
Setup localization properly To start translation of swift, we need to initially import the translation file - and place it at the proper place so that the usual CI scripts can handle it. The proper place is for all python projects $PROJECT/locale/$PROJECT.pot, so move locale/$PROJECT.pot to the new location and regenerate the file. Update setup.cfg with the new paths. Further imports will be done by the OpenStack Proposal bot. Change-Id: Ide4da91f2af71db529f4a06d6b1e30ba79883506 Partial-Bug: #608725 Closes-Bug: #1082805
2014-06-01 11:51:29 +02:00
input_file = swift/locale/swift.pot
add pybabel setup.py commands and initial .pot
2011-01-27 00:01:24 +00:00
[extract_messages]
keywords = _ l_ lazy_gettext
mapping_file = babel.cfg
Setup localization properly To start translation of swift, we need to initially import the translation file - and place it at the proper place so that the usual CI scripts can handle it. The proper place is for all python projects $PROJECT/locale/$PROJECT.pot, so move locale/$PROJECT.pot to the new location and regenerate the file. Update setup.cfg with the new paths. Further imports will be done by the OpenStack Proposal bot. Change-Id: Ide4da91f2af71db529f4a06d6b1e30ba79883506 Partial-Bug: #608725 Closes-Bug: #1082805
2014-06-01 11:51:29 +02:00
output_file = swift/locale/swift.pot
Add support for venv-based test run with tox. * Adds tox config - based on the config from python-quantumclient and updated for test, pep8 and coverage execution as per nova's run_tests.sh. * Adds nosetests defaults in setup.cfg * Adds runtime dependencies in tools/pip-requires - dependencies were gathered by referencing the packages used in creation of a Swift All In One. Versions were determined by checking the swift-core/trunk ppa or, failing that, the version available in lucid. * Adds test dependencies in tools/test-requires * Updates swift/common/middleware/formpost.py for pep8 compliance * Adds instructions for executing the tests with Tox to the developer_guidelines * Adds instructions for installing openstack.nose_plugin to developer_saio * Fixes bug 909177 Change-Id: I5407924d2181e9ab335aaf76bf30c8d40deccbb4
2012-01-31 15:27:29 -08:00
[nosetests]
Change setup.cfg style. Change-Id: I466470237ddd317740eb99bd55af7e7a50270ae3 Fixes: bug #1163258
2013-04-02 15:52:08 +04:00
exe = 1
verbosity = 2
detailed-errors = 1
Align tox.ini and fix coverage jobs in jenkins. The jenkins coverage jobs expect there to be a .coverage file, so deleting it is a bad idea. Also, coverage erase will do that for us. While we're in there, update tox.ini and setup.cfg to the latest. Change-Id: Icd0a8fc66a5146e0d94f62a9f65a4536981d2916
2012-06-08 19:59:26 -04:00
cover-package = swift
cover-html = true
cover-erase = true
Copy Permalink