13 Commits

Author SHA1 Message Date
Félix Cantournet
4f053c61d6 Fix swift-dispersion in multi-region setups
If you have 2 swift regions served by the same keystone,
then the client cannot get the correct URL for the swift endpoint
without specifying a region_name.

Closes-Bug: 1587088
Change-Id: Iaab883386e125c3ca6b9554389e63df17267a135
2016-06-01 15:35:47 +02:00
Falk Reimann
363a256e58 Support keystone v3 domains in swift-dispersion
This provides the capability to specify a project_name,
project_domain_name and user_domain_name in /etc/swift/dispersion.conf.
If this values are set in dispersion.conf they get populated to the
swift-client.  With this it is possible to have a specific dispersion
project specified, which is not the keystone default domain.  Changes
were applied to swift-dispersion-populate and swift-dispersion-report.
Relevant man pages, the example dispersion.conf and the admin guide were
updated accordingly.

DocImpact
Closes-Bug: #1468374

Change-Id: I0e716f8d281b4d0f510bc568bcee4a13fc480ff7
2015-07-24 13:40:24 -05:00
Matthew Oliver
090baa1fa9 Swift configuration parameter audit
This change is the result of an audit through the config parameters
provided by swift and how/if they are addressed in the swift
documentation. The documentation being the sample config files in
the /etc directory or the documentation.

This change is only concerned with the config files in etc/ next
I will look at the documentation in the doc/ folder.

This change makes the following assumptions:
  - Unless stated otherwise, the commented out parameter in the
    sample configuration is the default for swift.

  - When the default in the code differs from that of the sample
    configuration, the default in the code is correct.

Container reconciler:
  Parameter: interval
    - code: 30
    - config: 300
  Result: config = 30

Object Expirer:
  Parameter: recon_cache_path
    - code: /var/cache/swift
    - config: Parameter missing
  Result: Add parameter

swift-dispersion-populate && swift-dispersion-report
  Parameter: auth_version
    - code: 1.0
    - config: 2.0 (due to being a confusing example of how to setup
                   version 2.0).
  Result: Added 'auth_version = 1.0' to the right section (showing
          default and make the sample configuration for auth version
          2.0 easier to understand.

swift-drive-audit:
  Parameter: log_file_pattern
    - code: /var/log/kern.*[!.][!g][!z]
    - config: /var/log/kern*
  Result: config = /var/log/kern.*[!.][!g][!z]

  NOTE: swift-drive-audit uses a parameter called device_dir which
        defaults to '/srv/node'. In all other swift binaries/services
        there is a similar parameter called devices which stores the
        same thing. This is an inconsistency which I haven't fixed
        as this could break existing swift clusters out in the wild.

Proxy Server:
  Parameter: object_chunk_size
    - code: 65536
    - config: 8192
  Result: config = 65536

  Parameter: client_chunk_size
    - code: 65536
    - config: 8192
  Result: config = 65536

  Parameter: strict_cors_mode
    - code: True
    - config: No parameter
  Result: config = True

Account and Container replicator configuration confusion:
  NOTES:
    The account and container replicators have parameters:
      - interval
      - run_pause

    Both of these are loaded into the same variable in code:
      self.interval = int(conf.get('interval') or
                          conf.get('run_pause') or 30)

    If a user sets both to different values then interval is used.
  Result: Update the configuration to make this more clear.

DocImpact
Change-Id: Iaadbb1a6284f8b3e0801bc343b29772f70f4bf6e
2014-08-06 11:12:14 +10:00
Kun Huang
264766127e improve docs in etc/dispersion.conf-sample
1. add a comment to hint using a new account for using dispersion tools
2. change sample url for keystone from 'saio' to 'localhost'

Change-Id: I4683f5eb0af534b39112f1b7420f67d569c29b3a
2013-10-28 17:41:09 +08:00
Florian Hines
42f4b150e3 Faster swift-dispersion-populate
- Makes swift-dispersion-populate a bit faster when using a larger
  dispersion_coverage with a larger part_power.
- Adds option to only run population for container OR objects
- Adds option to let you resume population at given point (useful if you
  need to resume population after a previous run error'd out or the
  like) by specifying which suffix to start at.

The original populate just randomly used uuid4().hex as a suffix on the
container/object names until all the partition's required where covered.
This isn't a big deal if you're only doing 1% coverage on a ring with a
small part power but takes ages if you're doing 100% on a larger ring.

Change-Id: I52f890a774412c1d6179f12db9081aedc58b6bc2
2013-09-05 18:12:15 -05:00
Vincent Untz
7f1aa9d1e8 Allow dispersion tools to use keystone server with insecure certificate
The swift-dispersion-populate and swift-dispersion-report tools now
accept a --insecure option.

Also, dispersion.conf now has a keystone_api_insecure option.

Default is obviously to use the secure path.

DocImpact

Change-Id: I4000352e547d9ce5b08ade54e0c886281caff891
2013-08-05 22:44:12 +02:00
Thomas Leaman
5449155fb0 Allow floating point value for dispersion_coverage
For systems with very large numbers of partitions, 1% dispersion
coverage may simply be too much/take too long. This fix allows <1
values to be used for dispersion_coverage.

DocImpact

Change-Id: I5ed35b69754d55a410e66e658b3854de57c7666b
2013-07-09 09:29:07 +00:00
Mehdi Abaakouk
a1395ec672 Allow change the endpoint_type when use swift-dispersion tools
Fixes bug 1102319
DocImpact

Change-Id: I8fb0417ab9468e97ed01a6cb1e262630905e7f29
2013-01-31 16:10:37 +01:00
Florian Hines
e474dfb720 Add dispersion report flags to limit reports
- Add two optional flags that let you limit swift-dispersion-report to only
reporting on containers OR objects.
- Also make dispersion.conf and swift-dispersion-report manpages
  current.

DocImpact

Change-Id: Iad56133cad261241db27d0e2103098e3c2f3c245
2012-12-09 18:20:08 -06:00
Florian Hines
5f72a8db4a Fix Dispersion report and swift-bench on saio
We're still using saio:11000 in a few spots so a few things
don't work out of the box on the saio. Fixes bug #1024561

Change-Id: I226de54c2785b0d0b681c8d0cc24260adbd3d663
2012-07-13 17:48:37 -05:00
Julien Danjou
9a423d0b78 Allow to specify auth_version in swift-dispersion tools
Change-Id: I080d531471d8ea57c69a918d4f6930441f1e69f6
Signed-off-by: Julien Danjou <julien.danjou@enovance.com>
2012-04-02 16:42:05 +02:00
Florian Hines
5e4127ae2a Add json output option to swift-dispersion-report
Add's the configuration file option "dump_json" or command line
options [-j|--dump-json] to have swift-dispersion-report output
the report in json format. This allows the dispersion report to
be more easily consumed elsewhere.

There's also a few pep8 fixes and removal of unused imports.

Change-Id: I2374311ccbef43e6bbae24665c9584e60f3da173
2012-02-29 04:24:54 +00:00
gholt
6c13001244 Rename swift-stats-* to swift-dispersion-* to avoid confusion with log stats stuff 2011-03-31 22:32:41 +00:00