32 Commits

Author SHA1 Message Date
ZhiQiang Fan
f72704fc82 Change OpenStack LLC to Foundation
Change-Id: I7c3df47c31759dbeb3105f8883e2688ada848d58
Closes-bug: #1214176
2013-09-20 01:02:31 +08:00
Ksenia Demina
600e9c8672 Add delay in swift-bench
With enable write affinity, it's necessary to wait until
replication has moved things to their proper homes before
running delete request. With write affinity turned on, only
nodes in local region will get the object right after PUT request.

Fix bug #1198926

Change-Id: I3aa8933d45c47a010ae05561e12176479e7c9bcc
2013-08-02 11:47:33 +00:00
Darrell Bishop
5f295d0329 Don't delete containers when they weren't created.
When swift-bench is run in direct mode, don't try to delete the
containers which weren't created.

Fixes bug 1177960.

Change-Id: Ice07e8729bb776e2b215894cf95fb80b64167a8d
2013-05-11 13:37:16 -07:00
Victor Rodionov
7f534fac38 fix dict duplicated key
remove duplicated key from swift-bench bin; and also add .idea to gitignore

Change-Id: I31360489716d03324b8d2bb1c51e282393069c13
2013-04-22 01:31:53 +04:00
Kun Huang
45dda5e356 Change logger name to module itself instead of root
Using root logger will make public module log failed by unknow keyword.
Just change the logger name to module itself could dismiss this impact.
And disable logger.propagate could prevent double outputing.

Change-Id: I18696d124ebac9ca970d502558972e51de759097
Fixes: bug #1105133
2013-03-17 01:59:40 +08:00
Samuel Merritt
68479cc944 Add more command-line options to swift-bench.
These settings:
  1. are already exposed via config file
  2. were not exposed as command-line options, and
  3. are things I wanted on the command line while benchmarking recently.

I didn't include short options for --(get|put|delete)-concurrency
since I couldn't think of good single-letter ones, and optparse won't
take "-gc", "-pc", or "-dc".

Change-Id: I0bd7c7ae2892244eed37b8c10c9357dc7e1190d3
2013-01-22 18:41:08 -08:00
Joe Arnold
5e4ca30281 Fixed swift-bench when using -b and a conf file
Change-Id: I72448cd8578b53a658f151354a26066772e4b9ec
2012-11-07 16:12:04 -08:00
John Dickinson
8ac292595f changed TRUE_VALUES references to utils.config_true_value() call
cleaned up pep8 (v1.3.3) in all files this patch touches

Change-Id: I30e8314dfdc23fb70ab83741a548db9905dfccff
2012-10-29 13:59:01 -07:00
John Dickinson
af4b2851fc fix config parsing in swift-bench -x
This patch ensures that the command-line arg format (boolean)
doesn't conflict with the conf file format (string) and the
proper action is taken.

Change-Id: I3284096e1a9478897e1c3246ab190b46d2590243
2012-10-25 09:26:00 -07:00
Darrell Bishop
ed3b12d05c Can run swift-bench across multiple cores/servers.
You run one or more swift-bench-client processes like this:
    $ swift-bench-client 127.0.0.1 20001
    $ swift-bench-client 127.0.0.1 20002

Then you run swift-bench with a new option, --bench-clients (-b), which is
specified once for each swift-bench-client:
    $ swift-bench -b 127.0.0.1:20001 -b 127.0.0.1:20002

You get log lines from each client (interleaved) along with a final report
for all clients:
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:44:06,148 INFO Auth version: 1.0
    127.0.0.1:20001 swift-bench-server 2012-08-25 22:44:06,148 INFO Auth version: 1.0
    127.0.0.1:20001 swift-bench-server 2012-08-25 22:44:12,249 INFO 83 PUTS [0 failures], 41.5/s
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:44:14,430 INFO 74 PUTS [0 failures], 34.3/s
    ...
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:45:18,942 INFO Auth version: 1.0
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:45:20,946 INFO 238 DEL [2 failures], 118.9/s
    swift-bench 2012-08-25 22:45:27,549 INFO 2000 PUTS **FINAL** [0 failures], 56.8/s
    swift-bench 2012-08-25 22:45:27,550 INFO 30000 GETS **FINAL** [50 failures], 974.6/s
    swift-bench 2012-08-25 22:45:27,550 INFO 2000 DEL **FINAL** [20 failures], 237.1/s

The concurrency, PUT count, and GET count config settings are divided by
the number of bench_clients.  In other words, the same volume of work is
attempted (vs. not specifying --bench-clients), but it can now span
servers and CPU cores.

Benchmark containers are created (if use_proxy = yes) and deleted (if
delete = yes), with appropriate concurrency, in the initiating
swift-bench process, not any of the swift-bench-client processes.

Change-Id: Idbf31a23093244ab357a9bf77e6031257774f24a
2012-08-30 18:22:49 +00:00
Darrell Bishop
9bda92d54a Misc. swift-bench improvements.
swift-bench now honors the environment variables, ST_AUTH, ST_USER, and
ST_KEY like python-swiftclient does.

Added --lower-object-size (or -l) command-line option which, if
specified, will turn a specified --object-size into --upper-object-size.
It will raise a ValueError if --object-size is not specified or is <=
--lower-object-size.

BenchController how handles SIGINT (KeyboardInterrupt) similarly to the
swift command-line client: the first Ctrl-C will make it fast-track to
completion (no new PUT or GET operations are started, but everything PUT
is DELETE'ed).  A second Ctrl-C will immediately exit.  The behavior for
SIGTERM is unchanged (a single SIGTERM will immediately terminate the
process).

Added a sample configuration file for swift-bench, with documenting
comments.

Change-Id: I6f394ad995300fc8af3d565d95c3b45559ee510a
2012-08-27 20:25:44 +00:00
ning_zhang
aad7cdc364 To simulate the real workloads, the objects to be uploaded could be created in the random sizes, which are bounded (lower_object_size and upper_object_size) by the user inputs.
Change-Id: I64647c5d613a0794f0807886a4833cd5c31c0c5e
2012-08-03 11:47:39 -07:00
Florian Hines
5f72a8db4a Fix Dispersion report and swift-bench on saio
We're still using saio:11000 in a few spots so a few things
don't work out of the box on the saio. Fixes bug #1024561

Change-Id: I226de54c2785b0d0b681c8d0cc24260adbd3d663
2012-07-13 17:48:37 -05:00
François Charlier
ef3e556b72 swift-bench should be able to use auth version 2.0
Fixes Bug #1011727

Change-Id: I6681bd85a5ddb82a1059913ae90696e5994aa906
2012-06-13 09:06:57 +02:00
John Dickinson
1ecf5ebba1 updated copyright date for all files
Change-Id: Ifd909d3561c2647770a7e0caa3cd91acd1b4f298
2012-03-19 13:45:34 -05:00
David Goetz
51ce438f7f changing /usr/bin/python to /usr/bin/env python 2011-03-24 15:51:44 -07:00
Clay Gerrard
5d0bc6b9c7 logging refactor to support proxy access logs
New log level "notice" set to python log level 25 maps to syslog priority
LOG_NOTICE.  Used for some messages in the proxy server, but will be available
to all apps using the LogAdapter returned from get_logger.  Cleaned up some
code in get_logger so that console logging works with log_routes and removed
some unneeded bits.  NamedFormatter functionality was split between LogAdapter
(which now inherits from logging.LoggerAdapter) and TxnFormatter (which now is
only responsible for adding the log records txn_id).

The proxy server app now configures a separate logger for access line logging.
By default it will use the same settings as the regular proxy logger.
2011-02-10 14:59:52 -06:00
gholt
9dd1e2ae84 Updates to remove _ usage that is not i18n related 2011-01-19 15:21:57 -08:00
Clay Gerrard
ff0e62d914 reworked tnx logging 2011-01-07 21:17:29 +00:00
Clay Gerrard
6c1bf02a62 pep8 fo' realz 2011-01-05 09:32:19 -06:00
Anne Gentle
8823427161 Changed copyright notices on py files and the single rst file with a copyright notice 2011-01-04 17:34:43 -06:00
Clay Gerrard
68a9acf9b8 pep8 2011-01-04 16:12:56 -06:00
Clay Gerrard
97028e0b9c fixed missing NamedLogger import in bin/swift-bench, refactored
LogAdapter.tnx_id so that it works with multiple calls to get_logger, fixed
common.middleware.catch_errors to only call get_logger if it needs too, renamed
x-cf-trans-id to x-swift-tnx-id
2011-01-04 16:00:01 -06:00
gholt
57970bdeb5 Cleaned up the bins; patched the broken test (when run standalone) 2010-12-29 12:00:08 -08:00
Michael Barton
f432269013 add gettext to all binaries 2010-12-22 17:18:30 +00:00
Chuck Thier
43b3bf6095 Updated swift-bench to put to several containers (20 by default) instead of just 1 2010-10-20 22:04:03 +00:00
Chuck Thier
84a163d736 Somehow I missed the commit to print the whole help, not just usage 2010-10-07 21:36:39 +00:00
Chuck Thier
18245c1adc Updated to print usage by default if no args are passed, and added --saio to run with saio defaults 2010-10-04 15:09:53 +00:00
Chuck Thier
d7d0bd4253 added apache license to the new files 2010-10-01 19:50:34 +00:00
Chuck Thier
117aef456d Refactored swift-bench to reuse client.py and direct_client.py 2010-10-01 19:46:35 +00:00
Chuck Thier
f71a85a05b Missed a rename 2010-09-28 15:05:08 +00:00
Chuck Thier
6cef24dcd7 Initial checkin of benchmark code 2010-09-27 23:10:09 +00:00