This reverts commit 817a19c4b9ced01af9bbb6f7383283e3f0f0e9bd.
This apparently hits us in gate where I can see at least one
failure with 5.44 secs taken for a scenario.
Change-Id: Ied0516686e86167a0c7b1d480eb1db2789f7cada
New validation now enforces that times is >= concurrency because times
is total number of *runs*, not the number of concurrent *run sets*.
Change-Id: I454ef821e00bd5123a9640f472ad4b034dbec75e
Closes-Bug: #1680580
In a normal gate run these are returning in 2 seconds each
on average. Let's reduce the SLA of these from 15 to 5 now
to help prevent future performance regressions in this area.
Change-Id: Iae174c95d214c83d6726a3d3bd339dde7886af4f
During a normal run, the top three scenarios account for slightly
more than a half hour of runtime. Sample numbers:
Scenario Load Duration Full Duration
NeutronNetworks.create_and_update_subnets 562.189 1,182.400
NeutronTrunks.create_and_list_trunk_subports 427.475 600.721
NeutronNetworks.create_and_list_ports 310.167 540.144
This patch reduces the resources created by each of the 3 by 75%. This
should save us an additional ~20 minutes during a normal gate run, which
should change our window for timeout from approximately a half hour on a
normal node to about 50 minutes.
This additional buffer should hopefully be enough to reduce the failure
rate for the rally job when it gets scheduled to a slow node.
Change-Id: I923b625f7dd3ebf794b6a9e097f5ed12ce446bb5
This adds a basic rally scenario to create a trunk
with a bunch of subports so we can keep an eye on the
performance of the trunk API.
Change-Id: I12aaf6121b677e9696131601b3539a7091e2858c
This allows us to configure neutron when running the rally job in
the gate. This effort stems from patch [1]. Blame Kevin for not
wanting to squash the two together.
[1] I12aaf6121b677e9696131601b3539a7091e2858c
Change-Id: I006957784ac7900021bcfee57cbc83b5a6c533c4
The previous configuration of the task was taking up
to a half hour to run. Between this task and the others,
it was eating up all of our gate time, leaving no room
to add new jobs.
This reduces it 5 times in the number of runs, from 40
to 8. This still gives us a reasonable number to get an
average from, especially since each run creates 100 ports.
Change-Id: I0955e44df1a9e072c58fdacc337121b8621132df
Previous runs are showing that creating ports under this high load
ends up taking >5 secs per port on average. Lets set to 4, which is
still double the api_worker count in some cases for the current gate.
Change-Id: I05a0d28f5b035684e07288825f5b704a843dc9d7
Quoting the quota devref:
"""
For a reservation to be successful, the total amount of resources requested,
plus the total amount of resources reserved, plus the total amount of resources
already stored in the database should not exceed the project's quota limit.
"""
This means that in the absolute worst case scenario with 20 concurrent
workers, 19 could have made reservations, committed resources, but not
yet cleared their reservation. Because of the outstanding reservation
and the resources created by the 19 workers, they will all be
double-counted until their reservation is cleared (or it expires).
This adjusts the rally scenarios to handle the double-count for
concurrency.
Related-Bug: #1623390
Change-Id: I4808a92e7e6067aeeb62fc3b3d7f7ac71b179c44
This increases the rally port and network count to 100
and enables quotas to exercise the quota engine to
better simulate a real system.
Additionally, it reduces the SLA requirements because of
regressions that have snuck in throughout the cycle. As
they are fixed these should be reduced back down.
Change-Id: I042d64245b1e4486334996d834ad31561613fa50
Increase the ports per network in the create and list ports
test. This also adds a max average time SLA so we can catch
regressions in performance.
Change-Id: I2e7e3fd7406db77c8e44dce2ab0b4594ff6f2db9
* Since 24 Nov 2014 we added a lot of Neutron benchmarks
Running more Neutron related benchmarks in Neutron gate allows
to avoid performance regressions and races.
* Neutron benchmarks are described here:
https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/neutron/network.py
It's quite simple code be free to take a look.
* All changes in concurrency and times are related to optimization
of duration/usefulness
* To get description of benchmarks use:
rally info find NeutronNetworks.create_and_update_networks
New benchmarks:
- NeutronNetworks.create_and_update_networks
- NeutronNetworks.create_and_delete_networks
- NeutronNetworks.create_and_update_subnets
- NeutronNetworks.create_and_delete_subnets
- NeutronNetworks.create_and_update_routers
- NeutronNetworks.create_and_delete_routers
- NeutronNetworks.create_and_list_routers
- NeutronNetworks.create_and_update_ports
- NeutronNetworks.create_and_delete_ports
- NeutronNetworks.create_and_list_ports
- Quotas.neutron_update
related bug: #bug 1419723
Change-Id: Ie3c84e057fc96c0f35ad77b7297c564442ebcf10
*) Rename rally-scenarios that is quite misleading to rally-jobs.
rally-jobs makes much more sense, cause it actually contains files
related to rally job
*) Update rally-jobs/README.rst to add more info
*) Update rally-jobs/plugins/README.rst to expaling plugins
*) Add new directory rally-jobs/extra, this directory is copy pasted
in gates and can be used for files that are required by some of
benchmarks
Change-Id: I6d0c0435a4bb4658ddf4adb871bc36ab8c157f3e
