openstack/openstack-ansible-ops
Code Issues Proposed changes
bf71876abf
openstack-ansible-ops/bowling_ball
History
Nolan Brubaker 54f91a1b2f Add API query test for Swift 
This test will be useful in gauging API availability for Swift during
upgrades

Change-Id: I1e5ac7ab1cfed15771fdb205d5736da206929bc6
2017-06-05 13:13:28 -04:00
..
README.rst Implement CLI test picker 2017-05-19 11:47:49 +00:00
rolling_restart.py Unify output formatting, add logging 2017-03-15 16:36:32 -04:00
rolling_tests.py Add API query test for Swift 2017-06-05 13:13:28 -04:00

README.rst

Bowling Ball - OpenStack-Ansible Rolling Downtime Simulator

date

2017-03-09

tags

rackspace, openstack, ansible

category

*openstack, *nix

About

This project aims to test for issues with rolling downtime on OpenStack-Ansible deployments. It's comprised of two main components:

  • The rolling_restart.py script
  • The rolling_test.py script

The rolling_restart.py script will stop containers from a specified group in a rolling fashion - node 1 will stop, then start, then node 2, then node 3 and so on. This script runs from the deployment host.

The tests directory contains scripts to generate traffic against the target services. system that will be restarted by rolling_restart.py in order to measure the effects. These scripts run from a utility container.

The rolling_test.py script contains tests to generate traffic against the target services. These vary per service, but attempt to apply usage to a system that will be restarted by rolling_restart.py in order to measure the effects. This script runs from a utility container.

Usage

  1. Start your test from a utility container. ./rolling_test.py keystone runs the Keystone test. ./rolling_test.py list will list tests and their descriptions

  2. From the deployment node, run rolling_restart.py in the playbooks directory (necessary to find the inventory script). Specify the service you're targeting with the -s parameter.

    rolling_restart.py -s keystone_container

    You can specify a wait time in seconds between stopping and starting individual nodes.

    rolling_restart.py -s keystone_container -w 60

Assumptions

These tools are currently coupled to OSA, and they assume paths to files as specified by the multi-node-aio scripts.

Container stopping and starting is done with an ansible command, and the physical host to target is derived from the current inventory.

rolling_restart.py must currently be run from the playbooks directory. This will be fixed later.

You must source openrc before running rolling_test.py.

Creating New Tests

Tests should subclass from the ServiceTest class in the same file and implement the following properties and methods:

  1. run - The actual test to run should be placed in this method. Timings

    will be gathered based on when this function starts and stops.

  2. pre_test - Any pre-test setup that needs to happen, like creating a

    file for Glance, Cinder, or Swift upload.

  3. post_test - Any post-test teardown that might be needed.
  4. service_name - The primary service that is being tested.
  5. description - Brief description of what the test does.

Finally, add the test to the available_tests dictionary with the invocation name as the key and the class as the value.

Why the name?

It sets 'em up and knocks em down.