manila/HACKING.rst
vponomaryov 16a04df3d0 Refactor test framework
Manila's test framework is pretty old and requires update.

Changes:
- usage of nose replaced with testr
- now all the tests are thread safe
- added new options for run_tests.sh, such as --concurrency, --debug, etc...
- new '--concurrency' option for run_tests.sh defaults to 1, examples:
    ./run_tests.sh  # will run in 1 thread
    ./run_tests.sh --concurrency 2  # will run tests in 2 threads
- added tools/colorizer.py for colorizing output of testrun with run_tests.sh
- tests running with tox use as much threads as cores available by default
- examples of testrun with tox:
    tox  # will run test suites defined with 'envlist' in tox.ini, now it is pep8,py26,py27
    tox -epy27  # amount of threads is equal to amount of cores
    tox -epy27 -- --concurrency=2  # amount of threads is 2
    tox -epy27 -- --concurrency=4  # amount of threads is 4
- Added 'Database' class to manila.test module, for more conveniant db testing
- updated policy file 'manila/tests/policy.json' to allow share-network actions
- removed nose-related requirements
- added new requirements for testrepository, subunit

With merge of this change all old installed virtual environments become
incompatible and should be removed with "rm -rf .tox .venv" before testrun.

Implements blueprint testr-with-unittests

Change-Id: I9579ecd538e29d478dbc12adc7dcc33fc668b397
2014-07-22 14:58:48 +03:00

8.2 KiB

Manila Style Commandments =======================

General

  • Put two newlines between top-level code (funcs, classes, etc)

  • Put one newline between methods in classes and anywhere else

  • Long lines should be wrapped in parentheses in preference to using a backslash for line continuation.

  • Do not write "except:", use "except Exception:" at the very least

  • Include your name with TODOs as in "#TODO(termie)"

  • Do not shadow a built-in or reserved word. Example:

    def list():
        return [1, 2, 3]
    
    mylist = list() # BAD, shadows `list` built-in
    
    class Foo(object):
        def list(self):
            return [1, 2, 3]
    
    mylist = Foo().list() # OKAY, does not shadow built-in
  • Use the "is not" operator when testing for unequal identities. Example:

    if not X is Y:  # BAD, intended behavior is ambiguous
        pass
    
    if X is not Y:  # OKAY, intuitive
        pass
  • Use the "not in" operator for evaluating membership in a collection. Example:

    if not X in Y:  # BAD, intended behavior is ambiguous
        pass
    
    if X not in Y:  # OKAY, intuitive
        pass
    
    if not (X in Y or X in Z):  # OKAY, still better than all those 'not's
        pass

Imports

  • Do not import objects, only modules (*)
  • Do not import more than one module per line (*)
  • Do not make relative imports
  • Order your imports by the full module path
  • Organize your imports according to the following template

(*) exceptions are:

  • imports from migrate package
  • imports from sqlalchemy package
  • imports from manila.db.sqlalchemy.session module

Example:

# vim: tabstop=4 shiftwidth=4 softtabstop=4
{{stdlib imports in human alphabetical order}}
\n
{{third-party lib imports in human alphabetical order}}
\n
{{manila imports in human alphabetical order}}
\n
\n
{{begin your code}}

Human Alphabetical Order Examples

Example:

import httplib
import logging
import random
import StringIO
import time
import unittest

import eventlet
import webob.exc

import manila.api.ec2
from manila.api import openstack
from manila.auth import users
from manila.endpoint import cloud
import manila.flags
from manila import test

Docstrings

Example:

"""A one line docstring looks like this and ends in a period."""


"""A multi line docstring has a one-line summary, less than 80 characters.

Then a new paragraph after a newline that explains in more detail any
general information about the function, class or method. Example usages
are also great to have here if it is a complex class for function.

When writing the docstring for a class, an extra line should be placed
after the closing quotations. For more in-depth explanations for these
decisions see http://www.python.org/dev/peps/pep-0257/

If you are going to describe parameters and return values, use Sphinx, the
appropriate syntax is as follows.

:param foo: the foo parameter
:param bar: the bar parameter
:returns: return_type -- description of the return value
:returns: description of the return value
:raises: AttributeError, KeyError
"""

Dictionaries/Lists

If a dictionary (dict) or list object is longer than 80 characters, its items should be split with newlines. Embedded iterables should have their items indented. Additionally, the last item in the dictionary should have a trailing comma. This increases readability and simplifies future diffs.

Example:

my_dictionary = {
    "image": {
        "name": "Just a Snapshot",
        "size": 2749573,
        "properties": {
             "user_id": 12,
             "arch": "x86_64",
        },
        "things": [
            "thing_one",
            "thing_two",
        ],
        "status": "ACTIVE",
    },
}

Calling Methods

Calls to methods 80 characters or longer should format each argument with newlines. This is not a requirement, but a guideline:

unnecessarily_long_function_name('string one',
                                 'string two',
                                 kwarg1=constants.ACTIVE,
                                 kwarg2=['a', 'b', 'c'])

Rather than constructing parameters inline, it is better to break things up:

list_of_strings = [
    'what_a_long_string',
    'not as long',
]

dict_of_numbers = {
    'one': 1,
    'two': 2,
    'twenty four': 24,
}

object_one.call_a_method('string three',
                         'string four',
                         kwarg1=list_of_strings,
                         kwarg2=dict_of_numbers)

Internationalization (i18n) Strings

In order to support multiple languages, we have a mechanism to support automatic translations of exception and log strings.

Example:

msg = _("An error occurred")
raise HTTPBadRequest(explanation=msg)

If you have a variable to place within the string, first internationalize the template string then do the replacement.

Example:

msg = _("Missing parameter: %s") % ("flavor",)
LOG.error(msg)

If you have multiple variables to place in the string, use keyword parameters. This helps our translators reorder parameters when needed.

Example:

msg = _("The server with id %(s_id)s has no key %(m_key)s")
LOG.error(msg % {"s_id": "1234", "m_key": "imageId"})

Creating Unit Tests

For every new feature, unit tests should be created that both test and (implicitly) document the usage of said feature. If submitting a patch for a bug that had no unit test, a new passing unit test should be added. If a submitted bug fix does have a unit test, be sure to add a new one that fails without the patch and passes with the patch.

For more information on creating unit tests and utilizing the testing infrastructure in OpenStack Manila, please read manila/testing/README.rst.

openstack-common

A number of modules from openstack-common are imported into the project.

These modules are "incubating" in openstack-common and are kept in sync with the help of openstack-common's update.py script. See:

http://wiki.openstack.org/CommonLibrary#Incubation

The copy of the code should never be directly modified here. Please always update openstack-common first and then run the script to copy the changes across.

Running Tests

The testing system is based on a combination of tox and testr. If you just want to run the whole suite, run tox and all will be fine. However, if you'd like to dig in a bit more, you might want to learn some things about testr itself. A basic walkthrough for OpenStack can be found at http://wiki.openstack.org/testr

OpenStack Trademark

OpenStack is a registered trademark of OpenStack, LLC, and uses the following capitalization:

OpenStack

Commit Messages

Using a common format for commit messages will help keep our git history readable. Follow these guidelines:

First, provide a brief summary (it is recommended to keep the commit title under 50 chars).

The first line of the commit message should provide an accurate description of the change, not just a reference to a bug or blueprint. It must be followed by a single blank line.

If the change relates to a specific driver (libvirt, xenapi, qpid, etc...), begin the first line of the commit message with the driver name, lowercased, followed by a colon.

Following your brief summary, provide a more detailed description of the patch, manually wrapping the text at 72 characters. This description should provide enough detail that one does not have to refer to external resources to determine its high-level functionality.

Once you use 'git review', two lines will be appended to the commit message: a blank line followed by a 'Change-Id'. This is important to correlate this commit with a specific review in Gerrit, and it should not be modified.

For further information on constructing high quality commit messages, and how to split up commits into a series of changes, consult the project wiki:

http://wiki.openstack.org/GitCommitMessages