8.7 KiB
Development Guidelines
Coding Guidelines
For the most part we try to follow PEP 8 guidelines which can be viewed here: http://www.python.org/dev/peps/pep-0008/
Testing Guidelines
Swift has a comprehensive suite of tests and pep8 checks that are run on all submitted code, and it is recommended that developers execute the tests themselves to catch regressions early. Developers are also expected to keep the test suite up-to-date with any submitted code changes.
Swift's tests and pep8 checks can be executed in an isolated environment with Tox: http://tox.testrun.org/
To execute the tests:
Install Tox:
pip install tox
Generate list of distribution packages to install for testing:
tox -e bindep
Now install these packages using your distribution package manager like apt-get, dnf, yum, or zypper.
Run Tox from the root of the swift repo:
tox
Remarks: If you installed using
cd ~/swift; sudo python setup.py develop
, you may need to docd ~/swift; sudo chown -R ${USER}:${USER} swift.egg-info
prior to running tox.By default
tox
will run all of the unit test and pep8 checks listed in thetox.ini
fileenvlist
option. A subset of the test environments can be specified on the tox command line or by setting theTOXENV
environment variable. For example, to run only the pep8 checks and python2.7 unit tests use:tox -e pep8,py27
or:
TOXENV=py27,pep8 tox
Note
As of tox version 2.0.0, most environment variables are not
automatically passed to the test environment. Swift's
tox.ini
overrides this default behavior so that variable
names matching SWIFT_*
and *_proxy
will be
passed, but you may need to run tox --recreate
for this to
take effect after upgrading from tox<2.0.0.
Conversely, if you do not want those environment variables to be
passed to the test environment then you will need to unset them before
calling tox
.
Also, if you ever encounter DistributionNotFound, try to use
tox --recreate
or remove the .tox
directory to
force tox to recreate the dependency list.
Swift's functional tests may be executed against a development_saio
or other
running Swift cluster using the command:
tox -e func
The endpoint and authorization credentials to be used by functional
tests should be configured in the test.conf
file as
described in the section setup_scripts
.
The environment variable SWIFT_TEST_POLICY
may be set to
specify a particular storage policy name that will be used for
testing. When set, tests that would otherwise not specify a policy or
choose a random policy from those available will instead use the policy
specified. Tests that use more than one policy will include the
specified policy in the set of policies used. The specified policy must
be available on the cluster under test.
For example, this command would run the functional tests using policy 'silver':
SWIFT_TEST_POLICY=silver tox -e func
To run a single functional test, use the --no-discover
option together with a path to a specific test method, for example:
tox -e func -- --no-discover test.functional.tests.TestFile.testCopy
In-process functional testing
If the test.conf
file is not found then the functional
test framework will instantiate a set of Swift servers in the same
process that executes the functional tests. This 'in-process test' mode
may also be enabled (or disabled) by setting the environment variable
SWIFT_TEST_IN_PROCESS
to a true (or false) value prior to
executing tox -e func.
When using the 'in-process test' mode some server configuration options may be set using environment variables:
- the optional in-memory object server may be selected by setting the
environment variable
SWIFT_TEST_IN_MEMORY_OBJ
to a true value. - encryption may be added to the proxy pipeline by setting the
environment variable
SWIFT_TEST_IN_PROCESS_CONF_LOADER
toencryption
. Or when using tox, specify the tox environmentfunc-in-process-encryption
- the proxy-server
object_post_as_copy
option may be set using the environment variableSWIFT_TEST_IN_PROCESS_OBJECT_POST_AS_COPY
. - logging to stdout may be enabled by setting
SWIFT_TEST_DEBUG_LOGS
.
For example, this command would run the in-process mode functional tests with the proxy-server using object_post_as_copy=False (the 'fast-POST' mode):
SWIFT_TEST_IN_PROCESS=1 SWIFT_TEST_IN_PROCESS_OBJECT_POST_AS_COPY=False \
tox -e func
This particular example may also be run using the
func-in-process-fast-post
tox environment:
tox -e func-in-process-fast-post
The 'in-process test' mode searches for
proxy-server.conf
and swift.conf
config files
from which it copies config options and overrides some options to suit
in process testing. The search will first look for config files in a
<custom_conf_source_dir>
that may optionally be
specified using the environment variable:
SWIFT_TEST_IN_PROCESS_CONF_DIR=<custom_conf_source_dir>
If SWIFT_TEST_IN_PROCESS_CONF_DIR
is not set, or if a
config file is not found in <custom_conf_source_dir>
,
the search will then look in the etc/
directory in the
source tree. If the config file is still not found, the corresponding
sample config file from etc/
is used (e.g.
proxy-server.conf-sample
or
swift.conf-sample
).
When using the 'in-process test' mode SWIFT_TEST_POLICY
may be set to specify a particular storage policy name that
will be used for testing as described above. When set, this policy must
exist in the swift.conf
file and its corresponding ring
file must exist in <custom_conf_source_dir>
(if
specified) or etc/
. The test setup will set the specified
policy to be the default and use its ring file properties for
constructing the test object ring. This allows in-process testing to be
run against various policy types and ring files.
For example, this command would run the in-process mode functional
tests using config files found in $HOME/my_tests
and policy
'silver':
SWIFT_TEST_IN_PROCESS=1 SWIFT_TEST_IN_PROCESS_CONF_DIR=$HOME/my_tests \
SWIFT_TEST_POLICY=silver tox -e func
Coding Style
Swift uses flake8 with the OpenStack hacking module to enforce coding style.
Install flake8 and hacking with pip or by the packages of your Operating System.
It is advised to integrate flake8+hacking with your editor to get it automated and not get caught by Jenkins.
For example for Vim the syntastic plugin can do this for you.
Documentation Guidelines
The documentation in docstrings should follow the PEP 257 conventions (as mentioned in the PEP 8 guidelines).
More specifically:
- Triple quotes should be used for all docstrings.
- If the docstring is simple and fits on one line, then just use one line.
- For docstrings that take multiple lines, there should be a newline after the opening quotes, and before the closing quotes.
- Sphinx is used to build documentation, so use the restructured text markup to designate parameters, return values, etc. Documentation on the sphinx specific markup can be found here: http://sphinx.pocoo.org/markup/index.html
- Installing Sphinx:
-
- Install sphinx (On Ubuntu: sudo apt-get install python-sphinx)
- python setup.py build_sphinx
Manpages
For sanity check of your change in manpage, use this command in the root of your Swift repo:
./.manpages
License and Copyright
You can have the following copyright and license statement at the top of each source file. Copyright assignment is optional.
New files should contain the current year. Substantial updates can have another year added, and date ranges are not needed.:
# Copyright (c) 2013 OpenStack Foundation.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.