docs: clean up SAIO formatting

Drive-by: use six.moves in s3api; fix "unexpected indent" warning when building
docs on py3

Change-Id: I2a354e2624c763a68fcea7a6404e9c2fde30d631
This commit is contained in:
Tim Burke 2019-03-04 17:37:09 -08:00
parent 96013436a1
commit d185b607bb
3 changed files with 332 additions and 291 deletions
doc/source
swift/common
bufferedhttp.py
middleware/s3api/controllers

@ -76,7 +76,8 @@ Installing dependencies
python2-netifaces python2-pip python2-dnspython \ python2-netifaces python2-pip python2-dnspython \
python2-mock python2-mock
Note: This installs necessary system dependencies and *most* of the python .. note::
This installs necessary system dependencies and *most* of the python
dependencies. Later in the process setuptools/distribute or pip will install dependencies. Later in the process setuptools/distribute or pip will install
and/or upgrade packages. and/or upgrade packages.
@ -116,14 +117,16 @@ another device when creating the VM, and follow these instructions:
# **Make sure to include the trailing slash after /srv/$x/** # **Make sure to include the trailing slash after /srv/$x/**
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
Note: For OpenSuse users, a user's primary group is `users`, so you have 2 options: .. note::
For OpenSuse users, a user's primary group is ``users``, so you have 2 options:
* Change `${USER}:${USER}` to `${USER}:users` in all references of this guide; or * Change ``${USER}:${USER}`` to ``${USER}:users`` in all references of this guide; or
* Create a group for your username and add yourself to it:: * Create a group for your username and add yourself to it::
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER} sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
Note: We create the mount points and mount the storage disk under .. note::
We create the mount points and mount the storage disk under
/mnt/sdb1. This disk will contain one directory per simulated swift node, /mnt/sdb1. This disk will contain one directory per simulated swift node,
each owned by the current swift user. each owned by the current swift user.
@ -173,14 +176,16 @@ these instructions:
# **Make sure to include the trailing slash after /srv/$x/** # **Make sure to include the trailing slash after /srv/$x/**
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
Note: For OpenSuse users, a user's primary group is `users`, so you have 2 options: .. note::
For OpenSuse users, a user's primary group is ``users``, so you have 2 options:
* Change `${USER}:${USER}` to `${USER}:users` in all references of this guide; or * Change ``${USER}:${USER}`` to ``${USER}:users`` in all references of this guide; or
* Create a group for your username and add yourself to it:: * Create a group for your username and add yourself to it::
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER} sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
Note: We create the mount points and mount the loopback file under .. note::
We create the mount points and mount the loopback file under
/mnt/sdb1. This file will contain one directory per simulated swift node, /mnt/sdb1. This file will contain one directory per simulated swift node,
each owned by the current swift user. each owned by the current swift user.
@ -250,8 +255,10 @@ Getting the code
cd $HOME/swift; sudo pip install --no-binary cryptography -r requirements.txt; sudo python setup.py develop; cd - cd $HOME/swift; sudo pip install --no-binary cryptography -r requirements.txt; sudo python setup.py develop; cd -
Note: Due to a difference in libssl.so naming in OpenSuse to other Linux distros the wheel/binary wont work so the .. note::
cryptography must be built, thus the ``--no-binary cryptography``. Due to a difference in how ``libssl.so`` is named in OpenSuse vs. other Linux distros the
wheel/binary won't work; thus we use ``--no-binary cryptography`` to build ``cryptography``
locally.
Fedora 19 or later users might have to perform the following if development Fedora 19 or later users might have to perform the following if development
installation of swift fails:: installation of swift fails::
@ -275,6 +282,7 @@ Setting up rsync
that is copied and fixed up above: that is copied and fixed up above:
.. literalinclude:: /../saio/rsyncd.conf .. literalinclude:: /../saio/rsyncd.conf
:language: ini
#. On Ubuntu, edit the following line in ``/etc/default/rsync``:: #. On Ubuntu, edit the following line in ``/etc/default/rsync``::
@ -379,6 +387,7 @@ Optional: Setting up rsyslog for individual logging
below: below:
.. literalinclude:: /../saio/rsyslog.d/10-swift.conf .. literalinclude:: /../saio/rsyslog.d/10-swift.conf
:language: ini
#. Edit ``/etc/rsyslog.conf`` and make the following change (usually in the #. Edit ``/etc/rsyslog.conf`` and make the following change (usually in the
"GLOBAL DIRECTIVES" section):: "GLOBAL DIRECTIVES" section)::
@ -434,70 +443,87 @@ commands are as follows:
#. ``/etc/swift/swift.conf`` #. ``/etc/swift/swift.conf``
.. literalinclude:: /../saio/swift/swift.conf .. literalinclude:: /../saio/swift/swift.conf
:language: ini
#. ``/etc/swift/proxy-server.conf`` #. ``/etc/swift/proxy-server.conf``
.. literalinclude:: /../saio/swift/proxy-server.conf .. literalinclude:: /../saio/swift/proxy-server.conf
:language: ini
#. ``/etc/swift/object-expirer.conf`` #. ``/etc/swift/object-expirer.conf``
.. literalinclude:: /../saio/swift/object-expirer.conf .. literalinclude:: /../saio/swift/object-expirer.conf
:language: ini
#. ``/etc/swift/container-reconciler.conf`` #. ``/etc/swift/container-reconciler.conf``
.. literalinclude:: /../saio/swift/container-reconciler.conf .. literalinclude:: /../saio/swift/container-reconciler.conf
:language: ini
#. ``/etc/swift/container-sync-realms.conf`` #. ``/etc/swift/container-sync-realms.conf``
.. literalinclude:: /../saio/swift/container-sync-realms.conf .. literalinclude:: /../saio/swift/container-sync-realms.conf
:language: ini
#. ``/etc/swift/account-server/1.conf`` #. ``/etc/swift/account-server/1.conf``
.. literalinclude:: /../saio/swift/account-server/1.conf .. literalinclude:: /../saio/swift/account-server/1.conf
:language: ini
#. ``/etc/swift/container-server/1.conf`` #. ``/etc/swift/container-server/1.conf``
.. literalinclude:: /../saio/swift/container-server/1.conf .. literalinclude:: /../saio/swift/container-server/1.conf
:language: ini
#. ``/etc/swift/object-server/1.conf`` #. ``/etc/swift/object-server/1.conf``
.. literalinclude:: /../saio/swift/object-server/1.conf .. literalinclude:: /../saio/swift/object-server/1.conf
:language: ini
#. ``/etc/swift/account-server/2.conf`` #. ``/etc/swift/account-server/2.conf``
.. literalinclude:: /../saio/swift/account-server/2.conf .. literalinclude:: /../saio/swift/account-server/2.conf
:language: ini
#. ``/etc/swift/container-server/2.conf`` #. ``/etc/swift/container-server/2.conf``
.. literalinclude:: /../saio/swift/container-server/2.conf .. literalinclude:: /../saio/swift/container-server/2.conf
:language: ini
#. ``/etc/swift/object-server/2.conf`` #. ``/etc/swift/object-server/2.conf``
.. literalinclude:: /../saio/swift/object-server/2.conf .. literalinclude:: /../saio/swift/object-server/2.conf
:language: ini
#. ``/etc/swift/account-server/3.conf`` #. ``/etc/swift/account-server/3.conf``
.. literalinclude:: /../saio/swift/account-server/3.conf .. literalinclude:: /../saio/swift/account-server/3.conf
:language: ini
#. ``/etc/swift/container-server/3.conf`` #. ``/etc/swift/container-server/3.conf``
.. literalinclude:: /../saio/swift/container-server/3.conf .. literalinclude:: /../saio/swift/container-server/3.conf
:language: ini
#. ``/etc/swift/object-server/3.conf`` #. ``/etc/swift/object-server/3.conf``
.. literalinclude:: /../saio/swift/object-server/3.conf .. literalinclude:: /../saio/swift/object-server/3.conf
:language: ini
#. ``/etc/swift/account-server/4.conf`` #. ``/etc/swift/account-server/4.conf``
.. literalinclude:: /../saio/swift/account-server/4.conf .. literalinclude:: /../saio/swift/account-server/4.conf
:language: ini
#. ``/etc/swift/container-server/4.conf`` #. ``/etc/swift/container-server/4.conf``
.. literalinclude:: /../saio/swift/container-server/4.conf .. literalinclude:: /../saio/swift/container-server/4.conf
:language: ini
#. ``/etc/swift/object-server/4.conf`` #. ``/etc/swift/object-server/4.conf``
.. literalinclude:: /../saio/swift/object-server/4.conf .. literalinclude:: /../saio/swift/object-server/4.conf
:language: ini
.. _setup_scripts: .. _setup_scripts:
@ -516,6 +542,7 @@ Setting up scripts for running Swift
The template ``resetswift`` script looks like the following: The template ``resetswift`` script looks like the following:
.. literalinclude:: /../saio/bin/resetswift .. literalinclude:: /../saio/bin/resetswift
:language: bash
If you are using a loopback device add an environment var to If you are using a loopback device add an environment var to
substitute ``/dev/sdb1`` with ``/srv/swift-disk``:: substitute ``/dev/sdb1`` with ``/srv/swift-disk``::
@ -535,6 +562,7 @@ Setting up scripts for running Swift
The template ``test.conf`` looks like the following: The template ``test.conf`` looks like the following:
.. literalinclude:: /../../test/sample.conf .. literalinclude:: /../../test/sample.conf
:language: ini
#. Add an environment variable for running tests below:: #. Add an environment variable for running tests below::
@ -555,12 +583,16 @@ Setting up scripts for running Swift
The ``remakerings`` script looks like the following: The ``remakerings`` script looks like the following:
.. literalinclude:: /../saio/bin/remakerings .. literalinclude:: /../saio/bin/remakerings
:language: bash
You can expect the output from this command to produce the following. Note You can expect the output from this command to produce the following. Note
that 3 object rings are created in order to test storage policies and EC in that 3 object rings are created in order to test storage policies and EC in
the SAIO environment. The EC ring is the only one with all 8 devices. the SAIO environment. The EC ring is the only one with all 8 devices.
There are also two replication rings, one for 3x replication and another There are also two replication rings, one for 3x replication and another
for 2x replication, but those rings only use 4 devices:: for 2x replication, but those rings only use 4 devices:
.. code-block:: console
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0 Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1 Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
@ -612,6 +644,7 @@ Setting up scripts for running Swift
The ``startmain`` script looks like the following: The ``startmain`` script looks like the following:
.. literalinclude:: /../saio/bin/startmain .. literalinclude:: /../saio/bin/startmain
:language: bash
#. Get an ``X-Storage-Url`` and ``X-Auth-Token``:: #. Get an ``X-Storage-Url`` and ``X-Auth-Token``::

@ -178,6 +178,14 @@ class BufferedHTTPConnection(HTTPConnection):
return ret return ret
def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0): def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0):
'''Send a request to the server.
:param method: specifies an HTTP request method, e.g. 'GET'.
:param url: specifies the object being requested, e.g. '/index.html'.
:param skip_host: if True does not add automatically a 'Host:' header
:param skip_accept_encoding: if True does not add automatically an
'Accept-Encoding:' header
'''
self._method = method self._method = method
self._path = url self._path = url
return HTTPConnection.putrequest(self, method, url, skip_host, return HTTPConnection.putrequest(self, method, url, skip_host,

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from urllib import quote from six.moves.urllib.parse import quote
from swift.common.utils import public from swift.common.utils import public
from swift.common.middleware.s3api.controllers.base import Controller from swift.common.middleware.s3api.controllers.base import Controller