Merge "docs: clean up SAIO formatting"

doc/source/development_saio.rst

@@ -76,7 +76,8 @@ Installing dependencies
                             python2-netifaces python2-pip python2-dnspython \
                             python2-mock
 
-  Note: This installs necessary system dependencies and *most* of the python
+.. note::
+   This installs necessary system dependencies and *most* of the python
    dependencies. Later in the process setuptools/distribute or pip will install
    and/or upgrade packages.
 
@@ -90,16 +91,16 @@ Using a partition for storage
 If you are going to use a separate partition for Swift data, be sure to add
 another device when creating the VM, and follow these instructions:
 
-  #. Set up a single partition::
+#. Set up a single partition::
 
       sudo fdisk /dev/sdb
       sudo mkfs.xfs /dev/sdb1
 
-  #. Edit ``/etc/fstab`` and add::
+#. Edit ``/etc/fstab`` and add::
 
       /dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
 
-  #. Create the mount point and the individualized links::
+#. Create the mount point and the individualized links::
 
       sudo mkdir /mnt/sdb1
       sudo mount /mnt/sdb1
@@ -116,14 +117,16 @@ another device when creating the VM, and follow these instructions:
       # **Make sure to include the trailing slash after /srv/$x/**
       for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
 
-     Note: For OpenSuse users, a user's primary group is `users`, so you have 2 options:
+   .. note::
+      For OpenSuse users, a user's primary group is ``users``, so you have 2 options:
 
-     * Change `${USER}:${USER}` to `${USER}:users` in all references of this guide; or
+      * Change ``${USER}:${USER}`` to ``${USER}:users`` in all references of this guide; or
       * Create a group for your username and add yourself to it::
 
          sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
 
-     Note: We create the mount points and mount the storage disk under
+   .. note::
+      We create the mount points and mount the storage disk under
       /mnt/sdb1. This disk will contain one directory per simulated swift node,
       each owned by the current swift user.
 
@@ -133,7 +136,7 @@ another device when creating the VM, and follow these instructions:
    exist. This prevents disk sync operations from writing to the root
    partition in the event a drive is unmounted.
 
-  #. Next, skip to :ref:`common-dev-section`.
+#. Next, skip to :ref:`common-dev-section`.
 
 
 .. _loopback-section:
@@ -144,7 +147,7 @@ Using a loopback device for storage
 If you want to use a loopback device instead of another partition, follow
 these instructions:
 
-  #. Create the file for the loopback device::
+#. Create the file for the loopback device::
 
       sudo mkdir /srv
       sudo truncate -s 1GB /srv/swift-disk
@@ -153,11 +156,11 @@ these instructions:
    Modify size specified in the ``truncate`` command to make a larger or
    smaller partition as needed.
 
-  #. Edit `/etc/fstab` and add::
+#. Edit `/etc/fstab` and add::
 
       /srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
 
-  #. Create the mount point and the individualized links::
+#. Create the mount point and the individualized links::
 
       sudo mkdir /mnt/sdb1
       sudo mount /mnt/sdb1
@@ -173,14 +176,16 @@ these instructions:
       # **Make sure to include the trailing slash after /srv/$x/**
       for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
 
-     Note: For OpenSuse users, a user's primary group is `users`, so you have 2 options:
+   .. note::
+      For OpenSuse users, a user's primary group is ``users``, so you have 2 options:
 
-     * Change `${USER}:${USER}` to `${USER}:users` in all references of this guide; or
+      * Change ``${USER}:${USER}`` to ``${USER}:users`` in all references of this guide; or
       * Create a group for your username and add yourself to it::
 
          sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
 
-     Note: We create the mount points and mount the loopback file under
+   .. note::
+      We create the mount points and mount the loopback file under
       /mnt/sdb1. This file will contain one directory per simulated swift node,
       each owned by the current swift user.
 
@@ -229,11 +234,11 @@ To persist this, edit and add the following to ``/etc/fstab``::
 Getting the code
 ----------------
 
-  #. Check out the python-swiftclient repo::
+#. Check out the python-swiftclient repo::
 
       cd $HOME; git clone https://github.com/openstack/python-swiftclient.git
 
-  #. Build a development installation of python-swiftclient::
+#. Build a development installation of python-swiftclient::
 
       cd $HOME/python-swiftclient; sudo python setup.py develop; cd -
 
@@ -242,23 +247,25 @@ Getting the code
 
       cd $HOME/python-swiftclient; sudo pip install -r requirements.txt; sudo python setup.py develop; cd -
 
-  #. Check out the swift repo::
+#. Check out the swift repo::
 
       git clone https://github.com/openstack/swift.git
 
-  #. Build a development installation of swift::
+#. Build a development installation of swift::
 
       cd $HOME/swift; sudo pip install --no-binary cryptography -r requirements.txt; sudo python setup.py develop; cd -
 
-     Note: Due to a difference in libssl.so naming in OpenSuse to other Linux distros the wheel/binary wont work so the
+   .. note::
-     cryptography must be built, thus the ``--no-binary cryptography``.
+      Due to a difference in how ``libssl.so`` is named in OpenSuse vs. other Linux distros the
+      wheel/binary won't work; thus we use ``--no-binary cryptography`` to build ``cryptography``
+      locally.
 
    Fedora 19 or later users might have to perform the following if development
    installation of swift fails::
 
       sudo pip install -U xattr
 
-  #. Install swift's test dependencies::
+#. Install swift's test dependencies::
 
       cd $HOME/swift; sudo pip install -r test-requirements.txt
 
@@ -266,7 +273,7 @@ Getting the code
 Setting up rsync
 ----------------
 
-  #. Create ``/etc/rsyncd.conf``::
+#. Create ``/etc/rsyncd.conf``::
 
       sudo cp $HOME/swift/doc/saio/rsyncd.conf /etc/
       sudo sed -i "s/<your-user-name>/${USER}/" /etc/rsyncd.conf
@@ -275,8 +282,9 @@ Setting up rsync
    that is copied and fixed up above:
 
    .. literalinclude:: /../saio/rsyncd.conf
+      :language: ini
 
-  #. On Ubuntu, edit the following line in ``/etc/default/rsync``::
+#. On Ubuntu, edit the following line in ``/etc/default/rsync``::
 
       RSYNC_ENABLE=true
 
@@ -288,7 +296,7 @@ Setting up rsync
 
    On OpenSuse, nothing needs to happen here.
 
-  #. On platforms with SELinux in ``Enforcing`` mode, either set to ``Permissive``::
+#. On platforms with SELinux in ``Enforcing`` mode, either set to ``Permissive``::
 
       sudo setenforce Permissive
 
@@ -296,7 +304,7 @@ Setting up rsync
 
       sudo setsebool -P rsync_full_access 1
 
-  #. Start the rsync daemon
+#. Start the rsync daemon
 
    * On Ubuntu 14.04, run::
 
@@ -322,7 +330,7 @@ Setting up rsync
 
       sudo service xinetd restart
 
-  #. Verify rsync is accepting connections for all servers::
+#. Verify rsync is accepting connections for all servers::
 
       rsync rsync://pub@localhost/
 
@@ -362,7 +370,7 @@ running, tokens cannot be validated, and accessing Swift becomes impossible.
 Optional: Setting up rsyslog for individual logging
 ---------------------------------------------------
 
-  #. Install the swift rsyslogd configuration::
+#. Install the swift rsyslogd configuration::
 
       sudo cp $HOME/swift/doc/saio/rsyslog.d/10-swift.conf /etc/rsyslog.d/
 
@@ -379,13 +387,14 @@ Optional: Setting up rsyslog for individual logging
    below:
 
    .. literalinclude:: /../saio/rsyslog.d/10-swift.conf
+      :language: ini
 
-  #. Edit ``/etc/rsyslog.conf`` and make the following change (usually in the
+#. Edit ``/etc/rsyslog.conf`` and make the following change (usually in the
    "GLOBAL DIRECTIVES" section)::
 
       $PrivDropToGroup adm
 
-  #. If using hourly logs (see above) perform::
+#. If using hourly logs (see above) perform::
 
       sudo mkdir -p /var/log/swift/hourly
 
@@ -393,7 +402,7 @@ Optional: Setting up rsyslog for individual logging
 
       sudo mkdir -p /var/log/swift
 
-  #. Setup the logging directory and start syslog:
+#. Setup the logging directory and start syslog:
 
    * On Ubuntu::
 
@@ -415,89 +424,106 @@ After performing the following steps, be sure to verify that Swift has access
 to resulting configuration files (sample configuration files are provided with
 all defaults in line-by-line comments).
 
-  #. Optionally remove an existing swift directory::
+#. Optionally remove an existing swift directory::
 
       sudo rm -rf /etc/swift
 
-  #. Populate the ``/etc/swift`` directory itself::
+#. Populate the ``/etc/swift`` directory itself::
 
       cd $HOME/swift/doc; sudo cp -r saio/swift /etc/swift; cd -
       sudo chown -R ${USER}:${USER} /etc/swift
 
-  #. Update ``<your-user-name>`` references in the Swift config files::
+#. Update ``<your-user-name>`` references in the Swift config files::
 
       find /etc/swift/ -name \*.conf | xargs sudo sed -i "s/<your-user-name>/${USER}/"
 
 The contents of the configuration files provided by executing the above
 commands are as follows:
 
-  #. ``/etc/swift/swift.conf``
+#. ``/etc/swift/swift.conf``
 
    .. literalinclude:: /../saio/swift/swift.conf
+      :language: ini
 
-  #. ``/etc/swift/proxy-server.conf``
+#. ``/etc/swift/proxy-server.conf``
 
    .. literalinclude:: /../saio/swift/proxy-server.conf
+      :language: ini
 
-  #. ``/etc/swift/object-expirer.conf``
+#. ``/etc/swift/object-expirer.conf``
 
    .. literalinclude:: /../saio/swift/object-expirer.conf
+      :language: ini
 
-  #. ``/etc/swift/container-reconciler.conf``
+#. ``/etc/swift/container-reconciler.conf``
 
    .. literalinclude:: /../saio/swift/container-reconciler.conf
+      :language: ini
 
-  #. ``/etc/swift/container-sync-realms.conf``
+#. ``/etc/swift/container-sync-realms.conf``
 
    .. literalinclude:: /../saio/swift/container-sync-realms.conf
+      :language: ini
 
-  #. ``/etc/swift/account-server/1.conf``
+#. ``/etc/swift/account-server/1.conf``
 
    .. literalinclude:: /../saio/swift/account-server/1.conf
+      :language: ini
 
-  #. ``/etc/swift/container-server/1.conf``
+#. ``/etc/swift/container-server/1.conf``
 
    .. literalinclude:: /../saio/swift/container-server/1.conf
+      :language: ini
 
-  #. ``/etc/swift/object-server/1.conf``
+#. ``/etc/swift/object-server/1.conf``
 
    .. literalinclude:: /../saio/swift/object-server/1.conf
+      :language: ini
 
-  #. ``/etc/swift/account-server/2.conf``
+#. ``/etc/swift/account-server/2.conf``
 
    .. literalinclude:: /../saio/swift/account-server/2.conf
+      :language: ini
 
-  #. ``/etc/swift/container-server/2.conf``
+#. ``/etc/swift/container-server/2.conf``
 
    .. literalinclude:: /../saio/swift/container-server/2.conf
+      :language: ini
 
-  #. ``/etc/swift/object-server/2.conf``
+#. ``/etc/swift/object-server/2.conf``
 
    .. literalinclude:: /../saio/swift/object-server/2.conf
+      :language: ini
 
-  #. ``/etc/swift/account-server/3.conf``
+#. ``/etc/swift/account-server/3.conf``
 
    .. literalinclude:: /../saio/swift/account-server/3.conf
+      :language: ini
 
-  #. ``/etc/swift/container-server/3.conf``
+#. ``/etc/swift/container-server/3.conf``
 
    .. literalinclude:: /../saio/swift/container-server/3.conf
+      :language: ini
 
-  #. ``/etc/swift/object-server/3.conf``
+#. ``/etc/swift/object-server/3.conf``
 
    .. literalinclude:: /../saio/swift/object-server/3.conf
+      :language: ini
 
-  #. ``/etc/swift/account-server/4.conf``
+#. ``/etc/swift/account-server/4.conf``
 
    .. literalinclude:: /../saio/swift/account-server/4.conf
+      :language: ini
 
-  #. ``/etc/swift/container-server/4.conf``
+#. ``/etc/swift/container-server/4.conf``
 
    .. literalinclude:: /../saio/swift/container-server/4.conf
+      :language: ini
 
-  #. ``/etc/swift/object-server/4.conf``
+#. ``/etc/swift/object-server/4.conf``
 
    .. literalinclude:: /../saio/swift/object-server/4.conf
+      :language: ini
 
 .. _setup_scripts:
 
@@ -505,17 +531,18 @@ commands are as follows:
 Setting up scripts for running Swift
 ------------------------------------
 
-  #. Copy the SAIO scripts for resetting the environment::
+#. Copy the SAIO scripts for resetting the environment::
 
       mkdir -p $HOME/bin
       cd $HOME/swift/doc; cp saio/bin/* $HOME/bin; cd -
       chmod +x $HOME/bin/*
 
-  #. Edit the ``$HOME/bin/resetswift`` script
+#. Edit the ``$HOME/bin/resetswift`` script
 
    The template ``resetswift`` script looks like the following:
 
    .. literalinclude:: /../saio/bin/resetswift
+      :language: bash
 
    If you are using a loopback device add an environment var to
    substitute ``/dev/sdb1`` with ``/srv/swift-disk``::
@@ -528,39 +555,44 @@ Setting up scripts for running Swift
       sed -i "/find \/var\/log\/swift/d" $HOME/bin/resetswift
 
 
-  #. Install the sample configuration file for running tests::
+#. Install the sample configuration file for running tests::
 
       cp $HOME/swift/test/sample.conf /etc/swift/test.conf
 
    The template ``test.conf`` looks like the following:
 
    .. literalinclude:: /../../test/sample.conf
+      :language: ini
 
-  #. Add an environment variable for running tests below::
+#. Add an environment variable for running tests below::
 
       echo "export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf" >> $HOME/.bashrc
 
-  #. Be sure that your ``PATH`` includes the ``bin`` directory::
+#. Be sure that your ``PATH`` includes the ``bin`` directory::
 
       echo "export PATH=${PATH}:$HOME/bin" >> $HOME/.bashrc
 
-  #. Source the above environment variables into your current environment::
+#. Source the above environment variables into your current environment::
 
       . $HOME/.bashrc
 
-  #. Construct the initial rings using the provided script::
+#. Construct the initial rings using the provided script::
 
       remakerings
 
    The ``remakerings`` script looks like the following:
 
    .. literalinclude:: /../saio/bin/remakerings
+      :language: bash
 
    You can expect the output from this command to produce the following.  Note
    that 3 object rings are created in order to test storage policies and EC in
    the SAIO environment.  The EC ring is the only one with all 8 devices.
    There are also two replication rings, one for 3x replication and another
-     for 2x replication, but those rings only use 4 devices::
+   for 2x replication, but those rings only use 4 devices:
+
+
+   .. code-block:: console
 
       Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
       Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
@@ -593,15 +625,15 @@ Setting up scripts for running Swift
       Reassigned 3072 (300.00%) partitions. Balance is now 0.00.  Dispersion is now 0.00
 
 
-  #. Read more about Storage Policies and your SAIO :doc:`policies_saio`
+#. Read more about Storage Policies and your SAIO :doc:`policies_saio`
 
-  #. Verify the unit tests run::
+#. Verify the unit tests run::
 
       $HOME/swift/.unittests
 
    Note that the unit tests do not require any swift daemons running.
 
-  #. Start the "main" Swift daemon processes (proxy, account, container, and
+#. Start the "main" Swift daemon processes (proxy, account, container, and
    object)::
 
       startmain
@@ -612,27 +644,28 @@ Setting up scripts for running Swift
    The ``startmain`` script looks like the following:
 
    .. literalinclude:: /../saio/bin/startmain
+      :language: bash
 
-  #. Get an ``X-Storage-Url`` and ``X-Auth-Token``::
+#. Get an ``X-Storage-Url`` and ``X-Auth-Token``::
 
       curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
 
-  #. Check that you can ``GET`` account::
+#. Check that you can ``GET`` account::
 
       curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>
 
-  #. Check that ``swift`` command provided by the python-swiftclient package works::
+#. Check that ``swift`` command provided by the python-swiftclient package works::
 
       swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
 
-  #. Verify the functional tests run::
+#. Verify the functional tests run::
 
       $HOME/swift/.functests
 
    (Note: functional tests will first delete everything in the configured
    accounts.)
 
-  #. Verify the probe tests run::
+#. Verify the probe tests run::
 
       $HOME/swift/.probetests
 

swift/common/bufferedhttp.py

@@ -178,6 +178,14 @@ class BufferedHTTPConnection(HTTPConnection):
         return ret
 
     def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0):
+        '''Send a request to the server.
+
+        :param method: specifies an HTTP request method, e.g. 'GET'.
+        :param url: specifies the object being requested, e.g. '/index.html'.
+        :param skip_host: if True does not add automatically a 'Host:' header
+        :param skip_accept_encoding: if True does not add automatically an
+           'Accept-Encoding:' header
+        '''
         self._method = method
         self._path = url
         return HTTPConnection.putrequest(self, method, url, skip_host,

swift/common/middleware/s3api/controllers/s3_acl.py

@@ -13,7 +13,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from urllib import quote
+from six.moves.urllib.parse import quote
 from swift.common.utils import public
 
 from swift.common.middleware.s3api.controllers.base import Controller