Merge "docs: clean up SAIO formatting"
This commit is contained in:
Zuul
Gerrit Code Review
commit
f9329ca5ff
@ -76,9 +76,10 @@ Installing dependencies
|
||||
python2-netifaces python2-pip python2-dnspython \
|
||||
python2-mock
|
||||
|
||||
Note: This installs necessary system dependencies and *most* of the python
|
||||
dependencies. Later in the process setuptools/distribute or pip will install
|
||||
and/or upgrade packages.
|
||||
.. note::
|
||||
This installs necessary system dependencies and *most* of the python
|
||||
dependencies. Later in the process setuptools/distribute or pip will install
|
||||
and/or upgrade packages.
|
||||
|
||||
Next, choose either :ref:`partition-section` or :ref:`loopback-section`.
|
||||
|
||||
@ -90,50 +91,52 @@ Using a partition for storage
|
||||
If you are going to use a separate partition for Swift data, be sure to add
|
||||
another device when creating the VM, and follow these instructions:
|
||||
|
||||
#. Set up a single partition::
|
||||
#. Set up a single partition::
|
||||
|
||||
sudo fdisk /dev/sdb
|
||||
sudo mkfs.xfs /dev/sdb1
|
||||
sudo fdisk /dev/sdb
|
||||
sudo mkfs.xfs /dev/sdb1
|
||||
|
||||
#. Edit ``/etc/fstab`` and add::
|
||||
#. Edit ``/etc/fstab`` and add::
|
||||
|
||||
/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
|
||||
/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
|
||||
|
||||
#. Create the mount point and the individualized links::
|
||||
#. Create the mount point and the individualized links::
|
||||
|
||||
sudo mkdir /mnt/sdb1
|
||||
sudo mount /mnt/sdb1
|
||||
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
|
||||
sudo chown ${USER}:${USER} /mnt/sdb1/*
|
||||
sudo mkdir /srv
|
||||
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done
|
||||
sudo mkdir -p /srv/1/node/sdb1 /srv/1/node/sdb5 \
|
||||
/srv/2/node/sdb2 /srv/2/node/sdb6 \
|
||||
/srv/3/node/sdb3 /srv/3/node/sdb7 \
|
||||
/srv/4/node/sdb4 /srv/4/node/sdb8 \
|
||||
/var/run/swift
|
||||
sudo chown -R ${USER}:${USER} /var/run/swift
|
||||
# **Make sure to include the trailing slash after /srv/$x/**
|
||||
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
|
||||
sudo mkdir /mnt/sdb1
|
||||
sudo mount /mnt/sdb1
|
||||
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
|
||||
sudo chown ${USER}:${USER} /mnt/sdb1/*
|
||||
sudo mkdir /srv
|
||||
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done
|
||||
sudo mkdir -p /srv/1/node/sdb1 /srv/1/node/sdb5 \
|
||||
/srv/2/node/sdb2 /srv/2/node/sdb6 \
|
||||
/srv/3/node/sdb3 /srv/3/node/sdb7 \
|
||||
/srv/4/node/sdb4 /srv/4/node/sdb8 \
|
||||
/var/run/swift
|
||||
sudo chown -R ${USER}:${USER} /var/run/swift
|
||||
# **Make sure to include the trailing slash after /srv/$x/**
|
||||
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
|
||||
|
||||
Note: For OpenSuse users, a user's primary group is `users`, so you have 2 options:
|
||||
.. note::
|
||||
For OpenSuse users, a user's primary group is ``users``, so you have 2 options:
|
||||
|
||||
* Change `${USER}:${USER}` to `${USER}:users` in all references of this guide; or
|
||||
* Create a group for your username and add yourself to it::
|
||||
* Change ``${USER}:${USER}`` to ``${USER}:users`` in all references of this guide; or
|
||||
* Create a group for your username and add yourself to it::
|
||||
|
||||
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
|
||||
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
|
||||
|
||||
Note: We create the mount points and mount the storage disk under
|
||||
/mnt/sdb1. This disk will contain one directory per simulated swift node,
|
||||
each owned by the current swift user.
|
||||
.. note::
|
||||
We create the mount points and mount the storage disk under
|
||||
/mnt/sdb1. This disk will contain one directory per simulated swift node,
|
||||
each owned by the current swift user.
|
||||
|
||||
We then create symlinks to these directories under /srv.
|
||||
If the disk sdb is unmounted, files will not be written under
|
||||
/srv/\*, because the symbolic link destination /mnt/sdb1/* will not
|
||||
exist. This prevents disk sync operations from writing to the root
|
||||
partition in the event a drive is unmounted.
|
||||
We then create symlinks to these directories under /srv.
|
||||
If the disk sdb is unmounted, files will not be written under
|
||||
/srv/\*, because the symbolic link destination /mnt/sdb1/* will not
|
||||
exist. This prevents disk sync operations from writing to the root
|
||||
partition in the event a drive is unmounted.
|
||||
|
||||
#. Next, skip to :ref:`common-dev-section`.
|
||||
#. Next, skip to :ref:`common-dev-section`.
|
||||
|
||||
|
||||
.. _loopback-section:
|
||||
@ -144,51 +147,53 @@ Using a loopback device for storage
|
||||
If you want to use a loopback device instead of another partition, follow
|
||||
these instructions:
|
||||
|
||||
#. Create the file for the loopback device::
|
||||
#. Create the file for the loopback device::
|
||||
|
||||
sudo mkdir /srv
|
||||
sudo truncate -s 1GB /srv/swift-disk
|
||||
sudo mkfs.xfs /srv/swift-disk
|
||||
sudo mkdir /srv
|
||||
sudo truncate -s 1GB /srv/swift-disk
|
||||
sudo mkfs.xfs /srv/swift-disk
|
||||
|
||||
Modify size specified in the ``truncate`` command to make a larger or
|
||||
smaller partition as needed.
|
||||
Modify size specified in the ``truncate`` command to make a larger or
|
||||
smaller partition as needed.
|
||||
|
||||
#. Edit `/etc/fstab` and add::
|
||||
#. Edit `/etc/fstab` and add::
|
||||
|
||||
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
|
||||
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
|
||||
|
||||
#. Create the mount point and the individualized links::
|
||||
#. Create the mount point and the individualized links::
|
||||
|
||||
sudo mkdir /mnt/sdb1
|
||||
sudo mount /mnt/sdb1
|
||||
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
|
||||
sudo chown ${USER}:${USER} /mnt/sdb1/*
|
||||
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done
|
||||
sudo mkdir -p /srv/1/node/sdb1 /srv/1/node/sdb5 \
|
||||
/srv/2/node/sdb2 /srv/2/node/sdb6 \
|
||||
/srv/3/node/sdb3 /srv/3/node/sdb7 \
|
||||
/srv/4/node/sdb4 /srv/4/node/sdb8 \
|
||||
/var/run/swift
|
||||
sudo chown -R ${USER}:${USER} /var/run/swift
|
||||
# **Make sure to include the trailing slash after /srv/$x/**
|
||||
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
|
||||
sudo mkdir /mnt/sdb1
|
||||
sudo mount /mnt/sdb1
|
||||
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
|
||||
sudo chown ${USER}:${USER} /mnt/sdb1/*
|
||||
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done
|
||||
sudo mkdir -p /srv/1/node/sdb1 /srv/1/node/sdb5 \
|
||||
/srv/2/node/sdb2 /srv/2/node/sdb6 \
|
||||
/srv/3/node/sdb3 /srv/3/node/sdb7 \
|
||||
/srv/4/node/sdb4 /srv/4/node/sdb8 \
|
||||
/var/run/swift
|
||||
sudo chown -R ${USER}:${USER} /var/run/swift
|
||||
# **Make sure to include the trailing slash after /srv/$x/**
|
||||
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
|
||||
|
||||
Note: For OpenSuse users, a user's primary group is `users`, so you have 2 options:
|
||||
.. note::
|
||||
For OpenSuse users, a user's primary group is ``users``, so you have 2 options:
|
||||
|
||||
* Change `${USER}:${USER}` to `${USER}:users` in all references of this guide; or
|
||||
* Create a group for your username and add yourself to it::
|
||||
* Change ``${USER}:${USER}`` to ``${USER}:users`` in all references of this guide; or
|
||||
* Create a group for your username and add yourself to it::
|
||||
|
||||
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
|
||||
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
|
||||
|
||||
Note: We create the mount points and mount the loopback file under
|
||||
/mnt/sdb1. This file will contain one directory per simulated swift node,
|
||||
each owned by the current swift user.
|
||||
.. note::
|
||||
We create the mount points and mount the loopback file under
|
||||
/mnt/sdb1. This file will contain one directory per simulated swift node,
|
||||
each owned by the current swift user.
|
||||
|
||||
We then create symlinks to these directories under /srv.
|
||||
If the loopback file is unmounted, files will not be written under
|
||||
/srv/\*, because the symbolic link destination /mnt/sdb1/* will not
|
||||
exist. This prevents disk sync operations from writing to the root
|
||||
partition in the event a drive is unmounted.
|
||||
We then create symlinks to these directories under /srv.
|
||||
If the loopback file is unmounted, files will not be written under
|
||||
/srv/\*, because the symbolic link destination /mnt/sdb1/* will not
|
||||
exist. This prevents disk sync operations from writing to the root
|
||||
partition in the event a drive is unmounted.
|
||||
|
||||
.. _common-dev-section:
|
||||
|
||||
@ -229,117 +234,120 @@ To persist this, edit and add the following to ``/etc/fstab``::
|
||||
Getting the code
|
||||
----------------
|
||||
|
||||
#. Check out the python-swiftclient repo::
|
||||
#. Check out the python-swiftclient repo::
|
||||
|
||||
cd $HOME; git clone https://github.com/openstack/python-swiftclient.git
|
||||
cd $HOME; git clone https://github.com/openstack/python-swiftclient.git
|
||||
|
||||
#. Build a development installation of python-swiftclient::
|
||||
#. Build a development installation of python-swiftclient::
|
||||
|
||||
cd $HOME/python-swiftclient; sudo python setup.py develop; cd -
|
||||
cd $HOME/python-swiftclient; sudo python setup.py develop; cd -
|
||||
|
||||
Ubuntu 12.04 users need to install python-swiftclient's dependencies before the installation of
|
||||
python-swiftclient. This is due to a bug in an older version of setup tools::
|
||||
Ubuntu 12.04 users need to install python-swiftclient's dependencies before the installation of
|
||||
python-swiftclient. This is due to a bug in an older version of setup tools::
|
||||
|
||||
cd $HOME/python-swiftclient; sudo pip install -r requirements.txt; sudo python setup.py develop; cd -
|
||||
cd $HOME/python-swiftclient; sudo pip install -r requirements.txt; sudo python setup.py develop; cd -
|
||||
|
||||
#. Check out the swift repo::
|
||||
#. Check out the swift repo::
|
||||
|
||||
git clone https://github.com/openstack/swift.git
|
||||
git clone https://github.com/openstack/swift.git
|
||||
|
||||
#. Build a development installation of swift::
|
||||
#. Build a development installation of swift::
|
||||
|
||||
cd $HOME/swift; sudo pip install --no-binary cryptography -r requirements.txt; sudo python setup.py develop; cd -
|
||||
cd $HOME/swift; sudo pip install --no-binary cryptography -r requirements.txt; sudo python setup.py develop; cd -
|
||||
|
||||
Note: Due to a difference in libssl.so naming in OpenSuse to other Linux distros the wheel/binary wont work so the
|
||||
cryptography must be built, thus the ``--no-binary cryptography``.
|
||||
.. note::
|
||||
Due to a difference in how ``libssl.so`` is named in OpenSuse vs. other Linux distros the
|
||||
wheel/binary won't work; thus we use ``--no-binary cryptography`` to build ``cryptography``
|
||||
locally.
|
||||
|
||||
Fedora 19 or later users might have to perform the following if development
|
||||
installation of swift fails::
|
||||
Fedora 19 or later users might have to perform the following if development
|
||||
installation of swift fails::
|
||||
|
||||
sudo pip install -U xattr
|
||||
sudo pip install -U xattr
|
||||
|
||||
#. Install swift's test dependencies::
|
||||
#. Install swift's test dependencies::
|
||||
|
||||
cd $HOME/swift; sudo pip install -r test-requirements.txt
|
||||
cd $HOME/swift; sudo pip install -r test-requirements.txt
|
||||
|
||||
----------------
|
||||
Setting up rsync
|
||||
----------------
|
||||
|
||||
#. Create ``/etc/rsyncd.conf``::
|
||||
#. Create ``/etc/rsyncd.conf``::
|
||||
|
||||
sudo cp $HOME/swift/doc/saio/rsyncd.conf /etc/
|
||||
sudo sed -i "s/<your-user-name>/${USER}/" /etc/rsyncd.conf
|
||||
sudo cp $HOME/swift/doc/saio/rsyncd.conf /etc/
|
||||
sudo sed -i "s/<your-user-name>/${USER}/" /etc/rsyncd.conf
|
||||
|
||||
Here is the default ``rsyncd.conf`` file contents maintained in the repo
|
||||
that is copied and fixed up above:
|
||||
Here is the default ``rsyncd.conf`` file contents maintained in the repo
|
||||
that is copied and fixed up above:
|
||||
|
||||
.. literalinclude:: /../saio/rsyncd.conf
|
||||
.. literalinclude:: /../saio/rsyncd.conf
|
||||
:language: ini
|
||||
|
||||
#. On Ubuntu, edit the following line in ``/etc/default/rsync``::
|
||||
#. On Ubuntu, edit the following line in ``/etc/default/rsync``::
|
||||
|
||||
RSYNC_ENABLE=true
|
||||
RSYNC_ENABLE=true
|
||||
|
||||
On Fedora, edit the following line in ``/etc/xinetd.d/rsync``::
|
||||
On Fedora, edit the following line in ``/etc/xinetd.d/rsync``::
|
||||
|
||||
disable = no
|
||||
disable = no
|
||||
|
||||
One might have to create the above files to perform the edits.
|
||||
One might have to create the above files to perform the edits.
|
||||
|
||||
On OpenSuse, nothing needs to happen here.
|
||||
On OpenSuse, nothing needs to happen here.
|
||||
|
||||
#. On platforms with SELinux in ``Enforcing`` mode, either set to ``Permissive``::
|
||||
#. On platforms with SELinux in ``Enforcing`` mode, either set to ``Permissive``::
|
||||
|
||||
sudo setenforce Permissive
|
||||
sudo setenforce Permissive
|
||||
|
||||
Or just allow rsync full access::
|
||||
Or just allow rsync full access::
|
||||
|
||||
sudo setsebool -P rsync_full_access 1
|
||||
sudo setsebool -P rsync_full_access 1
|
||||
|
||||
#. Start the rsync daemon
|
||||
#. Start the rsync daemon
|
||||
|
||||
* On Ubuntu 14.04, run::
|
||||
* On Ubuntu 14.04, run::
|
||||
|
||||
sudo service rsync restart
|
||||
sudo service rsync restart
|
||||
|
||||
* On Ubuntu 16.04, run::
|
||||
* On Ubuntu 16.04, run::
|
||||
|
||||
sudo systemctl enable rsync
|
||||
sudo systemctl start rsync
|
||||
sudo systemctl enable rsync
|
||||
sudo systemctl start rsync
|
||||
|
||||
* On Fedora, run::
|
||||
* On Fedora, run::
|
||||
|
||||
sudo systemctl restart xinetd.service
|
||||
sudo systemctl enable rsyncd.service
|
||||
sudo systemctl start rsyncd.service
|
||||
sudo systemctl restart xinetd.service
|
||||
sudo systemctl enable rsyncd.service
|
||||
sudo systemctl start rsyncd.service
|
||||
|
||||
* On OpenSuse, run::
|
||||
* On OpenSuse, run::
|
||||
|
||||
sudo systemctl enable rsyncd.service
|
||||
sudo systemctl start rsyncd.service
|
||||
sudo systemctl enable rsyncd.service
|
||||
sudo systemctl start rsyncd.service
|
||||
|
||||
* On other xinetd based systems simply run::
|
||||
* On other xinetd based systems simply run::
|
||||
|
||||
sudo service xinetd restart
|
||||
sudo service xinetd restart
|
||||
|
||||
#. Verify rsync is accepting connections for all servers::
|
||||
#. Verify rsync is accepting connections for all servers::
|
||||
|
||||
rsync rsync://pub@localhost/
|
||||
rsync rsync://pub@localhost/
|
||||
|
||||
You should see the following output from the above command::
|
||||
You should see the following output from the above command::
|
||||
|
||||
account6012
|
||||
account6022
|
||||
account6032
|
||||
account6042
|
||||
container6011
|
||||
container6021
|
||||
container6031
|
||||
container6041
|
||||
object6010
|
||||
object6020
|
||||
object6030
|
||||
object6040
|
||||
account6012
|
||||
account6022
|
||||
account6032
|
||||
account6042
|
||||
container6011
|
||||
container6021
|
||||
container6031
|
||||
container6041
|
||||
object6010
|
||||
object6020
|
||||
object6030
|
||||
object6040
|
||||
|
||||
------------------
|
||||
Starting memcached
|
||||
@ -362,50 +370,51 @@ running, tokens cannot be validated, and accessing Swift becomes impossible.
|
||||
Optional: Setting up rsyslog for individual logging
|
||||
---------------------------------------------------
|
||||
|
||||
#. Install the swift rsyslogd configuration::
|
||||
#. Install the swift rsyslogd configuration::
|
||||
|
||||
sudo cp $HOME/swift/doc/saio/rsyslog.d/10-swift.conf /etc/rsyslog.d/
|
||||
sudo cp $HOME/swift/doc/saio/rsyslog.d/10-swift.conf /etc/rsyslog.d/
|
||||
|
||||
Note: OpenSuse may have the systemd logger installed, so if you want this
|
||||
to work, you need to install rsyslog::
|
||||
Note: OpenSuse may have the systemd logger installed, so if you want this
|
||||
to work, you need to install rsyslog::
|
||||
|
||||
sudo zypper install rsyslog
|
||||
sudo systemctl start rsyslog.service
|
||||
sudo systemctl enable rsyslog.service
|
||||
sudo zypper install rsyslog
|
||||
sudo systemctl start rsyslog.service
|
||||
sudo systemctl enable rsyslog.service
|
||||
|
||||
Be sure to review that conf file to determine if you want all the logs
|
||||
in one file vs. all the logs separated out, and if you want hourly logs
|
||||
for stats processing. For convenience, we provide its default contents
|
||||
below:
|
||||
Be sure to review that conf file to determine if you want all the logs
|
||||
in one file vs. all the logs separated out, and if you want hourly logs
|
||||
for stats processing. For convenience, we provide its default contents
|
||||
below:
|
||||
|
||||
.. literalinclude:: /../saio/rsyslog.d/10-swift.conf
|
||||
.. literalinclude:: /../saio/rsyslog.d/10-swift.conf
|
||||
:language: ini
|
||||
|
||||
#. Edit ``/etc/rsyslog.conf`` and make the following change (usually in the
|
||||
"GLOBAL DIRECTIVES" section)::
|
||||
#. Edit ``/etc/rsyslog.conf`` and make the following change (usually in the
|
||||
"GLOBAL DIRECTIVES" section)::
|
||||
|
||||
$PrivDropToGroup adm
|
||||
$PrivDropToGroup adm
|
||||
|
||||
#. If using hourly logs (see above) perform::
|
||||
#. If using hourly logs (see above) perform::
|
||||
|
||||
sudo mkdir -p /var/log/swift/hourly
|
||||
sudo mkdir -p /var/log/swift/hourly
|
||||
|
||||
Otherwise perform::
|
||||
Otherwise perform::
|
||||
|
||||
sudo mkdir -p /var/log/swift
|
||||
sudo mkdir -p /var/log/swift
|
||||
|
||||
#. Setup the logging directory and start syslog:
|
||||
#. Setup the logging directory and start syslog:
|
||||
|
||||
* On Ubuntu::
|
||||
* On Ubuntu::
|
||||
|
||||
sudo chown -R syslog.adm /var/log/swift
|
||||
sudo chmod -R g+w /var/log/swift
|
||||
sudo service rsyslog restart
|
||||
sudo chown -R syslog.adm /var/log/swift
|
||||
sudo chmod -R g+w /var/log/swift
|
||||
sudo service rsyslog restart
|
||||
|
||||
* On Fedora and OpenSuse::
|
||||
* On Fedora and OpenSuse::
|
||||
|
||||
sudo chown -R root:adm /var/log/swift
|
||||
sudo chmod -R g+w /var/log/swift
|
||||
sudo systemctl restart rsyslog.service
|
||||
sudo chown -R root:adm /var/log/swift
|
||||
sudo chmod -R g+w /var/log/swift
|
||||
sudo systemctl restart rsyslog.service
|
||||
|
||||
---------------------
|
||||
Configuring each node
|
||||
@ -415,89 +424,106 @@ After performing the following steps, be sure to verify that Swift has access
|
||||
to resulting configuration files (sample configuration files are provided with
|
||||
all defaults in line-by-line comments).
|
||||
|
||||
#. Optionally remove an existing swift directory::
|
||||
#. Optionally remove an existing swift directory::
|
||||
|
||||
sudo rm -rf /etc/swift
|
||||
sudo rm -rf /etc/swift
|
||||
|
||||
#. Populate the ``/etc/swift`` directory itself::
|
||||
#. Populate the ``/etc/swift`` directory itself::
|
||||
|
||||
cd $HOME/swift/doc; sudo cp -r saio/swift /etc/swift; cd -
|
||||
sudo chown -R ${USER}:${USER} /etc/swift
|
||||
cd $HOME/swift/doc; sudo cp -r saio/swift /etc/swift; cd -
|
||||
sudo chown -R ${USER}:${USER} /etc/swift
|
||||
|
||||
#. Update ``<your-user-name>`` references in the Swift config files::
|
||||
#. Update ``<your-user-name>`` references in the Swift config files::
|
||||
|
||||
find /etc/swift/ -name \*.conf | xargs sudo sed -i "s/<your-user-name>/${USER}/"
|
||||
find /etc/swift/ -name \*.conf | xargs sudo sed -i "s/<your-user-name>/${USER}/"
|
||||
|
||||
The contents of the configuration files provided by executing the above
|
||||
commands are as follows:
|
||||
|
||||
#. ``/etc/swift/swift.conf``
|
||||
#. ``/etc/swift/swift.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/swift.conf
|
||||
.. literalinclude:: /../saio/swift/swift.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/proxy-server.conf``
|
||||
#. ``/etc/swift/proxy-server.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/proxy-server.conf
|
||||
.. literalinclude:: /../saio/swift/proxy-server.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/object-expirer.conf``
|
||||
#. ``/etc/swift/object-expirer.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/object-expirer.conf
|
||||
.. literalinclude:: /../saio/swift/object-expirer.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/container-reconciler.conf``
|
||||
#. ``/etc/swift/container-reconciler.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/container-reconciler.conf
|
||||
.. literalinclude:: /../saio/swift/container-reconciler.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/container-sync-realms.conf``
|
||||
#. ``/etc/swift/container-sync-realms.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/container-sync-realms.conf
|
||||
.. literalinclude:: /../saio/swift/container-sync-realms.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/account-server/1.conf``
|
||||
#. ``/etc/swift/account-server/1.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/account-server/1.conf
|
||||
.. literalinclude:: /../saio/swift/account-server/1.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/container-server/1.conf``
|
||||
#. ``/etc/swift/container-server/1.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/container-server/1.conf
|
||||
.. literalinclude:: /../saio/swift/container-server/1.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/object-server/1.conf``
|
||||
#. ``/etc/swift/object-server/1.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/object-server/1.conf
|
||||
.. literalinclude:: /../saio/swift/object-server/1.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/account-server/2.conf``
|
||||
#. ``/etc/swift/account-server/2.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/account-server/2.conf
|
||||
.. literalinclude:: /../saio/swift/account-server/2.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/container-server/2.conf``
|
||||
#. ``/etc/swift/container-server/2.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/container-server/2.conf
|
||||
.. literalinclude:: /../saio/swift/container-server/2.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/object-server/2.conf``
|
||||
#. ``/etc/swift/object-server/2.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/object-server/2.conf
|
||||
.. literalinclude:: /../saio/swift/object-server/2.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/account-server/3.conf``
|
||||
#. ``/etc/swift/account-server/3.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/account-server/3.conf
|
||||
.. literalinclude:: /../saio/swift/account-server/3.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/container-server/3.conf``
|
||||
#. ``/etc/swift/container-server/3.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/container-server/3.conf
|
||||
.. literalinclude:: /../saio/swift/container-server/3.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/object-server/3.conf``
|
||||
#. ``/etc/swift/object-server/3.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/object-server/3.conf
|
||||
.. literalinclude:: /../saio/swift/object-server/3.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/account-server/4.conf``
|
||||
#. ``/etc/swift/account-server/4.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/account-server/4.conf
|
||||
.. literalinclude:: /../saio/swift/account-server/4.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/container-server/4.conf``
|
||||
#. ``/etc/swift/container-server/4.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/container-server/4.conf
|
||||
.. literalinclude:: /../saio/swift/container-server/4.conf
|
||||
:language: ini
|
||||
|
||||
#. ``/etc/swift/object-server/4.conf``
|
||||
#. ``/etc/swift/object-server/4.conf``
|
||||
|
||||
.. literalinclude:: /../saio/swift/object-server/4.conf
|
||||
.. literalinclude:: /../saio/swift/object-server/4.conf
|
||||
:language: ini
|
||||
|
||||
.. _setup_scripts:
|
||||
|
||||
@ -505,139 +531,146 @@ commands are as follows:
|
||||
Setting up scripts for running Swift
|
||||
------------------------------------
|
||||
|
||||
#. Copy the SAIO scripts for resetting the environment::
|
||||
#. Copy the SAIO scripts for resetting the environment::
|
||||
|
||||
mkdir -p $HOME/bin
|
||||
cd $HOME/swift/doc; cp saio/bin/* $HOME/bin; cd -
|
||||
chmod +x $HOME/bin/*
|
||||
mkdir -p $HOME/bin
|
||||
cd $HOME/swift/doc; cp saio/bin/* $HOME/bin; cd -
|
||||
chmod +x $HOME/bin/*
|
||||
|
||||
#. Edit the ``$HOME/bin/resetswift`` script
|
||||
#. Edit the ``$HOME/bin/resetswift`` script
|
||||
|
||||
The template ``resetswift`` script looks like the following:
|
||||
The template ``resetswift`` script looks like the following:
|
||||
|
||||
.. literalinclude:: /../saio/bin/resetswift
|
||||
.. literalinclude:: /../saio/bin/resetswift
|
||||
:language: bash
|
||||
|
||||
If you are using a loopback device add an environment var to
|
||||
substitute ``/dev/sdb1`` with ``/srv/swift-disk``::
|
||||
If you are using a loopback device add an environment var to
|
||||
substitute ``/dev/sdb1`` with ``/srv/swift-disk``::
|
||||
|
||||
echo "export SAIO_BLOCK_DEVICE=/srv/swift-disk" >> $HOME/.bashrc
|
||||
echo "export SAIO_BLOCK_DEVICE=/srv/swift-disk" >> $HOME/.bashrc
|
||||
|
||||
If you did not set up rsyslog for individual logging, remove the ``find
|
||||
/var/log/swift...`` line::
|
||||
If you did not set up rsyslog for individual logging, remove the ``find
|
||||
/var/log/swift...`` line::
|
||||
|
||||
sed -i "/find \/var\/log\/swift/d" $HOME/bin/resetswift
|
||||
sed -i "/find \/var\/log\/swift/d" $HOME/bin/resetswift
|
||||
|
||||
|
||||
#. Install the sample configuration file for running tests::
|
||||
#. Install the sample configuration file for running tests::
|
||||
|
||||
cp $HOME/swift/test/sample.conf /etc/swift/test.conf
|
||||
cp $HOME/swift/test/sample.conf /etc/swift/test.conf
|
||||
|
||||
The template ``test.conf`` looks like the following:
|
||||
The template ``test.conf`` looks like the following:
|
||||
|
||||
.. literalinclude:: /../../test/sample.conf
|
||||
.. literalinclude:: /../../test/sample.conf
|
||||
:language: ini
|
||||
|
||||
#. Add an environment variable for running tests below::
|
||||
#. Add an environment variable for running tests below::
|
||||
|
||||
echo "export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf" >> $HOME/.bashrc
|
||||
echo "export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf" >> $HOME/.bashrc
|
||||
|
||||
#. Be sure that your ``PATH`` includes the ``bin`` directory::
|
||||
#. Be sure that your ``PATH`` includes the ``bin`` directory::
|
||||
|
||||
echo "export PATH=${PATH}:$HOME/bin" >> $HOME/.bashrc
|
||||
echo "export PATH=${PATH}:$HOME/bin" >> $HOME/.bashrc
|
||||
|
||||
#. Source the above environment variables into your current environment::
|
||||
#. Source the above environment variables into your current environment::
|
||||
|
||||
. $HOME/.bashrc
|
||||
. $HOME/.bashrc
|
||||
|
||||
#. Construct the initial rings using the provided script::
|
||||
#. Construct the initial rings using the provided script::
|
||||
|
||||
remakerings
|
||||
remakerings
|
||||
|
||||
The ``remakerings`` script looks like the following:
|
||||
The ``remakerings`` script looks like the following:
|
||||
|
||||
.. literalinclude:: /../saio/bin/remakerings
|
||||
.. literalinclude:: /../saio/bin/remakerings
|
||||
:language: bash
|
||||
|
||||
You can expect the output from this command to produce the following. Note
|
||||
that 3 object rings are created in order to test storage policies and EC in
|
||||
the SAIO environment. The EC ring is the only one with all 8 devices.
|
||||
There are also two replication rings, one for 3x replication and another
|
||||
for 2x replication, but those rings only use 4 devices::
|
||||
|
||||
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
|
||||
Device d2r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 2
|
||||
Device d3r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 3
|
||||
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
|
||||
Device d2r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 2
|
||||
Device d3r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 3
|
||||
Reassigned 2048 (200.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb5_"" with 1.0 weight got id 1
|
||||
Device d2r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 2
|
||||
Device d3r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb6_"" with 1.0 weight got id 3
|
||||
Device d4r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 4
|
||||
Device d5r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb7_"" with 1.0 weight got id 5
|
||||
Device d6r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 6
|
||||
Device d7r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb8_"" with 1.0 weight got id 7
|
||||
Reassigned 6144 (600.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
Device d0r1z1-127.0.0.1:6011R127.0.0.1:6011/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z2-127.0.0.2:6021R127.0.0.2:6021/sdb2_"" with 1.0 weight got id 1
|
||||
Device d2r1z3-127.0.0.3:6031R127.0.0.3:6031/sdb3_"" with 1.0 weight got id 2
|
||||
Device d3r1z4-127.0.0.4:6041R127.0.0.4:6041/sdb4_"" with 1.0 weight got id 3
|
||||
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
Device d0r1z1-127.0.0.1:6012R127.0.0.1:6012/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z2-127.0.0.2:6022R127.0.0.2:6022/sdb2_"" with 1.0 weight got id 1
|
||||
Device d2r1z3-127.0.0.3:6032R127.0.0.3:6032/sdb3_"" with 1.0 weight got id 2
|
||||
Device d3r1z4-127.0.0.4:6042R127.0.0.4:6042/sdb4_"" with 1.0 weight got id 3
|
||||
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
You can expect the output from this command to produce the following. Note
|
||||
that 3 object rings are created in order to test storage policies and EC in
|
||||
the SAIO environment. The EC ring is the only one with all 8 devices.
|
||||
There are also two replication rings, one for 3x replication and another
|
||||
for 2x replication, but those rings only use 4 devices:
|
||||
|
||||
|
||||
#. Read more about Storage Policies and your SAIO :doc:`policies_saio`
|
||||
.. code-block:: console
|
||||
|
||||
#. Verify the unit tests run::
|
||||
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
|
||||
Device d2r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 2
|
||||
Device d3r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 3
|
||||
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
|
||||
Device d2r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 2
|
||||
Device d3r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 3
|
||||
Reassigned 2048 (200.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb5_"" with 1.0 weight got id 1
|
||||
Device d2r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 2
|
||||
Device d3r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb6_"" with 1.0 weight got id 3
|
||||
Device d4r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 4
|
||||
Device d5r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb7_"" with 1.0 weight got id 5
|
||||
Device d6r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 6
|
||||
Device d7r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb8_"" with 1.0 weight got id 7
|
||||
Reassigned 6144 (600.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
Device d0r1z1-127.0.0.1:6011R127.0.0.1:6011/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z2-127.0.0.2:6021R127.0.0.2:6021/sdb2_"" with 1.0 weight got id 1
|
||||
Device d2r1z3-127.0.0.3:6031R127.0.0.3:6031/sdb3_"" with 1.0 weight got id 2
|
||||
Device d3r1z4-127.0.0.4:6041R127.0.0.4:6041/sdb4_"" with 1.0 weight got id 3
|
||||
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
Device d0r1z1-127.0.0.1:6012R127.0.0.1:6012/sdb1_"" with 1.0 weight got id 0
|
||||
Device d1r1z2-127.0.0.2:6022R127.0.0.2:6022/sdb2_"" with 1.0 weight got id 1
|
||||
Device d2r1z3-127.0.0.3:6032R127.0.0.3:6032/sdb3_"" with 1.0 weight got id 2
|
||||
Device d3r1z4-127.0.0.4:6042R127.0.0.4:6042/sdb4_"" with 1.0 weight got id 3
|
||||
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
|
||||
|
||||
$HOME/swift/.unittests
|
||||
|
||||
Note that the unit tests do not require any swift daemons running.
|
||||
#. Read more about Storage Policies and your SAIO :doc:`policies_saio`
|
||||
|
||||
#. Start the "main" Swift daemon processes (proxy, account, container, and
|
||||
object)::
|
||||
#. Verify the unit tests run::
|
||||
|
||||
startmain
|
||||
$HOME/swift/.unittests
|
||||
|
||||
(The "``Unable to increase file descriptor limit. Running as non-root?``"
|
||||
warnings are expected and ok.)
|
||||
Note that the unit tests do not require any swift daemons running.
|
||||
|
||||
The ``startmain`` script looks like the following:
|
||||
#. Start the "main" Swift daemon processes (proxy, account, container, and
|
||||
object)::
|
||||
|
||||
.. literalinclude:: /../saio/bin/startmain
|
||||
startmain
|
||||
|
||||
#. Get an ``X-Storage-Url`` and ``X-Auth-Token``::
|
||||
(The "``Unable to increase file descriptor limit. Running as non-root?``"
|
||||
warnings are expected and ok.)
|
||||
|
||||
curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
|
||||
The ``startmain`` script looks like the following:
|
||||
|
||||
#. Check that you can ``GET`` account::
|
||||
.. literalinclude:: /../saio/bin/startmain
|
||||
:language: bash
|
||||
|
||||
curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>
|
||||
#. Get an ``X-Storage-Url`` and ``X-Auth-Token``::
|
||||
|
||||
#. Check that ``swift`` command provided by the python-swiftclient package works::
|
||||
curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
|
||||
|
||||
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
|
||||
#. Check that you can ``GET`` account::
|
||||
|
||||
#. Verify the functional tests run::
|
||||
curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>
|
||||
|
||||
$HOME/swift/.functests
|
||||
#. Check that ``swift`` command provided by the python-swiftclient package works::
|
||||
|
||||
(Note: functional tests will first delete everything in the configured
|
||||
accounts.)
|
||||
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
|
||||
|
||||
#. Verify the probe tests run::
|
||||
#. Verify the functional tests run::
|
||||
|
||||
$HOME/swift/.probetests
|
||||
$HOME/swift/.functests
|
||||
|
||||
(Note: probe tests will reset your environment as they call ``resetswift``
|
||||
for each test.)
|
||||
(Note: functional tests will first delete everything in the configured
|
||||
accounts.)
|
||||
|
||||
#. Verify the probe tests run::
|
||||
|
||||
$HOME/swift/.probetests
|
||||
|
||||
(Note: probe tests will reset your environment as they call ``resetswift``
|
||||
for each test.)
|
||||
|
||||
----------------
|
||||
Debugging Issues
|
||||
|
||||
@ -178,6 +178,14 @@ class BufferedHTTPConnection(HTTPConnection):
|
||||
return ret
|
||||
|
||||
def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0):
|
||||
'''Send a request to the server.
|
||||
|
||||
:param method: specifies an HTTP request method, e.g. 'GET'.
|
||||
:param url: specifies the object being requested, e.g. '/index.html'.
|
||||
:param skip_host: if True does not add automatically a 'Host:' header
|
||||
:param skip_accept_encoding: if True does not add automatically an
|
||||
'Accept-Encoding:' header
|
||||
'''
|
||||
self._method = method
|
||||
self._path = url
|
||||
return HTTPConnection.putrequest(self, method, url, skip_host,
|
||||
|
||||
@ -13,7 +13,7 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from urllib import quote
|
||||
from six.moves.urllib.parse import quote
|
||||
from swift.common.utils import public
|
||||
|
||||
from swift.common.middleware.s3api.controllers.base import Controller
|
||||
|
||||
Loading…
Reference in New Issue
Block a user