f63f1f3082
This change adds an Ansible Galaxy requirements file including the openstack.kolla collection. A new 'kolla-ansible install-deps' command is provided to install the requirements. With the new collection in place, this change also switches to using the baremetal role from the openstack.kolla collection, and removes the baremetal role from this repository. Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/820168 Change-Id: I9708f57b4bb9d64eb4903c253684fe0d9147bd4a
296 lines
9.9 KiB
ReStructuredText
296 lines
9.9 KiB
ReStructuredText
.. _operating-kolla:
|
|
|
|
===============
|
|
Operating Kolla
|
|
===============
|
|
|
|
Tools versioning
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
Kolla and Kolla Ansible use the ``x.y.z`` `semver <https://semver.org/>`_
|
|
nomenclature for naming versions, with major version increasing with each
|
|
new series, e.g., Wallaby.
|
|
The tools are designed to, respectively, build and deploy Docker images of
|
|
OpenStack services of that series.
|
|
Users are advised to run the latest version of tools for the series they
|
|
target, preferably by installing directly from the relevant branch of the Git
|
|
repository, e.g.:
|
|
|
|
.. code-block:: console
|
|
|
|
pip3 install --upgrade git+https://opendev.org/openstack/kolla-ansible@|KOLLA_BRANCH_NAME|
|
|
|
|
Version of deployed images
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
By default, Kolla Ansible will deploy or upgrade using the series name embedded
|
|
in the internal config (``openstack_release``) and it is not recommended to
|
|
tweak this unless using a local registry and a custom versioning policy, e.g.,
|
|
when users want to control when services are upgraded and to which version,
|
|
possibly on a per-service basis (but this is an advanced use case scenario).
|
|
|
|
Upgrade procedure
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
.. note::
|
|
|
|
This procedure is for upgrading from series to series, not for doing updates
|
|
within a series.
|
|
Inside a series, it is usually sufficient to just update the
|
|
``kolla-ansible`` package, rebuild (if needed) and pull the images,
|
|
and run ``kolla-ansible deploy`` again.
|
|
Please follow release notes to check if there are any issues to be aware of.
|
|
|
|
.. note::
|
|
|
|
If you have set ``enable_cells`` to ``yes`` then you should read the
|
|
upgrade notes in the :ref:`Nova cells guide<nova-cells-upgrade>`.
|
|
|
|
Kolla's strategy for upgrades is to never make a mess and to follow consistent
|
|
patterns during deployment such that upgrades from one environment to the next
|
|
are simple to automate.
|
|
|
|
Kolla Ansible implements a single command operation for upgrading an existing
|
|
deployment.
|
|
|
|
Limitations and Recommendations
|
|
-------------------------------
|
|
|
|
.. note::
|
|
|
|
Please note that when the ``use_preconfigured_databases`` flag is set to
|
|
``"yes"``, you need to have the ``log_bin_trust_function_creators`` set to
|
|
``1`` by your database administrator before performing the upgrade.
|
|
|
|
.. note::
|
|
|
|
If you have separate keys for nova and cinder, please be sure to set
|
|
``ceph_nova_keyring: ceph.client.nova.keyring`` and ``ceph_nova_user: nova``
|
|
in ``/etc/kolla/globals.yml``
|
|
|
|
Ubuntu Focal 20.04
|
|
------------------
|
|
|
|
The Victoria release adds support for Ubuntu Focal 20.04 as a host operating
|
|
system. Ubuntu users upgrading from Ussuri should first upgrade OpenStack
|
|
containers to Victoria, which uses the Ubuntu Focal 20.04 base container image.
|
|
Hosts should then be upgraded to Ubuntu Focal 20.04.
|
|
|
|
CentOS Stream 8
|
|
---------------
|
|
|
|
The Wallaby release adds support for CentOS Stream 8 as a host operating
|
|
system. CentOS Stream 8 support will also be added to a Victoria stable
|
|
release. CentOS Linux users upgrading from Victoria should first migrate hosts
|
|
and container images from CentOS Linux to CentOS Stream before upgrading to
|
|
Wallaby.
|
|
|
|
Preparation (the foreword)
|
|
--------------------------
|
|
|
|
Before preparing the upgrade plan and making any decisions, please read the
|
|
`release notes <https://docs.openstack.org/releasenotes/kolla-ansible/index.html>`__
|
|
for the series you are targeting, especially the `Upgrade notes` that we
|
|
publish for your convenience and awareness.
|
|
|
|
Before you begin, **make a backup of your config**. On the operator/deployment
|
|
node, copy the contents of the config directory (``/etc/kolla`` by default) to
|
|
a backup place (or use versioning tools, like git, to keep previous versions
|
|
of config in a safe place).
|
|
|
|
Preparation (the real deal)
|
|
---------------------------
|
|
|
|
First, upgrade the ``kolla-ansible`` package:
|
|
|
|
.. code-block:: console
|
|
|
|
pip3 install --upgrade git+https://opendev.org/openstack/kolla-ansible@|KOLLA_BRANCH_NAME|
|
|
|
|
.. note::
|
|
|
|
If you are running from Git repository, then just checkout the desired
|
|
branch and run ``pip3 install --upgrade`` with the repository directory.
|
|
|
|
If upgrading to a Yoga release or later, install or upgrade Ansible Galaxy
|
|
dependencies:
|
|
|
|
.. code-block:: console
|
|
|
|
kolla-ansible install-deps
|
|
|
|
The inventory file for the deployment should be updated, as the newer sample
|
|
inventory files may have updated layout or other relevant changes.
|
|
The ``diff`` tool (or similar) is your friend in this task.
|
|
If using a virtual environment, the sample inventories are in
|
|
``/path/to/venv/share/kolla-ansible/ansible/inventory/``, else they are
|
|
most likely in
|
|
``/usr/local/share/kolla-ansible/ansible/inventory/``.
|
|
|
|
Other files which may need manual updating are:
|
|
|
|
- ``/etc/kolla/globals.yml``
|
|
- ``/etc/kolla/passwords.yml``
|
|
|
|
For ``globals.yml``, it is best to follow the release notes (mentioned above).
|
|
For ``passwords.yml``, one needs to use ``kolla-mergepwd`` and ``kolla-genpwd``
|
|
tools.
|
|
|
|
``kolla-mergepwd --old OLD_PASSWDS --new NEW_PASSWDS --final FINAL_PASSWDS``
|
|
is used to merge passwords from old installation with newly generated
|
|
passwords. The workflow is:
|
|
|
|
#. Save old passwords from ``/etc/kolla/passwords.yml`` into
|
|
``passwords.yml.old``.
|
|
#. Generate new passwords via ``kolla-genpwd`` as ``passwords.yml.new``.
|
|
#. Merge ``passwords.yml.old`` and ``passwords.yml.new`` into
|
|
``/etc/kolla/passwords.yml``.
|
|
|
|
For example:
|
|
|
|
.. code-block:: console
|
|
|
|
cp /etc/kolla/passwords.yml passwords.yml.old
|
|
cp kolla-ansible/etc/kolla/passwords.yml passwords.yml.new
|
|
kolla-genpwd -p passwords.yml.new
|
|
kolla-mergepwd --old passwords.yml.old --new passwords.yml.new --final /etc/kolla/passwords.yml
|
|
|
|
.. note::
|
|
|
|
``kolla-mergepwd``, by default, keeps old, unused passwords intact.
|
|
To alter this behavior, and remove such entries, use the ``--clean``
|
|
argument when invoking ``kolla-mergepwd``.
|
|
|
|
Run the command below to pull the new images on target hosts:
|
|
|
|
.. code-block:: console
|
|
|
|
kolla-ansible pull
|
|
|
|
It is also recommended to run prechecks to identify potential configuration
|
|
issues:
|
|
|
|
.. code-block:: console
|
|
|
|
kolla-ansible prechecks
|
|
|
|
At a convenient time, the upgrade can now be run.
|
|
|
|
Perform the Upgrade
|
|
-------------------
|
|
|
|
To perform the upgrade:
|
|
|
|
.. code-block:: console
|
|
|
|
kolla-ansible upgrade
|
|
|
|
After this command is complete, the containers will have been recreated from
|
|
the new images and all database schema upgrades and similar actions performed
|
|
for you.
|
|
|
|
Tips and Tricks
|
|
~~~~~~~~~~~~~~~
|
|
|
|
Kolla Ansible CLI
|
|
-----------------
|
|
|
|
When running the ``kolla-ansible`` CLI, additional arguments may be passed to
|
|
``ansible-playbook`` via the ``EXTRA_OPTS`` environment variable.
|
|
|
|
``kolla-ansible -i INVENTORY deploy`` is used to deploy and start all Kolla
|
|
containers.
|
|
|
|
``kolla-ansible -i INVENTORY destroy`` is used to clean up containers and
|
|
volumes in the cluster.
|
|
|
|
``kolla-ansible -i INVENTORY mariadb_recovery`` is used to recover a
|
|
completely stopped mariadb cluster.
|
|
|
|
``kolla-ansible -i INVENTORY prechecks`` is used to check if all requirements
|
|
are meet before deploy for each of the OpenStack services.
|
|
|
|
``kolla-ansible -i INVENTORY post-deploy`` is used to do post deploy on deploy
|
|
node to get the admin openrc file.
|
|
|
|
``kolla-ansible -i INVENTORY pull`` is used to pull all images for containers.
|
|
|
|
``kolla-ansible -i INVENTORY reconfigure`` is used to reconfigure OpenStack
|
|
service.
|
|
|
|
``kolla-ansible -i INVENTORY upgrade`` is used to upgrades existing OpenStack
|
|
Environment.
|
|
|
|
``kolla-ansible -i INVENTORY check`` is used to do post-deployment smoke
|
|
tests.
|
|
|
|
``kolla-ansible -i INVENTORY stop`` is used to stop running containers.
|
|
|
|
``kolla-ansible -i INVENTORY deploy-containers`` is used to check and if
|
|
necessary update containers, without generating configuration.
|
|
|
|
``kolla-ansible -i INVENTORY prune-images`` is used to prune orphaned Docker
|
|
images on hosts.
|
|
|
|
``kolla-ansible -i INVENTORY1 -i INVENTORY2 ...`` Multiple inventories can be
|
|
specified by passing the ``--inventory`` or ``-i`` command line option multiple
|
|
times. This can be useful to share configuration between multiple environments.
|
|
Any common configuration can be set in ``INVENTORY1`` and ``INVENTORY2`` can be
|
|
used to set environment specific details.
|
|
|
|
``kolla-ansible -i INVENTORY gather-facts`` is used to gather Ansible facts,
|
|
for example to populate a fact cache.
|
|
|
|
.. note::
|
|
|
|
In order to do smoke tests, requires ``kolla_enable_sanity_checks=yes``.
|
|
|
|
Using Hashicorp Vault for password storage
|
|
------------------------------------------
|
|
|
|
Hashicorp Vault can be used as an alternative to Ansible Vault for storing
|
|
passwords generated by Kolla Ansible. To use Hashicorp Vault as the secrets
|
|
store you will first need to generate the passwords, and then you can
|
|
save them into an existing KV using the following command:
|
|
|
|
.. code-block:: console
|
|
|
|
kolla-writepwd \
|
|
--passwords /etc/kolla/passwords.yml \
|
|
--vault-addr <VAULT_ADDRESS> \
|
|
--vault-token <VAULT_TOKEN>
|
|
|
|
.. note::
|
|
|
|
For a full list of ``kolla-writepwd`` arguments, use the ``--help``
|
|
argument when invoking ``kolla-writepwd``.
|
|
|
|
To read passwords from Hashicorp Vault and generate a passwords.yml:
|
|
|
|
.. code-block:: console
|
|
|
|
mv kolla-ansible/etc/kolla/passwords.yml /etc/kolla/passwords.yml
|
|
kolla-readpwd \
|
|
--passwords /etc/kolla/passwords.yml \
|
|
--vault-addr <VAULT_ADDRESS> \
|
|
--vault-token <VAULT_TOKEN>
|
|
|
|
Tools
|
|
-----
|
|
|
|
Kolla ships with several utilities intended to facilitate ease of operation.
|
|
|
|
``tools/cleanup-containers`` is used to remove deployed containers from the
|
|
system. This can be useful when you want to do a new clean deployment. It will
|
|
preserve the registry and the locally built images in the registry, but will
|
|
remove all running Kolla containers from the local Docker daemon. It also
|
|
removes the named volumes.
|
|
|
|
``tools/cleanup-host`` is used to remove remnants of network changes
|
|
triggered on the Docker host when the neutron-agents containers are launched.
|
|
This can be useful when you want to do a new clean deployment, particularly one
|
|
changing the network topology.
|
|
|
|
``tools/cleanup-images --all`` is used to remove all Docker images built by
|
|
Kolla from the local Docker cache.
|