Refactor that prepares kolla_container_facts
module for introducing more actions that will be moved
from kolla_container module and kolla_container_volume_facts.
This change is based on a discussion about adding a new action
to kolla_container module that retrieves all names of the running
containers. It was agreed that kolla-ansible should follow Ansible's
direction of splitting modules between action modules and facts
modules. Because of this, kolla_container_facts needs to be able
to handle different requests for data about containers or volumes.
Change-Id: Ieaec8f64922e4e5a2199db2d6983518b124cb4aa
Signed-off-by: Ivan Halomi <ivan.halomi@tietoevry.com>
The Kolla project supports building images with
user-defined prefixes. However, Kolla-ansible is unable
to use those images for installation.
This patch fixes that issue.
Closes-Bug: #2073541
Change-Id: Ia8140b289aa76fcd584e0e72686e3786215c5a99
Changes name of ansible module kolla_docker to
kolla_container.
Change-Id: I13c676ed0378aa721a21a1300f6054658ad12bc7
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
The dib_env_vars variable in the Bifrost's dib.yml file can contain
the DIB_BLOCK_DEVICE_CONFIG environment variable which is always the
Multiline-YAML data. By default, the format of the data is not
preserved while the configuration is merged and saved for the
bifrost-deploy container.
This is because Ansible uses the PyYAML library which has a default
80 symbol string length limit. The official Ansible documentation [1]
recommends using to_yaml or to_nice_yaml filters with width parameter.
This change adds the same ability to the merge_yaml Ansible plugin.
1. https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#formatting-data-yaml-and-json
The related change for the diskimage-builder to solve the issue with
incorrect data provided by Kolla-Ansible is also provided:
I3b74ede69eb064ad813a9108ec68a228e549e8bb
Closes-Bug: #2014980
Related-Bug: #2014981
Change-Id: Id79445c0311916ac6c1beb3986e14f652ee5a63c
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
Regularly, we experience issues in Kolla Ansible deployments because we
use wrong options in OpenStack configuration files. This is because
OpenStack services ignore unknown options. We also need to keep on top
of deprecated options that may be removed in the future. Integrating
oslo-config-validator into Kolla Ansible will greatly help.
Adds a shared role to run oslo-config-validator on each service. Takes
into account that services have multiple containers, and these may also
use multiple config files. Service roles are extended to use this shared
role. Executed with the new command ``kolla-ansible validate-config``.
Change-Id: Ic10b410fc115646d96d2ce39d9618e7c46cb3fbc
Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/
in which was suggested to split patch into smaller ones.
This change adds container_engine variable to kolla_container_facts
module, this prepares module to be used with docker and podman as well
without further changes in roles.
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I9e8fa30646844ab4a288555f3aafdda345b3a118
First part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/
in which was suggested to split patch into smaller ones.
This implements kolla_container_engine variable
in command calls of docker,so later on it can be
also used for podman without further change.
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Change-Id: Ic30b67daa2e215524096ad1f4385c569e3d41b95
Bifrost supports enabling TLS for the services it deploys, as well as
generating a self-signed TLS certificate. Let's use it.
Change-Id: I2a60ec780c37895e810cdba65bb485d0986a196d
By default Bifrost generates passwords for use by services, and stores
them in files in /root/.config/bifrost/ in the container. This directory
is not persistent, so the passwords are lost if the container is
recreated. This is generally not a problem, because recreating the
container is generally done when redeploying Bifrost, and new passwords
will be generated and written to configuration files. However, if you
access the Ironic or Inspector APIs outside of the Bifrost playbooks,
the credentials will have changed.
This change fixes the issue by persisting the credentials directory in a
Docker volume. Note that applying this change will cause existing
credentials to be removed.
Closes-Bug: #1983356
Change-Id: I45a899e228b7634ba86fab5822139252c48a7f07
With the parameter bifrost_deploy_verbosity it is possible
to set the verbosity of the bootstrap of Bifrost.
This makes it possible to reduce verbosity when running
/bifrost/playbooks/install.yaml if needed.
Change-Id: I5815220f2193a492ae7e1f63443075790ae7aaef
Role vars have a higher precedence than role defaults. This allows to
import default vars from another role via vars_files without overriding
project_name (see related bug for details).
Change-Id: I3d919736e53d6f3e1a70d1267cf42c8d2c0ad221
Related-Bug: #1951785
Bifrost in Yoga will change the default TFTP and HTTP boot directories
to reside under /var/lib/ironic/. We already avoid the cross-filesystem
linking issue that the patch aims to address, by overriding
tftp_master_path. Avoid this breaking us by reverting to the previous
defaults.
Needed-By: https://review.opendev.org/c/openstack/bifrost/+/822743
Change-Id: Idc54c78c618ae90b4d933c2c401bb1789b0abd36
By default those are saved in /var/log/ironic/deploy.
Change of path was introduced by [1] since Victoria.
[1]: https://review.opendev.org/c/openstack/bifrost/+/742854
Change-Id: I0fb69a06d6565d633f9ea0e37e0a780a70bb0ced
This reverts commit 9cae59be51e8d2d798830042a5fd448a4aa5e7dc.
Reason for revert: This patch was found to introduce issues with fluentd customisation. The underlying issue is not currently fully understood, but could be a sign of other obscure issues.
Change-Id: Ia4859c23d85699621a3b734d6cedb70225576dfc
Closes-Bug: #1906288
Main plays are action-redirect-stubs, ideal for import_tasks.
This avoids 'include' penalty and makes logs/ara look nicer.
Fixes haproxy and rabbitmq not to check the host group as well.
Change-Id: I46136fc40b815e341befff80b54a91ef431eabc0
Partially-Implements: blueprint performance-improvements
Currently there is no option to set container_proxy only for one service
(e.g. magnum). This change adds this option.
Change-Id: Ia938ee660ebe8ce84321f721b6292b0b58a06e20
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. For unconditionally included tasks, switching to
import_tasks provides a clear benefit.
Benchmarking of include vs. import is available at [1].
This change switches from include_tasks to import_tasks where there is
no condition applied to the include.
[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md#task-include-and-import
Partially-Implements: blueprint performance-improvements
Change-Id: Ia45af4a198e422773d9f009c7f7b2e32ce9e3b97
Bifrost no longer deploys RabbitMQ, so we should not try to stop it
during upgrade. In fact, if we do then it fails:
Failed to stop rabbitmq-server.service: Unit rabbitmq-server.service not
loaded.
Bifrost removed RabbitMQ in Train, so this is only required from Ussuri.
Change-Id: Ie86f85974fd7385e72a918065fc9c5172f9684ba
For the CentOS 7 to 8 transition, we will have a period where both
CentOS 7 and 8 images are available. We differentiate these images via a
tag - the CentOS 8 images will have a tag of train-centos8 (or
master-centos8 temporarily).
To achieve this, and maintain backwards compatibility for the
openstack_release variable, we introduce a new 'openstack_tag' variable.
This variable is based on openstack_release, but has a suffix of
'openstack_tag_suffix', which is empty except on CentOS 8 where it has a
value of '-centos8'.
Change-Id: I12ce4661afb3c255136cdc1aabe7cbd25560d625
Partially-Implements: blueprint centos-rhel-8
This performs the same as a deploy-bifrost, but first stops the
bifrost services and container if they are running.
This can help where a docker stop may lead to an ungraceful shutdown,
possibly due to running multiple services in one container.
Change-Id: I131ab3c0e850a1d7f5c814ab65385e3a03dfcc74
Implements: blueprint bifrost-upgrade
Closes-Bug: #1834332
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:
* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts
It also adds become for 'command' tasks that use docker CLI.
Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
During the Train cycle, Bifrost switched to using JSON-RPC by default
for Ironic's internal communication [1], avoiding the need to install
RabbitMQ. This simplifies things, so we may as well remove our custom
configuration of RabbitMQ.
[1] https://review.openstack.org/645093
Change-Id: I3107349530aa753d68fd59baaf13eb7dd5485ae6
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.
Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
The Bifrost CI job seems quite unstable, often failing while attempting
to reach the public epel package repositories. We shouldn't need to
install any packages when deploying the container - they should all have
been installed in the image already.
This change avoids running the scripts/env-setup.sh script, which is run
when the container image is built.
It also removes sourcing of /opt/stack/ansible/hacking/env-setup, which
is now just a stub.
Change-Id: I1786e5337a397cb7b427d6b87c21eaee600af170
Ironic creates hardlinks between the TFTP master image store and the
HTTP root path when iPXE is enabled. With Docker volumes used for these
locations we run into https://bugs.launchpad.net/ironic/+bug/1507894
during deployment. If we use a directory under /httpboot to store the
master images this issue is avoided.
This change uses the new bifrost config variable,
ironic_tftp_master_path added in [1] to configure the path, rather than
the existing hacked approach of modifying the ironic.conf config file
after the bifrost_deploy container has started.
[1] https://review.openstack.org/#/c/577071
Change-Id: I5c62999c4956bebd0d3920d756ce67ba194b0ebe
Add become to all tasks that use the module "kolla_docker"
Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks
Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
The permissions of rabbitmq-env.conf for bifrost were changed in
I6e32d94d4172dd96d09d8609e8a5221ab5586a31 to 0660. This breaks bifrost
deployment, as the rabbitmq user is no longer able to read the
root-owned file on startup.
This commit changes the ownership of the file in the container to
rabbitmq:rabbitmq after it has been copied.
Change-Id: I53418f5d4e40b7ca57e546e2e92a57f613fd381e
Closes-Bug: #1766196
This patchset implements yamllint test to all *.yml
files.
Also fixes syntax errors to make jobs to pass.
Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799
Bifrost uses the www-data user for nginx on debian/ubuntu, and nginx on
other platforms. Kolla-ansible currently uses the nginx user for all
platforms when creating a log directory. This change uses the www-data
user on debian/ubuntu when setting ownership of the bifrost nginx log
directory.
Change-Id: I142a158b6f2e8f8a46b684267f6fbb2a6e22a259
Closes-Bug: #1753750
Bifrost cannot modify /etc/hosts from within a container, so add a
host entry during bootstrap.
This was previously fixed by Ied378b4dd755788e75ad1814cecb5700732ba83e
but the logic in bifrost was changed [1], making it out of sync with
kolla-ansible, and triggering a write to /etc/hosts. This change applies
the new logic in bifrost to kolla-ansible, ensuring that the file will
not need to change in the container.
Longer term we should look at ways to make bifrost less eager to modify
/etc/hosts, accepting any file that will work for RabbitMQ.
[1]
60b9a9917e
Change-Id: I0ee05feae3630435b2ec52cfeddf33647a974ee0
Closes-Bug: #1661009
