chrony is not supported in Xena cycle, remove it from kolla
Moved tasks from chrony role to chrony-cleanup.yml playbook to avoid a
vestigial chrony role.
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Change-Id: I5a730d55afb49d517c85aeb9208188c81e2c84cf
Currently, if you override docker_yum_url, the repo must contain a GPG
key at {{ docker_yum_url }}/gpg, despite the fact that the GPG key URL
can be overridden separately via docker_yum_gpgkey. This change uses
docker_yum_gpgkey consistently, avoiding the need to keep the key in the
repo.
Closes-Bug: #1934913
Change-Id: If8e6a02ce0760123f7b076c711727ef575965192
Adds a new variable, 'disable_firewall', which defaults to true. If set
to false, then the host firewall will not be disabled during
kolla-ansible bootstrap-servers.
Change-Id: Ie5131013012f89c8c3b91ca359ad17d9cb77efc8
By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.
This change updates all references to Ansible facts within Kolla Ansible
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.
This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.
[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars
Change-Id: I7e9d5c9b8b9164d4aee3abb4e37c8f28d98ff5d1
Partially-Implements: blueprint performance-improvements
With the new default since Wallaby, starting Docker makes it
enable forwarding and not filter it at all.
This may pose a security risk and should be mitigated.
Closes-Bug: #1931615
Change-Id: I5129136c066489fdfaa4d93741c22e5010b7e89d
Interface names with dashes can cause problems in Ansible since dashes
are replaced with underscores when referencing facts. In the baremetal
role we reference the fact for api_interface without replacing dashes
with underscores. This may result in host entries being omitted from
/etc/hosts.
This change fixes the issue.
Change-Id: I667adc7d8a7dbd20dbfa293f389e02355f8275bb
Related-Bug: #1927357
docker-ce on Debian/Ubuntu gets started just after installation, before
baremetal role configures daemon.json - which results in iptables rules
being implemented - but not removed on docker engine restart.
Closes-Bug: #1923203
Change-Id: Ib1faa092e0b8f0668d1752490a34d0c2165d58d2
If docker_configure_for_zun is set to true, then Zun-specific
configuration for Docker is applied to all nodes. It should only be
applied based on the relevant inventory groups. In some cases this can
cause Docker to fail to start. See
https://storyboard.openstack.org/#!/story/2008544 for details.
This change applies the configuration based on the zun-compute and
zun-cni-daemon groups. It also modifies the expression to not assume
that these groups exist in the inventory.
Change-Id: I0141abf0dd83e3a567ea6dcca945f86db129becf
Closes-Bug: #1914378
Story: 2008544
Task: 41645
Co-Authored-By: Buddhika Sanjeewa <bsanjeewa@kln.ac.lk>
Docker is using 172.17.0.0/16 by default for bridge networking on
docker0, and this might cause routing problems for operator networks.
This change introduces docker_disable_default_network to disable the
bridge networking by putting "bridge: none"[1] to daemon.json
Bridge networking does not work without iptables, so we set the default
for docker_disable_default_network to
docker_disable_default_iptables_rules.
For better defaults, this feature will be enabled by default in
Wallaby.
[1] https://docs.docker.com/engine/reference/commandline/dockerd/
Change-Id: Ic745300b27e50132d80d03787fa4abfada2d0173
Closes-Bug: #1848249
Related-Bug: #1849275
The bootstrap process tries to removes existing apparmor profiles but
doesn't consider the case where those are disabled. This change fixes
the scenario where the libvirt profile exists but is disabled.
Closes-Bug: 1909874
Change-Id: Ied0f2acc420bd5cf1e092c8aee358cba35bd8d5d
Adding docker apt gpg key requires gpupg to be installed.
Task will fail on minimal Debian 10 install as gnupg absent.
Change-Id: I979f88162ad8a206e413b37ac7fb09bcc912e016
ansible_lsb.codename fact needs lsb-release package installed but it absent in Debian 10 minimal install.
It's better to use ansible_distribution_release fact which contain the same value and don't need extra packages.
Change-Id: Ifda71047ab041b4b9838e3480d6a301a43c1c156
This reverts commit 9cae59be51e8d2d798830042a5fd448a4aa5e7dc.
Reason for revert: This patch was found to introduce issues with fluentd customisation. The underlying issue is not currently fully understood, but could be a sign of other obscure issues.
Change-Id: Ia4859c23d85699621a3b734d6cedb70225576dfc
Closes-Bug: #1906288
Main plays are action-redirect-stubs, ideal for import_tasks.
This avoids 'include' penalty and makes logs/ara look nicer.
Fixes haproxy and rabbitmq not to check the host group as well.
Change-Id: I46136fc40b815e341befff80b54a91ef431eabc0
Partially-Implements: blueprint performance-improvements
If iptables is not installed, e.g. in the CentOS 8 cloud image, and
Docker iptables management is enabled, we get the following errors:
Failed to find iptables: exec: \"iptables\": executable file not found
in $PATH failed to start daemon: Error initializing network controller:
error obtaining controller instance: failed to create NAT chain DOCKER:
Iptables not found
This change installs the iptables package Docker iptables management is
enabled.
Change-Id: I3ba5318debccafb28c3cbce8e4e9813c28b086fc
Closes-Bug: #1899060
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. For unconditionally included tasks, switching to
import_tasks provides a clear benefit.
Benchmarking of include vs. import is available at [1].
This change switches from include_tasks to import_tasks where there is
no condition applied to the include.
[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md#task-include-and-import
Partially-Implements: blueprint performance-improvements
Change-Id: Ia45af4a198e422773d9f009c7f7b2e32ce9e3b97
Docker is manipulating iptables rules by default to provide network
isolation, and this might cause problems if the host already has an
iptables-based firewall.
This change introduces docker_disable_default_iptables_rules to
disable the iptables manipulation by putting "iptables: false" [1] to
daemon.json
For better defaults, this feature will be enabled by default in
Victoria.
[1] https://docs.docker.com/network/iptables/
Closes-Bug: #1849275
Change-Id: I165199fc98fb98f227f2a20284e1bab03ef65b5b
This patch is removing chrony package
from docker host when containerized chrony is enabled.
It is also fixing issue with chrony container running
under Ubuntu docker host as noted below.
+ exec /usr/sbin/chronyd -d -f /etc/chrony/chrony.conf
2020-06-08T08:19:09Z chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 -DEBUG)
2020-06-08T08:19:09Z Fatal error : Could not open configuration file /etc/chrony/chrony.conf : Permission denied
Added also removal apparmor profile for ubuntu when
containerized chrony is enabled, as chrony's package
is not removing apparmor profile, and therefore
containerized chrony is not working.
Change-Id: Icf3bbae38b9f5630b69d5c8cf6a8bee11786a836
Closes-Bug: #1882513
install sg3-utils-udev if multipath is enabled, else SCSI_IDENT*
vars are missing in udev.
Closes-Bug: 1877509
Change-Id: Ib205f3cdb775c9cfa719325f702f4fad196d346b
Zun has a new component "zun-cni-daemon" which should be
deployed in every compute nodes. It is basically an implementation
of CNI (Container Network Interface) that performs the neutron
port binding.
If users is using the capsule (pod) API, the recommended deployment
option is using "cri" as capsule driver. This is basically to use
a CRI runtime (i.e. CRI plugin for containerd) for supporting
capsules (pods). A CRI runtime needs a CNI plugin which is what
the "zun-cni-daemon" provides.
The configuration is based on the Zun installation guide [1].
It consits of the following steps:
* Configure the containerd daemon in the host. The "zun-compute"
container will use grpc to communicate with this service.
* Install the "zun-cni" binary at host. The containerd process
will invoke this binary to call the CNI plugin.
* Run a "zun-cni-daemon" container. The "zun-cni" binary will
communicate with this container via HTTP.
Relevant patches:
Blueprint: https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime
Install guide: https://review.opendev.org/#/c/707948/
Devstack plugin: https://review.opendev.org/#/c/705338/
Kolla image: https://review.opendev.org/#/c/708273/
[1] https://docs.openstack.org/zun/latest/install/index.html
Depends-On: https://review.opendev.org/#/c/721044/
Change-Id: I9c361a99b355af27907cf80f5c88d97191193495
Drops support for creating Python 2 virtualenvs in bootstrap-servers,
and looking for a python2 interpreter in the kolla-ansible script.
Also forces the use of Python 3 as the remote interpreter in CI on
Debian and Ubuntu hosts, since they typically symlink the unversioned
interpreter to python2.7.
Change-Id: Id0e977de381e7faafed738674a140ba36184727e
Partially-Implements: blueprint drop-py2-support
CentOS 8 support is now fairly complete - time to drop CentOS 7.
Partially-Implements: blueprint centos-rhel-8
Change-Id: I940b1d3eceb98e16fa366c243672f588b1412d70
line from /etc/hosts
Ubuntu always uses 127.0.1.1 for that with some tricky sauce
around `hostname` depending on whether it contains '.' or not.
And when I mean `hostname` it's the one returned by `hostname`
command with no arguments.
ansible_hostname is always a single word so we can match on that.
I did not want to remove just any 127.0.1.1 in case someone
is using it for other purposes. :-)
Change-Id: I8bd3d42a5e3bd0f63336ed60a0af90d52b1650d6
Closes-bug: #1862739
Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].
This change removes the Ansible code and associated CI jobs.
[1]: https://review.opendev.org/669214
Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2
* HAProxy is now 1.8 in CentOS 8
* Support python3 in baremetal role
* Remove support for environments without python2 installed (this could
not have worked since we gather facts before this point)
Workarounds:
* Using CentOS 7 yum repo for Docker, with module_hotfixes
Change-Id: I30bd3d58f6224ad4c9575ba66c74deabe6895cc4
Partially-Implements: blueprint centos-rhel-8
If you do the following:
* Install legacy Docker (1.12.0) using kolla-ansible bootstrap-servers
with the Rocky release or earlier.
* Update to Docker CE, using kolla-ansible bootstrap-servers with the
Stein release or later
The package is upgraded, but docker is stopped. This prevents the 'Wait
for Docker to start' task from completing, since Docker will not start.
Seen on CentOS 7.6, Docker CE 19.03.4.
This was tested and working previously, perhaps something changed with
the Docker package.
This change fixes the issue by starting and enabling Docker after the
upgrade.
Change-Id: If6e9c91f3e8d0ec366eea7ca506c6d10dbf11c3a
Closes-Bug: #1852066
If docker_storage_driver is set in globals.yml, then kolla-ansible
bootstrap-servers is run, it fails like so:
The conditional check 'docker_storage_driver' failed. The error was: error
while evaluating conditional (docker_storage_driver): 'devicemapper' is
undefined
Ansible does not like evaluating strings as conditionals. This change switches
to using the length filter.
Change-Id: Ib95cbdac2a659e7a0d5f113fe48046de3a39dcda
Closes-Bug: #1849691
Introduce kolla_address filter.
Introduce put_address_in_context filter.
Add AF config to vars.
Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]
Other changes:
globals.yml - mention just IP in comment
prechecks/port_checks (api_intf) - kolla_address handles validation
3x interface conditional (swift configs: replication/storage)
2x interface variable definition with hostname
(haproxy listens; api intf)
1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)
neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network
basic multinode source CI job for IPv6
prechecks for rabbitmq and qdrouterd use proper NSS database now
MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)
Ceph naming workaround in CI
TODO: probably needs documenting
RabbitMQ IPv6-only proto_dist
Ceph ms switch to IPv6 mode
Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)
haproxy upgrade checks for slaves based on ipv6 addresses
TODO:
ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.
ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.
rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.
ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.
KNOWN ISSUES (beyond us):
One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN
RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982
For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227
Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689
Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
cloud-init to manage /etc/hosts
1) Ubuntu includes a line in /etc/hosts that makes the local hostname and
nodename (if different) point to 127.0.1.1. This can break RabbitMQ,
which expects the hostname to resolve to the API network address.
2) The distribution might come with cloud-init installed, and manage_etc_hosts
configuration enabled. If so, it will override the file /etc/hosts from cloud-init
templates at every boot, which will break RabbitMQ.
This change fixes these issues.
Change-Id: I53261d0403b983ab419bd44e705b89f7b7a1c316
Closes-Bug: #1837699
Instead of changing Docker daemon command line let's change config
for Docker instead. In /etc/docker/daemon.json file as it should be.
Custom Docker options can be set with 'docker_custom_config' variable.
Old 'docker_custom_option' is still present but should be avoided.
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I1215e04ec15b01c0b43bac8c0e81293f6724f278
