4824 Commits

Author SHA1 Message Date
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Radosław Piliszek
e91860c357 Do not install ntp package on host when not used
This could badly affect containerized chrony
on Debian family distros.

Change-Id: I3c57c0fe254b6166db55fa33358be646a4a23192
Closes-bug: #1847863
2019-10-12 17:11:41 +02:00
Zuul
9e18c8fc69 Merge "Remove /etc/hosts entries pointing hostname to localhost and prevent cloud-init to manage /etc/hosts" 2019-10-11 18:08:35 +00:00
Mark Goddard
0b24a0f2f0 Remove /etc/hosts entries pointing hostname to localhost and prevent
cloud-init to manage /etc/hosts

1) Ubuntu includes a line in /etc/hosts that makes the local hostname and
nodename (if different) point to 127.0.1.1. This can break RabbitMQ,
which expects the hostname to resolve to the API network address.

2) The distribution might come with cloud-init installed, and manage_etc_hosts
configuration enabled. If so, it will override the file /etc/hosts from cloud-init
templates at every boot, which will break RabbitMQ.

This change fixes these issues.

Change-Id: I53261d0403b983ab419bd44e705b89f7b7a1c316
Closes-Bug: #1837699
2019-10-10 16:21:02 +03:00
Zuul
45c175c33e Merge "Change ceph_client caps to use profile rbd" 2019-10-10 10:43:28 +00:00
Zuul
5973708f78 Merge "Copy Nova role as a basis for the Nova cell role" 2019-10-10 10:14:45 +00:00
Michal Nasiadka
bdc8df0c90 Change ceph_client caps to use profile rbd
Using profiles in cephx is the recommended way since Mimic,
this also adds support for blacklist ops.

Change-Id: Ib9f65644637a5761c6cd7ca8925afc6bb2b8d5f5
Closes-Bug: #1760065
2019-10-09 14:18:17 +02:00
Zuul
19bdba2b85 Merge "Use internal API for heat -> heat communication" 2019-10-08 09:03:32 +00:00
Zuul
e689d14d56 Merge "Fix swift-proxy-server memcached configuration" 2019-10-07 18:37:38 +00:00
Mark Goddard
3488479d06 Fix swift-proxy-server memcached configuration
Currently, swift-proxy config uses hosts in the swift-proxy-server group
to generate the list of memcached servers. However, memcached is
deployed to hosts in the memcached group.

This change fixes the memcached_servers option for swift-proxy to be the
same as other services.

Change-Id: Ib850a1bb2a504ac3e1396846ca3f1d9a30e8fca0
Closes-Bug: #1774313
2019-10-07 09:22:27 +01:00
Viktor Michalek
d3ae9bf1a9 Horizon: fix for Debian
Depends-On: https://review.opendev.org/686316
Change-Id: I5f204541cc44bca94bed756bb3af3e102f81a1d2
2019-10-03 17:02:41 +02:00
Zuul
c8509dbeac Merge "Haproxy: fix haproxy_cmd for Debian" 2019-10-03 14:50:29 +00:00
Zuul
6bdd7dba75 Merge "[designate] Add coordination backend for designate workers" 2019-10-03 13:15:52 +00:00
Viktor Michalek
78aab26295 Haproxy: fix haproxy_cmd for Debian
Change-Id: I097082112b857444c3e2f73896be5832a776743b
2019-10-03 04:33:22 -07:00
Doug Szumski
952b53089b Copy Nova role as a basis for the Nova cell role
The idea is to factor out a role for deploying Nova related services
to cells. Since all deployments use cells, this role can be used
in both regular deployments which have just cell0 and cell1,
and deployments with many cells.

Partially Implements: blueprint support-nova-cells
Change-Id: Ib1f36ec0a773c384f2c1eac1843782a3e766045a
2019-10-01 14:48:19 +01:00
Mark Goddard
039cc2be50 Add service-rabbitmq role
This role can be used by other roles to register RabbitMQ resources.
Currently support is provided for creating virtual hosts and users.

Change-Id: Ie1774a10b4d629508584af679b8aa9e372847804
Partially Implements: blueprint support-nova-cells
Depends-On: https://review.opendev.org/684742
2019-10-01 14:44:43 +01:00
Joseph M
9cae608392 [designate] Add coordination backend for designate workers
Add coordination backend configuration to designate.conf which is
required in multinode environments. Fixes warning from designate:

WARNING designate.coordination [-] No coordination backend configured,
assuming we are the only worker. Please configure a coordination backend

Change-Id: I23c4d2de7e3f9368795c423000a4f9a6c3a431e2
Closes-Bug: #1843842
Related-Bug: #1840070
2019-09-30 11:02:27 -04:00
Zuul
87e671c6b4 Merge "Add a job that *only* deploys updated containers" 2019-09-30 13:19:28 +00:00
Zuul
78caa69440 Merge "Remove deprecated option 'ovsdb_interface'" 2019-09-30 10:33:25 +00:00
ZijianGuo
1171a8d552 Remove deprecated option 'ovsdb_interface'
The deprecated ovsdb_interface configuration option has been removed,
the default native driver is now always used. [1]

[1] cf37563c83

Change-Id: Idd4e8ad4b00064d180a50c7b98a4568804939f50
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-09-29 10:30:57 +08:00
Doug Szumski
0d7a34e8c7 Remove Nova legacy upgrade
The rolling upgrade has been the default since Stein. The legacy
upgrade has been removed because it doesn't follow the upgrade
guide [1].

[1] https://docs.openstack.org/nova/latest/user/upgrade.html

Change-Id: I2aa879699cb4e9955bf5c38053eada5a53fb6211
2019-09-26 18:04:50 +01:00
Kris Lindgren
2fe0d98ebb Add a job that *only* deploys updated containers
Sometimes as cloud admins, we want to only update code that is running
in a cloud.  But we dont need to do anything else.  Make an action in
kolla-ansible that allows us to do that.

Change-Id: I904f595c69f7276e71692696471e32fd1f88e6e8
Implements: blueprint deploy-containers-action
2019-09-26 17:51:14 +01:00
Zuul
7d95cfd737 Merge "Switch default cloudkitty storage backend to influxdb" 2019-09-26 10:58:00 +00:00
Zuul
8a4a0aa8fd Merge "Fix deployment of Grafana" 2019-09-26 09:42:28 +00:00
Zuul
340b6d9456 Merge "Add support for libvirt+tls" 2019-09-26 09:19:33 +00:00
Radosław Piliszek
84f439aa76 Fix deployment of Grafana
Some tasks were improperly generalized in change:
I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
(simplify handlers)
This patch reverts that.

Change-Id: I38fbe319da50fc7d5a3fa3c4890f039ae900a60c
Closes-bug: #1845258
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-25 08:58:33 +02:00
Mark Goddard
27f4876eed Switch default cloudkitty storage backend to influxdb
Backport: stein

In the Stein release, cloudkitty switched the default storage backend
from sqlalchemy to influxdb. In kolla-ansible stein configuration, we
did not explicitly set the storage backend, and so we automatically
picked up this change. However, prior to
https://review.opendev.org/#/c/615928/ we did not have full support for
InfluxDB as a storage backend, and so this has broken the Rocky-Stein
upgrade (https://bugs.launchpad.net/kolla-ansible/+bug/1838641), which
fails with this during the DB sync:

ERROR cloudkitty InfluxDBClientError: get_list_retention_policies()
requires a database as a parameter or the client to be using a database

This change synchronises our default with cloudkitty's (influxdb), and
also provides an upgrade transition to create the influxdb database.

We also move the cloudkitty_storage_backend variable to
group_vars/all.yml, since it is used to determine whether to enable
influxdb.

Finally, the section name in cloudkitty.conf was incorrect - it was
storage_influx,  but should be storage_influxdb.

Change-Id: I71f2ed11bd06f58e141d222e2709835b7ddb2c71
Closes-Bug: #1838641
2019-09-24 16:15:14 +00:00
Zuul
1982c9809f Merge "Add <project>_install_type for all projects" 2019-09-24 16:05:38 +00:00
Zuul
a89380375c Merge "Create and grant all keystone roles in service-ks-register" 2019-09-24 13:09:19 +00:00
Zuul
01ff25f440 Merge "Remove some deprecated config options" 2019-09-24 13:04:52 +00:00
Zuul
eb80a730be Merge "Swift: add swift_extra_ring_files variable to handle multi-policies deployment" 2019-09-24 09:53:29 +00:00
Zuul
2ba04ad26b Merge "Add [nova] section to ironic.conf" 2019-09-24 09:08:53 +00:00
Zuul
86326b32a3 Merge "Fix removed and deprecated options in ironic.conf" 2019-09-24 09:05:57 +00:00
Zuul
589508d9a3 Merge "Ensure keepalived is restarted during upgrade" 2019-09-24 08:45:04 +00:00
Mark Goddard
741f6d9be9 Create and grant all keystone roles in service-ks-register
This ensures we execute the keystone os_* modules in one place.

Also rework some of the task names and loop item display.

Change-Id: I6764a71e8147410e7b24b0b73d0f92264f45240c
2019-09-24 08:50:04 +01:00
Alexis Deberg
0adbbb268e Swift: add swift_extra_ring_files variable to handle multi-policies deployment
The current tasks only use a hardcoded list deploying only the required files.
When using multiple custom policies, additionnal object-*.builder and
object*.gz files are to be deployed as well.
This adds a new default-empty variable that can be overridden when needed

Change-Id: I29c8e349c7cc83e3a2e01ff702d235a0cd97340e
Closes-Bug: #1844752
2019-09-24 02:43:15 -04:00
Mark Goddard
6f05f1b844 Ensure keepalived is restarted during upgrade
During upgrade, we stop all slave keepalived containers. However, if the
keepalived container configuration has not changed, we never restart
them.

This change fixes the issue by notifying the restart handler when the
containers are stopped.

Change-Id: Ibe094b0c14a70a0eb811182d96f045027aa02c2a
Closes-Bug: #1836368
2019-09-23 15:27:34 +01:00
Mark Goddard
cc555c4196 Add <project>_install_type for all projects
This allows the install type for the project to be different than
kolla_install_type

This can be used to avoid hitting bug 1786238, since kuryr only supports
the source type.

Change-Id: I2b6fc85bac092b1614bccfd22bee48442c55dda4
Closes-Bug: #1786238
2019-09-23 10:23:54 +00:00
Dincer Celik
5ff7bab46b [prometheus] Added support for extra options
This change introduces the way to pass extra options to prometheus.

Currently, prometheus runs with nearly default options, and when clouds
start getting bigger, you need to pass extra parameters to prometheus.

Change-Id: Ic773c0b73062cf3b2285343bafb25d5923911834
2019-09-23 11:25:04 +03:00
Zuul
b7bbbae981 Merge "Adding Prometheus blackbox exporter" 2019-09-20 17:25:04 +00:00
Mark Goddard
e127627dea Remove some deprecated config options
Heat's [DEFAULT] deferred_auth_method is deprecated, and we are setting
the default value of 'trusts'.

Glance's [DEFAULT] registry_host is deprecated, and we do not deploy a
registry.

Change-Id: I80024907c575982699ce323cd9a93bab94c988d3
2019-09-20 14:52:34 +01:00
Mark Goddard
2ddf1fbfab Add retries to keystone resource registration tasks
Sometimes things go wrong. We shouldn't fail a Kolla Ansible run because
of a temporary failure when creating keystone resources.

This task adds retries to the tasks in the service-ks-tasks role.
Default is 5 retries with a 10 second delay, as is used in OpenStack
Ansible.

Change-Id: Ib692062fb93ba330bb9c8a35c684ad06652be8a2
2019-09-20 12:52:55 +01:00
Zuul
ec8437205e Merge "Fix Octavia to use keystone_admin_project variable" 2019-09-19 21:43:58 +00:00
Dincer Celik
a4be4c0dcf Fix Octavia to use keystone_admin_project variable
Project name shouldn't be static as user may override it with
keystone_admin_project

Change-Id: If41b9d8de17985d960104c8daf27ea7d706c27c0
2019-09-19 15:57:57 +00:00
Kris Lindgren
f8cfccb99e Add support for libvirt+tls
To securely support live migration between computenodes we should enable
tls, with cert auth, instead of TCP with no auth support.

Implements: blueprint libvirt-tls

Change-Id: I22ea6233933c840b853fdcc8e03400b2bf577271
2019-09-19 15:32:41 +01:00
Radosław Piliszek
70b4bf6cbf Fix for haproxy precheck failing on CentOS running non-root
Change-Id: I7f2b3a6f1eacd4cabcaa31de543b7489bc5e654b
Closes-bug: #1844636
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-19 11:02:28 +02:00
Zuul
a21b9b5430 Merge "Refactor service, endpoint and user registration" 2019-09-18 17:34:15 +00:00
Zuul
11a572ccb1 Merge "Remove support for OracleLinux" 2019-09-18 14:27:57 +00:00
Zuul
c11ada4148 Merge "Ignore create_cells and discover_computes when nova-api is disabled" 2019-09-18 11:43:18 +00:00
Zuul
6189a0c25c Merge "Update python_path for "source" install type" 2019-09-18 11:43:17 +00:00