10577 Commits

Author SHA1 Message Date
Eduardo Gonzalez
b96ade3cf0 [train] Add required Tacker Conductor config and docs
Tacker requires config for storing CSAR vnf packages.
This patch adds it as well as relevant docs.
Only one Tacker Conductor is deployed by default due to
lack of a shared filesystem.

Change-Id: Iad391f35105e79fa9319502256528990915df9b7
Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Closes-Bug: #1845142
2019-10-21 11:15:29 +02:00
Zuul
d783c57a91 Merge "Add IPv6 control plane feature release note" 2019-10-21 09:14:19 +00:00
Radosław Piliszek
9e5b3e3fef Add IPv6 control plane feature release note
IPv6 control plane implementation [1] follow-up.

[1] Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c

Change-Id: Ia18fdaf3fbfb2bfcb565eaa0f06447d00a3d46f5
2019-10-20 19:44:43 +02:00
Radosław Piliszek
099a33c87d [train] Finish configuring Zun to use Placement
This also enables Placement when Zun is enabled like Kolla Ansible
already does with Nova.

Change-Id: Id2a09f702e8503b49d2b9e73e06b2ce9f4d168a9
Closes-bug: #1840573
2019-10-20 19:33:56 +02:00
Radosław Piliszek
4d398f4b7f Fix placement being enabled always instead of with nova
Adds "| bool".

Backportable to Stein.

Change-Id: Ifa2aa387be46beb6da1d3c5a5e0da1b561af8cee
Closes-bug: #1848937
2019-10-20 19:30:21 +02:00
Radosław Piliszek
2447c161b8 Zun: add Cinder (volume) iSCSI support
This makes iSCSI with Zun and Cinder work out-of-the-box.
(LVM included)

Change-Id: I2c4aa666ff9f952e32c450347d31ddbf2e58209e
Related-bug: #1797448
Closes-bug: #1848933
2019-10-20 18:27:37 +02:00
Doug Szumski
8e068099de Add unit tests for Nova Cells filters
This moves the Nova Cells filters alongside the service filters
for ease of testing.

Partially Implements: blueprint support-nova-cells
Change-Id: I32d35c065812c6b46c64bacdf283a0bdad0f8a0f
2019-10-18 13:13:32 +00:00
Michal Nasiadka
2585788982 Use versioned python binary with fetch ceph keyrings
Depends-On: https://review.opendev.org/688636/

Change-Id: I9918ff6a91acde2a7d184e44b8a1014462596e39
2019-10-18 12:00:22 +02:00
Radosław Piliszek
ebf88232dd [nova][train] Remove deprecated RetryFilter
Affects config with Blazar and fake Nova only.
The default does not include it.

Upstream docs:
RetryFilter - Deprecated since version 20.0.0 (Train)
Since the 17.0.0 (Queens) release, the scheduler has provided
alternate hosts for rescheduling so the scheduler does not need to
be called during a reschedule which makes the RetryFilter useless.

Change-Id: I26bf45997005124e9166b5bf1d44cb276624430b
2019-10-18 10:30:37 +02:00
Zuul
b73b9eab26 Merge "Fix OpenSSL template" 2019-10-18 07:14:08 +00:00
Zuul
5aa77e5457 Merge "Neutron: add support to use legacy iptables" 2019-10-18 05:53:21 +00:00
Zuul
bcbcee90d0 Merge "Refactor NSS database var" 2019-10-18 03:38:34 +00:00
Zuul
bb3c3df400 Merge "Openvswitch: some ovs tools require ovs daemons pidfiles" 2019-10-17 19:19:25 +00:00
Zuul
dffbfd6c6d Merge "Fixes missing boolean for Neutron FWaaS" 2019-10-17 14:04:22 +00:00
Mark Goddard
52bc6f609a Docs: add nova cells
Add documentation about deploying nova with multiple cells.

Change-Id: I89ee276917e5b9170746e07b7f644c7593b03da1
Depends-On: https://review.opendev.org/#/c/675659/
Related: blueprint bp/support-nova-cells
2019-10-17 13:43:39 +01:00
Radosław Piliszek
baa85527c9 Do not disable IPv6 in Tempest
IPv6 need not have been disabled in Tempest before either.

Change-Id: I77f691b6b028d1a7f57537442a79557ede5f861e
2019-10-17 13:58:34 +02:00
Zuul
e033c53498 Merge "Add a spec for ceph-ansible" 2019-10-17 10:38:58 +00:00
Zuul
cdfc1c2344 Merge "Set RabbitMQ cluster_partition_handling to pause_minority" 2019-10-17 10:38:57 +00:00
Jan Vondra
e54edb55e4 Neutron: add support to use legacy iptables
neutron_legacy_iptables option sets the KOLLA_LEGACY_IPTABLES
environment variable in the neutron-l3-agent, neutron-linuxbridge-agent
and neutron_openvswich_agent container where it should be consumed
by kolla_extended_start script resulting in setting iptables-legacy.

Depends-On: https://review.opendev.org/#/c/683679/
Change-Id: Iaa8b46a2227b61a729b8d54bbe4b20f389f251d1
2019-10-17 09:42:00 +00:00
Michal Nasiadka
10ca56b981 Add a spec for ceph-ansible
Change-Id: I324c250e57125910b64fad6c06a4d68acf0600b4
2019-10-17 09:06:48 +00:00
Zuul
f2042a878e Merge "Support multiple nova cells" 2019-10-17 08:44:23 +00:00
Radosław Piliszek
4443a53b88 Fix OpenSSL template
OpenSSL certificate should default to FQDN if possible.
Using IP addresses is not recommended, complicates dual stack
and limits addressing flexibility.

IPv6 control plane implementation [1] follow-up.

[1] Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c

Change-Id: Ibfc02f933ddcc170e9d616d401e294ba0ff5e981
2019-10-17 08:49:32 +02:00
Radosław Piliszek
75862bc751 Refactor NSS database var
IPv6 control plane implementation [1] follow-up.

[1] Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c

Change-Id: I4c2bd81e77fc09a04838a62f008e5d6c5dc1483d
2019-10-17 08:37:09 +02:00
Zuul
e7a85726d2 Merge "Implement IPv6 support in the control plane" 2019-10-16 18:22:23 +00:00
Doug Szumski
78a828ef42 Support multiple nova cells
This patch adds initial support for deploying multiple Nova cells.

Splitting a nova-cell role out from the Nova role allows a more granular
approach to deploying and configuring Nova services.

A new enable_cells flag has been added that enables the support of
multiple cells via the introduction of a super conductor in addition to
cell-specific conductors. When this flag is not set (the default), nova
is configured in the same manner as before - with a single conductor.

The nova role now deploys the global services:

* nova-api
* nova-scheduler
* nova-super-conductor (if enable_cells is true)

The nova-cell role handles services specific to a cell:

* nova-compute
* nova-compute-ironic
* nova-conductor
* nova-libvirt
* nova-novncproxy
* nova-serialproxy
* nova-spicehtml5proxy
* nova-ssh

This patch does not support using a single cell controller for managing
more than one cell. Support for sharing a cell controller will be added
in a future patch.

This patch should be backwards compatible and is tested by existing CI
jobs. A new CI job has been added that tests a multi-cell environment.

ceph-mon has been removed from the play hosts list as it is not
necessary - delegate_to does not require the host to be in the play.

Documentation will be added in a separate patch.

Partially Implements: blueprint support-nova-cells
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Change-Id: I810aad7d49db3f5a7fd9a2f0f746fd912fe03917
2019-10-16 17:42:36 +00:00
Viktor Michalek
492940be7b Openvswitch: some ovs tools require ovs daemons pidfiles
Change-Id: I4050c243f05571bbebab07b08c101e61879cda67
Closes-Bug: 1848363
2019-10-16 19:02:50 +02:00
Zuul
21babd3f9d Merge "CI: Increase timeout for upgrade jobs by 30 minutes" 2019-10-16 13:11:43 +00:00
Zuul
5bf83cfed7 Merge "CI: Increase job run attempts to 5" 2019-10-16 12:06:24 +00:00
Dincer Celik
0346dc124e Fixes missing boolean for Neutron FWaaS
The missing boolean breaks Neutron FWaaS deployment.

Change-Id: I169246a6ce8b15af76fd84b32029437016bd3c42
Closes-Bug: #1847562
2019-10-16 14:21:52 +03:00
Zuul
7bde217ac5 Merge "Fixes glance image cache deployment." 2019-10-16 09:54:07 +00:00
Radosław Piliszek
f69a8a9bf4 CI: Increase timeout for upgrade jobs by 30 minutes
Upgrade jobs like to timeout in the 2-hour window when they must
build their images.
This increase is already applied in ceph jobs.

Change-Id: Ic1118760d9192cc15e1ebf37fb8adf3440f18a78
2019-10-16 10:55:44 +02:00
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Radosław Piliszek
f3f4a93e3b CI: Increase job run attempts to 5
Attempts affect pre failures.
This means we can increase stability of jobs by rejecting nodes
that fail pre without failing runs at the same time (unless we
are really unlucky and hit b0rken nodes 5 times in a row).

Change-Id: I17b7f878c742fa8db66f738526855a02ab9f1905
2019-10-16 09:31:38 +02:00
Zuul
ba228529b1 Merge "Fix CI failures" 2019-10-15 15:49:20 +00:00
Mark Goddard
e3e5f7f20f Fix CI failures
1. Fix yamllint errors in .yamllint file(!)

YAML lint is currently failling on its own configuration file,
.yamllint. This change fixes the issues.

2. Run bindep role in Zuul jobs

This fixes an issue where libffi is not available.

Change-Id: Ic08a8e53a6905a68f0fe26d4b28184e62a64324f
2019-10-15 13:27:55 +01:00
Ionut Balutoiu
e3fccdfa65 Fix Python3 compatibility for kolla-genpwd
The method `Fernet.generate_key()` generates a binary string in Python 3:
```
>>> Fernet.generate_key()
b'qSMZlOK23pZUw_Uyy-ZRPUfPskMXKGCGmhG6AHCFiV8='
```

Unless properly written as a string to the Kolla `passwords.yml` file,
the Fernet key will end up in the final Barbican config like this:
```
[simple_crypto_plugin]
kek = b'qSMZlOK23pZUw_Uyy-ZRPUfPskMXKGCGmhG6AHCFiV8='
```

Due to the fact that the key is incorrectly written to the barbican
config file (it should be written as a string), every barbican secret
store fails with:

```
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/store_crypto.py", line 83, in store_secret
barbican.api.controllers     encrypting_plugin, context.project_model)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/store_crypto.py", line 290, in _find_or_create_kek_objects
barbican.api.controllers     kek_meta_dto = plugin_inst.bind_kek_metadata(kek_meta_dto)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/barbican/plugin/crypto/simple_crypto.py", line 104, in bind_kek_metadata
barbican.api.controllers     encryptor = fernet.Fernet(self.master_kek)
barbican.api.controllers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/cryptography/fernet.py", line 38, in __init__
barbican.api.controllers     "Fernet key must be 32 url-safe base64-encoded bytes."
barbican.api.controllers ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
```

This commit fixes the issue described above by properly writing
the Fernet key as a string to the Kolla `passwords.yml` file.

Closes-Bug: #1848191
Change-Id: I27fc0159c889bc2e1576fdd69b7d02a320b620f8
2019-10-15 11:27:07 +00:00
Dincer Celik
cf1b3a73ae Fixes glance image cache deployment.
Deployment fails because the variable "glance_registry_port" in
"ansible/roles/glance/templates/glance-cache.conf.j2" hasn't been configured
anywhere.

Also, "registry_host" and "registry_port" were deprecated since Queens[1], so
they should be removed.

[1] https://specs.openstack.org/openstack/glance-specs/specs/queens/approved/glance/deprecate-registry.html

Closes-Bug: #1848146

Change-Id: I3dd5f5d2ba73d491366791986fdbdf16b75538ef
2019-10-15 11:39:42 +03:00
Zuul
7a135f9703 Merge "setup.cfg: Cleanup" 2019-10-14 13:14:23 +00:00
Zuul
43c9dd94d8 Merge "Do not install ntp package on host when not used" 2019-10-14 12:32:35 +00:00
jacky06
cde22bdddb tox: Keeping going with docs
Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.

[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045

Change-Id: I405812a0039274139e055c54ab7b451dc753c842
2019-10-14 18:20:35 +08:00
Gaëtan Trellu
5b0a281d51 Set RabbitMQ cluster_partition_handling to pause_minority
This is to avoid split-brain.

This change also adds relevant docs that sort out the
HA/quorum questions.

Change-Id: I9a8c2ec4dbbd0318beb488548b2cde8f4e487dc1
Closes-Bug: #1837761
Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-14 10:44:16 +02:00
Zuul
5478f362e7 Merge "Docs: improve Nova documentation" 2019-10-13 08:49:32 +00:00
huangliujie
232a7fa873 setup.cfg: Cleanup
- pbr hasn't need the hook configuration since forever [1]

[1] c84876dc0f

Change-Id: If14fab7d7997f1a324cd3335d627868de3fafd8a
2019-10-13 11:58:21 +08:00
Radosław Piliszek
e91860c357 Do not install ntp package on host when not used
This could badly affect containerized chrony
on Debian family distros.

Change-Id: I3c57c0fe254b6166db55fa33358be646a4a23192
Closes-bug: #1847863
2019-10-12 17:11:41 +02:00
Zuul
9e18c8fc69 Merge "Remove /etc/hosts entries pointing hostname to localhost and prevent cloud-init to manage /etc/hosts" 2019-10-11 18:08:35 +00:00
Zuul
e08f046daf Merge "CI: Use any_errors_fatal in pre.yml and run.yml" 2019-10-11 16:07:38 +00:00
Mark Goddard
0b24a0f2f0 Remove /etc/hosts entries pointing hostname to localhost and prevent
cloud-init to manage /etc/hosts

1) Ubuntu includes a line in /etc/hosts that makes the local hostname and
nodename (if different) point to 127.0.1.1. This can break RabbitMQ,
which expects the hostname to resolve to the API network address.

2) The distribution might come with cloud-init installed, and manage_etc_hosts
configuration enabled. If so, it will override the file /etc/hosts from cloud-init
templates at every boot, which will break RabbitMQ.

This change fixes these issues.

Change-Id: I53261d0403b983ab419bd44e705b89f7b7a1c316
Closes-Bug: #1837699
2019-10-10 16:21:02 +03:00
Zuul
45c175c33e Merge "Change ceph_client caps to use profile rbd" 2019-10-10 10:43:28 +00:00
Zuul
5973708f78 Merge "Copy Nova role as a basis for the Nova cell role" 2019-10-10 10:14:45 +00:00
Michal Nasiadka
bdc8df0c90 Change ceph_client caps to use profile rbd
Using profiles in cephx is the recommended way since Mimic,
this also adds support for blacklist ops.

Change-Id: Ib9f65644637a5761c6cd7ca8925afc6bb2b8d5f5
Closes-Bug: #1760065
2019-10-09 14:18:17 +02:00