be2bd0ea04
When Ironic's CI went to Ubuntu Noble, IPv6 testing broke. Ultimately, that was because the job was changed from BIOS boot mode to UEFI. Which in theory was a good thing. But in reviewing the EDK2 known bugs[0], it is clear IPv6 testing moving forward needs to be manual and cannot be done automatically. For the reader's context, EDK2 is the source of the firmware used for Libvirt UEFI VMs. In attempting to work through the issues, we discovered: - EDK2 doesn't care about stateless (auto discovery) mode, and thus doesn't work when that is set. It attempts to solicit an address, which is an open bug [1] from 2019, opened by an Ironic contributor. - EDK2 *always* attempts PXE v4 first, which takes 60 seconds to timeout, before attempting PXE v6. - PXE v6 *might* work before it times out, which takes 5 minutes to occur, if the dhcp server has cleared leases, but there is no guarentee. In most cases the result is the DHCP server views the address as handed out and that there are no free addresses available to supply. This is also rooted in bug [2], also opened in 2019. - EDK2 then switches gears and tries HTTPBoot... simiarly, but the way it does so, noted in bug [3], is also incompatible with dnsmasq. There are additional bugs, but one sort of gets the idea. Some of this is compounded by aspects like dnsmasq attempting to be strict about responding to requests in the DHCPv6 model. A different DHCP server *might* demonstrate a little differently, but fundimentally the same underlying issues in EDK2 will make testing difficult. In attempting to fix this issue, we also attempted to revert back to BIOS mode. This mode uses iPXE ROM images built for QEMU, yet we quickly discovered these pre-built ROM images lacked IPv6 support in Ubuntu Noble. This likely a regression of Ubuntu, but bug tracking points directly to Upstream iPXE which is not valid as it is a compile time option. Testing the ROMs showed only DHCPv4 being attempted and IPv6 router advertisements being entirely ignored. In a sense, if it did work, it would still kind of be cheating as the iPXE ROM is able to skip the first part of the complexity related to PXE in general. In other words, it is not an entirely realistic test when compared to Bare Metal. As such, we don't have a forward path to "fix" this CI job as is. We know the code works. We know vendor firmware sometimes has quarks like needing stateless or stateful operation, We know Ironic does the right thing... within it's capabilities. We just can't test this in CI. [0]: https://github.com/tianocore/edk2/issues?q=is%3Aissue%20state%3Aopen%20%20ipv6 [1]: https://github.com/tianocore/edk2/issues/9832 [2]: https://github.com/tianocore/edk2/issues/9828 [3]: https://github.com/tianocore/edk2/issues/9689 Change-Id: Ifc25bc1e1abb949892a1297a313d63f74937c9a1
128 lines
6.4 KiB
ReStructuredText
128 lines
6.4 KiB
ReStructuredText
Configuring services for bare metal provisioning using IPv6
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Use of IPv6 addressing for baremetal provisioning requires additional
|
|
configuration. This page covers the IPv6 specifics only. Please refer to
|
|
:doc:`/install/configure-tenant-networks` and
|
|
:doc:`/install/configure-networking` for general networking configuration.
|
|
|
|
.. WARNING::
|
|
IPv6 Networking is not actively tested in upstream Continuous
|
|
Integration pipelines, in large part because test virtual machines
|
|
have limited support and known issues. Hardware vendors often have
|
|
better firmware which enables this functionality, but may require
|
|
specific operating modes, such as ``stateless`` or ``stateful``
|
|
depending on the requirements placed upon them by their customers.
|
|
|
|
Configure ironic PXE driver for provisioning using IPv6 addressing
|
|
==================================================================
|
|
|
|
The PXE drivers operate in such a way that they are able to utilize
|
|
both IPv4 and IPv6 addresses based upon the deployment's operating state and
|
|
configuration. Internally, the drivers attempt to prepare configuration options for both formats, which allows ports which are IPv6 only to automatically
|
|
receive boot parameters. As a result of this, it is critical that the
|
|
:oslo.config:option:`DEFAULT.my_ipv6` configuration parameter is set to the conductor's
|
|
IPv6 address. This option is unique per conductor, and due to the nature
|
|
of automatic address assignment, it cannot be "guessed" by the software.
|
|
|
|
Provisioning with IPv6 stateless addressing
|
|
-------------------------------------------
|
|
|
|
When using stateless addressing DHCPv6 does not provide addresses to the client.
|
|
DHCPv6 however provides other configuration via DHCPv6 options such as the
|
|
bootfile-url and bootfile-parameters.
|
|
|
|
Once the PXE driver is set to operate in IPv6 mode no further configuration is
|
|
required in the Baremetal Service.
|
|
|
|
Creating networks and subnets in the Networking Service
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
When creating the Baremetal Service network(s) and subnet(s) in the Networking
|
|
Service's, subnets should have ``ipv6-address-mode`` set to ``dhcpv6-stateless``
|
|
and ``ip-version`` set to ``6``. Depending on whether a router in the Networking
|
|
Service is providing RA's (Router Advertisements) or not, the ``ipv6-ra-mode``
|
|
for the subnet(s) should either be set to ``dhcpv6-stateless`` or be left unset.
|
|
|
|
.. Note:: If ``ipv6-ra-mode`` is left unset, an external router on the network
|
|
is expected to provide RA's with the appropriate flags set for
|
|
automatic addressing and other configuration.
|
|
|
|
|
|
Provisioning with IPv6 stateful addressing
|
|
------------------------------------------
|
|
|
|
When using stateful addressing DHCPv6 is providing both addresses and other
|
|
configuration via DHCPv6 options such as the bootfile-url and
|
|
bootfile-parameters.
|
|
|
|
The "identity-association" (IA) construct used by DHCPv6 is challenging when
|
|
booting over the network. Firmware, and ramdisks typically end up using
|
|
different DUID/IAID combinations and it is not always possible for one
|
|
chain-booting stage to release its address before giving control to the next
|
|
step. In case the DHCPv6 server is configured with static reservations only
|
|
the result is that booting will fail because the DHCPv6 server has no
|
|
addresses available. To get past this issue either configure the DHCPv6 server
|
|
with multiple address reservations for each host, or use a dynamic range.
|
|
|
|
.. Note:: Support for multiple address reservations requires dnsmasq version
|
|
2.81 or later. Some distributions may backport this feature to
|
|
earlier dnsmasq version as part of the packaging, check the
|
|
distributions release notes.
|
|
|
|
If a different (not dnsmasq) DHCPv6 server backend is used with the
|
|
Networking service, use of multiple address reservations might not
|
|
work.
|
|
|
|
Using the ``flat`` network interface
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Due to the "identity-association" challenges with DHCPv6 provisioning using the
|
|
``flat`` network interface is not recommended. When ironic operates with the
|
|
``flat`` network interface the server instance port is used for provisioning and
|
|
other operations. Ironic will not use multiple address reservations in this
|
|
scenario. Because of this **it will not work in most cases**.
|
|
|
|
Using the ``neutron`` network interface
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
When using the ``neutron`` network interface the Baremetal Service will allocate
|
|
multiple IPv6 addresses (4 addresses per port by default) on the service
|
|
networks used for provisioning, cleaning, rescue and introspection. The number
|
|
of addresses allocated can be controlled via the
|
|
``[neutron]/dhcpv6_stateful_address_count`` option in the Bare Metal Service's
|
|
configuration file (``/etc/ironic/ironic.conf``). Using multiple address
|
|
reservations ensures that the DHCPv6 server can lease addresses to each step.
|
|
|
|
To enable IPv6 provisioning on neutron *flat* provider networks with no switch
|
|
management, the ``local_link_connection`` field of baremetal ports must be set
|
|
to ``{'network_type': 'unmanaged'}``. The following example shows how to set the
|
|
local_link_connection for operation on unmanaged networks::
|
|
|
|
baremetal port set \
|
|
--local-link-connection network_type=unmanaged <port-uuid>
|
|
|
|
The use of multiple IPv6 addresses must also be enabled in the Networking
|
|
Service's dhcp agent configuration (``/etc/neutron/dhcp_agent.ini``) by setting
|
|
the option ``[DEFAULT]/dnsmasq_enable_addr6_list`` to ``True`` (default
|
|
``False`` in Ussuri release).
|
|
|
|
.. Note:: Support for multiple IPv6 address reservations in the dnsmasq backend
|
|
was added to the Networking Service Ussuri release. It was also
|
|
backported to the stable Train release.
|
|
|
|
|
|
Creating networks and subnets in the Networking Service
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
When creating the ironic service network(s) and subnet(s) in the Networking
|
|
Service, subnets should have ``ipv6-address-mode`` set to ``dhcpv6-stateful``
|
|
and ``ip-version`` set to ``6``. Depending on whether a router in the Networking
|
|
Service is providing RA's (Router Advertisements) or not, the ``ipv6-ra-mode``
|
|
for the subnet(s) should be set to either ``dhcpv6-stateful`` or be left
|
|
unset.
|
|
|
|
.. Note:: If ``ipv6-ra-mode`` is left unset, an external router on the network
|
|
is expected to provide RA's with the appropriate flags set for managed
|
|
addressing and other configuration.
|