323 Commits

Author SHA1 Message Date
Jay Faulkner
d7c95306d6 Ironic doesn't use metering; don't start it in CI
We don't use metering. We do use every byte of ram we can get our hands
on.

Change-Id: I839c7fd4cb6fe8661a25e6b4e00650575ae17520
2022-12-13 13:51:11 -08:00
Sławek Kapłoński
7c47ad04fc [grenade] Explicitly enable Neutron ML2/OVS services in the CI job
As with [1] basic grenade job will be switched to run with OVN as
Neutron backend, which is default in Devstack, we need to explicitly
disable ML2/OVN neutron services in the ironic-grenade job and use
ML2/OVS related services in that job.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/867065

[1] https://review.opendev.org/c/openstack/grenade/+/862475

Change-Id: I2ef96d1b3e19004f05253dfae508e9f07ae58f63
2022-12-09 11:52:08 +00:00
Riccardo Pittau
ad0b8e4dce Cross test sushy with python 3.10
We don't test python 3.8 anymore in antelope

Change-Id: I4748f14f7a75ae9da204ffafb61c8e495822f040
2022-10-20 18:04:45 +02:00
1499023c4a Switch to 2023.1 Python3 unit tests and generic template name
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for antelope. Also,
updating the template name to generic one.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: Ie1e2138b16929c204235e459df5f9c26885140ab
2022-09-23 08:39:28 +00:00
Julia Kreger
d8fc96fd1f CI: Changes to support Anaconda CI jobs
Introduces additional job configuration to enable automated
integration testing via tempest of the anaconda deployment
interface.

Also, configures a private subnet with DNS, which is required
by anaconda executing, in order to facilitate processing of URLs.

Change-Id: I61b5205cf2c9f83dfcabf4314247c76fb6a56acd
2022-09-06 07:38:11 -07:00
Dmitry Tantsur
f0a1778766 Finally remove support for netboot and the boot_option capability
Instance network boot (not to be confused with ramdisk, iSCSI or
anaconda deploy methods) is insecure, underused and difficult to
maintain. This change removes a lot of related code from Ironic.

The so called "netboot fallback" is still supported for legacy boot when
boot device management is not available or is unreliable.

Change-Id: Ia8510e4acac6dec0a1e4f5cb0e07008548a00c52
2022-08-02 12:47:31 +02:00
Julia Kreger
af838cca79 CI: Pull in diskimage-builder until new release is cut
Change-Id: I88a4863cd24258eb0b395303738c23e3468615c0
2022-06-30 16:29:01 -07:00
Zuul
936414a3cc Merge "Remove netboot jobs from the gate" 2022-06-25 00:21:16 +00:00
Zuul
d807350f6e Merge "Use bifrost on centos9 integration job" 2022-06-24 10:44:22 +00:00
Zuul
6f90d5f6a7 Merge "Swap the metalsmith UEFI job for a legacy one" 2022-06-22 13:45:22 +00:00
Riccardo Pittau
089b0c8e63 Use bifrost on centos9 integration job
We don't support CentOS Stream 8 anymore.

Change-Id: I90f20395afbb5e4299dc807288d4a44863d56768
2022-06-16 14:07:58 +02:00
Dmitry Tantsur
5bbcabbabe Remove netboot jobs from the gate
Netboot option will be removed soon, this change stops covering it.
Some jobs have been renamed to reflect the new reality.

Change-Id: I7e248c3deb4778fcf59bc64821833987653fbbcd
2022-05-31 10:02:56 +02:00
Zuul
37d70d78d7 Merge "CI: Removing ironic job queue" 2022-05-26 19:54:24 +00:00
Dmitry Tantsur
81f583f69b devstack: use CentOS 9 for DIB IPA builds
Additionally bumps CPU model to host-model as centos9 builds now
require a subset of CPU processors which include advanced features.
Host-model also allows for the VM to still start when running with
pure qemu, as opposed to KVM passthrough.

https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level#architectural_considerations_for_rhel_9

Change-Id: Ic261efd4bf6f5929687df5e7b1b51b541554af18
2022-05-25 08:57:15 -07:00
Julia Kreger
63e53797ad CI: Removing ironic job queue
Change-Id: Idf7991d8f8a8cebcdc0413f26ad31542f8ffa8a8
See: https://zuul-ci.org/docs/zuul/latest/releasenotes.html#relnotes-4-1-0-deprecation-notes
2022-05-19 11:08:22 -07:00
Dmitry Tantsur
f1257c79ca Swap the metalsmith UEFI job for a legacy one
UEFI is well covered by existing jobs, while the legacy job is covering
calling install_bootloader.

Depends-On: https://review.opendev.org/c/openstack/metalsmith/+/841911
Change-Id: I18c5334a52591f3d8ccced15aee8eb332121dded
2022-05-17 12:46:53 +02:00
Zuul
f73639d72c Merge "Fix names of two jobs" 2022-05-09 19:45:25 +00:00
Zuul
49113385e8 Merge "Fix v6 CI job - Return it to normal non-voting status" 2022-05-05 14:46:27 +00:00
Julia Kreger
a9f4acfdb0 Fix v6 CI job - Return it to normal non-voting status
* Fixes the IPv6 job by utilizing HOST_IPV6 instead of
  SERVICE_IPV6, as Devstack now automatically wraps
  SERVICE_IPV6 with brackets as if it is for a URL.
* Locks ipv6 job to bios mode. Ubuntu Focal OVMF/EDK2 does not
  support IPv6 PXE boot by default.
* Split from Devstack in terms of IP usage, since full explicit
  V6 usage is not a thing anymore. 4+6 is the default in devstack
  and regardless of what we set on the job we see both now used.
  So we delineate apart our usage for our own sanity.
* Reduce VM Interface count for IPv6 in an attempt to eliminate
  in-kernel routing confusion by two interfaces on the same physical
  network.
* Set IPv6 mode to dhcpv6-stateless due to fun issues in dhcp clients.
  When we move to UEFI, this will need to be changed to stateful as
  stateless is not supported in general by OVMF/E2DK.

Once the job has run in normal non-voting for a while, and we
ensure that it seems to be stable, we can make it voting again.

Change-Id: Ia833bfb64c6c3cc8e48cbe34ed200536652a8adf
2022-05-04 11:32:29 -07:00
Ghanshyam Mann
2e94aa4241 Drop lower-constraints.txt and its testing
As discussed in TC PTG[1] and TC resolution[2], we are
dropping the lower-constraints.txt file and its testing.
We will keep lower bounds in the requirements.txt file but
with a note that these are not tested lower bounds and we
try our best to keep them updated.

[1] https://etherpad.opendev.org/p/tc-zed-ptg#L326
[2] https://governance.openstack.org/tc/resolutions/20220414-drop-lower-constraints.html#proposal

Change-Id: Ide6b3e9d6c1171f1ae568c256a3e9affa45de17b
2022-04-30 17:58:31 -05:00
Riccardo Pittau
b77a5d67da Fix names of two jobs
Making jobs names less misleading

This should impact sushy and sushy-tools only

sushy change https://review.opendev.org/c/openstack/sushy/+/838662
sushy-tools change https://review.opendev.org/c/openstack/sushy-tools/+/838664

Change-Id: I83f3ac7ddc0662e32c205cd8ec0fab073aeaec56
2022-04-20 08:56:55 +00:00
Julia Kreger
9df7e67e69 Grenade: Change to use bios because we have funky networking
Grenade, for some confusing reason, creates a separate network,
and uses that for upgrade testing as opposed to the original network
the VMs were bound to. If Julia's memory is correct, this was for
multinode upgrade testing.

Anyway, When in UEFI mode, it appears that the TFTP packets
don't get tracked nor cross the boundrary. We likley need to
explicitly address this, but first, lets get the job working as
it was and can then update it.

Also, update requirements because markupsafe removed soft_unicode
method taht was deprecated since a while. Jinja2 started using the
new soft_str method since version 3.0.0

Change-Id: Iaebe966569962b0d3d43774d57b570469479f159
2022-04-04 14:13:58 +02:00
30b0c56d23 Add Python3 zed unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for zed.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I7ac9a2c84d60ca722e2343a5ca21d8c8b2b1ba71
2022-03-24 07:49:59 +00:00
Dmitry Tantsur
5a9dd8b092 Deprecate instance network boot
It's insecure and not very popular. See this post for details:
http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026224.html

Change-Id: I9a2df47bb8c08cc991b3c615a9eb533aba3171f4
2022-02-23 12:15:33 +01:00
Dmitry Tantsur
2f09b7b102 CI: force config drive on the multinode job's subnode
We need configdrives to pass information reliably, and the new cirros
image does not work without them.

Change-Id: I6cafa050d5c1c8289483f968d26c50485fd4528a
2022-02-21 11:57:31 +01:00
Zuul
8452de687e Merge "CI: use a custom cirros partition image instead of the default" 2022-02-16 16:53:59 +00:00
Dmitry Tantsur
bbceca562e CI: use a custom cirros partition image instead of the default
Cirros partition images are not compatible with local boot since they
don't ship grub (nor a normal root partition). This change adds a script
that builds a partition image with UEFI artifacts present. It still
cannot be booted in legacy mode, but it's a progress.

Set the tempest plugin's partition_netboot option. We need it to inform
the tempest plugin about the ability to do local boot. This option
already exists but is never set.

Also set the new default_boot_option parameter, which will be introduced
and used in Iaba563a2ecbca029889bc6894b2a7f0754d27b88.

Remove netboot from most of the UEFI jobs.

Change-Id: I15189e7f5928126c6b336b1416ce6408a4950062
2022-02-16 10:12:06 +01:00
Dmitry Tantsur
f67bbeb9f6 Clean up jobs with legacy names
Not everyone on the team even knows what pxe_ipmitool used to mean :)
Furthermore, we don't need "ipa" in job names, everything uses IPA
for... even longer than pxe_ipmitool does not exist.

While here, one job was clearly meant to use BIOS boot, but it does not.

Change-Id: I8a37efa0f222361f30ddb7fa621548685a40f961
2022-02-03 19:01:28 +01:00
Dmitry Tantsur
6f389138ff Re-enable the standalone job
Change-Id: Ieefdfc0ec287b5fed42f7c7d6d213cbd2f8cd927
2022-02-01 14:40:37 +01:00
Dmitry Tantsur
720b42d538 Avoid non-Stream CentOS and temporary disable the standalone job
The standalone job is failing because of a bug in IPA. To fix it we need
to make DIB jobs operational, and they're failing because of CentOS repos.

Change-Id: I8bd051ea709d328cb5efa2c2cbd5a226bdb4cfd3
2022-01-31 16:32:08 +01:00
Zuul
f6f6ce1a31 Merge "CI: reduce api worker processes to 1" 2021-12-12 18:20:10 +00:00
Julia Kreger
cdc3b9538f CI: Lower test VM memory by 200MB
We're seeing OOM events in CI, hopefully this helps.

Change-Id: Id8c0e4830011ca2fa526df461ed5b9b01f769cf9
2021-12-08 22:43:10 +00:00
Julia Kreger
24184a449b CI: reduce api worker processes to 1
CI is memory intensive, and we realistically don't need 2 or
more API workers running for every single WSGI process which
does not implement it's own specific override value.

This should reduce the memory footprint by an average of six processes
which consume 60-90 MB each.

Change-Id: Ia0a986152c2b9fc9c5ff54cf698a351db452fbbd
2021-12-08 09:17:46 -08:00
Dmitry Tantsur
b37ee7c911 devstack: provide a default for OS_CLOUD
Not having it breaks the inspector grenade job.

Change-Id: I7ee28a85cb2005dd69e6711b301cd029b8ca40cc
2021-12-08 09:49:26 +01:00
Dmitry Tantsur
028448afe4 Add a unit test job with Sushy from source
The final goal is to run it on sushy itself to make sure there are
no regressions.

Change-Id: I6f4bee9a3fa439b1477c41c82304652a801ea55e
2021-11-25 10:00:03 +01:00
Julia Kreger
350c2f7a50 CI: Fix devstack plugin with RBAC changes
Changes a neutron call to be project scoped as system
scoped can't create a resource and, and removes the unset
which no longer makes sense now that
I86ffa9cd52454f1c1c72d29b3a0e0caa3e44b829
has merged removing the legacy vars from devstack.

Also renames intenral use setting of OS_CLOUD to IRONIC_OS_CLOUD
as some services were still working with system scope or some sort
of mixed state occuring previously as some of the environment variables
were present still, however they have been removed from devstack.

This change *does* explicitly set an OS_CLOUD variable as well on
the base ironic job. This is because things like grenade for Xena
will expect the variable to be present.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/818449
Change-Id: I912527d7396a9c6d8ee7e90f0c3fd84461d443c1
2021-11-19 08:22:22 -08:00
Julia Kreger
493b4f0caf Yoga: Change default boot mode to uefi
Change the default boot mode to UEFI, as discussed during the end
of the Wallaby release cycle and previously agreed a very long time
ago by the Ironic community.

Change-Id: I6d735604d56d1687f42d0573a2eed765cbb08aec
2021-10-04 11:57:55 -07:00
816dbfdada Add Python3 yoga unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for yoga.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I1972b0a70173a259d72a8f9d1a2df4221038ac61
2021-09-22 10:11:14 +00:00
Julia Kreger
8e173b88d1 Disable Neutron firewall
Neutron's firewall initialization with OVS seems
to be the source of our pain with ports not being found
by ironic jobs. This is because firewall startup errors
crashes out the agent with a RuntimeError while it is deep
in it's initial __init__ sequence.

This ultimately seems to be rooted with communication
with OVS itself, but perhaps the easiest solution is
to just disable the firewall....

Related: https://bugs.launchpad.net/neutron/+bug/1944201
Change-Id: I303989a825a7e35f1cb7b401134fd63553f6791c
2021-09-20 14:09:20 +00:00
Julia Kreger
34fd84560a Dial back gate job memory allocation
Observed an OOM incident causing
ironic-tempest-ipa-partition-pxe_ipmitool to fail.

One vm started, the other seemed to try to start twice, but both times
stopped shortly into the run and the base OS had recorded in it an OOM
failure.

It appears the actual QEMU memory footprint being consumed when
configured at 3GB is upwards of 4GB, which obviously is too big to
fit in our 8GB VM instance.

Dialing back slightly, in hopes it stabilizes the job.

Change-Id: Id8cef722ed305e96d89b9960a8f60f751f900221
2021-09-15 13:58:19 -07:00
Steve Baker
6af0eb374e Set postgresql password encryption for FIPS compliance
This is part of the work to add jobs which confirm ironic works with
FIPS enabled, but this change is also appropriate non-FIPS jobs.

Change-Id: I4af4e811104088d28d7be6df53c26e72db039e08
2021-08-05 11:47:11 +12:00
Riccardo Pittau
e5d35ba65c Add lower-constraints job to current development branch
The lower-constraints test was removed becuase of an issue where pip
could not correctly determine the required packages versions to install,
ending in an almost infinite loop that would end up in timeout, failure,
and general mayhem.
Recently the issue has been fixed and, if properly configured, the
lower-constraints test can provide good indication of which minimum
versions are required to support the current code.
This patch adds the test back to the current development branch.
The long term goal is to keep the lower-constraints file in the stable
branches, but remove the test job to avoid issues during backports.

Change-Id: I5fff32ec5dd1a045116bcf02349650b1f5e3a196
2021-07-30 14:20:28 +02:00
Zuul
c71583fc8a Merge "Scoped RBAC Devstack Plugin support" 2021-07-21 11:27:17 +00:00
Julia Kreger
b5872c9032 Set glance limit for baremetal friendly images
The devstack default limit enforcement for glance defaults
to 1GB, and unfortunately this is too small for many to use
larger images such as centos which includes hardware firmware
images for execution on baremetal where drivers need the vendor
blobs in order to load/run.

Sets ironic-base to 5GB, and updates examples accordingly.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/801309
Change-Id: I41294eb571d07a270a69e5b816cdbad530749a94
2021-07-19 10:32:43 -07:00
Julia Kreger
2cd6468346 Scoped RBAC Devstack Plugin support
Adds support to the ironic devstack plugin to configure
ironic to be used in a scope-enforcing mode in line with
the Secure RBAC effort. This change also defines two new
integration jobs *and* changes one of the existing
integration.

In these cases, we're testing functional crub interactions,
integration with nova, and integration with ironic-inspector.

As other services come online with their plugins and
devstack code being able to set the appropriate scope
enforcement configuration, we will be able to change the
overall operating default for all of ironic's jobs and
exclude the differences.

This effort identified issues in ironic-tempest-plugin,
tempest, devstack, and required plugin support in
ironic-inspector as well, and is ultimately required
to ensure we do not break the Secure RBAC.

Luckilly, it all works.

Change-Id: Ic40e47cb11a6b6e9915efcb12e7912861f25cae7
2021-07-15 21:58:31 +00:00
Dmitry Tantsur
5a8a77ec5b Don't run the inspector job on changes to inspector tests
The current irrelevant-files contains the ironic tests directory, but
not the inspector one.

Change-Id: I1654e1333f95d4c3ffd25daa2b80ae4cf1033d91
2021-06-17 19:01:24 +02:00
Julia Kreger
6b0fc9089f Add bifrost-benchmark-ironic job
Utilizes a simple bifrost job to stand up a simple ironic deployment
where fake nodes will be created added, and a simple benchmark will
then be executed.

Change-Id: I33e29ee303b2cf4987b36c7aad2482bc3673f669
2021-06-01 15:07:36 +00:00
Zuul
848ba44608 Merge "CI: Collect a snapshot of network connections" 2021-05-25 11:20:45 +00:00
Dmitry Tantsur
a7f4b4a21c Stop testing the iscsi deploy interface
Change-Id: If876d5bbb7e18f293673d56912ecab3170fe5dfb
2021-04-30 15:54:23 +02:00
94bd4c8e37 Add Python3 xena unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for xena.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I24a4b8fd3d7c086b79875a2d0dd5f2399357d988
2021-04-01 09:12:27 +00:00