Merge "Add Mellanox Spectrum to firmware to remove list"

The Mellanox Spectrum series of ASICs are switch controllers.
The firmware packages which presently represent over 10% of the
size of an average Centos Stream based ironic-python-agent image.
Since this is not something we expect folks to really do,
and since it is for a switch device, it doesn't make sense to
ship these firmware binaries in the stock IPA ramdisk image.

Change-Id: Ic8ecf289f1e03e5ee7f36cc6ded707a25cb138c8
Signed-off-by: Julia Kreger <juliaashleykreger@gmail.com>

dib/element/ironic-python-agent-podman/README.rst

@@ -0,0 +1,191 @@
+ironic-python-agent-podman
+===========================
+
+Adds Podman support and configuration files to ironic-python-agent-ramdisk.
+
+Compatibility
+-------------
+This DIB element currently supports Debian-based images only. Additional
+distribution support may be added in the future.
+
+ironic-python-agent-config
+--------------------------
+
+**allow_arbitrary_containers**
+  - **Description:** Defines whether arbitrary containers are allowed. Set to true or false.
+  - **Environment Variable:** ``DIB_ALLOW_ARBITRARY_CONTAINERS``
+  - **Default:** ``false``
+
+**allowed_containers**
+  - **Description:** Specifies a list of allowed container image URLs (ex "image1-url,image2-url").
+  - **Environment Variable:** ``DIB_ALLOWED_CONTAINERS``
+  - **Default:** Empty string (``""``)
+
+**container_steps_file**
+  - **Description:** Specifies the path in the RAM to the YAML file containing container steps to be executed.
+  - **Environment Variable:** ``DIB_CONTAINER_STEPS_FILE``
+  - **Default:** ``/etc/ironic-python-agent.d/mysteps.yaml``
+
+**runner**
+  - **Description:** Defines the container runtime to use, such as podman or docker.
+  - **Environment Variable:** ``DIB_RUNNER``
+  - **Default:** ``podman``
+
+**pull_options**
+  - **Description:** Container pull options (e.g., --tls-verify=false).
+  - **Environment Variable:** ``DIB_PULL_OPTIONS``
+  - **Default:** ``--tls-verify=false``
+
+**run_options**
+  - **Description:** Options passed when running the container (e.g., --rm --network=host).
+  - **Environment Variable:** ``DIB_RUN_OPTIONS``
+  - **Default:** ``--rm --network=host --tls-verify=false``
+
+ironic-python-agent-podman
+--------------------------
+
+**podman_conf_file**
+  - **Description:** The path to the configuration file created in the RAM.
+  - **Environment Variable:** ``DIB_PODMAN_CONF_FILE``
+  - **Default:** ``/etc/containers/containers.conf``
+
+ironic-python-agent-steps
+--------------------------
+
+**steps_file_path**
+  - **Description:** Path to the local step file to be copied to the RAM.
+  - **Environment Variable:** ``DIB_STEPS_FILE_PATH``
+  - **Default:** ``/etc/mysteps.yaml``
+
+Example mysteps.yaml
+--------------------
+
+.. code-block:: yaml
+
+    steps:
+      - name: manage_container_cleanup
+        image: docker://172.24.4.1:5000/cleaning-image:latest
+        interface: deploy
+        reboot_requested: true
+        pull_options:
+          - --tls-verify=false
+        run_options:
+          - --rm
+          - --network=host
+          - --tls-verify=false
+        abortable: true
+        priority: 20
+      - name: manage_container_cleanup2
+        image: docker://172.24.4.1:5000/cleaning-image2:latest
+        interface: deploy
+        reboot_requested: true
+        pull_options:
+          - --tls-verify=false
+        run_options:
+          - --rm
+          - --network=host
+          - --tls-verify=false
+        abortable: true
+        priority: 10
+
+Customization
+-------------
+You can override any of the default values by setting the corresponding
+environment variables during the build process. This allows the configuration
+to be dynamically adapted without modifying the script.
+
+ironic-python-agent-podman
+===========================
+
+Adds Podman support and configuration files to ironic-python-agent-ramdisk.
+
+Compatibility
+-------------
+This DIB element currently supports Debian-based images only. Additional
+distribution support may be added in the future.
+
+ironic-python-agent-config
+--------------------------
+
+**allow_arbitrary_containers**
+  - **Description:** Defines whether arbitrary containers are allowed. Set to true or false.
+  - **Environment Variable:** ``DIB_ALLOW_ARBITRARY_CONTAINERS``
+  - **Default:** ``false``
+
+**allowed_containers**
+  - **Description:** Specifies a list of allowed container image URLs (ex "image1-url,image2-url").
+  - **Environment Variable:** ``DIB_ALLOWED_CONTAINERS``
+  - **Default:** Empty string (``""``)
+
+**container_steps_file**
+  - **Description:** Specifies the path in the RAM to the YAML file containing container steps to be executed.
+  - **Environment Variable:** ``DIB_CONTAINER_STEPS_FILE``
+  - **Default:** ``/etc/ironic-python-agent.d/mysteps.yaml``
+
+**runner**
+  - **Description:** Defines the container runtime to use, such as podman or docker.
+  - **Environment Variable:** ``DIB_RUNNER``
+  - **Default:** ``podman``
+
+**pull_options**
+  - **Description:** Container pull options (e.g., --tls-verify=false).
+  - **Environment Variable:** ``DIB_PULL_OPTIONS``
+  - **Default:** ``--tls-verify=false``
+
+**run_options**
+  - **Description:** Options passed when running the container (e.g., --rm --network=host).
+  - **Environment Variable:** ``DIB_RUN_OPTIONS``
+  - **Default:** ``--rm --network=host --tls-verify=false``
+
+ironic-python-agent-podman
+--------------------------
+
+**podman_conf_file**
+  - **Description:** The path to the configuration file created in the RAM.
+  - **Environment Variable:** ``DIB_PODMAN_CONF_FILE``
+  - **Default:** ``/etc/containers/containers.conf``
+
+ironic-python-agent-steps
+--------------------------
+
+**steps_file_path**
+  - **Description:** Path to the local step file to be copied to the RAM.
+  - **Environment Variable:** ``DIB_STEPS_FILE_PATH``
+  - **Default:** ``/etc/mysteps.yaml``
+
+Example mysteps.yaml
+--------------------
+
+.. code-block:: yaml
+
+    steps:
+      - name: manage_container_cleanup
+        image: docker://172.24.4.1:5000/cleaning-image:latest
+        interface: deploy
+        reboot_requested: true
+        pull_options:
+          - --tls-verify=false
+        run_options:
+          - --rm
+          - --network=host
+          - --tls-verify=false
+        abortable: true
+        priority: 20
+      - name: manage_container_cleanup2
+        image: docker://172.24.4.1:5000/cleaning-image2:latest
+        interface: deploy
+        reboot_requested: true
+        pull_options:
+          - --tls-verify=false
+        run_options:
+          - --rm
+          - --network=host
+          - --tls-verify=false
+        abortable: true
+        priority: 10
+
+Customization
+-------------
+You can override any of the default values by setting the corresponding
+environment variables during the build process. This allows the configuration
+to be dynamically adapted without modifying the script.

dib/element/ironic-python-agent-podman/element-deps

@@ -0,0 +1,2 @@
+ironic-python-agent-ramdisk
+package-installs

dib/element/ironic-python-agent-podman/environment.d/10-setup.bash

@@ -0,0 +1,21 @@
+# Podman Config
+PODMAN_CONF_INSIDEDIR=/etc/containers
+
+export PODMAN_CONF_FILE=${DIB_IPA_PODMAN_CONF_FILE:-$TMP_BUILD_DIR/mnt/$PODMAN_CONF_INSIDEDIR/containers.conf}
+
+# Ipa Config
+IPA_CONF_INSIDEDIR=/etc/ironic-python-agent.d
+export IPA_CONFFILE=$TMP_BUILD_DIR/mnt/$IPA_CONF_INSIDEDIR/ironic_python_agent.conf
+
+export ALLOW_ARBITRARY_CONTAINERS="${DIB_ALLOW_ARBITRARY_CONTAINERS:-false}"
+export ALLOWED_CONTAINERS="${DIB_ALLOWED_CONTAINERS:-""}"
+export CONTAINER_STEPS_FILE="${DIB_CONTAINER_STEPS_FILE:-/etc/ironic-python-agent.d/mysteps.yaml}"
+export RUNNER="${DIB_RUNNER:-podman}"
+export PULL_OPTIONS="${DIB_PULL_OPTIONS:---tls-verify=false}"
+export RUN_OPTIONS="${DIB_RUN_OPTIONS:---rm --network=host --tls-verify=false}"
+
+# Steps Config
+STEPS_INSIDEDIR=/etc/ironic-python-agent.d
+export STEPS_FILE=$TMP_BUILD_DIR/mnt/$STEPS_INSIDEDIR/mysteps.yaml
+
+export STEPS_FILE_PATH="${DIB_STEPS_FILE_PATH:-/etc/mysteps.yaml}"

dib/element/ironic-python-agent-podman/package-installs.yaml

@@ -0,0 +1,2 @@
+podman:
+  phase: install.d

dib/element/ironic-python-agent-podman/pre-finalise.d/10-ipa-podman-config

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# Ensure the directory exists
+sudo mkdir -p $(dirname $PODMAN_CONF_FILE)
+
+# Create Podman configuration file inside the ramdisk with sudo tee
+echo "[engine]
+no_pivot_root = true" | sudo tee $PODMAN_CONF_FILE > /dev/null
+
+echo "Podman configuration written to $PODMAN_CONF_FILE"

dib/element/ironic-python-agent-podman/pre-finalise.d/20-ipa-config

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+# Ensure the directory exists
+sudo mkdir -p $(dirname $IPA_CONFFILE)
+
+# Write the configuration inside the ramdisk
+cat <<EOF | sudo tee $IPA_CONFFILE
+[container]
+allow_arbitrary_containers="$ALLOW_ARBITRARY_CONTAINERS"
+allowed_containers="$ALLOWED_CONTAINERS"
+container_steps_file="$CONTAINER_STEPS_FILE"
+runner="$RUNNER"
+pull_options="$PULL_OPTIONS"
+run_options="$RUN_OPTIONS"
+EOF
+
+echo "Ironic Python Agent config written to $IPA_CONFFILE"

dib/element/ironic-python-agent-podman/pre-finalise.d/30-ipa-steps

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# Copy the YAML file to the ramdisk location
+if [[ -f $STEPS_FILE_PATH ]]; then
+    sudo cp $STEPS_FILE_PATH $STEPS_FILE
+    echo "Local mysteps.yaml copied to ramdisk: $STEPS_FILE"
+else
+    echo "No valid mysteps.yaml file found at $STEPS_FILE_PATH. Please provide a valid file or check the default path."
+    exit 1
+fi

dib/extra-hardware/package-installs.yaml

@@ -13,5 +13,5 @@ smartmontools:
 sysbench:
 
 biosdevname:
-    not-arch: ppc64,ppc64el,ppc64le
+    not-arch: aarch64,arm64,ppc64,ppc64el,ppc64le
     when: IPA_DISTRO_FAMILY = rh

dib/ironic-python-agent-ramdisk/environment.d/10-defaults.bash

@@ -1,4 +1,4 @@
-export IPA_REMOVE_FIRMWARE=${IPA_REMOVE_FIRMWARE:-amdgpu,netronome,qcom,ti-communication,ti-keystone,ueagle-atm,rsi,mrvl,brcm,mediatek,ath10k,rtlwifi,rtw88,rtw89,libertas,ath11k}
+export IPA_REMOVE_FIRMWARE=${IPA_REMOVE_FIRMWARE:-amdgpu,netronome,qcom,ti-communication,ti-keystone,ueagle-atm,rsi,mrvl,brcm,mediatek,ath10k,rtlwifi,rtw88,rtw89,libertas,ath11k,mellanox/mlxsw_spectrum}
 
 # NOTE(TheJulia): List of what each item represents for future context
 # amdgpu == AMD/ATI Radeon/Vega/Raven firmware for drivers
@@ -17,3 +17,4 @@ export IPA_REMOVE_FIRMWARE=${IPA_REMOVE_FIRMWARE:-amdgpu,netronome,qcom,ti-commu
 # rtw89 == Realtek wireless
 # libertas == Marvell libertas wifi
 # auth11k == Qualcomm atheros WLAN
+# mellanox/mlxsw_spectrum = Mellanox Spectrum Switch ASIC

dib/ironic-python-agent-ramdisk/install.d/ironic-python-agent-ramdisk-source-install/60-ironic-python-agent-ramdisk-install

@@ -13,10 +13,17 @@ IPADIR=/tmp/ironic-python-agent
 UPPER_CONSTRAINTS=/tmp/requirements/upper-constraints.txt
 VENVDIR=/opt/ironic-python-agent
 
+# define default python
+PYTHON_BIN="python3"
+if [[ "${DISTRO_NAME}${DIB_RELEASE}" =~ "centos9" ]]; then
+  PYTHON_BIN="python3.12"
+fi
+
 # create the virtual environment using the default python
-python3 -m venv $VENVDIR
-REQUIRED_PIP_STR="21.3.1"
-REQUIRED_PIP_TUPLE="(21, 3, 1)"
+$PYTHON_BIN -m venv $VENVDIR
+
+REQUIRED_PIP_STR="25.1.1"
+REQUIRED_PIP_TUPLE="(25, 1, 1)"
 
 HAS_PIP=$($VENVDIR/bin/python -c \
     "import pip; print(tuple(map(int, pip.__version__.split('.'))) >= $REQUIRED_PIP_TUPLE)")

dib/ironic-python-agent-ramdisk/pkg-map

@@ -1,15 +1,15 @@
 {
   "release": {
     "rhel": {
-      "7": {
-        "python": "python",
-        "python-dev": "python-devel"
+      "9": {
+        "python": "python3.12",
+        "python-dev": "python3.12-devel"
       }
     },
     "centos": {
-      "7": {
-        "python": "python",
-        "python-dev": "python-devel"
+      "9-stream": {
+        "python": "python3.12",
+        "python-dev": "python3.12-devel"
       }
     },
     "debian": {

releasenotes/notes/tinyipa-ci-testing-ends-8748513e6512cf8f.yaml

@@ -0,0 +1,7 @@
+---
+other:
+  - |
+    Due to the deprecation of Python 3.9 support in OpenStack and the
+    evolution of python dependencies, CI test jobs for TinyIPA have
+    been removed. Overall code to build the test images should expect
+    to be removed from Ironic projects and tools as time progresses.

releasenotes/source/2025.1.rst

@@ -0,0 +1,6 @@
+===========================
+2025.1 Series Release Notes
+===========================
+
+.. release-notes::
+   :branch: stable/2025.1

releasenotes/source/index.rst

@@ -6,6 +6,7 @@ Ironic Python Agent Builder Release Notes
    :maxdepth: 1
 
    unreleased
+   2025.1
    2024.2
    2024.1
    2023.2

zuul.d/ironic-python-agent-builder-jobs.yaml

@@ -16,12 +16,6 @@
       ipa_raw_dir: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent/UPLOAD_RAW'
       ipa_tar_dir: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent/UPLOAD_TAR'
 
-- job:
-    name: ironic-python-agent-build-image-tinyipa
-    parent: ironic-python-agent-build-image-base
-    vars:
-      image_type: 'tinyipa'
-
 - job:
     name: ironic-python-agent-build-image-dib-centos9
     parent: ironic-python-agent-build-image-base
@@ -66,12 +60,6 @@
       ipa_raw_dir: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent/UPLOAD_RAW'
       ipa_tar_dir: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent/UPLOAD_TAR'
 
-- job:
-    name: ironic-python-agent-check-image-tinyipa
-    parent: ironic-python-agent-check-image-base
-    vars:
-      image_type: 'tinyipa'
-
 - job:
     name: ironic-python-agent-check-image-dib-centos9
     parent: ironic-python-agent-check-image-base

zuul.d/project.yaml

@@ -7,8 +7,6 @@
       jobs:
         - openstack-tox-pep8
         - ironic-standalone-ipa-src
-        - ipa-tempest-bios-ipmi-direct-tinyipa-src
-        - ironic-python-agent-check-image-tinyipa
         - ironic-python-agent-check-image-dib-centos9
         - ironic-python-agent-check-image-dib-centos9-extra
         - ironic-python-agent-check-image-dib-debian
@@ -31,8 +29,6 @@
       jobs:
         - openstack-tox-pep8
         - ironic-standalone-ipa-src
-        - ipa-tempest-bios-ipmi-direct-tinyipa-src
-        - ironic-python-agent-check-image-tinyipa
         - ironic-python-agent-check-image-dib-centos9
         - ironic-python-agent-check-image-dib-centos9-extra
         - ironic-python-agent-check-image-dib-debian
@@ -41,8 +37,6 @@
     post:
       jobs:
         - publish-openstack-python-branch-tarball
-        - ironic-python-agent-build-image-tinyipa:
-            branches: master
         - ironic-python-agent-build-image-dib-centos9:
             branches: master
         - ironic-python-agent-build-image-dib-debian: