Import example hardware managers from ipa-example-hardware-managers

They're not easily discoverable there, let's keep them in tree.

The examples have been restructured to have two different projects
ready to be copied and adjusted. PEP8 failures have been fixed.

Change-Id: I2af04f4b7f9a2109fe83ec517e716159331a48bb
Co-Authored-By: Jay Faulkner <jay@jvf.cc>

doc/source/contributor/hardware_managers.rst

@@ -46,6 +46,11 @@ may want to implement are list_hardware_info(), to add additional hardware
 the GenericHardwareManager is unable to identify and erase_devices(), to
 erase devices in ways other than ATA secure erase or shredding.
 
+The examples_ directory has two example hardware managers that can be copied
+and adapter for your use case.
+
+.. _examples: https://opendev.org/openstack/ironic-python-agent/src/branch/master/examples
+
 Custom HardwareManagers and Cleaning
 ------------------------------------
 One of the reasons to build a custom hardware manager is to expose extra steps

examples/README.rst

@@ -0,0 +1,43 @@
+Example Hardware Managers
+=========================
+
+``vendor-device``
+-----------------
+
+This example manager is meant to demonstrate good patterns for developing a
+device-specific hardware manager, such as for a specific version of NIC or
+disk.
+
+Use Cases include:
+
+* Adding device-specific clean-steps, such as to flash firmware or
+  verify it's still properly working after being provisioned.
+* Implementing erase_device() using a vendor-provided utility for a given
+  disk model.
+
+``business-logic``
+------------------
+
+This example manager is meant to demonstrate how cleaning and the agent can
+use the node object and the node itself to enforce business logic and node
+consistency.
+
+Use Cases include:
+
+* Quality control on hardware by ensuring no component is beyond its useful
+  life.
+* Asserting truths about the node; such as number of disks or total RAM.
+* Reporting metrics about the node's hardware state.
+* Overriding logic of get_os_install_device().
+* Inserting additional deploy steps.
+
+Make your own Manager based on these
+------------------------------------
+
+To make your own hardware manager based on these examples, copy a relevant
+example out of this directory. Modify class names and entrypoints in setup.cfg
+to be not-examples.
+
+Since the entrypoints are defined in setup.cfg, simply installing your new
+python package alongside IPA in a custom ramdisk should be enough to enable
+the new hardware manager.

examples/business-logic/example_business_logic.py

@@ -0,0 +1,98 @@
+# Copyright 2015 Rackspace, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import time
+
+from oslo_log import log
+
+from ironic_python_agent import errors
+from ironic_python_agent import hardware
+
+LOG = log.getLogger()
+
+
+class ExampleBusinessLogicHardwareManager(hardware.HardwareManager):
+    """Example hardware manager to enforce business logic"""
+
+    # All hardware managers have a name and a version.
+    # Version should be bumped anytime a change is introduced. This will
+    # signal to Ironic that if automatic node cleaning is in progress to
+    # restart it from the beginning, to ensure consistency. The value can
+    # be anything; it's checked for equality against previously seen
+    # name:manager pairs.
+    HARDWARE_MANAGER_NAME = 'ExampleBusinessLogicHardwareManager'
+    HARDWARE_MANAGER_VERSION = '1'
+
+    def evaluate_hardware_support(self):
+        """Declare level of hardware support provided.
+
+        Since this example is explicitly about enforcing business logic during
+        cleaning, we want to return a static value.
+
+        :returns: HardwareSupport level for this manager.
+        """
+        return hardware.HardwareSupport.SERVICE_PROVIDER
+
+    def get_clean_steps(self, node, ports):
+        """Get a list of clean steps with priority.
+
+        Define any clean steps added by this manager here. These will be mixed
+        with other loaded managers that support this hardware, and ordered by
+        priority. Higher priority steps run earlier.
+
+        Note that out-of-band clean steps may also be provided by Ironic.
+        These will follow the same priority ordering even though they are not
+        executed by IPA.
+
+        There is *no guarantee whatsoever* that steps defined here will be
+        executed by this HardwareManager. When it comes time to run these
+        steps, they'll be called using dispatch_to_managers() just like any
+        other IPA HardwareManager method. This means if they are unique to
+        your hardware, they should be uniquely named. For example,
+        upgrade_firmware would be a bad step name. Whereas
+        upgrade_foobar_device_firmware would be better.
+
+        :param node: The node object as provided by Ironic.
+        :param ports: Port objects as provided by Ironic.
+        :returns: A list of cleaning steps, as a list of dicts.
+        """
+        # While obviously you could actively run code here, generally this
+        # should just return a static value, as any initialization and
+        # detection should've been done in evaluate_hardware_support().
+        return [{
+            'step': 'companyx_verify_device_lifecycle',
+            'priority': 472,
+            # If you need Ironic to coordinate a reboot after this step
+            # runs, but before continuing cleaning, this should be true.
+            'reboot_requested': False,
+            # If it's safe for Ironic to abort cleaning while this step
+            # runs, this should be true.
+            'abortable': True
+        }]
+
+    # Other examples of interesting cleaning steps for this kind of hardware
+    # manager would include verifying node.properties matches current state of
+    # the node, checking smart stats to ensure the disk is not soon to fail,
+    # or enforcing security policies.
+    def companyx_verify_device_lifecycle(self, node, ports):
+        """Verify node is not beyond useful life of 3 years."""
+        create_date = node.get('created_at')
+        if create_date is not None:
+            server_age = time.time() - time.mktime(time.strptime(create_date))
+            if server_age > (60 * 60 * 24 * 365 * 3):
+                raise errors.CleaningError(
+                    'Server is too old to pass cleaning!')
+            else:
+                LOG.info('Node is %s seconds old, younger than 3 years, '
+                         'cleaning passes.', server_age)

examples/business-logic/setup.cfg

@@ -0,0 +1,19 @@
+[metadata]
+name = example-business-logic
+author = Jay Faulkner
+author-email = jay@jvf.cc
+summary = IPA Example Hardware Managers: Business Logic
+license = Apache-2
+classifier =
+    Intended Audience :: Developers
+    Operating System :: OS Independent
+    License :: OSI Approved :: Apache Software License
+    Programming Language :: Python :: 3
+
+[files]
+modules =
+    example_business_logic
+
+[entry_points]
+ironic_python_agent.hardware_managers =
+    example_business_logic = example_business_logic:ExampleBusinessLogicHardwareManager

examples/business-logic/setup.py

@@ -0,0 +1,6 @@
+#!/usr/bin/env python
+import setuptools
+
+setuptools.setup(
+    setup_requires=['pbr'],
+    pbr=True)

examples/vendor-device/example_device.py

@@ -0,0 +1,152 @@
+# Copyright 2015 Rackspace, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from oslo_log import log
+
+from ironic_python_agent import hardware
+
+LOG = log.getLogger()
+
+
+# All the helper methods should be kept outside of the HardwareManager
+# so they'll never get accidentally called by dispatch_to_managers()
+def _initialize_hardware():
+    """Example method for initalizing hardware."""
+    # Perform any operations here that are required to initialize your
+    # hardware.
+    LOG.debug('Loading drivers, settling udevs, and generally initalizing')
+    pass
+
+
+def _detect_hardware():
+    """Example method for hardware detection."""
+    # For this example, return true if hardware is detected, false if not
+    LOG.debug('Looking for example device')
+    return True
+
+
+def _is_latest_firmware():
+    """Detect if device is running latest firmware."""
+    # Actually detect the firmware version instead of returning here.
+    return True
+
+
+def _upgrade_firmware():
+    """Upgrade firmware on device."""
+    # Actually perform firmware upgrade instead of returning here.
+    return True
+
+
+class ExampleDeviceHardwareManager(hardware.HardwareManager):
+    """Example hardware manager to support a single device"""
+
+    # All hardware managers have a name and a version.
+    # Version should be bumped anytime a change is introduced. This will
+    # signal to Ironic that if automatic node cleaning is in progress to
+    # restart it from the beginning, to ensure consistency. The value can
+    # be anything; it's checked for equality against previously seen
+    # name:manager pairs.
+    HARDWARE_MANAGER_NAME = 'ExampleDeviceHardwareManager'
+    HARDWARE_MANAGER_VERSION = '1'
+
+    def evaluate_hardware_support(self):
+        """Declare level of hardware support provided.
+
+        Since this example covers a case of supporting a specific device,
+        this method is where you would do anything needed to initalize that
+        device, including loading drivers, and then detect if one exists.
+
+        In some cases, if you expect the hardware to be available on any node
+        running this hardware manager, or it's undetectable, you may want to
+        return a static value here.
+
+        Be aware all managers' loaded in IPA will run this method before IPA
+        performs a lookup or begins heartbeating, so the time needed to
+        execute this method will make cleaning and deploying slower.
+
+        :returns: HardwareSupport level for this manager.
+        """
+        _initialize_hardware()
+        if _detect_hardware():
+            # This actually resolves down to an int. Upstream IPA will never
+            # return a value higher than 2 (HardwareSupport.MAINLINE). This
+            # means your managers should always be SERVICE_PROVIDER or higher.
+            LOG.debug('Found example device, returning SERVICE_PROVIDER')
+            return hardware.HardwareSupport.SERVICE_PROVIDER
+        else:
+            # If the hardware isn't supported, return HardwareSupport.NONE (0)
+            # in order to prevent IPA from loading its clean steps or
+            # attempting to use any methods inside it.
+            LOG.debug('No example devices found, returning NONE')
+            return hardware.HardwareSupport.NONE
+
+    def get_clean_steps(self, node, ports):
+        """Get a list of clean steps with priority.
+
+        Define any clean steps added by this manager here. These will be mixed
+        with other loaded managers that support this hardware, and ordered by
+        priority. Higher priority steps run earlier.
+
+        Note that out-of-band clean steps may also be provided by Ironic.
+        These will follow the same priority ordering even though they are not
+        executed by IPA.
+
+        There is *no guarantee whatsoever* that steps defined here will be
+        executed by this HardwareManager. When it comes time to run these
+        steps, they'll be called using dispatch_to_managers() just like any
+        other IPA HardwareManager method. This means if they are unique to
+        your hardware, they should be uniquely named. For example,
+        upgrade_firmware would be a bad step name. Whereas
+        upgrade_foobar_device_firmware would be better.
+
+        :param node: The node object as provided by Ironic.
+        :param ports: Port objects as provided by Ironic.
+        :returns: A list of cleaning steps, as a list of dicts.
+        """
+        # While obviously you could actively run code here, generally this
+        # should just return a static value, as any initialization and
+        # detection should've been done in evaluate_hardware_support().
+        return [{
+            'step': 'upgrade_example_device_model1234_firmware',
+            'priority': 37,
+            # If you need Ironic to coordinate a reboot after this step
+            # runs, but before continuing cleaning, this should be true.
+            'reboot_requested': True,
+            # If it's safe for Ironic to abort cleaning while this step
+            # runs, this should be true.
+            'abortable': False
+        }]
+
+    def upgrade_example_device_model1234_firmware(self, node, ports):
+        """Upgrade firmware on Example Device Model #1234."""
+        # Any commands needed to perform the firmware upgrade should go here.
+        # If you plan on actually flashing firmware every cleaning cycle, you
+        # should ensure your device will not experience flash exhaustion. A
+        # good practice in some environments would be to check the firmware
+        # version against a constant in the code, and noop the method if an
+        # upgrade is not needed.
+        if _is_latest_firmware():
+            LOG.debug('Latest firmware already flashed, skipping')
+            # Return values are ignored here on success
+            return True
+        else:
+            LOG.debug('Firmware version X found, upgrading to Y')
+            # Perform firmware upgrade.
+            try:
+                _upgrade_firmware()
+            except Exception as e:
+                # Log and pass through the exception so cleaning will fail
+                LOG.exception(e)
+                raise
+            return True

examples/vendor-device/setup.cfg

@@ -0,0 +1,20 @@
+[metadata]
+name = example-vendor-device
+author = Jay Faulkner
+author-email = jay@jvf.cc
+summary = IPA Example Hardware Managers: Vendor Device
+license = Apache-2
+classifier =
+    Intended Audience :: Developers
+    Operating System :: OS Independent
+    License :: OSI Approved :: Apache Software License
+    Programming Language :: Python :: 3
+    Development Status :: 4 - Beta
+
+[files]
+modules =
+    example_device
+
+[entry_points]
+ironic_python_agent.hardware_managers =
+    example_device = example_device:ExampleDeviceHardwareManager

examples/vendor-device/setup.py

@@ -0,0 +1,6 @@
+#!/usr/bin/env python
+import setuptools
+
+setuptools.setup(
+    setup_requires=['pbr'],
+    pbr=True)

tox.ini

@@ -35,11 +35,11 @@ commands = stestr run {posargs}
 [testenv:pep8]
 whitelist_externals = bash
 commands =
-  flake8 {posargs:ironic_python_agent imagebuild}
+  flake8 {posargs:ironic_python_agent examples}
   # Run bashate during pep8 runs to ensure violations are caught by
   # the check and gate queues.
   {toxinidir}/tools/run_bashate.sh {toxinidir}
-  doc8 doc/source README.rst
+  doc8 doc/source README.rst examples/README.rst
 
 [testenv:cover]
 setenv = VIRTUAL_ENV={envdir}
@@ -123,3 +123,8 @@ deps =
 deps = -r{toxinidir}/test-requirements.txt
 commands = bandit -r ironic_python_agent -x tests -n5 -ll -c tools/bandit.yml
 
+[testenv:examples]
+commands =
+  pip install -e {toxinidir}/examples/business-logic
+  pip install -e {toxinidir}/examples/vendor-device
+  python -c 'import example_business_logic; import example_device'

zuul.d/ironic-python-agent-jobs.yaml

@@ -2,6 +2,7 @@
     name: ironic-ipa-base
     parent: ironic-base
     irrelevant-files:
+      - ^examples/.*$
       - ^test-requirements.txt$
       - ^.*\.rst$
       - ^doc/.*$
@@ -134,6 +135,23 @@
     timeout: 2400
     vars:
       tox_envlist: bandit
+    irrelevant-files:
+      - ^examples/.*$
+      - ^test-requirements.txt$
+      - ^.*\.rst$
+      - ^doc/.*$
+      - ^ironic_python_agent/tests/.*$
+      - ^releasenotes/.*$
+      - ^setup.cfg$
+      - ^tools/(?!bandit.yml).*$
+      - ^tox.ini$
+
+- job:
+    name: ipa-tox-examples
+    parent: openstack-tox
+    timeout: 600
+    vars:
+      tox_envlist: examples
     irrelevant-files:
       - ^test-requirements.txt$
       - ^.*\.rst$
@@ -212,4 +230,4 @@
 
 - job:
     name: ipa-tempest-bios-ipmi-iscsi-tinyipa-src
-    parent: ipa-tempest-partition-bios-ipmi-iscsi-tinyipa-src
+    parent: ipa-tempest-partition-bios-ipmi-iscsi-tinyipa-src

zuul.d/project.yaml

@@ -8,6 +8,7 @@
       - release-notes-jobs-python3
     check:
       jobs:
+        - ipa-tox-examples
         # NOTE(iurygregory) Only run this two jobs since we are testing
         # wholedisk + partition on tempest
         - ipa-tempest-bios-ipmi-direct-src
@@ -41,4 +42,4 @@
     post:
       jobs:
         - ironic-python-agent-build-image-tinyipa
-        - ironic-python-agent-build-image-dib-centos8
+        - ironic-python-agent-build-image-dib-centos8