watcher/doc/source/dev/plugin/strategy-plugin.rst

Build a new optimization strategy

Watcher Decision Engine has an external strategy <strategy_definition> plugin interface which gives anyone the ability to integrate an external strategy in order to make use of placement algorithms.

This section gives some guidelines on how to implement and integrate custom strategies with Watcher. If you wish to create a third-party package for your plugin, you can refer to our documentation for third-party package creation <plugin-base_setup>.

Pre-requisites

Before using any strategy, you should make sure you have your Telemetry service configured so that it would provide you all the metrics you need to be able to use your strategy.

Create a new strategy plugin

In order to create a new strategy, you have to:

• Extend the :py~.UnclassifiedStrategy class
• Implement its :py~.BaseStrategy.get_name class method to return the unique ID of the new strategy you want to create. This unique ID should be the same as the name of the entry point we will declare later on <strategy_plugin_add_entrypoint>.
• Implement its :py~.BaseStrategy.get_display_name class method to return the translated display name of the strategy you want to create. Note: Do not use a variable to return the translated string so it can be automatically collected by the translation tool.
• Implement its :py~.BaseStrategy.get_translatable_display_name class method to return the translation key (actually the English display name) of your new strategy. The value return should be the same as the string translated in :py~.BaseStrategy.get_display_name.
• Implement its :py~.BaseStrategy.execute method to return the solution you computed within your strategy.

Here is an example showing how you can write a plugin called NewStrategy:

# filepath: thirdparty/new.py
# import path: thirdparty.new
import abc

import six

from watcher._i18n import _
from watcher.decision_engine.strategy.strategies import base


class NewStrategy(base.UnclassifiedStrategy):

    def __init__(self, osc=None):
        super(NewStrategy, self).__init__(osc)

    def execute(self, original_model):
        self.solution.add_action(action_type="nop",
                                 input_parameters=parameters)
        # Do some more stuff here ...
        return self.solution

    @classmethod
    def get_name(cls):
        return "new_strategy"

    @classmethod
    def get_display_name(cls):
        return _("New strategy")

    @classmethod
    def get_translatable_display_name(cls):
        return "New strategy"

As you can see in the above example, the :py~.BaseStrategy.execute method returns a :py~.BaseSolution instance as required. This solution is what wraps the abstract set of actions the strategy recommends to you. This solution is then processed by a planner <watcher_planner_definition> to produce an action plan which contains the sequenced flow of actions to be executed by the Watcher Applier <watcher_applier_definition>. This solution also contains the various efficacy indicators <efficacy_indicator_definition> alongside its computed global efficacy <efficacy_definition>.

Please note that your strategy class will expect to find the same constructor signature as BaseStrategy to instantiate you strategy. Therefore, you should ensure that your __init__ signature is identical to the :py~.BaseStrategy one.

Strategy efficacy

As stated before, the NewStrategy class extends a class called :py~.UnclassifiedStrategy. This class actually implements a set of abstract methods which are defined within the :py~.BaseStrategy parent class.

One thing this :py~.UnclassifiedStrategy class defines is that our NewStrategy achieves the unclassified goal. This goal is a peculiar one as it does not contain any indicator nor does it calculate a global efficacy. This proves itself to be quite useful during the development of a new strategy for which the goal has yet to be defined or in case a new goal <implement_goal_plugin> has yet to be implemented.

Define Strategy Parameters

For each new added strategy, you can add parameters spec so that an operator can input strategy parameters when creating an audit to control the :py~.BaseStrategy.execute behavior of strategy. This is useful to define some threshold for your strategy, and tune them at runtime.

To define parameters, just implements :py~.BaseStrategy.get_schema to return parameters spec with jsonschema format. It is strongly encouraged that provide default value for each parameter, or else reference fails if operator specify no parameters.

Here is an example showing how you can define 2 parameters for DummyStrategy:

class DummyStrategy(base.DummyBaseStrategy):

    @classmethod
    def get_schema(cls):
        return {
            "properties": {
                "para1": {
                    "description": "number parameter example",
                    "type": "number",
                    "default": 3.2,
                    "minimum": 1.0,
                    "maximum": 10.2,
                },
                "para2": {
                    "description": "string parameter example",
                    "type": "string",
                    "default": "hello",
                },
            },
        }

You can reference parameters in :py~.BaseStrategy.execute:

class DummyStrategy(base.DummyBaseStrategy):

    def execute(self):
        para1 = self.input_parameters.para1
        para2 = self.input_parameters.para2

        if para1 > 5:
            ...

Operator can specify parameters with following commands:

$ watcher audit create -a <your_audit_template> -p para1=6.0 -p para2=hi

Pls. check user-guide for details.

Abstract Plugin Class

Here below is the abstract :py~.BaseStrategy class:

watcher.decision_engine.strategy.strategies.base.BaseStrategy

Add a new entry point

In order for the Watcher Decision Engine to load your new strategy, the strategy must be registered as a named entry point under the watcher_strategies entry point of your setup.py file. If you are using pbr, this entry point should be placed in your setup.cfg file.

The name you give to your entry point has to be unique and should be the same as the value returned by the :py~.BaseStrategy.get_name class method of your strategy.

Here below is how you would proceed to register NewStrategy using pbr:

[entry_points]
watcher_strategies =
    new_strategy = thirdparty.new:NewStrategy

To get a better understanding on how to implement a more advanced strategy, have a look at the :py~.BasicConsolidation class.

Using strategy plugins

The Watcher Decision Engine service will automatically discover any installed plugins when it is restarted. If a Python package containing a custom plugin is installed within the same environment as Watcher, Watcher will automatically make that plugin available for use.

At this point, Watcher will scan and register inside the Watcher Database <watcher_database_definition> all the strategies (alongside the goals they should satisfy) you implemented upon restarting the Watcher Decision Engine <watcher_decision_engine_definition>.

You should take care when installing strategy plugins. By their very nature, there are no guarantees that utilizing them as is will be supported, as they may require a set of metrics which is not yet available within the Telemetry service. In such a case, please do make sure that you first check/configure the latter so your new strategy can be fully functional.

Querying metrics

A large set of metrics, generated by OpenStack modules, can be used in your strategy implementation. To collect these metrics, Watcher provides a Helper to the Ceilometer API, which makes this API reusable and easier to used.

If you want to use your own metrics database backend, please refer to the Ceilometer developer guide. Indeed, Ceilometer's pluggable model allows for various types of backends. A list of the available backends is located here. The Ceilosca project is a good example of how to create your own pluggable backend.

Finally, if your strategy requires new metrics not covered by Ceilometer, you can add them through a Ceilometer plugin.

Read usage metrics using the Python binding

You can find the information about the Ceilometer Python binding on the OpenStack ceilometer client python API documentation

To facilitate the process, Watcher provides the osc attribute to every strategy which includes clients to major OpenStack services, including Ceilometer. So to access it within your strategy, you can do the following:

# Within your strategy "execute()"
cclient = self.osc.ceilometer
# TODO: Do something here

Using that you can now query the values for that specific metric:

query = None  # e.g. [{'field': 'foo', 'op': 'le', 'value': 34},]
value_cpu = cclient.samples.list(
    meter_name='cpu_util',
    limit=10, q=query)

Read usage metrics using the Watcher Cluster History Helper

Here below is the abstract BaseClusterHistory class of the Helper.

watcher.decision_engine.cluster.history.base.BaseClusterHistory

The following code snippet shows how to create a Cluster History class:

from watcher.decision_engine.cluster.history import ceilometer as ceil

query_history  = ceil.CeilometerClusterHistory()

Using that you can now query the values for that specific metric:

query_history.statistic_aggregation(resource_id=compute_node.uuid,
                              meter_name='compute.node.cpu.percent',
                              period="7200",
                              aggregate='avg'
                              )