diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index 40671f3c9..7905c4a30 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -34,8 +34,8 @@ Components AMQP Bus -------- -The AMQP message bus handles asynchronous communications between the different -Watcher components. +The AMQP message bus handles internal asynchronous communications between the +different Watcher components. .. _cluster_history_db_definition: @@ -51,6 +51,14 @@ MongoDB,...) but will probably be more performant when using which are optimized for handling time series data, which are arrays of numbers indexed by time (a datetime or a datetime range). +.. _cluster_model_db_definition: + +Cluster Model Database +------------------------ + +This component stores the data related to the +:ref:`Cluster Data Model `. + .. _archi_watcher_api_definition: Watcher API @@ -159,7 +167,7 @@ The :ref:`Strategy ` is then dynamically loaded (via `stevedore `_). The :ref:`Watcher Decision Engine ` calls the **execute()** method of the :ref:`Strategy ` class which -generates a set of :ref:`Actions `. +generates a solution composed of a set of :ref:`Actions `. These :ref:`Actions ` are scheduled in time by the :ref:`Watcher Planner ` (i.e., it generates an @@ -179,4 +187,187 @@ Audit, the :ref:`Strategy ` relies on two sets of data: So far, only one :ref:`Strategy ` can be associated to a given :ref:`Goal ` via the main Watcher configuration file. +.. _data_model: + +Data model +========== + +The following diagram shows the data model of Watcher, especially the +functional dependency of objects from the actors (Admin, Customer) point of +view (Goals, Audits, Action Plans, ...): + +.. image:: ./images/functional_data_model.svg + :width: 100% + +.. _sequence_diagrams: + +Sequence diagrams +================= + +The following paragraph shows the messages exchanged between the different +components of Watcher for the most often used scenarios. + +.. _sequence_diagrams_create_audit_template: + +Create a new Audit Template +--------------------------- + +The :ref:`Administrator ` first creates an +:ref:`Audit template ` providing at least the +following parameters: + +- A name +- A goal to achieve + +.. image:: ./images/sequence_create_audit_template.png + :width: 100% + +The `Watcher API`_ just makes sure that the goal exists (i.e. it is declared +in the Watcher configuration file) and stores a new audit template in the +:ref:`Watcher Database `. + +.. _sequence_diagrams_create_and_launch_audit: + +Create and launch a new Audit +----------------------------- + +The :ref:`Administrator ` can then launch a new +:ref:`Audit ` by providing at least the unique UUID of the +previously created :ref:`Audit template `: + +.. image:: ./images/sequence_create_and_launch_audit.png + :width: 100% + +A message is sent on the :ref:`AMQP bus ` which triggers +the Audit in the +:ref:`Watcher Decision Engine `: + +.. image:: ./images/sequence_trigger_audit_in_decision_engine.png + :width: 100% + +The :ref:`Watcher Decision Engine ` reads +the Audit parameters from the +:ref:`Watcher Database `. It instantiates the +appropriate :ref:`Strategy ` (using entry points) +associated to the :ref:`Goal ` of the +:ref:`Audit ` (it uses the information of the Watcher +configuration file to find the mapping between the +:ref:`Goal ` and the :ref:`Strategy ` +python class). + +The :ref:`Watcher Decision Engine ` also +builds the :ref:`Cluster Data Model `. This +data model is needed by the :ref:`Strategy ` to know the +current state and topology of the audited +:ref:`Openstack cluster `. + +The :ref:`Watcher Decision Engine ` calls +the **execute()** method of the instantiated +:ref:`Strategy ` and provides the data model as an input +parameter. This method computes a :ref:`Solution ` to +achieve the goal and returns it to the +:ref:`Decision Engine `. At this point, +actions are not scheduled yet. + +The :ref:`Watcher Decision Engine ` +dynamically loads the :ref:`Watcher Planner ` +implementation which is configured in Watcher (via entry points) and calls the +**schedule()** method of this class with the solution as an input parameter. +This method finds an appropriate scheduling of +:ref:`Actions ` taking into account some scheduling rules +(such as priorities between actions). +It generates a new :ref:`Action Plan ` with status +**RECOMMENDED** and saves it into the +:ref:`Watcher Database `. The saved action plan is +now a scheduled flow of actions. + +If every step executed successfully, the +:ref:`Watcher Decision Engine ` updates +the current status of the Audit to **SUCCEEDED** in the +:ref:`Watcher Database ` and sends a notification +on the bus to inform other components that the :ref:`Audit ` +was successful. + + +.. _sequence_diagrams_launch_action_plan: + +Launch Action Plan +------------------ + +The :ref:`Administrator ` can then launch the +recommended :ref:`Action Plan `: + +.. image:: ./images/sequence_launch_action_plan.png + :width: 100% + +A message is sent on the :ref:`AMQP bus ` which triggers +the :ref:`Action Plan ` in the +:ref:`Watcher Applier `: + +.. image:: ./images/sequence_launch_action_plan_in_applier.png + :width: 100% + +The :ref:`Watcher Applier ` will get the +description of the flow of :ref:`Actions ` from the +:ref:`Watcher Database ` and for each +:ref:`Action ` it will instantiate a corresponding +:ref:`Action ` handler python class. + +The :ref:`Watcher Applier ` will then call the +following methods of the :ref:`Action ` handler: + +- **validate_parameters()**: this method will make sure that all the + provided input parameters are valid: + + - If all parameters are valid, the Watcher Applier moves on to the next + step. + - If it is not, an error is raised and the action is not executed. A + notification is sent on the bus informing other components of the + failure. + +- **preconditions()**: this method will make sure that all conditions are met + before executing the action (for example, it makes sure that an instance + still exists before trying to migrate it). +- **execute()**: this method is what triggers real commands on other + OpenStack services (such as Nova, ...) in order to change target resource + state. If the action is successfully executed, a notification message is + sent on the bus indicating that the new state of the action is + **SUCCEEDED**. + +If every action of the action flow has been executed successfully, a +notification is sent on the bus to indicate that the whole +:ref:`Action Plan ` has **SUCCEEDED**. + + +.. _state_machine_diagrams: + +State Machine diagrams +====================== + +.. _audit_state_machine: + +Audit State Machine +------------------- + +The following diagram shows the different possible states of an +:ref:`Audit ` and what event makes the state change to a new +value: + +.. image:: ./images/audit_state_machine.png + :width: 100% + +.. _action_plan_state_machine: + +Action Plan State Machine +------------------------- + +The following diagram shows the different possible states of an +:ref:`Action Plan ` and what event makes the state +change to a new value: + +.. image:: ./images/action_plan_state_machine.png + :width: 100% + + + .. _Watcher API: webapi/v1.html diff --git a/doc/source/image_src/architecture.dia b/doc/source/image_src/architecture.dia deleted file mode 100644 index 2153355fe..000000000 Binary files a/doc/source/image_src/architecture.dia and /dev/null differ diff --git a/doc/source/image_src/dia/architecture.dia b/doc/source/image_src/dia/architecture.dia new file mode 100644 index 000000000..cf98ea93f Binary files /dev/null and b/doc/source/image_src/dia/architecture.dia differ diff --git a/doc/source/image_src/dia/functional_data_model.dia b/doc/source/image_src/dia/functional_data_model.dia new file mode 100644 index 000000000..bfbdfa5c2 Binary files /dev/null and b/doc/source/image_src/dia/functional_data_model.dia differ diff --git a/doc/source/image_src/plantuml/action_plan_state_machine.txt b/doc/source/image_src/plantuml/action_plan_state_machine.txt new file mode 100644 index 000000000..cb5395d40 --- /dev/null +++ b/doc/source/image_src/plantuml/action_plan_state_machine.txt @@ -0,0 +1,16 @@ +@startuml + +[*] --> RECOMMENDED: The Watcher Planner\ncreates the Action Plan +RECOMMENDED --> TRIGGERED: Administrator launches\nthe Action Plan +TRIGGERED --> ONGOING: The Watcher Applier receives the request\nto launch the Action Plan +ONGOING --> FAILED: Something failed while executing\nthe Action Plan in the Watcher Applier +ONGOING --> SUCCEEDED: The Watcher Applier executed\nthe Action Plan successfully +FAILED --> DELETED : Administrator removes\nAction Plan +SUCCEEDED --> DELETED : Administrator removes\nAction Plan +ONGOING --> CANCELLED : Administrator cancels\nAction Plan +RECOMMENDED --> CANCELLED : Administrator cancels\nAction Plan +TRIGGERED --> CANCELLED : Administrator cancels\nAction Plan +CANCELLED --> DELETED +DELETED --> [*] + +@enduml diff --git a/doc/source/image_src/plantuml/audit_state_machine.txt b/doc/source/image_src/plantuml/audit_state_machine.txt new file mode 100644 index 000000000..8f4c9a1d0 --- /dev/null +++ b/doc/source/image_src/plantuml/audit_state_machine.txt @@ -0,0 +1,14 @@ +@startuml + +[*] --> PENDING: Audit requested by Administrator +PENDING --> ONGOING: Audit request is received\nby the Watcher Decision Engine +ONGOING --> FAILED: Audit fails\n(no solution found, technical error, ...) +ONGOING --> SUCCEEDED: The Watcher Decision Engine\ncould find at least one Solution +FAILED --> DELETED : Administrator wants to\narchive/delete the Audit +SUCCEEDED --> DELETED : Administrator wants to\narchive/delete the Audit +PENDING --> CANCELLED : Administrator cancels\nthe Audit +ONGOING --> CANCELLED : Administrator cancels\nthe Audit +CANCELLED --> DELETED : Administrator wants to\narchive/delete the Audit +DELETED --> [*] + +@enduml diff --git a/doc/source/image_src/plantuml/sequence_create_and_launch_audit.txt b/doc/source/image_src/plantuml/sequence_create_and_launch_audit.txt new file mode 100644 index 000000000..135df1ee6 --- /dev/null +++ b/doc/source/image_src/plantuml/sequence_create_and_launch_audit.txt @@ -0,0 +1,24 @@ +@startuml + + +actor Administrator + +Administrator -> "Watcher CLI" : watcher audit-create -a + +"Watcher CLI" -> "Watcher API" : POST audit(parameters) +"Watcher API" -> "Watcher Database" : create new audit in database (status=PENDING) + +"Watcher API" <-- "Watcher Database" : new audit uuid +"Watcher CLI" <-- "Watcher API" : return new audit URL + +Administrator <-- "Watcher CLI" : new audit uuid + +"Watcher API" -> "AMQP Bus" : trigger_audit(new_audit.uuid) +"AMQP Bus" -> "Watcher Decision Engine" : trigger_audit(new_audit.uuid) + +ref over "Watcher Decision Engine" + Trigger audit in the + Watcher Decision Engine +end ref + +@enduml diff --git a/doc/source/image_src/plantuml/sequence_create_audit_template.txt b/doc/source/image_src/plantuml/sequence_create_audit_template.txt new file mode 100644 index 000000000..98f8dc8cd --- /dev/null +++ b/doc/source/image_src/plantuml/sequence_create_audit_template.txt @@ -0,0 +1,16 @@ +@startuml + +actor Administrator + +Administrator -> "Watcher CLI" : watcher audit-template-create +"Watcher CLI" -> "Watcher API" : POST audit_template(parameters) + +"Watcher API" -> "Watcher API" : make sure goal exist in configuration +"Watcher API" -> "Watcher Database" : create new audit_template in database + +"Watcher API" <-- "Watcher Database" : new audit template uuid +"Watcher CLI" <-- "Watcher API" : return new audit template URL in HTTP Location Header +Administrator <-- "Watcher CLI" : new audit template uuid + +@enduml + diff --git a/doc/source/image_src/plantuml/sequence_launch_action_plan.txt b/doc/source/image_src/plantuml/sequence_launch_action_plan.txt new file mode 100644 index 000000000..92469e654 --- /dev/null +++ b/doc/source/image_src/plantuml/sequence_launch_action_plan.txt @@ -0,0 +1,23 @@ +@startuml + +actor Administrator + +Administrator -> "Watcher CLI" : watcher action-plan-start + +"Watcher CLI" -> "Watcher API" : PATCH action_plan(state=TRIGGERED) +"Watcher API" -> "Watcher Database" : action_plan.state=TRIGGERED + +"Watcher CLI" <-- "Watcher API" : HTTP 200 + +Administrator <-- "Watcher CLI" : OK + +"Watcher API" -> "AMQP Bus" : launch_action_plan(action_plan.uuid) +"AMQP Bus" -> "Watcher Applier" : launch_action_plan(action_plan.uuid) + +ref over "Watcher Applier" + Launch Action Plan in the + Watcher Applier +end ref + +@enduml + diff --git a/doc/source/image_src/plantuml/sequence_launch_action_plan_in_applier.txt b/doc/source/image_src/plantuml/sequence_launch_action_plan_in_applier.txt new file mode 100644 index 000000000..fe9caab08 --- /dev/null +++ b/doc/source/image_src/plantuml/sequence_launch_action_plan_in_applier.txt @@ -0,0 +1,31 @@ +@startuml + +"AMQP Bus" -> "Watcher Applier" : launch_action_plan(action_plan.uuid) +"Watcher Applier" -> "Watcher Database" : action_plan.state=ONGOING +"Watcher Applier" -[#blue]> "AMQP Bus" : notify action plan state = ONGOING +"Watcher Applier" -> "Watcher Database" : get_action_list(action_plan.uuid) +"Watcher Applier" <-- "Watcher Database" : actions +loop for each action of the action flow +create Action +"Watcher Applier" -> Action : instantiate Action object with target resource id\n and input parameters +"Watcher Applier" -> Action : validate_parameters() +"Watcher Applier" <-- Action : OK +"Watcher Applier" -[#blue]> "AMQP Bus" : notify action state = ONGOING +"Watcher Applier" -> Action : preconditions() +"Watcher Applier" <-- Action : OK +"Watcher Applier" -> Action : execute() +alt action is "migrate instance" +Action -> "Nova API" : migrate(instance_id, dest_host_id) +Action <-- "Nova API" : OK +else action is "disable hypervisor" +Action -> "Nova API" : host-update(host_id, maintenance=true) +Action <-- "Nova API" : OK +end +"Watcher Applier" <-- Action : OK +"Watcher Applier" -> "Watcher Database" : action.state=SUCCEEDED +"Watcher Applier" -[#blue]> "AMQP Bus" : notify action state = SUCCEEDED +end +"Watcher Applier" -> "Watcher Database" : action_plan.state=SUCCEEDED +"Watcher Applier" -[#blue]> "AMQP Bus" : notify action plan state = SUCCEEDED + +@enduml diff --git a/doc/source/image_src/plantuml/sequence_trigger_audit_in_decision_engine.txt b/doc/source/image_src/plantuml/sequence_trigger_audit_in_decision_engine.txt new file mode 100644 index 000000000..91cfa1781 --- /dev/null +++ b/doc/source/image_src/plantuml/sequence_trigger_audit_in_decision_engine.txt @@ -0,0 +1,31 @@ +@startuml + +"AMQP Bus" -> "Watcher Decision Engine" : trigger_audit(new_audit.uuid) +"Watcher Decision Engine" -> "Watcher Database" : update audit.state = ONGOING +"AMQP Bus" <[#blue]- "Watcher Decision Engine" : notify new audit state = ONGOING +"Watcher Decision Engine" -> "Watcher Database" : get audit parameters(goal, ...) +"Watcher Decision Engine" <-- "Watcher Database" : audit parameters(goal, ...) +create Strategy +"Watcher Decision Engine" -[#red]> "Strategy": select appropriate\noptimization strategy +loop while enough data to build cluster data model + "Watcher Decision Engine" -> "Nova API" : get resource state (host, instance, ...) + "Watcher Decision Engine" <-- "Nova API" : resource state +end +"Watcher Decision Engine" -[#red]> "Watcher Decision Engine": build cluster_data_model +"Watcher Decision Engine" -> "Strategy" : execute(cluster_data_model) +loop while enough history data for the strategy +"Strategy" -> "Ceilometer API": get_aggregated_metrics\n(resource_id,meter_name,period,aggregate_method) +"Strategy" <-- "Ceilometer API": aggregated metrics +end +"Strategy" -> "Strategy" : compute solution to achieve goal +"Watcher Decision Engine" <-- "Strategy" : solution = array of actions (i.e. not scheduled yet) +create "Watcher Planner" +"Watcher Decision Engine" -[#red]> "Watcher Planner": select appropriate actions scheduler (i.e. Planner implementation) +"Watcher Decision Engine" -> "Watcher Planner": schedule(audit_id, solution) +"Watcher Planner" -> "Watcher Planner": schedule actions according to\nscheduling rules/policies +"Watcher Decision Engine" <-- "Watcher Planner": new action_plan +"Watcher Decision Engine" -> "Watcher Database" : save new action_plan in database +"Watcher Decision Engine" -> "Watcher Database" : update audit.state = SUCCEEDED +"AMQP Bus" <[#blue]- "Watcher Decision Engine" : notify new audit state = SUCCEEDED + +@enduml diff --git a/doc/source/images/action_plan_state_machine.png b/doc/source/images/action_plan_state_machine.png new file mode 100644 index 000000000..b8ac3a86f Binary files /dev/null and b/doc/source/images/action_plan_state_machine.png differ diff --git a/doc/source/images/architecture.svg b/doc/source/images/architecture.svg index 48b9b2c03..abd3872bf 100644 --- a/doc/source/images/architecture.svg +++ b/doc/source/images/architecture.svg @@ -41,36 +41,31 @@ - - - - + + + + - - - - - - - + + + + + + + - - - Watcher - Decision Engine + + + Watcher + Decision Engine - - - - - - - - - - + + + + + @@ -160,10 +155,10 @@ - - - - Nova + + + + Nova @@ -174,14 +169,9 @@ - - - - - - - - + + + @@ -191,21 +181,16 @@ - - - - - - - - - Ceilometer API + + + + Ceilometer API - - - + + + @@ -218,38 +203,62 @@ - - - - - - - - - - - - - - - - Strategy + + + + + + + + + + + + + + + + Strategy - - - + + + - - - + + + + + + + + + + + + + + + + Planner + - - - + + + + + + + + + + + + + @@ -260,16 +269,34 @@ - - - - - Cluster History DB + + + + + Cluster Model DB - - - + + + + + Cluster History DB + + + + + + + + + + + + + + + + diff --git a/doc/source/images/audit_state_machine.png b/doc/source/images/audit_state_machine.png new file mode 100644 index 000000000..58c64aa48 Binary files /dev/null and b/doc/source/images/audit_state_machine.png differ diff --git a/doc/source/images/functional_data_model.svg b/doc/source/images/functional_data_model.svg new file mode 100644 index 000000000..8cc5fb5b1 --- /dev/null +++ b/doc/source/images/functional_data_model.svg @@ -0,0 +1,139 @@ + + + + + + + Audit Template + + + + + OpenStack Cluster + + + + + Applies to + + + + + Audit + + + + + gets configuration from + + + + + Goal + + + + + Achieves + + + + + Action Plan + + + + + Generates + + + + + Action + + + + + + is composed of + + + + + + Next action + + + + + + + + + + + + + Administrator + + + + + + Triggers + + + + + Defines Audit configuration in + + + + + + + + + + Customer + + + + + + Consumes resources + + + + + Resources + + + + + + + + + + + + + Modifies + + + + + Launches + + + + + Strategy + + + + + uses + + diff --git a/doc/source/images/sequence_create_and_launch_audit.png b/doc/source/images/sequence_create_and_launch_audit.png new file mode 100644 index 000000000..0e13a4492 Binary files /dev/null and b/doc/source/images/sequence_create_and_launch_audit.png differ diff --git a/doc/source/images/sequence_create_audit_template.png b/doc/source/images/sequence_create_audit_template.png new file mode 100644 index 000000000..57c539aec Binary files /dev/null and b/doc/source/images/sequence_create_audit_template.png differ diff --git a/doc/source/images/sequence_launch_action_plan.png b/doc/source/images/sequence_launch_action_plan.png new file mode 100644 index 000000000..2c7ece74e Binary files /dev/null and b/doc/source/images/sequence_launch_action_plan.png differ diff --git a/doc/source/images/sequence_launch_action_plan_in_applier.png b/doc/source/images/sequence_launch_action_plan_in_applier.png new file mode 100644 index 000000000..0ae17d882 Binary files /dev/null and b/doc/source/images/sequence_launch_action_plan_in_applier.png differ diff --git a/doc/source/images/sequence_trigger_audit_in_decision_engine.png b/doc/source/images/sequence_trigger_audit_in_decision_engine.png new file mode 100644 index 000000000..eb9e1ecaa Binary files /dev/null and b/doc/source/images/sequence_trigger_audit_in_decision_engine.png differ