From 12b2eceb358481d5f63e4c19d5961d1f8373e75a Mon Sep 17 00:00:00 2001 From: Renat Akhmerov Date: Thu, 23 Jan 2020 13:36:33 +0700 Subject: [PATCH] Move FAQ from Wiki to the built-in docs * This patch moves the list of frequently asked questions from https://wiki.openstack.org/w/index.php?title=Mistral&oldid=152120 to the docs. However, it makes some changes because some info is already outdated. Change-Id: I55a80ed2906bb3fae101d9adc044ed028b6a9d24 Partial-Bug: #1779318 --- doc/source/index.rst | 10 +- doc/source/user/faq.rst | 203 ++++++++++++++++++++++++++++++++++++++ doc/source/user/index.rst | 1 + 3 files changed, 212 insertions(+), 2 deletions(-) create mode 100644 doc/source/user/faq.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index 06f49384a..1527fe901 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -9,11 +9,14 @@ Mistral is a workflow service project. It aims to provide a mechanism to define complex graphs of tasks (workflows) in a simple YAML-based language and manage and run them in a scalable and reliable manner. -Quick Overview -============== +Just to Get Started +=================== * :doc:`user/overview`: If you've just started with Mistral, this short article will help you understand the main Mistral ideas and concepts. +* :doc:`user/faq`: Some of the typical questions you have may have already + been answered here. + For End Users ============= @@ -37,6 +40,9 @@ For Administrators and Operators For Developers ============== +* :doc:`developer/contributor/coding_guidelines`: No matter what you're + going to develop regarding Mistral, please read the coding guidelines + we accept in our project. * :doc:`developer/index`: If you want to contribute to the project or write Mistral extensions, please start here. * :doc:`developer/extensions/index`: Read this section if you want to write diff --git a/doc/source/user/faq.rst b/doc/source/user/faq.rst new file mode 100644 index 000000000..c870a27cd --- /dev/null +++ b/doc/source/user/faq.rst @@ -0,0 +1,203 @@ +========================== +Frequently Asked Questions +========================== + +What is Mistral? +---------------- + +Mistral is a task management service. It can also be called Workflow Service. +Most business processes consist of multiple distinct interconnected +steps that need to be executed in a particular order. One can describe such +process as a set of tasks and task relations and upload such description to +Mistral so that it takes care of state management, correct execution order, +task distribution and high availability. Mistral also provides flexible task +scheduling so that we can run a process according to a specified schedule +(i.e. every Sunday at 4.00pm) instead of running it immediately. We call such +set of tasks and dependencies between them a workflow. Independent routes are +called flows and Mistral can execute them in parallel. + +Who are Mistral users? +---------------------- + +Potential Mistral users are: Developers. Both who work on OpenStack services +and those running in tenant’s VMs. Developers use Mistral DSL/API to access +it. System integrators. They customize workflows related with deployment +using either special scripts or manually using Mistral CLI/UI. System +administrators can use Mistral via additional toolset for common +administrative tasks. This can be distributed cron, mass deployment tasks, +backups etc. + +How does Mistral relate to OpenStack? +------------------------------------- + +Mistral was original started within the OpenStack community. It is still +used within a number of OpenStack projects for various purposes. Mistral +has integration with OpenStack: authentication/authorization with Keystone +identity service and actions to interact with all major OpenStack services +like Nova, Neutron, Heat etc. + +Why offload business processes to a 3rd party service? +------------------------------------------------------ + +* *Reason 1*: **High Availability**. A typical application’s workflow consists + of many independent tasks like collecting data, processing, resource + acquiring, obtaining user input, reporting, sending notifications, + replicating data etc. All of the steps must happen in appropriate time as + they depend on each other. Many such processes can run in parallel. Now if + your application crashes somewhere in the middle or a power outage occurs + your business process terminates at unknown stage in an unknown state. So + you need to track a state of every single flow in some external persistent + storage like database so that you can resume it (or roll it back) from the + place it crashed. You also need some health monitoring tool that would watch + your app and if it crashed schedule unfinished flows on another instance. + This is exactly what Mistral can do out of the box without reinventing the + wheel for each application time and time again. +* *Reason 2*: **Scalability**. Most workflows have steps that can be performed + in parallel (i.e. different routes in a workflow). Mistral can distribute + execution of such tasks across your application’s instances so that the + whole execution would scale. +* *Reason 3*: **Observable state**. Because flow state is tracked outside + of application it becomes observable. At any given moment system + administrator can access information on what is currently going on, what + tasks are in pending state and what has already been executed. You can + obtain metrics on your business processes and profile them. +* *Reason 4*: **Scheduling**. Using Mistral you can schedule your process + to be run periodically or at a fixed moment in future. You can have your + execution to be triggered on alarm condition from an external health + monitoring system or upon a new email in your mailbox. +* *Reason 5*: **Dependency management offloading**. Because you offload task + management to an external service you don’t have to specify all the + triggers and actions in advance. For example, you may say “here is the + task that must be triggered if my domain is down for 1 minute” without + specifying how exactly the event is obtained. System administrator can + setup Nagios to watch your domain and trigger the action and replace it + later with Ceilometer without your application being affected or even + aware of the change. Administrator can even manually trigger the task + using CLI or UI console. Or another example is having a task that triggers + each time a flow reaches some desired state and let administrator configure + what exactly needs to happen there (like send a notification mail and later + replace it with SMS). +* *Reason 6*: **Open additional points for integration**. As soon as your + business process is converted to a Mistral workflow that can be accessed + by others, other application can setup their own workflow to be triggered + by your application reaching a certain state. For example suppose + OpenStack Nova would declare a workflow for new VM instance spawning. + One application (or system administrator) can hook to a task “finish” + so that every time Nova spawns another instance you would receive + a notification. Or suppose you want your users to have flexible quotas + on how many instances one can spawn based on information in external + billing system. Normally you would have to patch Nova to access your + billing system but with Mistral you can just alter Nova’s workflow so + that it includes your custom tasks that would do it instead. +* *Reason 7*: + **Formalized graphs of tasks are just easier to manage and understand**. + They can be visualized, analyzed and optimized. They simplify program + development and debugging. You can model program workflows, replace task + actions with stubs, easily mock external dependencies, do task profiling. + +Why not use Celery or something similar? +---------------------------------------- + +While Celery is a distributed task engine it was designed to execute custom +Python code on pre-installed private workers. Again, this is a different use +case from Mistral, which assumes that tasks can be executed on a shared +service and they do not require (or allow) custom code uploaded in advance. +In other words, Celery itself could be implemented on top of Mistral, if it +was started now. + +How does Mistral relate to Amazon SWF? +-------------------------------------- + +Amazon SWF shares many ideas with Mistral but, in fact, is designed to be +language-oriented (Java, Ruby, Python). It is hard and mostly meaningless +to use SWF without its, for example, Java SDK that exposes its functionality +as a set of Java annotations and interfaces. In this sense SWF is closer to +Celery than to Mistral. Mistral on the other hand aims to be both simpler +and more user-friendly. We want to have a service that is usable without +an SDK in any programming language. At the same time it’s always possible +to implement additional convenient language-oriented bindings based on cool +features like Python decorators, Java annotations and aspects. + +How do I make Mistral know about my workflows? +---------------------------------------------- + +Workflows are described using the Mistral Workflow Language based on YAML. +There is a REST API that is used to upload workflows, execute them and make +run-time modifications against them. The Workflow Language describes + +* Workflows +* Tasks +* Transitions between tasks (what should run next once a task completed). + Applicable for "direct" workflow type. +* Dependencies between tasks (what tasks need to be run before this task + can be executed). Applicable for "reverse" workflow type. +* Various policies applied to how tasks should run. For example, "retry" + policies helps with running a task multiple times in case of failures. +* Ad-hoc actions that can be used for to transform input or output of other + actions for convenience. + +What are Mistral tasks? +----------------------- + +Tasks are entities written with the Mistral Workflow Language that define +certain workflow steps. Each task has: + +* Name +* Optional tag names +* List of tasks it depends on for reverse workflows or list of transitions + for direct workflows +* Optional YAQL expression that extracts data from current data context so + that it would go as a task execution input +* Optional task action (concrete work to do) +* Optional task workflow. If specified, such task is associated with another + workflow execution (subworkflow). + +What are Mistral Workflows? +--------------------------- + +A set of tasks and rules according to which these tasks run. Each workflow +is designed to solve a certain domain problem like auto-scaling a web +application. + +What are Mistral Workbooks? +--------------------------- + +Workbook is a convenience bag to carry multiple workflows and ad-hoc actions +within a single file. Workbooks can also be used like namespaces. + +What are Mistral actions and how does Mistral execute them? +----------------------------------------------------------- + +Action is what to do when a particular task runs. Examples are: + +* Run a shell script +* Send an email +* Call your app’s URI. Send an AMQP (RabbitMQ) message to some queue. +* Other types of signaling (email, UDP message, polling etc.). + +Mistral can be extended to include other general purpose actions like +calling Puppet, Chef, Ansible etc. etc. + +Is it possible to organize a data flow between different tasks in Mistral? +-------------------------------------------------------------------------- + +Yes, tasks belonging to the same workflow can take some input as a json +structure, query a subset of this structure interesting for this particular +task using YAQL expression (https://pypi.python.org/pypi/yaql) and pass it +along to a corresponding action. Once the action has done its processing it +returns the result back using similar json format. So in this case Mistral +acts as a data flow hub dispatching results of one tasks to inputs of other +tasks. + +Does Mistral provide a mechanism to run nested workflows? +--------------------------------------------------------- + +Instead of performing a concrete action associated with a task Mistral can +start a nested workflow. That is, given the input that came to the task, +Mistral starts a new workflow with that input and after completion execution +jumps back to the parent workflow and continues from the same point. The +closest analogy in programming would be calling one method from another +passing all required parameters and optionally getting back a result. It’s +worth noting that the nested workflow works in parallel with the rest of the +activities belonging to the parent execution and it has its own isolated +execution context observable via API. diff --git a/doc/source/user/index.rst b/doc/source/user/index.rst index 6c34f056b..6bf7721f0 100644 --- a/doc/source/user/index.rst +++ b/doc/source/user/index.rst @@ -16,6 +16,7 @@ info on concrete features. :maxdepth: 2 overview + faq terminology/index main_features wf_namespaces