Mistral install guide

First attempt at creating the mistral install guide

Change-Id: I30142b46e36270b573b9ec10201907811040d94b
Signed-off-by: visnyei <andrea.visnyei@nokia.com>

doc/source/configuration/index.rst

@@ -1,9 +1,9 @@
 Mistral Configuration and Policy Guide
---------------------------------------
+======================================
 
 .. toctree::
-    :maxdepth: 2
+   :maxdepth: 2
 
-    config-guide.rst
-    policy-guide.rst
-    samples/index.rst
+   ../configuration/config-guide.rst
+   ../configuration/policy-guide.rst
+   ../configuration/samples/index.rst

doc/source/install/get_started.rst

@@ -0,0 +1,37 @@
+=================================
+Workflow service overview
+=================================
+
+Mistral is a workflow service that enables setting up and scheduling tasks and
+task relations that have to be executed in a particular order, called workflows.
+During execution, Mistral handles state management, correct execution order,
+parallelism, synchronization and high availability.
+
+Mistral can be used, for example, for Cloud Cron-style task scheduling,
+Cloud Environment Deployment, handling long-running business processes, managing
+BigData analysis and tasks, and live migration.
+
+The Workflow service consists of the following components:
+
+``Mistral API`` service
+  Provides a REST API for operating and monitoring workflow executions.
+
+``mistral-dashboard`` service
+  Mistral Dashboard is a Horizon plugin.
+
+``Mistral Engine`` service
+  Controls workflow executions and handles their data flow, places finished
+  tasks in a queue, transfers data from task to task, and deals with condition
+  transitions, and so on.
+
+``Mistral Executor`` service
+  Executes task actions, picks up the tasks from the queue, runs actions, and
+  sends results back to the engine.
+
+``Mistral Notifier`` service
+
+``python-mistralclient``
+  Python client API and Command Line Interface.
+
+  ``mistral-lib``
+  A library to support writing custom Mistral actions.

doc/source/install/index.rst

@@ -2,8 +2,19 @@ Mistral Installation Guide
 ==========================
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
 
-   installation_guide
+   get_started.rst
+   install.rst
+   verify.rst
+   next-steps.rst
    dashboard_guide
+   installation_guide
    mistralclient_guide
+
+The Workflow service (mistral) enables setting up task relations that have to be
+ executed in a particular order, called workflows.
+
+This chapter assumes a working setup of OpenStack following the
+`OpenStack Installation Tutorial
+<https://docs.openstack.org/install-guide/>`_.

doc/source/install/install-obs.rst

@@ -0,0 +1,10 @@
+.. _install-obs:
+
+
+Install and configure for openSUSE and SUSE Linux Enterprise
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For information on how to install and configure the Workflow service
+for openSUSE and SUSE Linux Enterprise, refer to the :doc:`Installation guide
+for Ubuntu <install-ubuntu>`. Note that some commands vary by distribution and
+might differ from the ones described, for instance, package management.

doc/source/install/install-rdo.rst

@@ -0,0 +1,10 @@
+.. _install-rdo:
+
+Install and configure for Red Hat Enterprise Linux and CentOS
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+
+For information on how to install and configure the Workflow service
+for Red Hat Enterprise Linux 7 and CentOS 7, refer to the :doc:`Installation
+guide for Ubuntu <install-ubuntu>`. Note that some commands vary by distribution
+and might differ from the ones described, for instance, package management.

doc/source/install/install-ubuntu.rst

@@ -0,0 +1,209 @@
+.. _install-ubuntu:
+
+Install and configure for Ubuntu
+================================
+
+This section describes how to install and configure the Workflow Service
+service for Ubuntu.
+
+
+
+Prerequisites
+-------------
+
+#. Install the packages:
+
+   .. code-block:: console
+
+      # apt-get update
+
+      # apt-get install python-setuptools python-pip libffi-dev libxslt1-dev \
+        libxml2-dev libyaml-dev libssl-dev python3-dev tox mistral-common
+
+
+
+Installation
+------------
+
+**NOTE**: For instructions on how to install Mistral using devstack, refer to
+:doc:`Mistral Devstack Installation </contributor/devstack>`
+
+Clone the repo and go to the repo directory:
+
+.. code-block:: console
+
+    $ git clone https://git.openstack.org/openstack/mistral
+    $ cd mistral
+
+Generate the configuration file:
+
+.. code-block:: console
+
+    $ tox -egenconfig
+
+Create the mistral directory and copy the example configuration file:
+
+.. code-block:: console
+
+    $ mkdir /etc/mistral
+    $ cp etc/mistral.conf.sample /etc/mistral/mistral.conf
+
+
+Edit the configuration file:
+
+ .. code-block:: console
+
+    $ vi /etc/mistral/mistral.conf
+
+
+
+**Virtualenv installation**:
+
+.. code-block:: console
+
+    $ tox
+
+This installs the necessary virtual environments and runs all the project
+tests. Installing the virtual environments may take significant time (~10-15
+mins).
+
+**Local installation**:
+
+.. code-block:: console
+
+    $ pip install -e .
+
+or:
+
+.. code-block:: console
+
+    $ pip install -r requirements.txt
+    $ python setup.py install
+
+**NOTE**: There are some differences between *pip install -e* and *setup.py
+install*. **pip install -e** works similarly to **setup.py install**
+or the EasyInstall tool, however, it does not actually install anything.
+Instead, it creates a special .egg-link file in the deployment directory that
+links to your project’s source code.
+
+
+.. include:: ../configuration/index.rst
+
+
+Before the first run
+--------------------
+
+After the installation, you will see the **mistral-server** and
+**mistral-db-manage** commands in your environment, either in system or virtual
+environment.
+
+**NOTE**: If you use **virtualenv**, all Mistral-related commands can be
+accessed with **tox -evenv --**. For example, *mistral-server* is available via
+*tox -evenv -- mistral-server*.
+
+The **mistral-db-manage** command can be used for migrations.
+
+Updating the database to the latest revision type:
+
+.. code-block:: console
+
+    $ mistral-db-manage --config-file <path-to-mistral.conf> upgrade head
+
+Before starting the Mistral server, run the *mistral-db-manage populate*
+command. It creates the DB with all the standard actions and standard workflows
+that Mistral provides to all Mistral users.:
+
+.. code-block:: console
+
+    $ mistral-db-manage --config-file <path-to-mistral.conf> populate
+
+For more detailed information on the *mistral-db-manage* script, see
+the :doc:`Mistral Upgrade Guide </admin/upgrade_guide>`.
+
+**NOTE**: For users who want a dry run with an **SQLite** database backend (not
+used in production), the *mistral-db-manage* script is not recommended for
+database initialization because of
+`SQLite limitations <http://www.sqlite.org/omitted.html>`_.
+Use the sync_db script described below for database
+initialization instead.
+
+**If you use virtualenv**:
+
+.. code-block:: console
+
+    $ tools/sync_db.sh --config-file <path-to-mistral.conf>
+
+**Or run sync_db directly**:
+
+.. code-block:: console
+
+    $ python tools/sync_db.py --config-file <path-to-mistral.conf>
+
+Running Mistral API server
+--------------------------
+
+To run the Mistral API server, execute the following command in a shell:
+
+.. code-block:: console
+
+    $ mistral-server --server api --config-file <path-to-mistral.conf>
+
+Running Mistral Engines
+-----------------------
+
+To run the Mistral Engine, execute the following command in a shell:
+
+.. code-block:: console
+
+    $ mistral-server --server engine --config-file <path-to-mistral.conf>
+
+Running Mistral Executors
+-------------------------
+To run the Mistral Executor instance, execute the following command in a
+shell:
+
+.. code-block:: console
+
+    $ mistral-server --server executor --config-file <path-to-mistral.conf>
+
+Note that at least one Engine instance and one Executor instance should be
+running so that workflow tasks are processed by Mistral.
+
+Mistral Notifier
+----------------
+
+To run the Mistral Notifier, execute the following command in a shell:
+
+.. code-block:: console
+
+    $ mistral-server --server notifier --config-file <path-to-mistral.conf>
+
+Running Multiple Mistral Servers Under the Same Process
+-------------------------------------------------------
+To run more than one server (API, Engine, or Task Executor) on the same process,
+execute the following command in a shell:
+
+.. code-block:: console
+
+    $ mistral-server --server api,engine --config-file <path-to-mistral.conf>
+
+The --server command line option can be a comma delimited list. The valid
+options are "all" (by default if not specified) or any combination of "api",
+"engine", and "executor". It is important to note that the "fake" transport for
+the rpc_backend defined in the config file should only be used if "all" the
+Mistral servers are launched on the same process. Otherwise, messages do not
+get delivered if the Mistral servers are launched on different processes
+because the "fake" transport is using an in-process queue.
+
+
+.. include:: mistralclient_guide.rst
+
+
+Finalize installation
+---------------------
+
+Restart the Workflow services:
+
+.. code-block:: console
+
+   # service openstack-mistral-api restart

doc/source/install/install.rst

@@ -0,0 +1,24 @@
+.. _install:
+
+Install and configure
+~~~~~~~~~~~~~~~~~~~~~
+
+This section describes how to install and configure the
+Workflow Service, code-named mistral, on the controller node.
+
+**NOTE:** **Mistral can be used in standalone mode or it can work
+with OpenStack.**
+
+If Mistral is used with OpenStack, you must already have a working OpenStack
+environment with at least the following components installed:
+
+- Keystone with API v3 support
+
+Note that installation and configuration vary by distribution.
+
+.. toctree::
+   :maxdepth: 2
+
+   install-ubuntu.rst
+   install-rdo.rst
+   install-obs.rst

doc/source/install/mistralclient_guide.rst

@@ -1,5 +1,5 @@
 Mistral Client Installation Guide
-=================================
+---------------------------------
 
 To install ``python-mistralclient``, it is required to have ``pip``
 (in most cases). Make sure that ``pip`` is installed. Then type::
@@ -15,7 +15,7 @@ After ``python-mistralclient`` is installed you will see command ``mistral``
 in your environment.
 
 Configure authentication against Keystone
------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 If Keystone is used for authentication in Mistral, then the environment should
 have auth variables::
@@ -43,7 +43,7 @@ To make sure Mistral client works, type::
     $ mistral workbook-list
 
 Configure authentication against Keycloak
------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Mistral also supports authentication against Keycloak server via OpenID Connect
 protocol.
@@ -80,7 +80,7 @@ to True::
     $ export MISTRALCLIENT_INSECURE=True
 
 Targeting non-preconfigured clouds
-----------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^-
 
 Mistral is capable of executing workflows on external OpenStack clouds,
 different from the one defined in the `mistral.conf` file in the
@@ -114,7 +114,7 @@ service. Whereas the `OS_TARGET_*` parameters are used to define the user that
 executes the workflow on the external cloud, keystone2.example.com/.
 
 Use cases
-^^^^^^^^^
+"""""""""
 
 **Authenticate in Mistral and execute OpenStack actions with different users**
 
@@ -126,7 +126,7 @@ cloud.
 As a user of Mistral, I want to execute a workflow on a cloud of my choice.
 
 Special cases
-^^^^^^^^^^^^^
+"""""""""""""
 
 **Using Mistral with zero OpenStack configuration**:
 

doc/source/install/next-steps.rst

@@ -0,0 +1,9 @@
+.. _next-steps:
+
+Next steps
+~~~~~~~~~~
+
+Your OpenStack environment now includes the Mistral service.
+
+To add additional services, see `OpenStack Pike Installation Tutorials and
+Guides <https://docs.openstack.org/install/>`_.

doc/source/install/verify.rst

@@ -0,0 +1,51 @@
+.. _verify:
+
+Basic verification
+~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+   $ mistral run-action std.noop
+
+
+Verify operation of the Workflow service
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. note::
+
+   Perform these commands on the controller node.
+
+#. Create a workflow file:
+
+   .. code-block:: console
+
+      $ cat >/tmp/test.wf.yaml <<EOL
+        ---
+        version: '2.0'
+
+        test_wf:
+          tasks:
+            echo_task:
+              action: std.echo output="Hello"
+        EOL
+
+
+#. Create a workflow from the workflow file:
+
+   .. code-block:: console
+
+      $ mistral workflow-create /tmp/test.wf.yaml
+
+#. Create an execution based on the workflow:
+
+   .. code-block:: console
+
+     $ mistral execution-create test_wf
+
+#. Run the execution until its returning state is 'SUCCESS':
+
+   .. code-block:: console
+
+      $   mistral execution-list
+
+After performing the above steps, the Mistral service is ready for use.

doc/source/main_features.rst

@@ -333,4 +333,3 @@ or ``evaluation_interval`` options.
 
  Defines the maximum number of finished executions.
  It must be greater or equal to ``1``.
-

tox.ini

@@ -108,7 +108,7 @@ enable-extensions = H106,H203,H904
 exclude=.venv,.git,.tox,dist,doc,*lib/python*,*egg,tools,scripts
 
 [doc8]
-extensions = .rst, .yaml
+extensions = .rst, .yaml, .mistral
 # Maximal line length should be 80.
 max-line-length = 80