2016-11-25 11:00:57 +01:00
|
|
|
|
========================
|
|
|
|
|
Team and repository tags
|
|
|
|
|
========================
|
|
|
|
|
|
2017-04-17 19:09:53 +08:00
|
|
|
|
.. image:: https://governance.openstack.org/badges/mistral.svg
|
|
|
|
|
:target: https://governance.openstack.org/reference/tags/index.html
|
2016-11-25 11:00:57 +01:00
|
|
|
|
|
2013-10-11 15:18:44 +04:00
|
|
|
|
Mistral
|
2013-10-11 03:26:15 -07:00
|
|
|
|
=======
|
|
|
|
|
|
2017-01-19 16:15:39 +05:30
|
|
|
|
Workflow Service for OpenStack cloud. This project aims to provide a mechanism
|
|
|
|
|
to define tasks and workflows without writing code, manage and execute them in
|
|
|
|
|
the cloud environment.
|
2013-11-29 12:04:12 +07:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
Installation
|
|
|
|
|
~~~~~~~~~~~~
|
2013-11-29 12:04:12 +07:00
|
|
|
|
|
2017-01-19 16:15:39 +05:30
|
|
|
|
The following are the steps to install Mistral on debian-based systems.
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2017-04-26 18:18:48 +05:30
|
|
|
|
To install Mistral, you have to install the following prerequisites::
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2017-01-19 16:15:39 +05:30
|
|
|
|
$ apt-get install python-dev python-setuptools libffi-dev \
|
2017-04-26 18:18:48 +05:30
|
|
|
|
libxslt1-dev libxml2-dev libyaml-dev libssl-dev
|
2014-12-25 16:35:47 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
**Mistral can be used without authentication at all or it can work with
|
|
|
|
|
OpenStack.**
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
In case of OpenStack, it works **only with Keystone v3**, make sure **Keystone
|
|
|
|
|
v3** is installed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Install Mistral
|
|
|
|
|
---------------
|
2013-11-29 12:04:12 +07:00
|
|
|
|
|
2014-12-25 16:35:47 +03:00
|
|
|
|
First of all, clone the repo and go to the repo directory::
|
2014-11-12 08:52:44 +04:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ git clone https://git.openstack.org/openstack/mistral.git
|
|
|
|
|
$ cd mistral
|
2014-11-12 08:52:44 +04:00
|
|
|
|
|
|
|
|
|
|
2014-12-05 14:22:55 +03:00
|
|
|
|
**Devstack installation**
|
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
Information about how to install Mistral with devstack can be found
|
2017-07-19 17:02:16 +08:00
|
|
|
|
`here <https://docs.openstack.org/mistral/latest/contributor/devstack.html>`_.
|
2014-11-12 08:52:44 +04:00
|
|
|
|
|
2014-12-05 14:22:55 +03:00
|
|
|
|
Configuring Mistral
|
2016-03-09 03:27:07 -05:00
|
|
|
|
~~~~~~~~~~~~~~~~~~~
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
Mistral configuration is needed for getting it work correctly with and without
|
|
|
|
|
an OpenStack environment.
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
#. Install and configure a database which can be *MySQL* or *PostgreSQL*
|
|
|
|
|
(**SQLite can't be used in production.**). Here are the steps to connect
|
|
|
|
|
Mistral to a *MySQL* database.
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
* Make sure you have installed ``mysql-server`` package on your Mistral
|
|
|
|
|
machine.
|
|
|
|
|
* Install *MySQL driver* for python::
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ pip install mysql-python
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
or, if you work in virtualenv, run::
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ tox -evenv -- pip install mysql-python
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-06-21 17:18:04 +07:00
|
|
|
|
NOTE: If you're using Python 3 then you need to install ``mysqlclient``
|
|
|
|
|
instead of ``mysql-python``.
|
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
* Create the database and grant privileges::
|
2014-08-27 12:32:05 +04:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ mysql -u root -p
|
2017-07-25 19:36:55 +05:30
|
|
|
|
mysql> CREATE DATABASE mistral;
|
2017-08-08 16:11:17 +08:00
|
|
|
|
mysql> USE mistral;
|
2017-07-25 19:36:55 +05:30
|
|
|
|
mysql> GRANT ALL PRIVILEGES ON mistral.* TO 'mistral'@'localhost' IDENTIFIED BY 'MISTRAL_DBPASS';
|
|
|
|
|
mysql> GRANT ALL PRIVILEGES ON mistral.* TO 'mistral'@'%' IDENTIFIED BY 'MISTRAL_DBPASS';
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
#. Generate ``mistral.conf`` file::
|
2015-07-02 17:04:01 +08:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ oslo-config-generator \
|
|
|
|
|
--config-file tools/config/config-generator.mistral.conf \
|
2016-12-19 00:10:00 -05:00
|
|
|
|
--output-file etc/mistral.conf.sample
|
|
|
|
|
|
|
|
|
|
#. Copy service configuration files::
|
|
|
|
|
|
|
|
|
|
$ sudo mkdir /etc/mistral
|
|
|
|
|
$ sudo chown `whoami` /etc/mistral
|
2017-06-09 03:26:19 -04:00
|
|
|
|
$ cp etc/event_definitions.yml.sample /etc/mistral/event_definitions.yml
|
2016-12-19 00:10:00 -05:00
|
|
|
|
$ cp etc/logging.conf.sample /etc/mistral/logging.conf
|
|
|
|
|
$ cp etc/policy.json /etc/mistral/policy.json
|
|
|
|
|
$ cp etc/wf_trace_logging.conf.sample /etc/mistral/wf_trace_logging.conf
|
|
|
|
|
$ cp etc/mistral.conf.sample /etc/mistral/mistral.conf
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2017-07-10 16:51:58 +05:30
|
|
|
|
#. Edit file ``/etc/mistral/mistral.conf`` according to your setup. Pay attention
|
|
|
|
|
to the following sections and options::
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
[oslo_messaging_rabbit]
|
|
|
|
|
rabbit_host = <RABBIT_HOST>
|
|
|
|
|
rabbit_userid = <RABBIT_USERID>
|
|
|
|
|
rabbit_password = <RABBIT_PASSWORD>
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
[database]
|
|
|
|
|
# Use the following line if *PostgreSQL* is used
|
|
|
|
|
# connection = postgresql://<DB_USER>:<DB_PASSWORD>@localhost:5432/mistral
|
|
|
|
|
connection = mysql://<DB_USER>:<DB_PASSWORD>@localhost:3306/mistral
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
#. If you are not using OpenStack, add the following entry to the
|
2016-11-23 19:35:26 +08:00
|
|
|
|
``/etc/mistral/mistral.conf`` file and **skip the following steps**::
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
[pecan]
|
|
|
|
|
auth_enable = False
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
#. Provide valid keystone auth properties::
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
[keystone_authtoken]
|
2017-07-10 16:51:58 +05:30
|
|
|
|
auth_uri = http://keystone-host:port/v3
|
|
|
|
|
identity_uri = http://keystone-host:port
|
2016-03-09 03:27:07 -05:00
|
|
|
|
auth_version = v3
|
|
|
|
|
admin_user = <user>
|
|
|
|
|
admin_password = <password>
|
|
|
|
|
admin_tenant_name = <tenant>
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
#. Register Mistral service and Mistral endpoints on Keystone::
|
2015-08-09 23:20:00 +08:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ MISTRAL_URL="http://[host]:[port]/v2"
|
|
|
|
|
$ openstack service create --name mistral workflowv2
|
2016-12-19 00:10:00 -05:00
|
|
|
|
$ openstack endpoint create mistral public $MISTRAL_URL
|
|
|
|
|
$ openstack endpoint create mistral internal $MISTRAL_URL
|
|
|
|
|
$ openstack endpoint create mistral admin $MISTRAL_URL
|
2015-08-09 23:20:00 +08:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
#. Update the ``mistral/actions/openstack/mapping.json`` file which contains
|
|
|
|
|
all available OpenStack actions, according to the specific client versions
|
|
|
|
|
of OpenStack projects in your deployment. Please find more detailed
|
|
|
|
|
information in the ``tools/get_action_list.py`` script.
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
Before the First Run
|
2017-01-19 16:15:39 +05:30
|
|
|
|
--------------------
|
2014-08-27 12:32:05 +04:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
After local installation you will find the commands ``mistral-server`` and
|
|
|
|
|
``mistral-db-manage`` available in your environment. The ``mistral-db-manage``
|
|
|
|
|
command can be used for migrating database schema versions. If Mistral is not
|
|
|
|
|
installed in system then this script can be found at
|
|
|
|
|
``mistral/db/sqlalchemy/migration/cli.py``, it can be executed using Python
|
|
|
|
|
command line.
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
To update the database schema to the latest revision, type::
|
2014-08-27 12:32:05 +04:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ mistral-db-manage --config-file <path_to_config> upgrade head
|
2014-08-27 12:32:05 +04:00
|
|
|
|
|
2017-01-19 16:15:39 +05:30
|
|
|
|
To populate the database with standard actions and workflows, type::
|
|
|
|
|
$ mistral-db-manage --config-file <path_to_config> populate
|
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
For more detailed information about ``mistral-db-manage`` script please check
|
|
|
|
|
file ``mistral/db/sqlalchemy/migration/alembic_migrations/README.md``.
|
2014-12-05 14:22:55 +03:00
|
|
|
|
|
2014-04-30 10:24:27 +04:00
|
|
|
|
Running Mistral API server
|
2017-01-19 16:15:39 +05:30
|
|
|
|
--------------------------
|
2014-04-30 10:24:27 +04:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
To run Mistral API server::
|
2013-11-29 12:04:12 +07:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ tox -evenv -- python mistral/cmd/launch.py \
|
2017-04-26 18:18:48 +05:30
|
|
|
|
--server api --config-file <path_to_config>
|
2013-12-20 18:38:13 +07:00
|
|
|
|
|
2014-04-30 10:24:27 +04:00
|
|
|
|
Running Mistral Engines
|
2017-01-19 16:15:39 +05:30
|
|
|
|
-----------------------
|
2014-04-30 10:24:27 +04:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
To run Mistral Engine::
|
2014-04-19 17:31:04 -07:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ tox -evenv -- python mistral/cmd/launch.py \
|
2017-04-26 18:18:48 +05:30
|
|
|
|
--server engine --config-file <path_to_config>
|
2014-04-19 17:31:04 -07:00
|
|
|
|
|
2014-04-30 10:24:27 +04:00
|
|
|
|
Running Mistral Task Executors
|
2017-01-19 16:15:39 +05:30
|
|
|
|
------------------------------
|
2013-12-20 18:38:13 +07:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
To run Mistral Task Executor instance::
|
2013-12-20 18:38:13 +07:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
$ tox -evenv -- python mistral/cmd/launch.py \
|
2017-04-26 18:18:48 +05:30
|
|
|
|
--server executor --config-file <path_to_config>
|
2013-11-29 12:04:12 +07:00
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
Note that at least one Engine instance and one Executor instance should be
|
|
|
|
|
running in order for workflow tasks to be processed by Mistral.
|
|
|
|
|
|
|
|
|
|
If you want to run some tasks on specific executor, the *task affinity* feature
|
|
|
|
|
can be used to send these tasks directly to a specific executor. You can edit
|
|
|
|
|
the following property in your mistral configuration file for this purpose::
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
|
|
|
|
[executor]
|
|
|
|
|
host = my_favorite_executor
|
|
|
|
|
|
2016-03-09 03:27:07 -05:00
|
|
|
|
After changing this option, you will need to start (restart) the executor. Use
|
|
|
|
|
the ``target`` property of a task to specify the executor::
|
2014-12-08 17:13:35 +03:00
|
|
|
|
|
|
|
|
|
... Workflow YAML ...
|
|
|
|
|
task1:
|
|
|
|
|
...
|
2015-04-07 11:27:24 +03:00
|
|
|
|
target: my_favorite_executor
|
2014-12-08 17:13:35 +03:00
|
|
|
|
... Workflow YAML ...
|
|
|
|
|
|
2014-05-14 09:58:22 -07:00
|
|
|
|
Running Multiple Mistral Servers Under the Same Process
|
2017-01-19 16:15:39 +05:30
|
|
|
|
-------------------------------------------------------
|
2016-03-09 03:27:07 -05:00
|
|
|
|
|
|
|
|
|
To run more than one server (API, Engine, or Task Executor) on the same
|
|
|
|
|
process::
|
|
|
|
|
|
|
|
|
|
$ tox -evenv -- python mistral/cmd/launch.py \
|
2017-04-26 18:18:48 +05:30
|
|
|
|
--server api,engine --config-file <path_to_config>
|
2016-03-09 03:27:07 -05:00
|
|
|
|
|
|
|
|
|
The value for the ``--server`` option can be a comma-delimited list. The valid
|
|
|
|
|
options are ``all`` (which is the default if not specified) or any combination
|
|
|
|
|
of ``api``, ``engine``, and ``executor``.
|
|
|
|
|
|
|
|
|
|
It's important to note that the ``fake`` transport for the ``rpc_backend``
|
|
|
|
|
defined in the configuration file should only be used if ``all`` Mistral
|
|
|
|
|
servers are launched on the same process. Otherwise, messages do not get
|
|
|
|
|
delivered because the ``fake`` transport is using an in-process queue.
|
2014-05-14 09:58:22 -07:00
|
|
|
|
|
2017-04-13 18:11:09 +07:00
|
|
|
|
Project Goals 2017
|
|
|
|
|
------------------
|
|
|
|
|
|
|
|
|
|
#. **Complete Mistral documentation**.
|
|
|
|
|
|
|
|
|
|
Mistral documentation should be more usable. It requires focused work to
|
2017-06-05 16:58:29 +05:30
|
|
|
|
make it well structured, eliminate gaps in API/Mistral Workflow Language
|
|
|
|
|
specifications, add more examples and tutorials.
|
2017-04-13 18:11:09 +07:00
|
|
|
|
|
|
|
|
|
*Definition of done*:
|
|
|
|
|
All capabilities are covered, all documentation topics are written using
|
|
|
|
|
the same style and structure principles. The obvious sub-goal of this goal
|
|
|
|
|
is to establish these principles.
|
|
|
|
|
|
|
|
|
|
#. **Complete Mistral Custom Actions API**.
|
|
|
|
|
|
|
|
|
|
There has been the initiative in Mistral team since April of 2016 to
|
|
|
|
|
refactor Mistral actions subsystem in order to make the process of
|
|
|
|
|
developing Mistral actions easier and clearer. In 2017 we need to complete
|
|
|
|
|
this effort and make sure that all APIs are stable and it’s well-documented.
|
|
|
|
|
|
|
|
|
|
*Definition of done*:
|
|
|
|
|
All API interfaces are stable, existing actions are rewritten using this new
|
|
|
|
|
API, OpenStack actions are also rewritten based on the new API and moved to
|
|
|
|
|
mistral-extra repo. Everything is well documented and the doc has enough
|
|
|
|
|
examples.
|
|
|
|
|
|
|
|
|
|
#. **Finish Mistral multi-node mode**.
|
|
|
|
|
|
|
|
|
|
Mistral needs to be proven to work reliably in multi-node mode. In order
|
|
|
|
|
to achieve it we need to make a number of engine, executor and RPC
|
|
|
|
|
changes and configure a CI gate to run stress tests on multi-node Mistral.
|
|
|
|
|
|
|
|
|
|
*Definition of done*:
|
|
|
|
|
CI gate supports MySQL, all critically important functionality (join,
|
|
|
|
|
with-items, parallel workflows, sequential workflows) is covered by tests.
|
|
|
|
|
|
|
|
|
|
#. **Reduce workflow execution time**.
|
|
|
|
|
|
|
|
|
|
*Definition of done*: Average workflow execution time reduced by 30%.
|
|
|
|
|
|
2017-01-19 16:15:39 +05:30
|
|
|
|
Project Resources
|
|
|
|
|
-----------------
|
2014-05-14 09:58:22 -07:00
|
|
|
|
|
2017-07-19 17:02:16 +08:00
|
|
|
|
* `Mistral Official Documentation <https://docs.openstack.org/mistral/latest/>`_
|
2014-12-25 16:35:47 +03:00
|
|
|
|
|
2017-01-19 16:15:39 +05:30
|
|
|
|
* Project status, bugs, and blueprints are tracked on
|
|
|
|
|
`Launchpad <https://launchpad.net/mistral/>`_
|
2014-09-23 19:03:35 +04:00
|
|
|
|
|
2017-01-19 16:15:39 +05:30
|
|
|
|
* Additional resources are linked from the project
|
|
|
|
|
`Wiki <https://wiki.openstack.org/wiki/Mistral/>`_ page
|
2014-09-23 19:03:35 +04:00
|
|
|
|
|
2017-01-19 16:15:39 +05:30
|
|
|
|
* Apache License Version 2.0 http://www.apache.org/licenses/LICENSE-2.0
|