diff --git a/doc/source/admin/install/install.rst b/doc/source/admin/install/install.rst index ed8290790..f04306baf 100644 --- a/doc/source/admin/install/install.rst +++ b/doc/source/admin/install/install.rst @@ -35,10 +35,12 @@ The Workflow service consists of the following components: ``Mistral Notifier`` service Send notifications based on state of workflow and task executions. + `This service is optional`. ``Mistral Event Engine`` service Create workflow executions based on external events (like RabbitMQ, HTTP, kafka, etc.). + `This service is optional`. The mistral project is also providing the following python libraries: @@ -163,11 +165,11 @@ Update the database to the latest revision: .. code-block:: console # For MySQL / MariaDB / PostgreSQL - $ mistral-db-manage --config-file /etc/mistral/mistral.conf upgrade head + $ mistral-db-manage upgrade head # For SQLite - do not use sqlite in production! # e.g. connection = 'sqlite:////var/lib/mistral.sqlite' - $ python tools/sync_db.py --config-file /etc/mistral/mistral.conf + $ python tools/sync_db.py Before starting the Mistral server, run the *mistral-db-manage populate* command. It creates the DB with all the standard actions and standard workflows @@ -175,7 +177,7 @@ that Mistral provides to all Mistral users.: .. code-block:: console - $ mistral-db-manage --config-file /etc/mistral/mistral.conf populate + $ mistral-db-manage populate For more detailed information on the *mistral-db-manage* script, see the :doc:`Mistral Upgrade Guide `. @@ -188,7 +190,12 @@ To run the Mistral components, execute the following command in a shell: .. code-block:: console - $ mistral-server --server all --config-file /etc/mistral/mistral.conf + $ mistral-server --server all + +.. note:: + + in this situation API will start only one worker! If you need more than + worker for you API, you should start the API with uWSGI (see below) Running Mistral components separately ------------------------------------- @@ -198,14 +205,14 @@ server, e.g. to start only the engine: .. code-block:: console - $ mistral-server --server engine --config-file /etc/mistral/mistral.conf + $ mistral-server --server engine The --server command line option can be a comma delimited list, so you can build combination of components, like this: .. code-block:: console - $ mistral-server --server engine,executor --config-file /etc/mistral/mistral.conf + $ mistral-server --server engine,executor The valid options are: @@ -216,6 +223,102 @@ The valid options are: * event-engine * notifier +Running Mistral API with uWSGI +------------------------------ + +The WSGI application +~~~~~~~~~~~~~~~~~~~~ + +One downside of running ``mistral-server --server api`` directly is that it +will start only one process (worker) to handle HTTP requests. + +While this may be enough for small/dev deployments, it may not for production. + +In that situation, Mistral provides a WSGI application at +``mistral.wsgi:application`` that can be used with any WSGI server. + +The below example uses uWSGI + + +Using uWSGI +~~~~~~~~~~~ + +Install uWSGI: + +.. code-block:: console + + $ pip install uwsgi + + +Create a uWSGI configuration file (e.g., ``/etc/uwsgi/mistral.ini``): + +.. code-block:: cfg + + [uwsgi] + # Listen on port 8989 and start as a full web server + http-socket = 0.0.0.0:8989 + + # Stats on port 9191 + stats = 0.0.0.0:9191 + + # App to start + virtualenv = /opt/openstack/mistral/ + module = mistral.wsgi:application + + # load apps in each worker instead of the master + lazy-apps = true + + # Number of processes + processes = 4 + + # Will kill processes that run more that 60s + harakiri = 60 + + # Enable threads + enable-threads = true + + # Gracefully manage processes + master = true + + # Thunder-lock - serialize accept() usage (if possible) + thunder-lock = true + + +Start uWSGI: + +.. code-block:: console + + $ uwsgi --ini /etc/uwsgi/mistral.ini + + +Passing Configuration Options +------------------------------ + +By default, Mistral will use its standard configuration file search paths: + +* ``/etc/mistral/mistral.conf`` +* ``/etc/mistral/mistral.conf.d/`` +* ``/etc/mistral.conf.d/`` +* many others, see: + https://docs.openstack.org/oslo.config/latest/configuration/options.html + +You can also provide ``config-dir`` or ``config-file`` options to +``mistral-server`` command line to provide a custom file/folder: + +.. code-block:: console + + $ mistral-server --config-dir /etc/mycustomdir/ + +Note that, when using ``uwsgi``, you won't be able to provide such params. In +that situation, you can use ``MISTRAL_CONFIG_DIR`` and/or +``MISTRAL_CONFIG_FILE`` environment variable instead: + +.. code-block:: cfg + + [uwsgi] + ... + env = MISTRAL_CONFIG_DIR=/etc/mycustomdir/ + .. _install-osa: Deploying with OpenStack-Ansible diff --git a/mistral/api/wsgi.py b/mistral/api/wsgi.py index 27866f0c1..d8a564b1a 100644 --- a/mistral/api/wsgi.py +++ b/mistral/api/wsgi.py @@ -12,6 +12,15 @@ # See the License for the specific language governing permissions and # limitations under the License. +# NOTE(amorin) Deprecated wsgi entry point +# use wsgi/ instead now + +# NOTE(amorin) +# Hardcode the threading backend to avoid using eventlet until this will +# eventually become the default +import oslo_service.backend as service_backend +service_backend.init_backend(service_backend.BackendType.THREADING) + from mistral.api import app application = app.init_wsgi() diff --git a/mistral/config.py b/mistral/config.py index 5e1b4c311..46ccaa2cb 100644 --- a/mistral/config.py +++ b/mistral/config.py @@ -21,6 +21,7 @@ Configuration options registration and useful routines. """ import json +import os from keystoneauth1 import loading from oslo_config import cfg @@ -859,7 +860,13 @@ def list_opts(): ] -def parse_args(args=None, usage=None, default_config_files=None): +def parse_args(args=None, usage=None): + # NOTE(amorin) allow overwritting config dir and file from env + conf_d = os.environ.get('MISTRAL_CONFIG_DIR') + default_config_dirs = [conf_d] if conf_d else None + conf_f = os.environ.get('MISTRAL_CONFIG_FILE') + default_config_files = [conf_f] if conf_f else None + default_log_levels = log.get_default_log_levels() default_log_levels.extend(_DEFAULT_LOG_LEVELS) log.set_defaults(default_log_levels=default_log_levels) @@ -871,7 +878,8 @@ def parse_args(args=None, usage=None, default_config_files=None): project='mistral', version=version.version_string, usage=usage, - default_config_files=default_config_files + default_config_dirs=default_config_dirs, + default_config_files=default_config_files, ) diff --git a/mistral/wsgi/__init__.py b/mistral/wsgi/__init__.py new file mode 100644 index 000000000..c691ddd91 --- /dev/null +++ b/mistral/wsgi/__init__.py @@ -0,0 +1,29 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""WSGI application entry-point for Mistral API.""" + +# NOTE(amorin) +# Hardcode the threading backend to avoid using eventlet until this will +# eventually become the default +import oslo_service.backend as service_backend +service_backend.init_backend(service_backend.BackendType.THREADING) + +import threading + +from mistral.api import app + +application = None +lock = threading.Lock() +with lock: + if application is None: + application = app.init_wsgi() diff --git a/releasenotes/notes/config-dir-file-6d36f8d2f2ebb5f9.yaml b/releasenotes/notes/config-dir-file-6d36f8d2f2ebb5f9.yaml new file mode 100644 index 000000000..8b066d496 --- /dev/null +++ b/releasenotes/notes/config-dir-file-6d36f8d2f2ebb5f9.yaml @@ -0,0 +1,12 @@ +--- +other: + - | + Adding a new wsgi entrypoint to allow using it within WSGI servers (like + uWSGI) using ``mistral.wsgi:application``. + + Note that **this is the only way to start more than one mistral API + worker**. + + Also adding two environment variables ``MISTRAL_CONFIG_DIR`` and + ``MISTRAL_CONFIG_FILE`` to allow setting config-dir and config-file when + using the new wsgi entrypoint.