3828 Commits

Author SHA1 Message Date
Vlad Gusev
5d2be3458d Add release note for I04ba85488b27cb05c3b81ad8c973c3cc3fe56d36
Change-Id: I3ad0a7769e2d61665d5ea76b90485e582993ddd2
2019-04-04 16:01:28 +00:00
Ian Wienand
765225b109 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: Ia4da4f94ecada34246a4db95174fa42595f9ff86
2019-04-04 09:16:50 +00:00
Renat Akhmerov
97f4fc2776 Setting the lower version of sphinxcontrib-pecanwsme to 0.10.0
* Sphinx 2.0.0 was released on Mar 28 and sphinxcontrib-pecanwsme
  became incompatible with it. The version 0.10.0 was released
  to fix that.
* See https://review.openstack.org/#/c/649264/

Depends-On: I0378297e099412191a4adc41e53b0eb23f6c43ae
Change-Id: Icb0163f553af690192bfab0e192a32d793db28a9
2019-04-04 05:03:27 +00:00
Renat Akhmerov
d332d7fefe Add release notes for engine optimizations
Change-Id: I9a87c1e2a0bc5a1ba54d7f93f6b1d077088e5023
2019-03-27 14:25:49 +07:00
Zuul
db162f09f4 Merge "Add Python 3.7 to tox" 2019-03-26 11:06:12 +00:00
Zuul
7d4a37d145 Merge "Improve profiler logging" 2019-03-26 10:38:35 +00:00
Zuul
a822c71bb8 Merge "Add http_proxy_to_wsgi middleware" 2019-03-26 05:35:15 +00:00
Renat Akhmerov
d4a4f7a5ee Improve profiler logging
* The custom profiler notifier defined by Mistral now also prints
  the total elapsed time of a profiler trace into the log file .
  This makes it much easier to search for bottlenecks.

Change-Id: I1db7e66e1b3756cccfde73feee83baef9edf283f
2019-03-25 07:53:32 +00:00
3e42cffd6c Update master for stable/stein
Add file to the reno documentation build to show release notes for
stable/stein.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.

Change-Id: Icb8261ce3371296b8a87e5bfa527a5f64a25157c
Sem-Ver: feature
2019-03-22 18:48:52 +00:00
Zuul
3116a15ed2 Merge "Add "root_execution" mapped property to WorkflowExecution model" 2019-03-22 13:23:37 +00:00
Dougal Matthews
218ab5a76e Add Python 3.7 to tox
Change-Id: I612dc2beb9ab14c93e4c0dab775e8868a1bd8c45
2019-03-22 11:11:09 +00:00
Zuul
54c34492d0 Merge "Optimize action scheduling" 2019-03-18 17:19:25 +00:00
Renat Akhmerov
32c96b1b6c Add "root_execution" mapped property to WorkflowExecution model
* We need to avoid using direct DB queries w/o using a mapped
  model where possible for performance sake. Using a mapped entity
  property is always more efficient because it's cached in an
  SQLAlchemy session.

Change-Id: I2d7652ea0cff8f2db7259d285ac98c582bf15b62
2019-03-18 11:19:44 +07:00
Renat Akhmerov
a73e3defea Optimize action scheduling
* If "target" property of a task is empty, there's no need to
  evaluate an expression
* Refactored action input validation logic to avoid preparing
  an action input twice for ad-hoc actions. The method
  "_prepare_input" was previously called twice for ad-hoc actions
  although it may be pretty expensive in some cases.
* Added more profiler decorators to track performance of ad-hoc
  actions

Change-Id: Ia56e2fda57c3af1a86a638018c23dd1c3281debb
2019-03-18 09:28:13 +07:00
Oleg Ovcharuk
99ebc1b5f7 Retries shouldn't execute if join task failed because of child task
Change-Id: Ideaa9938497f74335af633044cb6e98fbb1522d8
Closes-Bug: #1819418
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
8.0.0.0rc1
2019-03-11 22:29:08 +03:00
Vlad Gusev
ca1acb656c Add http_proxy_to_wsgi middleware
This sets up the HTTPProxyToWSGI middleware in front of Mistral API. The
purpose of this middleware is to set up the request URL correctly in
the case there is a proxy (for instance, a loadbalancer such as HAProxy)
in front of the Mistral API.

The HTTPProxyToWSGI is off by default and needs to be enabled via a
configuration value.

It can be enabled with the option in mistral.conf:
[oslo_middleware]
enable_proxy_headers_parsing=True

Closes-Bug: #1590608
Closes-Bug: #1816364
Change-Id: I04ba85488b27cb05c3b81ad8c973c3cc3fe56d36
2019-03-09 01:51:59 +03:00
Zuul
608367f28d Merge "add python 3.7 unit test job" 8.0.0.0b2 2019-02-26 08:08:53 +00:00
Corey Bryant
d660d2b1b8 add python 3.7 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.7.

See ML discussion here [1] for context.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135626.html

Change-Id: I1f3065955283d75df140debaefc4315ee3ed5eaf
Story: #2004073
Task: #27429
2019-02-20 16:09:25 -05:00
Wes Hayutin
43f80eaa9d standalone/undercloud do not cover full mistral workflow
The standalone job is started via a heat process
that skips a portion of the mistral workflow that
is used in overcloud deployments.

An example of missed coverage:
https://bugs.launchpad.net/tripleo/+bug/1816026
https://review.openstack.org/#/c/637522/

containers-multinode is a more appropriate
job to fully cover mistral in tripleo.

Change-Id: Ie94e03bdcc14918eb4095e9581d4a935a59b0d6b
2019-02-20 06:46:41 -07:00
Zuul
6306e65770 Merge "Release note for fixing event-engines HA" 2019-02-20 13:01:37 +00:00
Zuul
af9a605689 Merge "Stop using deprecated keystone_authtoken/auth_uri" 2019-02-19 21:09:01 +00:00
Zuul
57ecd33c61 Merge "Update dogpile.cache to match global requirements" 2019-02-19 16:25:39 +00:00
Brad P. Crochet
0085d08baa Stop using deprecated keystone_authtoken/auth_uri
keystone_authtoken/auth_uri is deprecated [1]. Use www_authenticate_uri
instead.

keystonemiddleware in requirements and lower constraints should be increased
because www_authenticate_uri was introduced in keystonemiddleware 4.18.0.

[1] https://review.openstack.org/#/c/508522/

Change-Id: I99b0ee941d702a28fb4f392d9747d0e2257a42c8
Closes-Bug: #1788174
2019-02-19 10:37:23 +00:00
Dougal Matthews
aa30e18c5b Update dogpile.cache to match global requirements
Change-Id: I810bc2d11ae14c8e1b669f8f0160b7c142c4b9a3
2019-02-18 09:06:36 +00:00
Chandan Kumar
1d16d2ffba Revert "Fix how Mistral prepares data for evaluating a YAQL expression"
This reverts commit a39db2d3dc21afbe8b9e1d6f8dd870da272b9671.

Change-Id: Ib45ad50cd1523e079b780e953091ddedbd49a54a
Related-bug: #1816026
2019-02-15 11:01:54 +00:00
Vlad Gusev
ed7c351d0b Release note for fixing event-engines HA
Add missing release note for https://review.openstack.org/#/c/548044/

Change-Id: I297054f052b79143b2d0882286fb3e22d19c70c5
2019-02-15 13:51:38 +03:00
Renat Akhmerov
a39db2d3dc Fix how Mistral prepares data for evaluating a YAQL expression
* If we use the built-in YAQL function 'str' in a workflow then it
  doesn't represent lists as '[item1, item3, ...]' but instead
  creates '(item1, item2,...). This is because the standard YAQL
  function 'yaql_utils.convert_input_data', which is needed to
  convert a initial user data into an internal YAQL format,
  converts all sequences (except strings) into tuples.
  This patch overrides this behavior for sequences that are not
  strings and tuples so that they now get converted into lists.
  YAQL uses tuples because it needs to obtain a safe immutable
  structure to make calculations upon. But in Mistral list is
  more suitable because lots of users care about string
  representations. Immutability is not so important because
  Mistral code base guarantees that the initial data context
  for an expression won't be changed while an expression is
  being evaluated by YAQL.
* "str" YAQL function used to work well but it was broken in
  https://review.openstack.org/#/c/477816/ that added additional
  context preparation in order to fix the issue
  https://bugs.launchpad.net/mistral/+bug/1772864

Change-Id: I69d32f8772418d586d6c414842bb54aada217481
Closes-Bug: #1815710
2019-02-14 12:20:12 +07:00
Zuul
58d6634702 Merge "Add a workflow execution report endpoint" 2019-02-11 11:44:49 +00:00
Renat Akhmerov
ae2c5fdbbb Add a workflow execution report endpoint
* This patch adds the first version of the REST endpoint that
  can generate a report for a workflow execution. Without any
  query parameters, GET method of the endpoint returns a tree-like
  structure that includes information about all execution objects
  associated with the specified workflow execution. The root object
  is the workflow execution itself, its children are task executions,
  each task execution has either action executions or workflow
  executions and so on. So the structure describes the entire set
  of execution objects regardless of how deep there are from the
  root object. This kind of data itself can be used to better
  understand and visualise the process related with a parent workflow,
  see what paths were taken while the workflow was running.
  If additional parameters are provided in a query string, the
  endpoint can give only a subset of the entire tree. Currently,
  the filters are:
    1. "errors_only" (boolean) to retain only all execution objects
        with the error state, meaning that only all error paths are
        present in the report. It is useful when we need to do a root
        cause analysis of the workflow failure. False by default.
    2. "max_depth" (integer) to limit how deep the algorithm can go
        into nested workflows. If set to 0, only the root workflow
        execution will be in the report. If set to 1, then the report
        will have only the root workflow execution and its direct
	children. And so on. If negative (by default) then no limit
        is set.
  Additionally, the report contains statistics about task executions
  examined while the report was being generated, like the number of
  tasks in the error state, number of tasks that successfully
  finished and so on.
* Added all main tests for the endpoint. Note that despite the fact
  that this test verifies a REST API endpoint, unlike the other API
  tests it runs a Mistral engine to run workflows. This is done to
  simplify the test implementation so that we don't have to mock
  everything with huge data structures like we do in other API tests.

Possible changes that may be made based on the feedback:
* Statistics can contain not only number of tasks in certain states.
  We can also add things like number of actions, depth of the tree,
  number of nested workflows, average task/action/workflow execution
  time etc.
* Additional query parameters to configure a generated report. For
  example, "statistics_only" just to get a general information about
  the workflow execution tree, not the tree itself. Another example
  is "running_only" to retain only not finished workflow paths.

Implements blueprint: mistral-error-analysis

Change-Id: Id3e17821e04b7a1b84dfea5126d223d90ad8e3c2
2019-02-11 15:03:35 +07:00
Oleg Ovcharuk
87200f6aea Sending TASK_FAILED event in case of MistralException
If task was failed by unhandled exceptions, Mistral was not sending
TASK_FAILED event, which could be critical for Mistral's clients, who
use notifications mechanism to handle issues.

Change-Id: I460686c2852e3eb642506049ad5c33705697ecd8
Closes-Bug: #1803746
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
2019-02-07 19:33:10 +00:00
Andras Kovi
81af1b4838 Process all task batches in wf output evaluation
All batches must be processed in workflow output evaluation. An
empty batch means only that no tasks were end tasks in the queried
slice.

Closes-Bug: 1811775
Change-Id: I0ed4e690f67966ba2d145ad6430b517bd896ced6
2019-01-15 13:54:41 +01:00
Zuul
e907e0ed69 Merge "Replace tripleo-scenario003-multinode with scenario003-standalone" 2019-01-11 10:15:26 +00:00
Rafael Folco
2f4db0a61e Replace tripleo-scenario003-multinode with scenario003-standalone
Change-Id: I393855408d13eacc83e63ac98b1daced7177b0fc
Story: https://tree.taiga.io/project/tripleo-ci-board/us/533
2019-01-10 10:34:28 +00:00
lijunjie
c7509a9c85 Fix the misspelling of "default"
Change-Id: I0a96eebec595ed510abde253a517c4b73cf5ec7b
2019-01-10 17:22:36 +08:00
Zuul
fdd50f0e25 Merge "Omit the twice occured words in index.rst" 2019-01-09 04:52:21 +00:00
Zuul
fc479303fa Merge "Amend the spelling error of a word" 2019-01-07 12:35:57 +00:00
MaoyangLiu
efca6f2e95 fix typo mistakes
Change-Id: I274bb4b2e6ac97c6ed6d85367f96b72f3ecd06c5
2019-01-07 09:22:56 +00:00
chengebj5238
7301d8cbcd Amend the spelling error of a word
Change-Id: I940b0bea7350735bcf78045058fc23568ba2e29a
2019-01-07 09:22:02 +00:00
yfzhao
98589e12c9 Update mailinglist from dev to discuss
openstack-dev was decomissioned this night in https://review.openstack.org/621258
Update openstack-dev to openstack-discuss

Change-Id: I803a5acabe8ccdf18721707f5637d759d9b3d5b6
Closes-Bug: #1808899
2019-01-04 14:59:24 +00:00
guotao.bj
278fc3319a Remove those copy words occured twice times in wf_lang_v2.rst
Change-Id: I5e8e0fca235e34b0663c4557c9e3405abfcd5e36
2019-01-04 11:50:58 +00:00
akhiljain23
444541177e Gate fix for failing at openstack-tox-docs
Zuul has been failing lately because of version conflict in
requirement for dogpile.cache package. For example in logs[1]
This PS adds requirement to fix the gate.

[1] http://logs.openstack.org/75/625775/1/check/openstack-tox-docs/d9591b7/job-output.txt.gz#_2018-12-25_06_04_13_576175

Change-Id: I9094d0e56263cf4c9cca5fc5b845606110a110a3
2018-12-27 08:25:14 +05:30
Zuul
0eb1611327 Merge "[Event-engine] Allow event_engine to work in HA" 2018-12-09 11:31:19 +00:00
Zuul
d9e9391b4e Merge "Eliminating datetime.now()" 2018-12-04 21:03:09 +00:00
Renat Akhmerov
0fb96bbd2b Add loging for sending an action to executor
Closes-Bug: #1805579
Change-Id: I4dc2b1e059d66d16725417ea5aa27b978e2163f0
2018-11-28 14:14:16 +07:00
Zuul
c4ccb70feb Merge "Clarify REST field valiation error" 2018-11-27 13:59:49 +00:00
Zuul
acd3beacc1 Merge "Remove tripleo newton and ocata jobs" 2018-11-27 11:22:28 +00:00
Andras Kovi
99be4227cf Clarify REST field valiation error
Change the error message to better articulate the error.

Change-Id: Ie2203e9ad120ff1f603ac9f8fbb9632484faa1a6
2018-11-27 09:51:01 +01:00
Zuul
8c3d439f65 Merge "Add Python 3.6 classifier to setup.cfg" 2018-11-27 08:27:01 +00:00
Zuul
b4c0dd3842 Merge "Add missing ws separator between words" 2018-11-27 08:27:00 +00:00
Rafael Folco
12a2fe8795 Remove tripleo newton and ocata jobs
Non-containerized multinode scenario jobs are no longer supported
and should be removed from zuul config.

Change-Id: I9abc7c9db1bd216dcd61925061e8bbc8c7e00b85
2018-11-27 05:49:31 +00:00