3111 Commits

Author SHA1 Message Date
int32bit
f84952a3b1 Add get cron-trigger by id support
Currently we can only do a CURD action on a cron-trigger by name.
This patch refer to workflow implementation and re-encapsulate
DB API so that users can manage a cron-trigger by id or name.

Closes-Bug: 1684469

Change-Id: I9ff657b2604647e734b5539e9bd6a524a3a20efb
2017-10-05 11:41:00 +00:00
Jenkins
b791850ba2 Merge "Update README with Keystone authtoken config" 2017-10-04 12:02:21 +00:00
Brad P. Crochet
098845e691 Migrate Mistral jobs to Zuul v3
Change-Id: I2164953581efad6da2354f781b0567d6125bc5aa
Partial-Bug: #1720797
2017-10-03 09:46:16 -04:00
Jenkins
def783c60a Merge "Add project_id to API resources" 2017-09-28 07:05:07 +00:00
OpenStack Proposal Bot
c07f281116 Updated from global requirements
Change-Id: Ia947c164a699719a4fc79fe1ed4139d31d88f4cf
2017-09-26 03:28:03 +00:00
Brad P. Crochet
fbee074a3f Update README with Keystone authtoken config
Change-Id: Ic9f51dd55fe881417454b1b8d0c414acba7d7022
2017-09-22 15:42:53 -04:00
Jenkins
35ccebc284 Merge "Replace @loopingcall.RetryDecorator with @tenacity.retry" 2017-09-22 10:59:56 +00:00
Renat Akhmerov
6ad421b34c Replace @loopingcall.RetryDecorator with @tenacity.retry
* It turns out that the behaviour of looping.RetryDecorator from
  oslo.service is different than what was expected. See
  https://bugs.launchpad.net/oslo.service/+bug/1718635 for
  details. This patch replaces it with a similar decorator from
  tenacity library.

Change-Id: I2d8a7f2b430e26991cc13a88ad33c1266c82d113
2017-09-22 11:42:21 +07:00
Jenkins
466c45b1e4 Merge "Updated from global requirements" 2017-09-21 17:22:17 +00:00
Jenkins
455d495261 Merge "Removed NOT IN query from expiration policy." 2017-09-21 16:38:33 +00:00
OpenStack Proposal Bot
e1d881c1fe Updated from global requirements
Change-Id: I3046c5bfeecdb378d387a3b0b01f89307e65f0d0
2017-09-21 13:30:47 +00:00
Kupai József
5c0b720da2 Removed NOT IN query from expiration policy.
NOT IN subquery with limit is not supported by some databases.

Closes-bug: #1712585
Change-Id: Ieeb6629cc6c2fc0794ac9a0ef731816c2b6ea59b
2017-09-21 12:45:21 +02:00
Jenkins
43343dc87a Merge "Use @db_utils.retry_on_deadlock to retry scheduler transactions" 2017-09-20 16:46:48 +00:00
Jenkins
b8d1e096e4 Merge "Add README.mistral.conf doc in etc directory" 2017-09-20 14:16:49 +00:00
Jenkins
fd127e36ef Merge "Add root_execution_id to sub-workflow executions" 2017-09-20 12:00:13 +00:00
Jenkins
06cacce371 Merge "Update URL and indentations" 2017-09-20 09:51:18 +00:00
Renat Akhmerov
a995916f14 Use @db_utils.retry_on_deadlock to retry scheduler transactions
* Since scheduler has transactional logic we need to account for
  cases when these transactions hit DB deadlocks. It is possible
  just due to MySQL nature, it's recommended to always design
  based on that. This patch decomposes one big scheduler method
  that processes delayed calls into smaller methods so that we
  could apply @db_utils.retry_on_deadlock decorator to repeat
  transactions if they fail because of a deadlock in MySQL.
* Fixed taskk state update when we're assiging it to RUNNING.
  In this case we can't set it to RUNNING if there any paused
  child executions.
* Fixed the test for cascaded pausing which didn't account
  non-atomicity of this operation

Closes-Bug: #1715589
Change-Id: Iffa0fb540a5705c587d71d30af6ab913b26d3952
2017-09-20 16:09:02 +07:00
Jenkins
7627c5d9f9 Merge "TrivialFix: pretty format the json code block" 2017-09-18 08:53:24 +00:00
OpenStack Proposal Bot
d2fb3612e5 Updated from global requirements
Change-Id: Ia8cc08f84d0c38929bc93bc8441296d16e89f52e
2017-09-16 23:14:34 +00:00
int32bit
2018962568 Add project_id to API resources
Currently our API doesn't return any project info in many resources, like
execution, cron-trigger, workbook, etc. As a essential property in those
resources, it's really helpful for users to process resources if we
return project_id in resource endpoints.

Closes-Bug: 1707573
Closes-Bug: 1694398

Change-Id: I866d20cf1f5129b6249140063aa0836f63626767
2017-09-15 19:43:47 +08:00
junboli
47fae66fe0 Add README.mistral.conf doc in etc directory
Currently, the mistral.conf can be generated by tox, So add the
README.mistral.conf is useful.

Change-Id: I657b0d193a8f53dd86ed7b5bf228310eddbecd75
2017-09-15 04:21:12 +00:00
junboli
76bafad3ec TrivialFix: pretty format the json code block
In the file doc/source/user/dsl_v2.rst, the json format of
task publish result is not well formated, this change is to
pretty format the json code block

Change-Id: I783472c1d7c13cd7b6a464325cf7213abf3ac359
2017-09-15 01:35:33 +00:00
Dougal Matthews
053308e7c3 Add root_execution_id to sub-workflow executions
For sub-workflow executions this will be the ID of the initial workflow
execution. For example, given the following workflows.

workflows:
  wf1:
    tasks:
      task1:
        workflow: wf2

  wf2:
    tasks:
      task1:
        workflow: wf3

  wf3:
    tasks:
      task1:
        action: std.noop

When we start wf1, it calls wf2 which calls wf3. Currently, it is hard to
retrieve the full execution, including sub-workflow executions. This patch adds
the root_execution_id to the sub-workflows, so the execution for wf2 and wf3
have a root_execution_id that is the ID of the execution for wf1. The execution
for wf1 is the root execution, so root_execution_id is None. This basically
gives us a flat reference to the graph of workflow executions.

This change is useful as we can then expose it via the API and it makes
it easier and more efficient to find a full execution, including it's
sub-workflow executions.

Implements: blueprint mistral-root-execution-id
Change-Id: I24638812caa2e48e3c071925db5e552b21e15d47
2017-09-14 11:26:36 +01:00
OpenStack Proposal Bot
0b03640018 Updated from global requirements
Change-Id: I8fe4448be3551e43ddc641d714b7aa719d6b0b26
2017-09-13 12:52:59 +00:00
Jenkins
a03410a875 Merge "Cleanup test_std_http_action" 2017-09-13 10:56:47 +00:00
Jenkins
4a1e9ea2bd Merge "Add mistral/tests/unit/expressions/__init__.py" 2017-09-12 09:53:08 +00:00
Jenkins
33143d324d Merge "Updated from global requirements" 2017-09-12 04:26:31 +00:00
Jenkins
2b985feb0b Merge "Fixes issue rendering strings containing multiple jinja expressions" 2017-09-12 04:20:43 +00:00
Bob Haddleton
4dbacfe1a1 Add mistral/tests/unit/expressions/__init__.py
This patchset adds the __init__.py file to mistral/tests/unit/expressions
to enable those tests to run, and updates the Jinja tests to include
the created_at attribute.

Closes-Bug: 1716452

Change-Id: I5c7af1cd4a2777764a2a46adf85129e4cb90e4d8
2017-09-11 18:38:36 -05:00
OpenStack Proposal Bot
af7a9153f4 Updated from global requirements
Change-Id: I56ec40c6df64df5625cf66c4edc908143b6c5042
2017-09-11 21:39:11 +00:00
Andras Kovi
e416188ebc Cleanup test_std_http_action
Cleaned up redundant assertions to make tests independent and
focused.

Change-Id: I35ca39473e89a884f3e6babd0ba89d61f1434562
2017-09-11 10:27:10 +02:00
Nick Maludy
10b27fcdea Fixes issue rendering strings containing multiple jinja expressions
Closes-Bug: #1712133
Change-Id: I058e9858e4a4a9a94e0948df38758446e68bf5b9
2017-09-09 11:59:49 -04:00
Jenkins
eccd7a0bb7 Merge "Handle case with None encoding during std.http action execution" 2017-09-09 09:18:30 +00:00
Jenkins
fdaec7edd3 Merge "Clean up screen and tail_log references" 2017-09-08 10:59:23 +00:00
Anastasia Kuznetsova
178369459f Handle case with None encoding during std.http action execution
During some cases resp.content may have a context not in json format
and with 'None' in the resp.encoding. Need to properly handle this
situation and don't try to pass 'None' as an argument to decode.

Change-Id: Id87d650996f16b5ffab79d72413134a4c7fe9ca9
Closes-Bug: #1700608
2017-09-08 12:50:07 +02:00
Jenkins
5aad444d56 Merge "Fix services launcher to handle shutdown properly" 2017-09-08 10:00:41 +00:00
Jenkins
7a74b6d6fd Merge "Catch DBEntityNotFoundError exceptions for invalid AdHoc Actions" 2017-09-08 09:53:24 +00:00
Jenkins
978399c824 Merge "Using current pike stable release for devstack" 2017-09-08 09:53:13 +00:00
Brad P. Crochet
433a5b46a9 Clean up screen and tail_log references
devstack has removed screen completely in
I8c27182f60b0f5310b3a8bf5feb02beb7ffbb26a. This patch cleans up any
remaining references to screen from the Mistral devstack plugin.

Change-Id: I56d6fe2c6957f9a6b5cc30d7570288ceed8b9f35
Depends-On: I8c27182f60b0f5310b3a8bf5feb02beb7ffbb26a
2017-09-07 09:17:09 -04:00
Jenkins
6b5a9981dd Merge "Fix "with-items" locking" 2017-09-07 11:32:25 +00:00
Jenkins
d3d4efdca6 Merge "Fix Kombu RPC threading and use within multiprocess environment" 2017-09-07 10:58:46 +00:00
junboli
d4676bff0b Using current pike stable release for devstack
Since the pike has been released, so it is better to use pike
as the devstack repo.

Change-Id: I5006a7cb9eeacef930e04d2f7287c652d4ba9165
2017-09-06 11:09:26 +08:00
Renat Akhmerov
2daa3c0ab2 Fix Kombu RPC threading and use within multiprocess environment
* Added a call to clean up RPC clients into WSGIService start()
  method to make sure that we reinitialize RPC in every child
  API process. Otherwise, replies for synchornous RPC calls
  don't get routed to the right client process (see bug
  description)
* Mistral components engine, executor and event engine didn't
  start properly because KombuServer.run() was implemented
  w/o a new thread. This patch fixes it.
* The methods to get RPC clients like
  mistral/rpc/clients.py:get_engine_client() wasn't synchronized
  on the client instances. As a consequence, sometimes Mistral
  created more than one instance of the same client (violation
  of Singleton pattern). This patch fixes it by adding required
  locking.
* Other style changes

Closes-Bug: #1714186
Closes-Bug: #1714929

Change-Id: I242c48df9811459321d1cd4b1d37ed35af0a013a
2017-09-05 10:36:43 +00:00
Renat Akhmerov
7b45a50a42 Fix "with-items" locking
* Previously we thought that we can avoid locking task execution
  when we process completion of an individual action inside
  WithItemsTask. But that was wrong because even when we don't
  need to control "concurrency" property we still can get a
  situation when two different action completions will be
  processed via scheduler and both will see all actions in state
  SUCCESS (because scheduler handles it in a different later
  transaction) and hence both will complete the task and run
  tasks from "on-xxx" clauses. This patch makes WithItemsTask
  always use locking.
* Added db_utils.retry_on_deadlock for
  _scheduled_on_action_complete() method since it opens a new
  DB transaction and can potentially end up in a dead lock (due
  to MySQL nature)

Closes-Bug: #1715116

Change-Id: I6f34409e7182af3ca5b13c17e6d6fb4302f9efed
2017-09-05 17:34:46 +07:00
rajat29
d4ae172125 Fix to use . to source script files
Adhering to coding conventions. Refer to ``Code conventions`` at
https://docs.openstack.org/contributor-guide/ for details.
When you have to source a script file, for example, a credentials file
to gain access to user-only or admin-only CLI commands, use . instead of source.
https://docs.openstack.org/contributor-guide/writing-style/code-conventions.html

Change-Id: I97c0cc8b9215b0f580311b1850b13a72f9ced60a
2017-09-05 13:04:29 +05:30
Jenkins
9f5160f0dc Merge "Add "API server started." print statement for the API wsgi service" 2017-09-04 14:01:51 +00:00
OpenStack Proposal Bot
af3d451566 Updated from global requirements
Change-Id: I82a4e79dd02722f40d7468ef83a2dcbeb5591091
2017-09-02 12:04:31 +00:00
Winson Chan
8a4fe7f146 Fix services launcher to handle shutdown properly
When Mistral services are configured with systemd, the services time out
on shutdown. The root cause is that each of the server thread is
assigned to a different instance of the oslo.service ServiceLauncher.
The ServiceLauncher can manage multiple server threads. When the server
threads are added to the same instance of the ServiceLauncher, SIGTERM
is handled properly for each server threads.

Change-Id: I8b739844f6073890324189aee028c0d7da3cc76e
Closes-Bug: #1714351
2017-09-01 18:54:12 +00:00
Bob Haddleton
8b2f63cbdf Catch DBEntityNotFoundError exceptions for invalid AdHoc Actions
This patch will catch the DBEntityNotFoundError exception that is
raised when an AdHoc Action has an invalid base action defined.  The
exception is converted to an InvalidActionException which will
halt the task/workflow and cause them to enter the ERROR state.

Change-Id: I115b8243641da100474e8c8b34a724978078978b
Closes-Bug: 1714548
2017-09-01 11:52:17 -05:00
Jenkins
418e27f9c6 Merge "Fix test for decoding utf8" 2017-09-01 05:06:41 +00:00