In case we load definition of one workflow, there is no need to convert
it to yaml and back.
Change-Id: Ia7fb914843e56650c8f6ff5e8c16d1b57001c0fc
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
It is very important to prevent sending huge messages via rpc.
Notification data has a lot of potentially large fields, such as output,
spec and etc. Most of them are useless as a part of notification message
so we should omit them.
Change-Id: I0762a465742fcb1d1dc8d4d00d81cf8288b842ca
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
Delayed calls for nonexistent entities should not fail; they should do
nothing and be deleted in normal way.
Change-Id: I1b818d671468b95ce8ae06416b57fd4a22cc6eb2
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
To get the result we need only a number of tasks, so there is no
need to fetch full task executions from the database.
Change-Id: I34ea985e6de08f46e1c40c64996357c8e6f3d79c
New function update_action_execution_heartbeat was introduced in db api
to optimize updating of heartbeats.
Now to perform the update mistral downloads an action execution object,
then modifies it, and finally uploads back to the db using ORM.
This function removes initial object fetching and changes ORM update to
sqlalchemy core.
Change-Id: I4cb2d85c193531648540a37caca9ee868ede3a08
Some minor updates including fixing issues with the HTML5 output
generated by Sphinx 2.0, inlining the OpenStack logo rather than
referencing it from elsewhere, and adding a new SOURCE_DATE_EPOCH option
for use in reproducible builds in new release openstackdocstheme 1.30.0.
...to pick up many improvements, including the return of table borders.
Change-Id: I22b652e1eb007616e697cb238fcd9e5f010cd693
Context evaluation is not required for non-conditional transitions,
because the list of next tasks is known in advance.
For this reason we can skip this operation, that can be quite
time-consuming, and generate a list of task directly.
Also a little cleanup was done to remove unnecessary methods.
Change-Id: Ia419c47a7d71db46a5cae557fe8bc7512390715e
A little code clean-up: this commit replaces _get_next_clauses method
from direct_workflow.py with a built-in find_outbound_task_names
Change-Id: Ic5ea956ee45b8c455d42669c0f0e386d53e22425
This trick is similar to what we do in _get_join_logical_state.
Instead of bypassing the entire workflow graph and downloading
tasks executions one by one, we prepare a lightweight cache with
a single database query and then analyze it on the spot.
Change-Id: If1adb4ea408248e766bff449637de2b1b1356738
This method was used to find a task execution by its name in
_is_upstream_task_execution only. But this operation can be
simplified, because we already have the task execution,
and therefore there is no need to search for it again in the
database.
Change-Id: I45a175970bc2567f5ef63e161fa431c8e9245ec1
Now action caching is used only in engine.actions, but actually
there are a lot of place where this cache can be useful.
This commit moves the caching code to the db layer, that allows it to
be used in all other places, where load_action_definition is called.
Change-Id: I8c470cbb424e63c16e599758bb93192985666353
Current algorithm of finding the joins requires a recursive search
through the execution graph, which leads to a large number of calls
to the database.
To optimize it we introduce a new algorithm that requires only one db
request. It downloads all potencial join task ids and names, and then
analyzes them without any additional db calls.
Change-Id: Ic73f2112406e681ae8a2aa67bcbccebe488fc03c
Now to find an adhoc action definition by name action hanler takes
the name in runtime context first, and then provides this value along
with workflow name and workflow spec name to resolve_action_definition
method.
This method tries to find a definition by full name, i.e.
"wb_name.action_name" if wf_name != wf_spec_name, and, if nothing was
found, looks for a definition just by name, i.e. "action_name".
There is an issue, because runtime context already contains full name
for the action and substitution of a workbook name as a prefix is not
necessary.
It leads to the fact, that for all adhoc actions mistral tries to find
them by "my_wb_name.my_wb_name.my_action_name" first, and then, when it
fails, it looks for "my_wb_name.my_action_name".
This results in double the number of queries to the database and highly
innefficient.
This commit solves the issue by not providing workflow name and workflow
spec name to resolve_action_definition method for adhoc actions.
After that, the method will search for the action immediately by its
name.
For system actions this substitution is not necessary there either, so
the code can be safely deleted as well.
Change-Id: I3b880ef29057afb041edd9095c279a944c77f2dd
This release of the Cinder client broke support for the v3
volume-transfer APIs unless microversion 3.55 or higher was requested.
depend on https://review.opendev.org/#/c/587877/
Change-Id: I26bbb2e5ef5f8a5f963b47385c81a64c0227f7b8
NetworkX released 2.3, dropping support for Python 2.
https://github.com/networkx/networkx/blob/master/doc/news.rst
Also see I0a9700926c9a0db93e782c853c33f1aaee3d4876
fixed sphinx requirements also according to global requirements
jsonschema should be uncaped also according to global requirements
but there are conflicts with zakarclient designateclient and irconicclient
so for now I put it in remark
Change-Id: Iea3dbcfa1f6ddf2c413ede5e73fa076c6edccfdf
With the new joining mechanism lookup utils for task executions
become useless and can be removed from the codebase.
Now all requests to the database will be perforemed directly from
a workflow controller.
Action defention cache was moved to mistral/engine/actions.py,
because it's needed only there.
Change-Id: If0d4403f5c61883ecfec4cfa14b98cc39aae5618
* There was a weird typo in the list generator expression made in
https://review.opendev.org/#/c/652575 that led to calculating
a field value in the wrong way. Fixed. The added test was
previously failing.
Closes-Bug: #1829841
Change-Id: I6b8ea8dfa623acbb56f649630ddee62e43d484eb
Now to find upstream task executions mistral does a lookup for each
inbound task spec. This is not optimal, because it leads to a significant
number of db requests.
To optimize this behavior we prepare a list of task names in advance,
and then search executions using 'in' filter.
Change-Id: Ia7bf62c45b889f753671bdda048f91c46af41039
Current joining mechanism in some cases can be expensive because
it uses a multi-step recursive search, which leads to a huge amount
of db requests.
This work changes this behavior by precaching required task
executions to prevent hammering the database during the lookup.
Change-Id: I2d1b7e72c728a14c85b015dfdb0f8800b95f3749
* Mistral evaluates expressions under "on-xxx" clauses more than
once during the processing of a workflow. It's now been fixed by
storing crucial information about the result of those expressions
after it was first obtained in the DB.
* Added two boolean fields "has_next_tasks" and "error_handled" in
the TaskExecution class and the required migration. These fields
allow not to calculate expressions under "on-xxx" clauses many
times which leads to reducing execution time in case of heavy
expressions and/or their data contexts.
* Minor style changes.
Closes-Bug: #1824121
Change-Id: Ib236ba7a72d8e578f9c52460d2a7d8d4540f9c37
This removes the parameter insecure in the client creation as this
no longer exists in the client. It also changes the client class
to be called as other clients in openstack using sessions
Change-Id: I3cb7ed4255f8996b8bfd9a3e1edba6de50f4e492
Closes-Bug: #1800819