Add file to the reno documentation build to show release notes for
stable/wallaby.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/wallaby.
Sem-Ver: feature
Change-Id: I58e157deef1aac29cf7d53af54d4726824419cc4
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.
2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.
Also replace policy.json to policy.yaml ref from doc and tests.
[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I3b9aeb3379a76f7e40dab0c46e27f4447a0c3d03
* added dynamic actions:
these actions are created and modified in runtime,
each action needs a code source to be imported from and a
class name.
- there are 2 new endpoints:
- /v2/code_sources/:
used to add new code sources to mistral.
- /v2/dynamic_actions/:
used to add dynamic actions to mistral in runtime
- a new Action provider (DynamicActionProvider) was added:
it provides the actions created from the dynamic actions api.
Change-Id: I9fe8c28ffdef71016d9dc13aea60a288c8ebaa0a
Signed-off-by: ali <ali.abdelal@nokia.com>
Add file to the reno documentation build to show release notes for
stable/victoria.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/victoria.
Change-Id: Id9fc114b680e47e494f9ec8869c176152ccf764a
Sem-Ver: feature
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems
Update Sphinx version as well.
Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.
Disable openstackdocs_auto_name to use 'project' variable as name.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.
openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.
Remove python 3 versioning from lower-constraints and
requirements to pass requirements-check.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: Id478711abb1783422ebb4ee9937d1b7202fdf2f3
Add file to the reno documentation build to show release notes for
stable/ussuri.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.
Change-Id: Ie516ee33ca126c6d9750ecdfc1fbb99051fc4a97
Sem-Ver: feature
*the new endpoints are:
v2/executions/{ex_id}/executions
v2/tasks/{task_id}/executions
it returns a list of execution resources, by default the API will
return all sub-executions
the API can take 3 parameters:
errors_only: returns only the error routes
- default is False
include_output: includes the output field for the returned executions
- default is False
max_depth: the max depth for the returned executions
- if a negative value is given, then the API will return
all sub-executions
- default is -1
Partially-Implements: blueprint mistral-execution-origin
Change-Id: I2a4c5e6890dbb4de868ce885d51341b15e359233
Signed-off-by: ali <ali.abdelal@nokia.com>
added namespace for the actions, actions can have the same name if they
are not in the same namespace, when executing an action, if an action
with that name is not found in the workflow namespace or given
namespace mistral will look for that action in the default namespace.
* action base can only be in the same namespace,or in the
default namespace.
* Namespaces are not part of the mistral DSL.
* The default namespace is an empty string ''.
* all actions will be in a namespace, if not specified, they will be
under default namespace
Depends-On: I61acaed1658d291798e10229e81136259fcdb627
Change-Id: I07862e30adf28404ec70a473571a9213e53d8a08
Partially-Implements: blueprint create-and-run-workflows-within-a-namespace
Signed-off-by: ali <ali.abdelal@nokia.com>
* Some users rely on the presence of the root error related to
running an action and it's not convenient that it is now in
the end of the string, e.g. if we look at the corresponding
task execution "state_info" field. This patch includes the cause
error message in the beginning of the resulting error string
returned by the action executor so that it's clearly visible.
This message can be also truncated in some cases (depending on
the config option) so we need to make sure we keep the cause
error message.
Closes-Bug: #1847984
Change-Id: Ieb10c10401380410665c418f4688681e929b1e23
Add file to the reno documentation build to show release notes for
stable/train.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/train.
Change-Id: Icae14d39d714d0da3a945aeca3dcb70f73635162
Sem-Ver: feature
* For the sake of the service performance, it may make sense to
disable validation of the workflow language syntax if it is
affordable for a particular use case. For example, if all
workflows are auto-generated by a 3rd party system and tested
thoroughly (either by running them with Mistral or at least
validating them via the special Mistral endpoint) then we can
safely disable validation of the language syntax when uploading
workflow definitions. For production systems it makes a big
difference if workflow texts are large (thousands of tasks).
This patch adds the boolean parameter "skip_validation" for API
requests like "POST /v2/workflows" to disable validation, if
needed, and the new configuration property "validation_mode"
to set a desired validation mode.
The option is an enumeration and has the following valid values:
1) "enabled" - enabled for all API requests unless it's
explicitly disabled in the request itself
2) "mandatory" - enabled for all API requests regardless
of the flag in the request
3) "disabled" - disabled for all API requrests regardless
of the flag in the request
"mandatory" is choosen as the default value for this new
property to keep compatibility with the previous versions.
* Minor style changes.
Closes-Bug: #1844242
Change-Id: Ib509653d38254954f8449be3031457e5f636ccf2
* There was a weird typo in the list generator expression made in
https://review.opendev.org/#/c/652575 that led to calculating
a field value in the wrong way. Fixed. The added test was
previously failing.
Closes-Bug: #1829841
Change-Id: I6b8ea8dfa623acbb56f649630ddee62e43d484eb
* "__task_execution" wasn't included in this case into the
expression data context so the function task() didn't work
properly
Change-Id: I3cacae90f9031d09a5e6d8153d728ddc01e1bb21
Closes-Bug: #1823875
Add file to the reno documentation build to show release notes for
stable/stein.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.
Change-Id: Icb8261ce3371296b8a87e5bfa527a5f64a25157c
Sem-Ver: feature
This method allows to specify a private key and avoids its storage
in the filesystem of the executors. This can be used later in
combination of a secrets_retrieve to use keys stored in barbican.
Change-Id: Ide438a7f6d24c8bdc9eb2c82e935fd39a6acc2c6
Closes-Bug: #1806703
* See bug description for the example that didn't work. It was
caused by a simple mistake in a python expression of type
"my_set = my_set or set()" that didn't work as expected, i.e.
it created a new set even if my_set is already an empty set.
So, the proper expression that's needed is
"my_set = set() if my_set is None else my_set"
Change-Id: I2a787921449fecf3301013a770ffe712e9606baf
Closes-Bug: #1803677
* Once in a while we get DetachedInstanceError for action definitions
and it happens when they are fetched from cache. We must always
clone persistent objects before caching them.
Change-Id: I1d0cffea6775eb258dcefc0dbb8a6ee18effe597
Closes-Bug: #1803528
* Removed using scheduler from action execution heartbeat checker
in favor of regular threads.
* Added the new config options "batch_size" under [action_heartbeat]
group to limit a number of action executions being processed during
one iteration the checker.
* Added a test checking that an action execution is automatically
failed by the heartbeat checker.
Closes-Bug: #1802065
Change-Id: I18c0c2c3159b9294c8af96c93c65a6edfc1de1a1
* Added the new property 'execution_integrity_check_batch_size'
under the [engine] group to limit the number of task executions
that the integrity checker may process during one iteration.
Closes-Bug: #1801876
Change-Id: I3c5074c45c476ebff109617cb15d56c54575dd4f
* action_queue module is replaced with the more generic
post_tx_queue module that allows to register operations that must
run after the main DB transaction associated with processing a
workflow event such as completing action.
* Instead of calling workflow completion check from all places
where task may possibly complete, Mistral now registers a post
transactional operation that runs after the main DB transaction
(to make sure at least one needed consistent DB read) right
inside the task completion logic. It reduces clutter significantly.
* Workflow completion check is now registered only if the just
completed task may lead to workflow completion, i.e. if it's the
last one in a workflow branch.
* Join now checks delayed calls to reduce a number of join
completion checks created with scheduler and also uses post
transactional queue for that.
Closes-Bug: #1801872
Change-Id: I90741d4121c48c42606dfa850cfe824557b095d0
* This patch removes the approach with DB polling needed to
determine if a "join" task is ready to run. Instead of running
a periodic scheduled job, each task completion now runs the
algorithm that finds all potentially affected join tasks
and schedules just one job (instead of a periodic job) to check
their readiness.
This solves a problem of system cascaded overloading in case of
having many very large joins (when a workflow has many joins with
many dependencies each). Previously, in such case Mistral created
too many periodic jobs that just didn't let the workflow progress
well, i.e. most CPU was used by scheduler to run those periodic
jobs that very rarely switched "join" tasks to the RUNNING state.
Change-Id: I5ebc44c7a3f95c868d653689dc5cea689c788cd0
Closes-Bug: #1799356
Fix the issue where `next` link in the response of the REST API, when
used with pagination, gets corrupted and makes it unusable. After this
fix, next link should be readily usable.
Change-Id: Idf45a59e0b07d8306cc82391679fe30a9cd2f0c1
Closes-Bug: #1793344
* Workflow output sometimes is not calculated correctly due to
the race condition between different transactions: the one that
checks workflow completion (i.e. calls "check_and_complete") and
the one that processes action execution completion (i.e. calls
"on_action_complete"). Calculating output sometimes was based on
stale data cached by the SQLAlchemy session. To fix this, we just
need to expire all objects in the session so that they are
refreshed automatically if we read their state in order to make
required calculations. See the bug description for more details
on how the problem was observed.
* Added another test for direct workflow that formally checks
calculation of workflow output. It doesn't pretend to test the
aforementioned issue (it can be reproduced only with a big number
of attempts, and/or under load). It's for the sake of the test
module completeness.
Change-Id: I4a7e7fd9a4bbb6e93df169b4b40bc2d83ccfce89
Closes-Bug: #1792090
Bug 1785654 has been fixed in https://review.openstack.org/#/c/599606
but there was no release note. This change adds it.
Change-Id: I7bea9553c41336ba08d6fec551055290530769a9