159 lines
5.3 KiB
Python
Raw Normal View History

Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import io
import logging
from functools import partial
from .errors import JenkinsJobsException
from .loc_loader import LocLoader
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
from .yaml_objects import BaseYamlObject
from .expander import Expander, ParamsExpander, deprecated_yaml_tags, yaml_classes_list
from .roots import root_adders
logger = logging.getLogger(__name__)
class Loader(LocLoader):
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
@classmethod
def empty(cls, jjb_config):
return cls(io.StringIO(), jjb_config)
def __init__(
self, stream, jjb_config, source_path=None, source_dir=None, anchors=None
):
super().__init__(stream, source_path)
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
self.jjb_config = jjb_config
self.source_path = source_path
self.source_dir = source_dir
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
self._retain_anchors = jjb_config.yamlparser["retain_anchors"]
if anchors:
# Override default set by super class.
self.anchors = anchors
# Override the default composer to skip resetting the anchors at the
# end of the current document.
def compose_document(self):
# Drop the DOCUMENT-START event.
self.get_event()
# Compose the root node.
node = self.compose_node(None, None)
# Drop the DOCUMENT-END event.
self.get_event()
return node
def _with_stream(self, stream, source_path, source_dir):
return Loader(stream, self.jjb_config, source_path, source_dir, self.anchors)
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
def load_fp(self, fp):
return self.load(fp)
def load_path(self, path):
return self.load(path.read_text(), source_path=path, source_dir=path.parent)
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
def load(self, stream, source_path=None, source_dir=None):
loader = self._with_stream(stream, source_path, source_dir)
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
try:
return loader.get_single_data()
finally:
loader.dispose()
if self._retain_anchors:
self.anchors.update(loader.anchors)
def load_deprecated_yaml(tag, cls, loader, node):
logger.warning("Tag %r is deprecated, switch to using %r", tag, cls.yaml_tag)
return cls.from_yaml(loader, node)
for cls in yaml_classes_list:
Loader.add_constructor(cls.yaml_tag, cls.from_yaml)
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
for tag, cls in deprecated_yaml_tags:
Loader.add_constructor(tag, partial(load_deprecated_yaml, tag, cls))
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
def is_stdin(path):
return hasattr(path, "read")
def enum_expanded_paths(path_list):
visited_set = set()
def real(path):
real_path = path.resolve()
if real_path in visited_set:
logger.warning(
"File '%s' is already added as '%s'; ignoring reference to avoid"
" duplicating YAML definitions.",
path,
real_path,
)
else:
yield real_path
visited_set.add(real_path)
for path in path_list:
if is_stdin(path):
yield path
elif path.is_dir():
for p in path.iterdir():
if p.suffix in {".yml", ".yaml"}:
yield from real(p)
else:
yield from real(path)
def load_files(config, roots, path_list):
expander = Expander(config)
params_expander = ParamsExpander(config)
loader = Loader.empty(config)
for path in enum_expanded_paths(path_list):
if is_stdin(path):
data = loader.load_fp(path)
else:
data = loader.load_path(path)
if data is None:
continue
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
if not isinstance(data, list):
raise JenkinsJobsException(
f"The topmost collection must be a list, but is: {data}",
pos=data.pos,
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
)
for idx, item in enumerate(data):
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
if not isinstance(item, dict):
raise JenkinsJobsException(
f"Topmost list should contain single-item dict,"
f" not a {type(item)}. Missing indent?",
pos=data.value_pos[idx],
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
)
if len(item) != 1:
raise JenkinsJobsException(
f"Topmost dict should be single-item,"
f" but have keys {list(item.keys())}. Missing indent?",
pos=item.pos,
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
)
kind, contents = next(iter(item.items()))
if kind.startswith("_"):
continue
if isinstance(contents, BaseYamlObject):
contents = contents.expand(expander, params={})
try:
adder = root_adders[kind]
except KeyError:
raise JenkinsJobsException(
f"Unknown topmost element type : {kind!r};"
f" known are: {','.join(root_adders)}.",
pos=item.pos,
Rewrite YAML parser Rewrite YAML parser, YAML objects and parameters expansion logic to enable better control over expansion logic. Broken backward compatilibity: * More agressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before. * Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures. Prepend them with underscore to be ignored by parser. * Files included using '!include-raw:' elements and having formatting in it's path ('lazy-loaded' in previous implementation) are now expanded too. Use '!include-raw-escape:' for them instead. See changes in these tests for examples: tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml * Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error. See changes in this test for example: tests/yamlparser/job_fixtures/parameter_name_reuse_default.* -> tests/yamlparser/error_fixtures/parameter_name_reuse_default.* * When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing. See changes in this test for example: tests/yamlparser/job_fixtures/job_group_includes_missing_job.* -> tests/yamlparser/error_fixtures/job_group_includes_missing_job.* Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2022-07-21 11:26:50 +03:00
)
adder(config, roots, expander, params_expander, contents, item.pos)