201 Commits

Author SHA1 Message Date
Vsevolod Fedorov
d741ea02c6 Task: 47875
Story: 2010711
Search for included files relative to the current job first

Change-Id: I34a5743967bf7cbb8b464a0a36abad24180d61e4
Signed-off-by: Max Trunov <trunov_ms@taximaxim.ru>
2023-12-07 11:05:12 +03:00
Vsevolod Fedorov
af9e394c8e Add Loader.source_dir attribute
Currently, Loader.source_path is used for both error locations and
as base for include directory.
But, it is set to "<expanded j2-yaml>" by !j2-yaml: tag, which makes it
unusable as base for include directory.
Make separate source_dir attribute for this usage.

Change-Id: I67669eb42f761e2d76e89992e6ad89480ddd3df0
2023-12-06 12:37:22 +03:00
Vsevolod Fedorov
c1ac0e03d6 Story: 2010984 Fix folder tests and defaults for folders
Folders was not actually tested by tests. Fix tests and add more tests.
Fix bug in code unearthed by now-working tests: defaults were not checked
for job folders.

Task: 49188

Change-Id: I922af7a28b0ec0c99ef3f8a17b3d5a9c10f2dce0
2023-12-04 10:51:55 +03:00
Vsevolod Fedorov
a1d4f91d1a Add context for macro calls
Change-Id: I674b153770297c3a0abbfcee26d840e0f2be490b
2023-11-22 12:58:37 +03:00
Vsevolod Fedorov
d17820a405 Restore macros support for notifications
Bugfix. Notifications was forgotten when making list of available
modules for macros.

Change-Id: Ic79cdb34070bc82105a76896749b12047872e2f3
2023-11-22 11:33:49 +03:00
Vsevolod Fedorov
a82fa9d392 Update documentation for lazy-loaded files
Now, when tag !include-raw: has variable in included file path,
included file is expanded.
And lazy-loading mechanics is removed - now it can be expanded in
the usual way.

Change-Id: I7234d42793a080e7f4186707c4d1d2e9404bd7f3
2023-11-22 11:15:51 +03:00
Vsevolod Fedorov
1a65a1d44b Handle nested brace inside variable
Change-Id: I1004ea869bfe56b380109c7bcae866931b4d4725
2023-11-22 11:15:51 +03:00
Vsevolod Fedorov
2c4de9abb1 Replace obsolete tag with new one in example
Change-Id: I15f0b8204b8831b489da9bc2db7fb48392302b94
2023-11-21 11:59:14 +03:00
Vsevolod Fedorov
d62faa15d4 Fix parameters overrides in dimensions specification
Task: 48621
Story: 2010883
Change-Id: Ia267f66c3832fb4a42a54a86bda259e96bb191fd
2023-08-16 11:54:18 +03:00
Vsevolod Fedorov
0f29e2ea3a Expand axis params before enumerating values
Change-Id: Ib51c00ae6a000f8c87483fcdc4433a2d0d055942
2023-07-04 11:41:05 +03:00
Vsevolod Fedorov
2f8ac26e86 Add test for job instantiation with null parameters
Change-Id: Ie2c91b2e7c36804333ef8e96ca89977203defdd8
2023-04-17 10:42:30 +03:00
Vsevolod Fedorov
84fe78ad90 Fix macro call with null parameters
Change-Id: I48bfa50d4011068bcb0206456be1009c1f9fde54
2023-04-17 10:38:26 +03:00
Vsevolod Fedorov
eebfb9c961 Do not fail on empty YAML file
Change-Id: Ia12862f902c2f0b638fcbff798553ca404d81e24
2023-04-12 11:22:20 +03:00
Vsevolod Fedorov
60e8395c62 Add source location and context to error messages
Change-Id: I2e955c01b71a195bb6ff8ba2bb6f3a64cb3e1f58
2023-04-04 13:35:42 +03:00
Vsevolod Fedorov
b2ef49cd9f Tests: Add regression tests for stories fixed by parser rewrite
Change-Id: I0e6ac97c4c97a762cc7483393d6f85af4e902aa7
2023-02-28 20:16:57 +03:00
Vsevolod Fedorov
a701920d7b STORY: 2010534 Fix bug: j2-yaml is not allowed inside macro body
Change-Id: I24916df5f0592a5ea8197e7f372bd37fdba3a461
2023-02-28 20:16:57 +03:00
Vsevolod Fedorov
af9e03ec08 Rewrite YAML parser
Rewrite YAML parser, YAML objects and parameters expansion logic to
enable better control over expansion logic.
Broken backward compatilibity:
* More agressive parameter expansion. This may lead to parameters
  expanded in places where they were not expanded before.
* Top-level elements, which is not known to parser (such as 'job',
  'view', 'project' etc), are now lead to parse failures.
  Prepend them with underscore to be ignored by parser.
* Files included using '!include-raw:' elements and having formatting in
  it's path ('lazy-loaded' in previous implementation) are now expanded
  too.
  Use '!include-raw-escape:' for them instead.
  See changes in these tests for examples:
    tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml
    tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml
    tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml
* Parameters with template value using itself were substituted as is.
  For example: "timer: '{timer}'" was expanded to "{timer}". Now it
  leads to recursive parameter error.
  See changes in this test for example:
    tests/yamlparser/job_fixtures/parameter_name_reuse_default.*
    ->
    tests/yamlparser/error_fixtures/parameter_name_reuse_default.*
* When job group includes a job which was never declared, it was just
  ignored. Now it fails: job is missing.
  See changes in this test for example:
    tests/yamlparser/job_fixtures/job_group_includes_missing_job.*
    ->
    tests/yamlparser/error_fixtures/job_group_includes_missing_job.*

Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2023-02-28 20:16:57 +03:00
Vsevolod Fedorov
a47e4ee896 Tests: Add regression tests for already fixed stories
Change-Id: Ie31a557e5adb56db22254a074eebf3117ed4e268
2023-02-28 18:49:03 +03:00
Vsevolod Fedorov
e77555e242 Tests: Add tests for using tags in parameter definition
Change-Id: Ib9a6c567c50f37643a498c879b82d05d1c7a856e
2023-02-28 18:49:03 +03:00
Vsevolod Fedorov
de86b2c130 Tests: Add test for defaults defined at project
Change-Id: I0cafdd1b97625ef8a49d04f59c92d12694eb90d2
2023-02-16 11:53:50 +03:00
Vsevolod Fedorov
b3d9ad1cd2 Tests: project-type should also be taken from defaults
Change-Id: Idf5bff0f96eb34ffaedf44eafa7a6ccd4449dd3d
2023-02-15 11:36:11 +03:00
Vsevolod Fedorov
1f43cfb8a8 Tests: Add test for merging defaults for a job
Change-Id: I04cd8f080fea7b4cb87182db20ea4d164d6fa267
2023-02-14 13:50:19 +03:00
Vsevolod Fedorov
2a8df2ced9 Tests: Add tests for job and template using elements from defaults
Change-Id: I15ecc4a4b8328ff2cf5580e439e51b022a8f6089
2023-02-14 11:38:06 +03:00
Vsevolod Fedorov
fa1ac99ff0 Tests: Add test for dimension parameter used in template name
Change-Id: Ib37291de1d30fe694bb6e301f56428627bbc7eb5
2023-02-13 13:07:14 +03:00
Vsevolod Fedorov
8784d916a6 Tests: Add tests for including a list of files
There were no tests for !include: and !include-jinja2: with a list of
files.

Change-Id: Id786e0306ec0279e98ac3b3cb7da62e688bcb12e
2023-02-13 11:28:42 +03:00
Vsevolod Fedorov
0b17c8e793 Tests: Add test to demonstrate inter-parameter expansion
With new parser version, more inter-parameter expansions will be
available. Add test to demonstrate that: see it's diff on new parser
review.

Change-Id: I977aa2fddac7217f387da8ac513c748dd6a15bd1
2023-02-09 14:04:41 +03:00
Vsevolod Fedorov
5b0e350560 Tests: Add test for missing view
Change-Id: Ic70f056ed0ca0dd1a7dfb7cfd1bf2803d2725fcd
2023-01-26 11:11:13 +03:00
Vsevolod Fedorov
cd057a858a Tests: Add test for nested dimention parameter usage
Change-Id: I3c407b9ddfa5d2ad0c5041e2fb7953cea8935bbd
2023-01-26 11:11:13 +03:00
Vsevolod Fedorov
f66c4e6809 Tests: Add test for overriding dimention name parameter
Change-Id: I1f3c2bdf4301fbea200abc465629657d91957a05
2023-01-26 11:11:11 +03:00
Vsevolod Fedorov
c1a67471e8 Tests: Add test for project including missing job
Change-Id: I37aa76b4de4031f8302faf2848a6bb3dd186df30
2023-01-26 11:07:43 +03:00
Vsevolod Fedorov
5e5eee86c3 Tests: Add test for group including missing job
Change-Id: Ie3103257111f5f6a81f0aadd32f88f35222198c6
2023-01-26 11:07:43 +03:00
Vsevolod Fedorov
cb39c88c53 Tests: Add parameter name reuse tests
Change-Id: I9acfe721921500c49f2253210b6438b65269abbb
2023-01-26 11:07:43 +03:00
Vsevolod Fedorov
08196fe819 Tests: Add tests for missing and default parameters
Change-Id: Iaca0cd39576e99829b20373bf7203ec205a1088c
2023-01-26 11:07:43 +03:00
Vsevolod Fedorov
6b92807cd7 Tests: Rename yamlparser tests and fixture directories
Rename fixtures directory to job_fixtures to prepare for adding view_fixtures
directory in the following commits.

Change-Id: Ic20997cae020b542ddc22bf444fa6b92fbcae064
2023-01-26 11:06:23 +03:00
Vsevolod Fedorov
0e1b708ad5 Yaml parser tests: Add test for job parameters precedence
Change-Id: I6ebc93425b44a7981b3befc931486e0a7a5d253b
2022-12-20 10:44:43 +03:00
Vsevolod Fedorov
60b9c23d33 Yaml parser tests: Add tests for views expansion
Currently, views handling part in YamlParser class is not covered at all.

Change-Id: Ifaac0c44b0a0487e5673060d7b1f25393161f750
2022-12-20 10:44:43 +03:00
Vsevolod Fedorov
f4d64f9f66 Move tests to pytest
Pytest makes each scenario into individual selectable test.
To be able to run each scenario separately is very useful for development.

Change-Id: I4b1c990a1fd839ce327cd7faa27159a9b9632fed
2022-12-20 10:44:43 +03:00
Eren Atas
08678825a9 Update Gerrit Trigger Plugin Implementation
The changes between plugin version 2.30.0-2.32.0 have been
implemented. Current test cases are updated, and also new
test cases are added in order to verify old version XML
output has not been altered.

Documentation uses gerrit004.yaml as an exampple, therefore,
`build-cancellation-policy` have also been implemented to
show as example.

Signed-off-by: Eren Atas <eatas.contractor@libertyglobal.com>
Change-Id: I55fb6f8dd722d50cff369a6fba0e11c96b0c08aa
2021-05-20 15:14:46 +02:00
Ruslan Marinin
15a0896791 fixed broken active choice reactive param after job creation
Change-Id: I7e7362a73e9478bdbd445f5178f982afa346f58a
2021-04-29 20:24:20 +03:00
Zuul
57cd9e7dc2 Merge "Added support of Active Choice Reactive Parameter" 2021-04-10 14:00:37 +00:00
Igor Lakhtenkov
f810d8bc30 Added support of Active Choice Reactive Parameter
Change-Id: I25e926603682db543658aad82efb6eb41d1b5cb4
Task: #42221
2021-04-07 01:33:01 +03:00
sp-ricard-valverde
c1d8a41666 feat(scm): add git extension for lfs-pull
Add a missing extension to git scm which enables the option to do
"Git LFS pull after checkout"

Add lfs-pull fixtures

Change-Id: Ie8b98877f3f282ac0b32dbeb089e6d0d9ccc25c8
2021-03-16 18:29:56 +01:00
Zuul
4f615ee5ec Merge "Fix 'authorization' property one more time" 2021-02-15 19:53:03 +00:00
Adam Romanek
c0866c56b9 Fix 'authorization' property one more time
Due to the fact how dispatching works the 'authorization' property
handler was not always invoked through Properties.gen_xml(), leading to
a bug and an invalid test case: project-with-auth-properties.(yaml/xml)

This change pushes the logic for determining if the object is a
folder/multi-branch project from Properties.gen_xml() to the
authorization() function itself. For that to work the authorization()
function needed access to the top-level job object, which is now
conditionally passed to each dispatched function as a keyword argument,
if the function takes 'job_data' argument. Note that taking this
argument is completely optional so no changes were required in other
handlers. In the future the same approach could be taken to eliminate
the hacks for 'uno-choice' in Parameters.gen_xml().

Additionally ModuleRegistry.dispatch() now merges the top-level job
object with any template data before deep-formatting, so that job-level
properties are now available in Jinja templates. A very nice use case
is in project-with-auth-j2-yaml.yaml test case.

Change-Id: I9a49de74055cd9acfdc87dbad1fc454548643e8f
2021-02-02 16:20:52 +01:00
Adam Romanek
c4e2f03776 Fix support for !j2-yaml tag on project level (missing deepcopy impl)
The properties of 'project' objects are deepcopied in
YamlParser._expandYamlForTemplateJob(), including any Jinja2YamlLoader
objects produced by !j2-yaml tag. Because of the missing __deepcopy__()
definition in Jinja2YamlLoader, the parent class' implementation was
used, causing the deepcopied object be of Jinja2Loader type.
Consequently the rendered value was always a string, not an instance of
LateYamlLoader as it supposed to be.

This change fixes the issue in Jinja2YamlLoader and potentially in
LateYamlLoader, however the latter is currently not deepcopied anywhere
so no test could be written to cover it.

Change-Id: I24fda368a4af0e9aac7b78c478ac7b4eddf27b9b
2021-02-01 23:35:39 +01:00
Eren ATAS
6cad52986e Active Choices parameters support
Following parameters have been implemented in JJB:
- Active Choices Parameter
- Active Choices Reactive Reference Parameter

Test cases also have been written.

Change-Id: I6e953aa26114fde9ff293ed17cc17d5972df9af3
Signed-off-by: Eren ATAS <eatas.contractor@libertyglobal.com>
2020-10-12 14:00:54 +02:00
Zuul
cf8b2721ed Merge "Introduce a new '!j2-yaml:' tag" 2020-09-25 15:25:31 +00:00
Adam Romanek
6bc5398336 Introduce a new '!j2-yaml:' tag
The tag provides Jinja templating capabilities for generating parts of
YAML structures. Two potential use cases are provided as test cases and
are linked in the documentation.

The new tag should also help address some use cases people were asking
about, like here:
https://groups.google.com/g/jenkins-job-builder/c/HkVZVuBDlKM.

Change-Id: I96392e42c3c79a9be0a8f736506908701251dd62
2020-09-14 14:37:31 +02:00
Eren ATAS
a24a547de0 Updated ext-email support
Added also a test case for version specific changes

Change-Id: I5213dccffc441037c63ee7c042aee4a8a84d9732
Signed-off-by: Eren ATAS <eatas.contractor@libertyglobal.com>
2020-09-02 15:37:16 +02:00
Eren ATAS
15ee9899b8 Added property-multiline for parameterized build trigger plugin
Updated test cases as well, The tag has been introduced after 2.35.2
therefore the parameter is also >=2.35.2

Change-Id: Ie0801108a16969b420b341e2deaa6ba62deb116c
Signed-off-by: Eren ATAS <eatas.contractor@libertyglobal.com>
2020-07-31 14:12:00 +02:00