As far back as its introduction (2cefa400), the `--delete-old` option
has only ever deleted jobs that were marked with the special
"<!-- Managed by Jenkins Job Builder -->" comment that JJB adds to
descriptions.
This change fixes the documentation of the `--delete-old` option,
which erroneously used to state that even jobs that were never
managed by JJB would be subject to deletion.
The test for the feature was slightly enriched as well: the fact
that unmanaged jobs were not deleted, was not actually tested.
Change-Id: I438a7b555b6e122869988b3a2b9ea40896004122
The system-global path of
/etc/jenkins_jobs/jenkins_jobs.ini
is not applicable on Windows. To solve that, and as Linux users would
benefit from a per-user configuration file, too, introduce a fallback at
~/.config/jenkins_jobs/jenkins_jobs.ini
following the XDG Base Directory Specification. Try to use that per-user
file if no configuration file is given on the command line and none is
present in the script directory for development purposes.
Change-Id: I1a22454999ee8ac60d6f22545850c33aa03bb29b
Support taking the order of the parameters to be passed through to a
triggered parameterized build from the user defined yaml via config
option. Use the existing order by default so as not to break current
usage.
The XML ordering of the parameters matters since when the same parameter
is defined through multiple sources, Jenkins uses the ordering to
determine which will be the one that that should win.
ie if FOO=BAR is set as a build parameter of the current job and a
properties file is generated during build that attempts to set FOO=BAZ;
the XML ordering determines which value, BAZ or BAR, that the triggered
job sees for the parameter FOO.
Change-Id: Ia0dafad8ed3976e7b0b11feb6b3af844bbda9d4a
Document the ability to define default values for template variables
within the template definition thus removing the need to always having
to set such values in job definitions.
Change-Id: If4d8bfd898066b8e6350ed22f0c97eb939ae4348
It is useful to allow templates that can have different settings for
slightly different jobs while still following the same naming scheme.
Additionally this allows for shorter and more descriptive names to be
used without the confusion of the variable placeholders, and for the
resulting job names from the templates to be altered without needing to
update all references.
Change-Id: Idc3517b44873210a33f988ebff449ea2ed567054
* Only update_jobs uses the parallel features right now
* --workers N
- If set to 0, it will use parallel execution and
use the number of cores in the machine as thread count
- Any other value enables the parallel extensions and sets the
number of concurrent threads to that value
- Will use 1 by default
Add some tests to make sure the parallel execution helpers work as
expected.
Change-Id: Ib0abd34ea7525f75fff4ff480287a6e589deba90
Signed-off-by: David Caro <dcaroest@redhat.com>
We don't want to unconditionally disable plugin querying just because
--conf is not given. That would mean that the query_plugins_info field
in the default paths would always be ignored, even if set to True.
This patch tweaks the logic so that we remember in options.conf the
actual name of the config file that we read from, whether it be from
--conf or from one of the default paths.
This way, the `test` command will still work out of the box, as commit
f68831b envisioned. But query_plugins_info is now checked if the user
has a configuration file (whether it be a default one, or one specified
through --conf).
Change-Id: Id71ad207f0419f7cc52eca174dd859d4182d7250
Story: 2000387
Change If84778231b provided an option to set a connection timeout
in the python-jenkins library. This change allows JJB to override
that timeout value.
Add tests to ensure that the timeout is only set when specified via the
config options.
Change-Id: I3dfe9139469dd0e8549eeedb1833c55ac79ea8b5
As per now if you want to configure the stash publisher and give it
a username and password, you need to set them in clear text within
the yaml file. This pull request gives the possibility to set these
credentials within the jenkins_jobs.ini file and therefore makes it
possible to not expose them to everyone.
Change-Id: I2e5d6caefb87ded2468c7b7e015e20464ffef99e
This adds some notes on parameter expansion with macros and especially
calls out some of the details around using "{" with shell-scripts.
Change-Id: Ibef92a60b9f44f7dc70db65aff7d97b3a665a6bf
This change does the following:
1. add a quick start guide
2. move the info about executing jjb from installation.rst
to execution.rst and slightly update the verbage.
The intention is to kick off this guide with a few simple use
cases not to have a comprehensive set of use cases.
Change-Id: I719c6dec5cc5f2731ef95094b76ec54c12b41b8e
Add documentation to let users know that they can also define jobs
in JSON. Create a new test suite to verify that jjb can correctly
generate jenkins xml from json definitions.
Change-Id: Idb5216b0b56837d0f4dd2298986149be66bd217e
Anchors and aliases were expanded internally within JJB's yaml loading
calls so they were limited to individual documents. Now, included files
will have access to aliases of anchors already defined at previously
processed files.
Example:
- default:
name: default-timeout-wrapper
timeout: &timeout
fail: true
elastic-percentage: 150
elastic-default-timeout: 90
type: elastic
- wrapper: !include include002_1.yaml.inc
Previously was not possible to use '*timeout' alias inside
include002_1.yaml.inc file
Closes-Story: 2000173
Change-Id: Ic031ddbb0310bd11748183fbde9502735c3b7169
This reverts commit 7100a7f225e60936f23d24c757c1b14c9b136ec4.
The referenced commit, in addition to what it advertises, also
filters the result of update_jobs to include only changed jobs.
If there are no change jobs, then, if the user specifies
--delete-old, there are no jobs in the list of jobs to keep
which comes from the result of update_jobs. In that case, all
jobs are deleted. Or in other words, the logic in this change
is that all jobs not updated are deleted.
Proposing as a revert because this is currently causing serious
problems for continuously deployed JJB instances, and the very
common --delete-old code path should be properly tested with
this change before it lands again.
Change-Id: I98443f0c085e27ed8dfece6409434015ac24b306
'raw' allow users to have a fallback in case
a plugin is not yet supported or the plugin is not generating the
expected result.
Only intended as a last fallback, but useful when waiting for
review to complete.
Change-Id: If0d22d7d43d35649e78aa7481e1c0f1ed21a6025
* Only update_jobs uses the parallel features right now
* --workers N
- if set to 0, it will use parallel execution and
use the number of cores in the machine as thread count
- Any other value enables the paralllel extensions and sets the
number of concurrent threads to that value
- Will use 1 by default
Added some tests to make sure the parallel execution helpers work as
expected
Change-Id: Iee8426ff2dc4c5b4ebec4c15bdd21cae5b47eb83
Signed-off-by: David Caro <dcaroest@redhat.com>
The goal of this patch is simply to move some classes out of
jenkins_jobs.builder into more appropriately-named modules. This started with
simply moving YamlParser into jenkins_jobs.parser but led to other moves in
order to avoid cyclic imports since YamlParser uses other classes previously
defined in jenkins_jobs.builder.
That said, this patch doesn't intend to address all of the clutter in
jenkins_jobs.builder, mostly just what is necessary to get started working on
YamlParser independent of other classes in that module.
Change-Id: Ie88bf683e495033eb0b670fe29c256a70282735f
Limit querying for plugin information from Jenkins to where an explicit
configuration file has been provided. Where no file is provided, should
avoid any requests that could result in attempting connections.
Add a config options to control this behaviour explicitly so that if a
config file is provided, it may explicitly disable querying jenkins for
additional information from plugins.
The `test` command can use this behaviour to ensure that a basic run is
possible without providing any configuration files, and can be
explicitly configured to disable querying should a config file be used
for testing.
Ensure the default sample configuration skips attempting to query so
that users may use it as a quick test for a working installation without
any network or valid jenkins url dependencies.
Change-Id: I641a3188013debf3765aaff109c4f5fa82a4cedb
Added a new configuration option under the section "job_builder" named
"allow_empty_variables" that if set to true, will replace non existing variables
in strings with the empty string instead of raising an error.
It's very useful if you have a shell script, that has optional values, for
example:
EXTRA_PACKAGES=({extra-packages})
for package in "${EXTRA_PACKAGES[@]}"; do
install "$package"
done
Or modifying the script behavior with flags, with empty as default value:
WITH_AUTOGEN={with-autogen}
if [[ $WITH_AUTOGEN ]]; then
./autogen.sh
fi
Then if you have two different jobs that use that script in their builder, you
just have to set the extra parameter or ignore it to change the behavior:
- builder:
name: mybuilder
builders:
- shell: !include shell-scripts/myscript.sh
- job-template:
name: 'mytpl-{name}'
...
builders:
- mybuilder
- project:
name: myproj1
with-autogen: true
jobs:
- 'mytpl-{name}'
- project:
name: myproj2
extra-packages: |
extrapkg1
extrapkg2
Change-Id: Iad9f0e522725e6fd6681cd62d3e36f69baf09585
Signed-off-by: David Caro <dcaroest@redhat.com>
Adds two tests to ensure correct behaviour with referencing yaml
anchors and a third test to verify the expansion internally of yaml
anchors and aliases.
* test that anchors are not carried across subsequent top level
invocations of yaml.load(). This will be used subsequently to ensure
that where anchors are allowed across included files they are
correctly reset on each top level call.
* test that where anchors are defined in a top level file and
subsequently included files, duplicate anchors raise exceptions as
though they were defined within the same file.
* test that data returned from yaml loading contains the additional data
specified by the alias. Uses json to force conversion so that the
outputted yaml contains the results of the anchors and aliases instead
of them.
Update documentation to contain details of the use of anchors and
aliases including a refernce to a simple generic example from the
specification as well as a JJB specific example.
Change-Id: I0f2b55e1e2f2bad09f65b1b981baa0159372ee10
Reduce the links to the Jenkins wiki for references to plugin pages by
using the sphinx.ext.extlinks module to provide a custom alias for
generating links using the same base url.
Change-Id: Ifba460cd131b4630deb41356e2b32641efe3086a
Use sphinx's linkcheck builder to validate any external links.
Fix the broken links by converting the example link to a code segment
and correcting the link referencing the cgit mirror for the git clone
instructions.
Change-Id: Ic6236e250b7af5c7c000af1e67fb69dd720ad6d8
Include an additional option to exclude some paths/patterns from being
included in the search to allow a complex hierarchy of directories to be
referenced under a single path. Provide support to allow the user to
specify patterns, relative and/or absolute path references.
Implements support for excluding based on absolute path, relative path
and based on simple shell globing patterns.
Change-Id: I236b364c268fd2bf5511a2c6d9a1c87914f3b086
The jenkins puppet link in docs is invalid as it links to non-existent
project called 'config' . The correct link is updated in installation
docs.
Closes-Bug story 2000145
story link: https://storyboard.openstack.org/#!/story/2000145
Change-Id: I542280b99508e80a1cec4f87395e328cb567f134