151 Commits

Author SHA1 Message Date
Jonathan Perret
2a050be26d Fix documentation of --delete-old: affects only managed jobs.
As far back as its introduction (2cefa400), the `--delete-old` option
has only ever deleted jobs that were marked with the special
"<!-- Managed by Jenkins Job Builder -->" comment that JJB adds to
descriptions.

This change fixes the documentation of the `--delete-old` option,
which erroneously used to state that even jobs that were never
managed by JJB would be subject to deletion.

The test for the feature was slightly enriched as well: the fact
that unmanaged jobs were not deleted, was not actually tested.

Change-Id: I438a7b555b6e122869988b3a2b9ea40896004122
2016-07-13 17:26:22 +02:00
Alexander Couzens
ca40fb7f87 doc/definition.rst: add missing '/' into include path
Change-Id: If07710333cbb84ce70d6d4fa40602a74c898c08a
2016-06-30 22:15:50 +02:00
Jenkins
223cb3530d Merge "Document template variable default values" 2016-04-26 15:04:06 +00:00
Terry Howe
aa41127ef7 Change documentation to install from pypi
Change-Id: I46fe46d0af14563b2e87492c2abe1b39ec246714
2016-04-23 12:20:44 +00:00
Jenkins
88ae2df307 Merge "Take parameter order from yaml in trigger_parameterized_builds" 2016-04-22 20:38:37 +00:00
Jenkins
5d14a621a0 Merge "Allow id's for different templates to follow same naming" 2016-04-21 23:13:59 +00:00
Jenkins
dc7e1b0aa2 Merge "Added possibility to exclude project combinations" 2016-04-21 16:36:16 +00:00
Sebastian Schuberth
dbe63dd9d2 cmd: Add a config file fallback to the user's home directory
The system-global path of

    /etc/jenkins_jobs/jenkins_jobs.ini

is not applicable on Windows. To solve that, and as Linux users would
benefit from a per-user configuration file, too, introduce a fallback at

    ~/.config/jenkins_jobs/jenkins_jobs.ini

following the XDG Base Directory Specification. Try to use that per-user
file if no configuration file is given on the command line and none is
present in the script directory for development purposes.

Change-Id: I1a22454999ee8ac60d6f22545850c33aa03bb29b
2016-04-18 15:25:56 +02:00
David Caro
d4fa9d7b23 Added possibility to exclude project combinations
Change-Id: I40603f041d33a27d08bf6912b7d9d885a95278fd
Signed-off-by: David Caro <dcaroest@redhat.com>
2016-03-12 17:51:01 +00:00
Darragh Bailey
3e8c31fe94 Take parameter order from yaml in trigger_parameterized_builds
Support taking the order of the parameters to be passed through to a
triggered parameterized build from the user defined yaml via config
option. Use the existing order by default so as not to break current
usage.

The XML ordering of the parameters matters since when the same parameter
is defined through multiple sources, Jenkins uses the ordering to
determine which will be the one that that should win.

ie if FOO=BAR is set as a build parameter of the current job and a
properties file is generated during build that attempts to set FOO=BAZ;
the XML ordering determines which value, BAZ or BAR, that the triggered
job sees for the parameter FOO.

Change-Id: Ia0dafad8ed3976e7b0b11feb6b3af844bbda9d4a
2016-03-07 14:10:32 +00:00
Adam Harwell
9bb036379c Update docs to include installation via pypi
Change-Id: Iad36cba6cc27770c69499ec7c402ff48f1f0380e
2016-03-03 12:26:37 -06:00
Darragh Bailey
a54e623dc7 Document template variable default values
Document the ability to define default values for template variables
within the template definition thus removing the need to always having
to set such values in job definitions.

Change-Id: If4d8bfd898066b8e6350ed22f0c97eb939ae4348
2016-02-20 18:20:51 +00:00
Darragh Bailey
398e28b3c3 Allow id's for different templates to follow same naming
It is useful to allow templates that can have different settings for
slightly different jobs while still following the same naming scheme.
Additionally this allows for shorter and more descriptive names to be
used without the confusion of the variable placeholders, and for the
resulting job names from the templates to be altered without needing to
update all references.

Change-Id: Idc3517b44873210a33f988ebff449ea2ed567054
2016-02-20 18:20:51 +00:00
Jenkins
0fc023b911 Merge "Add parallelization options" 2015-12-23 22:43:49 +00:00
Jenkins
ffc5632a4e Merge "Add MultiJob Project Documentation." 2015-12-03 05:08:28 +00:00
Jonathan Lebon
d08ef26bbf Document the new {template-name} variable
Doc change only to document the {template-name} variable added in commit
7ef05f7.

Change-Id: I88cfd14e2fa642fd4d6b9ac8d002c17fb8ef7dd9
2015-12-01 10:56:07 -05:00
David Caro
333fdc65a1
Add parallelization options
* Only update_jobs uses the parallel features right now
* --workers N
    - If set to 0, it will use parallel execution and
      use the number of cores in the machine as thread count
    - Any other value enables the parallel extensions and sets the
      number of concurrent threads to that value
    - Will use 1 by default

Add some tests to make sure the parallel execution helpers work as
expected.

Change-Id: Ib0abd34ea7525f75fff4ff480287a6e589deba90
Signed-off-by: David Caro <dcaroest@redhat.com>
2015-12-01 10:05:04 -05:00
Jonathan Lebon
72b259c845 cmd.py: tweak plugin information querying logic
We don't want to unconditionally disable plugin querying just because
--conf is not given. That would mean that the query_plugins_info field
in the default paths would always be ignored, even if set to True.

This patch tweaks the logic so that we remember in options.conf the
actual name of the config file that we read from, whether it be from
--conf or from one of the default paths.

This way, the `test` command will still work out of the box, as commit
f68831b envisioned. But query_plugins_info is now checked if the user
has a configuration file (whether it be a default one, or one specified
through --conf).

Change-Id: Id71ad207f0419f7cc52eca174dd859d4182d7250
Story: 2000387
2015-10-30 09:30:28 -04:00
Sebastian Schuberth
55fa8fda09 quick-start: Fix the configuration file argument to read "--conf"
Change-Id: Id9f9290cd8e7eabd3c3dbb5ee27cf3e3e1d52176
2015-10-20 18:22:36 +02:00
Khai Do
55e0fdc86c Allow JJB to set connection timeout to jenkins server
Change If84778231b provided an option to set a connection timeout
in the python-jenkins library.  This change allows JJB to override
that timeout value.

Add tests to ensure that the timeout is only set when specified via the
config options.

Change-Id: I3dfe9139469dd0e8549eeedb1833c55ac79ea8b5
2015-09-21 09:36:55 -07:00
Nicolas Glayre
8855eb2523 Stash publisher's credentials configuration
As per now if you want to configure the stash publisher and give it
a username and password, you need to set them in clear text within
the yaml file. This pull request gives the possibility to set these
credentials within the jenkins_jobs.ini file and therefore makes it
possible to not expose them to everyone.

Change-Id: I2e5d6caefb87ded2468c7b7e015e20464ffef99e
2015-08-24 15:23:47 +02:00
Wayne
d919261bcd Add MultiJob Project Documentation.
Change-Id: I5a2f86050a59d7ee1ef23f25c51ec4591908d0ff
2015-08-19 09:53:07 -07:00
Ian Wienand
25d81389e1 Enhance macro expansion documentation
This adds some notes on parameter expansion with macros and especially
calls out some of the details around using "{" with shell-scripts.

Change-Id: Ibef92a60b9f44f7dc70db65aff7d97b3a665a6bf
2015-08-14 13:14:19 +10:00
Jenkins
19aed6592f Merge "Add a JJB quick start guide" 2015-08-12 21:45:17 +00:00
Jenkins
2854ad8942 Merge "Document that jjb supports json job definitions" 2015-08-12 19:48:02 +00:00
Khai Do
6685326052 Add a JJB quick start guide
This change does the following:
1. add a quick start guide
2. move the info about executing jjb from installation.rst
   to execution.rst and slightly update the verbage.

The intention is to kick off this guide with a few simple use
cases not to have a comprehensive set of use cases.

Change-Id: I719c6dec5cc5f2731ef95094b76ec54c12b41b8e
2015-08-12 12:03:06 -07:00
Jenkins
297078d405 Merge "Support for externaljob project-type for monitoring external jobs" 2015-08-07 10:16:43 +00:00
Somay Jain
92b7f446f9 Support for externaljob project-type for monitoring external jobs
Change-Id: I8904725f75d27180d44c55768605db7d67e79586
2015-08-01 12:45:40 -07:00
Khai Do
e52bb31fa1 Document that jjb supports json job definitions
Add documentation to let users know that they can also define jobs
in JSON.  Create a new test suite to verify that jjb can correctly
generate jenkins xml from json definitions.

Change-Id: Idb5216b0b56837d0f4dd2298986149be66bd217e
2015-07-28 17:20:29 -07:00
Atsushi SAKAI
0dc94847a8 Fix two typos and Fix readability
apropriate => appropriate
behaviour  => behavior

tha toption => that option

Change-Id: I97b771752062861f55f372b03cd1f5449482b6ef
2015-07-27 09:10:57 +09:00
Victor Seva
1191dcfccf Allow use of aliases defined previously inside included files
Anchors and aliases were expanded internally within JJB's yaml loading
calls so they were limited to individual documents. Now, included files
will have access to aliases of anchors already defined at previously
processed files.

Example:

- default:
    name: default-timeout-wrapper
    timeout: &timeout
      fail: true
      elastic-percentage: 150
      elastic-default-timeout: 90
      type: elastic

- wrapper: !include include002_1.yaml.inc

Previously was not possible to use '*timeout' alias inside
include002_1.yaml.inc file

Closes-Story: 2000173

Change-Id: Ic031ddbb0310bd11748183fbde9502735c3b7169
2015-05-14 14:01:11 +02:00
Jenkins
5be7b768de Merge "Revert "Added parallelization options"" 2015-05-05 22:58:08 +00:00
James E. Blair
f74ce34d36 Revert "Added parallelization options"
This reverts commit 7100a7f225e60936f23d24c757c1b14c9b136ec4.

The referenced commit, in addition to what it advertises, also
filters the result of update_jobs to include only changed jobs.
If there are no change jobs, then, if the user specifies
--delete-old, there are no jobs in the list of jobs to keep
which comes from the result of update_jobs.  In that case, all
jobs are deleted.  Or in other words, the logic in this change
is that all jobs not updated are deleted.

Proposing as a revert because this is currently causing serious
problems for continuously deployed JJB instances, and the very
common --delete-old code path should be properly tested with
this change before it lands again.

Change-Id: I98443f0c085e27ed8dfece6409434015ac24b306
2015-05-05 18:30:03 +00:00
Jenkins
85634eb37c Merge "Add support for 'raw' to inject direct xml to job configs." 2015-05-04 16:05:51 +00:00
Max Rydahl Andersen
cb876b481e Add support for 'raw' to inject direct xml to job configs.
'raw' allow users to have a fallback in case
a plugin is not yet supported or the plugin is not generating the
expected result.

Only intended as a last fallback, but useful when waiting for
review to complete.

Change-Id: If0d22d7d43d35649e78aa7481e1c0f1ed21a6025
2015-05-02 15:42:32 -07:00
David Caro
7100a7f225 Added parallelization options
* Only update_jobs uses the parallel features right now
* --workers N
    - if set to 0, it will use parallel execution and
      use the number of cores in the machine as thread count
    - Any other value enables the paralllel extensions and sets the
      number of concurrent threads to that value
    - Will use 1 by default

Added some tests to make sure the parallel execution helpers work as
expected

Change-Id: Iee8426ff2dc4c5b4ebec4c15bdd21cae5b47eb83
Signed-off-by: David Caro <dcaroest@redhat.com>
2015-04-30 21:33:08 +02:00
Jenkins
f6a8cfede5 Merge "Remove YamlParser from jenkins_jobs.builder" 2015-04-23 17:07:59 +00:00
Jenkins
49739b57ec Merge "Only query jenkins plugins if config provided" 2015-04-23 14:52:46 +00:00
Wayne
4a8b93b8c2 Remove YamlParser from jenkins_jobs.builder
The goal of this patch is simply to move some classes out of
jenkins_jobs.builder into more appropriately-named modules. This started with
simply moving YamlParser into jenkins_jobs.parser but led to other moves in
order to avoid cyclic imports since YamlParser uses other classes previously
defined in jenkins_jobs.builder.

That said, this patch doesn't intend to address all of the clutter in
jenkins_jobs.builder, mostly just what is necessary to get started working on
YamlParser independent of other classes in that module.

Change-Id: Ie88bf683e495033eb0b670fe29c256a70282735f
2015-04-22 11:59:07 -07:00
Wayne
d2f9258864 Add support for hipchat notifier 0.1.8
Change-Id: I834dc05465ea5a24e3743f400837482a82e02e70
Closes-Bug: 1377567
2015-04-17 14:52:59 -07:00
Darragh Bailey
f68831bd2e Only query jenkins plugins if config provided
Limit querying for plugin information from Jenkins to where an explicit
configuration file has been provided. Where no file is provided, should
avoid any requests that could result in attempting connections.

Add a config options to control this behaviour explicitly so that if a
config file is provided, it may explicitly disable querying jenkins for
additional information from plugins.

The `test` command can use this behaviour to ensure that a basic run is
possible without providing any configuration files, and can be
explicitly configured to disable querying should a config file be used
for testing.

Ensure the default sample configuration skips attempting to query so
that users may use it as a quick test for a working installation without
any network or valid jenkins url dependencies.

Change-Id: I641a3188013debf3765aaff109c4f5fa82a4cedb
2015-04-17 17:16:33 +01:00
David Caro
814ba7575f Added possibility to use non-existent keys
Added a new configuration option under the section "job_builder" named
"allow_empty_variables" that if set to true, will replace non existing variables
in strings with the empty string instead of raising an error.

It's very useful if you  have a shell script, that has optional values, for
example:

 EXTRA_PACKAGES=({extra-packages})
 for package in "${EXTRA_PACKAGES[@]}"; do
    install "$package"
 done

Or modifying the script behavior with flags, with empty as default value:

 WITH_AUTOGEN={with-autogen}
 if [[ $WITH_AUTOGEN ]]; then
    ./autogen.sh
 fi

Then if you have two different jobs that use that script in their builder, you
just have to set the extra parameter or ignore it to change the behavior:

 - builder:
   name: mybuilder
   builders:
      - shell: !include shell-scripts/myscript.sh

 - job-template:
   name: 'mytpl-{name}'
   ...
   builders:
     - mybuilder

 - project:
   name: myproj1
   with-autogen: true
   jobs:
     - 'mytpl-{name}'

 - project:
   name: myproj2
   extra-packages: |
     extrapkg1
     extrapkg2

Change-Id: Iad9f0e522725e6fd6681cd62d3e36f69baf09585
Signed-off-by: David Caro <dcaroest@redhat.com>
2015-03-24 19:20:49 +01:00
Darragh Bailey
1e70128d30 Add tests for yaml anchor behaviour
Adds two tests to ensure correct behaviour with referencing yaml
anchors and a third test to verify the expansion internally of yaml
anchors and aliases.

* test that anchors are not carried across subsequent top level
  invocations of yaml.load(). This will be used subsequently to ensure
  that where anchors are allowed across included files they are
  correctly reset on each top level call.
* test that where anchors are defined in a top level file and
  subsequently included files, duplicate anchors raise exceptions as
  though they were defined within the same file.
* test that data returned from yaml loading contains the additional data
  specified by the alias. Uses json to force conversion so that the
  outputted yaml contains the results of the anchors and aliases instead
  of them.

Update documentation to contain details of the use of anchors and
aliases including a refernce to a simple generic example from the
specification as well as a JJB specific example.

Change-Id: I0f2b55e1e2f2bad09f65b1b981baa0159372ee10
2015-03-10 14:27:30 +00:00
Jenkins
8ecfd7e07a Merge "Use sphinx extlinks to shorten links to Jenkins wiki" 2015-03-05 15:49:13 +00:00
Jenkins
3c3825a8dc Merge "Check external documentation links are valid" 2015-03-05 15:48:18 +00:00
Darragh Bailey
753b3b46dc Use sphinx extlinks to shorten links to Jenkins wiki
Reduce the links to the Jenkins wiki for references to plugin pages by
using the sphinx.ext.extlinks module to provide a custom alias for
generating links using the same base url.

Change-Id: Ifba460cd131b4630deb41356e2b32641efe3086a
2015-02-24 21:37:17 +00:00
Darragh Bailey
302255ac60 Check external documentation links are valid
Use sphinx's linkcheck builder to validate any external links.

Fix the broken links by converting the example link to a code segment
and correcting the link referencing the cgit mirror for the git clone
instructions.

Change-Id: Ic6236e250b7af5c7c000af1e67fb69dd720ad6d8
2015-02-24 21:36:54 +00:00
Darragh Bailey
c5c0eb4f16 Support excludes when recursively processing paths
Include an additional option to exclude some paths/patterns from being
included in the search to allow a complex hierarchy of directories to be
referenced under a single path. Provide support to allow the user to
specify patterns, relative and/or absolute path references.

Implements support for excluding based on absolute path, relative path
and based on simple shell globing patterns.

Change-Id: I236b364c268fd2bf5511a2c6d9a1c87914f3b086
2015-02-17 23:15:56 +00:00
Jenkins
74c8ac4561 Merge "Interpolate params inside of params." 2015-02-05 23:40:47 +00:00
imran
efe02049d6 Fix invalid link to jenkins puppet module
The jenkins puppet link in docs is invalid as it links to non-existent
project called 'config' . The correct link is updated in installation
docs.

Closes-Bug story 2000145
story link: https://storyboard.openstack.org/#!/story/2000145

Change-Id: I542280b99508e80a1cec4f87395e328cb567f134
2015-02-02 10:52:04 -08:00