168 Commits

Author SHA1 Message Date
Jenkins
bed0f0f8d7 Merge "enhancements to install_packages macro" 2016-08-16 17:02:27 +00:00
Jenkins
4adc366d92 Merge "Dockerfiles for Networking-SFC Container" 2016-08-15 15:37:18 +00:00
Abe Music
29fcb04cf3 enhancements to install_packages macro
- Adds debian_package_install method to the jinja environment
  that will return the necessary set of commands needed to
  install packages via apt-get and/or URLs to .deb packages
- Updates install_packages macro to make use of the method
  above

Change-Id: Ie9318a7def54b5034ba91375fd8fd3b589d18349
Partially-implements: blueprint third-party-plugin-support
2016-08-15 08:17:13 -05:00
Robin Naundorf
a7c41a8893 Fix inconsistencies in git url
Github is just a mirror of the OpenStack git-repo.
Changed from Github to OpenStack git url wherever possible.

Change-Id: I7941ef86967de4efe7f23ff9fb11ec86c793901e
2016-08-06 14:21:23 +02:00
prithiv
eb41179a73 Dockerfiles for Networking-SFC Container
Networking-SFC is a neutron big-tent project
Kolla does not have Networking-SFC agent support for now. This
is essential for Service Function Chaining use case. Build Networking-SFC
Container.

Change-Id: I67b4d6e061dbe31e2211cd3210726ab4c30cd087
Partially-Implements: blueprint enable-networking-sfc-support
2016-08-06 12:06:39 +00:00
Lu lei
684b483fa8 Using sys.exit(main()) instead of main()
As we known, Exceptions are raised by the sys.exit() function. When they
are not handled, no stack traceback is printed in the Python interpreter.

In this patch we have known main() got return values(eg:0, 1), but
it can't specifies exit status when terminate the main thread by
unusing sys.exit(). So when using sys.exit(main()) instead of main()
may be more readable and reasonable.
TrivialFix.

Change-Id: I184289d28b92a7e345907247c045535a0c42c974
2016-08-02 09:32:28 +08:00
Swapnil Kulkarni (coolsvap)
843f934ede Make kolla-build list-dependencies option work
Change-Id: I465cc8a79b12c188569c9537e14763a6bef8dd43
Closes-Bug:#1606450
2016-07-27 12:37:11 +05:30
qinchunhua
9f596ffe4f Fix dict.values() PY3 compatible
The dict.values()[0] will raise a TypeError in PY3
as dict.values() doesn't return a list any more in PY3
but a view of list.

TrivialFix.

Change-Id: I01297bb68e45db4d15800d2d42e08560da7346bd
2016-07-27 03:55:51 +00:00
Dave Walker (Daviey)
77296ebdee Build docker images for OpenStack Watcher
Watcher is part of the OpenStack big-tent and is formally
known as "OpenStack Infrastructure Optimization service".

Whilst it provides a range of default goals and strategies,
the most relevant case is to enable re-balancing of the
compute hosts by taking CPU usage (data from Ceilometer)
into account and live migrating instances as required.

Currently this only builds the Docker images for type source
but binary builds are gracefully ignored.

The ansible configurations will be part of a later commit.

Change-Id: I9bb81ee625d9fcf6513e44e2ed20384e34da2adc
Partial-bug: #1598929
Partially-implements: bp watcher
Signed-off-by: Dave Walker (Daviey) <email@daviey.com>
2016-07-08 11:29:38 +01:00
Jenkins
02eec8a20c Merge "Add Senlin Docker container" 2016-07-08 09:59:12 +00:00
Eduardo Gonzalez
1d399461ac Add Senlin Docker container
Change-Id: I52e08e2df77106969c5975ebda2510ea8aefa8b5
Partially-implements: bp senlin-container
Signed-off-by: Eduardo Gonzalez <dabarren@gmail.com>
2016-06-30 16:36:25 +02:00
Joshua Harlow
43a38c179a Fix broken save,list images/dependencies options
It's sorta required to actually build the image
list (and strucutre) to make any of these commands
do anything.

TrivalFix

Change-Id: Ib601bd4d2cc84af6d35a8623b77f9b512124d2ad
2016-06-29 17:53:14 -07:00
Joshua Harlow
3bd1f14c4f Be smarter about what to do when making a docker client
Instead of having the program sys.exit when a docker
client object can not be created (which kills all the threads
and messes up the program and its associated state) have a
exception be raised when this (for whatever reason) fails.

Also refactor so that there is a docker task that the docker
client using tasks can all inherit from.

TrivalFix

Change-Id: Ie81aff10cfe6f2fc5c65d53402200e3928fb460c
2016-06-27 14:09:25 -07:00
Jenkins
01d183da27 Merge "Stop using a global logger for all the things" 2016-06-27 06:48:08 +00:00
Michal (inc0) Jastrzebski
dfb0e55740 Customizations continued
To correctly customize lists, we need to specify *_append, *_remove or
*_override variable in customization file.

Change-Id: I18d67ab89089e2696399ff1b99c1047a2f554442
Partially-implements: blueprint third-party-plugin-support
2016-06-24 13:56:20 +00:00
Jenkins
24404a1b5a Merge "Update urls for Delorean repo files" 2016-06-24 11:29:18 +00:00
Paul Bourke
3af14c3402 Update urls for Delorean repo files
delorean.repo is now redirected from trunk.rdoproject.org to
buildlogs.centos.org [0]

[0] https://www.redhat.com/archives/rdo-list/2016-June/msg00206.html

Closes-Bug: 1595568

Change-Id: I650abc34d9cf0c00f9025bd2c62f1bf03fbd96b7
2016-06-24 11:19:00 +01:00
Jenkins
8790492c6b Merge "Update kollaglue repo on docker to kolla" 2016-06-23 15:36:49 +00:00
Hui Kang
480c479ebf Add Kuryr Docker container
Co-authored-by: Mohammad Banikazemi <mb@us.ibm.com>
Partially-implements: bp kuryr-docker-plugin
Change-Id: Ife54f71984ff6c9825769e81fd39e3444c6fe310
2016-06-21 22:54:44 -04:00
Joshua Harlow
929d84b143 Stop using a global logger for all the things
Instead of using the same logger for all the things
which makes it incredibly hard to figure out what the images
and there builds are doing; instead have it so that each
image gets its own logger which is then used by tasks that
process that image.

Each images logs now will (optionally) go to a logs directory
that can be more easily read and looked over and leaves
the main logger for informational/task kind of
output only.

Closes-Bug: #1586478

Change-Id: I939e073a2bc5ed8b5a1e969888fc743d80f468c9
2016-06-21 15:25:05 -07:00
Michal (inc0) Jastrzebski
f1c3422582 Macro install packages
Simplification of package installations

Change-Id: I78c0a84c55e891dec7c9bd21bc61700dba244526
Partially-implements: blueprint third-party-plugin-support
2016-06-15 19:42:02 +00:00
Jenkins
4133778a5f Merge "Change the search path to look locally" 2016-06-14 19:02:13 +00:00
Jeffrey Zhang
3afe149e51 Implement Rally Dockerfile
Partially Implements: blueprint rally-container

Change-Id: If8c4fdb51b8083078dddc68598d48c6e139a615a
2016-06-14 14:30:03 +00:00
Swapnil Kulkarni (coolsvap)
3f063626a2 Update kollaglue repo on docker to kolla
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-June/097227.html

Change-Id: I73af59dcf1e755dbe0579486f1fb0ebe5ca0119c
Closes-Bug: #1592286
2016-06-14 12:34:34 +05:30
Ryan Hallisey
d360d4e5af Change the search path to look locally
In order to use tools/build.py, we need to search
locally for imports.

Closes-bug: #1592030
Change-Id: Idfa651c1268f93366de9f4e3fa80c33be42c71c3
2016-06-13 07:05:26 -04:00
TuanLAF
3772ac05f7 Fix typos in Kolla
TrivialFix

Change-Id: Id916352b419a100d1cae463031e3182bd1080776
2016-06-13 17:46:42 +07:00
Jenkins
9afb69c8de Merge "Mechanism for Dockerfile customization" 2016-06-12 23:58:54 +00:00
Jenkins
319fd3e794 Merge "Add non-intrusive ctrl-c handling" 2016-06-10 03:14:05 +00:00
Joshua Harlow
869e117b53 Add non-intrusive ctrl-c handling
Instead of trying to use a signal handler to stop
when ctrl-c is triggered use technique that cooperates
better with the threads that are running and lets
them die a happy death vs being forced to die in
unpleasant ways.

Closes-Bug: #1586476

Change-Id: I7fdb6a77a144bdd02276cca07b616bbb0c2f1957
2016-06-08 22:00:25 -07:00
Joshua Harlow
a45dc94e8c Use 'image' not 'item'
The things in self.images are images, not items so
reflect these to use the shared 'image' name vs
having a one-off that uses 'item'.

TrivalFix

Change-Id: I781514d3e116de32b4fe61a7f55455282680fec1
2016-06-08 17:50:39 -07:00
Michal (inc0) Jastrzebski
51074b0a42 Mechanism for Dockerfile customization
This patchset enables potentially full customization of Dockerfiles
provided by Kolla without having to change main Kolla code. Mechanism is
based on jinja2 block mechanism. In our dockerfiles we'll add blocks
(mostly empty) with highly standarized naming, then in override file
users will be able to fill/override contents of these blocks. Example is
shown in this patch.
To enable cutomized override file call build.py

--template-override=path-to-file Only caveat is that header {% extends
.. %} has to stay

Change-Id: I5957024e5652f9677c439fb2fcd524e467b951af
Partially-implements: blueprint third-party-plugin-support
2016-06-07 21:03:51 +00:00
Jeffrey Zhang
c31688be36 Make build.py importable lib
First step to break it down is to make it
importable lib rather than self-contained script.

Change-Id: Ibbc924c2849cc79006046f62f31a1909ce959066
Partially-implements: bp build-refactor
2016-06-07 16:40:16 +00:00
Jeffrey Zhang
9585c1be9f Remove the deprecated kolla-build section
TrivialFix

Change-Id: I3ee10efc093ed394c3af483a229dcd9dc732ec9a
2016-06-07 09:04:57 +08:00
Christian Berendt
442b4b6a94 Cleanup help string of install_type parameter
The part "The valid types are: binary, source, rdo, rhos" is already a
part of the generated kolla-build configuration file ("# Allowed values:
binary, source, rdo, rhos").

TrivialFix

Change-Id: I87b913c1b650fe54402ea643fc9c774448850f6b
2016-06-06 11:47:53 +00:00
Joshua Harlow
7c6990ccec Use an image object, recorder object and status constants
Instead of using raw dicts and passing data around via
dictionaries (which makes it really hard to figure out
what is in those dictionaries at any point) prefer to
use objects. That way people can actually understand what
the object is supposed to be, vs guessing and/or having to
decipher its usage.

The same goes for raw string constants, prefer using
named constants instead.

Closes-Bug: #1586475

Change-Id: Ide179dc6593c50696d47a2d3d4cd000f343855d4
2016-06-02 11:21:24 -07:00
Joshua Harlow
e78ae9bc61 Use a task model and dumb workers
Instead of having smart workers that need to know
about the full logic of how they operate and what
they will do which makes it hard to reuse parts of
those workers switch the model so that workers are
dumb and they just run tasks which themselves can
request additional tasks to run when succesful.

This makes it more consistent when tasks are to
be retried and also makes it easier to know which
task is running and when.

Less specialized workers == good.

Closes-Bug: #1586474

Change-Id: Ie7b7303b8bf2a80b3e26b6d4ffc27787f8c794d6
2016-06-02 11:21:09 -07:00
Jenkins
3a6a748aba Merge "Don't always unpack image build results tuple, as it may be None" 2016-06-01 08:35:53 +00:00
Jenkins
40cd7550dd Merge "Add sahara containers" 2016-06-01 05:16:17 +00:00
Jenkins
2bed574032 Merge "Pin the base distro release version" 2016-06-01 05:13:10 +00:00
Mark Goddard
b3e8ac8371 Don't always unpack image build results tuple, as it may be None
Change-Id: I21b1327da1de1bd3cc3018ba37715b0a19c1dbef
Closes-Bug: #1587501
2016-05-31 12:19:17 -05:00
Jenkins
d860c7d4ba Merge "Fix kolla-build return codes" 2016-05-31 11:08:39 +00:00
Jeffrey Zhang
4044a3475e Pin the base distro release version
Pin the base distro release version in the master branch(Newton).
Only one distro version is supported. The supported versions are:

* CentOS: 7
* RedHat: 7
* OracleLinux: 7
* Debian: 8
* Ubuntu: 14.04

NOTE: Kolla will move to Ubuntu 16.04 in the Newton cycle

DocImpact
TrivialFix

Change-Id: I32c0edbee4de14c8a0cc7c961d566e84d70f19a4
2016-05-31 01:41:48 +00:00
Daneyon Hansen
ba30579b3b Adds Neutron LBaaS Support
Previously, kolla did not support neutron lbaas functionality.
Only Lbaasv2 is supported in Mitaka. Additional information can
be found here:
http://docs.openstack.org/mitaka/networking-guide/adv-config-lbaas.html
Magnum uses Neutron Lbaas to provide high availability to COE API
and Etcd endpoints within a bay. Therefore, Neutron Lbaas is required
for Kolla to support Magnum.

Co-Authored-By: Serguei Bezverkhi <sbezverk@cisco.com>
Partial-Bug: #1551992

Change-Id: I05360b7c447c601fcb3c2b6b2a913ef5cc0f3a1b
2016-05-27 11:41:25 -04:00
Will Miller
18ae114e75 Fix kolla-build return codes
Add a main wrapper function to return 1 if any images failed to build;
otherwise, return 0. Rename the old main() to run_build(), which still
returns image statuses, to avoid interfering with unit tests.

Change-Id: I5817cc9cebb4f42dc13e0535223e5dc8931a3aab
Co-Authored-By: Mark Goddard <mgoddard@cray.com>
Closes-Bug: #1578404
2016-05-27 06:39:52 -05:00
Swapnil Kulkarni (coolsvap)
353d077e14 Add sahara containers
Added sahara container for
centos,ubuntu
binary,source

partially-implements blueprint sahara-container

Change-Id: Id6dad447cd6ec07286e7a6cab0b13a6d4d89c1a7
2016-05-26 16:52:07 +05:30
Jenkins
857d8db60c Merge "Implement non-default signal handler for SIGINT" 2016-05-26 06:38:14 +00:00
Swapnil Kulkarni (coolsvap)
e1b5b149f2 Make passwords.yml file generation configurable
partially implements blueprint multiple-clouds

Change-Id: I676c4245e6f058ffbed345970ee78d1750dd0f2f
2016-05-21 01:51:17 +00:00
Jenkins
cf3e68213e Merge "Catch all the exception in the push thread" 2016-05-10 21:02:47 +00:00
Jeffrey Zhang
070bf25835 Support a custom base image
Add a new option(base-image) to support custom base image
In default, the base-image is equal to base. When using custom
image, please use command line like:

    ./tool/build.py --base-image my-image --base centos --base-tag 0.1

DocImpact

Closes-Bug: #1573544
Change-Id: If4b9c57a2b68f6f06d3e981f30b2173064e02193
2016-05-09 07:29:19 +00:00
Jeff Peeler
75492cf82b Implement non-default signal handler for SIGINT
Pressing Ctrl-C will stop a build, but does not clean up the /tmp
directory of dockerfiles. Implement a signal handler such that clean up
does occur.

Python apparently ignores signals while calling join on a queue, so
that's why the small modification was needed there.

Change-Id: I43faf67726c154b6640d3596aea61ffc59871999
Closes-Bug: #1579242
2016-05-06 17:37:42 -04:00