14 Commits

Author SHA1 Message Date
gong yong sheng
dd57e1df5a Add iter function to change list into iterator
Change-Id: Ia141a54b1bc0b1742a396b7029ee0b56ead648db
Closes-bug: 1623288
2016-09-14 10:04:10 +08:00
Robin Naundorf
a7c41a8893 Fix inconsistencies in git url
Github is just a mirror of the OpenStack git-repo.
Changed from Github to OpenStack git url wherever possible.

Change-Id: I7941ef86967de4efe7f23ff9fb11ec86c793901e
2016-08-06 14:21:23 +02:00
Joshua Harlow
929d84b143 Stop using a global logger for all the things
Instead of using the same logger for all the things
which makes it incredibly hard to figure out what the images
and there builds are doing; instead have it so that each
image gets its own logger which is then used by tasks that
process that image.

Each images logs now will (optionally) go to a logs directory
that can be more easily read and looked over and leaves
the main logger for informational/task kind of
output only.

Closes-Bug: #1586478

Change-Id: I939e073a2bc5ed8b5a1e969888fc743d80f468c9
2016-06-21 15:25:05 -07:00
Jeffrey Zhang
c31688be36 Make build.py importable lib
First step to break it down is to make it
importable lib rather than self-contained script.

Change-Id: Ibbc924c2849cc79006046f62f31a1909ce959066
Partially-implements: bp build-refactor
2016-06-07 16:40:16 +00:00
Joshua Harlow
7c6990ccec Use an image object, recorder object and status constants
Instead of using raw dicts and passing data around via
dictionaries (which makes it really hard to figure out
what is in those dictionaries at any point) prefer to
use objects. That way people can actually understand what
the object is supposed to be, vs guessing and/or having to
decipher its usage.

The same goes for raw string constants, prefer using
named constants instead.

Closes-Bug: #1586475

Change-Id: Ide179dc6593c50696d47a2d3d4cd000f343855d4
2016-06-02 11:21:24 -07:00
Joshua Harlow
e78ae9bc61 Use a task model and dumb workers
Instead of having smart workers that need to know
about the full logic of how they operate and what
they will do which makes it hard to reuse parts of
those workers switch the model so that workers are
dumb and they just run tasks which themselves can
request additional tasks to run when succesful.

This makes it more consistent when tasks are to
be retried and also makes it easier to know which
task is running and when.

Less specialized workers == good.

Closes-Bug: #1586474

Change-Id: Ie7b7303b8bf2a80b3e26b6d4ffc27787f8c794d6
2016-06-02 11:21:09 -07:00
Mark Goddard
b3e8ac8371 Don't always unpack image build results tuple, as it may be None
Change-Id: I21b1327da1de1bd3cc3018ba37715b0a19c1dbef
Closes-Bug: #1587501
2016-05-31 12:19:17 -05:00
Vikram Hosakote
320c1f9272 Add timeout to requests.get() in kolla/cmd/build.py
When kolla-build is running, if there are network issues or if the
source's location (like http://tarballs.openstack.org) fails to
respond due to high number of concurrent requests, kolla-build just
hangs/blocks indefinitely.

This patch set resolves this issue by adding a timeout of 120
seconds for requests.get() in kolla/cmd/build.py, adds a unit test
for it in kolla/tests/test_build.py and also the "timeout" argument
in kolla/common/config.py.

Change-Id: I7c8745a20b9bd1c3f5d6a55c72a794f16fd7e513
Closes-Bug: #1548614
2016-02-28 00:54:47 +00:00
Jeffrey Zhang
da0f0cf1e6 fix the custom profile raise exception issue
Closes-Bug: #1536272
Change-Id: I6c91e522923eade16ba23711e6014e6b4b8cd3e0
2016-02-15 15:25:15 +08:00
Paul Bourke
477fc18bd0 Fix image plugin functionality for oslo.config
The new oslo.config style parsing for kolla-build.conf stopped plugins
from working. This patch fixes it and adds a unit test.

Co-Authored-By: Jeffrey Zhang <zhang.lei.fly@gmail.com>
Closes-Bug: 1534556
Change-Id: I135797c733ae0cae040a009c364073769b19e4eb
2016-02-12 06:00:20 +00:00
Allen Gao
db53ad1ddf Move the install base type check to kolla-build
The install type is converted in kolla-build, so it will never
fail in dockerfile, move the check to kolla-build just above
the install type converting.

TrivialFix

Co-Authored-By: Jeffrey Zhang <jeffrey.zhang@99cloud.net>
Change-Id: I1500d3b47e909f94ea9f68c5245297733f63a70b
2016-02-04 16:48:12 +08:00
Paul Bourke
b82111d589 Pick up proxy buildargs from environment
We currently support any buildarg via --build-args. This patch picks up
the default supported proxy buildargs if set in the user's environment,
to allow for more transparent proxy support.

The --build-args will take precendence.

DocImpact
Implements blueprint: better-proxy-support

Change-Id: I084e5d1cc8f9993d65167878f9983ad58a68e86e
2016-01-18 11:53:25 +00:00
Jeffrey Zhang
7093d37f18 Expose the docker build_arg to build.py
With this implement, we can add variables at building stage.
For example, add HTTP_PROXY and NO_PROXY when needed like below.

    build.py --build-args \
        HTTP_PROXY:http://127.0.0.1:8080,NO_PROXY:127.0.0.1

More info about build_arg, pls check[0]

[0] https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables-build-arg

DocImpact

Implements: bp add-buildargs
Change-Id: I29ed7f536670fef59d697603dc562a69d90743c9
2016-01-18 11:42:00 +00:00
Jeffrey Zhang
d4490ea2bc add unittest code base
* add a base TestCase class
* load the tests from the project root rather than only tests dir
* add a test case for WorkerThread builder function

Change-Id: Icf878f9249b475a311123c8235c42b1212d02ca6
Partially-implements: bp add-ut-codebase
2016-01-18 11:42:00 +00:00