47 Commits

Author SHA1 Message Date
Paul Bourke
b41247c656 Add header blocks to all Dockerfiles
Change needed to add header blocks to all Dockerfiles, similar to the
base.

Use case is to easily run something before packages are installed, e.g.
to COPY a local rpm in that can be added to the package list.

Change-Id: I1bbfdf0b762da0a392aa8bf47781315b45377bee
Closes-Bug: 1618969
2016-09-13 16:53:31 +01:00
Jeffrey Zhang
7f8b40538f
Use the standard start method for kolla-toolbox container
TrivialFix

Change-Id: Icb092e47713e0b89859747d6ed4c02b3950d7ced
Signed-off-by: Jeffrey Zhang <zhang.lei.fly@gmail.com>
2016-09-02 00:06:47 +08:00
Jenkins
e37314aaab Merge "Use dumb-init to manage the pid 1 process" 2016-08-31 16:30:13 +00:00
Jenkins
6c4d06ed8f Merge "Make the kolla_keystone_service can update fields" 2016-08-30 03:18:14 +00:00
Jenkins
1fd2d434b1 Merge "Fix bandit gate jobs" 2016-08-28 15:51:00 +00:00
Paul Bourke
fc30d583f9 Fix bandit gate jobs
* Inspected each error and fixed / added nosec where appropriate.
* build-swift-ring.py which was throwing sec errors is no longer used so
  removed it.
* Removed the dev/ directory from being checked.

Closes-Bug: #1617713
Change-Id: I25664cabca4137e5c9f499c1af3f5ce78b86fb56
2016-08-28 08:52:44 +00:00
Stephan Michaud
29c542186b Add mariadb client binaries to kolla-toolbox
The kolla-toolbox container contains ansible which has the mysql_db
module which helps on various mysql tasks such as db creation. The
mysql_db module requires certain mysql binaries in order to accomplish
tasks such as restoring a database from a dump.

This change adds those client libraries which weren't previously
included in the container.

Change-Id: I6516838381bf9327c8901fc4c32ebd5151fb053f
Signed-off-by: Stephan Michaud <michauds90@gmail.com>
Closes-Bug: #1616155
2016-08-23 16:54:18 -04:00
Jenkins
e1e28f7505 Merge "Remove Kazoo dependency and Kolla Zookeeper helper" 2016-08-23 10:57:30 +00:00
Jeffrey Zhang
1a7af3e10b Pin Ansible version to 2.1.1.0 in kolla-toolbox
Use a fixed and released Ansible version to keep stability.

Change-Id: I885816d85249b15be91b5a101718a05841a20c18
Closes-Bug: #1615350
2016-08-21 16:53:39 +08:00
Jeffrey Zhang
6710bbeb7c Use dumb-init to manage the pid 1 process
Normally, when you launch a Docker container, the process you're
executing becomes PID 1, giving it the quirks and responsibilities that
come with being the init system for the container.

There are two common issues this presents:

* In most cases, signals won't be handled properly.
* Orphaned zombie processes aren't properly reaped.

the dumb-init acting like a simple init system. It launches a single
process and then proxies all received signals to a session rooted at
that child process.

Closes-Bug: #1614509
Change-Id: I9d3d04648e151ddc7c6732b92ffd3b6c9fe467ec
2016-08-20 17:27:17 +00:00
Szymon Konefal
e0a9347ebc Remove Kazoo dependency and Kolla Zookeeper helper
Related-Bug: #1577194

Change-Id: Idad2a78d3b4251a321c1a05b435bff4625d5fe56
2016-08-19 17:07:19 +02:00
Shaun Smekel
a9d08726f5 Handle empty package list for install_packages
Currently if the install_packages macro is run with an empty
package list, it will add a yum or apt-get command with no
packages listed.

This bug fix aims to omit this line when no packages have
been given, or, the operator wants to use the "_override" /
"_remove" functionality to disable all packages being
installed in a Dockerfile.

Co-Authored-By: Paul Bourke <paul.bourke@oracle.com>
Change-Id: Ifaaaebfccc3adb0f2f68a35ac08e59378bc87fdb
Closes-bug: 1612446
2016-08-19 10:49:03 +00:00
Kevin Fox
53cb9f7950 Add crudini
Add the crudini tool that allows easy manipulation and data extraction
from ini config files.

This is intended to be used by kolla-kubernetes.

TrivialFix

Change-Id: Icbdd14ad2a68f7c176ee65f1d132c552e8331652
2016-08-17 08:36:17 -07:00
Kevin Fox
b234976a80 Add jq
Add the jq tool that allows easy manipulation and data extraction
from json documents.

This is intended to be used by kolla-kubernetes.

Change-Id: I478be7ec403e3444a412e1bcd47fd52f3a4e8095
2016-08-16 09:23:49 -07:00
Jeffrey Zhang
cae30a913b Make the kolla_keystone_service can update fields
Make the kolla_keystone_service can update the following fields

* service.name
* service.description
* endpoint.url

TrivialFix

Change-Id: If239a518b0217cfc105388ae5e298a8904ac2af5
2016-08-07 18:05:57 +08:00
Revanth Alampally
464542b4a5 Customizations for kolla-toolbox
This patchset contains customization of Dockerfile
of kolla-toolbox containers.

Change-Id: Idd141935012874e1184c1d88cd0e0fdc37e1b6c9
2016-08-05 17:11:22 -07:00
Patrick Powell
4f3efd7d4b Clean up Glance image after sanity check
The sanity check creates an image that needs to be
deleted after in order for the sanity check to
pass so that we complete deployment with a clean
enviroment.
Closes-bug: #1606580

Change-Id: Ib347f69b69229e39925f0c1ff11069b15afcf192
2016-07-26 10:12:43 -04:00
Jenkins
5d0459b147 Merge "Improve glance sanity check" 2016-07-16 08:57:45 +00:00
Patrick Powell
e7251f1640 Improve glance sanity check
Checks to see if a blank glance image was loaded correctly
partially-implements: blueprint sanity-check-container

Change-Id: I29549518e726b9530897ef31b120dd3759d1b279
2016-07-14 10:49:46 -04:00
Stephen Hindle
cadf8afdcf Read GPT label using sgdisk rather than udev
The Ceph osd bootstrap/startup logic depends upon reading/writing
partition names.  Some older versions of udev have trouble reading these
(/dev/disk/by-label is missing).

To work around this, we shell out to sgdisk and scrape the partition
name data directly.

Co-authored-by: Paul Bourke <paul.bourke@oracle.com>
Closes-Bug: 1585185
Change-Id: I362b3f8e91de79687fc84e256996fbcaf303b6af
2016-07-07 10:26:34 +01:00
Christian Berendt
76f97b4067 Create ansible home directory in kolla-toolbox container
This solves the following issue when running an ansible command inside
the kolla-toolbox container:

[Errno 13] Permission denied: '/home/ansible/.ansible'

Change-Id: I654d27c49b7fcc50e018b61146084660894352aa
Closes-bug: #1592268
2016-06-14 07:47:22 +02:00
Jenkins
9fbf4ef485 Merge "Bump the ansible version to >2 in the kolla-toolbox" 2016-06-09 11:19:29 +00:00
Jeffrey Zhang
4d1fa3cb90 Bump the ansible version to >2 in the kolla-toolbox
Kolla do not depend on certain version of Ansible. So
install the ansible using pip

Change-Id: Id1bf103a1f5c2a1cde653d53110f28c903ae8768
Partial-Implements: blueprint ansible2
2016-06-07 14:58:53 +08:00
Benedikt Trefzer
5df0903cfa Make container kolla-toolbox build with debian
- handle different path for os_client_config

Change-Id: I1134a932845ee22e8318c14ec02f0478eee11b9a
Partially-implements: blueprint build-debian
2016-06-04 03:44:23 +00:00
Jeffrey Zhang
476da0f43a Log the traceback in the kolla libraries
The traceback is logged also and will be helpful when the
library failed. We can get which line is failed from the log.

TrivialFix

Change-Id: Ia417232007e2f34c83d2e7cad6ed08dee49ce836
2016-05-31 01:40:55 +00:00
Steven Dake
025d57f820 Remove dependencies on kazoo and friends
Since kolla-mesos has been retired, there is no need in the code
base to keep the zookeeper implementation.  As a result, just remove
it.  If we were to keep it we need virtualenvs for that part of the
code base which installs the python dependencies related to it
for from-binary installs.

This just simplifies the implementation tremendously and culls dead
code.

Change-Id: Ieda226e652d67f5b5667112f4f2556f3171366d3
Closes-Bug: #1577194
2016-05-02 06:12:50 -07:00
Serguei Bezverkhi
9c8f3ffee0 Fixes kolla_toolbox failure due to missing pytz module
kolla_toolbox contantly fails to build due to a missing module pytz,
add it before the dependent module installation.

Change-Id: Ia497f4eef830259e8e27307d6b3100dc7660b99f
Closes-Bug: #1569633
2016-04-13 06:30:42 -04:00
Jenkins
65041fbcc2 Merge "Fix gate by adding MariaDB-shared dependnecy" 2016-03-30 15:00:24 +00:00
Jenkins
2068bc009a Merge "Revert "Fix consistently failing CentOS gate"" 2016-03-30 14:59:46 +00:00
Steven Dake
5f381dfbdd Fix gate by adding MariaDB-shared dependnecy
With the reorganization of the MariaDB repository, the C libraries
for mariadb were not included in the toolbox container.  This results
in a non-functional CentOS environment which is resolved by this
simple patch.

Change-Id: Ib4a2f7e55b6efc36bc3ff2d93447cb97b7b109f5
Closes-Bug: #1561347
2016-03-29 14:08:24 -04:00
Sam Yaple
3f707d78df Revert "Fix consistently failing CentOS gate"
This reverts commit 17e0de6aa6c32c5adebeffc90c96f15285877cf5.

This change should have never merged. There is something else going on
here and the bug doesnt fully invetigate the issue. Using a mix of
system and pip packages is not acceptable and can cause problems.
In this case the packaged mysql has a KNOWN issue that we fixed in
liberty by switching to the pip installed package.

Change-Id: Ie10116010156c13742a0a5d37c440c713e4698e5
2016-03-28 20:54:55 -04:00
Jeffrey Zhang
d2ad5f290a Remove meaningless char in regexp of find_disks.py file
the $ char is useless and not needed.

TrivialFix

Change-Id: I18df48ca86badadf6ec18757c17ed32222ead30a
2016-03-25 22:25:41 +08:00
Steven Dake
17e0de6aa6 Fix consistently failing CentOS gate
This fixes a gate regression and probably regression in general
with building images.  I'm not sure where the regression came from,
but I suspect a new verson of MySQL-python from pip is to blame.

Closes-Bug: #1561347
Change-Id: Ib502d0090a3fc3b3ea5d27a91ff59df1f53c01a3
2016-03-24 01:51:39 -04:00
SamYaple
5250a00781 Allow external ceph journals and fix bootstrap
This allows us to specify external journals for osds which can greatly
improve performance when the external journals are on the solid-state
drives.

The new lookup and startup methods fix the previous races we had
preventing osds from being created properly.

This retains the same functionality as before and is completely
compatible with the previous method and labels, however this does set
new labels for all new bootstrap OSDs. This was due to a limitation
in the length of the name of a GPT partition.

Closes-Bug: #1558853
DocImpact
Partially-Implements: blueprint ceph-improvements
Change-Id: I61fd10cb35c67dabc53bd82270f26909ef51fc38
2016-03-19 22:36:30 +00:00
Steven Dake
a577634fd3 Reorder operations of mkdir and useradd in toolbox
The home directory for Ansible already exists and does not
need to be created.  Reorder useradd and mkdir at suggestion
of Jeff Peeler to avoid warning.

The warning looks like this:

INFO:__main__:kolla-toolbox:Step 6 : RUN mkdir -p /etc/ansible /usr/share/ansible /home/ansible     && echo 'localhost ansible_connection=local' > /etc/ansible/hosts     && useradd --user-group ansible --groups kolla     && sed -i 's|  "identity_api_version": "2.0",|  "identity_api_version": "3",|' /usr/lib/python2.7/site-packages/os_client_config/defaults.json
INFO:__main__:kolla-toolbox: ---> Running in ccded9c01fca
INFO:__main__:kolla-toolbox:useradd: warning: the home directory already exists.
INFO:__main__:kolla-toolbox:Not copying any file from skel directory into it.
INFO:__main__:kolla-toolbox:

TrivialFix

Change-Id: I031ade6597f8071f5be1768a7e70b1008dd94e52
2016-03-17 13:30:28 -07:00
Steven Dake
0d273e077d mariadb-devel is deprecated
mariadb 10 uses the packaging convention MariaDB-* to not
conflict with mariadb 5.  As such, MariaDB-devel was indicating
a deprecation warning which this patch removes.

TrivialFix

Change-Id: I4b51fcedce3ef9e44c4a6830cd38ca8040a4d3f6
2016-03-09 00:44:50 -05:00
SamYaple
6f23746177 Bump ansible version to head of devel
bcoca has asked we use head of devel to fix the loader issue

TrivialFix

Change-Id: Ie6f4b6bbdffed9a85c840a4f425d866fc5402f4b
2016-03-04 16:06:36 +00:00
SamYaple
57124620ab Fix Keystone v3 and Horizon
After our switch to keystone-manage bootstrap Horizon is not happy
due to v3 not being setup correctly. This patch fixes that

This also includes removal of unused variables (transforms them into
endpoint url variables)

TrivialFix
Change-Id: I1e04db8c24049f80e974c063f03068a2ab32a563
2016-03-03 15:44:53 +00:00
SamYaple
a2fe37fe89 Bump to latest ansible and deps
Ansible is pinned to single commit in devel branch to pull in the
latest shade module we need (domain control). It will be available in
ansible 2.1

TrivialFix
Change-Id: I4c21fa1d2cec30d4aeb80e050ef4a62332f9ed45
2016-03-03 15:19:00 +00:00
Eugene Bagdasaryan
790675be4f Sanity check for swift
This runs first sanity check for swift. Once
swift is deployed it checks list()

Change-Id: I613bf9f2893d66814863893ec5acde5aa252548d
Partially-Implements: blueprint sanity-check-container
2016-02-29 18:23:13 +03:00
jvora
c31729cdaf Sanity check for cinder
This runs first sanity check for cinder. Once
cinder is deployed it checks volumes.list()

Change-Id: I1b4cc57f21cf0fa52a391229c2c2b3fa995d32a8
Partially-Implements: blueprint sanity-check-container
2016-02-27 16:22:34 -08:00
Martin André
ded2967ea6 Remove unneeded deps
Percona has fixed their packaging issue and now
MySQL-python installs cleanly without the need of MariaDB-shared
package.

TrivialFix

Change-Id: I42672c269185352ab4999683bd0ad7c5f23de3bb
2016-02-23 12:15:14 +09:00
Jeffrey Zhang
937c771ec8 Fix building centos kolla-toolbox
Percona recently updated their package which is now conflicting with
mariadb-libs.

Closes-Bug: #1548589

Change-Id: I9db7ffb8b94b623784cffa3e63804c47375097d1
2016-02-23 02:54:00 +00:00
Éric Lemoine
0251ac73d5 Make Ansible write its logs into "kolla_logs"
The Ansible logs are currently not collected by Heka.  This can be
done later, with an Ansible-specific decoder for Heka.

Partially implements: blueprint heka
Change-Id: I8d3ba4edb527f61c0a8234024b4be953c6e6c565
2016-02-19 21:49:21 +00:00
Allen Gao
3853a8bc89 Fix the incorrectly removed dependent packages
Change-Id: I3d963cd58ef791d30a90f3a74cac2477dac3edfd
Closes-Bug: #1546449
2016-02-17 17:57:40 +08:00
Steven Dake
5dd79bbbef Make kolla-toolbox container build on CentOS bin
Ansible installs pytz from pip which is newer then then pytz in
RDO repositories.  As part of this process, it removes pytz
which corrupts the python installation in some way.  This causes
further pip usage to produce a backtrace causing the kolla-toolbox
container not to build.

Mixing pip and RPM is not good.

Change-Id: I413468641bad5476cfcba61cb0e2189c31f3d682
Closes-Bug: #1545215
2016-01-29 18:51:30 -07:00
SamYaple
80b7266ed1 Rename kolla_ansible to kolla_toolbox
This change is needed for clarity. We have a kolla-ansible script.
We have a kolla-mesos repo. We plan to have a kolla-ansible repo.
Already we have had far too much confusion about whether we are
talking about the container or the project. Naming this kolla-toolbox
eliminates all of that confusion and its probably a bit more accurate
of a name too.

Closes-Bug: #1541053
Change-Id: I8fd1f49d5a22b36ede5b10f46b9fe02ddda9007e
2016-02-02 18:12:15 +00:00