96 Commits

Author SHA1 Message Date
Dan Prince
64500bf2d1 Move nova-scheduler data into puppet_config
This is now required per the puppet_config interfaces for docker
services (per I208c1ef1550ab39ab0ee47ab282f9b1937379810)

Change-Id: Iab96919cb0a6b15942f3c19f8d28205261174edc
2017-03-08 08:59:11 +01:00
Martin André
46b5911a15 Enable Docker service for Compute role
A recent commit [1] change how docker is installed and configured on
the overcloud nodes, from a cloud-init script to a proper puppet
profile in puppet-tripleo but forgot to enable the docker service on
the compute nodes.

[1] Ia50169819cb959025866348b11337728f8ed5c9e

Change-Id: I202723d0e48f110e5b0dbfe3dcf6646da9f37948
2017-03-08 08:57:00 +01:00
Dan Prince
38a0dc7e7b Add puppet_config to docker neutron-l3 service
This patch makes the neutron-l3 docker service adhere
to the new puppet_config interface.

Change-Id: If5b73ec90637e878af55c8404d1eff8c18e857c3
2017-03-07 13:15:18 -05:00
Jenkins
a4c8067eec Merge "Enable composable upgrades for docker service templates" 2017-03-07 10:08:20 +00:00
Jenkins
d4e5dc1455 Merge "Overwrite nova placement with stub for docker" 2017-03-07 00:29:31 +00:00
Steven Hardy
fb748ba307 Enable composable upgrades for docker service templates
This aligns the docker based services with the new composable upgrades
architecture we landed for ocata, and does a first-pass adding upgrade_tasks
for the services (these may change, atm we only disable the service on
the host).

To run the upgrade workflow you basically do two steps:

openstack overcloud deploy --templates \
  -e environments/major-upgrade-composable-steps-docker.yaml

This will run the ansible upgrade steps we define via upgrade_tasks
then run the normal docker PostDeploySteps to bring up the containers.

For the puppet workflow there's then an operator driven step where
compute nodes (and potentially storage nodes) are upgrades in batches
and finally you do:

openstack overcloud deploy --templates \
  -e environments/major-upgrade-converge-docker.yaml

In the puppet case this re-applies puppet to unpin the nova RPC API
so I guess it'll restart the nova containers this affects but otherwise
will be a no-op (we also disable the ansible steps at this point.

Depends-On: I9057d47eea15c8ba92ca34717b6b5965d4425ab1
Change-Id: Ia50169819cb959025866348b11337728f8ed5c9e
2017-03-06 15:53:46 +00:00
Jenkins
47fcabee46 Merge "Put docker puppet config in puppet_config dict" 2017-03-06 10:32:52 +00:00
Pradeep Kilambi
9bbd43b3e7 Fix httpd dir create to not error if exists
In cases where /var/log/httpd already exists, this exits with error
code 1.

$ sudo docker logs keystone-init-log
mkdir: cannot create directory '/var/log/httpd': File exists

Change-Id: I62bf08d9fc9e02d5f3016bd14bb0a090b76ac837
2017-03-03 12:58:37 +00:00
Dan Prince
c68704b4b0 Overwrite nova placement with stub for docker
This updates kolla config to overwrite the stock
version with the puppet-nova generated mock.

Depends-On: Ie16a60c604ecf9f4012b0630f91e6ece2b6855db

Change-Id: I320f024adc88102ea24c0212702fe2dce826874f
Closes-bug: #440612
2017-03-02 20:36:27 +00:00
Jenkins
09794f09aa Merge "Containerize neutron-l3 agent" 2017-03-01 23:18:08 +00:00
Steve Baker
610850140d Put docker puppet config in puppet_config dict
This approach removes the need for the yaql zip to build the
docker-puppet data by building the data in a puppet_config dict.

This allows a future change to make docker-puppet.py only accept dict
data.

Currently the step_config is left where it is and referenced inside
puppet_config, but feedback is welcome whether this is necessary or
desirable.

Change-Id: I4a4d7a6fd2735cb841174af305dbb62e0b3d3e8c
2017-03-01 21:37:54 +00:00
John Trowbridge
c5931fdf1e Containerize neutron-l3 agent
This allows to run a containerized neutron on the overcloud.

Co-Authored-By: Martin André <m.andre@redhat.com>
Depends-On: Iaf6536b1c4d0b2b118af92295136378cdfeee9d1
Change-Id: I86a12248d4f28f4dbe7708be928bcd8a45968d01
2017-03-01 15:54:55 +00:00
John Trowbridge
b8695b8b2e Associate unmapped hosts with cell mappings
Otherwise the containerized nova running in the overcloud fails with
"Host 'overcloud-novacompute-0' is not mapped to any cell, Code: 400".

Co-Authored-By: Martin André <m.andre@redhat.com>
Change-Id: I9ff77f25bfd1f37167b0638a32fe5049951bc5b4
2017-03-01 16:37:46 +01:00
Steve Baker
93561bde0a Switch to dict format for docker_puppet_tasks
This change gives the option of docker-puppet.py data to be in a dict
as well as a list. This allows docker_puppet_tasks data to use the
same keys as the top level puppet config data.

If the yaql fu can be worked out to build the top level data,
docker-puppet.py can later drop the list format entirely.

Change-Id: I7e2294c6c898d2340421c93516296ccf120aa6d2
2017-03-01 03:14:49 +00:00
Jenkins
6a8850e10e Merge "Write out a json file containing container startup info and create tool to use it." 2017-02-28 10:58:17 +00:00
Ian Main
28b908abfc Add option to diff containers after config stage.
This allows you to show the changes made to a container during
configuration stage for fast development.

Change-Id: Id9c72cf2b07486f0a80bf3572a7ba349888d877f
2017-02-24 16:49:20 +00:00
Jenkins
1930d31d0e Merge "Containers: workaround for neutron DB sync" 2017-02-24 16:17:45 +00:00
Jiri Stransky
55c587d50b Containers: workaround for neutron DB sync
Neutron DB sync didn't have permission to read the config files, we now
run neutron-db-manage as root until we can find a more permanent
solution.

Change-Id: I502a8514adc523c7cac1da059be10480eef71cb9
Closes-Bug: #1667300
2017-02-24 10:48:53 +01:00
Dan Prince
f09f50f96a Add step to docker_puppet_tasks
This patch sets the step correctly for docker_puppet_tasks.
This is now required in order to match the 'step' in some
puppet manifests explicitly so that things like keystone
initialization run correctly.

Closes-bug: #1667454

Change-Id: If2bdd0b1051125674f116f895832b48723d82b3a
2017-02-23 14:45:47 -05:00
Jenkins
c510b10d44 Merge "Parallelize docker-puppet.py" 2017-02-23 15:43:01 +00:00
Jenkins
1e1a4d415e Merge "Add docker mysql service" 2017-02-23 06:20:41 +00:00
Flavio Percoco
301cb391ae Add docker mysql service
Co-Authored-By: Dan Prince <dprince@redhat.com>
Co-Authored-By: Martin André <m.andre@redhat.com>

Change-Id: If0ee671acbf6a9931622003a859089d61e2050b3
2017-02-23 03:25:06 +00:00
Jenkins
758eeff854 Merge "Containerized Swift Services." 2017-02-22 21:39:48 +00:00
Jenkins
1343079855 Merge "Containerized MongoDB" 2017-02-22 21:39:22 +00:00
Jenkins
7b2da488f5 Merge "Add docker Glance API service" 2017-02-22 21:22:55 +00:00
Jenkins
7af2a54baa Merge "Add docker keystone service" 2017-02-22 21:22:37 +00:00
Jenkins
10d5b61b92 Merge "Add docker rabbitmq service" 2017-02-22 20:27:58 +00:00
Jenkins
bfaa613ee4 Merge "Add docker mistral services" 2017-02-22 20:27:44 +00:00
Jenkins
7668e24d10 Merge "Add docker zaqar service" 2017-02-22 20:27:02 +00:00
Jenkins
686deffb39 Merge "Add docker heat services" 2017-02-22 20:25:42 +00:00
Jenkins
0d4e65ba6a Merge "Add docker nova services" 2017-02-22 20:25:34 +00:00
Jenkins
751ad87dc7 Merge "Add docker memcached service" 2017-02-22 20:22:36 +00:00
Ian Main
5a74923bc5 Parallelize docker-puppet.py
Use a pool of worker processes to run the puppet modules so they
can all be done in parallel.  Defaults to cpu count processes.

Change-Id: I083d302b8cf6538569e4d165221c21df152266bc
2017-02-22 14:56:42 -05:00
Jenkins
cf3ba3835a Merge "Add docker neutron services" 2017-02-22 19:37:22 +00:00
Ian Main
27b6188779 Write out a json file containing container startup info and create tool to use it.
This adds a bit to the post.yaml for docker to write out a json file
containing all the information on how we are start docker containers
(thanks Dan!).  I've then written a script that parses this that can
be used to execute docker run commands in various ways for debugging
purposes.

Change-Id: I36d66b42d1ac5030db8841820d4fc512a71d1285
Co-Authored-by: Dan Prince <dprince@redhat.com>
2017-02-22 18:29:51 +00:00
Ian Main
ae1ce1c1eb Containerized Swift Services.
Running swift in containers. Oh yeah.

Co-Authored-by: Dan Prince <dprince@redhat.com>

Change-Id: I2d96514fb7aa51dffe8fe293bc950e0e99df5e94
2017-02-22 18:29:19 +00:00
Dan Prince
f196adfe38 Add docker Glance API service
Co-Authored-By: Dan Prince <dprince@redhat.com>
Change-Id: I02cc769df5ec7afd608c502abc63fde8aa52be82
2017-02-22 18:27:36 +00:00
Flavio Percoco
3705549745 Containerized MongoDB
Change-Id: Ic3fd3bfd76d31ba515dbabdda7dfd06b9833a2ca
2017-02-22 18:26:50 +00:00
Dan Prince
03608215f7 Add docker memcached service
Change-Id: I055677a5ba62ee1c34a207fb83273f70558d21f1
2017-02-20 14:37:43 -05:00
Dan Prince
41fe57951b Add docker rabbitmq service
Change-Id: I3d30e55716a464d423358ff23bdd184d34da765a
2017-02-20 14:37:08 -05:00
Dan Prince
8eda7b10a2 Add docker zaqar service
Change-Id: I09379b4959c38ae4e7b4e667b97e87d35c8e7e83
2017-02-20 14:36:50 -05:00
Dan Prince
e75e5dde02 Add docker mistral services
This patch adds docker services for Mistral API, Engine, and Executor.

NOTE: the Mistral API container for TripleO must be built with
openstack-tripleo-common installed in order for TripleO specific
actions to get installed for the undercloud.

Change-Id: I0075840d14eb43c50d80b87acd986b80cfd38d4a
2017-02-20 14:36:21 -05:00
Dan Prince
0466def813 Add docker neutron services
This patch adds docker services for Neutron API(server), the
Neutron DHCP agent, and Neutron ML2 plugin configuration.

Co-Authored-By: Ian Main <imain@redhat.com>

Change-Id: If31e095cae533597cf0d73b4277425dfc3459564
2017-02-20 14:35:38 -05:00
Dan Prince
d010442153 Add docker nova services
This patch adds docker services for Nova for the Api, conductor,
scheduler, ironic, placement, and pass-thru configuration for metadata (it
simply enables metadata to be configured as part of the nova-api.

The nova-api DB initialization commands depend on a new heat-agent
feature (see patch below) to accommodate exit codes returned by
the new cells setup commands.

Change-Id: I39436783409ed752b08619b07b0a0c592bce0456
Depends-On: Ia6ca4b01982a0b33b26eca2a907d9d9f87c19922
2017-02-20 14:35:01 -05:00
Dan Prince
838d54b4ac Add docker heat services
This patch adds docker services for Heat API, API Cfn, and Engine.

Change-Id: I50caad9b9cf5482a8872cb0f2a67477b5e829ead
2017-02-20 14:34:20 -05:00
Dan Prince
2902640adc Add docker ironic services
This patch adds docker services for Ironic API and Ironic
Conductor.

Change-Id: Icea5ae2ecd7e749ce95ed8444c3a5ef3e035bb4e
2017-02-15 13:13:21 -05:00
Dan Prince
84626c82cc Add docker_puppet_tasks initialization on primary node
This patch adds a new (optional) section to the docker post.j2.yaml
that collects any 'docker_puppet_tasks' data from enabled
services and applies it on the primary role node (the
first node in the primary (first) role).

The use case for this is although we are generally only using
puppet for configuration there are several exceptions that we
desire to make use of today for parity with baremetal. This
includes things like database creation and keystone endpoint
initialization which we rely on configuration via hiera variables
controlled by the puppet services.

Change-Id: Ic14ef48f26de761b0d0eabd0e1c0eae52d90e68a
2017-02-15 13:09:59 -05:00
Dan Prince
500bec5449 Add docker keystone service
Depends-On: Icabdb30369c8ca15e77d169dc441bee8cfd3631f

Change-Id: Icec07f75f81953c4bf81ca21b4b02bc02e157562
Co-Authored-By: Martin André <m.andre@redhat.com>
Co-Authored-By: Ian Main <imain@redhat.com>
2017-02-15 13:05:56 -05:00
Dan Prince
ad2ea290be docker: new hybrid deployment architecture and configuration
This patch implements a new docker deployment architecture that
should us to install docker services in a stepwise manner alongside
of baremetal puppet services. This works by using Yaql to select
docker specific services (docker/services/*.yaml) vs the puppet
specific ones and then applying the selected Json to relevant Heat
software deployments for docker and baremetal puppet in a stepwise
fashion.

Additionally the new architecture
leverages new composable services interfaces from Newton to
allow configuration of per-service container configuration
sets (directories that are bind mounted into kolla containers) by
using the Kolla containers themselves. It does this by spinning up
a throw away "configuration only" version of the container being
configured itself, then running the puppet apply in that container and
copying the generated config files into /var/lib/config-data. This
avoids having to install all of the OpenStack dependency packages
in the heat-agent-container itself (our previous approach) and should
allow us to configure a much wider variety of container config files
that would otherwise be impossible with the previous shared approach.

The new approach (combined) should allow us to configure containers in
both the undercloud and overcloud and incrementally add CI coverage to
services as we containerize them.

Co-Authored-By: Martin André <m.andre@redhat.com>
Co-Authored-By: Ian Main <imain@redhat.com>
Co-Authored-By: Flavio Percoco <flavio@redhat.com>

Change-Id: Ibcff99f03e6751fbf3197adefd5d344178b71fc2
2017-02-15 12:56:44 -05:00
Jiri Stransky
246370e490 Containers: Add required EndpointMap parameter
This parameter is passed in by the parent overcloud.yaml template, so we
have to listen accept it in docker/post.j2.yaml, otherwise the
deployment fails.

Change-Id: Ia3fdcfa01d52006a6e9fd0bb02c7379411f3d900
Closes-Bug: #1664569
2017-02-14 14:44:30 +01:00