59 Commits

Author SHA1 Message Date
Saravanan KR
ec3e347092 Add volumes list of docker-puppet container when merging config volumes
When config volumes of same types are merged, the volumes to be mounted
for the continer is not merged. In order to run neutron container with
ovs agent, '/run/openvswitch' directory mount is required to execute
ovs-vsctl commands. This patch merges the volumes list of the same
config volumes instead of taking only the first one.

Closes-Bug: #1766759
Change-Id: I70972ad2fc339c9316befd74dda8555982f23316
2018-04-25 07:34:35 +05:30
Sergii Golovatiuk
bf99d30f56 Replace LOG.warn with LOG.warning
logging.warn is deprecated in Python 3 [1].

[1] https://docs.python.org/3/library/logging.html#logging.warning

Change-Id: I61218125ebf85b7951163360279645d7ad4d176c
2018-04-11 16:11:21 +00:00
Dan Prince
a28892df4f docker-puppet.py: don't pull if image exists
This updates the pull function here so that it matches
how paunch also handle's images, only pulling them if
the existing image doesn't already exist on the host.

Change-Id: I90ea41ccdfdb0b9206a63901554d002a5ec0fd3a
2018-03-08 12:26:31 -05:00
Zuul
5334c679f0 Merge "Enable docker-puppet.py for a single config_volume" 2018-01-13 06:40:48 +00:00
Emilien Macchi
eb324768d0 puppet apply: add --summarize
... so we can know how long take resources configuration in Puppet
catalogs, and more easily debug why we have timeouts.

Change-Id: If3fae8837140caae91120e46b4880146ffe22afc
2018-01-04 09:37:46 -08:00
Jiri Stransky
51a3b05f4e Improve debugging possibilities for docker-puppet.py time handling
We faced issue where on some environments docker-puppet.py picks up
/etc/hosts as modified, even though that shouldn't be the case (LP bug
1709689). On the last occasion we found the cause to be desynced time
on the host machine of a virtual setup, and subsequent NTP sync of
overcloud nodes (causing a 5 hour skip back in time) racing with
docker-puppet.py.

Still, more info to debug these kinds of issues would be nice to
have. Printing the timestamp of origin_of_time, saving it to a
persistent directory and making sure /etc/localtime is mounted into
docker-puppet containers should make finding the root cause on such
occasions easier.

Change-Id: I2ea197673b470379ead295058b6952cce3a69606
Closes-Bug: #1737954
2017-12-13 15:13:38 +01:00
James Slagle
c4e6a70864 Enable docker-puppet.py for a single config_volume
If docker-puppet.py fails on any config_volume, it can be difficult to
reproduce the failure given all the other entries in docker-puppet.json.
Often to reproduce a single failure, one has to modify the json file,
and remove all other entries, save the result to a new file, then pass
that new file as $CONFIG.

This commit adds the ability to specify $CONFIG_VOLUME, which will cause
docker-puppet.py to only run the configuration for the specified entry
in docker-puppet.json whose config_volume value matches the user
specified value.

Change-Id: I2889647a27a8b891696a6a3e7f78b59a015c2c79
Closes-Bug: #1737043
2017-12-07 17:25:35 -05:00
Sofer Athlan-Guyot
4a708af34a Add modulepath option when applying puppet inside docker.
When new module are added, we may miss the symlink in
/etc/puppet/modules.  And for consistency as we mount the
/usr/share/openstack-puppet/modules directory it’s better to add it
to the modulepath.

Change-Id: I963aede41403ebbe3b9afb55a725b304a30a0cbb
Closes-Bug: #1736980
2017-12-07 20:09:13 +01:00
Bogdan Dobrelya
1fc9285125 Allow containerized undercloud deploy with SELinux
When SELinux is enforcing, use the docker volume mount flag
:z for the docker-puppet tool's bind-mounted volumes in RW mode.
Note, if a volume mount with a Z, then the label will be specific
to the container, and not be able to be shared between containers.

Volumes from /etc/pki mounted RO do not require the context changes.
For those RO volumes that do require it, use :ro,z.

For deploy-steps, make sure ansible file resources in /var/lib/
are enforced the same SELinux context attributes what docker's :z
provides.

Partial-bug: #1682179
Related-bug: #1723003

Change-Id: Idc0caa49573bd88e8410d3d4217fd39e9aabf8f2
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2017-11-06 15:04:18 +01:00
Michele Baldessari
11e599d116 Add --detailed-exitcodes when running puppet via ansible
puppet run on never fails, even when it should, since we moved
to the ansible way of applying it. The reason is the current following code:

    - name: Run puppet host configuration for step {{step}}
      command: >-
        puppet apply
        --modulepath=/etc/puppet/modules:/opt/stack/puppet-modules:/usr/share/openstack-puppet/modules
        --logdest syslog --logdest console --color=false
        /var/lib/tripleo-config/puppet_step_config.pp

The above is missing the --detailed-exitcodes switch and so puppet will never
really error out on us and the deployment will keep on running all the
steps even though a previous puppet manifest might have failed. This
cause extra hard-to-debug failures.

Initially the issue was observed on the puppet host runs, but this
parameter is missing also from docker-puppet.py, so let's add it there
as well as it makes sense to return proper error codes whenever we call
puppet.

Besides this being a good idea in general, we actually *have* to do it
because puppet does not fail correctly without this option due to the
following puppet bug:
https://tickets.puppetlabs.com/browse/PUP-2754

Depends-On: I607927c2ee5c29b605e18e9294b0f91d37337680
Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>

Change-Id: Ie9df4f520645404560a9635fb66e3af42b966f54
Closes-Bug: #1723163
2017-10-26 20:03:11 +00:00
Jenkins
466da79636 Merge "Take all mounted config_volumes into account" 2017-10-13 10:10:01 +00:00
Jenkins
3e7ea2202f Merge "Also match config volumes for /var/lib/config-data/puppet-generated/" 2017-10-12 13:22:47 +00:00
Martin André
8aef24de81 Take all mounted config_volumes into account
We need to account for all the mounted config volumes when generating
the TRIPLEO_CONFIG_HASH in order for paunch to know to restart the
container when any one of the config_volume gets updated.

Change-Id: I473a71f49bd446694da48bb5b7b0a49126df7845
Closes-Bug: #1721306
2017-10-09 18:04:31 +01:00
Jenkins
8c1ae8b73e Merge "Fix logging for docker-puppet" 2017-10-03 22:23:58 +00:00
Steven Hardy
3a932b0569 Also match config volumes for /var/lib/config-data/puppet-generated/
Some services only mount this directory, not /var/lib/config-data/$service
so handle this case in the docker-puppet code that maps the mounted
volumes to the services when adding the config hash to the container
environment.

Change-Id: I3bdb7609f322458584ac9597ffbfefb057b84646
Closes-Bug: #1720208
2017-09-29 11:25:22 +01:00
Ian Main
6ba04cce6b Fix logging for docker-puppet
Logging in docker puppet was changed.  This just makes it a bit more verbose so
you get a sense of what's going on.

Change-Id: Ibda35b67f608a95eb39cb84f0457751d9a61d216
2017-09-27 16:05:48 -04:00
Jenkins
211d7f32dc Merge "Make our json output prettier." 2017-09-25 20:51:36 +00:00
Ian Main
9a7f62ff43 Remove healthcheck override in docker-puppet.
Now that the healthchecks are set explicitly, we can remove this
workaround from docker-puppet.

Change-Id: I37e8ade86263d8c5b86366c5441279bfc16debd6
2017-09-21 20:25:41 -04:00
Ian Main
f1632feb61 Make our json output prettier.
Add an indent=2 so these files are human readable.

Change-Id: Ie75c3a85e9c9dc890c4e138e9453bf055cd265a7
2017-09-14 15:19:29 -04:00
Jenkins
b5c18ded6a Merge "Add a docker pull retry to docker-puppet.py" 2017-09-10 04:06:37 +00:00
Dan Prince
d3b3361a76 Add a docker pull retry to docker-puppet.py
Co-Authored-By: Ian Main <imain@redhat.com>

Change-Id: Iad6d38690340f4a064a4527c58ed439d91fa5188
Closes-bug: #1715136
2017-09-07 16:49:14 -04:00
Attila Fazekas
c9c3d98b26 docker-puppet.py duplicated import
The sys module was imported twice.
Trivial whitespace changes also included into this change.

Change-Id: I1ac6ddade5c6ea2f63feff9273db19f644aec3d4
2017-09-05 14:13:37 +02:00
Jenkins
5b968f58ae Merge "Set mode for ansible written files" 2017-09-05 10:59:29 +00:00
Steven Hardy
94c7752cfa Set mode for ansible written files
Use a more restrictive mode for these files, as some may contain sensitive data
which shouldn't be world readable

Closes-Bug: #1714986
Change-Id: Ib1e79b1d4e25d6e329938402b1ca776bdab81bdd
2017-09-04 16:38:26 +01:00
Jenkins
9395804698 Merge "Stop hardcoding host's config volume path" 2017-09-04 12:13:21 +00:00
Dan Prince
20e1f0e8c9 Set docker-puppet --health-cmd = /bin/true
Change-Id: Idf627a348cad8d5287c82cb393367210f1c760cf
Closes-bug: #1713185
2017-08-25 22:35:47 -04:00
Martin André
0c62b6cd8d Stop hardcoding host's config volume path
Get the path from the CONFIG_VOLUME_PREFIX environment variable.

This is useful for debugging and generate configuration files to
a different directory.

Change-Id: Ib85e3898804312ebb6677a5fa189fbfc357ce27c
2017-08-16 12:41:41 +02:00
Bogdan Dobrelya
6073155d62 Also log docker-puppet.py puppet output to console
Running puppet apply with --logdest syslog results in all the output
being redirected to syslog. You get no error messages. In the case where this fails, the subsequent debug task shows nothing useful
as there was no stdout/stderr.

Also pass --logdest console to docker-puppet's puppet apply so that
we get the output for the debug task.

Related-Bug: #1707030

Change-Id: I67df5eee9916237420ca646a16e188f26c828c0e
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2017-07-28 11:25:42 +02:00
Oliver Walsh
4a7f3398f1 Add support for nova live/cold-migration with containers
Updates hieradata for changes in https://review.openstack.org/471950.
Creates a new service - NovaMigrationTarget. On baremetal this just configures
live/cold-migration. On docker is includes a container running a second sshd
services on an alternative port.
Configures /var/lib/nova/.ssh/config and mounts in nova-compute and libvirtd
containers.

Change-Id: Ic4b810ff71085b73ccd08c66a3739f94e6c0c427
Implements: blueprint tripleo-cold-migration
Depends-On: I6c04cebd1cf066c79c5b4335011733d32ac208dc
Depends-On: I063a84a8e6da64ae3b09125cfa42e48df69adc12
2017-07-23 02:26:55 +01:00
Jenkins
58e04d321b Merge "Improve logs from ansible, puppet, docker-puppet.py" 2017-07-18 02:45:57 +00:00
Bogdan Dobrelya
c011a34f5e Improve logs from ansible, puppet, docker-puppet.py
* Debug ansible 'puppet apply' stderr joined stdout, split
  by lines.
* Do 'puppet apply' w/o colors, logdest syslog, and given a wanted
  modulepath instead of the module puppet, that can't support those
  options.
* Bind-mount syslog socket for docker-puppet.py to pass puppet logs
  to host OS syslog.
* Fix logging handlers for multiprocess workers in docker-puppet.py.

Related-bug: #1698172
Closes-bug: #1700086

Change-Id: I84112a836e968aa5c3596a6544e0392980529963
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2017-07-14 10:55:27 +02:00
Oliver Walsh
ea4188cb4a Add support for running crontabs in containers
This change enables the puppet cron resource in docker-puppet.py and adds user
crontabs to the paths copied from the config containers.

Only the nova crontab is configured for now. Other services will require
similar changes to run their crontabs.

Partial-Bug: 1701254

Change-Id: I2d1d0f0d77908a132472cf4bc475f8bd526af504
Depends-On: Ie16fb4539481a3c192cff8220a97daa4c70467fc
2017-07-12 11:01:07 +01:00
Martin André
cf18e865d1 Copy only generated puppet files into the container
This solves a problem with bind-mounts when the containers are holding
files descriptors open.

At the same time this makes the template more robust to puppet changes
since new config files will be available in the containers without
needing to update the templates.

Partial-Bug: #1698323
Change-Id: Ia4ad6d77387e3dc354cd131c2f9756939fb8f736
2017-07-10 11:13:25 +02:00
Tim Rozet
24e855dae3 Adds docker OpenDaylight
Depends-On: I020550ede0ef981582392cf6c48dd5cb5823a074
Depends-On: I610b07a3c2bcf1c3288f76112a08b81c50e06913
Depends-On: I3d378044b3da5309b60967a12df7800520a254dc
Depends-On: I9c32b41ef865a09587f3ebfe8b8a896031fbd285
Depends-On: Ib31bf29bc69f5c58e98b99c3e598b19c99efc77f

Change-Id: I36c7390ddb4192e55ee56006fd6e9c5f8704445c
Signed-off-by: Tim Rozet <trozet@redhat.com>
2017-07-03 23:47:19 -04:00
Steven Hardy
f739fd0ccf Force mtime for tar used in container config md5sums
The checksum is changing each run because the mtime is different, so force
a specific date such that we only compare the directory contents.

Change-Id: I5ed2b50176f902d7af12b96e650b67b736d59a4a
2017-06-29 17:05:31 +01:00
Dan Prince
d2e0dcdb20 Default docker-puppet.py logging to INFO
If you want debug logging you can set the new DockerPuppetDebug
heat parameter to 'True'.

Change-Id: Iae7bb67379351ea15d61c331867d7005f07ba98e
Closes-bug: 1700570
2017-06-28 10:26:32 -04:00
Martin André
74595a7321 Make a copy of files touched by puppet in container
This should help determine what exactly needs to be bind mounted in the
container and should also help limit the size of collected logs in CI,
as collecting the entire /etc directory from each container can grow
pretty quickly in size and is not that useful.

Related-Bug: #1698172
Change-Id: Ie2bded39cdb82a72f0c28f1c552403cd11b5af45
2017-06-16 16:09:17 +02:00
Steve Baker
f600d459f0 Replace NO_ARCHIVE block with single call to rsync
Also attempts to move the workaround for bug #1696283 to before the
puppet apply call.

Closes-Bug: #1696622
Change-Id: I3a195466a5039e7641e843c11e5436440bfc5a01
2017-06-14 02:22:35 +00:00
Steven Hardy
e77de0d5ff Write md5sum for service config directories
The configuration generated by docker-puppet may change on update,
so checksum the combined files from the config-data directories,
to enable detecting those that have changed and restarting the
appropriate containers - we need to merge this checksum into
the environment passed to the containters, as this will cause
paunch to correctly restart containers when the configuration
generated changes, even if the rest of the json definition
provided by heat does not.

Change-Id: I40d9080cf3ad708ef4ed91e46d2b2ae1138bb9c3
2017-06-08 16:06:20 +01:00
Oliver Walsh
e7b7d4a0e3 Ensure /etc/ssh/ssh_known_hosts exist in docker config-data.
Works around the issue encountered in 1696283.

Change-Id: I1947d9d1e3cabc5dfe25ee1af994d684425bdbf7
Resolves-Bug: #1696283
2017-06-07 12:16:01 +01:00
Steve Baker
7962dd4d44 Don't delete failed docker-puppet containers
This helps a bit with debugging issues, and the container will be
deleted on the next run when the same volume is configured.

Change-Id: I4f2f219bd7e40abafd0eb31c1275fdd8ed4db4da
2017-05-17 03:46:25 +00:00
Steve Baker
c6ea4bc85f Make docker-puppet.sh a static file.
Variables are now passed in with --env in the docker run call.

This will allow docker-puppet.sh to be baked into the image instead of
having it as a custom entrypoint.

Change-Id: Icbaefe033becc6b2226535f28ee202917bdc1074
2017-05-10 09:21:38 +12:00
Bogdan Dobrelya
746c4d264d Improve logging for docker-puppet.py
Log prepared docker command
Use logger stdout instead of print command
Log stderr as debug as well

Change-Id: I3d48fbf4fa3381d325e3be3788b041e06d4bb294
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2017-05-02 07:15:18 +00:00
Jenkins
8391468d16 Merge "Do not log errors on non-existing container" 2017-04-13 14:00:15 +00:00
Juan Antonio Osorio Robles
39f63c5efa docker/all: Bind-mount OpenSSL CA bundle
The containers also need to trust the CA's that the overcloud node
trusts, else we'll get SSL verification failures.

bp tls-via-certmonger-containers

Change-Id: I7d3412a6273777712db2c90522e365c413567c49
2017-04-12 09:45:13 +00:00
Martin André
2eddc12be7 Do not log errors on non-existing container
This is cluttering up the logs with useless error messages, making it
more difficult than necessary to debug the CI job.

Change-Id: Icbdc4c74d99fea39b8722955dab56e5f538849aa
2017-04-11 09:29:52 +02:00
Jenkins
cd6128d0a5 Merge "docker-puppet.py fail if any worker fails" 2017-04-06 10:50:37 +00:00
Emilien Macchi
91053af09d Allow to configure policy.json for OpenStack projects
For both containers and classic deployments, allow to configure
policy.json for all OpenStack APIs with new parameters (hash,
empty by default).

Example of new parameter: NovaApiPolicies.
See environments/nova-api-policy.yaml for how the feature can be used.

Note: use it with extreme caution.

Partial-implement: blueprint modify-policy-json
Change-Id: I1144f339da3836c3e8c8ae4e5567afc4d1a83e95
2017-03-28 22:21:28 +00:00
Juan Antonio Osorio Robles
7f5c616a94 docker-puppet: skip empty volume entries
This allows to optionally add volumes, where we could use a heat
conditional to either put the volume path we want or put an empty string
which should be safely skipped.

Change-Id: I68f91ffdd8ceb14735adad1322fcf124c47b160c
2017-03-22 10:11:21 +02:00
Jenkins
0162749f71 Merge "Pass the DOCKER_* env vars when running docker" 2017-03-13 20:13:41 +00:00