60 Commits

Author SHA1 Message Date
Emilien Macchi
81c1cae40d Rename /var/lib/docker-puppet to /var/lib/container-puppet
The /var/lib/docker-puppet is deprecated and can now be found under
/var/lib/container-puppet. We don't have Docker anymore so we try to avoid
confusion in the directories. The directory still exists but a readme
file points to the right directory.

Change-Id: Ie3d05d18e2471d25c0c4ddaba4feece840b34196
2019-03-01 14:59:37 -05:00
Michele Baldessari
7133394c35 Be able to know when we are running inside a minor update workflow
With this change we add an ansible variable called
'tripleo_minor_update' set to true only during the update_steps_playbook
which get run during a minor update.
Then inside common/deploy-steps-tasks when starting containers with
paunch we export this 'tripleo_minor_update' ansible variable and
push it inside the 'TRIPLEO_MINOR_UPDATE' environment variable.

Inside change Id1d671506d3ec827bc311b47d9363952e1239ce3 we will then
use the env variable and export it to the restart_bundles in order
to detect if we're inside a minor update workflow (as opposed to
a redeploy - aka stack update). The testing that has been done is
described in the above change.

Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>

Change-Id: Ib3562adbd83f7162c2aeb450329b7cc4ab200fc2
2019-02-20 15:49:09 +01:00
Cédric Jeanneret
3d07ad4326 New parameter: ContainerLogStdoutPath
This is used in order to point where podman must push its logs.
Two scripts are using it:
- docker-puppet.py
- paunch (near future - see https://review.openstack.org/#/c/635438/)

This will allow to get the stdout for all containers, even when they
are removed before we can actually run "podman logs container_name".

Related-Bug: #1814897
Change-Id: Idc220047d56ce0eb41ac43903877177c4f7b75c2
2019-02-18 09:42:05 +01:00
Zuul
d5298e2f79 Merge "run docker_puppet_tasks on any role" 2019-02-08 12:57:29 +00:00
Cédric Jeanneret
edfe180634 Inject log-driver for podman containers
Currently, docker daemon runtime has a default --log-driver set
to journald.
Podman lack of daemon prevent such a global application, meaning
we have to set that driver for each and every container when we
either create or run them.

Notes:
- podman only supports "json-file", and it's not even a json.
- docker json-file doesn't support "path" option, making this output
  unusable in the end: logs end in
  /var/lib/docker/containers/ID/ID-json.log

Related-Bug: #1814897
Change-Id: Ia613fc3812aa34376c3fe64c21abfed51cfc9cab
2019-02-07 08:05:21 +01:00
Steven Hardy
a0a09d29aa run docker_puppet_tasks on any role
Currently this assumes all tasks will run on the primary controller
but because of composable roles, that may not be the case.

An example is if you deploy keystone on any role other than the
role tagged primary e.g Controller by default, we don't create
any of the users/endpoints because the tasks aren't written to
the role unless keystone actually runs there.

Closes-Bug: #1792613
Change-Id: Ib6efd03584c95ed4ab997f614aa3178b01877b8c
2019-01-31 18:17:56 +00:00
Bogdan Dobrelya
c5d1b6fb63 Fix paunch logs verbosity control
Make ConfigDebug also controlling the paunch logs verbosity.

Depends-On: https://review.openstack.org/614166
Related-Bug: #1799182

Change-Id: I89fd73eaa2120f06ab245be148a60bb08f0cb512
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2019-01-16 15:06:32 +00:00
Zuul
52a70658ab Merge "Be explicit when passing vars into deploy steps" 2019-01-15 18:16:40 +00:00
Bogdan Dobrelya
35aae87301 Be explicit when passing vars into deploy steps
Implicit defaults hide issues with overring ansible variables as we
pass values in from deploy-steps.j2.

Make no implicit defaults for variables passed into deploy steps via
ansible vars. Only expect those take the values defined in the caller
deploy-steps.j2 playbook template. Add missing params and vars for
templates to propagate ansible values for external deploy/upgrade,
upgrade/update and post upgrade steps playbooks.

Make DockerPuppetDebug boolean to align with other booleans we pass
into deploy steps via ansible vars. Fix its processing in
docker-puppet.py, which is defaults for DockerPuppetDebug: ''
converted into 'false' in deploy steps tasks playbook, and then
that becomes always True in docker-puppet.py.

Related-Bug: #1799914

Change-Id: Ia630f08f553bd53656c76e5c8059f15d314a17c0
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2019-01-15 10:59:50 +01:00
Emilien Macchi
d8ee4b9e73 docker-puppet.py: only create docker-puppet.sh when it doesn't exist
In docker-puppet.py, we only create docker-puppet.sh script if it
doesn't exist yet. It's not useful to re-create it and it can be
dangerous to regenerate the script while docker-puppet.py is running,
since we bind mount the script to the containers.
It's possible that during a multi-process task, the script changes and
then the entrypoint fails to run correctly if the interpreter is not
present in the script.

This patch makes sure that we create the script only when needed, and
also that we remove it before running docker-puppet.py, which will be
useful when doing clean deployments or upgrades.

Context: https://github.com/containers/libpod/issues/1844
Change-Id: I0ac69adb47f59a9ca82764b5537532014a782913
2019-01-08 21:55:10 -05:00
Cédric Jeanneret
0576e26234 Ensure we get dedicated logging file for HAProxy
With the current configuration, HAProxy logs are in the host journal.
This isn't really friendly when you want to debug issues with this service.

This patches ensures HAProxy logs are in a dedicated file, using the syslog
facility set in its configuration.

Depends-On: I8fee040287940188f6bc6bc35bdbdaf6c234cbfd
Change-Id: Ia615ac07d0c559deb65e307bb6254127e989794d
2018-12-12 10:16:42 +01:00
Dan Prince
0cbbdac604 Add DockerPuppetMountHostPuppet parameter
This can be used to control whether puppet modules are consumed
from the baremetal host or from the container. Our default
is to consume these from the host so that deployment
archive tarballs can be used to extra puppet modules from
the host.

Since I61e35d8118c1de4c2976f496e8a6c9c529f3d91f we've had
puppet-tripleo in our containers however so using this
location would be possible as well.

Change-Id: I73026e66bcfafd1c582916141b5b1cf0ce0dc36c
2018-11-30 07:39:59 -05:00
Jose Luis Franco Arza
9527808075 Run docker-puppet.py in debug mode.
Allow running docker-puppet.py in
debug mode, depending on the value
of the ansible variable docker_puppet_debug.

This variable takes its value from DockerPuppetDebug,
which is set to true in the env file
environments/config-debug.yaml.

Change-Id: I7c88aa22dce3396c6a79843ac13db479ed987f9d
2018-11-15 11:24:10 +01:00
Quique Llorente
851bd0953a Use correct python version for puppet-docker.py
Now that we are running this at fedora28 with python3 we need to use
python3 to run python scripts at playbooks

Depends-On: I2c471724374da44eeddc4680b268bc362572ee27
Closes-Bug: #1802531
Change-Id: I42b18b228bfe361d19b580a853328c1a6c896257
2018-11-12 06:51:19 +00:00
James Slagle
16dff38eb4 Fix tasks in check mode
These tasks should have check_mode:no set so that they run in check
mode, as the variables they register are used in later tasks. Otherwise,
ansible in check mode fails with undefined variable errors.

Also, some tasks may fail due to not all requirements being available
since those requirements were not created by previous tasks that were
also ran in check mode.

This adds ignore_errors to these tasks, and sets the value to the
boolean ansible_check_mode which is provided by ansible and set based on
whether or not --check was passed to the ansible command line.

Change-Id: I84bc3c14ede37959a4078fd14ce4661b7bd23f84
2018-11-01 19:14:14 +00:00
Zuul
0797f537dc Merge "check mode: paunch configs" 2018-10-20 08:01:47 +00:00
Zuul
557e69c0dc Merge "check mode: docker_puppet_tasks" 2018-10-20 08:01:45 +00:00
Zuul
c262998fcf Merge "check mode: docker_puppet" 2018-10-16 23:12:05 +00:00
Cédric Jeanneret
245da47a9d Add SELinux configurations for a proper Standalone deploy
With this patch, we're able to deploy a "standalone" stack using
podman on a fully-enabled SELinux system.

Change-Id: I4bfa2e1d3fe6c968c4d4a2ee1c2d4fb00a1667a1
2018-10-10 11:57:21 +00:00
James Slagle
b76a36eca1 check mode: paunch configs
Adds initial check mode support for the paunch container startup
configuration and kolla config files. This cleans up the formatting of
the generated files so that the diff shown duing check mode with --diff
is useful.

We can't actually run paunch during check mode as it doesn't yet have
any support for a dry run mode.

Change-Id: I9add7b9fda50847c111e91735bd55a1ddf32f696
2018-10-03 07:47:20 -04:00
James Slagle
29f05e1e6f check mode: docker_puppet_tasks
Adds check mode support for docker_puppet_tasks.

Since it's not possible to reliably determine what these tasks do, we
can't actually run them to get an idea of what might be changed. We can
however show the diff of the json file to get an idea of what would be
run.

Change-Id: I19e8bc9eb93d8acc8ee7d737770f9cc7e63f7a27
2018-10-03 07:47:20 -04:00
James Slagle
a6f9821821 check mode: docker_puppet
Adds check mode support for docker_puppet. The updated json file is
written to /var/lib/docker-puppet/check-mode/docker-puppet.json
during check mode and then diffed with the existing version at
/var/lib/docker-puppet/docker-puppet.json.

When docker-puppet.py is run during check mode, the updated json file
under the check-mode directory is passed to the command. All generated
config files are then written under /var/lib/config-data/check-mode,
which is then recursively diffed with the existing config under just
/var/lib/config-data to report on all changed config files.

Change-Id: I5c831e9546f8b6edaf3b0fda6c9fbef86c825a4c
2018-10-03 07:47:16 -04:00
James Slagle
6d0f16d430 check mode: puppet host
Adds check mode support for puppet host tasks.

This works by writing the new puppet host manifest under
/var/lib/tripleo-config/check-mode, and diffing it against the existing
version of the manifest.

Puppet is also run with --noop, so that it only reports on what changes
would have been made.

It also uses the check mode hiera configuration at
/etc/puppet/check-mode/hiera.yaml if it exists so that the updated hiera
data is also accounted for when puppet runs with --noop.

Depends-On: Ibe0c2ab79c35f04ce51e7a1ade0e8ff72b430163
Change-Id: I112b63096c8dce05176b0939a7678bec02987294
2018-10-01 13:59:59 -04:00
James Slagle
1b0c827930 Convert with_dict tasks to use loop and be less chatty
with_dict is replaced by ansible's loop:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html#with-dict

This migrates tasks using with_dict over to use loop instead.

Additionally, when using loop (or with_dict), the entire loop item is
logged by default. This makes these tasks very verbose since we're
looping over large json/yaml files. Instead, use loop_control and label
ot only log the item key. The entire data structure already exists in
the config-download directory anyway, so there's no need to log the
whole thing to the console.

Change-Id: I1fc7431dfc662212b6ca64f4f738760f25b0c30b
2018-10-01 13:59:59 -04:00
Zuul
e516e8bfec Merge "Tag tasks in in common tasks" 2018-09-28 11:37:13 +00:00
Zuul
fd52b25c8b Merge "Remove "when failed" from debug task names" 2018-09-28 00:54:50 +00:00
Cédric Jeanneret
57154fd084 Dropped "recurse" for idempotency
We should not need that recurse anyway.

Change-Id: I504b52a2bb3c89e75ac3402f259c317889c054e6
Closes-Bug: #1794251
2018-09-25 13:20:12 +02:00
James Slagle
961fdc22ee Tag tasks in in common tasks
Adds the following tags to relevant tasks in deploy-steps-tasks.yaml
that are common to all roles:

- host_config
- container_config
- container_config_tasks
- container_config_scripts
- container_startup_configs

The tags are tool agnostic, so hopefully they won't have to be updated
over time. They allow users to run only specific parts of the common
tasks.

Change-Id: Ia7476da222218411caddae887f99c029b4bccf23
2018-09-24 09:21:53 -04:00
James Slagle
13aaf52a46 Remove "when failed" from debug task names
This commit removes the "when failed" from the task title to eliminate
confusion.

These tasks always run to show the debug output for the previous task,
regardless of whether the the previous task failed or not. They will
show the debug output as long as the previous task finished (success or
failure).

Change-Id: I4e52bdc18885f13793550e5032fb1316a13b248c
2018-09-24 09:17:15 -04:00
Cédric Jeanneret
7719263c27 Ensure some directories are created with the correct SELinux label
As Podman doesn't create host location for bind-mount, we have to ensure
directories actually exists.
SELinux labels are also important, since Podman has selinux enabled by default,
and there is currently no way to disable it like in Docker.

Change-Id: Ic1bede203e8199a296944273cb334027dab940fe
2018-09-21 22:11:11 +00:00
Emilien Macchi
e175e5ab2f Initial support for Podman in docker-puppet
Create a new parameter in TripleO: ContainerCli.
The default is set to 'docker' for backward compatibility but it allows
to also set to 'podman'.
When podman is selected, the right commands will be run so docker-puppet
can configure the containers when Podman is the selected container
library backend.

It removes the tripleo_logs:/var/log/tripleo/ mount that was used
by tripleo-ui but we shouldn't do that here. We'll create a bind mount
in tripleo-ui container later.

It run puppet with FACTER_hostname only if NET_HOST is disabled.

Change-Id: I240b15663b720d6bd994d5114d43d51fa26d76cc
Co-Authored-by: Martin André <m.andre@redhat.com>
2018-09-08 05:23:00 +00:00
Quique Llorente
2f44dbd938 Revert "Fix deploy health checks"
This reverts commit bd1d5d72caf25010e373f1ad2ed6ebc5aee96914.

Closes-Bug: #1784307
Change-Id: Ia2c12d7455564b6297c5f0934812b10fabbdc914
2018-07-30 13:20:11 +01:00
Oliver Walsh
bd1d5d72ca Fix deploy health checks
Allow up to 5 minutes for unhealthy and restarting containers to stabilise.

Change-Id: Icb0ef7648920e77fe368409f07612cdcba83e4cf
Related-Bug: 1782598
2018-07-24 13:30:33 +00:00
Bogdan Dobrelya
a4175d42f4 Revert "Check container health as part of the deploy"
There is a CI blocker LP 1782598 to deal with ASAP.
Then, we can fix this in the scope of
https://review.openstack.org/#/c/584119/

This reverts commit 915c1ebdd79fecb57a0719997a56c34685307431.

Change-Id: I8f03d8a588e58202c3628c72144a232729041c89
2018-07-20 12:15:13 +03:00
Alex Schultz
7fbaee67d0 Limit deploy health checks to paunch managed ones
If an operator has non-paunch managed containers (ceph/openshift), we
may not want to fail the deployment if those are unhealthy.

Change-Id: Ifd3e67a66b3224d0ed5f7ef12ba27b06f78c8556
2018-07-19 01:52:27 +00:00
Alex Schultz
915c1ebdd7 Check container health as part of the deploy
After starting the containers, we should make sure they are healthy
before continuing. If any containers are unhealthy we should fail
quickly and provide output showing which container is unhealthy.

Change-Id: I785ddb45779b6699fc839fdddb9c804dd1b1da5d
2018-07-19 01:51:59 +00:00
Raoul Scarazzini
c494a508f8 Remove unuseful become: true from deploy-steps
The ansible command generated in ansible-playbook-command.sh by default
have "--become" in it.
This commit removes "become: true" where is used to avoid confusion in
deploy steps. Today we explicitly set "become: false" in deploy-steps.j2
for certain actions, so there's no meaning of having also "become: true"
for the other ones.
We have a release note [1] that explains why the "become" was
introduces, but maybe we can revisit it.

[1] releasenotes/notes/use-become-true-in-deploy-steps-playbook-01decb18d895879f.yaml

Change-Id: Ic666b4ecaecf0591dd8bb0406f239649b20b9623
2018-06-13 16:28:48 +02:00
Sam Doran
1deab1217d Optimized Ansible tasks in deplay-steps-tasks.yaml
- do not use set_fact when a lookup can be done directly in the task
- use multi-line YAML for easier legibility
- ignore errors in file lookup plugin when file does not exist and set defaults

Change-Id: I832a2ec34f4ed4a87e30d0c88f4c60bcf2f4c151
2018-05-31 19:59:06 -04:00
Emilien Macchi
1bec01137e deploy-steps: switch to tripleo_role_name
"role_name" is internal to Ansible, we should not use it.
This patch uses the new variable set in the inventory to use a specific
TripleO var: tripleo_role_name which is the TripleO role name and not
the Ansible role names, both things are very different.

Depends-On: I57c4eac87e2f96dfe5490b111cd2508505715d56
Change-Id: Iecaf6f1b830e65be2f9e2e44431054fe46f9f565
Related-Bug: #1771171
2018-05-15 16:38:29 +00:00
Zuul
3fdb4c85a9 Merge "Add spacing for readability" 2018-04-10 03:54:22 +00:00
James Slagle
f044539b1c Don't log set_fact tasks for role_data
set_fact logs the fact value. In the case of reading the role_data_*
files, this is very verbose as the files can be large. Use no_log: True
to make these tasks less verbose. The content is saved in the
config-download output already, so no useful info is lost.

Change-Id: Ie6f75113194961628a0c9bdfbfbf5d88a18059eb
Closes-Bug: #1760996
2018-04-04 07:29:53 -04:00
James Slagle
0b23ff7ec9 Add spacing for readability
Add blank lines between the Ansible tasks and plays in the stack
outputs. This is an improvement in readability for the user.

Change-Id: I52ebd9081cacf213ac29f1d24e73db6ea5cfe33f
2018-04-03 11:56:15 -04:00
Dan Prince
cb16252474 Add EnablePuppet (defaults to true)
This wires in a heat parameter that can be used to disable the
baremetal (Puppet) deployment tasks. Useful for testing
some lightweight/containers only deployments.

Change-Id: I376418c618616b7755fafefa80fea8150cf16b99
2018-03-13 09:14:27 -04:00
Dan Prince
d87325990c Treat enable_debug in Ansible as a bool
Without the extra bool this when block gets evaluated as a string.
Given that it is always present this means enable_debug has been
enabled regardless of the end user setting.

Change-Id: I9f53f3bca4a6862966e558ea20fe001eabda7bcf
Closes-bug: #1754481
2018-03-08 16:51:08 -05:00
James Slagle
c334ad38d5 Consume RoleData config from config download files
In https://review.openstack.org/#/c/525260/, we moved the creation of
various RoleData driven config files to deploy-steps-tasks.yaml, and to
consume the values from various role_data_* variables that were written in
the inventory (see https://review.openstack.org/#/c/528354/).

However, we were already downloading and saving the RoleData to separate
files via config download. We should consume from those files instead of
the inventory. That has the advantage that one can quickly modify and
iterate on the local files, and have those changes applied. That is
harder to do when these values are in the inventory, and not possible to
do when using dynamic inventory.

Since the tasks will fail trying to read from the files when not using
config-download, conditional local_action tasks that use the stat module
first verify the existence of the files before attempting to read their
contents. If they don't exist, the values fall back to whatever has been
defined by the ansible variable.

Change-Id: Idfdce6f0a778b0a7f2fed17ff56d1a3e451868ab
Closes-Bug: #1749784
2018-03-07 13:57:08 -05:00
Steven Hardy
7f4811779f Add name to debug tasks
This makes it clearer that the previous task failed, which isn't
immediately evident from the ansible task output due to the failed_when
on those tasks.

Change-Id: I765208d5865f6e5a292e5b52c572e2e79540c663
Closes-Bug: #1748443
2018-02-12 10:13:25 +00:00
Zuul
71accf3415 Merge "Add {{step}} var to Task name" 2018-02-02 20:12:46 +00:00
James Slagle
ba0719c1b7 Add {{step}} var to Task name
Add the {{step}} var to a couple of task names from
deploy-steps-tasks.yaml where it was missing. Makes the output a bit
more consistent and user friendly.

Change-Id: I0a1b3f7f62543107b2f82ee57d75e65ecc7e02d4
2018-01-25 16:42:07 -05:00
Steven Hardy
a2a0ba9300 Move step 1 preparation to deploy-steps-tasks.yaml
This moves the writing of various files that are consumed by the
tasks in deploy-steps-tasks.yaml, hopefully this is clearer, and
it also means we can drive the creation of these files via ansible
directly using https://review.openstack.org/528354

Change-Id: I173d22ebcbc986cefdef47f81298abe10ce8591b
2018-01-18 11:18:01 +00:00
Steven Hardy
41988eab39 Default empty map for docker_config steps
In the event a step has no services defined, we must still write the
config, as this is needed if services are disabled on update such that
a step becomes empty - we must run paunch on every step or the cleanup
of the "old" services does not happen.

Closes-Bug: 1742915
Change-Id: Iee01002f56b5311560557f2bf6f053601b9d43d7
2018-01-16 09:22:39 +00:00