40 Commits

Author SHA1 Message Date
Carlos Camacho
44ef2a3ec1 Change template names to rocky
The new master branch should point now to rocky.

So, HOT templates should specify that they might contain features
for rocky release [1]

Also, this submission updates the yaml validation to use only latest
heat_version alias. There are cases in which we will need to set
the version for specific templates i.e. mixed versions, so there
is added a variable to assign specific templates to specific heat_version
aliases, avoiding the introductions of error by bulk replacing the
the old version in new releases.

[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#rocky
Change-Id: Ib17526d9cc453516d99d4659ee5fa51a5aa7fb4b
2018-05-09 08:28:42 +02:00
Emilien Macchi
77aa6763f6 Undercloud upgrades will use upgrade_tasks
Instead of using host_prep_tasks (which are part of deployment tasks),
we'll use the upgrade tasks that are now well known and tested in
previous releases, when the we containerized the overcloud.

Depends-On: Id25e6280b4b4f060d5e3f78a50ff83aaca9e6b1a
Change-Id: Ic199c7d431e155e2d37996acd0d7b924d14af2b7
2018-05-05 06:52:34 +00:00
Zuul
dd558e656d Merge "FFU: Use yum shell instead of ansible yum module" 2018-04-16 12:37:35 +00:00
Lukas Bezdicka
c2536e22f1 FFU: Use yum shell instead of ansible yum module
Ansible yum module installs all packages available in the repo
 if you use asterix. We instead will use yum -y update name*.

Change-Id: I8e71367ae91faa06313711c6a954c61af705fd8f
Resolves: rhbz#1549845
2018-04-13 14:45:43 +02:00
Emilien Macchi
d86025593b Handle undercloud upgrades via host_prep_tasks
Using host_prep_tasks interface to handle undercloud teardown before we
run the undercloud install.
The reason of not using upgrade_tasks is because the existing tasks were
created for the overcloud upgrade first and there are too much logic
right now so we can easily re-use the bits for the undercloud. In the
future, we'll probably use upgrade_tasks for both the undercloud and
overcloud but right now this is not possible and a simple way to move
forward was to implement these tasks that work fine for the undercloud
containerization case.

Workflow will be:
- Services will be stopped and disabled (except mariadb)
- Neutron DB will be renamed, then mariadb stopped & disabled
- Remove cron jobs
- All packages will be upgraded with yum update.

Change-Id: I36be7f398dcd91e332687c6222b3ccbb9cd74ad2
2018-04-12 18:14:28 -07:00
Lukas Bezdicka
5038364cb2 FFU: Fix Cinder services action order
We should try to disable pacemaker resource and shut down services
in step 1 and make sure we check running services only once.

Change-Id: I5676132be477695838c59a0d59c62e09e335a8f0
2018-03-14 14:20:09 +01:00
Zuul
41fffa9f8c Merge "Enable cinder-api health check" 2018-03-14 03:43:22 +00:00
Martin Mágr
d93ac6b491 Enable cinder-api health check
This patch enables health check execution for cinder-api docker container.

Change-Id: I4c51f8b7260eb04cd8a8c502ed1b80df4f025880
Depends-On: Ib82cb849540694106a869ec81694f1159967ee79
2018-02-26 14:33:49 +00:00
Lee Yarwood
d2073a13a2 ffu: Introduce Cinder fast-forward upgrade tasks
fast_forward_upgrade_tasks for Glance covering Ocata and Pike.
- Service status check
- Stop services when updating from Ocata to Pike
- Update cinder packages
- Db sync

Resolves: rhbz#1536010
Closes-Bug: #1744056
bp fast-forward-upgrades
Change-Id: I172c3a1868a8b7a94b282cbe5c2f6b323f7ca101
2018-02-20 13:22:04 -05:00
Lukas Bezdicka
0cb5c847f3 Always evaluate step first in conditional
If we use variables defined in later step in conditional before
checking which step are we on we will fail.

Resolves: rhbz#1535457
Closes-Bug: #1743764
Change-Id: Ic21f6eb5c4101f230fa894cd0829a11e2f0ef39b
2018-02-09 17:12:29 +01:00
Damien Ciabrini
6381879a1b Bind mount the database client settings in cinder_api_db_sync
cinder_api_db_sync does not use the database settings which are
generated by the puppet_config step. Consequently, it loses some
important client settings for accessing the DB, e.g. it breaks
when TLS everywhere is enabled.

Bind mount the tripleo.cnf file to expose the proper DB settings.

Change-Id: I17f3304d546eeb78803b4a3cc859255bfb3f71eb
Closes-Bug: #1746491
2018-01-31 11:14:28 +00:00
marios
dec003def8 Convert tags to when statements for Q major upgrade workflow
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.

This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)

The yaml-validate also now checks for duplicate 'when:' statements

Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
2018-01-08 13:57:47 +02:00
Zuul
1942568e87 Merge "Add validation task in docker services [cinder]" 2017-12-01 05:29:24 +00:00
Carlos Camacho
927495fe3d Change template names to queens
The new master branch should point now to queens instead of pike.

So, HOT templates should specify that they might contain features
for queens release [1]

[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#queens

Change-Id: I7654d1c59db0c4508a9d7045f452612d22493004
2017-11-23 10:15:32 +01:00
Zuul
6a72a9f1b4 Merge "Drop step_config as top level docker requirement" 2017-11-17 08:33:29 +00:00
Dan Prince
a307fe7ffc Drop step_config as top level docker requirement
Step config is only required within the puppet_configs section
of docker/services/*. This patch drops the top level 'step_config'
and updates the unit tests accordingly.

Change-Id: I7dc7cfae3ef1965ec95b1d9ef23e7f162418c034
2017-11-15 16:01:16 -05:00
Jiri Stransky
85ec193403 Write readme.txt into old log directories
This should help operators find the new log files. We do have them
documented, but not everybody reads every word in the docs :)

The readme creation has ignore_errors: true so that if the directory
isn't present at all (e.g. on deployed server environments, which
don't have openstack packages installed), we don't fail the deployment
when we're not able to create the readme.

Change-Id: I6b36db7b7ce8b3e4da566eb7828d0c3b8646a14f
Partial-Bug: #1730957
2017-11-14 10:35:11 +01:00
Zuul
c8f6b8ec97 Merge "Explicitly set healthcheck command." 2017-11-08 18:06:02 +00:00
Carlos Camacho
d4477a8ea3 Add validation task in docker services [cinder]
Docker services are missing the pre-upgrade validation task
in the upgrade_tasks section which verifies if the service
is running before stopping it.

Change-Id: I3d0b68eaf11b78f3422e026710d062f7e9455508
Partial-Bug: #1704389
2017-10-19 12:10:32 +02:00
marios
ce0ef2fa20 Remove package if service stopped and disabled
Adds a UpgradeRemoveUnusedPackages param to use
in the ansible when conditional for the removal

Adds package removal to step2 right after a service
is stopped and disabled on step2. Package updates
happen in step3 so ideally remove before that.

The package removal task has ignore_errors true
so dependencies or other issue removing packages will
not fail the upgrade workflow.

Also adds this to the upgrade environment files
for visibility and defaulting false

Change-Id: Ie4e4a2d41f7752c5a13507a7c15c6f68e203cfca
Related-Bug: 1701501
2017-10-05 11:50:58 +00:00
Ian Main
627971b2a1 Explicitly set healthcheck command.
We were setting them in the Dockerfile's previously.  However this
caused the healtcheck commands to always run regardless of which
process we were running in the container.  This caused 'unhealthy'
containers at times they were never intended to be checked.  This
change makes it so they are explicitly set.

Change-Id: I7bc12d236b3cc7a52d3e6aa706fd04675dad3a9a
2017-09-28 17:20:00 -04:00
Juan Badia Payno
5dbe1121e9 docker: add logging(source & groups)
The services that docker depends on, have logging_sources and logging_groups;
but those are not set on the docker outputs so they are not used when dockers
are deployed.

Added logging_source & logging_groups as docker optional parameters in
tools/yaml-validate.py

Closes-Bug: #1718110
Change-Id: I8795eaf4bd06051e9b94aa50450dee0d8761e526
2017-09-27 07:37:14 +00:00
Dan Prince
1128271b46 Add tags to baremetal cron removal tasks
In 59e29b17f4a9f5f65b6f8a7b8e82ef6426d8a51 we forgot to
add tags to the Ansible tasks to remove the baremetal
cron jobs at step 2.

Change-Id: I23fb134b88336ebc4eb1a97a69a2d73d4ef0edb2
Related-bug: #1708466
2017-09-06 11:27:47 +02:00
Jenkins
0bdf8da5ba Merge "Manually set healthchecks for _cron services" 2017-09-01 13:47:27 +00:00
Dan Prince
d1aaf0aadf Manually set healthchecks for _cron services
The docker _cron services show up as (unhealthy) due to
them sharing the containers for the OpenStack services.
As such we need to manually override the health checks
for these services. By setting them to /bin/true
the services should show up has healthy.

Change-Id: I46e12bcec226fbe2768c7fe8f0e7719df46401a9
Closes-bug: #1713183
2017-08-25 22:27:24 -04:00
Bogdan Dobrelya
287e84585c Persist containerized services httpd logs
Store the httpd logs under dedicated /var/log/containers/httpd/
paths.
Additionally, add release notes describing upgrade impact
for containerized services logs.

Closes-bug: #1700045

Change-Id: I8120c56f2315700862bd0f708b8baa8910275b09
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2017-08-25 14:36:53 +00:00
Dan Prince
59e29b17f4 Remove baremetal cron jobs on docker upgrade
Change-Id: I072a3f582cdb978187d14233ea1ba636d12a1293
Closes-bug: #1708466
2017-08-03 10:20:57 -04:00
Jenkins
7aff429265 Merge "Fix keystone, cinder, heat-api cron containers" 2017-08-03 00:44:41 +00:00
Jenkins
56d4563935 Merge "Generate MySQL client config if service requires database" 2017-08-01 20:00:18 +00:00
Dan Prince
975d862bc3 Fix keystone, cinder, heat-api cron containers
The cron containers need to run as root in order to create PID files
correctly.

Additionally, the keystone_cron container was misconfigured to
use /usr/bin/cron instead of the correct /usr/bin/crond.

Additionally we have an issue where the Kolla keystone container has
hard coded ARGS for the docker container which causes -DFOREGROUND
(an Apache specific argument) to get appended onto the kolla_start
command thus causing crond to fail to startup correctly. This
works around the issue by overriding the command and calling
kolla_set_configs manually. Once we fix this in Kolla we can
revisit this.

Change-Id: Ib8fb2bef9a3bb89131265051e9ea304525b58374
Related-bug: 1707785
2017-08-01 10:06:47 -04:00
Juan Antonio Osorio Robles
e3ee5965fd Add missing metadata_settings from docker services
These are needed for the TLS everywhere bits.

Change-Id: I81fcf453fc1aaa2545e0ed24013f0f13b240a102
2017-07-31 18:22:44 +03:00
Damien Ciabrini
0cb45d65c6 Generate MySQL client config if service requires database
Services that access database have to read an extra MySQL configuration file
/etc/my.cnf.d/tripleo.cnf which holds client-only settings, like client bind
address and SSL configuration. The configuration file is thus used by
containerized services, but also by non-containerized services that still
run on the host.

In order to generate that client configuration file appropriately both on the
host and for containers, 1) the MySQLClient service must be included by the
role; 2) every containerized service which uses the database must include the
mysql::client profile in the docker-puppet config generation step.

By including the mysql::client profile in each containerized service, we ensure
that any change in configuration file will be reflected in the service's
/var/lib/config-data/{service}, and that paunch will restart the service's
container automatically.

We now only rely on MySQLClient from puppet/services, to make it possible to
generate /etc/my.cnf.d/tripleo.cnf on the host, and to set the hiera keys that
drive the generation of that config file in containers via docker-puppet.

We include a new YAML validation step to ensure that any service which depends
on MySQL will initialize the mysql::client profile during the docker-puppet
step.

Change-Id: I0dab1dc9caef1e749f1c42cfefeba179caebc8d7
2017-07-27 13:41:13 -04:00
Dan Prince
cd857451b8 Add cinder cron container to run db purge
The cinder db purge cron job is created by puppet in
/var/spool/cron/cinder. This creates a cron container to run
that in an environment where it has access to cinder.conf
and the cinder-manage binaries.

Change-Id: I02ae32a6dcd8569e2e2390063d4d935d05545a78
Partial-bug: #1701254
2017-07-20 20:12:18 -04:00
Jenkins
2185b83560 Merge "Use a single configuration file for specifying docker containers." 2017-07-15 06:19:13 +00:00
Ian Main
e76d84f784 Use a single configuration file for specifying docker containers.
This removes the default container names from all the templates
and uses a single environment file to specify the full container
name and registry from which to pull.  Also does away with most
of DockerNamespace.

Change-Id: Ieaedac33f0a25a352ab432cdb00b5c888be4ba27
Depends-On: Ibc108871ebc2beb1baae437105b2da1d0123ba60
Co-Authored-By: Dan Prince <dprince@redhat.com>
Co-Authored-By: Steve Baker <sbaker@redhat.com>
2017-07-14 22:23:02 +00:00
Giulio Fidente
baf6eee501 Adds network/cidr mapping into a new service property
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.

Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).

Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
2017-07-14 13:44:04 +02:00
Martin André
cf18e865d1 Copy only generated puppet files into the container
This solves a problem with bind-mounts when the containers are holding
files descriptors open.

At the same time this makes the template more robust to puppet changes
since new config files will be available in the containers without
needing to update the templates.

Partial-Bug: #1698323
Change-Id: Ia4ad6d77387e3dc354cd131c2f9756939fb8f736
2017-07-10 11:13:25 +02:00
Martin André
a474ae82d5 Add heat parameter for all of config_volume images
This commit consistently defines a heat template parameter in the form
of DockerXXXConfigImage where XXX represents the name of the
config_volume that is used by docker-puppet.

The goal is to mitigate hard to debug errors where the templates would
set different defaults for the image docker-puppet.py uses to run, for
the same config_volume name.

This fixes a couple of inconsistencies on the way.

Change-Id: I212020a76622a03521385a6cae4ce73e51ce5b6b
Closes-Bug: #1699791
2017-06-28 10:48:53 +02:00
Jiri Stransky
248099db8c Fix race conditions between containers
In many occasions we had log directory initialization containers
without `detach: false`, which didn't guarantee that they'll finish
before the container depending on them will start using the log
directory.

This is now fixed by moving the initialization container one global
step earlier, so that we can keep the concurrency when creating the
log dirs. (Using `detach: false` makes paunch handle just one
container at a time, and as such it can have negative performance
impact.)

For services which have their container(s) starting in step_1,
initialization cannot be moved to an earlier step, so the solution
here was to just add `detach: false`.

As a minor related change, cinder DB sync container now mounts the log
directory from host to put cinder-manage.log into the expected
location.

Change-Id: I1340de4f68dd32c2412d9385cf3a8ca202b48556
2017-06-14 15:58:55 +02:00
Dan Prince
402b2283a8 Docker services for Cinder Api and Scheduler
Adds docker services for Cinder API and Scheduler.

Co-Authored-By: Gorka Eguileor <geguileo@redhat.com>
Co-Authored-By: Jon Bernard <jobernar@redhat.com>
Co-Authored-By: Martin André <m.andre@redhat.com>
Co-Authored-By: Alan Bishop <abishop@redhat.com>

Depends-On: Ic1585bae27c318bd6bafc287e905f2ed250cce0f

Change-Id: I5cff9587626a3b2a147e03146d5268242d1c9658
Partial-bug: #1668920
2017-06-12 10:04:59 -04:00