1113 Commits

Author SHA1 Message Date
Zuul
5a219d53bf Merge "Cinder containers: volume and env customizations" 2018-01-15 21:21:22 +00:00
Zuul
b993f5aba7 Merge "Add support for OVN Metadata Agent" 2018-01-13 10:47:15 +00:00
Zuul
1af7729939 Merge "Convert tags to when statements for Q major upgrade workflow" 2018-01-13 09:39:38 +00:00
Zuul
5334c679f0 Merge "Enable docker-puppet.py for a single config_volume" 2018-01-13 06:40:48 +00:00
Dan Prince
7bd89420f2 Cinder containers: volume and env customizations
This patch will allow custom volume and env variables for the
cinder-volume container.

This is likely going to be needed by some Cinder backends who may not
have in-tree TripleO integration yet and need these types of
customizations.

Change-Id: I825c5373c7c4ab6896579eae705bc034f67fb68f
2018-01-12 16:50:56 -05:00
Daniel Alvarez
85e006d19d Add support for OVN Metadata Agent
This patch adds support for networking-ovn-metadata-agent.
It will deploy the agent on compute nodes and disable Nova
force_config_drive.

The following two patches have been squashed into this one:
* https://review.openstack.org/#/c/525164/
* https://review.openstack.org/#/c/522813/
The reason behind the squash is that we had interdepenencies
and this patch alone wouldn't be testing the code properly
without the two other ones since scenario007 job in baremetal
has been removed for this cycle.

UpgradeImpact

Depends-On: I678652294cb8f964c34b742a0bc0ea360d736fb9
Depends-On: If3dffde5e0db8f7607a9708d36d54d1600fe5da8
Depends-On: I38f775479d178f5b252619635b67f876bc8c5ed5
Depends-On: Ifdd42437333730a3b3e6f36cbab6df0a2971a5a1
Depends-On: I940cec6d670df39ac6e2a3559a028acbeee99331

Change-Id: Idc2bb4e31a64502ac6fcdac771d823509dc328e7
Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
2018-01-12 09:40:06 +00:00
Martin André
5e8bec8d53 Remove unused env var during mysql bootstrap
In TripleO we exported the KOLLA_KUBERNETES to skip the cluster
readiness check and workaround a limitation of the mariadb boostrap
script in Kolla that expected MariaDB 10.0 coming from the MariaDB
repository and didn't work with the MariaDB 10.1 from RDO.

Luckily this was fixed in Kolla with
Ia2acb09e877a586243fc1acb49d8d140cf27d7b5 and we can now remove this
tech debt from t-h-t.

Change-Id: Iba62e436a16ddb3cfc87fc4ec03b599e55841681
Related-Bug: #1740060
2018-01-11 10:40:15 +01:00
Zuul
077bbd2525 Merge "Use docker_config_scripts for puppet apply" 2018-01-11 06:20:21 +00:00
Zuul
d03199743f Merge "Set tftp to only listen to the provisioning network" 2018-01-10 07:15:19 +00:00
Alex Schultz
6f834f60e6 Use docker_config_scripts for puppet apply
There are some configuration applies that we need to do during the
deployment. These currently live as manually constructed bash runs which
are missing the --detailed-exitcode handling to know when we have
failures.  In order to reduce the duplicated code and simplify this
exeuction, this change creates a docker_config_scripts with
docker_puppet_run.sh in containers-common that can be reused by any of
the docker services. This allows use to properly handle
--detailed-exitcodes while also reducing the amount of duplicated code
bits that we have within THT.

Additionally this change adds a new shared value for ContainersCommon to
pull the required volumes for the docker_puppet_apply.sh script into a
single place. Unfortunately the existing volumes from ContainersCommon
includes a mount for /etc/puppet to /etc/puppet which causes problems
because we need to be able to write out a hiera value.  The /etc/puppet
mount is needed for the bootstrap_host_exec function which is consumed
by various docker_config tasks but the mount conflicts with the puppet
apply logic being used.

Depends-On: I24e5e344b7f657ce5d42a7c7c45be7b5ed5e6445
Change-Id: Icf4a64ed76635e39bbb34c3a088c55e1f14fddca
Related-Bug: #1741345
Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
2018-01-09 17:17:13 -07:00
Martin André
8eb351d588 Fix path for iscsi config file
We changed the bind mount to be /etc/iscsi in
I838427ccae06cfe1be72939c4bcc2978f7dc36a8, we need to copy the files to
/etc/iscsi so that they do not end up at '/' in the container.

Change-Id: Id5c1f16d08ffd36a35a6669d64460a7b2240d401
Closes-Bug: #1741850
2018-01-09 10:56:31 +01:00
Zuul
1b341f76b1 Merge "Allow for optinal volumes and env variables" 2018-01-08 12:42:51 +00:00
marios
dec003def8 Convert tags to when statements for Q major upgrade workflow
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.

This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)

The yaml-validate also now checks for duplicate 'when:' statements

Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
2018-01-08 13:57:47 +02:00
Zuul
587cd86c54 Merge "Parameterize ceph-ansible environment variables" 2018-01-07 18:00:54 +00:00
Zuul
f9d9919e5d Merge "Remove _member_ role from the keystone accepted roles" 2018-01-06 06:38:34 +00:00
Zuul
3de239d41b Merge "puppet apply: add --summarize" 2018-01-06 04:13:18 +00:00
Michael Henkel
b83a62fe00 Allow for optinal volumes and env variables
This patch will allow to attach optional volumes and env variables to the
neutron-api, heat-api and nova-compute containers.

Change-Id: I95cd5017fdbbec257d274b805be4509ec32f9019
Closes-Bug: 1741464
2018-01-05 13:24:24 +01:00
Emilien Macchi
eb324768d0 puppet apply: add --summarize
... so we can know how long take resources configuration in Puppet
catalogs, and more easily debug why we have timeouts.

Change-Id: If3fae8837140caae91120e46b4880146ffe22afc
2018-01-04 09:37:46 -08:00
Derek Higgins
7d69b51dbf Set tftp to only listen to the provisioning network
It doesn't need to be listening to all IP's.

Change-Id: Ib51fe69b32533f5d8814cc0529cd79cf93fbac63
2018-01-03 15:50:25 +00:00
Bogdan Dobrelya
82f128f15b Fix puppet config volume for iscsid in containers
Bind mount the /etc/iscsi host path for iscsi container puppet config.
Use the real host path /etc/iscsi for containers dependsing on it.

Closes-bug: #1735425

Change-Id: I838427ccae06cfe1be72939c4bcc2978f7dc36a8
Depends-on: I7e9f0641164691682516ac3e72e2145c7d112409
Co-authored-by: Alan Bishop <abishop@redhat.com>
Co-authored-by: Martin André <m.andre@redhat.com>
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2018-01-02 17:14:17 +00:00
Keith Schincke
45e96e5fa5 Remove _member_ role from the keystone accepted roles
As per the development mailing list: Keystone removed _member_ role management

Change-Id: I4f553431d1c38ca8d2c18a213d07f957c388d914
2017-12-27 13:37:40 -05:00
Zuul
843f6dc4ca Merge "Add validation task in docker services [Heat]" 2017-12-23 07:20:36 +00:00
Zuul
28cbfa8d33 Merge "Improve debugging possibilities for docker-puppet.py time handling" 2017-12-23 04:25:17 +00:00
Zuul
17c227f5e2 Merge "Add proper debug switch on init_bundles" 2017-12-23 01:24:10 +00:00
Zuul
ed38a7dafd Merge "Wait for rabbitmq_ready tag" 2017-12-21 22:38:17 +00:00
Dan Prince
f5754bfe53 swift_rsync: don't bind mount /run
This resolves an issue where the pid file exists from a previous
run of the container.

Change-Id: Id051172407f0e879d3edf18c8b2ec13734794ed2
Closes-bug: #1724559
2017-12-20 16:03:32 -05:00
John Fulton
ba2169d0a3 Parameterize ceph-ansible environment variables
Add CephAnsibleEnvironmentVariables which allows a user to
override any Ansible environment variable.

Depends-On: I5d69af146ca6ca8b3d5f78445cd1b47828daa955
Change-Id: Ic731c8f0c988c485c5b3448182a568b8514cab0a
Closes-Bug: 1738276
2017-12-20 17:48:05 +00:00
Michele Baldessari
bab6ec2532 Wait for rabbitmq_ready tag
We need to wait for rabbitmq_ready exec so that rabbit is fully
up. This can only happen if we add the tag for it.
Also we need to make sure that launching the epmd process cannot
happen. The reason for this is the following:
When the puppet-rabbitmq module gets invoked (a simple facter run
will be sufficient) inside the rabbitmq_init_bundle container it spawns
an epmd process.
Now if we wait for the Exec[rabbitmq-ready], it means that this epmd
process is staying around until rabbit is up, but then will disappear
suddenly when the rabbitmq_init_bundle container exits, which will
subsequently confuse the rabbitmq cluster and make it fail.

Partial-Bug: #1739026

Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
Co-Authored-By: John Eckersberg <jeckersb@redhat.com>

Change-Id: Ie74a13a6c8181948900ea0de8ee9717f76f3ce79
2017-12-19 15:36:12 +01:00
Zuul
6c2bd0d9a6 Merge "Add ceph-rbdmirror ansible container service" 2017-12-18 17:13:00 +00:00
Zuul
541b39ea8a Merge "Passes NodeDataLookup to ceph-ansible workflow" 2017-12-18 16:30:35 +00:00
Michele Baldessari
4d7e03be85 Add proper debug switch on init_bundles
When deploying with -e environments/config-debug.yaml, which sets
ConfigDebug to true, it is expected that puppet is run with --debug
--verbose. This has happened for most of the puppet uses (via
LP#1722752), but we missed enabling it for the init_bundle under
docker/services.

While we're at it we also add '--color=false' to the puppet apply
command of the init_bundle containers as that is what we use in the
other puppet apply runs.

Closes-Bug: #1738764

Change-Id: If529b83a7342b3ad17d705517978539d1c6b949e
2017-12-18 15:27:36 +01:00
Zuul
9f7c5e70db Merge "Expose logs from nova_libvirt container" 2017-12-15 08:08:11 +00:00
Keith Schincke
3a94c99bde Add ceph-rbdmirror ansible container service
Change-Id: I84faa8176d7967068d715af58f1377cec397b5e6
Depends-on: I2288d965da98b637aa91fd49b961f6e524610f60
2017-12-14 08:24:07 -05:00
Zuul
c15fc7f59b Merge "Remove Cinder UID from CephX keyrings' ACLs" 2017-12-14 13:03:44 +00:00
Zuul
2ecc12a37a Merge "Search for containers within stopped containers." 2017-12-14 10:22:54 +00:00
Zuul
6ce8b7f981 Merge "Set barbican to be configured in step 3" 2017-12-14 03:22:22 +00:00
Zuul
2fa7509bad Merge "Add missing keystone_domain_config" 2017-12-13 15:47:29 +00:00
Sven Anderson
22b5fd7928 Expose logs from nova_libvirt container
By default logs for libvirt are disabled. However, for debugging
purposes they might be enabled but are only available within the
container. This change bind mounts the log directory to the host.

Change-Id: I4a69e39355a332872ab40663d85a0bc15b98dcf4
2017-12-13 15:42:34 +01:00
Jiri Stransky
51a3b05f4e Improve debugging possibilities for docker-puppet.py time handling
We faced issue where on some environments docker-puppet.py picks up
/etc/hosts as modified, even though that shouldn't be the case (LP bug
1709689). On the last occasion we found the cause to be desynced time
on the host machine of a virtual setup, and subsequent NTP sync of
overcloud nodes (causing a 5 hour skip back in time) racing with
docker-puppet.py.

Still, more info to debug these kinds of issues would be nice to
have. Printing the timestamp of origin_of_time, saving it to a
persistent directory and making sure /etc/localtime is mounted into
docker-puppet containers should make finding the root cause on such
occasions easier.

Change-Id: I2ea197673b470379ead295058b6952cce3a69606
Closes-Bug: #1737954
2017-12-13 15:13:38 +01:00
Giulio Fidente
1971e7b049 Passes NodeDataLookup to ceph-ansible workflow
Per-node customizations were only dumped as hieradata, so the
ceph-ansible workflow could not consume them.
This change passes the structure to the mistral workflow so that it
can consume the data and populate the inventory accordingly.

Change-Id: Ie7a9f10f0c821b8c642494a4d3933b2901f39d40
Depends-On: Ia23825aea938f6f9bcf536e35cad562a1b96c93b
Closes-Bug: #1736707
2017-12-13 14:38:02 +01:00
Yurii Prokulevych
09dcd7e26c Search for containers within stopped containers.
During minor update pcs cluster is stopped during step 1.
Then we search for pcs managed containers at step 2.
But since pcs cluster is stopped, 'docker ps' won't report stopped
containers.
This change adds '--all' option to show all the containers.

Change-Id: If38a4f7e25d4d1f4679d9684ad2c0db8475d679b
Closes-Bug: #1737548
2017-12-13 11:34:26 +01:00
Zuul
5d8c117bff Merge "Add parameters for Barbican worker image" 2017-12-13 06:58:29 +00:00
Zuul
314063b703 Merge "Add modulepath option when applying puppet inside docker." 2017-12-13 00:56:04 +00:00
Alex Schultz
40530c0e8c Add missing keystone_domain_config
When configuring the keystone LDAP integration we need to write out
domain configuration items using the keystone_domain_config provider.
Since this tag was missed in the docker conversion, the configuration
was not actually available in the docker container.

Change-Id: I7abdfdd55e38da80768c907863fa06429debf9cd
Closes-Bug: #1737799
2017-12-12 18:06:56 +00:00
Ade Lee
a0c7f5580f Set barbican to be configured in step 3
Barbican is set to be configured in step 3 after keystone.
This allows other services (like swift) to use barbican to store
and retrieve configuration secrets.

Change-Id: Ie486473d001b10a23374b55369431b4c2bb85419
2017-12-11 15:35:36 -05:00
Ade Lee
f464e3d99f Add parameters for Barbican worker image
Change-Id: Idb1e776b6fa24d6be09b02300d4a57440bd9e05c
2017-12-11 15:26:08 -05:00
Zuul
e612bd769d Merge "Add parameters for Barbican keystone listener" 2017-12-07 22:58:08 +00:00
James Slagle
c4e6a70864 Enable docker-puppet.py for a single config_volume
If docker-puppet.py fails on any config_volume, it can be difficult to
reproduce the failure given all the other entries in docker-puppet.json.
Often to reproduce a single failure, one has to modify the json file,
and remove all other entries, save the result to a new file, then pass
that new file as $CONFIG.

This commit adds the ability to specify $CONFIG_VOLUME, which will cause
docker-puppet.py to only run the configuration for the specified entry
in docker-puppet.json whose config_volume value matches the user
specified value.

Change-Id: I2889647a27a8b891696a6a3e7f78b59a015c2c79
Closes-Bug: #1737043
2017-12-07 17:25:35 -05:00
Sofer Athlan-Guyot
4a708af34a Add modulepath option when applying puppet inside docker.
When new module are added, we may miss the symlink in
/etc/puppet/modules.  And for consistency as we mount the
/usr/share/openstack-puppet/modules directory it’s better to add it
to the modulepath.

Change-Id: I963aede41403ebbe3b9afb55a725b304a30a0cbb
Closes-Bug: #1736980
2017-12-07 20:09:13 +01:00
Zuul
d7c03dfafc Merge "Make CephPools override properties of other pools" 2017-12-07 15:08:38 +00:00