7624 Commits

Author SHA1 Message Date
Zuul
a151ed15e8 Merge "Add a way to override base path when file driver is used" 2018-06-12 05:54:48 +00:00
Zuul
a2c8a130db Merge "undercloud: set OS_CACERT when TLS is used" 2018-06-11 16:29:17 +00:00
Zuul
15c5e566ca Merge "rerun *_init_bundles all the time" 2018-06-11 13:36:05 +00:00
Zuul
f9ad91703c Merge "Introduce restart_bundle containers to detect config changes and restart pacemaker resources" 2018-06-11 13:36:04 +00:00
Zuul
e2bbd55a45 Merge "Add docker service to OpenShift CI roles" 2018-06-11 08:22:00 +00:00
Zuul
39939708b7 Merge "Fix update/upgrade or config change for ODL" 2018-06-09 17:20:47 +00:00
Michele Baldessari
a0dfc6c0c6 rerun *_init_bundles all the time
In the same spirit as change I1f07272499b419079466cf9f395fb04a082099bd
we want to rerun all pacemaker _init_bundles all the time. For a few main
reasons:
1) We will eventually support scaling-up roles that contain
   pacemaker-managed services and we need to rerun _init_bundles so that
   pacemaker properties are created for the newly added nodes.
2) When you replace a controller the pacemaker properties will be
   recreated for the newly added node.
3) We need to create appropriate iptables rules whenever we add a
   service to an existing deployment.

We do this by adding the DeployIdentifier to the environment so that
paunch will retrigger a run at every redeploy.

Partial-Bug: #1775196
Change-Id: Ifd48d74507609fc7f4abc269b61b2868bfbc9272
2018-06-09 10:22:15 +02:00
Zuul
7bad4f3bf9 Merge "Fix fail_if_empty invocation with pipes" 2018-06-09 02:51:34 +00:00
Tim Rozet
dc9c7315ae Fix update/upgrade or config change for ODL
OpenDaylight creates multiple files the first time it boots, which we do
not mount to the host. After the first boot, it creates a cache which we
do mount to the host. This means that on a config change or
update/upgrade of ODL the cache will not be removed, but the files will
be. This causes ODL to fail to start.

The solution is to stop the container in update/upgrade and then remove
the cache before the update happens. This will trigger the new ODL to
rebuild the cache with the new ODL version.  For config change, we also
need to remove the cache in the host_prep_tasks so that we do not end up
in a similar state.

Closes-Bug: 1775919

Change-Id: Ia457b90b765617822e9adbf07485c9ea1fe179e5
Signed-off-by: Tim Rozet <trozet@redhat.com>
2018-06-08 15:10:30 -04:00
Michele Baldessari
a6389da22d Introduce restart_bundle containers to detect config changes and restart pacemaker resources
During the containerization work we regressed on the restart of
pacemaker resources when a config change for the service was detected.
In baremetal we used to do the following:
1) If a puppet config change was detect we'd touch a file with the
   service name under /var/lib/tripleo/pacemaker-restarts/<service>
2) A post deployment bash script (extraconfig/tasks/pacemaker_resource_restart.sh)
   would test for the service file's existence and restart the pcs service via
   'pcs resource restart --wait=600 service' on the bootstrap node.

With this patchset we make use of paunch's ability do detect if a config
hash change happened to respawn a temporary container (called
<service>_restart_bundle) which will simply always restart the pacemaker
service from the bootstrap node whenever invoked, but only if the pcmk
resource already exists. For this reason we add config_volume and bind
mount it inside the container, so that the TRIPLEO_CONFIG_HASH env
variable gets generated for these *_restart_bundle containers.

We tested this change as follows:
A) Deployed an HA overcloud with this change and observed that pcmk resources
   were not restarted needlessly during initial deploy
B) Rerun the exact same overcloud deploy with no changes, observed that
   no spurious restarts would take place
C) Added an env file to trigger the of config of haproxy[1], redeployed and observed that it restarted
   haproxy only:
   Jun 06 16:22:37 overcloud-controller-0 dockerd-current[15272]: haproxy-bundle restart invoked
D) Added a trigger [2] for mysql config change, redeployed and observed restart:
   Jun 06 16:40:52 overcloud-controller-0 dockerd-current[15272]: galera-bundle restart invoked
E) Added a trigger [3] for a rabbitmq config change, redeployed and observed restart:
   Jun 06 17:03:41 overcloud-controller-0 dockerd-current[15272]: rabbitmq-bundle restart invoked
F) Added a trigger [4] for a redis config change, redeployed and observed restart:
   Jun 07 08:42:54 overcloud-controller-0 dockerd-current[15272]: redis-bundle restart invoked
G) Rerun a deploy with no changes and observed that no spurious restarts
   were triggered

[1] haproxy config change trigger:
parameter_defaults:
  ExtraConfig:
    tripleo::haproxy::haproxy_globals_override:
      'maxconn': 1111

[2] mysql config change trigger:
parameter_defaults:
  ExtraConfig:
    mysql_max_connections: 1111

[3] rabbitmq config change trigger (default partition handling is 'ignore'):
parameter_defaults:
  ExtraConfig:
    rabbitmq_config_variables:
      cluster_partition_handling: 'pause_minority'
      queue_master_locator: '<<"min-masters">>'
      loopback_users: '[]'

[4] redis config change trigger:
parameter_defaults:
  ExtraConfig:
    redis::tcp_backlog: 666
    redis::params::tcp_backlog: 666

Change-Id: I62870c055097569ceab2ff67cf0fe63122277c5b
Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
Closes-Bug: #1775196
2018-06-08 16:06:24 +02:00
Dustin Schoenbrun
5e3d90f851 Fix typo with Manila upgrade template
There was a typo in the update_tasks for Manila which was causing
updates and upgrades to fail. This patch fixes the typo.

Closes-Bug: 1775667

Change-Id: I88dd16fa94111a4eb56aeaa32b560cf7d12b9f82
2018-06-07 18:59:31 +00:00
Zuul
e0139adfda Merge "Decouple OpenShiftGlusterDisks from openshift-master.yaml" 2018-06-07 11:09:21 +00:00
Martin André
b9caa67a62 Add docker service to OpenShift CI roles
This ensures the docker service on the openshift nodes is able to pull
from a local registry if configured this way.

Change-Id: Ifd48b2e6500b10d108985a4a9f1d73493d404134
Depends-On: I31494ff8524b90343e6e8c67bd08a354837ecc45
2018-06-07 10:45:26 +02:00
Zuul
c9a29bb632 Merge "Add sanity checks for xinetd service restart." 2018-06-06 23:19:57 +00:00
Zuul
d3f8ac065f Merge "Revert "Only add internal_api_virtual_ip if InternalApi is in network_data"" 2018-06-05 20:23:09 +00:00
Zuul
d00ada6d2c Merge "Add OPNFV scenario environment" 2018-06-05 18:33:01 +00:00
Zuul
3489a01acf Merge "Add flag to enable QoS DSCP marking in ODL" 2018-06-05 18:20:42 +00:00
Sergii Golovatiuk
9a37e5ac42 Fix fail_if_empty invocation with pipes
* Since $@ parameter may have pipe '|' it should be processed correctly.
  Currenly the part before pipe is assigned as $@ so bash runs pipe with
  commands after it after execution of function. However, we want to assign
  command with pipe to $@ thus "" around command with pipe are required.
* replace $() with eval as $() doesn't work correctly with pipe as it tries to
  escape pipe so output variable contains wrong data.
* This patch adds tonumber to first invocation

Change-Id: I958e14c0a4ea4b5782d2c74dc895471b0f70b875
2018-06-05 17:20:52 +02:00
Zuul
837a58df1a Merge "Add release note for PasswordAuthentication parameter" 2018-06-05 13:33:11 +00:00
Zuul
68c3856b83 Merge "Enable inspector dnsmasq dhcp filter" 2018-06-05 11:16:28 +00:00
Yurii Prokulevych
302af5066e Add sanity checks for xinetd service restart.
xinetd.service is not installed on pre-provisioned nodes,
 so we'll add extra check for its restart.

Change-Id: I4fbac81ceb4aba534395cf8c0a842fb732559234
Closes-Bug: 1775154
2018-06-05 12:01:37 +02:00
Steven Hardy
487e0617d6 Decouple OpenShiftGlusterDisks from openshift-master.yaml
In the event you have different disks in nodes assigned to each role
you may need to pass role specific parameters, e.g like:

parameter_defaults:
  OpenShiftMasterParameters:
    OpenShiftGlusterDisks:
      - /dev/vdc
  OpenShiftWorkerParameters:
    OpenShiftGlusterDisks:
      - /dev/vdd

To enable that we create an inventory file per role, and pass the directory
of files to ansible.

Change-Id: I8b4d8698405ffb004b081e1f097f300216edfa77
2018-06-05 09:45:52 +01:00
Zuul
e3d6a47a31 Merge "undercloud: enable KernelIpNonLocalBind" 2018-06-05 07:21:23 +00:00
Pradeep Kilambi
b3b027e731 Add a way to override base path when file driver is used
The goal is to be able to point the Gnocchi file driver directory
to an NFS share.

A new parameter GnocchiFileBasePath allows to customize
the bind mount to /var/lib/gnocchi.

Change-Id: I868a368161f4a529e5e7dc3593dc6862e3196247
2018-06-05 07:49:22 +02:00
Zuul
542f9e00ab Merge "Merge values for same key from multiple services" 2018-06-05 02:45:25 +00:00
Emilien Macchi
32ea5028fd undercloud: enable KernelIpNonLocalBind
We need KernelIpNonLocalBind on the undercloud to bind non local ips
among other ip forward options. This sysctl parameter was managed by
instack-undercloud but never ported to the containerized undercloud.
We need the same sysctl parameters for parity with non containerized
undercloud.

Change-Id: Idd3d432b8f7eb573d94cd56be8e05614510ebddf
Related-Bug: #1774898
2018-06-05 01:38:51 +00:00
Zuul
30ddce3e3d Merge "ssh: enable PasswordAuthentication for containerized undercloud" 2018-06-05 01:07:26 +00:00
Tim Rozet
3b81a45281 Revert "Only add internal_api_virtual_ip if InternalApi is in network_data"
This reverts commit 8cfaf354d2673c9d4b8bc955ab8adc81de167b1d.

Change-Id: Ib8ff69a4bc869de21ad838b3bc6c38a8676036c6
2018-06-04 20:26:31 +00:00
Emilien Macchi
b749e027a0 Add release note for PasswordAuthentication parameter
Add release note for I10b112e8bffff30879606ddd970dfd3ec67fd9c7.

Change-Id: I9475e6386a3747e7aeddb997e2e0bf585652060c
2018-06-04 10:41:14 -07:00
michaeltchapman
f3f4e2856c Add OPNFV scenario environment
Add an OPNFV scenario environment that uses ODL for overcloud
networking and OVS for virthost networking.

Depends-On: I33602ac5521c4f059c1a0d08e3e828fb64d3c817
Depends-On: Ib7968c46a59f266c20628c36178d2235ad833915
Depends-On: I37405e41ec0f85249cef87c09c966cbe0f9baddf
Change-Id: If1f476bb933106456df3568978b4555dde190621
2018-06-04 10:46:30 -04:00
Milan Kováčik
a1a2048d47 Enable inspector dnsmasq dhcp filter
Modify both the inspector and dnsmasq containers for the inspector to be
able to modify dnsmasq configuration on the fly to filter the dhcp
traffic.

The upgrade_tasks moved to the puppet service in order to be shared
between both the containerised and regular deployment.  The upgrade_tasks
were amended with steps to clean-up the iptables inspector chain&rules.

With inspector no longer managing iptables rules, create new rules to
allow DHCP traffic on IronicInspectorInterface.

Co-Authored-By: Harald Jensås <hjensas@redhat.com>
Change-Id: Ic7e32acb8559a7a12cd8767dc68c343872a6a4e3
Depends-On: I056cdadc025f35d8b6fd22f510a7c0a8e259a1f0
2018-06-04 16:36:14 +02:00
Zuul
9108e56456 Merge "Support containerized DVR in compute role" 2018-06-04 14:34:57 +00:00
Zuul
34b6e5e7ec Merge "standalone: cleanup environment" 2018-06-04 14:34:54 +00:00
Zuul
939a32f246 Merge "Create docker-puppet.sh only once to avoid ETXTBSY errors" 2018-06-04 14:04:55 +00:00
Zuul
d5a2c0b3d0 Merge "Pike to Queens controller upgrade guard rerun with no images" 2018-06-04 09:13:47 +00:00
Emilien Macchi
70901ab69a ssh: enable PasswordAuthentication for containerized undercloud
We don't expect our operators to have SSH keys setup on the undercloud
node, so we don't want to block the PasswordAuthentication in
sshd_config.

Depends-On: I88b24c82fb3cf2309f45d5d447a9b0c403da7fc9
Change-Id: I10b112e8bffff30879606ddd970dfd3ec67fd9c7
Closes-Bug: #1772519
2018-06-03 01:49:26 +00:00
Zuul
a33d05d30d Merge "Run image prepare after registry install" 2018-06-01 22:23:51 +00:00
Zuul
97d6aa7b51 Merge "Ensure WorkflowSteps resource is enabled in ceph-upgrade-prepare" 2018-06-01 20:40:25 +00:00
Zuul
fdef4acb68 Merge "Upgrades: Refactor httpd_enabled variable" 2018-06-01 17:43:59 +00:00
Zuul
c099f863c9 Merge "Clear up Upgrade params on converge" 2018-06-01 17:43:57 +00:00
Zuul
096cef093c Merge "Manage public certificate with ansible" 2018-06-01 17:37:48 +00:00
Brent Eagles
f51f84e781 Support containerized DVR in compute role
This patch adds the required parameters to the Compute role so the
agents are configured properly on upgrade.

Related-Bug: #1774199
Change-Id: Iab42ae0fb13e8e92cc9903432a95e04a94a5913c
2018-06-01 11:31:50 -02:30
Zuul
125b0a6109 Merge "Optimized Ansible tasks in deplay-steps-tasks.yaml" 2018-06-01 13:53:31 +00:00
Giulio Fidente
ba8168fe96 Ensure WorkflowSteps resource is enabled in ceph-upgrade-prepare
To trigger ceph-ansible we need to make sure the WorkfowSteps
resource is enabled in ceph-upgrade-prepare env file.

Change-Id: Id760305971a68c397f9334265dd023b1e1884295
Closes-Bug: 1774647
2018-06-01 15:51:48 +02:00
mandreou
adb10e6586 Pike to Queens controller upgrade guard rerun with no images
As discussed at [0] if current overcloud resources are removed
manually/some error, and even container images deleted, the
upgrade tasks should be able to guard.

We already have a similar guard for the fetch&retag at [1]

[0] https://bugzilla.redhat.com/show_bug.cgi?id=1584809
[1] 8824e7abcd/docker/services/pacemaker/haproxy.yaml (L305-L324)

Change-Id: I2c81e6d0f73fbef0f2a347b9fd4d27df91c2fdd1
2018-06-01 12:13:42 +03:00
Michele Baldessari
1037786b6c Create docker-puppet.sh only once to avoid ETXTBSY errors
We currently create /var/lib/docker-puppet/docker-puppet.sh
inside the mp_puppet_config() function which then gets
invoked in parallel via the following:

  p = multiprocessing.Pool(process_count)
  returncodes = list(p.map(mp_puppet_config, process_map))

This is problematic because we have the following potential race:
1) Process A opens /var/lib/docker-puppet/docker-puppet.sh for writing
2) Process B runs docker run and has the following bind mount:
   /var/lib/docker-puppet/docker-puppet.sh:/var/lib/docker-puppet/docker-puppet.sh:z
3) Process B will fail because an exec of a file being written to
   will return ETXTBSY

The deployment can fail due to the above with the following error:
[root@overcloud-controller-2 ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a401108cd827 192.168.24.1:8787/tripleoqueens/centos-binary-glance-api:current-tripleo-rdo "/var/lib/docker-p..." 19 minutes ago Exited (1) 19 minutes ago docker-puppet-glance_api
[root@overcloud-controller-2 ~]# docker logs docker-puppet-glance_api
standard_init_linux.go:178: exec user process caused "text file busy"

Since /var/lib/docker-puppet/docker-puppet.sh never changes
there is really no need to create it multiple times. Let's just
create it once before spawning the multiple docker run commands
so we avoid any ETXTBSY errors.

Ran 20 successful deployments in sequence with this change applied.

Change-Id: I16b19488ce9f1411273459576db76d16b318dacb
Closes-Bug: #1760787
2018-06-01 07:13:19 +02:00
Zuul
5a2ac547b6 Merge "Add site id parameter to cisco vts ml2 template" 2018-06-01 04:14:51 +00:00
Zuul
f0b6d58db9 Merge "Pass designate configuration to neutron" 2018-06-01 02:12:29 +00:00
Zuul
7c88c50d2c Merge "Add basics for standalone node" 2018-06-01 01:17:22 +00:00
Zuul
c6d5c11784 Merge "Drop old ceilometer services" 2018-06-01 01:07:42 +00:00