openstack-ansible-rabbitmq_.../tasks/rabbitmq_cluster_join.yml
Markos Chandras aec44599f8 rabbitmq_cluster_join: Retry if starting the RabbitMQ application fails
Sometimes starting the RabbitMQ application could fail with the
following error:

container2:~ # rabbitmqctl start_app
Starting node rabbit@container2 ...

BOOT FAILED
===========

Error description:
   enoent

Log files (may contain more information):
   /var/log/rabbitmq/rabbit@container2.log
   /var/log/rabbitmq/rabbit@container2-sasl.log

Stack trace:
   [{erlang,open_port,
            [{spawn_executable,false},
             [{args,["unix-sendto:/run/systemd/notify","STDIO"]},
              use_stdio,out]],
            []},
    {rabbit,sd_open_port,0,[{file,"src/rabbit.erl"},{line,345}]},
    {rabbit,sd_notify_socat,1,[{file,"src/rabbit.erl"},{line,351}]},
    {rabbit,maybe_sd_notify,0,[{file,"src/rabbit.erl"},{line,291}]},
    {rabbit,broker_start,0,[{file,"src/rabbit.erl"},{line,283}]},
    {rabbit,start_it,1,[{file,"src/rabbit.erl"},{line,403}]}]

Error: enoent

This seems to go away if we retry so it's probably a race condition
somewhere. As such, try a few more times before giving up.

Change-Id: I7b9a6f6cedd5e9a6494e4a2ca129601638cfafd5
2017-06-20 15:35:02 +01:00

52 lines
1.6 KiB
YAML

---
# Copyright 2014, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# If cluster name is our own hostname, we assume we're not properly clustered
# TODO(someone): implement a more robust way of checking
# if node is clustered or not
- block:
- name: Check cluster status
shell: |
rabbitmqctl -q cluster_status | grep '{cluster_name,<<"{{ rabbitmq_cluster_name }}">>}'
changed_when: false
tags:
- rabbitmq-cluster
rescue:
- name: Stop rabbitmq app
shell: |
rabbitmqctl stop_app; sleep 5
tags:
- rabbitmq-cluster
- name: Join rabbitmq cluster
command: >
rabbitmqctl join_cluster "rabbit@{{ rabbitmq_primary_cluster_node.split('.')[0] }}"
register: rabbit_join_cluster
until: rabbit_join_cluster|success
retries: 5
delay: 2
tags:
- rabbitmq-cluster
- name: Start rabbitmq app
command: rabbitmqctl start_app
register: rabbit_start_app
until: rabbit_start_app|success
retries: 5
delay: 2
tags:
- rabbitmq-cluster