Jens Rosenboom 7ad0d7eaf9 Fix reconnect race condition with RabbitMQ cluster
Retry Queue creation to workaround race condition
that may happen when both the client and broker race over
exchange creation and deletion respectively which happen only
when the Queue/Exchange were created with auto-delete flag.

Queues/Exchange declared with auto-delete instruct the Broker to
delete the Queue when the last Consumer disconnect from it, and
the Exchange when the last Queue is deleted from this Exchange.

Now in a RabbitMQ cluster setup, if the cluster node that we are
connected to go down, 2 things will happen:

 1. From RabbitMQ side, the Queues w/ auto-delete will be deleted
    from the other cluster nodes and then the Exchanges that the
    Queues are bind to if they were also created w/ auto-delete.
 2. From client side, client will reconnect to another cluster
    node and call queue.declare() which  create Exchanges then
    Queues then Binding in that order.

Now in a happy path the queues/exchanges will be deleted from the
broker before client start re-creating them again, but it also
possible that the client first start by creating queues/exchange
as part of the queue.declare() call, which are no-op operations
b/c they alreay existed, but before it could bind Queue to
Exchange, RabbitMQ nodes just received the 'signal' that the
queue doesn't have any consumer so it should be delete, and the
same with exchanges, which will lead to binding fail with
NotFound error.

Illustration of the time line from Client and RabbitMQ cluster
respectively when the race condition happen:

       e-declare(E)      q-declare(Q)       q-bind(Q, E)
     -----+------------------+----------------+----------->
                               e-delete(E)
     ------------------------------+---------------------->

Change-Id: Ideb73af6f246a8282780cdb204d675d5d4555bf0
Closes-Bug: #1318721
2014-11-27 13:49:52 +01:00
2014-01-15 04:08:18 +00:00
2014-06-05 22:48:44 +02:00
2014-04-30 02:45:47 +00:00
2014-10-31 20:29:28 +00:00

Oslo Messaging Library

The Oslo messaging API supports RPC and notifications over a number of different messaging transports.

See also: Library Documentation

Description
OpenStack library for messaging
Readme 36 MiB
Languages
Python 99.8%
Shell 0.2%