ZeroMQ manages failover automatically when socket is
being connected to multiple hosts. Dynamic connections
mode assumes to connect once per message so we connect
only to a single host. However we cannot say for sure
if this host is still alive. For failover reasons
we can connect to a primary host and then add some
more hosts as failover and let ZeroMQ to decide where
it finally sends the message.
Change-Id: I19cbb75aaea8a0b725dd6a8ff665392738e164a1
Now that there exists only a gate job for Python 3.5 and not 3.4,
we should remove those references to the 3.4 that is untested.
Change-Id: I2a00ead626d9eced96487ee82b5d7857126d8355
The driver incorrectly set up SSL client authentication. This patch
fixes the configuration. Unit tests added.
Change-Id: I64b1736f5a708c70013d9fedba73da4fa8d9ccfb
Closes-Bug: #1606985
For zmq.DEALER socket there are two round-robin modes exist,
controlled by zmq.IMMEDIATE option. Immediate True means sending
only to running servers and False means to send to all connected
servers (may be not up and running at the moment).
If we do all messaging over DEALER socket as we do for direct
connections, immediate=True better suits for direct CALL/CAST
message types and immediate=False is better for fanout.
This patch fixes things for dynamic connections as for static
we already used the approach.
Change-Id: I70c7aeb3c7a2c63c128bec394827577cab580def
This change makes the monkeypatching the first thing to do.
It removes the try/except since the tests requires eventlet.
Change-Id: Ic6bbe303d7f4a606212b46add2fa51ce14c569ae
A message ack status of 'RELEASED' or 'MODIFIED' indicates that the
message was not accepted by the destination due to some temporary
issue. These status are used to indicate to the sender that the
message can be safely re-transmitted without risk of duplication
(i.e. the delivery is not 'in-doubt'). For example this may happen
during a message bus topology update if a message is sent before the
topology stabilizes.
This change implements re-send for these cases.
Closes-Bug: #1646586
Change-Id: I419e23b59e3eb90fda3f1c0e7ddf54ef98870e4b
Redis as a key-value storage has limitations on values
expiration by ttl, only keys can expire. Regarding this
we need to introduce some redundancy and register hostnames
as keys along with general target keys to make them expire
on ttl timeout.
Change-Id: Ic0b06d8ec27b63ca49afe941a32020b5e532598c
When the connection fails due to an authentication issue raise a
MessageDeliveryFailure on all pending send requests. Include the
authentication error message in the exception.
Change-Id: I06b40c6c480a4d082dce64801736b3d12f696c26
Closes-Bug: 1508512
consume() must return only if user timeout is reached and not
when driver consumer_timeout is reached.
Change-Id: I6b2b2a28038a194224e79fa37285436ca6787a0a
This producer singleton works only if one transport use kafka.
If multiple transports with kafka are used, their overrides each other
the producer on each send.
This change creates only one producer per connection.
A later optimisation can be one producer per driver instance.
Change-Id: I429f7c5efcb41690dd1b17f856bda2425c788e53
Currently Kafka driver for an oslo.messaging uses kafka-python==0.9.5
and mostly broken. This package version supports only low level Kafka
producer and consumer API which are marked as deprecated now [1]. Using
of these interfaces bring a big concern to the message processing,
because current KafkaConsumer has not any consuming coordination. This
fact causes a message duplication for the several consumers of one
topic. This behavior is specific to Ceilometer and services which read
and process notifications from other services.
New version of kafka-python allows to use async thread safe message
producers and coordinated consumers [1].
[1] http://kafka-python.readthedocs.io/en/master/changelog.html#feb-15-2016
The driver is currently experimental, python-kafka<1.0.0 API have major
issue described above that can't make the oslo.messaging driver works,
so we prefer having a working driver with a non-synced dependencies, that the
reverse.
Co-Authored-By: Mehdi Abaakouk <sileht@redhat.com>
Change-Id: I29862ed7bf56b9d8878fa8e9fb1cbd9d643908a4
