The retry variable can be None which causes this exception to be
formatted incorrectly. Switching from %(retry)d to %(retry)s
should handle the case where retry is None more gracefully.
Change-Id: I592ea3e44506afb4a676d8dbe1e659a0e649424e
Closes-bug: #1440755
To make sure notify.logger always play well with messaging(v2) driver.
Achieve this by seting up a logger with target logging handler, doing real
logging, finally check notifications that receive from message broker.
Change-Id: I234457974ca48689b9dfc9f7c72b4ea37a57a40b
When publish_errors is True, and using `messaging` notify driver,
produces a infinite loop that report sending notification error.
The reason is we always passing None to the content argument in
logger handlers (LoggingNotificationHandler, LoggingErrorNotificationHandler),
when pack the context object which expected as a dict, raise TypeError
exception, so result in infinite retry looping.
To match what document said, passing a empty dict rather than None. Also
modify unittests to make sure context is a empty dict.
Change-Id: Ic2c4c43f5bbafa8107ea370ba959da16cfa4a24c
Closes-bug: #1346466
The decaying timer can now just use the functionality
of the oslo_utils stop watch to avoid having to maintain
similar information itself; now it just becomes a thin
layer ontop of that object that provides its functionality.
Change-Id: I1b014b821a6b980590ca5b4d850a515d55c42208
Different OpenStack processes log that line when idle, but it doesn't
offer actionable information to developers or users. Ideally process
logs should be silent when idle, even in debug mode.
Closes-Bug: #1434727
Change-Id: I6f9f2977358d86ada7178c09b04ff6b290a6a8ad
Url constructed for kombu cannot contain ipv6 address without
brackets. The url.hostname returns hosts without brackets so
we have to readd them.
Change-Id: Ifb1d358a67655af99a84b77ca813fc2dd87d87dc
With the new heartbeat code, there may be a lot of tracebacks
we should mute them for normal ops and log them only when
we need (enable debug level).
Change-Id: I4b729ed1a6ddad2a0e48102852b2ce7d66423eaa
Currently, the option amqp_durable_queues is both deprecated with name
and group that cause the option [DEFAULT]amqp_durable_queues can't work.
This patch use multi deprecated options to make it work.
Change-Id: Ied28bcf415362a976928bac75225018030304ac7
Closes-Bug: #1433956
This patch was proposed by this review:
https://review.openstack.org/#/c/31231
The unit test is added.
Change-Id: Ie124dc22f4ef3bbaeed76186cf08924a39b52812
Co-Authored-By: Eric Windisch <ewindisch@docker.com>
Closes-Bug: #1186310
This change ensures that connections that fail to return to the pool are
cleanly closed and exception raised are not returned to the caller.
For rabbit, we also try to reconnection in case of connection failure,
before dropping the connection.
Closes-bug: #1433458
Change-Id: Ic714db7b8be9df8b6935a903732c60aaea0bc404
For all transport errors, we first got a log of a failure with a
backtrace like 'Fail to publish message on topic' or 'fail to consume
messages' and then another message like 'AMQP is unreachable'.
But in most case, we retry to consume/publish messages when the reason
is a connection lost.
So, now we don't log the failure message anymore in case of connection lost,
but just message related to rabbit deconnection/reconnection.
The error message and the backtrace are only logged in case of we really fail to
publish a message.
Change-Id: Ifa1b04b348b347bad0b2abec3e759a2ce7815b86
AMQP offers a heartbeat feature to ensure that the application layer
promptly finds out about disrupted connections (and also completely
unresponsive peers). If the client requests heartbeats on connection, rabbit
server will regularly send messages to each connections with the expectation of
a response.
To acheive this, each driver connection object spawn a thread that
send/retrieve heartbeat packets exchanged between the server and the
client.
To protect the concurrency access to the kombu connection between the
driver and this thread use a lock that always prioritize the
heartbeat thread. So when the heartbeat thread wakes up it will acquire the
lock quickly, to ensure we have no heartbeat starvation when the driver
sends a lot of messages.
Also when we are polling the broker, the lock can be held for a long
time by the 'consume' method, so this one does the heartbeat stuffs itself.
DocImpact: 2 new configuration options for Rabbit driver
Co-Authored-By: Oleksii Zamiatin <ozamiatin@mirantis.com>
Co-Authored-By: Ilya Pekelny <ipekelny@mirantis.com>
Related-Bug: #1371723
Closes-Bug: #856764
Change-Id: I1d3a635f3853bc13ffc14034468f1ac6262c11a3
When starting heartbeat in matchmaker-redis, keys may be changed
during iteration and a runtime exception is thrown as follows:
RuntimeError: dictionary changed size during iteration
Change-Id: I016a449674619294196055614c62c2856002291c
Closes-Bug: #1432966
This guide introduces the basic scenario of zeromq
driver and how to correctly deploy zeromq driver
for OpenStack.
Change-Id: I23eeafc5863b44bfbeb89ce3571a4e79b4b01d01
Closes-Bug: #1382325
