For a rationale for this patch, see the discussion surrounding Bug
When reconnecting to a RabbitMQ cluster with mirrored queues in
use, the attempt to release the connection can hang "indefinitely"
somewhere deep down in Kombu. Blocking the thread for a bit
prior to release seems to kludge around the problem where it is
otherwise reproduceable.
DocImpact
Change-Id: Ic2ede3046709b831adf8204e4c909c589c1786c4
Partial-Bug: 856764
With mirrored queues and clustered rabbit nodes a queue is still
mastered by a single rabbit node. When the rabbit node dies an
election occurs amongst the remaining nodes and a new master is
elected. When a slave is promoted to master it will close all the
open channels to its consumers but it will not close the
connections. This is reported to consumers as a consumer cancel
notification (CCN). Consumers need to re-subscribe to these queues
when they recieve a CCN.
kombu 2.1.4+ reports CCNs as channel errors. This patch updates
the ensure function to be more inline with the upstream kombu
functionality. We now monitor for channel errors as well as
connection errors and initiate a reconnect if we detect an error.
Change-Id: Ie00f67e65250dc983fa45877c14091ad4ae136b4
Partial-Bug: 856764
Move log_handler from oslo-incubator to oslo.messaging and include
unit tests. This patch also adds an additional test case from the
original version in oslo-incubator. Originally, the test case only
checked to make sure the notification was not sent. Now a positive test
case exists to test both paths.
bp graduate-notifier-log-handler
Co-Authored By: Ben Nemec <bnemec@redhat.com>
Change-Id: I9ea051ec7e3614579341a31d8dc1c13d9514b8a9
The patch in I6e37780cc28737cfd56b6719ec8d9cebbc9bb278 added a new class
for the QpidMessage which was wrongly initialized when processing a
message.
Change-Id: Iaf525fc6ee9e56100c89863f9c651f20c148ecb5
Closes-Bug: 1282038
In an HA deployment, a 60 seconds delay between reconnects can be quite
problematic. This patch changes the delay calculation by setting the max
delay to 5s and by changing the way it is increased.
Unfortunately, this is one of the places where both our main drivers are
not consistent. Rabbit's driver uses configuration parameters for this
whereas qpid's driver has never had one. However, I would prefer not
adding configuration paremeters to qpid's driver for the following
reasons:
1. Most of OpenStack services depend on the messaging layer, hence
they need it to be available. A 5s delay seems to be reasonable and
I could argue the need of tune it further. Although so frequent
reconnects can add load to the network, that wouldn't be the main
issue if one of the brokers go down.
2. We're trying to move away from configuration options towards using
transport URL. This path is still not clear and I would
prefer avoiding adding new options until we clear it out.
Change-Id: I537015f452eb770acba41fdedfe221628f52a920
Closes-bug: #1281148
This patch make the dispatcher responsible of the message
acknowledgement.
This is the preliminar step to be able to requeue message.
Partial implements blueprint notification-subscriber-server
Change-Id: If74b47d5e23976d407deb27df7395b1982963c75
The notification listener doesn't have anything to send to the notifier
and the notifier doesn't attend to receive something.
So this patch remove the message reply when the listener is a
notification
listener.
Partial implements blueprint notification-subscriber-server
Change-Id: Ic989947ba3b6894cde788422842fca19159ea261
The patch add ta abstraction layer to acknowledge a message.
Partial implements blueprint notification-subscriber-server
Change-Id: I6e37780cc28737cfd56b6719ec8d9cebbc9bb278
Having the sphinx theme in the same namespace package
as the production code has been causing issues with
devstack installations. The solution settled on was
to rename oslo.sphinx to oslosphinx because oslo.sphinx
is not a "production" library.
See the linked bug report for more background details.
Closes-Bug: #1277168
Change-Id: I220b8901cef36499e91b92719f1e8e5461a95e92
Follow oslo.config style guide for help strings better to create
consistent help strings:
* Capitalize first word of each help string
* Finish help strings with "."
Change-Id: Ia08fa09593661e6e5b834d98bbd92689c2674075
Catch expected exceptions in a way that supports subclasses also being
seen as "expected."
Log a little more detail about unexpected exceptions in the ExecutorBase
so it is easier to add them to the list of expected exceptions.
Change-Id: I1bdd628eba717308f0afe1a889efd8711bad6296
Closes-bug: #1276163
Running the tests installs pbr in a way that leaves a .egg directory in
the source tree, so update the ignore file to ignore it.
Change-Id: I7d702c9511992efc518195def0967bc0191d93a9
In Qpid implementation, when using multiple qpid_hosts, we don't want to
immediately retry failed connection for the same failed broker. This was not
the case in existing implementation though, where we've always attempted to
reconnect starting from the first broker in the list of candidates. So if the
first broker failed, we initiated reconnect to the same failed broker.
This change makes reconnect() implementation to select the next broker in the
list. This also means that non-failure reconnect attempts will also switch the
current broker. All in all, users should not rely on any particular order to
use brokers from the list, so this should not constitute an issue.
Change-Id: I257c2ad6d7ebead356c0239134340975da6dbc07
Partial-Bug: 1261631
In RabbitMQ implementation, when using multiple rabbit_hosts, we don't want to
immediately retry failed connection for the same failed broker. This was not
the case in existing implementation though, where we've always attempted to
reconnect starting from the first broker in the list of candidates. So if the
first broker failed, we initiated reconnect to the same failed broker.
This change makes reconnect() implementation to select the next broker in the
list. This also means that non-failure reconnect attempts will also switch the
current broker. All in all, users should not rely on any particular order to
use brokers from the list, so this should not constitute an issue.
Change-Id: I76965ea06c1fb6fd0cd384a5b3edc14703470b8f
Partial-Bug: 1261631
The dispatcher is now responsible to configure and to get the listener from
the transport.
The server just ask to the dispatcher to build and return a configured
listener for a provider transport.
Partial implements blueprint notification-subscriber-server
Change-Id: I4a6d9620b8239f6d377bc5788b8a90a860b2f02c
This patch allow the fake driver to comsume multiple topics
with one listener.
Partial implements blueprint notification-subscriber-server
Change-Id: Ib52dc181e10b487854fbb398eda9f758232a1251
Currently it is not possible to modify the login method used with
kombu connections via configuration files, and requires modifying
the source code to actually add a login method in the first place.
This adds a configuration option (and default) which are passed
along to the kombu connection.
RabbitMQ allows three login method strings by default, but server
side plugins allow arbitrarily named strings to be used as login
methods and as such it is not possible to provide a fixed list of
options to the user.
DocImpact
Change-Id: I5c62a8dd4125d42bb429d3985061e37a9d519fff
Closes-Bug: #1269890
Replace the use of TestManager with make_test_instance
to get an instance of the type of manager expected.
The old TestManager class is deprecated
Change-Id: I717c0296bf8ae07b1dad8f0045bfe12c8620ad0a
Partial-bug: #1273455
Register a oslo.messaging entry point in the oslo.config.opts namespace
which, when called, returns a list of the configuration options which
may be registered by the library at runtime.
The idea here is that the sample config file generator can query this
and include the returned options in the sample config file of any
applications which use the library.
Some sample code for fetching available options:
import pkg_resources
for ep in pkg_resources.iter_entry_points('oslo.config.opts'):
if ep.name == "oslo.messaging":
list_opts = ep.load()
for g, opts in list_opts():
opts.sort(key=lambda x: x.name)
print "[%s]" % g
for o in opts:
print "%s = %s" % (o.name, o.default)
print
Related-Bug: #1241566
Change-Id: Ic28351258652d2ea38974e2f4d1aa6b1d3dd7192
This change corrects the UnboundLocalError exception that
can occur if something bad happens on reconnect to RabbitMQ
Closes-Bug: #1272271
Change-Id: I9e157810caa7a45e9a61ec571cfe9024fabacf93
In order to transition to a python 3.x compatible world,
lets switch to mox3 which is python 3.x compatible drop-in
replacement of mox.
Change-Id: Iaa22779cd516f842c9f70828f863f90ddae0bab5
Takes a yaml-based config file
(see etc/oslo/routing_notifier.yaml.sample) via
routing_notifier_config option.
Events may be routed by priority or event_type.
Implements: blueprint configurable-notification
Change-Id: I437dfac348f387044e6da3d6a0bbb208323c1741
The set_time_override function in timeutils was written as a
helper function to mock utcnow for unittests before 'mock' was
generally used. Now that we have mock and fixture, we no longer
need to use it.
Change-Id: I9a0727edab12ccd5f1e4eb4f5f62d588f5a0faee
Partial-Bug: #1266962
