The client call() and cast() methods take a request context argument
which is expected to be a dictionary. The RPC endpoint methods on the
server are invoked with a dictionary context argument.
However, Nova passes a nova.context.RequestContext on the client side
and the oslo-incubator RPC code deserializes that as a amqp.RpcContext
which looks vaguely compatible with Nova's RequestContext.
Support the serialization and deserialization of RequestContext objects
with an additional (de)serialize_context() hook on our Serializer class.
Note: this is a backwards incompatible API change because Serializer
implementations which do not implement the new abstract methods would
no longer be possible to instantiate. Not a problem with this commit,
but shows the type of compat concerns we'll need to think about once the
API is locked down for good.
Change-Id: I20782bad77fa0b0e396d082df852ca355548f9b7
Currently we have a allowed_rpc_exception_modules configuration variable
which we use to configure a per-project list of modules which we will
allow exceptions to be instantiated from when deserializing remote
errors.
It makes no sense for this to be user configurable, instead the list of
modules should be set when you create a transport.
Closes-Bug: #1031719
Change-Id: Ib40e92cb920996ec5e8f63d6f2cbd88fd01a90f2
If a remote endpoint raises an exception which the client is not allowed
to (or cannot) deserialize, then RPCClient.call() raises a RemoteError
exception instead.
Make this exception type part of the public API.
Change-Id: I70be0ab7d40af3224d93d6bd0522c1a82f6303c3
Review I4e7b19dc730342091fd70a717065741d56da4555 gives a lot of the
background here, but the idea is that some exceptions raised by an RPC
endpoint method do not indicate any sort of failure and should not be
logged by the server as an error.
The classic example of this is conductor's instance_get() method raising
InstanceNotFound. This is perfectly normal and should not be considered
an error.
The new API is a decorator which you can use with RPC endpoints methods
to indicate which exceptions are expected:
@messaging.expected_exceptions(InstanceNotFound)
def instance_get(self, context, instance_id):
...
but we also need to expose the ExpectedException type itself so that
direct "local" users of the endpoint class know what type will be used
to wrap expected exceptions. For example, Nova has an ExceptionHelper
class which unwraps the original exception from an ExpectedException and
re-raises it.
I've changed from client_exceptions() and ClientException to make it
more clear it's intent. I felt that the "client" naming gave the
impression it was intended for use on the client side.
Change-Id: Ieec4600bd6b70cf31ac7925a98a517b84acada4d
Notifications are an unusual case in that we need users to manually opt
in to new incompatible message formats by editing configuration because
there may be external consumers expecting the old format.
Add a send_notification() method to the driver interface and add a
format version paramater to the method, to make it clear that this
version selection is specifically for notifications.
In the case of the rabbit/qpid drivers, the 2.0 format is where we added
the message envelope.
Change-Id: Ib4925c308b1252503749962aa16f043281f2b429
The target preconditions (e.g. you need at least a topic to send) are
the same for all drivers, so enforce them before we ever call into a
driver.
Change-Id: Ic4e9bd94bd9f060ec0662d2bb778c699903dddc4
This method is fairly gnarly, so break my usual preference to make the
intent clear from the code and instead include detailed comments in the
method.
Change-Id: I107272a7eab85c70581652488a3c14ce0e18b906
These FIXMEs don't need fixing now that I think it through some more.
The reply format is specific to drivers and the 'ending' flag is part of
the existing wire protocol that we need to support even if we don't
support multicall().
Change-Id: I834f0bb01513b5318f0b365948a7d9247feb49bf
We appear to not have a use for this. I had originally thought we might
use this to ack messages one they've been processed and replied to, but
we actually have always acked messages as soon as they have been
deserialized and queued for dispatching.
Change-Id: I8e1fd565814f3b5e3ba0f1bc77e62ed52ff08661
I typically avoided using e.g. assertIsNone() thinking we couldn't use
it on 2.7, but now that we use testtools.TestCast there are a bunch of
useful assertion methods we can use.
Change-Id: I7696dc4744cdfd2466773326f202bc08dcfcbf0f
Make the rabbit driver properly serialize exceptions before sending them
back in a reply and then properly re-raise them on the client side.
Also, extend the rabbit driver test to cover this case.
Change-Id: I6b3d03edcd41810125ba6442db5515754f0c1ac9
The trickiest logic in the rabbit driver is to handle the situation
where multiple threads are waiting for a reply on the same reply queue.
This commit adds unit testing for that scenario and fixes some bugs with
it.
Change-Id: I5c8fbeec49572a4f3badbcdae414dc44dc690b6a
While we're iterating over the queues in ReplyWaiters.wake_all(), new
queues can be registered and we get:
RuntimeError: dictionary changed size during iteration
Instead of using an iterator, take a snapshot list of message IDs and
operate on that.
We don't actually care about any new queues added after wake_all() is
called because the connection lock has already been dropped so one of
the other waiters must have picked it up.
We also don't need to worry about queues being removed - if we write to
a removed queue, that's not going to be a problem.
Change-Id: Ib572cbfd3a7346b76579f82b64aa85a03c1a4fb2
Add two test cases:
- send messages using the rabbit driver and check that the message
gets sent to the expected queue in the expected format
- send messages to a given queue in the correct format and check that
we can happily consume them using the rabbit driver
i.e. these are tests that check that the driver sends and receives
messages according to the wire protocol.
Change-Id: I6a48085585981b17609b31f216bdca438e90e068
Drivers should not be returning messages with any driver-specific
wire format fields included. Right now, the rabbit and qpid drivers are
leaking the _unique_id field.
Avoid this simply by popping _unique_id when we're checking for dups.
Change-Id: Icbfb02ee66aebe5ef97a9a2502d8a0745e07bd8b
Pretty obvious that we need this.
The rabbit/qpid implementations just empty the connection pool, in the
same way their module-level cleanup() methods do now.
Change-Id: I70ba5cab3eb7a30f74cdd6cafe60087769a77b57
These aren't really appropriate for dumping in the repo like this
so I've moved them here:
https://github.com/markmc/oslo.messaging-notes
Change-Id: Iba4c19cefd8a02831543d5c2c3b1886d5d31fa63
This just adds a QpidDriver which re-uses AMQPDriver.
The test-rabbit-{client,server}.py test scripts works fine with this
driver and the URL changed to qpid:///
Change-Id: Ifb72853e83b678b311fecb21fe016af19a67b272
We absolutely do not want this to be a requirement, but for the moment
the rabbit and qpid drivers import eventlet and avoid using it at
runtime. This should be cleaned up shortly and we can remove the dep
again.
Change-Id: Iaef75964a502598a94d95aa77f0e44d1f7faba50
This is unused. I think I added it before iter_free(), i.e. empty()
was going to be:
for item in self.get(only_free=True):
item.close()
rather than:
for item in self.iter_free():
item.close()
Change-Id: Ie6d0de3b7453ee036412db10f37d26f04de73cf7
These are pretty comprehensive tests for the exception serialization
code. The code itself code do with a bunch of refactoring, but that will
come later.
The main TODO item is that we need a way for users of the API to
register the list of modules that it wants to allow exceptions to be
deserialized from. There is no need for this to be an end-user
configuration option, but we do need each service to be able to specify
its own list.
Change-Id: Ib2fc3f1ca5da708b41bdc00e8ac6ce0c92399ce8
We're going to be using iterconsume(limit=1) and this basically seems
to be broken right now because you get an error if you call consume()
multiple times on the same connection.
Set the 'do_consume' flag at connection time rather than on entry into
iterconsume().
Change-Id: I988e4074ae0e267384931d6e1994e9cbe5248196
We don't have any infrastructure for localizations in oslo.messaging
so using this is pointless right now. I'm also generally not convinced
we want to translate any of the strings in this library anyway.
For now, just add a dummy _() function. We can can unmark the strings
later.
Change-Id: I1b6a698ee5558c50dc5eafee1f5f05ee2570435e
There's obviously an awful lot of work left to do on this. Some 20
FIXMEs, for a start :)
However, the test client can successfully invoke a call() and get a
reply back from the server!
The main complexity is in how the client waits for a reply, especially
where there are multiple threads all waiting for replies. Rather than
follow the current method of spawning off a greenthread (and the implied
dependency on eventlet) to read the replies and pass them to the queue
for the appropriate waiting thread, we instead have one of the waiting
threads take on that responsibility.
Change-Id: I20d3d66a5cc9820752e7eaebd8871ffb235d31c9
This means we no longer set the request context for the current thread
so that it can be used in logging.
We do need to add this back later, but it might be in the form of a
get_current_context() method.
Change-Id: I3f08a85e2019affddec829e2ea008b5c10707660
