__metaclass__ cannot be used in python3.six be used in general
for python 3 compatibility.
Porting Change-Id I9fc7a59df3af29b4cc1287c40fa4e883d994a961
from oslo-incubator
Change-Id: Icdacdcf5556b6d3b8450d1350c6f62b4f5a9690b
This makes the RPC version support three elements, adding a "revision"
in addition to the major and minor version. The revision would always
be zero for the master branch of a service, but could be incremented
for stable versions. This provides us some room to backport fixes
that affect RPC versions in such a way that would avoid breaking
the version lineage for systems running stable versions that may
some day be involved in a rolling upgrade to a version from master.
I didn't find any tests for version_is_compatible(), so I added
some for existing version scenarios, as well as new ones with
revisions. They also serve to validate that this doesn't break
anything for code using two-element versions (the expectation is that
two-element versions will still be used everywhere until a third
is needed).
Porting chages from Change-Id I239c17a3e305f572493498c4b96ee3c7514c5881
to oslo-incubator
Change-Id: I4fa7b0be14a7afba36136a746b76036355f119b2
packages is recursive, so there is no need to list subpackages
versions are set by tags, so the version in the file is not needed
non-d2to1-based pbr does not need the hook specification
Change-Id: Id5e6c19dfe81c630862e9b87b7f9e5f67a965945
This implements the server side of the driver without modifying the
existing code by allowing the driver to spawn off multiple greenthreads
as before, but queueing any dispatched messages so that the executor
can still do listener.poll() to dispatch messages itself.
This is a hack, but it's a starting point.
Change-Id: Ie299c2695d81d0473cea81d40114326b89de0011
This is the ZeroMQ server which acts as a proxy for all messages
destined to a particular host. Again, there are a bunch of FIXMEs
here. This still needs work.
Change-Id: I9384f486e44b0b0cbca028e219ad66f1990d5181
Get sending working with an initial version of the driver. There's a
bunch of FIXMEs inline reflecting that even the client side needs a
tonne of work yet.
Change-Id: I6d69ebc9ae3b3999832209e0c4100ffe26e35919
Modifications are:
- use stdlib logging; no huge need for oslo logging here
- stub out the _() function; we don't have any l10n infrastructure in
the project and may never have
- change imports to oslo.messaging.openstack.common and
oslo.messaging._drivers as appropriate
Change-Id: I87b85b79a33dec65e51ed95fff90cc56042240c5
Concurrency. Sigh.
A sequence of events like this is possible:
- We send a request from thread A
- Thread B, who is waiting for a response gets scheduled
- Thread B receives our response and queues it up
- Thread B receives its own response and drops the connection lock
- Thread A grabs the connection lock and wait for a response to arrive
The obvious solution is that when we grab the connection lock, we should
check whether a previous lock-holding thread had already received our
response and queued it up.
Change-Id: I88b0d55d5a40814a84d82ed4f42d5ba85d2ef7e0
There are a number of situations in which we log a message if an
exception occurs during the handling of a message:
1) Something goes wrong pulling the message from the queue and
de-serializing it - here we print "Failed to process message"
2) An RPC endpoint method raises an expected exception - here we
print an 'Expected exception during message handling' debug
message
3) An RPC endpoint method raises any other exception - here we
should print an 'Exception during message handling' error message
However, in the latter case, we are currently printing out the 'Failed
to process' error message.
Change-Id: I4f7042b8ec978aaff8f4e20e62ba1ac765fe6ba5
On the server side, we only send replies if the request included a
_msg_id key. Also, the _reply_q key is only used when we wish to send a
reply.
So, in order to retain the exact same on-the-wire behaviour and ensure
servers aren't sending replies where none is needed, only include these
keys if we're doing a call (i.e. wait_for_reply=True).
Change-Id: Iac329493252be7d94b1ebe24f00e4d3f5c61d269
I added check_for_lock because I assumed it was enabled by default and
actively in use by Nova. However, it actually isn't used by Nova yet and
enabling spews a tonne of warnings.
It's a rather clunky API and there's a good chance we can design a
better API for it, so let's leave it out until we're ready to actually
start using it in Nova.
Related-Bug: #1063222
Change-Id: Ib890978398059f360cd0f3352f4755262b8111c6
Nova sends notifications with a bunch of different publisher_ids, so we
instantiate quite a lot of Notifier objects. Loading the noification
drivers for each of these is a substantial amount of overhead.
One obvious answer would be to make publisher_id an argument to the
error(), info(), etc. methods, but I think it's nice to encapsulate the
publisher_id in a notifier instance.
Instead, add a prepare() method which mirrors the approach in RPCClient.
You use this method to create a specialized notifier instance with a new
publisher_id.
Change-Id: Ia45fda3164086bb7a9ef6dee0587f726ab8b1a97
Handle e.g. foo://u:p@/bar. Right now we get:
IndexError: string index out of range
from:
if hostname[0] == '[':
Change-Id: I0bccebb14ad1d37862955e8988d160240bd1cf6d
Just like the bug in the fake driver, we have a similar rather
embarassing and obvious bug - we're currently not allowing endpoint
methods to send replies of None, '', False, [], {}, etc.
Change-Id: Icf0abdfcf122c5757dd3737f66130b3a53769ef6
There's no need to make the fixtures, testtools, etc. libraries a
runtime requirement, so let's move it from the toplevel oslo.messaging
API so you need to explicitly import it.
Change-Id: I9e2f32a898d78489f2d8d9c218c81f35cda14e34
The driver reply() method is actually passed a full sys.exc_info()
tuple.
This was masked in the unit tests because the driver ended up basically
doing:
raise (ValueError, ValueError, ...)
which caused a new ValueError to be instantiated and the test was
satisified. However, if an exception type has some required arguments,
you'll get a TypeError when this statement attempts to instantiate it
with no arguments.
Change-Id: I4af9c5084954d7b9c5f02cdae3387d17c206985b
Just copies code from Nova's cells code.
See I6b18f7643ab694f5ff34206b80865c40b1ec2680 for where this code was
first introduced.
Change-Id: If9b1503ed47f5910c0ab44edfbe3f42fcce9bf18
A rather embarassing and obvious bug - we're currently not allowing
endpoint methods to send replies of None, '', False, [], {}, etc.
Change-Id: Ifc5ff8f308f526197559a4df7bed244cff6ed3c1
The configuration options registered by oslo.messaging should not be
directly relied upon by users of the library, since those config option
names could change in future.
Add an API which allows API users to override specific configuration
options e.g.
self.messaging_conf = self.useFixture(messaging.ConfFixture(cfg.CONF))
self.messaging_conf.transport_driver = 'fake'
Change-Id: If0d837e1b86e3b04237fde522551cfb81505a543
Nova's cells/rpc_driver.py has some code which allows user of the REST
API to update elements of a cell's transport URL (say, the host name of
the message broker) stored in the database. To achieve this, it has
a parse_transport_url() method which breaks the URL into its constituent
parts and an unparse_transport_url() which re-forms it again after
updating some of its parts.
This is all fine and, since it's fairly specialized, it wouldn't be a
big deal to leave this code in Nova for now ... except the unparse
method looks at CONF.rpc_backend to know what scheme to use in the
returned URL if now backend was specified.
oslo.messaging registers the rpc_backend option, but the ability to
reference any option registered by the library should not be relied upon
by users of the library. Imagine, for instance, if we renamed the option
in future (with backwards compat for old configurations), then this
would mean API breakage.
So, long story short - an API along these lines makes some sense, but
especially since not having it would mean we'd need to add some way to
query the name of the transport driver.
In this commit, we add a simple new TransportURL class:
>>> url = messaging.TransportURL.parse(cfg.CONF, 'foo:///')
>>> str(url), url
('foo:///', <TransportURL transport='foo'>)
>>> url.hosts.append(messaging.TransportHost(hostname='localhost'))
>>> str(url), url
('foo://localhost/', <TransportURL transport='foo', hosts=[<TransportHost hostname='localhost'>]>)
>>> url.transport = None
>>> str(url), url
('kombu://localhost/', <TransportURL transport='kombu', hosts=[<TransportHost hostname='localhost'>]>)
>>> cfg.CONF.set_override('rpc_backend', 'bar')
>>> str(url), url
('bar://localhost/', <TransportURL transport='bar', hosts=[<TransportHost hostname='localhost'>]>)
The TransportURL.parse() method equates to parse_transport_url() and
TransportURL.__str__() equates to unparse_transport().
The transport drivers are also updated to take a TransportURL as a
required argument, which simplifies the handling of transport URLs in
the drivers.
Change-Id: Ic04173476329858e4a2c2d2707e9d4aeb212d127
This is apparently fixed in pbr since I3972b3132619e8e2dd7e362ca5fe9d1e3add43b8
but I'm seeing the issue with pbr 0.5.21 on Fedora.
Related-Bug: #1194742
Change-Id: I136b8493c8d8d48a0116facf5f23c2a1479c070f
If a transport URL is supplied, transform it into the server_params
format which was previously used for cast_to_server() etc.
Change-Id: I453734a71748dc8d3ffc02ead7bfb92ffb0a6c7c
My original thinking was that if you're using the exchange name to
separate two instances of the applications (so their queues don't
collide) then the exchange name is pretty key to transport
configuration. In fact, it's really a virtual host that you'd use for
this (at least in the case of rabbit and qpid).
Also, Nova's cells code has already moved ahead with the assumption that
the path specifies a virtual host, so it'd only make sense to deviate
from that if there was a really good reason to.
Change-Id: Ic8b5dc3538b6b17afec524047acc2efa76366377
We shouldn't be logging expected exceptions. Add a failing rabbit driver
test to check this and fix it.
Change-Id: I78b758957117be7c11c5826a27dd6d1d4fffe9cb
Oslo's logging code has some useful support for including bits of the
request context in log messages. While this isn't exclusively about the
request context in a dispatching RPC method, it seems useful for
oslo.messaging to support the concept for at least this use case simply
by recording the context in a thread local store before dispatching an
endpoint method and immediately clearing it when the method returns.
Note, we don't need to store weak refs in our store because we will
clear the reference in all cases rather than ever leaving a stale
reference around in the store.
Change-Id: I70ac06ed3a2a891a7a7b388b1823a0f3b08f2dd1
The client call() and cast() methods take a request context argument
which is expected to be a dictionary. The RPC endpoint methods on the
server are invoked with a dictionary context argument.
However, Nova passes a nova.context.RequestContext on the client side
and the oslo-incubator RPC code deserializes that as a amqp.RpcContext
which looks vaguely compatible with Nova's RequestContext.
Support the serialization and deserialization of RequestContext objects
with an additional (de)serialize_context() hook on our Serializer class.
Note: this is a backwards incompatible API change because Serializer
implementations which do not implement the new abstract methods would
no longer be possible to instantiate. Not a problem with this commit,
but shows the type of compat concerns we'll need to think about once the
API is locked down for good.
Change-Id: I20782bad77fa0b0e396d082df852ca355548f9b7
