974 Commits

Author SHA1 Message Date
Mehdi Abaakouk
1f8ccd3ac5 rabbit: Add logging on blocked connection
When the broker will block the connection for a server-side issue
like disk full, it notifies the client.

This change adds the callback methods when this occurs to inform
the deployer about the reason of this blocking.

Change-Id: I5164b9e1b720f022b45a5718258df036ba8808ed
Closes-bug: #1454449
2015-06-09 10:35:24 +02:00
Jenkins
85c069e154 Merge "Set places to 0 in self.assertAlmostEqual()" 2015-06-08 16:41:39 +00:00
Jenkins
02ab25ecc6 Merge "Reduce magic conf attribute usage" 2015-06-08 16:40:22 +00:00
Jenkins
63bbdfc7c9 Merge "rabbit: test for new reply behavior" 2015-06-05 11:20:37 +00:00
Jenkins
2d133bee84 Merge "Enable fanout_target scenarios in test_impl_rabbit" 2015-06-05 11:20:11 +00:00
Victor Sergeyev
3044abbaf4 Reduce magic conf attribute usage
Connection class contains `magic` attribute conf, which contains (or at
least should contain) all config options. Set these config options as
class attributes to make the code more clear

Change-Id: Ib67f148e2ecf3def37e57d6e3359bced932fbc90
2015-06-05 11:12:58 +03:00
Jenkins
f51a4ef6c6 Merge "Imported Translations from Transifex" 2015-06-04 18:11:35 +00:00
Jenkins
d685e6f80a Merge "replace rpc_response_timeout use in rabbit driver" 2015-06-04 17:50:16 +00:00
OpenStack Proposal Bot
9f146bc299 Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I55f2a8f6f81f12735f0cf5c0ad20401283bb4e08
2015-06-04 06:08:05 +00:00
Ben Nemec
94bb8adfd0 Remove leftover oslo.config reference
There was still a reference to the oslo.config namespace package
in _drivers/base.py.  This converts it to the new package name.

Change-Id: I9c3878094bcf8015c30d87f693f51e0d48b31a33
2015-06-04 00:06:40 +00:00
Doug Hellmann
a0b33a4659 replace rpc_response_timeout use in rabbit driver
The rabbit driver was using the rpc_response_timeout configuration
option as a reconnect timeout value, even though the option was defined
only in oslo_messaging.client and was not always being registered before
being accessed. An earlier patch fixed this by registering the option
here in the driver, too, but that breaks several levels of
abstraction. This changes the driver to define a new option with the
same default value, so that the driver is only using options it defines
itself. It also removes the old hacky fix.

Closes-bug: #1461182

Change-Id: Ia96c815d157219e12a10d94b87b0156503369a6b
2015-06-03 19:56:50 +00:00
Victor Sergeyev
91dd104e13 Enable fanout_target scenarios in test_impl_rabbit
Tests with `fanout_target` scenarios were skipped because of issue [1].
This issue already fixed, so we can enable these tests.

[1] https://github.com/celery/kombu/issues/195

Change-Id: I6600c6b9089e6093cbad2b0b96f2682ac9d25c69
2015-06-03 14:50:40 +03:00
Jenkins
5b4038ce37 Merge "Ensure rpc_response_timeout is registered before using it" 2015-06-02 22:49:13 +00:00
Doug Hellmann
887e5a0423 Ensure rpc_response_timeout is registered before using it
The response code in the rabbit driver doesn't use a Client object, so
the option is not being registered in servers that don't instantiate
Client instances (ironic-conductor, for example).

Change-Id: I7def5e6d4960938a17344db024585a0492d6969d
Partial-bug: #1461182
2015-06-02 18:48:33 +00:00
Mehdi Abaakouk
bc6e2a8c15 rabbit: test for new reply behavior
Blueprint: remove-double-reply

Change-Id: I17f3b2511e976de275dfcb50e6ed6a9a2d328941
2015-05-30 12:57:20 +03:00
Jenkins
27efb36b87 Merge "Allow to remove second _send_reply() call" 2015-05-30 00:11:05 +00:00
Jenkins
ec5a789f22 Merge "Fix condition in _publish_and_retry_on_missing_exchange()" 2015-05-29 22:55:32 +00:00
Victor Sergeyev
5181361f7c Fix condition in _publish_and_retry_on_missing_exchange()
The current condition logic in _publish_and_retry_on_missing_exchange()
replaces passed `timeout` value with `rpc_response_timeout` config value.

This patch fix it.

Change-Id: I451cd2b5da78a3d7011466dfb14669aa1dcf1d5c
2015-05-29 17:28:28 +03:00
Victor Sergeyev
fa2a501c52 Set places to 0 in self.assertAlmostEqual()
We are checking time of thread run in test_impl_rabbit.py and sometimes
it can be a bit longer, because of server load.

Set places=0 instead of 1 to make tests less restrictive.

Change-Id: I2ef6f1474047e1da64402e93696cb73b4d3318e8
2015-05-29 15:00:48 +03:00
Victor Sergeyev
e93f62303c Allow to remove second _send_reply() call
Added a new configuration option `send_single_reply` which allows to
send a single AMQP reply instead of two. This will reduce amount of RPC
calls and increase transport productivity.

The new behaviour is not compatible with the old logic, so isn't
backward compatible and disabled by default.

DocImpact
A new configuration option added.

Blueprint: remove-double-reply

Change-Id: Idab118b22163e734aca010f325cddfaec26bfa0f
2015-05-28 16:22:38 +03:00
Victor Sergeyev
c1c0af2069 Don't create a new channel in RabbitMQ Connection.reset()
Current implementation of RabbitMQ driver in in Connection.reset() change
the channel to use and create a new channel for it. This happens after the
each message send.

There no big need to create a new channel each time, so we can cancel all
consumer queues, instead of creating a channel in reset().

Test test_connection_reset_always_succeed() removed, because we are not
create channel on reset() anymore.

Co-Authored-By: Mehdi Abaakouk <sileht@redhat.com>
Change-Id: Ie164840e6c055b01525b13aabdb8b9c7f5d1b98b
2015-05-28 15:32:37 +03:00
Jenkins
baddce34a2 Merge "Adding Publisher Acknowledgements/confirms" 2015-05-28 11:13:11 +00:00
Jenkins
ed432e59c4 Merge "Fix deprecated_group of rpc_conn_pool_size" 2015-05-28 11:00:04 +00:00
OpenStack Proposal Bot
29c3cfaa3c Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I8490386a5664b74c2c8d76a7afa632412a9466d9
2015-05-28 06:08:50 +00:00
Jenkins
dda4e698b3 Merge "consumer connections not closed properly" 2015-05-27 16:20:33 +00:00
Davanum Srinivas
cd63a70235 Adding Publisher Acknowledgements/confirms
During the vancouver summit, Michael Klishin from
pivotal proposed we should use publisher confirms
as the default setting to reduce messages being
dropped on the floor.

https://www.rabbitmq.com/confirms.html
http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/

This setting is a lightweight way of keeping track of which messages
have been processed by the broker and which would need re-publishing
in case of broker shutdown or network failure.

Change-Id: I25bc955df130dad4725f5281211d37fd73e7ea63
2015-05-27 16:43:29 +02:00
Jenkins
02fae06de5 Merge "Refactor processing reply in ReplyWaiter" 2015-05-27 13:17:12 +00:00
Mehdi Abaakouk
18c32bc30c Fix deprecated_group of rpc_conn_pool_size
When rpc_conn_pool_size have been moved from amqp.py to base.py:

  87137e7af05f12a99bd04566036fbf71824f45cf

We loose the deprecated_group, this change reintroduces it.

Change-Id: I8cdea7f042afebcc162bafef881ebe61a1cac989
2015-05-27 13:06:40 +02:00
Victor Sergeyev
75cba16d15 Refactor processing reply in ReplyWaiter
At the moment for each msg_id we receive two amqp message - first one
with the payload, a second one to ensure the other have finish to send
the payload. This was made, because a long time ago 'reply' allowed
generator as payload to send multiple messages on one 'rpc.call' - [1]

It's a bad idea - to double RPC messages for each call, so we are going
to remove this the second AMQP message sending. This patch allows
receiver side to proceed correctly old case - two AMQP messages (first
with data and second with 'ending' parameter) same as the new one (a
single message with data 'ending' parameter)

Blueprint: remove-double-reply

[1] - https://github.com/openstack/oslo-incubator/blob/stable/icehouse/openstack/common/rpc/amqp.py#L464

Change-Id: Ic09fe619694c300c4502acb7157d7ecdd47c5fd7
2015-05-27 11:08:17 +03:00
Mehdi Abaakouk
336e3c8f60 rabbit: doc fixes
Change-Id: I945b30ebad35f463220eedc092c7fcd958740bd7
2015-05-27 06:37:54 +00:00
gordon chung
a450841264 consumer connections not closed properly
heartbeat_thread is not set for listeners. when closing connection,
it blindly checks heartbeat_thread and will throw an error causing
connection to remain open. this patch explicitly sets
heartbeat_thread to None.

Change-Id: Ief3bf02f952882ecadf742cdd0bac8edd7812473
Closes-Bug: #1458917
2015-05-26 21:22:16 -04:00
Jenkins
cf98f07f1f Merge "rabbit: smart timeout on missing exchange" 2015-05-26 17:04:30 +00:00
Mehdi Abaakouk
f3d4ba7f6b rabbit: smart timeout on missing exchange
When we send a reply and the exchange is missing, they
no need to wait more that the timeout set by the application.

Change-Id: I7eb19346d72cb0a4813c964df1435d7f4c79e218
2015-05-26 15:06:52 +00:00
gtt116
708d9d842b rabbit: Fix message ttl not work
impl_rabbit set timeout into message's header with {'ttl': (timeout * 1000)},
this mean doesn't work in real, messages still stays in queue after the ttl.

As RabbitMQ document said (http://www.rabbitmq.com/ttl.html#per-message-ttl),
we should passing "expiration" into message's property rather than header to
make it work.

Change-Id: I5d6ae72e69f856c56fb83fb939ed35246716e04d
Closes-bug: #1444854
2015-05-26 16:50:08 +02:00
Jenkins
ecb7803d5f Merge "rabbit: remove publisher classes" 2015-05-26 14:39:20 +00:00
Jenkins
cc42675a15 Merge "Add one more functional test for MessagingTimeout" 2015-05-26 13:39:59 +00:00
Mehdi Abaakouk
ccbe866d32 rabbit: remove publisher classes
Each times we send a message with the rabbit driver we create useless
object that's not more that the kombu exchange and a wrapper method around
the kombu producer.

So, this change just creates the exchange not our useless custom Publisher
and move the wrapped methods into the Connection object.

Change-Id: Id221f4363d897cd904f7aeccbc90cbd288db2db1
2015-05-26 08:51:27 +02:00
Mehdi Abaakouk
77f952a1f7 rabbit: Set timeout on the underlying socket
They are some case where the underlying can be stuck
until the system socket timeout is reached, but in oslo.messaging
we very often known that is not needed to wait for ever because
the upper layer (usualy the application) expect to return after
a certain period.

So this change set the timeout on the underlying socket each we can
determine that is not needed to wait more.

Closes-bug: #1436788
Change-Id: Ie71ab8147c56eaf672585da107bec8b22af9da6c
2015-05-26 08:51:26 +02:00
Davanum Srinivas
8af6b2ff82 Remove stale copy of context.py
Change-Id: I2d09873ddb7a9ef688308cfdbe6deb1959cdd2ac
2015-05-24 03:13:43 +00:00
Jenkins
29c573b3f2 Merge "make it possible to import amqp driver without dependencies" 2015-05-23 01:08:42 +00:00
Victor Sergeyev
cc4ca1f9ef Add one more functional test for MessagingTimeout
Add a check, that MessagingTimeout raises on long-running queries,
if client sends another queries at the same time.

Added a long_running_task() to TestServerEndpoint and allowed to pass a
message executor into the RpcServerFixture.

Related bug: #1338732

Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Change-Id: Icafb6838e2d9fb76b6d1c202465c09c174a3bed9
2015-05-22 16:35:25 +03:00
Victor Sergeyev
3d483fda70 Fix list_opts test to not check all deps
The list_opts entrypoint test failed unnecessarily when the
dependencies in the packages were inconsistent. This test doesn't
need to verify that the dependencies are consistent, only that the
entrypoint is available and provides the expected function.

Change-Id: I0bb0f2b591c402202104af8daf07d56b514cbb2f
2015-05-22 16:30:26 +03:00
Jenkins
7bbd6e4769 Merge "rabbit: fixes a logging issue" 2015-05-17 21:54:11 +00:00
Doug Hellmann
1da2231116 make it possible to import amqp driver without dependencies
All plugins are supposed to be importable without their dependencies so
we can discover options and documentation. Restructure the parts of the
AMQP1 driver that depend on having proton and pyngus installed so the
driver can load without them.

Change-Id: Id0c8c2a6ae44d13f061e651c33efc9e38750a049
2015-05-16 14:21:32 -04:00
Jenkins
7ded4a972f Merge "Remove outdated release notes" 2015-05-15 14:15:51 +00:00
Jenkins
4fa7795f42 Merge "rabbit: smarter declaration of the notif. queue" 2015-05-13 19:54:35 +00:00
Jenkins
cab3f20c36 Merge "rabbit: redeclare consumers when ack/requeue fail" 2015-05-13 19:50:41 +00:00
Mehdi Abaakouk
65fecf2866 Remove outdated release notes
Oslo.messaging have an outdated release in the code tree.
but now the release note is published on the mailing.

This change removes it.

Change-Id: I0a3401b7c9bc8230169e75727e45a99e6c3c780f
2015-05-13 11:19:45 +02:00
Jenkins
cfc28916f9 Merge "Remove support for Python 3.3" 2015-05-12 19:43:45 +00:00
Mehdi Abaakouk
1ba55c03b5 rabbit: smarter declaration of the notif. queue
The NotifyPublisher was redeclaring again and again,
the same exchange and queue each times a notification is sent.

This change fixes that by caching the already declared exchange
and queue for each channel.

Also, to make the test pass. 'Connection.ensure' have been updated
to have the same behavior for amqp and memory driver about
kombu recoverable_errors. And the hostname and port of the memory
driver are set to not fail when we print a log message.

Closes bug: #1437902

Change-Id: I20d133ac67b8a8a4c51d51b6a1b2369aa44ffe2f
2015-05-12 10:19:42 +02:00