27 Commits

Author SHA1 Message Date
Kenneth Giusti
73c0c0071e Add the "transport_options" parameter to the amqp1 and kafka drivers.
Change-Id: I5ca6ec2cb30b8d7f18e1770f80024957bc029bf3
2019-06-20 16:12:39 -04:00
Hervé Beraud
4f385720d7 Remove log translation and i18n
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

Change-Id: I9ddb6595fc52e46ed8844e39d2fa71029c90b65c
Closes-Bug: #1674567
2019-04-24 11:55:38 +02:00
Kenneth Giusti
684e3f0e41 Enable RPC call monitoring in AMQP 1.0 driver
The call monitoring feature was introduced in commit
b34ab8b1cc9f4d513a2927c102dbbe82031d9c2a for RabbitMQ.  This patch
enables the feature on the AMQP 1.0 driver - currently the only other
driver that supports RPC.

Change-Id: Ic787696852690b59779fb4716aec1e78c48bbe6a
2018-06-13 11:01:38 -04:00
Dan Smith
b34ab8b1cc [rabbitmq] Implement active call monitoring
This adds an optional call_monitor_timeout parameter to the RPC client,
which if specified, will enable heartbeating of long-running calls by
the server. This enables the user to increase the regular timeout to
a much larger value, allowing calls to take a very long time, but
with heartbeating to indicate that they are still running on the server
side. If the server stops heartbeating, then the call_monitor_timeout
takes over and we fail with the usual MessagingTimeout instead of waiting
for the longer overall timeout to expire.

Change-Id: I60334aaf019f177a984583528b71d00859d31f84
2018-05-09 10:08:37 -07:00
Dan Smith
930e6189e2 Add heartbeat() method to RpcIncomingMessage
This adds a heartbeat() method to RpcIncomingMessage to be used by a
subsequent patch implementation of active-call heartbeating. This is
unimplemented in all drivers for the moment.

Change-Id: If8ab0dc16e3bef69d5a826c31c0fe35e403ac6a1
2018-04-24 07:35:56 -07:00
Jenkins
ff905655e9 Merge "Add support for virtual hosts" 2017-09-07 03:27:50 +00:00
Rajath Agasthya
d15a6deb17 Remove envelope argument from driver send() interface
It is ignored by most drivers and hardcoded to True in the rabbit
driver

Closes-Bug: #1712877
Change-Id: Ic6b0d07bff3c544aca4cccdfcb4c59b8705e9128
2017-08-25 08:17:47 -07:00
Kenneth Giusti
263dce9ea8 Add support for virtual hosts
Adds the 'pseudo_vhost' option which when enabled will incorporate the
virtual host into the address semantics.  This creates a 'subnet' like
address space for each virtual host.  Use this when the messaging bus
does not provide virtual hosting support.  It is enabled by default as
to date none of the supported AMQP 1.0 message buses natively support
virtual hosting.

It also updates SSL support: SSL can either use the connection
hostname or the vhost name when validating a server's
certificate. This is controlled by the 'ssl_verify_vhost' option.
This option is disabled by default as it requires both vhost and SNI
support from the server.  By default SSL will use the DNS name from
the TransportURL.

Change-Id: I49bb99d1b19e8e7e6fded76198da92ca5f7d65ab
Closes-Bug: #1700835
Partial-Bug: #1706987
2017-07-28 16:37:26 -04:00
Kenneth Giusti
d778aa5a44 [AMQP 1.0] if RPC call is configured as presettled ignore acks
The pre-settled configuration option was being ignored in the case of
RPC call transfers.

Change-Id: I35989c8a653bac0d9a7c3e33a8af2027574f4bd9
Closes-Bug: 1680905
2017-04-07 13:23:55 -04:00
Mehdi Abaakouk
037dee19ed Validate the transport url query string
When driver load we allow to override option unrelated to the driver and
to set option useless for the driver.

This change validates the query string when the driver load to report as
soon as possible invalid options. And allow to override only option
of the driver option group (ie: [oslo_messaging_<driver_name>].

Related-bug: #1666903

Change-Id: Iaf23f773279c10bf37d545883ada7c2f6a9ffbbf
2017-02-27 13:10:31 +01:00
Mehdi Abaakouk
58b026a2aa drivers: use common.ConfigOptsProxy everywhere
ConfigOptsProxy have been implemented only pika driver while
the oslo.messaging allow to pass the query string for all drivers.

This change fixes that.
Closes-Bug: #1666903
Closes-Bug: #1607889

Next step is to validate the query with ConfigOptsProxy, to
raise appropriate exception in case of mis-configuration.

Change-Id: I573334e774ccf33ecd27a85067045f3c6489ee89
2017-02-27 13:10:31 +01:00
Kenneth Giusti
9fab0bdbc8 [AMQP 1.0] Resend messages that are released or modified
A message ack status of 'RELEASED' or 'MODIFIED' indicates that the
message was not accepted by the destination due to some temporary
issue.  These status are used to indicate to the sender that the
message can be safely re-transmitted without risk of duplication
(i.e. the delivery is not 'in-doubt').  For example this may happen
during a message bus topology update if a message is sent before the
topology stabilizes.

This change implements re-send for these cases.

Closes-Bug: #1646586
Change-Id: I419e23b59e3eb90fda3f1c0e7ddf54ef98870e4b
2017-01-09 09:30:57 -05:00
Kenneth Giusti
bc46e64711 [AMQP 1.0] Avoid unnecessary thread switch on ack
Avoid blocking the RPC Server thread when it attempts to acknowledge
the RPC request message.  Blocking is unnecessary as the acknowledge
command does not return a status and can be processed asynchronously
from the server thread.

Avoiding this context switch improves overall RPC call throughput
according to the simulator tool by approximately ten percent on my
test systems.

Change-Id: I71548eb6f9f7dcaf74cb426d4e9b369b54856419
2016-10-11 10:11:35 -04:00
Kenneth Giusti
9775ed3cce Remove debug logs from fast path
Decrease the size of the debug log output drastically by removing
debug messages issued for each message.  This patch only removes those
log statements that log correct behavior - errant behavior is still
logged for debug.

Change-Id: I1c67e6ed7e503ef3b9543fdfce31b91c46bd851a
2016-10-03 16:31:35 -04:00
Kenneth Giusti
6c973dae22 [AMQP 1.0] Make the default settlement behavior configurable
This patch allows the ack processing behavior to be configured based
on the type of message sent.  This allows the proper ack behavior for
a given backend to be customized. By default Cast messages are now
sent pre-acked (best effort).

Change-Id: I0d4e44a7a87f3c917a3eb44f6e02af74a9af10a5
Closes-Bug: #1616612
2016-08-24 15:56:51 -04:00
Jenkins
ba387dfd1c Merge "[AMQP 1.0] small fixes to improve timer scalability" into feature/amqp-dispatch-router 2016-08-17 05:49:40 +00:00
Kenneth Giusti
b7717e1616 [AMQP 1.0] small fixes to improve timer scalability
This patch introduces the following tweaks to the timer
implementation:

Reduce the number of timers that need to be tracked by reducing the
timer granularity to units of seconds.

Decrease the default timeout values to further reduce the total number
of tracked timers.

Batch multiple expiring events that share the same deadline.

Inline the timer comparison code in the main event loop.

Avoid using an expensive comparison method in the heap sort by using
an integer primitive instead.

Use monotonic time instead of time.time()

Change-Id: I83e86bf203e6a641085e482c7ccf0e01f4fb4d86
2016-08-15 13:30:37 -04:00
Jenkins
cdd279b516 Merge "Remove the max_send_retries option" into feature/amqp-dispatch-router 2016-08-06 20:44:39 +00:00
Kenneth Giusti
3109941e5e Log a warning when connected to a routable message bus
Newton will be the first release of support for this new message bus
technology.  Make users aware that this feature is 'bleeding edge' and
queuing is not supported in this release.

Change-Id: I44ba5507d941823e2dfa92d52dae4b85e9bda493
2016-08-04 18:25:50 -04:00
Kenneth Giusti
5236e99155 Remove the max_send_retries option
This option was added for feature parity with the rabbitmq driver.
Since then the option was removed from rabbitmq.  Since this option
was never released we can simply remove it rather than deprecating it.

Change-Id: Ic10dd374c865eaf7c7563f379426e171996184a0
Closes-Bug: #1599518
2016-08-01 11:19:06 -04:00
Andrew Smith
ded89418a2 [AMQP 1.0] Add configuration parameters for send message deadline
This patch add driver configuration option parameters for oslo_messaging_amqp
reply, send and notification timeout. These timeouts are used when the caller
does not provide a timeout expiry.

Change-Id: I398a4686abf3df6e5c802d94d6bda7ba9995833d
Depends-On: I6b60b6944a4c44da9b2e41ab2a83ab59f15e396e
Closes-Bug: #1596641
2016-07-07 08:55:04 -04:00
Kenneth Giusti
4c0674d194 Re-factor the AMQP 1.0 addressing semantics
This patch introduces a new routable addressing format.  This new
address format not only works with traditional broker-based messaging
backends but can be used to optimally route messages across a mesh
messaging topology.  This new addressing format may be enabled either
via configuration or by message bus identification.

This patch moves all of the AMQP 1.0 addressing logic into a new
Addresser class.  This class is used to map an oslo.messaging Target
address and notification flag into a corresponding AMQP 1.0
message/subscription address based on the addressing mode.  This hides
the addressing details from the rest of the driver.

For backward compatibility with previous releases the Addresser can be
configured to automatically detect a non-routable messaging back end
and fall back to using the legacy addressing scheme.

The intention is to use routable addressing as the only form of
addressing supported by the driver regardless of messaging back end.
However before that can happen the legacy addressing must follow the
standard deprecation process.  Until then legacy addressing will
remain available for backwards compatibility.

Change-Id: I6b60b6944a4c44da9b2e41ab2a83ab59f15e396e
2016-07-06 10:28:19 -04:00
Andrew Smith
35ea442a0e [AMQP 1.0] Add acknowledge and requeue handling for incoming message
This patch schedules a controller task to handle the acknowledge or
requeue disposition of the incoming message

Change-Id: I9a53959ce205417481f6442f712a8f238394bdae
Depends-On: Icf4dfbb0be1336e81a2815fe17dbe8428d3df619
Closes-Bug: #1593759
2016-07-01 09:58:36 -04:00
Kenneth Giusti
b1b4cfce8c Refactor AMQP 1.0 command task to support timers
Refactors the Task() classes to support per-send timers and
consolidate message sending logic.

Change-Id: Icf4dfbb0be1336e81a2815fe17dbe8428d3df619
2016-06-14 10:12:51 -04:00
Kenneth Giusti
890ce0e543 Refactor link management to support link recovery
This refactors the driver to process send/subscribe requests as they
arrive from the application rather than blocking these requests until
the reply link is activated.  This allows recoverable failures
occurring at the connection or link level to be applied to the sender
retry count.  This patch also implements link re-connect for
subscriptions.

These changes are required to support a brokerless routed mesh
backend.

Change-Id: I14606de55e2e94d03d865fefe8c8c2327207ba5c
Closes-Bug: #1434540
2016-06-13 11:15:13 -04:00
Gevorg Davoian
39749c77a8 Remove logging from serialize_remote_exception
This patch removes log_failure argument from the function
serialize_remote_exception and from driver implementations
using it (because it is never used and always defaults to True)
and prevents error logging in this function (because these errors
are already logged by servers while processing incoming messages).

Change-Id: Ic01bb11d6c4f018a17f3219cdbd07ef4d30fa434
Closes-Bug: 1580352
2016-05-19 15:18:28 +03:00
Kenneth Giusti
e72a8d5bc4 Reorganize the AMQP 1.0 driver source files
This patch simply moves the existing AMQP 1.0 driver files into a
directory layout that is consistent with the other drivers.  The
driver unit tests are also moved to the proper test directory. There
are no changes in driver functionality or API.

Change-Id: I83a5d5433be8c3b317597100af69192ec5be81f1
Closes-Bug: #1579823
2016-05-13 14:04:37 -04:00