From 36fb5bceabe08a982ebd52e4a8f005cd26fdf6b8 Mon Sep 17 00:00:00 2001 From: Arnaud Morin Date: Tue, 28 Feb 2023 07:25:09 +0100 Subject: [PATCH] Set default heartbeat_rate to 3 Kombu recommend to run heartbeat_check every seconds but we use a lock around the kombu connection so, to not lock to much this lock to most of the time do nothing except waiting the events drain, we start heartbeat_check and retrieve the server heartbeat packet only two times more than the minimum required for the heartbeat works: heartbeat_timeout / heartbeat_rate / 2.0 Because of this, we are not sending the heartbeat frames at correct intervals. E.G. If heartbeat_timeout=60 and rate=2, AMQP protocol expects to send a frame every 30sec. With the current heartbeat_check implementation, heartbeat_check will be called every: heartbeat_timeout / heartbeat_rate / 2.0 = 60 / 2 / 2.0 = 15 Which will result in the following frame flow: T+0 --> do nothing (60/2 > 0) T+15 --> do nothing (60/2 > 15) T+30 --> do nothing (60/2 > 30) T+45 --> send a frame (60/2 < 45) ... With heartbeat_rate=3, the heartbeat_check will be executed more often: heartbeat_timeout / heartbeat_rate / 2.0 = 60 / 3 / 2.0 = 10 Frame flow: T+0 --> do nothing (60/3 > 0) T+10 --> do nothing (60/3 > 10) T+20 --> do nothing (60/3 > 20) T+30 --> send a frame (60/3 < 30) ... Now we are sending the frame with correct intervals Closes-bug: #2008734 Signed-off-by: Arnaud Morin Change-Id: Ie646d254faf5e45ba46948212f4c9baf1ba7a1a8 --- oslo_messaging/_drivers/impl_rabbit.py | 2 +- releasenotes/notes/heartbeat-rate-3-7ada9edbccc11a3f.yaml | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) create mode 100644 releasenotes/notes/heartbeat-rate-3-7ada9edbccc11a3f.yaml diff --git a/oslo_messaging/_drivers/impl_rabbit.py b/oslo_messaging/_drivers/impl_rabbit.py index 50e0a66a8..1cd20aebe 100644 --- a/oslo_messaging/_drivers/impl_rabbit.py +++ b/oslo_messaging/_drivers/impl_rabbit.py @@ -209,7 +209,7 @@ rabbit_opts = [ "considered down if heartbeat's keep-alive fails " "(0 disables heartbeat)."), cfg.IntOpt('heartbeat_rate', - default=2, + default=3, help='How often times during the heartbeat_timeout_threshold ' 'we check the heartbeat.'), cfg.BoolOpt('direct_mandatory_flag', diff --git a/releasenotes/notes/heartbeat-rate-3-7ada9edbccc11a3f.yaml b/releasenotes/notes/heartbeat-rate-3-7ada9edbccc11a3f.yaml new file mode 100644 index 000000000..3efc51e0c --- /dev/null +++ b/releasenotes/notes/heartbeat-rate-3-7ada9edbccc11a3f.yaml @@ -0,0 +1,5 @@ +--- +fixes: + - | + Change heartbeat_rate default from 2 to 3 in order to send AMQP heartbeat + frames at correct interval