From 5a52d8e4a0c5d4c246deb8851ef893df63ee0847 Mon Sep 17 00:00:00 2001 From: Pierre Riteau <pierre@stackhpc.com> Date: Wed, 19 Aug 2020 17:56:30 +0200 Subject: [PATCH] Add workaround for keystonemiddleware/neutron memcached issue There is an issue where keystonemiddleware connections to memcached from neutron-server grow beyond configured values [1], eventually reaching the maximum number of connections accepted by memcached servers. Other services do not appear to be affected by this issue. A workaround is to use the advanced memcached pool. Despite its documentation claiming to only work with Python 2, it appears to work fine on Python 3. [1] https://bugs.launchpad.net/keystonemiddleware/+bug/1883659 Change-Id: Ifbbc2022839cbc575848d830600241c61603c80b Closes-Bug: #1892210 --- ansible/roles/neutron/templates/neutron.conf.j2 | 3 +++ ...utron-memcached-connection-issue-84f5affa217b4612.yaml | 8 ++++++++ 2 files changed, 11 insertions(+) create mode 100644 releasenotes/notes/fix-neutron-memcached-connection-issue-84f5affa217b4612.yaml diff --git a/ansible/roles/neutron/templates/neutron.conf.j2 b/ansible/roles/neutron/templates/neutron.conf.j2 index 699465f8c2..115169ac53 100644 --- a/ansible/roles/neutron/templates/neutron.conf.j2 +++ b/ansible/roles/neutron/templates/neutron.conf.j2 @@ -115,6 +115,9 @@ memcache_secret_key = {{ memcache_secret_key }} memcached_servers = {% for host in groups['memcached'] %}{{ 'api' | kolla_address(host) | put_address_in_context('memcache') }}:{{ memcached_port }}{% if not loop.last %},{% endif %}{% endfor %} +# Workaround for https://bugs.launchpad.net/keystonemiddleware/+bug/1883659 +memcache_use_advanced_pool = True + [oslo_messaging_notifications] transport_url = {{ notify_transport_url }} {% if neutron_enabled_notification_topics %} diff --git a/releasenotes/notes/fix-neutron-memcached-connection-issue-84f5affa217b4612.yaml b/releasenotes/notes/fix-neutron-memcached-connection-issue-84f5affa217b4612.yaml new file mode 100644 index 0000000000..a7bc30f7f3 --- /dev/null +++ b/releasenotes/notes/fix-neutron-memcached-connection-issue-84f5affa217b4612.yaml @@ -0,0 +1,8 @@ +--- +fixes: + - | + Fixes `LP#1892210 + <https://bugs.launchpad.net/kolla-ansible/+bug/1892210>`__ where the number + of open connections to Memcached from ``neutron-server`` would grow over + time until reaching the maximum set by ``memcached_connection_limit`` (5000 + by default), at which point the Memcached instance would stop working.