Add L3 agent extension API object

In L2 agent extensions, when the agent extension needed access to a
datastructure within the L2 agent, an agent extension API object was created.
This API object would be the interface permitting agent extensions to have
access to those objects internal to the L2 agent.

This change implements a similar agent extension API object for the L3 agent
extensions.  This is necessary to allow L3 agent extensions to have access to
the RouterInfo class, so that they can do lookups on it, for example
determining the namespace for a specific router.  Without this API object, the
L3 agent extension would not have access to this structure.

Co-Authored-By: Margaret Frances <margaret_frances@cable.comcast.com>

Partially-Implements: blueprint l3-agent-extensions

Change-Id: I85f89accbeefd820130335674fd56cb54f1449de

doc/source/devref/agent_extensions.rst

@@ -35,6 +35,27 @@ thereby allowing an extension to access resources internal to the agent. At
 layer 2, for instance, upon each port event the agent is then able to trigger a
 handle_port method in its extensions.
 
+Interactions with the agent API object are in the following order:
+
+#. The agent initializes the agent API object.
+#. The agent passes the agent API object into the extension manager.
+#. The manager passes the agent API object into each extension.
+#. An extension calls the new agent API object method to receive, for instance, bridge wrappers with cookies allocated.
+
+
+  +-----------+
+  | Agent API +--------------------------------------------------+
+  +-----+-----+                                                  |
+        |                                   +-----------+        |
+        |1                               +--+ Extension +--+     |
+        |                                |  +-----------+  |     |
+  +---+-+-+---+  2  +--------------+  3  |                 |  4  |
+  |   Agent   +-----+ Ext. manager +-----+--+   ....    +--+-----+
+  +-----------+     +--------------+     |                 |
+                                         |  +-----------+  |
+                                         +--+ Extension +--+
+                                            +-----------+
+
 Each extension is referenced through a stevedore entry point defined within a
 specific namespace. For example, L2 extensions are referenced through the
 neutron.agent.l2.extensions namespace.
@@ -71,5 +92,11 @@ this object.
 This agent API object is part of neutron's public interface for third parties.
 All changes to the interface will be managed in a backwards-compatible way.
 
-At the moment, only the L2 Open vSwitch agent provides an agent API object to
+At this time, on the L2 side, only the L2 Open vSwitch agent provides an agent
-extensions. See :doc:`L2 agent extensions <agent_extensions>`.
+API object to extensions. See :doc:`L2 agent extensions <l2_agent_extensions>`. 
+For L3, see :doc:`L3 agent extensions <l3_agent_extensions>`.
+
+The relevant modules are:
+
+* neutron.agent.l2.agent_extension_api
+* neutron.agent.l3.agent_extension_api

doc/source/devref/l2_agent_extensions.rst

@@ -33,7 +33,7 @@ Open vSwitch agent API
 * neutron.plugins.ml2.drivers.openvswitch.agent.ovs_agent_extension_api
 
 Open vSwitch agent API object includes two methods that return wrapped and
-hardened bridge objects with cookie values allocated for calling extensions.
+hardened bridge objects with cookie values allocated for calling extensions::
 
 #. request_int_br
 #. request_tun_br
@@ -41,29 +41,3 @@ hardened bridge objects with cookie values allocated for calling extensions.
 Bridge objects returned by those methods already have new default cookie values
 allocated for extension flows. All flow management methods (add_flow, mod_flow,
 ...) enforce those allocated cookies.
-
-Extensions are able to use those wrapped bridge objects to set their own flows,
-while the agent relies on the collection of those allocated values when
-cleaning up stale flows from the previous agent session::
-
-  +-----------+
-  | Agent API +--------------------------------------------------+
-  +-----+-----+                                                  |
-        |                                   +-----------+        |
-        |1                               +--+ Extension +--+     |
-        |                                |  +-----------+  |     |
-  +---+-+-+---+  2  +--------------+  3  |                 |  4  |
-  |   Agent   +-----+ Ext. manager +-----+--+   ....    +--+-----+
-  +-----------+     +--------------+     |                 |
-                                         |  +-----------+  |
-                                         +--+ Extension +--+
-                                            +-----------+
-
-Interactions with the agent API object are in the following order:
-
-#. The agent initializes the agent API object (bridges, other internal state).
-#. The agent passes the agent API object into the extension manager.
-#. The manager passes the agent API object into each extension.
-#. An extension calls the new agent API object method to receive bridge wrappers with cookies allocated.
-
-Call #4 also registers allocated cookies with the agent bridge objects.

doc/source/devref/l3_agent_extensions.rst

@@ -0,0 +1,38 @@
+..
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+
+      Convention for heading levels in Neutron devref:
+      =======  Heading 0 (reserved for the title in a document)
+      -------  Heading 1
+      ~~~~~~~  Heading 2
+      +++++++  Heading 3
+      '''''''  Heading 4
+      (Avoid deeper levels because they do not render well.)
+
+
+L3 agent extensions
+===================
+
+L3 agent extensions are part of a generalized L2/L3 extension framework. See
+:doc:`agent extensions <agent_extensions>`.
+
+L3 agent extension API
+~~~~~~~~~~~~~~~~~~~~~~
+
+The L3 agent extension API object includes several methods that expose
+router information to L3 agent extensions::
+
+#. get_routers_in_project
+#. get_router_hosting_port
+#. is_router_in_namespace

neutron/agent/l3/agent.py

@@ -32,6 +32,7 @@ from neutron.agent.l3 import dvr_edge_router as dvr_router
 from neutron.agent.l3 import dvr_local_router as dvr_local_router
 from neutron.agent.l3 import ha
 from neutron.agent.l3 import ha_router
+from neutron.agent.l3 import l3_agent_extension_api as l3_ext_api
 from neutron.agent.l3 import l3_agent_extensions_manager as l3_ext_manager
 from neutron.agent.l3 import legacy_router
 from neutron.agent.l3 import namespace_manager
@@ -369,10 +370,12 @@ class L3NATAgent(ha.AgentMixin,
 
     def init_extension_manager(self, connection):
         l3_ext_manager.register_opts(self.conf)
+        self.agent_api = l3_ext_api.L3AgentExtensionAPI(self.router_info)
         self.l3_ext_manager = (
             l3_ext_manager.L3AgentExtensionsManager(self.conf))
         self.l3_ext_manager.initialize(
-            connection, lib_const.L3_AGENT_MODE)
+            connection, lib_const.L3_AGENT_MODE,
+            self.agent_api)
 
     def router_deleted(self, context, router_id):
         """Deal with router deletion RPC message."""

neutron/agent/l3/l3_agent_extension.py

@@ -20,7 +20,7 @@ from neutron.agent import agent_extension
 
 
 @six.add_metaclass(abc.ABCMeta)
-class L3AgentCoreResourceExtension(agent_extension.AgentCoreResourceExtension):
+class L3AgentCoreResourceExtension(agent_extension.AgentExtension):
     """Define stable abstract interface for l3 agent extensions.
 
     An agent extension extends the agent core functionality.

neutron/agent/l3/l3_agent_extension_api.py

@@ -0,0 +1,67 @@
+# Copyright 2016 Comcast
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from neutron.agent.linux import ip_lib
+
+
+class L3AgentExtensionAPI(object):
+    '''Implements the Agent API for the L3 agent.
+
+    Extensions can gain access to this API by overriding the consume_api
+    method which has been added to the AgentCoreResourceExtension class.
+
+    The purpose of this API is to give L3 agent extensions access to the
+    agent's RouterInfo object.
+    '''
+
+    def __init__(self, router_info):
+        self._router_info = router_info
+
+    def _local_namespaces(self):
+        root_ip = ip_lib.IPWrapper()
+        local_ns_list = root_ip.get_namespaces()
+        return set(local_ns_list)
+
+    def get_router_hosting_port(self, port_id):
+        """Given a port_id, look up the router associated with that port in
+        local namespace. Returns a RouterInfo object (or None if the router
+        is not found).
+        """
+        if port_id:
+            local_namespaces = self._local_namespaces()
+            for router_info in self._router_info.values():
+                if router_info.ns_name in local_namespaces:
+                    for port in router_info.internal_ports:
+                        if port['id'] == port_id:
+                            return router_info
+
+    def get_routers_in_project(self, project_id):
+        """Given a project_id, return a list of routers that are all in
+        the given project.  Returns empty list if the project_id provided
+        doesn't evaluate to True.
+        """
+        if project_id:
+            return [ri for ri in self._router_info.values()
+                    if ri.router['project_id'] == project_id]
+        else:
+            return []
+
+    def is_router_in_namespace(self, router_id):
+        """Given a router_id, make sure that the router is in a local
+        namespace.
+        """
+        local_namespaces = self._local_namespaces()
+        ri = self._router_info.get(router_id)
+        return ri and ri.ns_name in local_namespaces

neutron/tests/unit/agent/l3/test_l3_agent_extension_api.py

@@ -0,0 +1,96 @@
+# Copyright 2016 Comcast
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+#
+
+import uuid
+
+import mock
+
+from neutron.agent.l3 import l3_agent_extension_api as l3_agent_api
+from neutron.agent.l3 import router_info
+from neutron.agent.linux import ip_lib
+from neutron.tests import base
+
+
+class TestL3AgentExtensionApi(base.BaseTestCase):
+
+    def _prepare_router_data(self, ports=None):
+        self.router_id = str(uuid.uuid4())
+        self.project_id = str(uuid.uuid4())
+        ri_kwargs = {'router': {'id': self.router_id,
+                                'project_id': self.project_id},
+                     'agent_conf': mock.ANY,
+                     'interface_driver': mock.ANY,
+                     'use_ipv6': mock.ANY}
+        ri = router_info.RouterInfo(self.router_id, **ri_kwargs)
+        ri.internal_ports = ports
+        return {ri.router_id: ri}, ri
+
+    def test_get_router_hosting_port_for_router_not_in_ns(self):
+        port_ids = [1, 2]
+        ports = [{'id': pid} for pid in port_ids]
+        router_info, ri = self._prepare_router_data(ports)
+
+        with mock.patch.object(ip_lib.IPWrapper,
+                               'get_namespaces') as mock_get_namespaces:
+
+            mock_get_namespaces.return_value = []
+            api_object = l3_agent_api.L3AgentExtensionAPI(router_info)
+            router = api_object.get_router_hosting_port(port_ids[0])
+
+        mock_get_namespaces.assert_called_once_with()
+        self.assertFalse(router)
+
+    def test_get_router_hosting_port_for_router_in_ns(self):
+        port_ids = [1, 2]
+        ports = [{'id': pid} for pid in port_ids]
+        router_info, ri = self._prepare_router_data(ports)
+
+        with mock.patch.object(ip_lib.IPWrapper,
+                               'get_namespaces') as mock_get_namespaces:
+            mock_get_namespaces.return_value = [ri.ns_name]
+            api_object = l3_agent_api.L3AgentExtensionAPI(router_info)
+            router = api_object.get_router_hosting_port(port_ids[0])
+            self.assertEqual(ri, router)
+
+    def test_get_routers_in_project(self):
+        router_info, ri = self._prepare_router_data()
+
+        with mock.patch.object(ip_lib.IPWrapper,
+                               'get_namespaces') as mock_get_namespaces:
+            mock_get_namespaces.return_value = [ri.ns_name]
+            api_object = l3_agent_api.L3AgentExtensionAPI(router_info)
+            routers = api_object.get_routers_in_project(self.project_id)
+            self.assertEqual([ri], routers)
+
+    def test_is_router_in_namespace_for_in_ns(self):
+        router_info, ri = self._prepare_router_data()
+
+        with mock.patch.object(ip_lib.IPWrapper,
+                               'get_namespaces') as mock_get_namespaces:
+            mock_get_namespaces.return_value = [ri.ns_name]
+            api_object = l3_agent_api.L3AgentExtensionAPI(router_info)
+            router_in_ns = api_object.is_router_in_namespace(ri.router_id)
+            self.assertTrue(router_in_ns)
+
+    def test_is_router_in_namespace_for_not_in_ns(self):
+        router_info, ri = self._prepare_router_data()
+
+        with mock.patch.object(ip_lib.IPWrapper,
+                               'get_namespaces') as mock_get_namespaces:
+            mock_get_namespaces.return_value = [str(uuid.uuid4())]
+            api_object = l3_agent_api.L3AgentExtensionAPI(router_info)
+            router_in_ns = api_object.is_router_in_namespace(ri.router_id)
+            self.assertFalse(router_in_ns)

releasenotes/notes/l3-agent-extensions-b348ff26aec0fe88.yaml

@@ -2,4 +2,8 @@
 features:
   - The neutron L3 agent now has the ability to load
     agent extensions, which allows other services to
-    integrate without additional agent changes.
+    integrate without additional agent changes.  An 
+    API for exposing the l3 agent's router info data 
+    to the extensions is also provided so that 
+    extensions can remain consistent with router
+    state.