openstack/neutron
Code Issues Proposed changes

[doc] Follow-up logging service for firewall_group

Browse Source 
This patch updates the networking guide to include 'firewall_group'
as a new resource-type in 'Neutron Packet Logging Framework'

Change-Id: I94d779fcabc64417d4d561c659270501c0651a66
Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Partial-Bug: #1720727
This commit is contained in:
Kim Bao Long 2018-07-13 17:23:21 +07:00
parent 8f27c2a165
commit e82a17527d
2 changed files with 323 additions and 244 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

265
doc/source/admin/config-logging.rst
View File

@ -1,53 +1,58 @@
.. _config-logging: .. _config-logging:
=========================== ================================
Logging for security groups Neutron Packet Logging Framework
=========================== ================================
Logging is designed as a service plug-in that captures events for Packet logging service is designed as a Neutron plug-in that captures network
relevant resources (for example, security groups or firewalls) when they occur. packets for relevant resources (e.g. security group or firewall group) when the
registered events occur.
.. image:: figures/logging-framework.png
:width: 100%
:alt: Packet Logging Framework
Supported loggable resource types
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
From Rocky release, both of ``security_group`` and ``firewall_group`` are
supported as resource types in Neutron packet logging framework.
Supported logging resource types Service Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~
As of the Queens release, the ``security_group`` resource type is supported. To enable the logging service, follow the below steps.
#. On Neutron controller node, add ``log`` to ``service_plugins`` setting in
Configuration ``/etc/neutron/neutron.conf`` file. For example:
~~~~~~~~~~~~~
To enable the service, follow the steps below.
#. On Neutron server node:
#. Add the Logging service to the ``service_plugins`` setting in
``/etc/neutron/neutron.conf``. For example:
.. code-block:: none .. code-block:: none
service_plugins = router,metering,log service_plugins = router,metering,log
#. Add the Logging extension to the ``extensions`` setting in #. To enable logging service for ``security_group`` in Layer 2, add ``log`` to
``/etc/neutron/plugins/ml2/ml2_conf.ini``. For example: option ``extensions`` in section ``[agent]`` in ``/etc/neutron/plugins/ml2/ml2_conf.ini``
for controller node and in ``/etc/neutron/plugins/ml2/openvswitch_agent.ini``
for compute/network nodes. For example:
.. code-block:: ini .. code-block:: ini
[agent] [agent]
extensions = log extensions = log
#. On compute/network nodes: #. To enable logging service for ``firewall_group`` in Layer 3, add
``fwaas_v2_log`` to option ``extensions`` in section ``[AGENT]`` in
#. In ``/etc/neutron/plugins/ml2/openvswitch_agent.ini``, add ``log`` ``/etc/neutron/l3_agent.ini`` for network nodes. For example:
to the ``extensions`` setting in the ``[agent]`` section. For example:
.. code-block:: ini .. code-block:: ini
[agent] [AGENT]
extensions = log extensions = fwaas_v2,fwaas_v2_log
#. In ``/etc/neutron/plugins/ml2/openvswitch_agent.ini``, add configuration #. On compute/network nodes, add configuration for logging service to
options for logging feature in the ``[network_log]`` section. For example: ``[network_log]`` in ``/etc/neutron/plugins/ml2/openvswitch_agent.ini`` as
shown bellow:
.. code-block:: ini .. code-block:: ini
@ -58,42 +63,42 @@ To enable the service, follow the steps below.
In which, ``rate_limit`` is used to configure the maximum number of packets In which, ``rate_limit`` is used to configure the maximum number of packets
to be logged per second (packets per second). When a high rate triggers to be logged per second (packets per second). When a high rate triggers
``rate_limit``, logging queues packets to be logged. ``burst_limit`` is used ``rate_limit``, logging queues packets to be logged. ``burst_limit`` is
to configure the maximum of queued packets. And logged data can be stored used to configure the maximum of queued packets. And logged packets can be
anywhere by using ``local_output_log_base``. stored anywhere by using ``local_output_log_base``.
.. note:: .. note::
- Logging currently works with ``openvswitch`` firewall driver only. - It requires at least ``100`` for ``rate_limit`` and at least ``25``
- It requires at least 100 for ``rate_limit`` and at least 25 for ``burst_limit``. for ``burst_limit``.
- If ``rate_limit`` is unset, logging will log unlimited. - If ``rate_limit`` is unset, logging will log unlimited.
- If we don't specify ``local_output_log_base``, logged data will be stored - If we don't specify ``local_output_log_base``, logged packets will be
in system journal like ``/var/log/syslog``. stored in system journal like ``/var/log/syslog`` by default.
Trusted projects policy.json configuration Trusted projects policy.json configuration
------------------------------------------ ----------------------------------------------
With the default ``/etc/neutron/policy.json``, administrators must With the default ``/etc/neutron/policy.json``, administrators must set up
set up resource logging on behalf of the cloud projects. resource logging on behalf of the cloud projects.
If projects are trusted to administer their own resource logging in your cloud, If projects are trusted to administer their own loggable resources in their
neutron's file ``policy.json`` can be modified to allow this. cloud, neutron's policy file ``policy.json`` can be modified to allow this.
Modify ``/etc/neutron/policy.json`` policy entries as follows: Modify ``/etc/neutron/policy.json`` entries as follows:
.. code-block:: none .. code-block:: none
"get_loggable_resources": "rule:regular_user", "get_loggable_resources": "rule:regular_user",
"create_log": "rule:regular_user", "create_log": "rule:regular_user",
"get_log": "rule:regular_user",
"get_logs": "rule:regular_user",
"update_log": "rule:regular_user", "update_log": "rule:regular_user",
"delete_log": "rule:regular_user", "delete_log": "rule:regular_user",
"get_logs": "rule:regular_user",
"get_log": "rule:regular_user",
Operator workflow Service workflow for Operator
~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#. Confirm logging resources are supported: #. To check the loggable resources that are supported by framework:
.. code-block:: console .. code-block:: console
@ -102,19 +107,30 @@ Operator workflow
| Supported types | | Supported types |
+-----------------+ +-----------------+
| security_group | | security_group |
| firewall_group |
+-----------------+ +-----------------+
#. Create a logging resource with an appropriate resource type: .. note::
- In VM ports, logging for ``security_group`` in currently works with
``openvswitch`` firewall driver only. ``linuxbridge`` is under
development.
- Logging for ``firewall_group`` works on internal router ports only. VM
ports would be supported in the future.
#. Log creation:
* Create a logging resource with an appropriate resource type
.. code-block:: console .. code-block:: console
$ openstack network log create --resource-type security_group \ $ openstack network log create --resource-type security_group \
--description "Collecting all security events in project demo" \ --description "Collecting all security events" \
--enable --event ALL Log_Created --event ALL Log_Created
+-----------------+------------------------------------------------+ +-----------------+------------------------------------------------+
| Field | Value | | Field | Value |
+-----------------+------------------------------------------------+ +-----------------+------------------------------------------------+
| Description | Collecting all security events in project demo | | Description | Collecting all security events |
| Enabled | True | | Enabled | True |
| Event | ALL | | Event | ALL |
| ID | 8085c3e6-0fa2-4954-b5ce-ff6207931b6d | | ID | 8085c3e6-0fa2-4954-b5ce-ff6207931b6d |
@ -129,19 +145,45 @@ Operator workflow
| updated_at | 2017-07-05T02:56:43Z | | updated_at | 2017-07-05T02:56:43Z |
+-----------------+------------------------------------------------+ +-----------------+------------------------------------------------+
.. warning::
In the case of ``--resource`` and ``--target`` are not specified from the
request, these arguments will be assigned to ``ALL`` by default. Hence,
there is an enormous range of log events will be created.
* Create logging resource with a given resource (sg1 or fwg1)
.. code-block:: console
$ openstack network log create my-log --resource-type security_group --resource sg1
$ openstack network log create my-log --resource-type firewall_group --resource fwg1
* Create logging resource with a given target (portA)
.. code-block:: console
$ openstack network log create my-log --resource-type security_group --target portA
* Create logging resource for only the given target (portB) and the given
resource (sg1 or fwg1)
.. code-block:: console
$ openstack network log create my-log --resource-type security_group --target portB --resource sg1
$ openstack network log create my-log --resource-type firewall_group --target portB --resource fwg1
.. note:: .. note::
The ``Enabled`` field is set to ``True`` by default. If enabled, - The ``Enabled`` field is set to ``True`` by default. If enabled, logged
log information is written to the destination if configured in events are written to the destination if ``local_output_log_base`` is
``local_output_log_base`` or system journal like ``/var/log/syslog``. configured or ``/var/log/syslog`` in default.
- The ``Event`` field will be set to ``ALL`` if ``--event`` is not specified
from log creation request.
#. Enable/Disable log
Enable/Disable log We can ``enable`` or ``disable`` logging objects at runtime. It means that
------------------ it will apply to all registered ports with the logging object immediately.
We can enable or disable logging objects at runtime. It means that it will
apply to all attached ports with the logging object immediately.
For example: For example:
.. code-block:: console .. code-block:: console
@ -151,7 +193,7 @@ For example:
+-----------------+------------------------------------------------+ +-----------------+------------------------------------------------+
| Field | Value | | Field | Value |
+-----------------+------------------------------------------------+ +-----------------+------------------------------------------------+
| Description | Collecting all security events in project demo | | Description | Collecting all security events |
| Enabled | False | | Enabled | False |
| Event | ALL | | Event | ALL |
| ID | 8085c3e6-0fa2-4954-b5ce-ff6207931b6d | | ID | 8085c3e6-0fa2-4954-b5ce-ff6207931b6d |
@ -166,51 +208,49 @@ For example:
| updated_at | 2017-07-05T03:12:01Z | | updated_at | 2017-07-05T03:12:01Z |
+-----------------+------------------------------------------------+ +-----------------+------------------------------------------------+
Logged events description
~~~~~~~~~~~~~~~~~~~~~~~~~
Events collected description Currently, packet logging framework supports to collect ``ACCEPT`` or ``DROP``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ or both events related to registered resources. As mentioned above, Neutron
packet logging framework offers two loggable resources through the ``log``
service plug-in: ``security_group`` and ``firewall_group``.
Logging will collect ``ACCEPT`` or ``DROP`` or both events related to security group, The general characteristics of each event will be shown as the following:
with the following general characteristics:
* Log every ``DROP`` event: Every ``DROP`` security event will be generated when * Log every ``DROP`` event: Every ``DROP`` security events will be generated
an incoming or outgoing session is dropped, that is the new session is not when an incoming or outgoing session is blocked by the security groups or
allowed for the security group and because of that blocked. firewall groups
* Log an ``ACCEPT`` event: An ``ACCEPT`` security event will be generated for each * Log an ``ACCEPT`` event: The ``ACCEPT`` security event will be generated only
**NEW** incoming or outgoing session that is allowed by the ports security group. for each ``NEW`` incoming or outgoing session that is allowed by security
More details for the events follow below: groups or firewall groups. More details for the ``ACCEPT`` events are shown
as bellow:
* North/South ``ACCEPT``: For a North/South session there would be a single ``ACCEPT`` * North/South ``ACCEPT``: For a North/South session there would be a single
event irrespective of direction. ``ACCEPT`` event irrespective of direction.
* East/West ``ACCEPT``/``ACCEPT``: In an intra-project East/West session where the * East/West ``ACCEPT``/``ACCEPT``: In an intra-project East/West session
security group on the originating port allows the session and the security where the originating port allows the session and the destination port
group on the destination port allows the session, i.e. the traffic is allowed, allows the session, i.e. the traffic is allowed, there would be two
there would be two ``ACCEPT`` security events generated, one from the perspective ``ACCEPT`` security events generated, one from the perspective of the
of the originating port and one from the perspective of the destination port. originating port and one from the perspective of the destination port.
* East/West ``ACCEPT``/``DROP``: In an intra-project East/West session initiation * East/West ``ACCEPT``/``DROP``: In an intra-project East/West session
where the security group on the originating port allows the session and the initiation where the originating port allows the session and the
security group on the destination port does not allow the session there would destination port does not allow the session there would be ``ACCEPT``
be ``ACCEPT`` security events generated from the perspective of the originating security events generated from the perspective of the originating port and
port and ``DROP`` security events generated from the perspective of the ``DROP`` security events generated from the perspective of the destination
destination port. port.
General data requirements: The security event should include: #. The security events that are collected by security group should include:
* A timestamp of the flow.
* A status of the flow ``ACCEPT``/``DROP``. * A status of the flow ``ACCEPT``/``DROP``.
* An indication of the originator of the flow, e.g which project or log resource * An indication of the originator of the flow, e.g which project or log resource
generated the event. generated the events.
* A timestamp of the flow.
* An identifier of the associated instance interface (neutron port id). * An identifier of the associated instance interface (neutron port id).
* An identifier of the matching security group rule. * A layer 2, 3 and 4 information (mac, address, port, protocol, etc).
* A layer 3 and 4 information (address, port, protocol, etc).
.. note::
No other extraneous events are generated within the security event logs,
e.g. no debugging data, etc.
* Security event record format: * Security event record format:
@ -242,3 +282,42 @@ General data requirements: The security event should include:
TCPOptionSACKPermitted(kind=4,length=2), TCPOptionTimestamps(kind=8,length=10,ts_ecr=0,ts_val=196418896), TCPOptionSACKPermitted(kind=4,length=2), TCPOptionTimestamps(kind=8,length=10,ts_ecr=0,ts_val=196418896),
TCPOptionNoOperation(kind=1,length=1), TCPOptionWindowScale(kind=3,length=3,shift_cnt=3)], TCPOptionNoOperation(kind=1,length=1), TCPOptionWindowScale(kind=3,length=3,shift_cnt=3)],
seq=3284890090,src_port=47825,urgent=0,window_size=14600) seq=3284890090,src_port=47825,urgent=0,window_size=14600)
#. The events that are collected by firewall group should include:
* A timestamp of the flow.
* A status of the flow ``ACCEPT``/``DROP``.
* The identifier of log objects that are collecting this event
* An identifier of the associated instance interface (neutron port id).
* A layer 2, 3 and 4 information (mac, address, port, protocol, etc).
* Security event record format:
* Logged data of an ``ACCEPT`` event would look like:
.. code-block:: console
Jul 26 14:46:20:
action=ACCEPT, log_resource_ids=[u'2e030f3a-e93d-4a76-bc60-1d11c0f6561b'], port=9882c485-b808-4a34-a3fb-b537642c66b2
pkt=ethernet(dst='fa:16:3e:8f:47:c5',ethertype=2048,src='fa:16:3e:1b:3e:67')
ipv4(csum=47423,dst='10.10.1.16',flags=2,header_length=5,identification=27969,offset=0,option=None,proto=1,src='10.10.0.5',tos=0,total_length=84,ttl=63,version=4)
icmp(code=0,csum=41376,data=echo(data='\xe5\xf2\xfej\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00',id=29185,seq=0),type=8)
* Logged data of a ``DROP`` event:
.. code-block:: console
Jul 26 14:51:20:
action=DROP, log_resource_ids=[u'2e030f3a-e93d-4a76-bc60-1d11c0f6561b'], port=9882c485-b808-4a34-a3fb-b537642c66b2
pkt=ethernet(dst='fa:16:3e:32:7d:ff',ethertype=2048,src='fa:16:3e:28:83:51')
ipv4(csum=17518,dst='10.10.0.5',flags=2,header_length=5,identification=57874,offset=0,option=None,proto=1,src='10.10.1.16',tos=0,total_length=84,ttl=63,version=4)
icmp(code=0,csum=23772,data=echo(data='\x8a\xa0\xac|\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00',id=25601,seq=5),type=8)
.. note::
No other extraneous events are generated within the security event logs,
e.g. no debugging data, etc.

BIN
doc/source/admin/figures/logging-framework.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 68 KiB