fluentd: rework openstack logs match

Add global_request_id according to oslo.log defaults [1],
also support lines like:

2024-12-23 20:48:29.905 2 ERROR oslo_service.periodic_task ['Traceback (most recent call last):\\n'

[1]: https://docs.openstack.org/oslo.log/latest/configuration/#DEFAULT.logging_context_format_string

Closes-Bug: #2044370

Change-Id: I80c3e20de0b7503de6331b55879e85892323b3d6
This commit is contained in:
Michal Nasiadka
2024-12-27 16:40:55 +01:00
parent 6d553b3d79
commit 5a3fb2b4de
2 changed files with 8 additions and 1 deletions

View File

@@ -16,7 +16,7 @@
<parse>
@type multiline
format_firstline /^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3} \d+ \S+ \S+ \[.*\]/
format1 /^(?<Timestamp>\S+ \S+) (?<Pid>\d+) (?<log_level>\S+) (?<python_module>\S+) \[(\S+ req-)?((?<request_id>\S+) (?<user_id>\S+) (?<tenant_id>\S+) (?<domain_id>\S+) (?<system_scope>\S+) (?<user_domain>\S+) (?<project_domain>\S+)|-)\] (?<Payload>.*)?$/
format1 /^(?<Timestamp>\S+ \S+) (?<Pid>\d+) (?<log_level>\S+) (?<python_module>\S+) ((?:\[)(?:None|req-)(?<global_request_id>\S+)? (?:None|req-)(?<request_id>\S+) (?<user_id>\S+) (?<tenant_id>\S+) (?<domain_id>\S+) (?<system_scope>\S+) (?<user_domain>\S+) (?<project_domain>\S+)(?:\]))?(?<Payload>.*)?$/
time_key Timestamp
keep_time_key true
time_format %F %T.%L

View File

@@ -0,0 +1,7 @@
---
fixes:
- |
Fixes cases when fluentd parser fails on Python traceback.
OpenStack services regex has been reworked to include both
global_request_id and handling cases with Python traceback.
`LP#2044370 <https://launchpad.net/bugs/2044370>`_