In order to orchestrate smooth transition to fluentd 0.14.x
aka 1.0 stable branch aka td-agent 3
from td-agent repository - use image labels (fluentd_version
and fluentd_binary).
Depends-On: https://review.opendev.org/676411
Change-Id: Iab8518c34ef876056c6abcdb5f2e9fc9f1f7dbdd
Currently, we have a lot of logic for checking if a handler should run,
depending on whether config files have changed and whether the
container configuration has changed. As rm_work pointed out during
the recent haproxy refactor, these conditionals are typically
unnecessary - we can rely on Ansible's handler notification system
to only trigger handlers when they need to run. This removes a lot
of error prone code.
This patch removes conditional handler logic for all services. It is
important to ensure that we no longer trigger handlers when unnecessary,
because without these checks in place it will trigger a restart of the
containers.
Implements: blueprint simplify-handlers
Change-Id: I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
We don't add extra volumes support for all services in patch [1].
In order to unify the management of the volume, so we need add extra volumes
support for these services.
[1] 12ff28a693
Change-Id: Ie148accdd8e6c60df6b521d55bda12b850c0d255
Partially-Implements: blueprint support-extra-volumes
Signed-off-by: ZijianGuo <guozijn@gmail.com>
This change formats internal Fluent logs in a similar way to other
logs. It makes it easier for a user to identify issues with Fluent
parsing logs. Any failure to parse a log will be ingested into the
logging framework and can easily be located by searching for
'pattern not match' or by filtering for Fluent log warnings.
Change-Id: Iea6d12c07a2f4152f2038d3de2ef589479b3332b
Kolla service logs which don't match a Fluentd rewriterule get dropped.
This change prevents that by tagging them with 'unmatched'.
Change-Id: I0a2484d878d5c86977fb232a57c52f874ca7a34c
Monasca Python service logs prior to this change were being dropped
due to missing entries in the Fluent record_transformer config file.
This change adds support for ingesting those logs, and explicitly
removes support for ingesting Monasca Log API logs to reduce the risk
of feedback, for example if debug logging is turned on in the Monasca
Log API.
Change-Id: I9e3436a8f946873867900eed5ff0643d84584358
Presently, errors can appear in Fluentd and Monasca Log API logs due
to log output from some Monasca services, which do not use Oslo log,
being processed alongside other OpenStack logs which do.
This change parses these log files separately to prevent these errors.
Change-Id: Ie3cbb51424989b01727b5ebaaeba032767073462
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:
* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts
It also adds become for 'command' tasks that use docker CLI.
Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
Qinling is an OpenStack project to provide "Function as a Service".
This project aims to provide a platform to support serverless functions.
Change-Id: I239a0130f8c8b061b531dab530d65172b0914d7c
Implements: blueprint ansible-qinling-support
Story: 2005760
Task: 33468
Add options for configuring TLS and authentication for elasticsearch
connections in in fluentd.
Change-Id: I936adc2aeaa3c87081be1c44aa0221caf2124e23
Closes-Bug: #1831078
Let the Monasca Fluentd output plugin handle mapping of the log
message.
Change-Id: I4a74a91b9b38d5c172397a7e7204e626bcedcfac
Closes-Bug: #1830184
Depends-On: https://review.opendev.org/#/c/660988/
Several config file permissions are incorrect on the host. In general,
files should be 0660, and directories and executables 0770.
Change-Id: Id276ac1864f280554e98b937f2845bb424d521de
Closes-Bug: #1821579
The message field for these format files wasn't renamed when
the input field was renamed. This problem was masked by Monasca
which renames the message field back to 'message' for the Fluentd
Monasca output plugin.
Change-Id: I2674958b1e64efddbca4765fefa55431261648e6
Closes-Bug: #1819168
In some scenarios it may be useful to perform custom formatting of logs
before forwarding them. For example, the JSON formatter plugin can be
used to convert an event to JSON.
Change-Id: I3dd9240c5910a9477456283b392edc9566882dcd
Because kolla-ansible not have cyborg so should add it.
Implements: blueprint add-cyborg-to-kolla-ansible
Depend-On: I497e67e3a754fccfd2ef5a82f13ccfaf890a6fcd
Change-Id: I6f7ae86f855c5c64697607356d0ff3161f91b239
We're duplicating code to build the keystone URLs in nearly every
config, where we've already done it in group_vars. Replace the
redundancy with a variable that does the same thing.
Change-Id: I207d77870e2535c1cdcbc5eaf704f0448ac85a7a
For other architectures, fluentd is not installed using the
td-agent package.
Change-Id: I508bf16a54dcadac4b5cba03e184b4f620a2b395
Depends-On: https://review.openstack.org/638692
The path /var/lib/docker/volumes/kolla_logs/_data/ is too long
shorter log path will help to debug from log.
The volume path is compatible with docker-engine and docker-ce.
Change-Id: I9195d5f24d938f5060fe748aac3ae58c79ec5abf
Vitrage Collector service has been removed from Vitrage in change:
Ie713456b2df96e24d0b15d2362a666162bfb4300.
Change-Id: I45023940c1d2573bfed49d4ce3fac16ed2d559e4
Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>
Co-Authored-By: Kien Nguyen <kiennt65@viettel.com.vn>
OpenDaylight logs have different format than openstack,
is a karaf log with java error traces.
This PS add required config to make fluentd parse properly
ODL logs.
Change-Id: I34fb96c8a424679b3b618f2ff6a840b8dc165bec
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.
Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
