393 Commits

Author SHA1 Message Date
Michal Nasiadka
1aac65de0c Fix issues introduced by ansible-lint 6.6.0
mainly jinja spacing and jinja[invalid] related

Change-Id: I6f52f2b0c1ef76de626657d79486d31e0f47f384
2022-09-21 14:34:54 +00:00
Stig Telfer
ffb4767c05 Increase the Fluentd request timeout for ES
Fluentd has a default timeout of 5s for flushing data to ElasticSearch.
If there is a significant backlog of unsent log messages, this timeout
can be exceeded, resulting in Fluentd failing to make further progress.

Raise the default timeout to 60s.

This patch adopts the configuration parameters previously proposed by
Krzysztof Klimonda.

Closes-Bug: #1983031
Closes-Bug: #1896611
Change-Id: I1aaab654a5a0752fccef2cfb8cc0bde4a0ee2562
2022-09-05 10:33:16 +00:00
Zuul
981699f554 Merge "Add clouds.yaml file and use it" 2022-08-09 13:14:55 +00:00
Monty Taylor
73a1812c58 Add clouds.yaml file and use it
clouds.yaml[0] is a richer way to express configuration for OpenStack
clouds. It's also fully supported by Ansible's OpenStack modules as
well as python-openstackclient and openstacksdk. It's the future - who
doesn't like the future?

Write a file using both the public (default) and the internal endpoints
for the admin user. Also, change all of the examples to reference it
and to get python-openstackclient to use it too.

[0] https://docs.openstack.org/openstacksdk/latest/user/guides/connect_from_config.html

Implements: blueprint use-clouds-yaml
Change-Id: I557d2e4975c7b3d3c713a556b9ba47af9567ce6e
2022-08-08 12:19:47 +00:00
Radosław Piliszek
72b63dfee7 Further Keystone-related cleanups
Per comments on [1].

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/843727

Change-Id: I60162b54bc06e158534d29311d4474b34750c64d
2022-06-20 08:40:03 +00:00
Radosław Piliszek
7ca9349b09 Do not use keystone_admin_url et al
Following up on [1].
The 3 variables are only introducing noise after we removed
the reliance on Keystone's admin port.

[1] I5099b08953789b280c915a6b7a22bdd4e3404076

Change-Id: I3f9dab93042799eda9174257e604fd1844684c1c
2022-05-28 18:19:01 +02:00
LinPeiWen
6503644140 fluentd: remove deprecated parameter
In fluentd v1.0, "format" became a deprecated parameter,
replaced by "<parse>"

Change-Id: I0825e81fcd90fbc7f64c8df3ea9ae199ad79294a
2022-04-16 09:36:13 +00:00
Zuul
ec4af6c90e Merge "Adds FluentD configuration for OpenvSwitch log matching" 2022-03-31 15:23:50 +00:00
Juan Pablo Suazo
ab7bb56df6 Adds FluentD configuration for OpenvSwitch log matching
Adds a new configuration file that provides fluentd with an appropiate regex to match with OpenvSwitch logs in both default files.

The regex is segmented with variable as to isolate the relevant parts of each log message.

Closes-Bug: #1965815
Signed-off-by: Juan Pablo Suazo <jsuazo@whitestack.com>
Change-Id: Ife83c50c048d517a5c8a5dee588f8f7846fcee00
2022-03-25 14:01:28 +00:00
Michal Nasiadka
8fe9872031 monasca: Remove monasca-grafana leftovers
In Xena [1] we removed Monasca Grafana service, but some components were left
to support cleanup operations.

[1]: https://review.opendev.org/c/openstack/kolla-ansible/+/788228

Change-Id: Iccc7bc3628bb7cbab1ac28f41c7b7dc7695894c6
2022-03-23 07:04:57 +00:00
jinyuanliu
3ccb176f13 ADD venus for kolla-ansible
This project [1] can provide a one-stop solution to log collection,
cleaning, indexing, analysis, alarm, visualization, report generation
and other needs, which involves helping operator or maintainer to
quickly solve retrieve problems, grasp the operational health of the
platform, and improve the level of platform management.

[1] https://wiki.openstack.org/wiki/Venus

Change-Id: If3562bbed6181002b76831bab54f863041c5a885
2022-03-17 20:35:08 +08:00
Zuul
02a3cbcde3 Merge "Make cron logfile minsize,maxsize configurable" 2022-03-08 16:33:27 +00:00
Juan Pablo Suazo
80ee3f2e5c Add openvswitch and prometheus to logrotate
Closes-Bug: #1961795

Change-Id: I5547cce5c389846ed216bb898b78e45b8f231e1e
2022-02-24 08:03:17 +00:00
Isaac Prior
b3e2fcc793 Fix fluentd v1 buffer syntax issue
Change-Id: I5b3ab3ab8153cda283dec772bf1393af0caf4137
Closes-Bug: 1919179
2022-02-11 11:33:38 +00:00
Michal Nasiadka
b97832dd4f Refactor fluentd syslog logging
Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: I75ca59d981bcd2dd51faa296ab0b4223a891f5cb
2022-02-11 11:33:38 +00:00
Zuul
d8b2c72fdf Merge "Fix log rotation for fluentd created files" 2022-01-28 12:24:37 +00:00
Zuul
f1c68d1d53 Merge "fluentd: remove legacy parsing config" 2022-01-26 17:07:26 +00:00
Zuul
cb69f7d379 Merge "Fluentd cleanup" 2022-01-25 16:22:21 +00:00
Zuul
d241a3dc8c Merge "fluentd: add modern parsing config" 2022-01-25 14:48:49 +00:00
Isaac Prior
79b59e2c0b Fix log rotation for fluentd created files
Overrides default fluentd buffer config to stop log files from using
datestamped filenames, allowing logrotate to manage them.

Closes-Bug: #1940118
Change-Id: I40c4e209470d21e0a02fd447fb628acfdae9fa9d
2022-01-25 12:02:20 +00:00
Zuul
b41268308b Merge "Delete the logrotate files because they do not have separate logs" 2022-01-25 10:54:53 +00:00
Zuul
9bbba35b2f Merge "Add logrotate configuration for placement service" 2022-01-11 16:02:51 +00:00
lixuehai
0fae876944 Add logrotate configuration for placement service
Closes-Bug: #1954723
Change-Id: I6ffc8caae1aef757d37c629fbc05bb129f80147b
2022-01-10 15:48:46 +08:00
Michal Arbet
f3d5425808 Make cron logfile minsize,maxsize configurable
This patch is moving minsize and maxsize logrotate
options to defaults and use them in template.

Change-Id: Ic2dfc747faedef8a7a4bc5e61db7c0cb5bf93e88
2021-12-31 09:44:08 +00:00
Radosław Piliszek
72221aa9f0 fluentd: remove legacy parsing config
After we remove the problematic, legacy "parser" plugin, the
config only needs the new, modern syntax.
This removes warnings that were logged.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/823071
Change-Id: I33aa06de6ce88288769a8291ebd59e78e07cdbaf
2021-12-29 13:37:47 +00:00
Radosław Piliszek
1a3c4f1bbf fluentd: add modern parsing config
To satisfy the needs of the modern parser plugin.

Needed-By: https://review.opendev.org/c/openstack/kolla/+/823071
Co-Authored-By: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I2b748d0544c14bebefe8c62aa5aafaaa5371ce53
2021-12-28 17:51:26 +00:00
Mark Goddard
1fd1f131e7 Remove unused template wsgi_python.conf.j2
We stopped using this file in Queens
(https://review.opendev.org/494635), but the file was not removed at
that time.

Change-Id: Ibe5fb291e7c39965f5c4ff5ee4ea0bb1f8e6e9c2
Closes-Bug: #1840158
2021-12-28 14:37:33 +00:00
Radosław Piliszek
df2a29e483 Fluentd cleanup
We no longer have to support multiple fluentd versions and rely
on the workaround.

Change-Id: Id4083a9f07317fc04467e52176850f3909703169
2021-12-28 14:33:02 +00:00
likui
867f1ddaea Delete the logrotate files because they do not have separate logs
Change-Id: I67995073b92d6ea9ccc208ae96e8a6045369b801
2021-12-23 16:15:42 +08:00
likui
f17bd2db77 Add logrotate to libvirt service
Change-Id: I5d6af3707c3de4466699f1a0e6ff2f2e2fe4e123
2021-12-20 19:44:49 +08:00
Maksim Malchuk
d559e2c39d Update dnsmasq logging
This change adds the dnsmasq.log for the ironic-dnsmasq container and
also enables more verbose logging when debug logging enabled.
This can be triggered globbaly via 'openstack_logging_debug' or per
service via 'ironic_logging_debug' or 'neutron_logging_debug'.

Change-Id: I0e6b089beb88827effbcc365625eb2df902f5470
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2021-10-12 22:30:42 +03:00
Zuul
61b98f6899 Merge "Do not create haproxy and swift log dirs needlessly" 2021-10-06 14:55:05 +00:00
wu.chunyang
1f71df1a8b Remove chrony role from kolla
chrony is not supported in Xena cycle, remove it from kolla

Moved tasks from chrony role to chrony-cleanup.yml playbook to avoid a
vestigial chrony role.

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: I5a730d55afb49d517c85aeb9208188c81e2c84cf
2021-09-30 18:56:14 +02:00
Radosław Piliszek
2c6bc0bd1a Do not create haproxy and swift log dirs needlessly
Closes-Bug: #1945070
Change-Id: I1b2a82b57cb9884b6c3c3ad07f6449ae29042a3d
2021-09-27 16:01:47 +00:00
Zuul
a1a22b3a89 Merge "toolbox: Allow different users logging to ansible.log" 2021-09-09 15:55:27 +00:00
Michał Nasiadka
24e6a6ced0 toolbox: Allow different users logging to ansible.log
Currently only operations done with default kolla_toolbox user are logged
to /var/log/kolla/ansible.log.

In order to fix logging, permissions to ansible.log must allow writing
for other users in kolla group - and then a separate patch will follow
to make custom ansible.cfg file usable by other toolbox users.

Partial-Bug: #1942846
Change-Id: I1be60ac7647b1a838e97f05f15ba5f0e39e8ae3c
2021-09-07 14:12:11 +02:00
Zuul
c48469d2ac Merge "Allow override of rabbitmq config in kolla toolbox" 2021-09-07 10:34:54 +00:00
Zuul
39931aedb4 Merge "Fix kolla-toolbox with IPv6 and disabled RabbitMQ" 2021-09-07 10:34:52 +00:00
Zuul
83c5d95b47 Merge "Support monitoring Fluentd with Prometheus" 2021-08-27 09:34:12 +00:00
Jan Horstmann
f8e3e169c3 Allow override of rabbitmq config in kolla toolbox
Certain overrides for rabbitmq may need to be set for `rabbitmqctl` in
kolla-toolbox aswell.
This commit allows to override `rabbitmq-env.conf` and `erl_inetrc` in
kolla-toolbox.

Change-Id: Idef6adcf9700f75a2db503444a8de093ee21a9c5
2021-08-13 12:57:16 +00:00
Radosław Piliszek
13200ace33 Fix kolla-toolbox with IPv6 and disabled RabbitMQ
Closes-Bug: #1939883
Change-Id: Ica311acba445cccac1d20757ced6f15a064ebcaa
2021-08-13 12:47:41 +00:00
Zuul
578ca1d647 Merge "Add suppress_type_name to stop warnings with ES7" 2021-08-10 16:26:11 +00:00
Doug Szumski
b692ce7af1 Support monitoring Fluentd with Prometheus
This patch adds support for integrating Prometheus with Fluentd.
This can be used to extract useful information about the status
of Fluentd, such as output buffer capacity and logging rate,
and also to extract metrics from logs via custom Fluentd
configuration. More information can be found here in [1].

[1] https://docs.fluentd.org/monitoring-fluentd/monitoring-prometheus

Change-Id: I233d6dd744848ef1f1589a462dbf272ed0f3aaae
2021-08-09 10:12:20 +01:00
Zuul
295c69b5ee Merge "Remove tempest role" 2021-08-06 14:04:55 +00:00
Zuul
e7c39aa419 Merge "Robustify fluentd output tag matching" 2021-07-19 11:16:46 +00:00
wu.chunyang
5261998467 Remove tempest role
Remove tempest role as planned

Change-Id: If3cf073e88c83f670c867a49afe48845f9e81008
2021-07-07 21:58:39 +08:00
Mark Goddard
ade5bfa302 Use ansible_facts to reference facts
By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.

This change updates all references to Ansible facts within Kolla Ansible
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.

This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.

[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars

Change-Id: I7e9d5c9b8b9164d4aee3abb4e37c8f28d98ff5d1
Partially-Implements: blueprint performance-improvements
2021-06-23 10:38:06 +01:00
Zuul
5665d5e619 Merge "Stop fluentd deprecation warnings of type vs @type" 2021-06-20 11:46:30 +00:00
Zuul
0d31dd9f1f Merge "Fix parsing of infra.mariadb.xinetd logs" 2021-06-20 11:00:56 +00:00
Zuul
754206477c Merge "Remove rally deployment" 2021-06-20 10:55:30 +00:00