415 Commits

Author SHA1 Message Date
Michal Nasiadka
06baa8f63f fluentd: Use labels for transition to v5
Depends-On: https://review.opendev.org/c/openstack/kolla/+/901508
Change-Id: I8c7d3de95d0f1f8e57a993b8c3417d90459e19be
2023-11-28 15:13:00 +00:00
Michal Arbet
b4dd2cc773 Add fluent config for letsencrypt
Depends-On: https://review.opendev.org/c/openstack/kolla/+/899773
Change-Id: I016c7a54ee2c43b6bbad0cb3eb3fe216f3d66238
2023-11-07 10:59:52 +01:00
James Kirsch
5581a28253 Add support for LetsEncrypt-managed certs
Add support for automatic provisioning and renewal of HTTPS
certificates via LetsEncrypt.

Spec is available at:
https://etherpad.opendev.org/p/kolla-ansible-letsencrypt-https

Depends-On: https://review.opendev.org/c/openstack/kolla/+/887347
Co-Authored-By: Michal Arbet <michal.arbet@ultimum.io>
Implements: blueprint letsencrypt-https
Change-Id: I35317ea0343f0db74ddc0e587862e95408e9e106
2023-11-07 10:59:51 +01:00
Zuul
8e302cc47d Merge "Improve grok pattern matching for apache logs" 2023-10-30 15:09:30 +00:00
Ivan Halomi
9a3f463345 Add support of podman deployment
This change adds basic deployment based on Podman
container manager as an alternative to Docker.

Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
Signed-off-by: Petr Tuma <p.tuma@partner.samsung.com>
Change-Id: I2b52964906ba8b19b8b1098717b9423ab954fa3d
Depends-On: Ie4b4c1cf8fe6e7ce41eaa703b423dedcb41e3afc
2023-10-20 17:51:52 +02:00
Laurie Fisher
63213c0e23 Improve grok pattern matching for apache logs
Apache grok pattern updated to match the size of response in bytes,
the time taken to serve the request and the user-agent that made
the request.

Change-Id: I8ce6e6e10bbf55ef8aa21f983f4a6c4b6dc24402
2023-10-13 14:03:56 +00:00
Zuul
d0262bdc3e Merge "toolbox: use script handling signals" 2023-09-18 18:44:38 +00:00
joek-office
b9f2ad94a7 Change regex for the multiline parser at the fluentd rabbitmq input
Related-Bug: #2015184

Closes-Bug: #2015184

Change-Id: I6ea0949d58e38e2cda41c9702001e7bd500f23f1
2023-08-30 12:25:00 +00:00
Michal Nasiadka
28e9f6e50c toolbox: use script handling signals
Since we use "sleep infinity" in kolla_toolbox container - stopping it
always ends up with exit code 143.

Let's use a wrapper script that handles signals correctly.

Closes-Bug: #2031692

Depends-On: https://review.opendev.org/c/openstack/kolla/+/891777
Change-Id: I84295b909d788ceb67bf09ef60fb0583108b111b
2023-08-17 19:41:21 +00:00
Zuul
0fc3418b8f Merge "Fix the fluentd regexp to collect the logs" 2023-05-24 13:50:25 +00:00
lishuliang
900c2d20bb Fix the fluentd regexp to collect the logs
The default value of logging_context_format_string and
logging_user_identity_format has changed [1][2][3].
This is to correct fluentd to use the same format.

[1]ebdee7f399
[2]7ed065f38d

[3]https://review.opendev.org/c/openstack/oslo.log/+/838185

Closes-Bug: #1872220

Change-Id: Ide0079ef4d28e3a3fa60288a74b936531c52ce60
2023-05-24 09:04:18 +00:00
Maksim Malchuk
df6472e0af Fix Bash variable expansion issues in openrc file
Sometimes passwords, URLs and other values of the variables can
contain special symbols, for example the dollar sign, using these
values can lead to unpredictable attempts of the variable expansions
in the Bash scripts, such as openrc file, so we need to use single
quotes for all variables values.

Change-Id: Ib2aabadd0ffd6a8dc2591245f29b4478e03d92fc
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2023-04-25 14:46:11 +03:00
Zuul
ff0fca8fdc Merge "Switch trove-api to wsgi running under apache." 2023-02-20 16:46:42 +00:00
wu.chunyang
303998e294 Switch trove-api to wsgi running under apache.
This change also adds support for Trove backend TLS.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/854744
Change-Id: I2acf7820b24b112b57b0c00a01f5c4b8cb85ce25
2023-02-02 01:22:59 +00:00
yangshaoxue
113b77c8cb Add skyline service
Support to deploy skyline by kolla-ansible.

Implements: blueprint skyline
Depends-On: https://review.opendev.org/c/openstack/kolla/+/826948

Change-Id: Ice5621491a432ba32138abd6f62d1f815cc219e0
2023-01-31 13:47:18 +08:00
Michal Nasiadka
673ca8c7e7 Drop skydive
Change-Id: I8855bd60c2fd77f33fb55d4123131a94327bd166
2023-01-05 14:55:53 +01:00
Michal Nasiadka
f128d19957 Remove kafka, storm, zookeeper
Their cleanup has been added to monasca cleanup command.

Change-Id: I19a846e2683ae70b33ca64d2aba7ac71eb724588
2022-12-08 06:50:15 +00:00
Zuul
113242c864 Merge "Replace ElasticSearch and Kibana with OpenSearch" 2022-12-01 14:38:51 +00:00
Michal Nasiadka
e1ec02eddf Replace ElasticSearch and Kibana with OpenSearch
This change replaces ElasticSearch with OpenSearch, and Kibana
with OpenSearch Dashboards. It migrates the data from ElasticSearch
to OpenSearch upon upgrade.

No TLS support is in this patch (will be a followup).

A replacement for ElasticSearch Curator will be added as a followup.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/830373

Co-authored-by: Doug Szumski <doug@stackhpc.com>
Co-authored-by: Kyle Dean <kyle@stackhpc.com>
Change-Id: Iab10ce7ea5d5f21a40b1f99b28e3290b7e9ce895
2022-12-01 10:27:50 +00:00
Doug Szumski
04d3bb3666 Monasca removal followup
A few minor fixes were noted in this review [1], and they
are addressed here.

TrivialFix

[1]: https://review.opendev.org/c/openstack/kolla-ansible/+/861392/

Change-Id: If30d9c2b48615dfb54edcb8d782c4c24b968ac4b
2022-11-18 14:55:04 +00:00
Doug Szumski
adb8f89a36 Remove support for deploying OpenStack Monasca
Kolla Ansible is switching to OpenSearch and is dropping support for
deploying ElasticSearch. This is because the final OSS release of
ElasticSearch has exceeded its end of life.

Monasca is affected because it uses both Logstash and ElasticSearch.
Whilst it may continue to work with OpenSearch, Logstash remains an
issue.

In the absence of any renewed interest in the project, we remove
support for deploying it. This helps to reduce the complexity
of log processing configuration in Kolla Ansible, freeing up
development time.

Change-Id: I6fc7842bcda18e417a3fd21c11e28979a470f1cf
2022-11-11 15:48:11 +00:00
Michal Arbet
6c9d1f9b55 Fix missing logrotate configuration for proxysql logs
This trivial patch is just adding missed logrotate
configuration for proxysql.

Closes-Bug: #1995248
Change-Id: I3ad88d03836930160b6db43a7cad63b34ffc62b0
2022-10-31 11:26:43 +01:00
Michal Nasiadka
1aac65de0c Fix issues introduced by ansible-lint 6.6.0
mainly jinja spacing and jinja[invalid] related

Change-Id: I6f52f2b0c1ef76de626657d79486d31e0f47f384
2022-09-21 14:34:54 +00:00
Stig Telfer
ffb4767c05 Increase the Fluentd request timeout for ES
Fluentd has a default timeout of 5s for flushing data to ElasticSearch.
If there is a significant backlog of unsent log messages, this timeout
can be exceeded, resulting in Fluentd failing to make further progress.

Raise the default timeout to 60s.

This patch adopts the configuration parameters previously proposed by
Krzysztof Klimonda.

Closes-Bug: #1983031
Closes-Bug: #1896611
Change-Id: I1aaab654a5a0752fccef2cfb8cc0bde4a0ee2562
2022-09-05 10:33:16 +00:00
Zuul
981699f554 Merge "Add clouds.yaml file and use it" 2022-08-09 13:14:55 +00:00
Monty Taylor
73a1812c58 Add clouds.yaml file and use it
clouds.yaml[0] is a richer way to express configuration for OpenStack
clouds. It's also fully supported by Ansible's OpenStack modules as
well as python-openstackclient and openstacksdk. It's the future - who
doesn't like the future?

Write a file using both the public (default) and the internal endpoints
for the admin user. Also, change all of the examples to reference it
and to get python-openstackclient to use it too.

[0] https://docs.openstack.org/openstacksdk/latest/user/guides/connect_from_config.html

Implements: blueprint use-clouds-yaml
Change-Id: I557d2e4975c7b3d3c713a556b9ba47af9567ce6e
2022-08-08 12:19:47 +00:00
Radosław Piliszek
72b63dfee7 Further Keystone-related cleanups
Per comments on [1].

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/843727

Change-Id: I60162b54bc06e158534d29311d4474b34750c64d
2022-06-20 08:40:03 +00:00
Radosław Piliszek
7ca9349b09 Do not use keystone_admin_url et al
Following up on [1].
The 3 variables are only introducing noise after we removed
the reliance on Keystone's admin port.

[1] I5099b08953789b280c915a6b7a22bdd4e3404076

Change-Id: I3f9dab93042799eda9174257e604fd1844684c1c
2022-05-28 18:19:01 +02:00
LinPeiWen
6503644140 fluentd: remove deprecated parameter
In fluentd v1.0, "format" became a deprecated parameter,
replaced by "<parse>"

Change-Id: I0825e81fcd90fbc7f64c8df3ea9ae199ad79294a
2022-04-16 09:36:13 +00:00
Zuul
ec4af6c90e Merge "Adds FluentD configuration for OpenvSwitch log matching" 2022-03-31 15:23:50 +00:00
Juan Pablo Suazo
ab7bb56df6 Adds FluentD configuration for OpenvSwitch log matching
Adds a new configuration file that provides fluentd with an appropiate regex to match with OpenvSwitch logs in both default files.

The regex is segmented with variable as to isolate the relevant parts of each log message.

Closes-Bug: #1965815
Signed-off-by: Juan Pablo Suazo <jsuazo@whitestack.com>
Change-Id: Ife83c50c048d517a5c8a5dee588f8f7846fcee00
2022-03-25 14:01:28 +00:00
Michal Nasiadka
8fe9872031 monasca: Remove monasca-grafana leftovers
In Xena [1] we removed Monasca Grafana service, but some components were left
to support cleanup operations.

[1]: https://review.opendev.org/c/openstack/kolla-ansible/+/788228

Change-Id: Iccc7bc3628bb7cbab1ac28f41c7b7dc7695894c6
2022-03-23 07:04:57 +00:00
jinyuanliu
3ccb176f13 ADD venus for kolla-ansible
This project [1] can provide a one-stop solution to log collection,
cleaning, indexing, analysis, alarm, visualization, report generation
and other needs, which involves helping operator or maintainer to
quickly solve retrieve problems, grasp the operational health of the
platform, and improve the level of platform management.

[1] https://wiki.openstack.org/wiki/Venus

Change-Id: If3562bbed6181002b76831bab54f863041c5a885
2022-03-17 20:35:08 +08:00
Zuul
02a3cbcde3 Merge "Make cron logfile minsize,maxsize configurable" 2022-03-08 16:33:27 +00:00
Juan Pablo Suazo
80ee3f2e5c Add openvswitch and prometheus to logrotate
Closes-Bug: #1961795

Change-Id: I5547cce5c389846ed216bb898b78e45b8f231e1e
2022-02-24 08:03:17 +00:00
Isaac Prior
b3e2fcc793 Fix fluentd v1 buffer syntax issue
Change-Id: I5b3ab3ab8153cda283dec772bf1393af0caf4137
Closes-Bug: 1919179
2022-02-11 11:33:38 +00:00
Michal Nasiadka
b97832dd4f Refactor fluentd syslog logging
Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: I75ca59d981bcd2dd51faa296ab0b4223a891f5cb
2022-02-11 11:33:38 +00:00
Zuul
d8b2c72fdf Merge "Fix log rotation for fluentd created files" 2022-01-28 12:24:37 +00:00
Zuul
f1c68d1d53 Merge "fluentd: remove legacy parsing config" 2022-01-26 17:07:26 +00:00
Zuul
cb69f7d379 Merge "Fluentd cleanup" 2022-01-25 16:22:21 +00:00
Zuul
d241a3dc8c Merge "fluentd: add modern parsing config" 2022-01-25 14:48:49 +00:00
Isaac Prior
79b59e2c0b Fix log rotation for fluentd created files
Overrides default fluentd buffer config to stop log files from using
datestamped filenames, allowing logrotate to manage them.

Closes-Bug: #1940118
Change-Id: I40c4e209470d21e0a02fd447fb628acfdae9fa9d
2022-01-25 12:02:20 +00:00
Zuul
b41268308b Merge "Delete the logrotate files because they do not have separate logs" 2022-01-25 10:54:53 +00:00
Zuul
9bbba35b2f Merge "Add logrotate configuration for placement service" 2022-01-11 16:02:51 +00:00
lixuehai
0fae876944 Add logrotate configuration for placement service
Closes-Bug: #1954723
Change-Id: I6ffc8caae1aef757d37c629fbc05bb129f80147b
2022-01-10 15:48:46 +08:00
Michal Arbet
f3d5425808 Make cron logfile minsize,maxsize configurable
This patch is moving minsize and maxsize logrotate
options to defaults and use them in template.

Change-Id: Ic2dfc747faedef8a7a4bc5e61db7c0cb5bf93e88
2021-12-31 09:44:08 +00:00
Radosław Piliszek
72221aa9f0 fluentd: remove legacy parsing config
After we remove the problematic, legacy "parser" plugin, the
config only needs the new, modern syntax.
This removes warnings that were logged.

Depends-On: https://review.opendev.org/c/openstack/kolla/+/823071
Change-Id: I33aa06de6ce88288769a8291ebd59e78e07cdbaf
2021-12-29 13:37:47 +00:00
Radosław Piliszek
1a3c4f1bbf fluentd: add modern parsing config
To satisfy the needs of the modern parser plugin.

Needed-By: https://review.opendev.org/c/openstack/kolla/+/823071
Co-Authored-By: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I2b748d0544c14bebefe8c62aa5aafaaa5371ce53
2021-12-28 17:51:26 +00:00
Mark Goddard
1fd1f131e7 Remove unused template wsgi_python.conf.j2
We stopped using this file in Queens
(https://review.opendev.org/494635), but the file was not removed at
that time.

Change-Id: Ibe5fb291e7c39965f5c4ff5ee4ea0bb1f8e6e9c2
Closes-Bug: #1840158
2021-12-28 14:37:33 +00:00
Radosław Piliszek
df2a29e483 Fluentd cleanup
We no longer have to support multiple fluentd versions and rely
on the workaround.

Change-Id: Id4083a9f07317fc04467e52176850f3909703169
2021-12-28 14:33:02 +00:00