153 Commits

Author SHA1 Message Date
Zuul
a396284a83 Merge "Fix AlertManager's external web url" 2022-09-09 13:17:03 +00:00
Piotr Parczewski
61ff6f811a Fix AlertManager's external web url
Remove hard-coded internal address; introduce variable to control
external web url.

Closes-bug: #1972817
Change-Id: Ib834a9f8b4a0238960dca65b2ebc1da840cec626
2022-09-09 10:05:07 +00:00
Zuul
0411ff3bb4 Merge "Allow exposing OpenStack exporter via HAProxy" 2022-09-07 09:10:14 +00:00
Franco Mariotti
7219279215 Allow exposing OpenStack exporter via HAProxy
Signed-off-by: Franco Mariotti <fmariotti@whitestack.com>
Change-Id: Ie151cd97d3e0ba3bfec9e95a5b8bdfef0b54806c
2022-08-31 13:29:02 -03:00
Zuul
5eb81ca2a9 Merge "Fix creation of prometheus user and grant permissions" 2022-08-30 17:18:36 +00:00
Michal Arbet
c5500bdfd2 Fix creation of prometheus user and grant permissions
Prometheus is creating user and granting permissions
to database from which is gathering metrics. This
process is different when haproxy/proxysql is used.

Proxysql:

  - kolla-ansible should use root_shard_ID user to connect
    to ProxySQL endpoint and it is routed to proper shard.

Haproxy:

  - kolla-ansible should use root user to connect to HAProxy
    endpoint and that's all.

If proxysql is not used, mariadb role will not create user
shard_root_ID user in bootstrap (from my perspective of view
it should), and therefore it will fail when HAProxy is used.

This patch is just fixing user to connect.

Change-Id: Icd07807b2c404eb4d3f398879639b17f1e7949c2
2022-08-30 15:37:04 +02:00
Pierre Riteau
b8928ce54b Switch Prometheus Alertmanager to active/passive
Closes-Bug: #1987866
Change-Id: Iaf352a15b9e6c9607e0d33c803c132d9267ca727
2022-08-29 18:25:52 +02:00
Pierre Riteau
f7b1221032 Use active_passive parameter for existing services
MariaDB is left unchanged because its custom_member_list uses a
different group (mariadb_default_database_shard_hosts).

Change-Id: Icefd5a3d02ae4dfeb27401696c35ca2c38e203d3
2022-08-29 14:38:02 +02:00
Piotr Parczewski
09adcf0181 Add prometheus-msteams to get container facts loop
A follow up patch to I563f2438b3cb0895606b029b5269ce2e50c413e3

Change-Id: I1e4c5db46413668d4b5df2f2dcedc5d9aaecd63a
2022-08-12 07:34:27 +00:00
Zuul
6cab1fcbd0 Merge "Add support for deploying prometheus-msteams" 2022-08-10 17:14:13 +00:00
Pierre Riteau
c1155a2879 Add support for deploying prometheus-msteams
This can be used to forward Prometheus Alertmanager notifications to
Microsoft Teams.

Change-Id: I563f2438b3cb0895606b029b5269ce2e50c413e3
Depends-On: https://review.opendev.org/c/openstack/kolla/+/812678
2022-08-10 13:21:41 +02:00
Michal Arbet
4838591c6c Add loadbalancer-config role and wrap haproxy-config role inside
This patch adds loadbalancer-config role
which is "wrapper" around haproxy-config
and proxysql-config role which will be added
in follow-up patches.

Change-Id: I64d41507317081e1860a94b9481a85c8d400797d
2022-08-09 12:15:49 +02:00
Michal Arbet
baad47ac61 Edit services roles to support database sharding
Depends-On: https://review.opendev.org/c/openstack/kolla/+/769385
Depends-On: https://review.opendev.org/c/openstack/kolla/+/765781

Change-Id: I3c4182a6556dafd2c936eaab109a068674058fca
2022-08-09 12:15:26 +02:00
Michal Nasiadka
dcf5a8b65f Fix var-spacing
ansible-lint introduced var-spacing - let's fix our code.

Change-Id: I0d8aaf3c522a5a6a5495032f6dbed8a2be0251f0
2022-07-25 22:15:15 +02:00
zhangmeng
41a9402910 make "external_labels" optional on prometheus.yml
Closes-bug: 1944699
Change-Id: I6d0bb3b88983846fdd9c8af09456a106a940d191
2022-07-20 15:25:30 +08:00
Pierre Riteau
13b0f3b861 Make external access to monitoring services configurable
Change-Id: Iaf6bf36ae0adce3342981c36c859fc138b172f6b
2022-06-27 11:57:53 +02:00
Zuul
5865b0e9a8 Merge "Support setting Nova API microversion for openstack-exporter" 2022-06-24 08:48:43 +00:00
Pierre Riteau
41fba3c5df Support setting Nova API microversion for openstack-exporter
Starting from v1.5.0 of the exporter, OS_COMPUTE_API_VERSION can be set
to configure the Nova API version to be used [1]. Microversion 2.1 can
be used to keep metrics unmodified from the previous exporter version
deployed by Kolla (v1.3.0).

Support it with prometheus_openstack_exporter_compute_api_version,
defaulting to using the latest version.

[1] https://github.com/openstack-exporter/openstack-exporter/pull/201

Change-Id: I7605a3f9f74effb29ecec3b28e4709fd5f7f8cd4
2022-06-23 17:11:50 +02:00
Pierre Riteau
06223d651b Fix typo in prometheus-node-exporter restart handler
Change-Id: Ib05569a08e2fe21dae31cdacad3b622d17cb5db3
2022-06-22 16:51:49 +02:00
Radosław Piliszek
72b63dfee7 Further Keystone-related cleanups
Per comments on [1].

[1] https://review.opendev.org/c/openstack/kolla-ansible/+/843727

Change-Id: I60162b54bc06e158534d29311d4474b34750c64d
2022-06-20 08:40:03 +00:00
Zuul
4336ffbe44 Merge "Add support for custom alert notification templates" 2022-06-02 10:05:06 +00:00
Radosław Piliszek
7ca9349b09 Do not use keystone_admin_url et al
Following up on [1].
The 3 variables are only introducing noise after we removed
the reliance on Keystone's admin port.

[1] I5099b08953789b280c915a6b7a22bdd4e3404076

Change-Id: I3f9dab93042799eda9174257e604fd1844684c1c
2022-05-28 18:19:01 +02:00
k-s-dean
fcba927d7b talk TLS to openstack exporter via haproxy
Closes-Bug: #1975598
Change-Id: If4c85f8e960141d08a89accdc11a3271f31974c1
2022-05-24 16:23:42 +01:00
Radosław Piliszek
3e75a33ad4 Use the new image naming scheme
Change-Id: Ib4b15ed4feac82d8492b1c0f0238a752eac668e6
2022-05-23 06:37:25 +00:00
Piotr Parczewski
77e811e25c Add support for custom alert notification templates
Change-Id: I9e4e1287ea69960ae3b13734acc3eb0b087a8d86
2022-05-18 12:03:47 +02:00
k-s-dean
656f6cdb08 Put openstack exporter behind HAproxy so only one is queried at a time
Closes-Bug: #1972818

Change-Id: I9e36b9169b6725bf6db953e464fc099087747778
2022-05-12 07:41:57 +00:00
Zuul
2c15d36fed Merge "Adds prometheus_scrape_interval" 2022-04-21 16:55:35 +00:00
Marcin Juszkiewicz
1620ab5be9 drop install_type from image names
We have only one value for install_type now and it gets removed from
image names.

Change-Id: I8bf95fd7aa9dd26b80d618ca0fcb097003b4cb0a
2022-04-20 12:29:12 +02:00
Zuul
27bf4e9351 Merge "Switch prometheus to active/passive mode" 2022-04-15 13:05:19 +00:00
Will Szumski
6906b275ef Switch prometheus to active/passive mode
This uses the same approach as the mariadb role (and others).

Closes-Bug: #1928193
Co-Authored-By: John Garbutt <johng@stackhpc.com>
Change-Id: I79a7a8c80327cfd9ef31d17fe71f450a181a638c
2022-04-15 10:10:50 +00:00
Nathan Taylor
0f2794a075 Adds etcd endpoints as a Prometheus scrape target
Add "enable_prometheus_etcd_integration" configuration parameter which
can be used to configure Prometheus to scrape etcd metrics endpoints.
The default value of "enable_prometheus_etcd_integration" is set to
the combined values of "enable_prometheus" and "enable_etcd".

Change-Id: I7a0b802c5687e2d508e06baf55e355d9761e806f
2022-03-08 08:42:19 -07:00
Zuul
63706667e1 Merge "Add support for deploying Prometheus libvirt exporter" 2022-02-21 21:35:55 +00:00
Pierre Riteau
b210dcd6e2 Configure node-exporter to report correct file system metrics
Without this configuration, all mount points are reporting the same
utilisation metrics [1]. With the rslave option, all root mounts from
the host are visible in the container, so we can remove the bind mounts
for /proc and /sys.

[1] https://github.com/prometheus/node_exporter#docker

Change-Id: I4087dc81f9d1fa5daa24b9df6daf1f9e1ccd702f
Closes-Bug: #1961438
2022-02-18 18:36:22 +01:00
Pierre Riteau
dcba829792 Allow to define extra parameters for Prometheus exporters
The following variables are added:

* prometheus_blackbox_exporter_cmdline_extras
* prometheus_elasticsearch_exporter_cmdline_extras
* prometheus_haproxy_exporter_cmdline_extras
* prometheus_memcached_exporter_cmdline_extras
* prometheus_mysqld_exporter_cmdline_extras
* prometheus_node_exporter_cmdline_extras
* prometheus_openstack_exporter_cmdline_extras

Change-Id: I5da2031b9367115384045775c515628e2acb1aa4
2022-02-18 10:12:22 +01:00
Will Szumski
033db44f1c Adds prometheus_scrape_interval
Grafana requires the scrape interval to be set to be able to compute
$__rate_interval. The default is 15s which does not match the kolla
default of 60s. The symptom of not setting this is that you will see
"no data" when zooming graphs that use rate queries. This occurs as the
interval will be set to a period shorter than the scrape interval.
The recommendation is that you use a common scrape interval for all
jobs. See:

- https://grafana.com/blog/2020/09/28/new-in-grafana-7.2-__rate_interval-for-prometheus-rate-queries-that-just-work/
- https://stackoverflow.com/questions/66369969/set-scrape-interval-in-provisioned-prometheus-data-source-in-grafana

Change-Id: I7e5c1e20c7b66b64cbd333f669ef8d8da60daaa8
2022-02-14 11:10:44 +00:00
Zuul
2d72fc5da4 Merge "prometheus: add tls_connect blackbox module" 2022-01-28 12:24:35 +00:00
Zuul
2146015cf0 Merge "Revert "Use friendly target names in Prometheus"" 2022-01-25 09:55:44 +00:00
Zuul
dc5eaa4ec7 Merge "Use Volume V3 API in OpenStack exporter" 2022-01-07 19:19:09 +00:00
Doug Szumski
491d418476 Add support for deploying Prometheus libvirt exporter
Add support for deploying the Kolla Prometheus libvirt exporter image to
facilitate gathering metrics from the Nova libvirt service.

Co-Authored-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: Ib27e60c39297b86ae674297370f9543ab08cda05
Partially-Implements: blueprint libvirt-exporter
2022-01-05 13:30:45 +01:00
Angelos Kolaitis
4410ca7802
Use Volume V3 API in OpenStack exporter
Kolla has removed the Volume V2 API by default since OpenStack Wallaby.
However, openstack-exporter attempts to use the Volume V2 API by
default, resulting in clean installs failing to fetch Cinder metrics
in Prometheus.

This patch updates the clouds.yml configuration file for
openstack-exporter to use the Volume V3 API instead.

Closes-Bug: #1938194
Change-Id: Ifbb601be3ef1a1e853d5a7e832adf556c0ae38b9
2022-01-05 13:19:08 +02:00
Pierre Riteau
56fc74f231 Move project_name and kolla_role_name to role vars
Role vars have a higher precedence than role defaults. This allows to
import default vars from another role via vars_files without overriding
project_name (see related bug for details).

Change-Id: I3d919736e53d6f3e1a70d1267cf42c8d2c0ad221
Related-Bug: #1951785
2021-12-31 09:26:25 +00:00
Mark Goddard
c358a2d586 Revert "Use friendly target names in Prometheus"
This reverts commit 4ff65b7661ea06e9fa8631c4eb82232e03af77d7.

Reason for revert: adds assumptions about inventory_hostname being resolvable.

Closes-Bug: #1955563
Change-Id: Ifa2b2ea8622f56c34b8f7f37fee53133272ff925
2021-12-23 15:14:33 +00:00
Zuul
612937de0f Merge "Fix privileges for MariaDB 10.5" 2021-10-11 11:15:04 +00:00
Radosław Piliszek
c7c14e1c43 Fix privileges for MariaDB 10.5
"BINLOG MONITOR" and "SLAVE MONITOR" replace
"REPLICATION CLIENT" (which is now an alias for "BINLOG MONITOR").
The validation in Ansible MySQL collection is too simple to
understand aliases and breaks. Hence, let's use the canonical
names and adapt per service according to its needs.

Change-Id: I1175e4846384accd19942620dc155d0c5728e64b
2021-10-07 09:24:31 +00:00
Zuul
01470fc7e9 Merge "Use friendly target names in Prometheus" 2021-10-06 16:27:21 +00:00
Christian Berendt
4f78c696c2 Do not become root when searching for custom prometheus alert rules files
Change-Id: I6da412d6d3e7d067c8d903ee884711ac509d24aa
2021-10-04 09:49:58 +02:00
Piotr Parczewski
4ff65b7661 Use friendly target names in Prometheus
Change-Id: I16fdb2f93ddb656eeacd3f2b84190f9bdcfaa21c
2021-09-22 11:09:32 +02:00
Zuul
7cf30017ea Merge "Add Alertmanger metric target(s)" 2021-09-20 18:08:56 +00:00
Zuul
83c5d95b47 Merge "Support monitoring Fluentd with Prometheus" 2021-08-27 09:34:12 +00:00
Radosław Piliszek
9ff2ecb031 Refactor and optimise image pulling
We get a nice optimisation by using a filtered loop instead
of task skipping per service with 'when'.

Partially-Implements: blueprint performance-improvements
Change-Id: I8f68100870ab90cb2d6b68a66a4c97df9ea4ff52
2021-08-10 11:57:54 +00:00