Currently, we have a lot of logic for checking if a handler should run,
depending on whether config files have changed and whether the
container configuration has changed. As rm_work pointed out during
the recent haproxy refactor, these conditionals are typically
unnecessary - we can rely on Ansible's handler notification system
to only trigger handlers when they need to run. This removes a lot
of error prone code.
This patch removes conditional handler logic for all services. It is
important to ensure that we no longer trigger handlers when unnecessary,
because without these checks in place it will trigger a restart of the
containers.
Implements: blueprint simplify-handlers
Change-Id: I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
We don't add extra volumes support for all services in patch [1].
In order to unify the management of the volume, so we need add extra volumes
support for these services.
[1] 12ff28a693
Change-Id: Ie148accdd8e6c60df6b521d55bda12b850c0d255
Partially-Implements: blueprint support-extra-volumes
Signed-off-by: ZijianGuo <guozijn@gmail.com>
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:
* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts
It also adds become for 'command' tasks that use docker CLI.
Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
This change ensures that URLs returned from these services reference
the HAProxy endpoint, rather than the host on which the service is
running.
Closes-Bug: #1825150
Change-Id: I7f966ff749ea37620f1bde7019a598cb9505fa45
Several config file permissions are incorrect on the host. In general,
files should be 0660, and directories and executables 0770.
Change-Id: Id276ac1864f280554e98b937f2845bb424d521de
Closes-Bug: #1821579
All Prometheus services should use the Prometheus install type which
defaults to the Kolla install type, rather than directly using the
Kolla install type.
Change-Id: Ieaa924986dff33d4cf4a90991a8f34534cfc3468
This patch implements the support for the elasticsearch-exporter in
kolla-ansible
The configuration and prechecks are reused from the other exporters
Depends-On: Id138f12e10102a6dd2cd8d84f2cc47aa29af3972
Change-Id: Iae0eac0179089f159804490bf71f1cf2c38dde54
The patch that this depends on in the Kolla repo updates various Prometheus
exporters. In some cases the command line syntax has changed which prevents
them from starting. This commit updates the command line syntax in-line with
the new versions.
Depends-On: I846989b16fa7f76b11b309b7a9764cec8aaf538d
Change-Id: I1c8c56059e51442d7bf2248b9632021cb529b4ba
This allows an operator to pin the Prometheus docker image
tag for all Prometheus images to that specified by the `prometheus_tag`.
Without this change, the alert manager and cadvisor tags would also need
to be set.
Change-Id: Iadef001af7d3be5b2a39ce5e2363d05a33a775e4
This patch implements the initial support for the
openstack-exporter[0] in the kolla-ansible
prometheus monitoring system.
The configuration and prechecks are reused from the other
exporters and a new template is provided for generating
a os-client-config file required by the exporter.
The default scrape interval is 60 seconds, but it can
be extended via a configuration option.
[0] https://github.com/Linaro/openstack-exporter
Change-Id: I4a34c4bb56e74b5cd544972cbd6540d9acb6e4a1
Vitrage has already supported Prometheus as
datasource. Kolla can config it automatically,
just need a little changes, for example in
wsgi config file [1].
Co-Authored-By: Hieu LE <hieulq2@viettel.com.vn>
[1] https://review.openstack.org/#/c/584649/8/devstack/apache-vitrage.template
Change-Id: I64028a0dfd9887813b980a31c30c2c1b1046da61
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.
Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.
Available are two new templates:
* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend
For now the default will be the single listen block, for ease of
transition.
Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
Without this, kolla-ansible check fails with the following error:
Unable to retrieve file contents
Could not find or access '/path/to/kolla-ansible/ansible/check.yml'"}
Also adds the check command to the CI tests, to ensure that it does not
break again.
Change-Id: I9fc2f9999f55cb742ac3ac38579dcf26524a9fc7
Closes-Bug: #1790653
This patch extends the configuration task for prometheus
to allow the operator to pass a(set) of prometheus alert
rules files, that will be used by alertmanager to produce
alerts.
This functionality is only enabled when the prometheus-alertmanager
service is enabled.
Change-Id: I882759c3774f43640631c1058f8a9cb24e7a60d2
Closes-Bug: #1776529
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@linaro.org>
Change https://review.openstack.org/#/c/571826/4/
introduced the usage of merge_yaml for rendering the
prometheus-alertmanager.yml configuration.
However the merge_yaml module doesn't do a deep copy of >=
second level properties, so it doesn't works for
most configurations.
Bug: #1786077
Change-Id: I35297c6e2a3800582fb1fd3782a5d93558562b1d
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@linaro.org>
This commit is the final commit to apply resource-constraints
to all OpenStack services.
Depends-on: I39004f54281f97d53dfa4b1dbcf248650ad6f186
Change-Id: I072d69be9698be54775cb0ae286ea2b6ed78776c
Implements: blueprint resource-constraints
This patch enables the ceph mgr prometheus exporter.
If enable_prometheus_ceph_mgr_exporter is set to true,
the ceph mgr prometheus plugin is enabled on the hosts that are part
of the ceph-mgr group, then the exporter is added into the prometheus-server
configuration file.
Change-Id: Ia2f879401e585e6043f69cc5e3ab1a1f72f7f033
This patch extends the prometheus role for being able
to deploy the prometheus-alertmanager[0] container.
The variable enable_prometheus_alertmanager
decides if the container should be deployed and enabled.
If enabled, the following configuration and actions are performed:
- The alerting section on the prometheus-server configuration
is added pointing the prometheus-alertmanager host group as targets.
- HAProxy is configured to load-balance over the prometheus-alertmanager
host group. (external/internal).
Please note that a default (dummy) configuration is provided, that
allows the service to start, the operator should extend it via a node custom config
[0] https://github.com/openstack/kolla/tree/master/docker/prometheus/prometheus-alertmanager
Change-Id: I3a13342c67744a278cc8d52900a913c3ccc452ae
Closes-Bug: 1774725
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@linaro.org>
Add become to all tasks that use the module "kolla_docker"
Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
cAdvisor (Container Advisor) provides metrics on resource usage and
performance characteristics of running containers. This change
deploys a cadvisor container and configures prometheus to scrape
data from it.
Change-Id: I55dd4fee954f9be68efda397746861ddaaa0a565
Partially-Implements: blueprint prometheus
This patch adds the prometheus_memcached_exporter[0] to the
list of available exporters, following the conventions
used by the previously integrated exporters.
[0] https://github.com/openstack/kolla/tree/master/docker/prometheus-memcached-exporter
Change-Id: I103b0ee19ef2fd17ce19a27d60773675ad234c1c
Closes-Bug: #1773303
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@linaro.org>
In change I78cb60168aaa40bb6439198283546b7faf33917c, action was changed
to kolla_action, and serial to kolla_serial, to avoid Ansible warnings
due to use of reserved keywords. In that change, some keywords were
missed, and some changes that were merged since then have not switched
to the new variables. This change fixes all current instances of those
issues.
Change-Id: I357dffdfcb2b405e280a962d366ee65eebf0a8d1
Implements: blueprint migrate-to-ansible-2-2-0
This patch adds the ansible role to deploy the prometheus service which
can be used to collect performance metrics accross the environment
Partially-Implements: blueprint prometheus
Change-Id: I908b9c9dad63ab5c9b80be1e3a80a4fc8191cb9e
