54 Commits

Author SHA1 Message Date
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Lakshmi Prasanna Goutham Pratapa
76210a2d85 Apply Resource Constraints to Openstack Services
This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa
2018-07-25 17:05:04 +01:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
Kevin Tibi
1d249ca966 Fix Grafana config for extra conf
Previous change[1] of config for grafana introduced
new bugs.

[1]https://review.openstack.org/#/c/548291/

Change-Id: Ib62f0670003612073ff08f702febe81c753b72e5
2018-05-16 16:29:22 +02:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Mark Giles
6777c903a0 Automatically add Prometheus as a Grafana datasource
Assuming both Prometheus and Grafana are enabled, this change
configures Grafana such that it can use data collected by the
Prometheus server.

Partially-Implements: blueprint prometheus
Change-Id: I0290806d0c622b1de33cf7047f099795b4c780b9
2018-05-03 11:29:27 -04:00
Zuul
910180a5e2 Merge "Custom configuration for grafana" 2018-04-16 08:06:37 +00:00
Kevin TIBI
965669b461 Custom configuration for grafana
Add custom configuration for grafana

Change-Id: I03e3ab57ce538c0f86d1442c7bd30d3ce9012637
2018-04-13 10:43:44 +02:00
Doug Szumski
b865bada8c Support accessing InfluxDB via HAProxy
This change allows access to InfluxDB via HAProxy on the internal
network.

If HA is required the Enterprise version of InfluxDB will need to
be installed. This could be implemented by customising the InfluxDB
Docker file.

There are other alternatives to HA, such as the InfluxDB Relay [1].
Support for this is considered to be outside of the scope of this
bugfix.

[1] https://docs.openstack.org/developer/performance-docs/
    methodologies/monitoring/influxha.html

Closes-Bug: #1751283
Change-Id: I4624efbd99c0cddd1361f2438866ad3a82e5557b
2018-04-06 17:48:24 +01:00
Duong Ha-Quang
9965cc46ff Specify 'become' for only neccesary tasks (all other roles)
Add become to only neccesary tasks in roles:

- aodh
- barbican
- bifrost
- ceilometer
- ceph
- chrony
- cinder
- cloudkitty
- collectd
- congress
- designate
- elasticsearch
- etcd
- freezer
- gnocchi
- grafana
- influxdb
- ironic
- iscsi
- karbor
- kibana
- kuryr
- magnum
- manila
- mistral
- mongodb
- multipathd
- murano
- octavia
- panko
- qdrouterd
- rally
- sahara
- searchlight
- senlin
- skydive
- solum
- swift
- swift
- tacker
- telegraf
- tempest
- trove
- vmtp
- watcher
- zun

Change-Id: I6e32d94d4172dd96d09d8609e8a5221ab5586a31
Partial-Implements: blueprint ansible-specific-task-become
2018-03-12 09:37:43 +07:00
Doug Szumski
b8979c2083 Support configuring grafana local admin username
The grafana local admin username can be configured by overriding
the admin user field in the grafana.ini file. However, this will
fail when kolla-ansible attempts to configure any enabled
datasources for grafana because the local admin password is
hardcoded to 'admin'. This change allows the grafana local admin
password to be configured via group vars so that the correct
username is used when configuring datasources.

Closes-Bug: #1750408
Change-Id: I0962200894f7a0452da1c249a68f9230b6fab13f
2018-02-19 16:58:50 +00:00
Doug Szumski
dafe39dfb6 Fix retry mechanism
If the wrong status code is returned, ansible sets failed: true.
This means that a retry is never attempted.

Use .get('status') to work around the possibility that a status
may not be returned in some corner cases. For example:

TASK [grafana : Wait for grafana application ready]
***************** fatal: [some_remote_node]: FAILED! =>
{"failed": true, "msg": "The conditional check 'result.status
== 200' failed. The error was: error while evaluating
conditional (result.status == 200): 'dict object' has no
attribute 'status'"}

Change-Id: I14207d1a9e994fa25c1c061025fff5b64a71d690
Closes-Bug: #1742501
2018-01-24 16:01:17 +00:00
Alexandru Bogdan Pica
8e3b79440c Implement external MariaDB and pre-configured Databases support
This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121
2018-01-23 13:07:40 +00:00
caoyuan
b63ecd7e3e Update kolla_internal_fqdn to database_address since it defined in all.yml
through the database_address has beed defined in groups_vars/all.yml, we should
better use it, this way, if we want to use external database, we just need to
redefined in all.yml

refer to https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L83
Co-Authored-By: chenqiaomin <chen.qiaomin@99cloud.net>

Change-Id: Ie559301451954e16347ceaabf02f594c5c5cbe56
2018-01-17 13:01:48 +08:00
Doug Szumski
2854671018 Local connection to remote Grafana can fail
Following deployment of Grafana, a check is made to ensure that the
login page is accessible. If the Grafana container has been deployed
to a remote host, but is not directly accessible over the deployment
network then the check will fail because the task is set to run with
connection: local. For example, if Ansible is using a different
network to the network on which grafana is exposed to connect to the
remote host then the task may fail. A similar problem exists for the
remaining two tasks in the post_config file.

Further to this, if Grafana is not accessible, the ansible uri
module does not return a status field, so no retry attempts are made
and the task fails with an obscure message:

TASK [grafana : Wait for grafana application ready] *****************
fatal: [some_remote_node]: FAILED! => {"failed": true, "msg": "The
conditional check 'result.status == 200' failed. The error was: error
while evaluating conditional (result.status == 200): 'dict object' has
no attribute 'status'"}

$ ansible --version
ansible 2.3.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.5 (default, Aug 4 2017, 00:39:18) [GCC 4.8.5 20150623
(Red Hat 4.8.5-16)]

Change-Id: I77e2129b4a1c5f2f0802fb92eb03395ff72cf5f3
Closes-Bug: #1742501
2018-01-11 09:38:02 +00:00
guochao
deadbdb43e Disable grafana Getting Started panel
I think that we do not need this in a production-ready project.

Change-Id: Ief1392c5e8becc90034fca782211a1da30da14ae
2017-08-16 09:57:49 +00:00
Jenkins
b4d7592adb Merge "Support customizing grafana home dashboard" 2017-08-16 03:52:34 +00:00
Mathias Ewald
f4e7b3af7f Add Elasticsearch to Grafana
This patch adds Elasticsearch to Grafana as a data source

Change-Id: I7c6452bba47274f9d971b8000e9d3e911617874d
2017-07-25 09:18:12 +00:00
Mathias Ewald
fa52fb3514 Fix grafana post-config check
Check grafana's /login url for status code 200 instead of root url since the
latter does not return an http 200 code

Change-Id: Ide7da26c0e217a4a60ad3d2ac693d6f7ac7bb722
Closes-Bug: #1705911
2017-07-24 11:13:22 +00:00
guochao
08753b404e Support customizing grafana home dashboard
Replace grafana home dashboard with file in node_custom_config folder.

Change-Id: I00e3963ee287c526935e34169b63b1a40e6b4025
2017-07-24 13:11:31 +08:00
Mathias Ewald
b7aef07d28 Fix grafana data source registration
Make data source creation more dynamic to support a variable number of data
source of different types

Only create data sources actually deployed by kolla

Change-Id: I26a2d3c71b178562b906623ab77eaff673c81dce
2017-07-21 09:35:55 +00:00
Jenkins
a71aa61ea7 Merge "Use unified vars in task" 2017-07-12 01:56:12 +00:00
caoyuan
36c530f9ed Fix the database user name for all roles
when create database user, it should use
database_user, not database_name.

Change-Id: I4dfa01d1a5a46c5c58f1fc47b0be71b186462764
Closes-Bug: #1698762
2017-06-20 21:19:31 +08:00
Jenkins
322eb353a2 Merge "Fix "Enable influxdb datasource" random error" 2017-06-16 11:22:46 +00:00
Bertrand Lallau
991695a272 Simplify condition check on standalone services
Condition check are already performed here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/site.yml
In order to simplify codebase, these checks can be removed for
standalone services.

Change-Id: Ib9842cd5363a1c7e56234a5e91dc264f89e2838f
2017-06-15 17:15:17 +02:00
Jeffrey Zhang
03e1b05f99 Use unified vars in task
Ansible task support vars directive, no need implement another one in
merge_config. This patch remove the vars directive in merge_config
action plugin.

Change-Id: I33648a2b6e39b4d49ce76eb66fbf2522721f8c68
2017-06-15 22:37:28 +08:00
Bertrand Lallau
49c083f555 Fix "Enable influxdb datasource" random error
Sometimes the "Enable influxdb datasource" task failed with "503 Service
Unavailable" error. Waiting for grafana port UP seems not always enough,
it requires to wait for application ready.

Change-Id: I3a27b3086698b89a80a3f585beb810831289ae7a
Closes-Bug: #1697213
2017-06-12 10:56:44 +00:00
Bertrand Lallau
bb1b0857b3 Grafana post_config checks only need to run locally
Recently added post_config for Grafana does some uri queries,
these can be problematic if run across every node.

Change-Id: I580212474879da18780fc5f66a21952a9e6f8daa
2017-06-02 21:30:51 +00:00
Bertrand Lallau
afdd11b9a2 Generalize api_interface_address variable usage
Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455
2017-05-23 08:35:15 +00:00
Jenkins
b077f6e7bb Merge "Add timeout parameter for precheck tasks" 2017-05-07 15:04:24 +00:00
Jeffrey Zhang
b1f015616c Add timeout parameter for precheck tasks
wait_for module waits 300 seconds for the port started or stopped.  This
is meaningless and useless in precheck. This patch change timeout to 1
seconds.

Change-Id: I9b251ec4ba17ce446655917e8ef5e152ef947298
Closes-Bug: #1688152
2017-05-04 09:02:25 +08:00
Jenkins
abd86ecd13 Merge "Fix Grafana config file template to use variables" 2017-05-03 13:33:26 +00:00
Bertrand Lallau
29844253c1 Grafana: automatically set InfuxDB datasource
In order to automate Grafana configuration, this fix automatically
set the InfluxDB datasource. This avoid doing a annoying manual
configuration after Kolla-ansible deployment.

Change-Id: I2b1a63fd371966192f1df0a82cee4711c6324710
2017-04-25 22:20:32 +02:00
Bertrand Lallau
a49e18a73f Fix Grafana config file template to use variables
Grafana session/provider_config property actually uses a static user
and a static database name.
If grafana_database_user and grafana_database_name default value is
changed, the config generated will not be updated accordingly.

Change-Id: I517daab5439ab1aef56a4b3a11104bd0717223bf
Closes-Bug: #1682810
2017-04-14 14:40:53 +02:00
Bertrand Lallau
dfdf3efb3f Fix error during Grafana container restart
grafana_ini file used in test condition doesn't exists,
use grafana_conf file instead.

Change-Id: Id5dca94aed98b0d6cf6c4f841e7380f8b8154416
Closes-Bug: #1673932
2017-03-21 09:28:18 +01:00
Paul Bourke
5418ada148 Enable sanity checks from kolla-ansible
Add a new subcommand 'check' to kolla-ansible, used to run the
smoke/sanity checks.

Add stub files to all services that don't currently have checks.

Change-Id: I9f661c5fc51fd5b9b266f23f6c524884613dee48
Partially-implements: blueprint sanity-check-container
2017-03-09 10:37:06 +00:00
Eduardo Gonzalez
7e53d2e74e Fix grafana database creation
Kolla use kolla_toolbox container to interact
with the database.
Grafana fails to deploy if mysql-python is not installed
in the host.

Change-Id: I033f1828ff027532de9be94aaf7dee6dbeab8fb6
Closes-Bug: #1661646
2017-02-03 23:32:04 +00:00
Mauricio Lima
43813b8a6d Optimize reconfiguration for grafana
Change-Id: I672d6fea935cbe7416ebd0cfff5006fddb9cd534
Partially-implements: blueprint better-reconfigure
2017-01-25 10:35:34 -03:00
Jenkins
3f8cb8adda Merge "Refactor bootstrap.yml file" 2017-01-18 10:17:50 +00:00
Jeffrey Zhang
7233b45d29 Refactor bootstrap.yml file
Change-Id: I00d2dcb0895548ba169ab85764cf546c2214cbf5
2017-01-12 23:37:50 +08:00
caoyuan
02d9de8636 Move grafana precheck into its own role
Change-Id: Ic2ad2b38d7e1c988f9dca9dc0d377cfee7b89504
Partially-implements: blueprint condition-pre-check
2017-01-11 16:25:07 +08:00
Jeffrey Zhang
fa45801946 Move precheck into its own role
* Merge prechecks.yml and site.yml playbook
* Create empty precheck.yml into all roles.

Change-Id: I8a138558a26c0a2a66c5fd48ed37be657c99c1dd
Implements: blueprint condition-pre-check
2016-11-03 14:48:58 +08:00
prameswar
c1d80d3410 grafana added to haproxy to listen on VIP
Closes-bug: #1636579

Change-Id: I5628196885bddebab84abb71c89753582123418c
2016-10-26 15:51:54 +00:00
Jeffrey Zhang
aed67d9cc5 Add missing closing brace in grafana configuration file
Change-Id: Iebcca62cccb02b71953c43e1b595b731fc9d0b92
Closes-Bug: #1634506
2016-10-18 21:36:09 +08:00
zhubingbing
ce3126c54a
Fix grafana upgrade permission issue
Change-Id: I15d96a8ee04f860ce149b8a64f306e181383d2fa
Partial-Bug: #1631503
2016-10-18 11:09:34 +08:00
Jenkins
45b0f2cf63 Merge "Remove do_reconfigure.yml file for all roles" 2016-10-13 16:20:30 +00:00
Jeffrey Zhang
0c15da00c8 Fix the wrong setting for Grafana session
The provider_config is configured wrong. And Grafana only supports one
Memcached instance for session, which is bad for high available.

Use mysql to provider the session storage.

TrivialFix

Change-Id: I889a961d7f36f44701654fbac04b4bff05043506
2016-10-10 19:47:32 +08:00
Jenkins
8bde923bba Merge "Add newline in grafana template" 2016-09-28 10:51:28 +00:00