openstack/openstack-ansible-ops
Code Issues Proposed changes
1021a5b945
openstack-ansible-ops/elk_metrics_6x
History
Per Abildgaard Toft 48e2b8e998 Updatev version of ELK stack for openstack ansible 
This addition is an updated of the curent elk_metrics which will install Elasticsearc, Logstash and Kibana 6.x.
It also include configuration guide for haproxy endpoints

Change-Id: Iac4dec6d17bc75433e5fe672f3b9781536b8e619
2018-03-06 14:21:23 +00:00
..
conf.d Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
env.d Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
templates Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
vars Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
installElastic.yml Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
installKibana.yml Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
installLogstash.yml Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
installMetricbeat.yml Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
readme.rst Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00
reverseProxyKibana.yml Updatev version of ELK stack for openstack ansible 2018-03-06 14:21:23 +00:00

readme.rst

install Elk stack with topbeat to gather metrics

tags

openstack, ansible

Changelog

2018-03-06 Per Abildgaard Toft (per@minfejl.dk): Updated to version Elasticsearch,Logstash and Kibana 6.x. Changed Topebeat (deprecated) to metricbeat. Included haproxy endpoint configuration.

About this repository

This set of playbooks will deploy elk cluster (Elasticsearch, Logstash, Kibana) with topbeat to gather metrics from hosts metrics to the ELK cluster.

Process

Clone the elk-osa repo

cd /opt
git clone https://github.com/openstack/openstack-ansible-ops

Copy the env.d file into place

cd openstack-ansible-ops
cp env.d/elk.yml /etc/openstack_deploy/env.d/

Copy the conf.d file into place

cp conf.d/elk.yml /etc/openstack_deploy/conf.d/

In elk.yml, list your logging hosts under elastic-logstash_hosts to create the elasticsearch cluster in multiple containers and one logging host under kibana_hosts to create the kibana container

vi /etc/openstack_deploy/conf.d/elk.yml

Create the containers

cd /opt/openstack-ansible-playbooks
openstack-ansible lxc-containers-create.yml -e 'container_group=elastic-logstash:kibana'

install master/data elasticsearch nodes on the elastic-logstash containers

cd /opt/openstack-ansible-ops
openstack-ansible installElastic.yml -e elk_hosts=elastic-logstash -e node_master=true -e node_data=true

Install an Elasticsearch client on the kibana container to serve as a loadbalancer for the Kibana backend server

openstack-ansible installElastic.yml -e elk_hosts=kibana -e node_master=false -e node_data=false

Install Logstash on all the elastic containers

openstack-ansible installLogstash.yml

Install Kibana, nginx reverse proxy and metricbeat on the kibana container

openstack-ansible installKibana.yml

Conigure haproxy endpoints:

Edit the /etc/openstack_deploy/user_variables.yml file and add fiel following lines:

haproxy_extra_services:
 - service:
      haproxy_service_name: kibana
      haproxy_ssl: False
      haproxy_backend_nodes: "{{ groups['kibana'] | default([]) }}"
      haproxy_port: 81
      haproxy_balance_type: tcp

and then run the haproxy-install playbook .. code-block:: bash cd /opt/openstack-ansible/playbooks/ openstack-ansible haproxy-install.yml --tags=haproxy-service-config

install Metricbeat everywhere to start shipping metrics to our logstash instances

openstack-ansible installMetricbeat.yml

Trouble shooting:

If everything goes bad, you can clean up with the following command: