This change creates a new option to enable or disbale rollup jobs. This
is also providing the default basic index patterns for kibana index
patterns and elastic indexes.
Change-Id: I60e96a2cdbe27de760b54c4d9d43bcde4d09bbf5
Signed-off-by: cloudnull <kevin@cloudnull.com>
This change updates the roles / playbooks to begin using Elasticsearch
relesae 6.5.x. Core to this change is the conversion of the journalbeat
role from custom compiled go, to simple package install which was made
possible by the folks at elastic within this release. Because of the
conversion the "beats-community" playbook has been removed given its now
empty.
A change to the bootstrap script was made allowing it to parse an OS id
with a "-" in it, like "opensuse-tumbleweed".
Change-Id: Ic9b80234d6a6ce876bff885f3223874602d55dd6
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
As the role based macro links are pointing to unreachable destination
and are not required, they will be removed.
Additionally the timesouts are increased for API commands to ES, along
with minor changes around the upstart system manager
Change-Id: I2572bce230af2fd43261c9b0bf903bfd9655959e
This change adds Ubuntu 14.04 support to the project.
Change-Id: I20695e19409b63c6e1def4ccf8929c6d52be647e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change adds the scaffolding required to get multi-distro support
running in the roles. The change breaks up our playbooks converting all
of the tasks into various roles with internal dependencies. While this
will improve execution time, the change is being done to reduce boiler
plate and to allow us to build on the pattern used in OSA to provide
multi-distro capabilities.
A side effect of this change is a major improvement in idempotency. The
playbooks should now be 100% idempotent.
All of the templates have been left in the main playbook directory. This
was done to help ease the transition. In a future PR the template
structure will be moved into the roles where it needs to be.
The main variable files has been left intact. This file will be carved
up into role defaults in a future PR.
Change-Id: I938a10564128ce4078fa12edcf614dcdbd684b25
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Systemd overrides have been added to the service unit files for all
beats and services. All of the playbooks have been updated to make them
look and feel uniform.
This also sets handlers within the playbooks so that we're improving the
idempotence.
Change-Id: I2dd3183dae4bfddc607cc74f9dfb7af115b80abc
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The following change will ensure that the elastic-static is logging to
the journal and that systemd is able to report how well the elastic
slice is running.
Change-Id: I79a9074b5f14a41dec421d6691fd04c0e6be15b7
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change breaks out the basic install and setup playbooks into
different playbooks. This is done to ensure an operator can easily rerun
any part of the playbook as they upgrade, change, or modify a setup over
the lifetime of the deployment.
Change-Id: I66c53fcb21880c950ea3fee202e7d2224dfdff3a
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The tasks with external deps should all have a retry so to cater for
cases of intermittent service interruptions. This change adds retires
and tags to of the tasks with external deps.
Change-Id: Icd0a8143347bcbffc2a1e0a02da92091b59ed483
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
To ensure users can upgrade packages the variable
`"{{ elk_package_state | default('present') }}"` has been added
to all package installs.
Change-Id: I0238d9e1ed991cb1480bd924f2d5a09687890da3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When running a multi-node ELK stack (5+ nodes) the elasticsearch server
needs to be able to mark and identify nodes that will be data and master
nodes. This change ensures that the first set of available master nodes
is mark accordingly as both data and master and that all other nodes are
marked as cluster members with alternating nodes being marked as data
nodes. With this change the cluster will be able to grow as well as
handle large data sets more efficiently.
This change ensures that sharding is correctly started when the cluster
is deployed and that the beat services are using the recommended
sharding practices.
Change-Id: Id555132199f268b21aaa939a84760c744238dd2b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change allows EL to have customizable heap sizes which will
automatically set themselves to the recommended size as advised by the
elastic documentation.
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/heap-size.html
Change-Id: I55d014dab37bf5e5a4806bd19d4e07e072f9fa7e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Most of the changes in this PR are for style and to adapt the playbooks
so that the system can operate on a multi-node cloud.
Functional change includes the removal of mainline Java 8 in favor of
OpenJDK 8.
A site playbook was add to allow an operator to just run everything.
Old tools that no longer function within the stack have been removed.
Packetbeat was added to the install list
Auditbeat was added to the install list
All of the config files have been updated for the recent ElasticStack
6.x changes.
Change-Id: I01200ad4772ff200b9c5c93f8f121145dfb88170
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This addition is an updated of the curent elk_metrics which will install Elasticsearc, Logstash and Kibana 6.x.
It also include configuration guide for haproxy endpoints
Change-Id: Iac4dec6d17bc75433e5fe672f3b9781536b8e619