This change adds the scaffolding required to get multi-distro support
running in the roles. The change breaks up our playbooks converting all
of the tasks into various roles with internal dependencies. While this
will improve execution time, the change is being done to reduce boiler
plate and to allow us to build on the pattern used in OSA to provide
multi-distro capabilities.
A side effect of this change is a major improvement in idempotency. The
playbooks should now be 100% idempotent.
All of the templates have been left in the main playbook directory. This
was done to help ease the transition. In a future PR the template
structure will be moved into the roles where it needs to be.
The main variable files has been left intact. This file will be carved
up into role defaults in a future PR.
Change-Id: I938a10564128ce4078fa12edcf614dcdbd684b25
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This allows alternate algorithms to be developed and enables the
same tasks to be included in a test to verify the planned retention
does not exceed the cluster storage capacity.
Change-Id: Ie3d80d6cfad16b946ccd790859bc7cd92b90fdef
This change adds xpack monitoring by default for kibana, logstash, and
elasticsearch.
Change-Id: I03c8b8390d33bb996201671aae481415bfc1e691
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Systemd overrides have been added to the service unit files for all
beats and services. All of the playbooks have been updated to make them
look and feel uniform.
This also sets handlers within the playbooks so that we're improving the
idempotence.
Change-Id: I2dd3183dae4bfddc607cc74f9dfb7af115b80abc
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
To install curator facts are required, this change ensures we gather
them.
Change-Id: I510692095cdf8ecb5806a43c714b7bbbace47022
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The curator retention policies will now query the storage nodes within
a given deployment and set a suitable index retention policy based on
the total amount of storage each index is assumed to produce every day.
To ensure we're minimizing the storage required and optimizing search
performance several actions are now being taken:
* Indexes will be shrunk after a quarter of their retention time.
* Indexes will be deleted should they exceed the retention time.
Change-Id: I8bf548620b5404d25deaadba8fda93452ef64fa0
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change breaks out the basic install and setup playbooks into
different playbooks. This is done to ensure an operator can easily rerun
any part of the playbook as they upgrade, change, or modify a setup over
the lifetime of the deployment.
Change-Id: I66c53fcb21880c950ea3fee202e7d2224dfdff3a
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The tasks with external deps should all have a retry so to cater for
cases of intermittent service interruptions. This change adds retires
and tags to of the tasks with external deps.
Change-Id: Icd0a8143347bcbffc2a1e0a02da92091b59ed483
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
To run the curator playbook pip is required. This change installs the
package `python-virtualenv` to resolve the dependency and installs the
curator application into a virtualenv within /opt. This will keep the
host clean by isolating the dependencies and ensure that we can easily
clean up should we ever need to.
Change-Id: If872bb30281948d79a3f79a0e3a39e8a457c6fdb
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The elasticsearch-curator package requires certifi 2018.4.16 but this
is constrained to version 2018.1.18. Use isolated mode to allow
installation of the upstream version.
Change-Id: I4ba387afab35230e70b45fab42b6bfaf9ffeba19
At present we're collecting too much info by default. We're seeing
+500GB on a <50 node environment in just two weeks. While we dont expect
the data set to grow much larger given the use of curator, this change
lowers the default collection intervals of the various beats and updates
the retention / detection policies so we're not storing too much
information.
To correct a unicode problem with py2 the host index loops have been
updated.
Curator has also been updated to run everyday.
Change-Id: Ic202eb19806d1b805fa314d3d8bde05b286740e0
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Curator has been added to automatically maintain the cluster with
sensible defaults when it pertains to data retention.
The index counts have been modified such that they're determined by the
size of the initial cluster. While these shard counts can be modified
post deployment by reindexing the data, it's not something being done at
this time.
Depends-On: https://review.openstack.org/c/565807
Change-Id: I249d715ae5241ab57c4117b14377e4d07cb6e984
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>