This change allows the embedded ansible process to be configurable by
the end user.
* Python requirements and ansible roles will all now be user
configurable.
* Setup is now a local only playbook. This playbook replaces the bash
commands we were rerunning when the `bootstrap-embedded-ansible.sh`
script was executed.
* Embedded ansible version is now 2.7.5 as default.
* Deprecation warnings have been resolved.
* Tests impacted by this change have been updated.
Change-Id: I4303c44e249cda31457a4f05a681e298d225a8b7
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change reduces code throughout the playbooks thereby speeding up
the task execution.
* A new role named `elastic_beat_setup` was created to
facilitate template setup as needed.
* Beats retention policies are now defined on the elastic-logstash
nodes instead of on all target hosts. This method will speed-up
deployments on massive installations while streamlining all deployments.
* Kibana variable assumptions have been fixed. This will allow for
deployments without Kibana to be accomplished.
Change-Id: I36343264042e81dfcb68bad0f6c3a503e525eceb
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The template creates can take a lot of time when dealing with large data
sets. This change makes it so template create will only happen on a
greenfield installation or when upgrading. A simple rerun of the
playbooks will not trigger template creations which will allow deployers
the ability to better change or modify deploymens without having to
worry about extended runtimes due to template interactions.
Change-Id: Ia9b77277553fbdbe0444737f39ec3de75f07cc0f
As the role based macro links are pointing to unreachable destination
and are not required, they will be removed.
Additionally the timesouts are increased for API commands to ES, along
with minor changes around the upstart system manager
Change-Id: I2572bce230af2fd43261c9b0bf903bfd9655959e
Previously the beat setup tasks were tagged with 'setup' but the include
statements were not, so the tasks were always skipped when using '--tags
setup'. This change adds tags to the includes so that the tasks are executed
as expected
Change-Id: If16069cd273d84a22b229b8140e5a8d56eed86d1
The multi-logstash pipeline setup, while amazingly fast, was crashing
and causing index errors when under high load for a long period of time.
Because of the crashing behavior and the fact that the folks from
Elastic describe multi-pipeline queues to be "beta" at this time the
logstash pipelines have been converted back into a single pipeline.
The memory backed queue options are now limited by a ram disk (tmpfs)
which will ensure that a burst within the queue does not cause OOM
issues and ensures a highly performant deployment and limiting memory
usage at the same time. Memory backed queues will be enabled when the
underlying system is using "rotational" media as detected by ansible
facts. This will ensure a fast and consistent experience across all
deployment types.
Pipeline/ml/template/dashboard setup has been added to the beat
configurations which will ensure beats are properly configured even
when running in an isolated deployment and outside of normal operations
where beats are generally configured on the first data node.
Change-Id: Ie3c775f98b14f71bcbed05db9cb1c5aa46d9c436
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
There is no guarantee that all container IP addressess will be included
in an existing no_proxy environment variable. This will cause failures
when an http proxy is configured, but the proxy does not allow traffic
to 'hairpin' back to internal addresses.
This change forces no_proxy to the specific address of the kibana
and coordinator endpoints when the uri module is used to load dashboards
and configure rollups.
Change-Id: I669334c722cce79459b522e6e2d7e1aaec49ef24
It looks like when using a normal include, Ansible tries to include
all sorts of extra things, further breaking the environment by trying
to resolve unaccessible variables.
This patch refactors all of those includes to macros therefore including
a single file and also decreasing the memory usage to avoid copying
over the entire context.
Change-Id: Ie8733c7d52b1fc5bde484855988bddf6a06dbe00
This change adds SUSE 42.3 support to the elastic telemetry solutions.
Change-Id: Ibe93ea0d1ead9e7fe6da16d89989cfe5ade0f43e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change will help with organization throughout the stack.
Change-Id: I2ad865db534ae1d377bbdecd4b421ee0fc802536
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change adds Ubuntu 14.04 support to the project.
Change-Id: I20695e19409b63c6e1def4ccf8929c6d52be647e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change adds the scaffolding required to get multi-distro support
running in the roles. The change breaks up our playbooks converting all
of the tasks into various roles with internal dependencies. While this
will improve execution time, the change is being done to reduce boiler
plate and to allow us to build on the pattern used in OSA to provide
multi-distro capabilities.
A side effect of this change is a major improvement in idempotency. The
playbooks should now be 100% idempotent.
All of the templates have been left in the main playbook directory. This
was done to help ease the transition. In a future PR the template
structure will be moved into the roles where it needs to be.
The main variable files has been left intact. This file will be carved
up into role defaults in a future PR.
Change-Id: I938a10564128ce4078fa12edcf614dcdbd684b25
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>