openstack/kolla-ansible
Code Issues Proposed changes
Ansible deployment of the Kolla containers
3,254 Commits 5 Branches 109 Tags 175 MiB
Jinja 48.2%
Python 39%
Shell 12.6%
Ruby 0.2%
491aff0b88
Go to file
Éric Lemoine 491aff0b88 Make Heka send logs to Elasticsearch 
This patch includes changes relative to integrating Heka with
Elasticsearch and Kibana.

The main change is the addition of an Heka ElasticSearchOutput plugin
to make Heka send the logs it collects to Elasticsearch.

Since Logstash is not used the enable_elk deploy variable is renamed
to enable_central_logging.

If enable_central_logging is false then Elasticsearch and Kibana are
not started, and Heka won't attempt to send logs to Elasticsearch.

By default enable_central_logging is set to false. If
enable_central_logging is set to true after deployment then the Heka
container needs to be recreated (for Heka to get the new
configuration).

The Kibana configuration used property names that are deprecated in
Kibana 4.2. This is changed to use non-deprecated property names.

Previously logs read from files and from Syslog had a different Type
in Heka. This is changed to always use "log" for the Type. In this
way just one index instead of two is used in Elasticsearch, making
things easier to the user on the visualization side.

The HAProxy configuration is changed to add entries for Kibana.
Kibana server is now accessible via the internal VIP, and also via
the external VIP if there's one configured.

The HAProxy configuration is changed to add an entry for
Elasticsearch. So Elasticsearch is now accessible via the internal
VIP. Heka uses that channel for communicating with Elasticsearch.

Note that currently the Heka logs include "Plugin
elasticsearch_output" errors when Heka starts. This occurs when Heka
starts processing logs while Elasticsearch is not yet started. These
are transient errors that go away when Elasticsearch is ready. And
with buffering enabled on the ElasticSearchOuput plugin logs will be
buffered and then retransmitted when Elasticsearch is ready.

Change-Id: I6ff7a4f0ad04c4c666e174693a35ff49914280bb
Implements: blueprint central-logging-service
2016-03-05 11:51:49 +01:00
ansible Make Heka send logs to Elasticsearch 2016-03-05 11:51:49 +01:00
demos Extension .md is changed to .rst 2015-08-24 22:14:22 +05:30
dev Fix issue in newest_dhcp_lease.py 2016-02-26 15:48:51 +01:00
doc Move ironic-discoverd to ironic-inspector 2016-02-29 11:47:52 +08:00
docker Make Heka send logs to Elasticsearch 2016-03-05 11:51:49 +01:00
etc Remove keystone admin token 2016-03-01 00:14:06 +00:00
kolla Add timeout to requests.get() in kolla/cmd/build.py 2016-02-28 00:54:47 +00:00
specs Merge "Logging with Heka spec" 2016-02-17 11:14:44 +00:00
tests Move ironic-discoverd to ironic-inspector 2016-02-29 11:47:52 +08:00
tools Make Heka send logs to Elasticsearch 2016-03-05 11:51:49 +01:00
.gitignore Added *.DS_Store To .gitignore 2016-02-24 15:36:51 -05:00
.gitreview Update .gitreview for project rename 2015-09-11 20:57:54 +00:00
.testr.conf Merge "Revert "Capture the log in default"" 2016-01-19 15:36:52 +00:00
LICENSE Add ASL license 2014-09-20 17:29:35 -07:00
loc Fix up loc with change to devenv 2015-10-12 09:02:30 -07:00
README.rst Tidy up README.rst 2016-02-17 17:21:00 -07:00
requirements.txt Updated from global requirements 2016-02-26 01:48:31 +00:00
setup.cfg Remove unexisting data_files in setup.cfg 2016-02-02 17:46:17 +01:00
setup.py Updated from global requirements 2015-11-13 17:11:37 +00:00
test-requirements.txt Updated from global requirements 2016-01-19 13:48:15 +00:00
tox.ini Remove the tox linters env 2016-02-26 19:26:26 +08:00

README.rst

Kolla Overview

The Kolla project is a member of the OpenStack Big Tent Governance. Kolla's mission statement is:

Kolla provides production-ready containers and deployment tools for
operating OpenStack clouds.

Kolla provides Docker containers and Ansible playbooks to meet Kolla's mission. Kolla is highly opinionated out of the box, but allows for complete customization. This permits operators with little experience to deploy OpenStack quickly and as experience grows modify the OpenStack configuration to suit the operator's exact requirements.

Getting Started

Learn about Kolla by reading the documentation online docs.openstack.org.

Get started by reading the Developer Quickstart.

Kolla provides images to deploy the following OpenStack projects:

As well as these infrastructure components:

  • Ceph implementation for Cinder, Glance and Nova
  • Openvswitch and Linuxbridge backends for Neutron
  • MongoDB as a database backend for Ceilometer and Gnocchi
  • RabbitMQ as a messaging backend for communication between services.
  • HAProxy and Keepalived for high availability of services and their endpoints.
  • MariaDB and Galera for highly available MySQL databases

Docker Images

The Docker images are built by the Kolla project maintainers. A detailed process for contributing to the images can be found in the image building guide.

The Kolla developers build images in the kollaglue namespace for every tagged release and implement an Ansible deployment for many but not all of them.

You can view the available images on Docker Hub or with the Docker CLI:

$ sudo docker search kollaglue

Directories

  • ansible - Contains Ansible playbooks to deploy Kolla in Docker containers.
  • demos - Contains a few demos to use with Kolla.
  • dev/heat - Contains an OpenStack-Heat based development environment.
  • dev/vagrant - Contains a vagrant VirtualBox/Libvirt based development environment.
  • doc - Contains documentation.
  • etc - Contains a reference etc directory structure which requires configuration of a small number of configuration variables to achieve a working All-in-One (AIO) deployment.
  • docker - Contains jinja2 templates for the docker build system.
  • tools - Contains tools for interacting with Kolla.
  • specs - Contains the Kolla communities key arguments about architectural shifts in the code base.
  • tests - Contains functional testing tools.

Getting Involved

Need a feature? Find a bug? Let us know! Contributions are much appreciated and should follow the standard Gerrit workflow.

  • We communicate using the #kolla irc channel.
  • File bugs, blueprints, track releases, etc on Launchpad.
  • Attend weekly meetings.
  • Contribute code.

Contributors

Check out who's contributing code and contributing reviews.