Previous work on Watcher added the Docker images, this
change adds the ansible configuration.
There is support for HA, via haproxy to balance across the
Watcher API hosts.
There is also a hook into nova.conf to conditionally add
Nova compute Host metrics via Ceilometer if Watcher is enabled.
This defaults to enabled false.
Change-Id: I8763528bb6ff12943b810212c71396d2d7cf6836
Partial-bug: #1598929
Partially-implements: bp watcher
Signed-off-by: Dave Walker (Daviey) <email@daviey.com>
If one is using an existing or alternative mysql deployment and sets
enable_mariadb to false, heka would fail to deploy.
TrivialFix
Change-Id: Ieb793e11a40358e5d4fc1f3955f200e2cddef4f5
All the images follow the same naming schema where the container
name is part of the ansible variable name. Currently, the
kolla_toolbox container is named ansible_image_full. Change
this to match what the container name is.
Could impact operators who are specifying ansible_image_full
in globals.yml.
TrivialFix
Change-Id: I9f0cbe4bd69ce17489310a1f5734a254e6ebe27b
Added general_log to ansible/roles/mariadb/templates/galera.cnf.j2
to improve mariadb logging.
This will be helpful to debug mariadb issues especially when
mariadb is scaled.
Test results of this patch set are at:
http://paste.openstack.org/show/492852/
Change-Id: I80438d1bbdd1ed2a1f47489c6f9c45b8107340a0
Closes-Bug: #1563668
Currently Heka writes the keepalived logs in
/var/log/kolla/haproxy/keepalived.log.
This commit changes this to /var/log/kolla/keepalived/keepalived.log.
Closes-Bug: #1565499
Change-Id: I3033097bd77ddbf72948697b34a6a499ea903083
Proposed patch to change keystone and heka log dir
from /var/log/kolla/apache2/ to /var/log/kolla/keystone/
Closes-Bug: #1560620
Change-Id: I70c65ceba5a301cc56880313ca86f01bd35676cb
This is consistent with the patch set in
https://review.openstack.org/#/c/296012/ for stable/mitaka
The new lua parser for libvirtd.log so that libvirtd.log can be
parsed in the Kibana dashboard will be implemented in a separate
patch set (in both master and backported to Mitaka).
TrivialFix
Related-Bug: 1560298
Related-Id: I1bc8bdb3180f30dc69af28a35fa057ac801bde0f
Change-Id: I59938a4e99abc676c391e535405417701aae2767
Eric spotted this on IRC. Submitted a bug and patch
on his behalf since he isn't available to work on Kolla
in the next few hours.
Co-Authored-By: Eric Lemoine <elemoine@mirantis.com>
Change-Id: I91efdd540170a391228d247c2e6e533e6c87b951
Closes-Bug: #1560819
Swift and heka were broken by an optimization that resulted in
a regression in commit:
0a1486a2c73275009f7158c911e7bfd683e40a58
This patch fixes that problem.
Change-Id: I02c6cf4de4f807c3ec66e2d4f59f182d0b864ea3
Closes-Bug: #1560788
Changes location of Horizon log, they will be stored on common log volume
kolla_logs.
Change-Id: Ie9d56999a83efd05ab7c3dcb00b4dc42c9bce8f8
Closes-Bug: 1560250
These values are optional only when the services are not enabled.
If the file does not exist we should not warn, but rather inform.
Ceph-mon is an exception here since its bootstrap process means
the files may or may not exist initially.
TrivialFix
Change-Id: Ic02bece76d480e99deecf612036f37abb5604135
Now that we have reconfigure we dont have to copy over all config
files at the same time.
TrivialFix
Change-Id: I866be0f05e604151e081974a64f17bf65787f604
Add cron image and playbook for logrotate.
The "common" Ansible playbook includes configuration files for
logrotate. At this point the operator cannot customize/override the
logrotate configuration.
Closes-Bug: #1553244
Change-Id: Ic9fdda9a273c9ccd90502f0acc7614d2c7157dca
This was an attempt to get storage_interface to work properly but that
work will not be completed and functional this cycle. There are design
topics that need to be discussed about it that were brought to light
by the RAX gate failing for it.
TrivialFix
Change-Id: I65579f9e0e0dcf3fa51c0ea031ff474145457c40
Heka logs almost done. When testing, The collected logs can copy
to the /tmp/logs folder, which will be handle by the gate. This
will be very useful for our gate debug.
TrivialFix
Change-Id: I06ca8ebd13933bfd15cb2fb5e53fc3038b17b8c3
The code in docker/base/start.sh that waits for the log socket does
not work because it includes a bad "space" character after "-S".
This patch changes that character to a real "space" character. It
also sets the SKIP_LOG_SETUP envvar for the Heka container.
TrivialFix
Change-Id: I17a3005df5bbd57678350fc77030e44b5ad6acae
This patch includes changes relative to integrating Heka with
Elasticsearch and Kibana.
The main change is the addition of an Heka ElasticSearchOutput plugin
to make Heka send the logs it collects to Elasticsearch.
Since Logstash is not used the enable_elk deploy variable is renamed
to enable_central_logging.
If enable_central_logging is false then Elasticsearch and Kibana are
not started, and Heka won't attempt to send logs to Elasticsearch.
By default enable_central_logging is set to false. If
enable_central_logging is set to true after deployment then the Heka
container needs to be recreated (for Heka to get the new
configuration).
The Kibana configuration used property names that are deprecated in
Kibana 4.2. This is changed to use non-deprecated property names.
Previously logs read from files and from Syslog had a different Type
in Heka. This is changed to always use "log" for the Type. In this
way just one index instead of two is used in Elasticsearch, making
things easier to the user on the visualization side.
The HAProxy configuration is changed to add entries for Kibana.
Kibana server is now accessible via the internal VIP, and also via
the external VIP if there's one configured.
The HAProxy configuration is changed to add an entry for
Elasticsearch. So Elasticsearch is now accessible via the internal
VIP. Heka uses that channel for communicating with Elasticsearch.
Note that currently the Heka logs include "Plugin
elasticsearch_output" errors when Heka starts. This occurs when Heka
starts processing logs while Elasticsearch is not yet started. These
are transient errors that go away when Elasticsearch is ready. And
with buffering enabled on the ElasticSearchOuput plugin logs will be
buffered and then retransmitted when Elasticsearch is ready.
Change-Id: I6ff7a4f0ad04c4c666e174693a35ff49914280bb
Implements: blueprint central-logging-service
The storage hostname is used in many conf files all over the place and
should be registered upon each run for every service
TrivialFix
Change-Id: I1d5a20cb6d51cee9f529cf2e1fb144158f7718a9
Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.
This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.
kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn
The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.
Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla
Currently Heka fails to parse the RabbitMQ logs. There are two
problems:
1. The rabbit-sasl.log file is ignored but the file_match expression
does not match.
2. The delimiter used in the RegexSplitter makes Heka stop on the
very first log entry. '\n\n(=[^=]+====' (with two \n's) is
a better delimiter. deliver_incomplete_final is used to get the
final log entry.
TrivialFix
Change-Id: I94720340d5b2d6fd5d7641b9ff58733f6cd882ee
HAProxy: change to use option forwardfor to pass origin IP address
to backend via X-Forwarded-For header
Keystone: Apache does the audit logs for keystone. Change the
LogFormat to display the passed address instead of the connection
address which is that of the load balancer.
Nova, Cinder, Glance: these services can make use of the address
passed in X-Forwarded-For. With this setting the API logs for
these services include the client IP address.
Change-Id: Ia861ecc11a7c7d463d0366586926d1a842853f69
Closes-Bug: #1548935
This bootstrap was non-idempotent. This patch follows the style
first implemented with nova to make this idempotent.
TrivialFix
Change-Id: Id04e59c5274a7d8a5bffd3ce018f3bbb84839d75
