This patch includes changes relative to integrating Heka with
Elasticsearch and Kibana.
The main change is the addition of an Heka ElasticSearchOutput plugin
to make Heka send the logs it collects to Elasticsearch.
Since Logstash is not used the enable_elk deploy variable is renamed
to enable_central_logging.
If enable_central_logging is false then Elasticsearch and Kibana are
not started, and Heka won't attempt to send logs to Elasticsearch.
By default enable_central_logging is set to false. If
enable_central_logging is set to true after deployment then the Heka
container needs to be recreated (for Heka to get the new
configuration).
The Kibana configuration used property names that are deprecated in
Kibana 4.2. This is changed to use non-deprecated property names.
Previously logs read from files and from Syslog had a different Type
in Heka. This is changed to always use "log" for the Type. In this
way just one index instead of two is used in Elasticsearch, making
things easier to the user on the visualization side.
The HAProxy configuration is changed to add entries for Kibana.
Kibana server is now accessible via the internal VIP, and also via
the external VIP if there's one configured.
The HAProxy configuration is changed to add an entry for
Elasticsearch. So Elasticsearch is now accessible via the internal
VIP. Heka uses that channel for communicating with Elasticsearch.
Note that currently the Heka logs include "Plugin
elasticsearch_output" errors when Heka starts. This occurs when Heka
starts processing logs while Elasticsearch is not yet started. These
are transient errors that go away when Elasticsearch is ready. And
with buffering enabled on the ElasticSearchOuput plugin logs will be
buffered and then retransmitted when Elasticsearch is ready.
Change-Id: I6ff7a4f0ad04c4c666e174693a35ff49914280bb
Implements: blueprint central-logging-service
Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.
This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.
kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn
The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.
Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla
Working towards the blueprint that will add TLS protection
for the external endpoints, kolla needs certificates.
When kolla deploys OpenStack, the external VIP will need
a server side certifcate. Clients that access those endpoints will
need the public CA certificate that signed that certificate.
This ansible script will create these two certificates to make
it easy to use TLS in a test environment. The generated
certificate files are:
/etc/kolla/certificates/haproxy.pem (server side certificate)
/etc/kolla/certificates/haproxy-ca.pem (CA certificate)
The generated certificates are not suitable for use in a
production environment, but will be useful for testing and
verifying operations.
Partially-implements: blueprint ssl-kolla
Change-Id: I208777f9e5eee3bfb06810c7b18a2727beda234d
I accidentally merged a patch with +W that does not work on all gates
that infra uses. Specifically it breaks on RAX for unknown reasons.
This is a quick patch to not fully revert, but allow the gate to stay
green while I work out the issue
Change-Id: I02fc66bbc53103f2d71c648d5e2959f10f0130c0
Partially-Implements: blueprint functional-testing-gate
Make the containers and volumes names of ironic to be
consistent with others.
Partially-implements: blueprint ironic-container
Change-Id: If78b020c81157a948c5c3283f1410bcb6e5c1e61
It was run before ansible was installed and did not properly fail.
This prevented the mirrors for apt from being properly used.
TrivialFix
Change-Id: I5331020993b62c8635a4a97dc3a3d2e01e3d4360
When breaking neutron-agents into thin containers, we forgot to update
the cleanup-containers script.
TrivialFix
Change-Id: Idad2eb73a4de7c66109ad5bffcce3173970e2af9
The gate currently doesn't provide any context when ansible fails. This
commit adds a new --verbose option to `kolla-ansible` that is passed to
`ansible-playbook`, and sets the default to -vvv for `kolla-ansible
deploy` in the gate.
TrivialFix
Change-Id: If248a4186b140e1ec0d2e787c27481cec7e81eec
Both apt and pip mirrors are added here
Related-Id: I26f5fc5cec224cf4f4ea361e0f71cecbf320fbb0
TrivialFix
Change-Id: Ib3cb1d2c6fb27887e3ea819d277e21ea3a530f56
Sometimes the keyserver has a mirror failure which results in a failed
gate. Add a retry to help prevent that failure.
TrivialFix
Change-Id: I143626dd6d799b4ea0f82f6649d2155c2f45a115
Docker 1.10 has broken the gate and this patch will correct that
breakage.
The issue comes with rsyslog. Due to a commit in Docker 1.10 [1] we
must change the way we get the log socket for rsyslog. The /dev/
folder will no longer populate as we used it. So instead we simply
make a new socket in a path we control and share that to the correct
location in the containers.
Additionally, adjust the gate for new Docker daemon.
[1] https://github.com/docker/docker/pull/16639
Partially-Implements: blueprint kolla-upgrade
Change-Id: I881a2ecdf6d7b35991e1d38a3f3e60d022d6577f
Remove the '..' in the path like:
/home/jenkins/workspace/gate-kolla-dsvm-deploy-centos-binary/tools/../ansible/inventory/all-in-one
TrivialFix
Change-Id: I80724a9e876ed1826c65e08b55cfa08124d70eb9
This change is needed for clarity. We have a kolla-ansible script.
We have a kolla-mesos repo. We plan to have a kolla-ansible repo.
Already we have had far too much confusion about whether we are
talking about the container or the project. Naming this kolla-toolbox
eliminates all of that confusion and its probably a bit more accurate
of a name too.
Closes-Bug: #1541053
Change-Id: I8fd1f49d5a22b36ede5b10f46b9fe02ddda9007e
Container maybe exit after deployed. But the print_failure
never runs if the kolla-ansible run success.
This PS checks all the containers status after deploy and
failed the test if the container status is exited
TrivialFix
Change-Id: Ia461b280855eda500e143ee1d6cfd5f215eaf6fe
The passed keyfile path is used to decrypt the passwords.yml
which encrypted by the ansible vault.
Partially-Implements: blueprint encrypt-sensitive-info
DocImpact
Change-Id: Ifaebf464478cde99e896c13b375cfaf9282ee576
we can deploy specific services by using tags feature. It is very
useful, especially in develop/test stage.
TrivaiFix
Change-Id: I21cc55e6f1a32a5c57570bc2c2fb9b084e513af8
