1313 Commits

Author SHA1 Message Date
Jonathan Rosser
b9c995415c Collect hardware facts for elastic data nodes
This is needed to have the required ansible facts for doing
memory calculations.

Change-Id: I8f260b723bbdca4c26ba62b01fcf8cb589ed31ce
2023-03-08 13:55:01 +00:00
Jonathan Rosser
1afff2760f Ensure python3-pexpect is installed on Ubuntu Bionic
CI jobs are failing on Bionic as this is missing, probably becasue
ansible is finding a python3 interpreter when previously it found
python2.

Change-Id: I80c66bd93e57d5e46af85cbf4a271a90d97533cc
2023-03-08 13:27:09 +00:00
Jonathan Rosser
9b1ea92f88 Remove "warn" parameter from command module
This is removed in ansible 2.14.

Change-Id: Ic228ba62d33e3056be71a1d2285b068fc82f32d9
2023-01-10 08:40:13 +00:00
Dmitriy Rabotyagov
2a6c571078 Update tox.ini to work with 4.0
With tox release of 4.0, some parameters were deprecated and are ignored now
which causes tox failures. One of the most spread issues we have is using
`whitelist_externals` isntead of `allowlist_externals`


Change-Id: I2b9d561d7e7a63c3540197b4ebc58f1e5b8846ec
2022-12-27 17:53:17 +01:00
Kirill Tyugaev
11efaaec95 Improve venv cleanup regexp discovery
Change-Id: If52687cb1fabed575eb3467216163b64b0c89635
2022-12-20 16:13:52 +03:00
Zuul
44ae9af3f5 Merge "kibana: fix installation on ubuntu jammy and other py3 only systems" 2022-11-25 13:31:25 +00:00
Andrew Bonney
2f1449b04c kibana: fix installation on ubuntu jammy and other py3 only systems
Change-Id: I0231b0b00277f306066d4827988f9640d9a1fba7
2022-11-11 10:29:27 +00:00
Andrew Bonney
e13281a246 Set permissions on elastic data directories
Change-Id: I70cc0192da9809798c90859937cd339ac1e1a971
2022-11-11 10:26:37 +00:00
Andrew Bonney
2b65764a2e Enable cluster ID monitoring setting for beats using logstash
Change-Id: I59e7c92ebd91ceca161577e73d9f24826abedda1
2022-11-02 09:47:04 +00:00
Andrew Bonney
5437ddbd86 Add support for enabling ELK stack security
Change-Id: I661662c0784010ca2fcc3b3d31df1a1d79dbed1e
2022-11-02 09:17:45 +00:00
Andrew Bonney
29080ab4f6 Add support for apt package pinning
Adds variables to pin elastic package versions to ensure they
can't move out of sequence. An individual variable is added
for journalbeat given it is withdrawn from 7.16.

Includes a version bump to a more recent 7.x release which is
still compatible with this role

Change-Id: I5e6c42398acf70012bf879f41953d409abbd89f4
2022-10-28 14:53:58 +01:00
Andrew Bonney
1480703942 Fix role installation in tests
Previous collection addition commit assumed multiple yaml files
could be passed in at once.

Change-Id: Ia23f1102e76f57d20af755bd0fb26018c1cf632a
2022-10-28 14:28:03 +01:00
Matthew Thode
f74ad7d93b
gather facts for elk_metrics
Change-Id: I418b7d050ba30e212b09c21b289900106a31bdf4
Signed-off-by: Matthew Thode <mthode@mthode.org>
2022-09-26 22:32:09 -05:00
James Denton
effd1542c9 Correct OVNL3RouterPlugin path
This patch updates the OVNL3RouterPlugin path to
match previously updated documentation.

Change-Id: I7873a2c2575a7a1184d9b2c5534b7135eee3cc0d
2022-09-08 19:00:34 -05:00
James Denton
ff7cbf5548 Update MNAIO to use Ansible Collections
This patch updates various playbooks within the MNAIO to use ansible
collections, which is required for the latest Ansible versions.

Change-Id: Ia7694e6b182dc8898b56730537cc99759a722b20
2022-07-23 21:01:25 +00:00
Zuul
6cc3b9b03d Merge "Address ELK deprecations necessary before v8 upgrades" 2022-06-01 14:21:15 +00:00
Zuul
a2db066e0e Merge "beats: allow definition of custom hostname where necessary" 2022-06-01 14:21:13 +00:00
Zuul
b8f1466605 Merge "Serialise operations on components which may be highly available" 2022-06-01 10:20:52 +00:00
Andrew Bonney
7dbfa3025c beats: allow definition of custom hostname where necessary
Adds a variable to enable overriding of the name which beats
use to identify themselves to elasticsearch.

Change-Id: Ic1353431336394d03aadee8dd03333ef138e437e
2022-06-01 08:39:24 +01:00
Andrew Bonney
6f117524db Address ELK deprecations necessary before v8 upgrades
Various configuration options have changed since v7 was initially
released. This patch addresses the obvious issues in ElasticSearch
and Kibana.

Change-Id: Ie9b6bf768dabe258171e5f53cfa02c462f44a557
2022-06-01 08:37:49 +01:00
Dmitriy Rabotyagov
a5d31f3b47 Switch sphinx language to en
With sphinx release of 5.0.0, they changed default for language variable
to 'en' from None. With that current None valuable is not valid and should
not be used.


Change-Id: Id540deef52eea623aacff607709def993e869e05
2022-05-30 16:01:26 +02:00
Andrew Bonney
825aa0d9ea Serialise operations on components which may be highly available
Minimal playbook changes to reduce the risk out outages. This
could be further improved by contacting the elasticsearch API
during actions to ensure that each node recovers before moving
on to the next.

Change-Id: If106efd5700c770a8256adb11cb4735d11cbed5c
2022-05-30 08:41:04 +01:00
Zuul
dff4647b70 Merge "Add journald input support to filebeat" 2022-05-26 16:29:45 +00:00
Zuul
ccf14390f1 Merge "Update beat templates from 7.11 reference" 2022-05-26 16:29:43 +00:00
Zuul
60be6b1ae9 Merge "kibana: fix bug in templating variable selection" 2022-05-26 16:29:39 +00:00
Zuul
82e542eff1 Merge "kibana: add configuration option for base url" 2022-05-26 16:27:19 +00:00
Zuul
d2b39a0032 Merge "beats: fix potential for indexes to be set up incorrectly" 2022-05-26 16:27:17 +00:00
Andrew Bonney
34d981cab6 kibana: add configuration option for base url
Recent versions of ELK complain when this URL is not
set. This parameters allows users to set the URL which
is used to access Kibana.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-ops/+/843423

Change-Id: I4cafc29bde353f4cf1286af73617b1ad3d5fea3d
2022-05-26 13:26:01 +01:00
Andrew Bonney
e7e62f7b98 kibana: fix bug in templating variable selection
A long standing typo means that highly available
kibana instances could become confused. This becomes
more apparent when ELK security is enabled.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-ops/+/843423

Change-Id: I7fa759da107337c78c44b950b86a769df5e20d87
2022-05-26 13:23:13 +01:00
Andrew Bonney
79c11c0348 Add journald input support to filebeat
Journalbeat is removed in ELK 7.16 and merged into
filebeat. This patch adds defaults which allow this
input to be enabled via a single variable.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-ops/+/843423
Change-Id: I5563d6493a6a53374170c4bdaa1edfbff85b300d
2022-05-26 13:21:24 +01:00
Andrew Bonney
27df34985e beats: fix potential for indexes to be set up incorrectly
If a deployment is running a mix of versions, running once
would only run against a single version, missing out the index
setup for other versions of the beats.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-ops/+/843423

Change-Id: Ib6b0e77b96aec33779062a81dbad0931eeaa8806
2022-05-26 13:20:01 +01:00
Erik Berg
f97ebbf990 Update beat templates from 7.11 reference
Depends-On: Ida8668f5548a15a8f597839bd9002585aeea5d1a
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-ops/+/843423

Change-Id: Ib889cc7ac7ad2540031016075cb5baab091bd6e3
2022-05-26 12:19:17 +00:00
Andrew Bonney
7cfcbbeccf elk7: minimal fixes to CI jobs
This is a minimal patch to address failing CI to enable
other patches to be submitted.

Similar patches are likely to be required for other OSA
ops components, but this will be tackled at a later date.

Change-Id: I6020d7b820c1f7d781691df9a1d657aef0ea010b
2022-05-26 12:18:50 +00:00
OpenStack Proposal Bot
f2fe9aa59f Updated from OpenStack Ansible Tests
Change-Id: Icbef33cd5163e4bfe51e6ce1f8fd43848b649462
2022-03-29 17:17:41 +00:00
James Denton
ab91446804 Update MNAIO for Focal
This patch removes legacy support for 14.04/16.04/18.04
on the deploy node and moves the default deploy to
Xena on 20.04 LTS. Root disk size has been bumped to support
upgrades (8 GB -> 12 GB).

Change-Id: I81a13464b9daa90090cb380e2b0d89e5eb8fe89a
2022-02-16 09:46:59 -06:00
Zuul
5c1bd46f91 Merge "Fixed venvs pattern to be more specific" 2021-03-25 11:00:20 +00:00
Adrien Cunin
ddedfcff76 Fixed venvs pattern to be more specific
Closes-Bug: #1910223
Change-Id: Icb58263a2a59f131715a5120652a39ed67673e39
2021-03-25 10:26:18 +01:00
Adrien Cunin
a7de5d89a8 Removed task to cleanup venv tgz
venv tgz in /var/cache/ no longer exist.

Change-Id: Ib91712b5663ec24e29de28e2522046c8dc1b90c6
2021-03-24 15:13:01 +01:00
Zuul
518159ef40 Merge "It's always systemd. Remove references to upstart" 2021-03-15 09:33:38 +00:00
OpenStack Proposal Bot
6fe509e486 Updated from OpenStack Ansible Tests
Change-Id: Ic1bb36734cc11e3f2ab33feccc114bb71ab51f32
2021-03-12 22:17:10 +00:00
Zuul
0fcbc4afaa Merge "Remove tasks for v6.5 beats (aarch64) from elk7" 2021-03-12 14:45:01 +00:00
Zuul
09a310f758 Merge "Allow longer timeouts for elasticsearch.service" 2021-03-12 14:43:08 +00:00
Erik Berg
ae879fc6e6 It's always systemd. Remove references to upstart
Upstart was last seen on ubuntu-14

Change-Id: I5cfbcc10fd1fbb76e5606e3f550327c4c8f3ff45
2021-03-11 16:31:32 +00:00
Erik Berg
16782bf43b Remove tasks for v6.5 beats (aarch64) from elk7
These beats don't seem to be available at the urls, and they're
version 6.5.0.

Change-Id: I0022250dd58ce0f39c02e0733a17e53d753a2675
2021-03-11 14:45:07 +00:00
Erik Berg
501803569e remove argsplat warning
Since we can't be sure how deployers would override this setting,
we try applying all parameters, and omit those not defined.

Change-Id: Ie83b708ac7ee094e5ab0a72099d83418cc4cb58a
2021-03-10 19:11:45 +00:00
Daniel Pawlik
ea16ee62de Change get-pip urls
The path for get-pip.py script in versions 3.2 and 3.4 have been changed
with this commit [1].

[1] 2360f025eb

Change-Id: Iedef41b53c4e5fa28dac8206c5ba39dab5033815
2021-03-08 15:14:05 +01:00
Erik Berg
de88293f24 Allow longer timeouts for elasticsearch.service
/usr/lib/systemd/system/elasticsearch.service comes with a
TimeoutStartSec=75, which is not enough time for a somewhat busy
E5-2430L v2 @ 2.40GHz to get things going.

We don't want to permanently override that with a default of 75 in
case they decide to bump or remove that timeout in the future.

Change-Id: Ida8668f5548a15a8f597839bd9002585aeea5d1a
2021-03-05 13:01:55 +00:00
Zuul
397ed3f1eb Merge "apm-server gets fussy about file-permissions" 2021-03-04 21:56:09 +00:00
Zuul
baadacde86 Merge "Collect keystone apache federation files" 2021-03-04 16:47:59 +00:00
Erik Berg
dc262581c6 apm-server gets fussy about file-permissions
"Exiting: error loading config file: config file
("/etc/apm-server/apm-server.yml") must be owned by the user identifier
(uid=0) or root"

apm-server is normally run under its own user, and permissions on
/etc/apm-server/apm-server.yml reflect that, so we disable the check
while running running `setup` under root.

Change-Id: If25b446b49a29d1ac86261fe632bb6fe791f13ce
2021-03-03 15:54:23 +01:00