10766 Commits

Author SHA1 Message Date
Zuul
9b03f51c7b Merge "Make fluentd-elasticsearch configuration more robust" 2019-12-10 21:01:32 +00:00
Pierre Riteau
49f334284b Template custom HAProxy service configuration
Change Id84e3b6e62e544582d6917047534e846e026798d added support for
custom HAProxy service config using a plain copy of files in services.d.

Use a template action instead of a copy so that we can use variables and
iterate over group of hosts.

Change-Id: I1f07785932de4e4540422bd18af95241f05a67bf
2019-12-10 15:31:45 +01:00
Zuul
516cb3d95d Merge "Neutron: Added loading ip6_tables kernel module" 2019-12-10 13:56:10 +00:00
Radosław Piliszek
7714ea724f CI: Test Swift
Partially-implements: blueprint kolla-deployment-scenario-testing
Change-Id: I82266829dba47eac81f440a53706c2e9064b8e62
2019-12-10 13:19:17 +00:00
Hongbin Lu
c6481469e6 CI: Test Zun with Cinder LVM backend (iSCSI)
Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Depends-on: https://review.opendev.org/694476
Change-Id: I6e7f2f4229c8b579dcc17dacffeb74160875ae29
2019-12-10 13:18:59 +00:00
Zuul
a46292f6b6 Merge "Support configuration of Docker client timeout" 2019-12-10 12:38:57 +00:00
Zuul
8f3e6aca99 Merge "Use local python interpreter for keystone cron generator" 2019-12-10 11:15:55 +00:00
Zuul
6dbddd4ee7 Merge "CI: Use python 3 for local kolla-ansible execution" 2019-12-10 11:15:53 +00:00
Zuul
558b745ae1 Merge "Support deploying Elasticsearch Curator" 2019-12-10 10:52:46 +00:00
Zuul
06cf664cfb Merge "Improve Apache logging" 2019-12-10 10:44:45 +00:00
Zuul
4b62841384 Merge "CI: Move ansible installation & configuration to Ansible" 2019-12-10 10:39:15 +00:00
Zuul
1907607a9d Merge "CI: Don't fail on expected critical log messages" 2019-12-09 22:33:36 +00:00
Mark Goddard
e46d602536 Use local python interpreter for keystone cron generator
We generate the keystone cron schedule via a python script on localhost.
Currently this always uses 'python', however this may not be available
on some systems.

This change switches to use the same python interpreter as used by
ansible-playbook.

Partially-Implements: blueprint python-3

Change-Id: I6007f8d6880f418a503766cec21a330c44e5b80f
2019-12-09 17:07:10 +00:00
Mark Goddard
a5408f425b CI: Use python 3 for local kolla-ansible execution
This change switches the CI jobs to use python 3 for local execution of
the kolla-ansible commands.

For upgrades, we use python 2 for the previous (Train) deploy, then
reinstall using python 3 for the (Ussuri) upgrade.

NOTE: This is separate from the python interpreter used on remote hosts,
which is configured via ansible_python_interpreter.

Partially Implements: blueprint python-3
Related: blueprint drop-py2-support

Change-Id: I5bdc165f68b7bde1f9ef30fe8216f2a44e6d4706
2019-12-09 17:06:19 +00:00
Zuul
73b634415c Merge "Support python 3 in kolla-ansible script" 2019-12-09 16:54:07 +00:00
Jan Vondra
59b058e23d Neutron: Added loading ip6_tables kernel module
ip6_tables is needed for HybridFwDriver

Change-Id: I66c50b74a89c046dc59e59dd2422a80e0642ab72
2019-12-09 16:26:13 +00:00
Doug Szumski
d4381456fd Support deploying Elasticsearch Curator
This allows users to supply an Elasticsearch Curator actions file
to manage log retention [1]. Curator then runs on a cron job, which
defaults to every day. A default curator actions file is provided,
which can be customised by the end user if required.

[1] https://www.elastic.co/guide/en/elasticsearch/client/curator/current/actionfile.html

Change-Id: Ide9baea9190ae849e61b9d8b6cff3305bdcdd534
2019-12-09 14:49:05 +00:00
Mark Goddard
a32cddca49 Fix fluentd parsing of WSGI logs
WSGI log files use a different input configuration than OpenStack log
files. Currently this depends on log files matching either *-access.log
or *-error.log. Some services use *_access.log or *_error.log, so are
not parsed correctly.

This change modifies the fluentd configuration to accept an underscore
or hyphen for WSGI log file names.

Change-Id: I566d6cac0b6749054fd5422ec8f36f99dacb1db7
Closes-Bug: #1720371
2019-12-09 14:12:14 +00:00
Krzysztof Klimonda
0c573062fc Make fluentd-elasticsearch configuration more robust
Enable reconnect_on_error option so that ES plugin re-establishes
a new session to the ES cluster on errors. Also, enable buffering
to the file, so that the buffer survives container restarts.

Co-Authored-By: Michal Nasiadka <mnasiadka@gmail.com>
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Co-Authored-By: Doug Szumski <doug@stackhpc.com>
Closes-Bug: #1830724
Change-Id: Ia40685b9d4fc02194e03c8791ddeb3d29d7f07f6
2019-12-09 13:38:36 +00:00
Mark Goddard
c320077f08 CI: Move ansible installation & configuration to Ansible
Continue to reduce the scope of setup_gate.sh. Allows us to more easily
select python 2 or 3.

Change-Id: If2eeeacbbbdf58afb765b4a39772b5a1af7b952b
Partially Implements: blueprint python-3
2019-12-09 13:30:35 +00:00
Mark Goddard
3a6577bada Support python 3 in kolla-ansible script
The kolla-ansible script uses a python interpreter to detect the
location of playbooks and to check that the kolla_ansible python module
is importable. This change updates the script to support a python
interpreter named python or python3.

Partially Implements: blueprint python-3

Change-Id: Id5dcc53cc5dd9780632c04b6b73c56ea5da484a2
2019-12-09 13:29:43 +00:00
Mark Goddard
2f7640d396 CI: Don't fail on expected critical log messages
There is a number of critical log messages that we see in CI from time
to time. While these should be fixed, let's not fail jobs unnecessarily.

This change introduces one expected critical message in
placement-api.log:

    Failed to fetch token data from identity server

Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Related-bug: #1847727
Change-Id: I92ad0be70ed05925612f0c709907ab62280326b8
2019-12-08 08:36:33 +00:00
Radosław Piliszek
58b5acbf65 Default to etcd3gw driver for etcd-based coordination
To fix instability and availability issues:

etcd3 is not available in repos for binary kolla images.

etcd3 does not support eventlet-based services [1].

[1] https://review.opendev.org/466098

Change-Id: I430bab735da204fc81696130b17931a89214c876
Closes-bug: #1852086
Closes-bug: #1854932
2019-12-07 08:52:12 +00:00
Michal Nasiadka
3f55b87069 Improve Apache logging
Currently we don't put global Apache error logs into /var/log/kolla,
this change adds statements that redirect those logs there.

Adapted the logfile names to catch into openstack wsgi logging fluentd
input config and existing logrotate cron entries.

Change-Id: I21216e688a1993239e3e81411a4e8b6f13e138c2
2019-12-06 13:11:49 +00:00
Zuul
c66a00e8f1 Merge "Start README.rst with a better title" 2019-12-05 09:57:15 +00:00
Scott Solkhon
68b9224425 Add also_notifies to Infoblox backend for Designate
Change-Id: Ia02f83dfaaba53f95e373b2b2be3f74cfb7ae578
Closes-Bug: #1855085
2019-12-04 15:30:13 +00:00
Zuul
f52082abf7 Merge "Fix Swift with kernel 4.19 and later" 2019-12-04 10:16:10 +00:00
Zuul
f8862c6cc2 Merge "[Train] Fix Tacker config for the local Glance store" 2019-12-03 10:21:34 +00:00
Erik Olof Gunnar Andersson
619ccf1242 Added senlin-conductor and senlin-health-manager
Depends-On: https://review.opendev.org/692948/
Depends-On: https://review.opendev.org/692691/
Change-Id: I07827b896d36c3723697540fcff164224f6729af
2019-11-30 16:49:33 -08:00
Scott Solkhon
991bdc5f55 Fix Prometheus template generation
In a deployment where Prometheus is enabled and
Alertmanager is disabled the task "Copying over
prometheus config file" in
'ansible/roles/prometheus/tasks/config.yml' will
fail to template the Prometheus configuration file
'ansible/roles/prometheus/templates/prometheus.yml.j2'
as the variable 'prometheus_alert_rules' does not
contain the key 'files'. This commit fixes this bug.

Change-Id: Idbe1e52dd3693a6f168d475f9230a253dae64480
Closes-Bug: #1854540
2019-11-30 22:54:22 +00:00
Radosław Piliszek
21488710d6 Fix Swift with kernel 4.19 and later
We mount Swift volumes with xfs.
The 'nobarrier' option we used was made noop [1]
and deprecated [2] (with warning) in kernel 4.10.
In 4.19 it was removed [3] resulting in an error
when using e.g. Debian Buster as host.
The noop patch was backported to CentOS 7 so
it is safe to remove this option with no behavior
change and backport to where needed.
Ubuntu Bionic uses 4.15 which only warns.
CentOS 8 uses 4.18 which only warns as well.
Debian Buster uses 4.19 exactly which breaks.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2291dab2c9d1880efd19469df2042e2277c8b7a4
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cf4573d899cd80d8578c050061dc342f99f3a32
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1c02d502c20809a2a5f71ec16a930a61ed779b81

Change-Id: I006dea21321146c7fc738d0b41c401b72d271a99
Closes-bug: #1800132
2019-11-30 12:56:36 +01:00
Zuul
baef98a232 Merge "CI: Refactor a lot" 2019-11-28 12:16:18 +00:00
Mark Goddard
01050dc01f Support configuration of Docker client timeout
Adds support for configuration of the Docker client timeout via
'docker_client_timeout'.

This change also increases the default timeout to 120 seconds, as we
sometimes see timeouts in CI and heavily loaded or underpowered
environments. Increasing 'docker_client_timeout' further may be helpful
in cases where Docker reports 'Read timed out'.

Change-Id: I73745771078cb2c0ebae2b1d87ba2c4c12958d82
Closes-Bug: #1809844
2019-11-28 10:10:00 +00:00
Q.hongtao
4405c848ac Start README.rst with a better title
Now that we are using gitea the contents of our README.rst are
more prominently displayed. Starting it with a "Team and repository
tags" title is a bit confusing. This change makes it start with the
name of the project instead.

Change-Id: Icc946b883e769aaaf2300477ab3e4d9af1e7dc74
2019-11-28 09:51:40 +08:00
Zuul
734e86a02f Merge "Split out OVS/Linuxbridge agent configs from ml2_conf.ini" 2019-11-27 18:14:43 +00:00
Zuul
296fdfad46 Merge "Fix 'nova_cephx_raw_key' is undefined" 2019-11-27 16:02:17 +00:00
Pierre Riteau
4234cc4b5b [neutron] Adjust neighbour table thresholds
When clouds have a large number of hosts, the default size of the ARP
cache is too small. The cache can overflow, which means that the system
has no way to reach some IP addresses.

Increasing threshold limits addresses the situation, in a reasonably
safe way (the maximum impact is 5MB or so of additional RAM used).

More context on this issue:

* http://man7.org/linux/man-pages/man7/arp.7.html
* https://bugs.launchpad.net/charm-nova-compute/+bug/1780348
* https://bugs.launchpad.net/fuel/+bug/1488938
* https://bugs.launchpad.net/tripleo/+bug/1690087
* 0583a0c949
* 3dd21ea62a
* 1651a1805a

Change-Id: I60c871e8eb9f2c086818ff077987f2390930800c
Closes-Bug: #1844349
2019-11-27 16:32:47 +01:00
Michal Nasiadka
0947974ff6 Split out OVS/Linuxbridge agent configs from ml2_conf.ini
Change-Id: I799993728112a525e34cfbc4e786a10f0ed03be9
2019-11-27 13:42:48 +00:00
Mark Goddard
6c783b74fc Add internal TLS variables to globals.yml
Change-Id: I283ccee11e360020c3e537b2f55a6be02cbdb2dd
Related: blueprint add-ssl-internal-network
2019-11-26 17:52:13 +00:00
Mark Goddard
692d176cd5 Fix kolla_ansible python module check
The kolla-ansible script now (Train+) checks whether the kolla_ansible
python module is installed, and emits the following message if not:

ERROR: kolla_ansible has to be available in the PYTHONPATH (e.g.
installed)

It is supposed to exit 1 if this check fails, but does not.

This change fixes this by removing a subshell from the kolla-ansible
script.

Change-Id: I2c1a7398f2b8f876abcf2508874835154cb4ce57
Closes-Bug: #1854049
2019-11-26 17:50:15 +00:00
Radosław Piliszek
a2fc684164 CI: Refactor a lot
Separate upgrade logic to is_upgrade job var and rename
scenarios to match.

Rename "ACTION" to "SCENARIO" (as it is a scenario).

Separate testing of dashboard (aka Horizon) and increase
its timeout to 5 minutes (CentOS 7 slow as always).

Separate initialization of core OpenStack.

Use gate setup script from ./tests/

Remove useless tox setupenv.

Do not deploy Heat when not really necessary.

Change-Id: I4fca319ccc3de7188f8b7b44c9c71321e3899467
2019-11-26 17:30:14 +01:00
Mark Goddard
cacf308170 Add python3-dev[el] to bindep.txt
We should probably have had this in previously.

Change-Id: I14f690194085729bf500ed707d3bb99b1c055845
2019-11-26 10:59:31 +00:00
Radosław Piliszek
c5f9ea9590 [Train] Fix Tacker config for the local Glance store
It turned out the previous fix ([1]) was incomplete.
Additionally, it seems we have to limit Tacker server
to one instance co-located with conductor.

[1] https://review.opendev.org/684275
commit b96ade3cf01009d822f85744efee523127f2674c

Change-Id: I9ce27d5f68f32ef59e245960e23336ae5c5db905
Closes-bug: #1853715
Related-bug: #1845142
2019-11-25 20:13:15 +01:00
Zuul
119523fd5b Merge "CI/Debian: Push images after building" 2019-11-25 17:09:30 +00:00
Andrei Nistor
6ff6337230 Fix 'nova_cephx_raw_key' is undefined
When using external ceph without nova integration, kolla-ansible fails
because 'nova_cephx_raw_key' is undefined.
This patch fixes the issue by applying the 'default' filter to
prevent failure on undefined.
The change in behavior was introduced by [1].

[1] https://review.opendev.org/689753
commit 44709f413246a05b891f250e88c43226e1f1b0c9

Change-Id: I2fdca1a6a78e78623733a387a2d8c7e29d449083
Closes-Bug: #1853862
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
2019-11-25 16:33:22 +00:00
Zuul
f8ec9f714b Merge "Remove endpoint_type from neutron.conf [DEFAULT] group" 2019-11-25 14:27:55 +00:00
Zuul
83eff5f493 Merge "Fix hard-coded admin project name and username in blazar task" 2019-11-25 12:38:03 +00:00
Pierre Riteau
c697dc98bc Remove endpoint_type from neutron.conf [DEFAULT] group
This option appears to have been removed in 2015 in Neutron change
I9cf36e1fd3a009c175e0d475af407a30f4e5c408.

Change-Id: Ib2d94743aeeff328c900ab1607900844acb4462b
2019-11-25 12:02:55 +01:00
Zuul
e04d03e764 Merge "Default to disabling haproxy for memcached" 2019-11-25 10:30:48 +00:00
Zuul
6fc3d762d6 Merge "CI: Wait for Zun to delete the test container" 2019-11-24 16:47:41 +00:00