10192 Commits

Author SHA1 Message Date
Gaetan Trellu
edb3489820 Adds Qinling Ansible role
Qinling is an OpenStack project to provide "Function as a Service".
This project aims to provide a platform to support serverless functions.

Change-Id: I239a0130f8c8b061b531dab530d65172b0914d7c
Implements: blueprint ansible-qinling-support
Story: 2005760
Task: 33468
2019-05-31 10:25:28 -04:00
Pierre Riteau
abf10736c7 Remove unused [service_credentials] config sections
Several services inherited [service_credentials] config sections which
they don't use in their code.

Change-Id: Iccf4358e85fb3d7ed25bc1762ff532b2c32bea4a
2019-05-31 13:25:00 +01:00
Zuul
41e634e204 Merge "Fix quickstart for virtual environments" 2019-05-31 12:15:34 +00:00
Isaac Prior
84edfd09b6 Fix monasca grafana organisation check
"Create default control plane organisation if it doesn't exist" task
fails when organisation already exists.
The list organisation task currently returns project domain id.
The create organisation task currently provides project domain name.
Change the create task to use default_project_domain_id instead.

TrivialFix

Change-Id: Ice70d55e6729fe55164dcf85e98acdc1d7925209
2019-05-31 12:25:21 +01:00
Zuul
6c042adfa2 Merge "there is no easy_install in Debian 'buster'" 2019-05-31 11:19:20 +00:00
ZijianGuo
e87eb9e788 Fix the document of external ceph for gnocchi
Change-Id: I75955012a839e52281e9a409eeab4a2c8d778cd2
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-05-30 21:42:26 +08:00
Krzysztof Klimonda
fc24e9af7c Add support for elasticsearch TLS and authentication in fluentd
Add options for configuring TLS and authentication for elasticsearch
connections in in fluentd.

Change-Id: I936adc2aeaa3c87081be1c44aa0221caf2124e23
Closes-Bug: #1831078
2019-05-30 13:26:50 +00:00
chenxing
e1fe928885 Add deploy guide stuff to irrelevant file list of zuul
It shouldn't trigger kolla-ansible-base job if changing deploy guide.

Change-Id: I8cb5d3ae7825690279f78ec847b940110bd60216
2019-05-30 21:09:32 +08:00
Dincer Celik
56b2e947a1 Fixes VMTP deploys.
Merged config files outputs to python_path which is incorrect. Changing it
to node_config_directory.

Change-Id: If801f54709114b931677adb605dffb75cfab25cd
Closes-Bug: #1830346
2019-05-29 16:20:01 +03:00
Krzysztof Klimonda
4a60dc21e5 Elevate privileges for copying gnocchi policy.json
Change-Id: I616a30acf38a61265f3005c712767d5221abf5c5
Closes-Bug: #1830723
2019-05-28 12:30:48 +00:00
ZijianGuo
b374ea33e4 Update service_provider for FWaaS v2
As described in document [1], the service_provider should be set to
'FIREWALL_V2:fwaas_db:neutron_fwaas.services.firewall.service_drivers.
agents.agents.FirewallAgentDriver:default' for FWaaS v2.

[1] https://docs.openstack.org/neutron/stein/admin/fwaas-v2-scenario.html#enable-fwaas-v2

Change-Id: Iad3e906af325ba3209a77a91ab2061e515b4a219
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-05-28 17:25:31 +08:00
dommgifer
5eab9cbe1b Remove ceilometer transformers config
According ceilometer commit: 9db5c6c9bfc66018aeb78c4a262e1bfa9b326798
ceilometer was removed transformer support
so, remove about transformers config.

Change-Id: I47fc90aa6dff6d0843c90b27a785e0c6b3d2961e
Close-bug: #1830601
2019-05-27 18:06:46 +08:00
David Moreau Simard
86e9b0831c
Pin ARA to <1.0.0 and use stable setup interface to find callback plugin
ARA 1.0 will be released in the near future and isn't backwards
compatible. Pin it so it doesn't break things unexpectedly.

ARA ships simple setup modules to help figure out the paths
to where things are located.
These are backwards compatible from ARA 1.0 to 0.x.

Change-Id: I3fe3f4082279c2fd9a629605619a97aa5f5b0b73
2019-05-24 10:55:32 -04:00
Doug Szumski
f23901677c Remove obsolete roles middleware
Change I0ca38f2cc7d63b9b47eedb304ba7b00a94816f9a removed the roles
middleware from the example paste pipeline.

Change-Id: Ie9a3b0fef395aaf414407f6bae1ac4bca158240d
2019-05-24 11:31:07 +01:00
Doug Szumski
0351ec2e62 Set log message field name in Monasca Fluentd plugin
Let the Monasca Fluentd output plugin handle mapping of the log
message.

Change-Id: I4a74a91b9b38d5c172397a7e7204e626bcedcfac
Closes-Bug: #1830184
Depends-On: https://review.opendev.org/#/c/660988/
2019-05-23 13:59:43 +01:00
ZijianGuo
c89b901524 Add support for ceilometer custom pipeline.yaml
This file can be modified to adjust polling intervals or other configurations.
We can add a custom 'pipeline.yaml' file to override it.

Change-Id: I325523edc4f7e37db55a2e21fe52e76138e6d114
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-05-23 17:57:16 +08:00
Pierre Riteau
14bf98ddd5 Make "Check if firewalld is installed" task quieter
Stop showing the task has having made changes and silence warning about
not using the yum module (which we could use for the check, but not as
easily).

Change-Id: I9e3608b5db521930409a29981767f468ea234679
2019-05-22 16:38:54 +01:00
ZijianGuo
136210c403 Add support for ceilometer custom event yaml
* event_definitions.yaml:
  This file provides a standard set of events and corresponding traits
that may be of interest.

* event_pipeline.yaml:
  This file can be modified to adjust which notifications to capture and
where to publish the events.

Change-Id: I9c1698e07b65102af9b3ee448ad07f8fa6428b74
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-05-22 20:44:56 +08:00
Pierre Riteau
37899026bf Add ansible_nodename (system hostname) to /etc/hosts
Kolla-Ansible populates /etc/hosts with overcloud hosts using their API
interface IP address. When configured correctly, this allows Nova to use
the API interface for live migration of instances between compute hosts.

The hostname used is from the `ansible_hostname` variable, which is a
short hostname generated by Ansible using the first dot as a delimiter.
However, Nova defaults to use the result of socket.gethostname() to
register nova-compute services.

In deployments where hostnames are set to FQDNs, for example when using
FreeIPA, nova-compute would try to reach the other compute node using
its FQDN (as registered in the Nova database), which was absent from
/etc/hosts. This can result in failures to live migrate instances if
DNS entries don't match.

This commit populates /etc/hosts with `ansible_nodename` (hostname as
reported by the system) in addition to `ansible_hostname`, if they are
different.

Change-Id: Id058aa1db8d60c979680e6a41f7f3e1c39f98235
Closes-Bug: #1830023
2019-05-22 12:27:37 +01:00
Rafael Weingärtner
f0e19ba647 Method to override the default ceilometer meters.yaml via Kolla-ansible
Similar to what we did here: https://review.opendev.org/#/c/655276 but,
 for ceilometer/data/meters.d/meters.yaml file.

The idea is to create a method for operators to manage custom meters
 YAML files via Kolla-ansible. To do that, we enable them (operators)
 to use a folder called by default "meters.d" in their local
 ceilometer configurations, where all of the custom meters YAML files
 will be read from. If this folder exist and has YAML files in it, we
 copy them for the default "/etc/ceilometer/meters.d" path in the
 containers. We do not inject things in the container though. We copy
 the files for the control node, and then we map them via
 ceilometer*.json container configuration files.

Change-Id: I712edcf39bfdb64887e25437f0aff30a45a829dd
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2019-05-21 10:55:10 -03:00
Mark Goddard
0b27baf3ce Fix quickstart for virtual environments
The etc_examples and inventory should be copied from the virtual
environment rather than the system.

Change-Id: I3ac1e057971b7481a0bce2a15351031e51bf97d6
Closes-Bug: #1829435
2019-05-21 10:29:24 +01:00
Marcin Juszkiewicz
6cf60d2385 there is no easy_install in Debian 'buster'
Change-Id: I49588f9ceef60472428d835b95d26a36eaf2b47e
2019-05-20 16:29:26 +02:00
Marcin Juszkiewicz
1c866201e3 prechecks: be nice, use print in Py3 way
Change-Id: I7a99567b3d16830e1db641e82a7c9e4eca1ae4e6
2019-05-20 12:53:13 +02:00
Mark Goddard
002eec951f nova: Fix DBNotAllowed during compute startup
backport: stein, rocky

During startup of nova-compute, we see the following error message:

Error gathering result from cell 00000000-0000-0000-0000-000000000000:
DBNotAllowed: nova-compute

This issue was observed in devstack [1], and fixed [2] by removing
database configuration from the compute service.

This change takes the same approach, removing DB config from nova.conf
in the nova-compute* containers.

[1] https://bugs.launchpad.net/devstack/+bug/1812398
[2] 8253787137

Change-Id: I18c99ff4213ce456868e64eab63a4257910b9b8e
Closes-Bug: #1829705
2019-05-20 10:24:28 +01:00
Rafael Weingärtner
a01294a4de Use'openstack_region_name' in cloudkitty collectors and fetchers
Change-Id: I9fb4151261fba227dda09241bc141826589ad169
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2019-05-17 14:19:40 -03:00
Rafael Weingärtner
0f359dc9cb Use 'openstack_service_workers' as the nb of Cloudkitty workers
Add the ability to Kolla-ansible to manage the 'max_workers' parameter
 in Cloudkitty. We will use the 'openstack_service_workers' variable
 to control the number of workers that Cloudkitty is able to use.

Change-Id: I2f4e7e5c45d71a7e01d1b743d2eb4850cc339419
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2019-05-17 12:53:11 -03:00
Mark Goddard
6c1442c385 Fix keystone fernet key rotation scheduling
Right now every controller rotates fernet keys. This is nice because
should any controller die, we know the remaining ones will rotate the
keys. However, we are currently over-rotating the keys.

When we over rotate keys, we get logs like this:

 This is not a recognized Fernet token <token> TokenNotFound

Most clients can recover and get a new token, but some clients (like
Nova passing tokens to other services) can't do that because it doesn't
have the password to regenerate a new token.

With three controllers, in crontab in keystone-fernet we see the once a day
correctly staggered across the three controllers:

ssh ctrl1 sudo cat /etc/kolla/keystone-fernet/crontab
0 0 * * * /usr/bin/fernet-rotate.sh
ssh ctrl2 sudo cat /etc/kolla/keystone-fernet/crontab
0 8 * * * /usr/bin/fernet-rotate.sh
ssh ctrl3 sudo cat /etc/kolla/keystone-fernet/crontab
0 16 * * * /usr/bin/fernet-rotate.sh

Currently with three controllers we have this keystone config:

[token]
expiration = 86400 (although, keystone default is one hour)
allow_expired_window = 172800 (this is the keystone default)

[fernet_tokens]
max_active_keys = 4

Currently, kolla-ansible configures key rotation according to the following:

   rotation_interval = token_expiration / num_hosts

This means we rotate keys more quickly the more hosts we have, which doesn't
make much sense.

Keystone docs state:

   max_active_keys =
     ((token_expiration + allow_expired_window) / rotation_interval) + 2

For details see:
https://docs.openstack.org/keystone/stein/admin/fernet-token-faq.html

Rotation is based on pushing out a staging key, so should any server
start using that key, other servers will consider that valid. Then each
server in turn starts using the staging key, each in term demoting the
existing primary key to a secondary key. Eventually you prune the
secondary keys when there is no token in the wild that would need to be
decrypted using that key. So this all makes sense.

This change adds new variables for fernet_token_allow_expired_window and
fernet_key_rotation_interval, so that we can correctly calculate the
correct number of active keys. We now set the default rotation interval
so as to minimise the number of active keys to 3 - one primary, one
secondary, one buffer.

This change also fixes the fernet cron job generator, which was broken
in the following cases:

* requesting an interval of more than 1 day resulted in no jobs
* requesting an interval of more than 60 minutes, unless an exact
  multiple of 60 minutes, resulted in no jobs

It should now be possible to request any interval up to a week divided
by the number of hosts.

Change-Id: I10c82dc5f83653beb60ddb86d558c5602153341a
Closes-Bug: #1809469
2019-05-17 14:05:48 +01:00
Mark Goddard
25ac955a4e Add unit test for keystone fernet cron generator
Before making changes to this script, document its behaviour with a unit
test.

There are two major issues:

* requesting an interval of more than 1 day results in no jobs
* requesting an interval of more than 60 minutes, unless an exact
  multiple of 60 minutes, results in no jobs

Change-Id: I655da1102dfb4ca12437b7db0b79c9a61568f79e
Related-Bug: #1809469
2019-05-17 09:13:31 +01:00
binhong.hua
12ff28a693 Make kolla-ansible support extra volumes
When integrating 3rd party component into openstack with kolla-ansible,
maybe have to mount some extra volumes to container.

Change-Id: I69108209320edad4c4ffa37dabadff62d7340939
Implements: blueprint support-extra-volumes
2019-05-17 11:55:04 +08:00
Serhat Demircan
c4ebd99030 link kolla_logs volume to docker_runtime_directory if docker_runtime_directory variable exists
Change-Id: I053c4a47e657effe25e8d75a3ef55477635729d2
2019-05-16 14:22:37 +03:00
Zuul
22a6e0216e Merge "Add support for Ceilometer custom gnocchi_resources.yaml" 2019-05-16 11:00:27 +00:00
Zuul
be9eaee662 Merge "Tiny fix of quickstart guide" 2019-05-16 11:00:25 +00:00
Zuul
042f2a0406 Merge "Provide support for neutron dev mode in kolla" 2019-05-16 11:00:24 +00:00
Zuul
ffefbfdef2 Merge "Configure region_name in cloudkitty.conf" 2019-05-15 09:18:28 +00:00
Zuul
df87a9434f Merge "Update Python 3 test runtimes for Train" 2019-05-15 08:22:30 +00:00
Erol Guzoglu
11823e632d Configure region_name in cloudkitty.conf
Change-Id: I3395810f353c1a26eafcf0b97eb416d7083efde1
2019-05-14 16:41:40 +03:00
Rafael Weingärtner
b316587dae Enable kolla-ansible to use a custom metrics files with Cloudkitty
Cloudkitty has a default (built-in the container) metrics.yml file
 in the /etc/cloudkitty/metrics.yml files. We would like to be able
 to overwrite/customize these metrics configurations via kolla-ansible.
 Cloudkitty is able to use a custom metric file via "metrics_conf".
 Therefore, we are enabling this configuration via Kolla-ansible.

Change-Id: Id9019298482c040be05f540e71dacfdf0bd77469
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2019-05-14 08:25:07 -03:00
chenxing
fffb6678f1 Tiny fix of quickstart guide
Remove some unexpected indents.

Change-Id: I50b8239ab7270814ffc8be9affb3bd28ce1f855b
2019-05-14 17:23:41 +08:00
Zuul
959e473a1c Merge "Ansible flush_handlers ignores conditional clauses" 2019-05-14 08:04:09 +00:00
ZijianGuo
6532c62c0d Install a list of packages to speed up package installation
Change-Id: I89313d7c279f95e80c593cf02309c8a9f8e6e318
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-05-09 21:45:42 +08:00
Doug Szumski
b805726ca1 Fix issues obtaining Keystone token with Monasca Grafana
When using the the default domain name there are issues authenticating
with Keystone. For example, you can only log in on the second attempt
and the Monasca datasource fails to authenticate. Switching to the
default domain id resolves these issues.

Change-Id: I2cb4b2608c74dd853c97e4fc27078930bc72fdf8
2019-05-09 12:02:54 +01:00
Raimund Hook
6804a5a682 Ansible flush_handlers ignores conditional clauses
The flush_handlers clause doesn't honour conditional clauses.
Instead, it prints a warning and runs anyway:
[WARNING]: flush_handlers task does not support when conditional

See: https://github.com/ansible/ansible/pull/41126

TrivialFix

Change-Id: Iaf70c2e932ae6dfb723bdb2ba658acdbfe74ebe2
2019-05-09 11:51:59 +01:00
confi-surya
157704d902 Update Python 3 test runtimes for Train
Following the process set down in the TC resolution on updating Python
versions:

https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html

Change-Id: If36099eb2ab0c34c18544a0996d2bbaea255b539
2019-05-09 12:46:50 +05:30
Zuul
0d5ccfefae Merge "Enable deflate when using tls for horizon" 2019-05-07 19:34:21 +00:00
Zuul
eef2bdb80f Merge "Updating Jinja filters to conform to Ansible 2.5+" 2019-05-07 19:32:06 +00:00
Jeffrey Zhang
06a8161a18 Enable deflate when using tls for horizon
deflate is disable because of breach attach[0] issue. But it has be
fixed on horizon size through[1], so we cloud enable deflate all the
time.

compress application/json too in default.

[0] https://wiki.openstack.org/wiki/OSSN/OSSN-0037
[1] https://review.openstack.org/#/c/596549/

Change-Id: I364c8a71633fac846dbaac8eaa0b78191e6d7d0e
Closes-Bug: #1827976
2019-05-07 11:30:56 +08:00
Raimund Hook
726ba75341 Updated ansible uri module HEADERS_ field
This fixes a deprecation warning that gets displayed when running
the kibana/post_config 'Get kibana default indexes' task.

HEADERS_ has been deprecated since ansible 2.1 and will be
removed in 2.9.

https://docs.ansible.com/ansible/latest/modules/uri_module.html

TrivialFix

Change-Id: I177113c606119505c6cb69c66a326f7cbdaf2196
2019-05-04 18:48:03 +01:00
Raimund Hook
84ea42bd7c Updating Jinja filters to conform to Ansible 2.5+
Since Ansible 2.5, the use of jinja tests as filters has been
deprecated.

I've run the script provided by the ansible team to 'fix' the
jinja filters to conform to the newer syntax.

This fixes the deprecation warnings.

Change-Id: I844ecb7bec94e561afb09580f58b1bf83a6d00bd
Closes-bug: #1827370
2019-05-02 14:58:09 +01:00
Raimund Hook
fe71a5c059 fixed spelling of 'Bootstrapping' in tools/kolla-ansible
'Bootstrapping' was spelt with one p - added the second p so the
word becomes a verb nicely.

TrivialFix

Change-Id: I126a5c253408af70d6d0a3be6e59270f385a00e3
2019-04-26 11:48:57 +01:00
Zuul
d12e3ef67a Merge "Check if glance_upgrading is defined before using it" 2019-04-26 09:48:33 +00:00