4928 Commits

Author SHA1 Message Date
Zuul
888e50f01b Merge "Use become for all docker tasks" 2019-06-07 10:47:23 +00:00
Zuul
4eb141e160 Merge "Fix monasca grafana organisation check" 2019-06-07 09:08:12 +00:00
Zuul
cbc09a9131 Merge "Add support for elasticsearch TLS and authentication in fluentd" 2019-06-07 09:08:10 +00:00
Zuul
796980aa3f Merge "Add ansible_nodename (system hostname) to /etc/hosts" 2019-06-07 09:08:08 +00:00
Zuul
a4431930c6 Merge "Remove /%(tenant_id)s suffix from v2.1 endpoints" 2019-06-07 09:08:07 +00:00
Zuul
20e1e639bb Merge "[cloudkitty] Remove unused option from configuration file template" 2019-06-07 09:08:06 +00:00
Zuul
0a1ad98105 Merge "Support multi-region discovery of Nova cells" 2019-06-07 09:08:04 +00:00
Zuul
01f0f2387d Merge "Hide logs when looping over passwords" 2019-06-07 08:53:40 +00:00
Zuul
fc5a1b4d71 Merge "[heat] Multi-region support for bootstrap" 2019-06-07 08:53:38 +00:00
Zuul
c9bc1f3ad1 Merge "Add blazar to fluentd aggregation" 2019-06-07 08:53:37 +00:00
Mark Goddard
b123bf6621 Use become for all docker tasks
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
2019-06-06 19:04:58 +01:00
Zuul
ff2b2f44ba Merge "Fix keystone fernet key rotation scheduling" 2019-06-06 16:33:03 +00:00
Zuul
2d3344e8cd Merge "Fix Blazar Nova aggregate in multi-region setup" 2019-06-06 15:46:30 +00:00
Zuul
4c4a9dc5fb Merge "Make bootstrap more idempotent" 2019-06-06 14:04:12 +00:00
Radosław Piliszek
c8a57d0c19 Make bootstrap more idempotent
No need to touch the sudoers.d file each time
Creation and mode setting is handled by lineinfile itself

Change-Id: Ia36e21b04d3a08fab3c748f6298f142c1d73ee6d
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-06-06 12:34:17 +02:00
Zuul
fbad0d9cce Merge "Spelling mistake in manila.conf.j2" 2019-06-05 14:50:43 +00:00
Zuul
f9d40d2d77 Merge "Remove unused [service_credentials] config sections" 2019-06-05 14:19:03 +00:00
Cody Hammock
2c343562ee
Add blazar to fluentd aggregation
If Blazar is enabled, ensure that fluentd processes its logs.

Change-Id: If71d5c056c042667388dae8e4ee6d51a5ecab46e
2019-06-05 08:30:14 -05:00
Jason
44da1963ba
[heat] Multi-region support for bootstrap
When bootstrapping, Heat was not setting a region explicitly, so it
could default to a region other than the one being deployed.

Change-Id: I0a0596a020fbff91ccc5b9f44f271eab220c88cd
2019-06-05 08:24:07 -05:00
Jason
f20cbf4958
Fix Blazar Nova aggregate in multi-region setup
The Nova aggregate was always defaulting to some region (usually first
in the Keystone endpoint list) when registering the Nova aggregate for
Blazar. Add in a region override to ensure we are always writing to the
region being deployed.

Change-Id: I3f921ac51acab1b1020a459c07c755af7023e026
2019-06-05 08:18:42 -05:00
Jason
30c619d1bc
Hide logs when looping over passwords
When ansible goes in to a loop, by default it prints all the keys for
the item it is looping over. Some roles, when setting up the databases,
iterate over an object that includes the database password.

Override the loop label to hide everything but the database name.

Change-Id: I336a81a5ecd824ace7d40e9a35942a1c853554cd
2019-06-05 08:09:51 -05:00
Jason
328e14253d
Support multi-region discovery of Nova cells
In a multi-region environment, each region is being deployed separately.
Cell discovery, however, would sometimes fail due to it picking a region
different than the one being deployed. Most likely, an internal endpoint
for region A will not be visible from region B. Furthermore, it is not
very useful to discover hosts on a region you're not modifying.

This changes the check to only run against nova compute services located
in the region being deployed.

Change-Id: I21eb1164c2f67098b81edbd5cc106472663b92cb
2019-06-05 08:07:13 -05:00
Luka Peschke
288b9e06f4 [cloudkitty] Remove unused option from configuration file template
The unused "services" option has been removed from the [collect] section.

Change-Id: Id99fc99e1dff2b8f953b242abda2174cbbecd11d
2019-06-04 11:47:12 +02:00
Zuul
9d5b405328 Merge "nova: Fix DBNotAllowed during compute startup" 2019-06-04 03:58:15 +00:00
Zuul
2208b0214e Merge "Adds Qinling Ansible role" 2019-06-03 20:29:41 +00:00
Zuul
0f6cd86f3a Merge "Add support for ceilometer custom pipeline.yaml" 2019-06-03 19:35:55 +00:00
Zuul
3d1b53f301 Merge "Make kolla-ansible support extra volumes" 2019-06-03 18:51:24 +00:00
Zuul
300d9ffcc1 Merge "Add support for ceilometer custom event yaml" 2019-06-03 18:29:39 +00:00
Zuul
c3f011f128 Merge "Enable kolla-ansible to use a custom metrics files with Cloudkitty" 2019-06-03 18:06:57 +00:00
sunguangning
f4b49ae5e9 Spelling mistake in manila.conf.j2
Co-Authored-By: liaoweizhong <liaoweizhong@unionpay.com>

Change-Id: I04a9c4bff817601f718bb43f41f3418571b11cf4
Closes-bug: #1830853
2019-06-03 10:55:01 +00:00
Pierre Riteau
82551a2bfb Remove /%(tenant_id)s suffix from v2.1 endpoints
The installation guide [1] uses endpoints ending at the /v2.1 suffix.

[1] https://docs.openstack.org/nova/stein/install/controller-install.html

Change-Id: I92af045da67f9e746fd6e4b94e56bb8aa72850c4
2019-05-31 18:42:49 +01:00
Gaetan Trellu
edb3489820 Adds Qinling Ansible role
Qinling is an OpenStack project to provide "Function as a Service".
This project aims to provide a platform to support serverless functions.

Change-Id: I239a0130f8c8b061b531dab530d65172b0914d7c
Implements: blueprint ansible-qinling-support
Story: 2005760
Task: 33468
2019-05-31 10:25:28 -04:00
Pierre Riteau
abf10736c7 Remove unused [service_credentials] config sections
Several services inherited [service_credentials] config sections which
they don't use in their code.

Change-Id: Iccf4358e85fb3d7ed25bc1762ff532b2c32bea4a
2019-05-31 13:25:00 +01:00
Isaac Prior
84edfd09b6 Fix monasca grafana organisation check
"Create default control plane organisation if it doesn't exist" task
fails when organisation already exists.
The list organisation task currently returns project domain id.
The create organisation task currently provides project domain name.
Change the create task to use default_project_domain_id instead.

TrivialFix

Change-Id: Ice70d55e6729fe55164dcf85e98acdc1d7925209
2019-05-31 12:25:21 +01:00
Zuul
6c042adfa2 Merge "there is no easy_install in Debian 'buster'" 2019-05-31 11:19:20 +00:00
Krzysztof Klimonda
fc24e9af7c Add support for elasticsearch TLS and authentication in fluentd
Add options for configuring TLS and authentication for elasticsearch
connections in in fluentd.

Change-Id: I936adc2aeaa3c87081be1c44aa0221caf2124e23
Closes-Bug: #1831078
2019-05-30 13:26:50 +00:00
ZijianGuo
c89b901524 Add support for ceilometer custom pipeline.yaml
This file can be modified to adjust polling intervals or other configurations.
We can add a custom 'pipeline.yaml' file to override it.

Change-Id: I325523edc4f7e37db55a2e21fe52e76138e6d114
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-05-23 17:57:16 +08:00
Pierre Riteau
14bf98ddd5 Make "Check if firewalld is installed" task quieter
Stop showing the task has having made changes and silence warning about
not using the yum module (which we could use for the check, but not as
easily).

Change-Id: I9e3608b5db521930409a29981767f468ea234679
2019-05-22 16:38:54 +01:00
ZijianGuo
136210c403 Add support for ceilometer custom event yaml
* event_definitions.yaml:
  This file provides a standard set of events and corresponding traits
that may be of interest.

* event_pipeline.yaml:
  This file can be modified to adjust which notifications to capture and
where to publish the events.

Change-Id: I9c1698e07b65102af9b3ee448ad07f8fa6428b74
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-05-22 20:44:56 +08:00
Pierre Riteau
37899026bf Add ansible_nodename (system hostname) to /etc/hosts
Kolla-Ansible populates /etc/hosts with overcloud hosts using their API
interface IP address. When configured correctly, this allows Nova to use
the API interface for live migration of instances between compute hosts.

The hostname used is from the `ansible_hostname` variable, which is a
short hostname generated by Ansible using the first dot as a delimiter.
However, Nova defaults to use the result of socket.gethostname() to
register nova-compute services.

In deployments where hostnames are set to FQDNs, for example when using
FreeIPA, nova-compute would try to reach the other compute node using
its FQDN (as registered in the Nova database), which was absent from
/etc/hosts. This can result in failures to live migrate instances if
DNS entries don't match.

This commit populates /etc/hosts with `ansible_nodename` (hostname as
reported by the system) in addition to `ansible_hostname`, if they are
different.

Change-Id: Id058aa1db8d60c979680e6a41f7f3e1c39f98235
Closes-Bug: #1830023
2019-05-22 12:27:37 +01:00
Marcin Juszkiewicz
6cf60d2385 there is no easy_install in Debian 'buster'
Change-Id: I49588f9ceef60472428d835b95d26a36eaf2b47e
2019-05-20 16:29:26 +02:00
Marcin Juszkiewicz
1c866201e3 prechecks: be nice, use print in Py3 way
Change-Id: I7a99567b3d16830e1db641e82a7c9e4eca1ae4e6
2019-05-20 12:53:13 +02:00
Mark Goddard
002eec951f nova: Fix DBNotAllowed during compute startup
backport: stein, rocky

During startup of nova-compute, we see the following error message:

Error gathering result from cell 00000000-0000-0000-0000-000000000000:
DBNotAllowed: nova-compute

This issue was observed in devstack [1], and fixed [2] by removing
database configuration from the compute service.

This change takes the same approach, removing DB config from nova.conf
in the nova-compute* containers.

[1] https://bugs.launchpad.net/devstack/+bug/1812398
[2] 8253787137

Change-Id: I18c99ff4213ce456868e64eab63a4257910b9b8e
Closes-Bug: #1829705
2019-05-20 10:24:28 +01:00
Mark Goddard
6c1442c385 Fix keystone fernet key rotation scheduling
Right now every controller rotates fernet keys. This is nice because
should any controller die, we know the remaining ones will rotate the
keys. However, we are currently over-rotating the keys.

When we over rotate keys, we get logs like this:

 This is not a recognized Fernet token <token> TokenNotFound

Most clients can recover and get a new token, but some clients (like
Nova passing tokens to other services) can't do that because it doesn't
have the password to regenerate a new token.

With three controllers, in crontab in keystone-fernet we see the once a day
correctly staggered across the three controllers:

ssh ctrl1 sudo cat /etc/kolla/keystone-fernet/crontab
0 0 * * * /usr/bin/fernet-rotate.sh
ssh ctrl2 sudo cat /etc/kolla/keystone-fernet/crontab
0 8 * * * /usr/bin/fernet-rotate.sh
ssh ctrl3 sudo cat /etc/kolla/keystone-fernet/crontab
0 16 * * * /usr/bin/fernet-rotate.sh

Currently with three controllers we have this keystone config:

[token]
expiration = 86400 (although, keystone default is one hour)
allow_expired_window = 172800 (this is the keystone default)

[fernet_tokens]
max_active_keys = 4

Currently, kolla-ansible configures key rotation according to the following:

   rotation_interval = token_expiration / num_hosts

This means we rotate keys more quickly the more hosts we have, which doesn't
make much sense.

Keystone docs state:

   max_active_keys =
     ((token_expiration + allow_expired_window) / rotation_interval) + 2

For details see:
https://docs.openstack.org/keystone/stein/admin/fernet-token-faq.html

Rotation is based on pushing out a staging key, so should any server
start using that key, other servers will consider that valid. Then each
server in turn starts using the staging key, each in term demoting the
existing primary key to a secondary key. Eventually you prune the
secondary keys when there is no token in the wild that would need to be
decrypted using that key. So this all makes sense.

This change adds new variables for fernet_token_allow_expired_window and
fernet_key_rotation_interval, so that we can correctly calculate the
correct number of active keys. We now set the default rotation interval
so as to minimise the number of active keys to 3 - one primary, one
secondary, one buffer.

This change also fixes the fernet cron job generator, which was broken
in the following cases:

* requesting an interval of more than 1 day resulted in no jobs
* requesting an interval of more than 60 minutes, unless an exact
  multiple of 60 minutes, resulted in no jobs

It should now be possible to request any interval up to a week divided
by the number of hosts.

Change-Id: I10c82dc5f83653beb60ddb86d558c5602153341a
Closes-Bug: #1809469
2019-05-17 14:05:48 +01:00
binhong.hua
12ff28a693 Make kolla-ansible support extra volumes
When integrating 3rd party component into openstack with kolla-ansible,
maybe have to mount some extra volumes to container.

Change-Id: I69108209320edad4c4ffa37dabadff62d7340939
Implements: blueprint support-extra-volumes
2019-05-17 11:55:04 +08:00
Zuul
22a6e0216e Merge "Add support for Ceilometer custom gnocchi_resources.yaml" 2019-05-16 11:00:27 +00:00
Zuul
042f2a0406 Merge "Provide support for neutron dev mode in kolla" 2019-05-16 11:00:24 +00:00
Erol Guzoglu
11823e632d Configure region_name in cloudkitty.conf
Change-Id: I3395810f353c1a26eafcf0b97eb416d7083efde1
2019-05-14 16:41:40 +03:00
Rafael Weingärtner
b316587dae Enable kolla-ansible to use a custom metrics files with Cloudkitty
Cloudkitty has a default (built-in the container) metrics.yml file
 in the /etc/cloudkitty/metrics.yml files. We would like to be able
 to overwrite/customize these metrics configurations via kolla-ansible.
 Cloudkitty is able to use a custom metric file via "metrics_conf".
 Therefore, we are enabling this configuration via Kolla-ansible.

Change-Id: Id9019298482c040be05f540e71dacfdf0bd77469
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2019-05-14 08:25:07 -03:00
Zuul
959e473a1c Merge "Ansible flush_handlers ignores conditional clauses" 2019-05-14 08:04:09 +00:00