568 Commits

Author SHA1 Message Date
Zuul
ec78645928 Merge "Bump minimum Ansible version to 2.5" 2019-07-08 09:21:53 +00:00
Zuul
70b7cddd2b Merge "Add parameters to configure number of processes and threads of horizon" 2019-07-05 09:17:14 +00:00
Mark Goddard
e6d0e610c5 Deprecate Ceph deployment
There are now several good tools for deploying Ceph, including Ceph
Ansible and ceph-deploy. Maintaining our own Ceph deployment is a
significant maintenance burden, and we should focus on our core mission
to deploy OpenStack. Given that this is a significant part of kolla
ansible currently we will need a long deprecation period and a migration
path to another tool.

Change-Id: Ic603c85c04d8794580a19f9efaa7a8589565f4f6
Partially-Implements: blueprint remove-ceph
2019-07-04 19:05:54 +01:00
Christian Berendt
dc3489df18 Add parameters to configure number of processes and threads of horizon
Change-Id: Ib5490d504a5b7c9a37dda7babf1257aa661c11de
2019-07-04 17:23:50 +02:00
Zuul
2ad7b50010 Merge "Cloudkitty InfluxDB Storage backend via Kolla-ansible" 2019-07-04 03:45:40 +00:00
Rafael Weingärtner
97cb30cdd8 Cloudkitty InfluxDB Storage backend via Kolla-ansible
This proposal will add support to Kolla-Ansible for Cloudkitty
 InfluxDB storage system deployment. The feature of InfluxDB as the
 storage backend for Cloudkitty was created with the following commit
 https://github.com/openstack/cloudkitty/commit/
 c4758e78b49386145309a44623502f8095a2c7ee

Problem Description
===================

With the addition of support for InfluxDB in Cloudkitty, which is
achieving general availability via Stein release, we need a method to
easily configure/support this storage backend system via Kolla-ansible.

Kolla-ansible is already able to deploy and configure an InfluxDB
system. Therefore, this proposal will use the InfluxDB deployment
configured via Kolla-ansible to connect to CloudKitty and use it as a
storage backend.

If we do not provide a method for users (operators) to manage
Cloudkitty storage backend via Kolla-ansible, the user has to execute
these changes/configurations manually (or via some other set of
automated scripts), which creates distributed set of configuration
files, "configurations" scripts that have different versioning schemas
and life cycles.

Proposed Change
===============

Architecture
------------

We propose a flag that users can use to make Kolla-ansible configure
CloudKitty to use InfluxDB as the storage backend system. When
enabling this flag, Kolla-ansible will also enable the deployment of
the InfluxDB via Kolla-ansible automatically.

CloudKitty will be configured accordingly to [1] and [2]. We will also
externalize the "retention_policy", "use_ssl", and "insecure", to
allow fine granular configurations to operators. All of these
configurations will only be used when configured; therefore, when they
are not set, the default value/behavior defined in Cloudkitty will be
used. Moreover, when we configure "use_ssl" to "true", the user will
be able to set "cafile" to a custom trusted CA file. Again, if these
variables are not set, the default ones in Cloudkitty will be used.

Implementation
--------------
We need to introduce a new variable called
`cloudkitty_storage_backend`. Valid options are `sqlalchemy` or
`influxdb`. The default value in Kolla-ansible is `sqlalchemy` for
backward compatibility. Then, the first step is to change the
definition for the following variable:
`/ansible/group_vars/all.yml:enable_influxdb: "{{ enable_monasca |
bool }}"`

We also need to enable InfluxDB when CloudKitty is configured to use
it as the storage backend. Afterwards, we need to create tasks in
CloudKitty configurations to create the InfluxDB schema and configure
the configuration files accordingly.

Alternatives
------------
The alternative would be to execute the configurations manually or
handle it via a different set of scripts and configurations files,
which can become cumbersome with time.

Security Impact
---------------
None identified by the author of this spec

Notifications Impact
--------------------
Operators that are already deploying CloudKitty with InfluxDB as
storage backend would need to convert their configurations to
Kolla-ansible (if they wish to adopt Kolla-ansible to execute these
tasks).

Also, deployments (OpenStack environments) that were created with
Cloudkitty using storage v1 will need to migrate all of their data to
V2 before enabling InfluxDB as the storage system.

Other End User Impact
---------------------
None.

Performance Impact
------------------
None.

Other Deployer Impact
---------------------
New configuration options will be available for CloudKitty.
* cloudkitty_storage_backend
* cloudkitty_influxdb_retention_policy
* cloudkitty_influxdb_use_ssl
* cloudkitty_influxdb_cafile
* cloudkitty_influxdb_insecure_connections
* cloudkitty_influxdb_name

Developer Impact
----------------
None

Implementation
==============

Assignee
--------
* `Rafael Weingärtner <rafaelweingartne>`

Work Items
----------
 * Extend InfluxDB "enable/disable" variable
 * Add new tasks to configure Cloudkitty accordingly to these new
 variables that are presented above
 * Write documentation and release notes

Dependencies
============
None

Documentation Impact
====================
New documentation for the feature.

References
==========
[1] `https://docs.openstack.org/cloudkitty/latest/admin/configuration/storage.html#influxdb-v2`
[2] `https://docs.openstack.org/cloudkitty/latest/admin/configuration/collector.html#metric-collection`

Change-Id: I65670cb827f8ca5f8529e1786ece635fe44475b0
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2019-07-02 11:14:05 -03:00
Mark Goddard
0a769dc30b Bump minimum Ansible version to 2.5
This is necessary for some Ansible tests which were renamed in 2.5 -
including 'version' and 'successful'.

Change-Id: Iacf88ef5589c7571fcf56ba8b99d3dbe76975195
2019-07-01 09:38:01 +01:00
Zuul
85b9dabcd4 Merge "Add support for neutron custom dnsmasq.conf" 2019-06-27 13:59:42 +00:00
Zuul
e7c19b7413 Merge "Enable InfluxDB TSI by default" 2019-06-27 11:44:51 +00:00
Christian Berendt
a3f1ded357 Add support for neutron custom dnsmasq.conf
Change-Id: Ia7041be384ac07d0a790c2c5c68b1b31ff0e567a
2019-06-27 12:20:12 +02:00
Zuul
a956c53181 Merge "Remove `hnas_iscsi` from the supported storage backends list of Cinder" 2019-06-24 13:08:24 +00:00
chenxing
b7ca065edf Remove `hnas_iscsi` from the supported storage backends list of Cinder
The Hitachi NAS Platform iSCSI driver was marked as not supported by
Cinder in the Ocata realease[1].

[1] https://review.opendev.org/#/c/444287/

Change-Id: I1a25789374fddaefc57bc59badec06f91ee6a52a
Closes-Bug: #1832821
2019-06-24 09:04:14 +00:00
Doug Szumski
015ddb6e37 Enable InfluxDB TSI by default
The TSI is recommended for all users. Some of the key benefits are
a reduction in memory requirements and an increase in the maximum
number of time series. For more information see this link:

https://docs.influxdata.com/influxdb/v1.7/concepts/tsi-details/

Change-Id: I4b29eb5a4ae82f6c39059d0b6de41debdfd75508
2019-06-21 14:48:12 +01:00
Marek Svensson
10bf6b05fa Fix default deployment of freezer, use mariadb.
This change defaults freezer to use mariadb as default backend for database
and adds elasticsearch as an optional backend due to the requirement of
freezer to use elasticsearch version 2.3.0. The default elasticsearch in
kolla-ansible is 5.6.x and that doesn't work with freezer.

Added needed options to the elasticsearch backend like:
 - protocol
 - address
 - port
 - number of replicas

Change-Id: I88616c285bdb297fd1f738846ddffe1b08a7a827
Signed-off-by: Marek Svensson <marek@marex.st>
2019-06-18 15:12:36 -04:00
Zuul
6cae4dedfe Merge "Remove nova-consoleauth" 2019-06-17 16:28:45 +00:00
Jeffrey Zhang
4e032923c0 Remove nova-consoleauth
The nova-consoleauth service was deprecated during the Rocky release [1]
and has not been necessary since unless you're using cells v1. As Kolla
has never supported cells v1, which is finally being removed during
Train [2], we can get ahead of the curve and stop deploying
nova-consoleauth immediately.

[1] https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/convert-consoles-to-objects.html
[2] https://blueprints.launchpad.net/nova/+spec/remove-cells-v1/

Change-Id: I099080979f5497537e390f531005a517ab12aa7a
2019-06-16 16:39:07 +08:00
Zuul
29b755eb15 Merge "Remove Neutron LBaaS support" 2019-06-13 19:15:42 +00:00
Zuul
ee895413eb Merge "Stop duplicating Nova cells" 2019-06-13 18:56:00 +00:00
Carlos Goncalves
f427920daf Remove Neutron LBaaS support
The project has been retired and there will be no Train release [1].
This patch removes Neutron LBaaS support in Kolla.

[1] https://review.opendev.org/#/c/658494/

Change-Id: Ic0d3da02b9556a34d8c27ca21a1ebb3af1f5d34c
2019-06-07 13:50:19 +01:00
Zuul
796980aa3f Merge "Add ansible_nodename (system hostname) to /etc/hosts" 2019-06-07 09:08:08 +00:00
Pierre Riteau
19b8dbe460 Stop duplicating Nova cells
Check if a base Nova cell already exists before calling `nova-manage
cell_v2 create_cell`, which would otherwise create a duplicate cell when
the transport URL or database connection change.

If a base cell already exists but the connection values have changed, we
now call `nova-manage cell_v2 update_cell` instead. This is only
possible if a duplicate cell has not yet been created. If one already
exists, we print a warning inviting the operator to perform a manual
cleanup. We don't use a hard fail to avoid an abrupt change of behavior
if this is backported to stable branches.

Change-Id: I7841ce0cff08e315fd7761d84e1e681b1a00d43e
Closes-Bug: #1734872
2019-06-06 18:10:06 +01:00
Zuul
ff2b2f44ba Merge "Fix keystone fernet key rotation scheduling" 2019-06-06 16:33:03 +00:00
Zuul
2208b0214e Merge "Adds Qinling Ansible role" 2019-06-03 20:29:41 +00:00
Gaetan Trellu
edb3489820 Adds Qinling Ansible role
Qinling is an OpenStack project to provide "Function as a Service".
This project aims to provide a platform to support serverless functions.

Change-Id: I239a0130f8c8b061b531dab530d65172b0914d7c
Implements: blueprint ansible-qinling-support
Story: 2005760
Task: 33468
2019-05-31 10:25:28 -04:00
Pierre Riteau
37899026bf Add ansible_nodename (system hostname) to /etc/hosts
Kolla-Ansible populates /etc/hosts with overcloud hosts using their API
interface IP address. When configured correctly, this allows Nova to use
the API interface for live migration of instances between compute hosts.

The hostname used is from the `ansible_hostname` variable, which is a
short hostname generated by Ansible using the first dot as a delimiter.
However, Nova defaults to use the result of socket.gethostname() to
register nova-compute services.

In deployments where hostnames are set to FQDNs, for example when using
FreeIPA, nova-compute would try to reach the other compute node using
its FQDN (as registered in the Nova database), which was absent from
/etc/hosts. This can result in failures to live migrate instances if
DNS entries don't match.

This commit populates /etc/hosts with `ansible_nodename` (hostname as
reported by the system) in addition to `ansible_hostname`, if they are
different.

Change-Id: Id058aa1db8d60c979680e6a41f7f3e1c39f98235
Closes-Bug: #1830023
2019-05-22 12:27:37 +01:00
Mark Goddard
6c1442c385 Fix keystone fernet key rotation scheduling
Right now every controller rotates fernet keys. This is nice because
should any controller die, we know the remaining ones will rotate the
keys. However, we are currently over-rotating the keys.

When we over rotate keys, we get logs like this:

 This is not a recognized Fernet token <token> TokenNotFound

Most clients can recover and get a new token, but some clients (like
Nova passing tokens to other services) can't do that because it doesn't
have the password to regenerate a new token.

With three controllers, in crontab in keystone-fernet we see the once a day
correctly staggered across the three controllers:

ssh ctrl1 sudo cat /etc/kolla/keystone-fernet/crontab
0 0 * * * /usr/bin/fernet-rotate.sh
ssh ctrl2 sudo cat /etc/kolla/keystone-fernet/crontab
0 8 * * * /usr/bin/fernet-rotate.sh
ssh ctrl3 sudo cat /etc/kolla/keystone-fernet/crontab
0 16 * * * /usr/bin/fernet-rotate.sh

Currently with three controllers we have this keystone config:

[token]
expiration = 86400 (although, keystone default is one hour)
allow_expired_window = 172800 (this is the keystone default)

[fernet_tokens]
max_active_keys = 4

Currently, kolla-ansible configures key rotation according to the following:

   rotation_interval = token_expiration / num_hosts

This means we rotate keys more quickly the more hosts we have, which doesn't
make much sense.

Keystone docs state:

   max_active_keys =
     ((token_expiration + allow_expired_window) / rotation_interval) + 2

For details see:
https://docs.openstack.org/keystone/stein/admin/fernet-token-faq.html

Rotation is based on pushing out a staging key, so should any server
start using that key, other servers will consider that valid. Then each
server in turn starts using the staging key, each in term demoting the
existing primary key to a secondary key. Eventually you prune the
secondary keys when there is no token in the wild that would need to be
decrypted using that key. So this all makes sense.

This change adds new variables for fernet_token_allow_expired_window and
fernet_key_rotation_interval, so that we can correctly calculate the
correct number of active keys. We now set the default rotation interval
so as to minimise the number of active keys to 3 - one primary, one
secondary, one buffer.

This change also fixes the fernet cron job generator, which was broken
in the following cases:

* requesting an interval of more than 1 day resulted in no jobs
* requesting an interval of more than 60 minutes, unless an exact
  multiple of 60 minutes, resulted in no jobs

It should now be possible to request any interval up to a week divided
by the number of hosts.

Change-Id: I10c82dc5f83653beb60ddb86d558c5602153341a
Closes-Bug: #1809469
2019-05-17 14:05:48 +01:00
binhong.hua
12ff28a693 Make kolla-ansible support extra volumes
When integrating 3rd party component into openstack with kolla-ansible,
maybe have to mount some extra volumes to container.

Change-Id: I69108209320edad4c4ffa37dabadff62d7340939
Implements: blueprint support-extra-volumes
2019-05-17 11:55:04 +08:00
Zuul
a16576e9c0 Merge "Do some Train TODOs" 2019-04-23 06:02:10 +00:00
Zuul
b28ffeb27d Merge "Remove RabbitMQ support from Bifrost" 2019-04-15 11:30:57 +00:00
Mark Goddard
33564a0097 Remove RabbitMQ support from Bifrost
During the Train cycle, Bifrost switched to using JSON-RPC by default
for Ironic's internal communication [1], avoiding the need to install
RabbitMQ. This simplifies things, so we may as well remove our custom
configuration of RabbitMQ.

[1] https://review.openstack.org/645093

Change-Id: I3107349530aa753d68fd59baaf13eb7dd5485ae6
2019-04-10 11:30:50 +01:00
Zuul
7eb0da0d71 Merge "Use ironic inspector 'dnsmasq' PXE filter by default" 2019-04-10 09:17:41 +00:00
Mark Goddard
86e83faeb1 Use ironic inspector 'dnsmasq' PXE filter by default
With Docker CE, the daemon sets the default policy of the iptables
FORWARD chain to DROP. This causes problems for provisioning bare metal
servers when ironic inspector is used with the 'iptables' PXE filter.
It's not entirely clear why these two things interact in this way,
but switching to the 'dnsmasq' filter works around the issue, and is
probably a good move anyway because it is more efficient.

We have added a migration task here to flush and remove the ironic-inspector
iptables chain since inspector does not do this itself currently.

Change-Id: Iceed5a096819203eb2b92466d39575d3adf8e218
Closes-Bug: #1823044
2019-04-08 17:00:52 +00:00
Mark Goddard
bb9d51e25b Do some Train TODOs
Make an early start on the TODOs for the Train cycle.

1. Remove the task that removes the vitrage_collector container, which
   was added in the Stein cycle to clean up this container which is no
   longer deployed.

2. Remove globals.yml configuration in CI to disable Heat for upgrade
   jobs. Heat is now enabled in the previous release (Stein).

3. Remove the deprecated variable cinder_iscsi_helper, which was renamed
   to cinder_target_helper in Stein.

Change-Id: I774bf395e0bdd4db9c20c6289a22cf059fa42e1a
2019-04-08 12:25:27 +01:00
ce6222ae8b Update master for stable/stein
Add file to the reno documentation build to show release notes for
stable/stein.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.

Change-Id: I4a9a0eab03f3dd06bf2214ed6d6e8db6af5bd032
Sem-Ver: feature
2019-04-05 14:00:22 +00:00
Mark Goddard
3a6a9384cb Tidy up release notes for Stein release
Change-Id: I0d66e49f09313de8abb89f510c7a5098507c572a
2019-03-29 13:34:43 +00:00
Zuul
03d3885a56 Merge "Add cyborg to kolla-ansible" 2019-03-28 08:20:13 +00:00
Zuul
33a92b9f7d Merge "Add ceilometer_ipmi container into ceilometer role" 2019-03-22 12:02:22 +00:00
Zuul
e35c32c0d7 Merge "Support separate Swift storage networks" 2019-03-14 16:19:59 +00:00
Scott Solkhon
a781c64319 Support separate Swift storage networks
Adds support to seperate Swift access and replication traffic from other storage traffic.

In a deployment where both Ceph and Swift have been deployed,
this changes adds functionalality to support optional seperation
of storage network traffic. This adds two new network interfaces
'swift_storage_interface' and 'swift_replication_interface' which maintain
backwards compatibility.

The Swift access network interface is configured via 'swift_storage_interface',
which defaults to 'storage_interface'. The Swift replication network
interface is configured via 'swift_replication_interface', which
defaults to 'swift_storage_interface'.

If a separate replication network is used, Kolla Ansible now deploys separate
replication servers for the accounts, containers and objects, that listen on
this network. In this case, these services handle only replication traffic, and
the original account-, container- and object- servers only handle storage
user requests.

Change-Id: Ib39e081574e030126f2d08f51de89641ddb0d42e
2019-03-14 14:00:18 +00:00
Zuul
4b4fc498af Merge "Support customising Fluentd formatting" 2019-03-14 09:26:50 +00:00
caoyuan
16900c2e37 Add ceilometer_ipmi container into ceilometer role
refer to [0]

[0]: https://docs.openstack.org/ceilometer/latest/install/install-compute-rdo.html
Co-Authored-By: zhulingjie <easyzlj@gmail.com>

Change-Id: I4cda336dedb3d807b80d13bcc219268a8d667b4d
2019-03-14 16:24:15 +08:00
Zuul
ce08b31f5e Merge "Support the prometheus elasticsearch exporter" 2019-03-13 17:00:50 +00:00
Erol Guzoglu
14ab9a7c4e Support the prometheus elasticsearch exporter
This patch implements the support for the elasticsearch-exporter in
kolla-ansible

The configuration and prechecks are reused from the other exporters

Depends-On: Id138f12e10102a6dd2cd8d84f2cc47aa29af3972
Change-Id: Iae0eac0179089f159804490bf71f1cf2c38dde54
2019-03-11 17:25:51 +03:00
Doug Szumski
c8a22f1090 Support customising Fluentd formatting
In some scenarios it may be useful to perform custom formatting of logs
before forwarding them. For example, the JSON formatter plugin can be
used to convert an event to JSON.

Change-Id: I3dd9240c5910a9477456283b392edc9566882dcd
2019-03-08 11:20:33 +00:00
Bai Yongjun
ed2fd243d1 Add cyborg to kolla-ansible
Because kolla-ansible not have cyborg so should add it.

Implements: blueprint add-cyborg-to-kolla-ansible

Depend-On: I497e67e3a754fccfd2ef5a82f13ccfaf890a6fcd

Change-Id: I6f7ae86f855c5c64697607356d0ff3161f91b239
2019-03-08 10:46:53 +08:00
Zuul
a2975ef592 Merge "Improve standalone ironic support" 2019-03-06 09:03:31 +00:00
Zuul
a628deefc1 Merge "Use new cinder target_helper option" 2019-03-04 12:09:39 +00:00
Mark Goddard
1c22da32ff Use new cinder target_helper option
The iscsi_helper option was deprecated in favour of target_helper in
Queens, and will be removed in the Stein release.

This also renames the cinder_iscsi_helper variable to
cinder_target_helper, deprecating but still supporting the former name
until the Train release.

Change-Id: Ie38c09b2dd8598f62b0733c8444eec5f6ce3daac
2019-03-01 14:36:25 +00:00
Mark Goddard
54965c878b Improve standalone ironic support
Adds a new flag, 'enable_openstack_core', which defaults to 'yes'.
Setting this flag to 'no' will disable the core OpenStack services,
including Glance, Heat, Horizon, Keystone, Neutron, and Nova.

Improves the default configuration of OpenStack Ironic when used in
standalone mode. In particular, configures a noauth mode when Keystone
is disabled, and allows the iPXE server to be used for provisioning as
well as inspection if Neutron is disabled.

Documentation for standalone ironic will be updated separately.

This patch was developed and tested using Bikolla [1].

[1] https://github.com/markgoddard/bikolla

Change-Id: Ic47f5ad81b8126a51e52a445097f7950dba233cd
Implements: blueprint standalone-ironic
2019-02-22 17:22:48 +00:00
Zuul
c2f3ba3d5d Merge "hinese quotes" 2019-02-12 14:32:11 +00:00