10489 Commits

Author SHA1 Message Date
Radosław Piliszek
84f439aa76 Fix deployment of Grafana
Some tasks were improperly generalized in change:
I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
(simplify handlers)
This patch reverts that.

Change-Id: I38fbe319da50fc7d5a3fa3c4890f039ae900a60c
Closes-bug: #1845258
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-25 08:58:33 +02:00
Mark Goddard
27f4876eed Switch default cloudkitty storage backend to influxdb
Backport: stein

In the Stein release, cloudkitty switched the default storage backend
from sqlalchemy to influxdb. In kolla-ansible stein configuration, we
did not explicitly set the storage backend, and so we automatically
picked up this change. However, prior to
https://review.opendev.org/#/c/615928/ we did not have full support for
InfluxDB as a storage backend, and so this has broken the Rocky-Stein
upgrade (https://bugs.launchpad.net/kolla-ansible/+bug/1838641), which
fails with this during the DB sync:

ERROR cloudkitty InfluxDBClientError: get_list_retention_policies()
requires a database as a parameter or the client to be using a database

This change synchronises our default with cloudkitty's (influxdb), and
also provides an upgrade transition to create the influxdb database.

We also move the cloudkitty_storage_backend variable to
group_vars/all.yml, since it is used to determine whether to enable
influxdb.

Finally, the section name in cloudkitty.conf was incorrect - it was
storage_influx,  but should be storage_influxdb.

Change-Id: I71f2ed11bd06f58e141d222e2709835b7ddb2c71
Closes-Bug: #1838641
2019-09-24 16:15:14 +00:00
Zuul
1982c9809f Merge "Add <project>_install_type for all projects" 2019-09-24 16:05:38 +00:00
Zuul
a89380375c Merge "Create and grant all keystone roles in service-ks-register" 2019-09-24 13:09:19 +00:00
Zuul
875cdec9f9 Merge "Catch errors and changes in kolla_toolbox module" 2019-09-24 13:04:53 +00:00
Zuul
01ff25f440 Merge "Remove some deprecated config options" 2019-09-24 13:04:52 +00:00
Zuul
32ceccbd33 Merge "CI: Reinstate use of Docker registry mirror" 2019-09-24 13:03:51 +00:00
Dincer Celik
bb7fc2e5b1 [prometheus] Added documentation for deployment and setting extra options
Change-Id: I8bb39eaf8a4239c37fcbf91b55ec8003542e2506
2019-09-24 15:58:31 +03:00
Zuul
eb80a730be Merge "Swift: add swift_extra_ring_files variable to handle multi-policies deployment" 2019-09-24 09:53:29 +00:00
Zuul
2ba04ad26b Merge "Add [nova] section to ironic.conf" 2019-09-24 09:08:53 +00:00
Zuul
86326b32a3 Merge "Fix removed and deprecated options in ironic.conf" 2019-09-24 09:05:57 +00:00
Zuul
589508d9a3 Merge "Ensure keepalived is restarted during upgrade" 2019-09-24 08:45:04 +00:00
Mark Goddard
741f6d9be9 Create and grant all keystone roles in service-ks-register
This ensures we execute the keystone os_* modules in one place.

Also rework some of the task names and loop item display.

Change-Id: I6764a71e8147410e7b24b0b73d0f92264f45240c
2019-09-24 08:50:04 +01:00
Alexis Deberg
0adbbb268e Swift: add swift_extra_ring_files variable to handle multi-policies deployment
The current tasks only use a hardcoded list deploying only the required files.
When using multiple custom policies, additionnal object-*.builder and
object*.gz files are to be deployed as well.
This adds a new default-empty variable that can be overridden when needed

Change-Id: I29c8e349c7cc83e3a2e01ff702d235a0cd97340e
Closes-Bug: #1844752
2019-09-24 02:43:15 -04:00
Mark Goddard
6f05f1b844 Ensure keepalived is restarted during upgrade
During upgrade, we stop all slave keepalived containers. However, if the
keepalived container configuration has not changed, we never restart
them.

This change fixes the issue by notifying the restart handler when the
containers are stopped.

Change-Id: Ibe094b0c14a70a0eb811182d96f045027aa02c2a
Closes-Bug: #1836368
2019-09-23 15:27:34 +01:00
Mark Goddard
cc555c4196 Add <project>_install_type for all projects
This allows the install type for the project to be different than
kolla_install_type

This can be used to avoid hitting bug 1786238, since kuryr only supports
the source type.

Change-Id: I2b6fc85bac092b1614bccfd22bee48442c55dda4
Closes-Bug: #1786238
2019-09-23 10:23:54 +00:00
Mark Goddard
5c9a798324 CI: Reinstate use of Docker registry mirror
After modernising docker configuration
(I1215e04ec15b01c0b43bac8c0e81293f6724f278), we lost our
registry-mirrors configuration in CI that lets us use a mirror of
Dockerhub.

This change uses the new docker_custom_config variable to configure the
registry mirror.

Change-Id: I1430413c12e9d0b59e4f216ff66372de0f3a4f21
2019-09-23 11:14:01 +01:00
Dincer Celik
5ff7bab46b [prometheus] Added support for extra options
This change introduces the way to pass extra options to prometheus.

Currently, prometheus runs with nearly default options, and when clouds
start getting bigger, you need to pass extra parameters to prometheus.

Change-Id: Ic773c0b73062cf3b2285343bafb25d5923911834
2019-09-23 11:25:04 +03:00
Zuul
b7bbbae981 Merge "Adding Prometheus blackbox exporter" 2019-09-20 17:25:04 +00:00
Zuul
94ece4a702 Merge "CI: collect more system configs (name resolution)" 2019-09-20 17:20:54 +00:00
Zuul
6211aab111 Merge "Add retries to keystone resource registration tasks" 2019-09-20 17:20:53 +00:00
Zuul
2764bde147 Merge "CI: Use VXLAN overlay network" 2019-09-20 16:57:59 +00:00
Zuul
7424edb2b0 Merge "CI: Configure the upgrade jobs from the current branch" 2019-09-20 16:38:01 +00:00
Mark Goddard
bfd8ee1978 CI: Fix check-logs.sh
This script has a few issues:

* It catches false positives, due to log levels in config options.

* It doesn't fail on CRITICAL logs, due to variable reset issue.

This change fixes these.

Change-Id: I50c859eb2991e498eeb64bca45daf1e6f237761f
2019-09-20 15:36:50 +00:00
Mark Goddard
e127627dea Remove some deprecated config options
Heat's [DEFAULT] deferred_auth_method is deprecated, and we are setting
the default value of 'trusts'.

Glance's [DEFAULT] registry_host is deprecated, and we do not deploy a
registry.

Change-Id: I80024907c575982699ce323cd9a93bab94c988d3
2019-09-20 14:52:34 +01:00
Mark Goddard
2ddf1fbfab Add retries to keystone resource registration tasks
Sometimes things go wrong. We shouldn't fail a Kolla Ansible run because
of a temporary failure when creating keystone resources.

This task adds retries to the tasks in the service-ks-tasks role.
Default is 5 retries with a 10 second delay, as is used in OpenStack
Ansible.

Change-Id: Ib692062fb93ba330bb9c8a35c684ad06652be8a2
2019-09-20 12:52:55 +01:00
Radosław Piliszek
e7d5c58415 CI: collect more system configs (name resolution)
This patch adds configs relevant to name resolution.

Change-Id: I7ebc2409e9ec0bd875abf0bf4e452bc89efe940d
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-20 10:30:49 +02:00
Zuul
ec8437205e Merge "Fix Octavia to use keystone_admin_project variable" 2019-09-19 21:43:58 +00:00
Zuul
91c68f5da8 Merge "Update "openstack_release" variable to static brach name" 2019-09-19 21:21:57 +00:00
Zuul
3d74bf0c2f Merge "Fix for haproxy precheck failing on CentOS running non-root" 2019-09-19 19:08:16 +00:00
Dincer Celik
a4be4c0dcf Fix Octavia to use keystone_admin_project variable
Project name shouldn't be static as user may override it with
keystone_admin_project

Change-Id: If41b9d8de17985d960104c8daf27ea7d706c27c0
2019-09-19 15:57:57 +00:00
Kris Lindgren
f8cfccb99e Add support for libvirt+tls
To securely support live migration between computenodes we should enable
tls, with cert auth, instead of TCP with no auth support.

Implements: blueprint libvirt-tls

Change-Id: I22ea6233933c840b853fdcc8e03400b2bf577271
2019-09-19 15:32:41 +01:00
Mark Goddard
8e40629161 CI: Use VXLAN overlay network
VXLAN is necessary to run HA in CI (due to floating VIP
address handled by keepalived).
It also turned out to be required to have private
IPv6 address assignments.
This patch is based on linux bridge rather than OVS
to avoid problems with OVS deployed in containers.

This patch enables haproxy in multinode jobs.

Includes saving of linux networking details.

Makes DASHBOARD_URL agree with OS_AUTH_URL - properly uses the
pre-upgrade value for testing.

Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Depends-on: https://review.opendev.org/683068
Depends-on: https://review.opendev.org/682957
Change-Id: I66888712da80c3d6f84ee4949762961664d3adea
2019-09-19 11:07:02 +02:00
Radosław Piliszek
70b4bf6cbf Fix for haproxy precheck failing on CentOS running non-root
Change-Id: I7f2b3a6f1eacd4cabcaa31de543b7489bc5e654b
Closes-bug: #1844636
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-19 11:02:28 +02:00
Q.hongtao
fa88d0984f Update many invaild links for README
Change-Id: I13a72c9312b712c4832dd0b6ead8fad2c0ddbcb2
2019-09-19 14:11:03 +08:00
Zuul
a21b9b5430 Merge "Refactor service, endpoint and user registration" 2019-09-18 17:34:15 +00:00
Zuul
68b9d78cb3 Merge "Build pdf doc" 2019-09-18 17:11:11 +00:00
Zuul
11a572ccb1 Merge "Remove support for OracleLinux" 2019-09-18 14:27:57 +00:00
Radosław Piliszek
e2f511b7d9 CI: Configure the upgrade jobs from the current branch
This lets us control the upgrade process entirely from the
current branch.

Change-Id: Ic8c39e415846596c23dae93c2839375a24e8b888
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-18 13:11:42 +00:00
Zuul
c11ada4148 Merge "Ignore create_cells and discover_computes when nova-api is disabled" 2019-09-18 11:43:18 +00:00
Zuul
6189a0c25c Merge "Update python_path for "source" install type" 2019-09-18 11:43:17 +00:00
Zuul
73844d3ba7 Merge "Fix prometheus-alertmanager cluster bug" 2019-09-18 11:43:03 +00:00
Mark Goddard
15e35333dd Remove support for OracleLinux
We have agreed to remove support for Oracle Linux.

http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006896.html

Change-Id: If11b4ff37af936a0cfd34443e8babb952307882b
2019-09-18 12:25:12 +01:00
Scott Solkhon
b22375ebfd Adding Prometheus blackbox exporter
This commit follows up the work in Kolla to provide deploy and configure the
Prometheus blackbox exporter.

An example blackbox-exporter module has been added (disabled by default)
called os_endpoint. This allows for the probing of endpoints over HTTP
and HTTPS. This can be used to monitor that OpenStack endpoints return a status
code of either 200 or 300, and the word 'versions' in the payload.

This change introduces a new variable `prometheus_blackbox_exporter_endpoints`.
Currently no defaults are specified because the configuration is heavily
dependent on the deployment.

Co-authored-by: Jack Heskett <Jack.Heskett@gresearch.co.uk>
Change-Id: I36ad4961078d90e2fd70c9a3368f5157d6fd89cd
2019-09-18 11:06:19 +01:00
Q.hongtao
8efc3d4ec6 Update stackalytics link for README
Change-Id: Ib0222d5bfd67819b86ffbd1d0a98dc49cc55c370
2019-09-18 11:30:57 +08:00
Zuul
6a53b0bb95 Merge "[designate] Fix admin api configuration" 2019-09-17 17:45:08 +00:00
Mark Goddard
3522d235bd Refactor service, endpoint and user registration
Use upstream Ansible modules for registration of services, endpoints,
users, projects, roles, and role grants.

Change-Id: I7c9138d422cc91c177fd8992347176bb54156b5a
2019-09-17 10:13:56 -07:00
Mark Goddard
ef3abc455f Build pdf doc
The is one of community goals that each project should produce a
single PDF file. The pdf should be in the output of openstack-tox-docs
job.

TeX packages are required to build PDF locally, following is
recommended:

* inkscape
* texlive-latex-base
* texlive-latex-extra
* texlive-fonts-recommended

More about the goal:
https://governance.openstack.org/tc/goals/train/pdf-doc-generation.html
https://etherpad.openstack.org/p/train-pdf-support-goal
https://etherpad.openstack.org/p/pdf-goal-train-common-problems

Change-Id: Ia0f8b73bcbda6295319f5c8de1aa02a40844207b
2019-09-17 14:14:18 +01:00
Zuul
92fe23b6d1 Merge "[designate] Removed deprecated pool manager settings" 2019-09-17 08:40:27 +00:00
Mark Goddard
70b515bf12 Catch errors and changes in kolla_toolbox module
The kolla_toolbox Ansible module executes as-hoc ansible commands in the
kolla_toolbox container, and parses the output to make it look as if
ansible-playbook executed the command. Currently however, this module
sometimes fails to catch failures of the underlying command, and also
sometimes shows tasks as 'ok' when the underlying command was changed.
This has been tested both before and after the upgrade to ansible 2.8.

This change fixes this issue by configuring ansible to emit output in
JSON format, to make parsing simpler. We can now pick up errors and
changes, and signal them to the caller.

This change also adds an ansible playbook, tests/test-kolla-toolbox.yml,
that can be executed to test the module. It's not currently integrated
with any CI jobs.

Note that this change cannot be backported as the JSON output callback
plugin was added in Ansible 2.5.

Change-Id: I8236dd4165f760c819ca972b75cbebc62015fada
Closes-Bug: #1844114
2019-09-16 14:34:11 +01:00