257 Commits

Author SHA1 Message Date
Mark Goddard
3522d235bd Refactor service, endpoint and user registration
Use upstream Ansible modules for registration of services, endpoints,
users, projects, roles, and role grants.

Change-Id: I7c9138d422cc91c177fd8992347176bb54156b5a
2019-09-17 10:13:56 -07:00
Scott Solkhon
09e02ef8f1 Support configuration of trusted CA certificate file
This commit adds the functionality for an operator to specify
their own trusted CA certificate file for interacting with the
Keystone API.

Implements: blueprint support-trusted-ca-certificate-file
Change-Id: I84f9897cc8e107658701fb309ec318c0f805883b
2019-08-16 12:47:42 +00:00
Zuul
b59791ca92 Merge "Fix handling of docker restart policy" 2019-08-03 16:27:46 +00:00
Mark Goddard
1abd15d4a6 Fix glance bootstrap with file backend
Change https://review.opendev.org/#/c/670247/ attempted to fix glance
deployment with the file backend. However, it added a new bug by being
more strict about only generating configuration where the container will
be deployed. This means that the current method of running the glance
bootstrap container on any host in glance-api group could be broken,
since it needs the container configuration.

This change only runs the bootstrap container on hosts in the
glance_api_hosts list, which in the case of the file backend typically
only contains one host.

This change also fixes up some logic during rolling upgrade, where we
might not generate new configuration for the bootstrap host.

Change-Id: I83547cd83b06ddefb3a9e1f39844537bdb32bd7f
Related-Bug: #1836151
2019-07-18 14:36:56 +00:00
Radosław Piliszek
6a737b1968 Fix handling of docker restart policy
Docker has no restart policy named 'never'. It has 'no'.
This has bitten us already (see [1]) and might bite us again whenever
we want to change the restart policy to 'no'.

This patch makes our docker integration honor all valid restart policies
and only valid restart policies.
All relevant docker restart policy usages are patched as well.

I added some FIXMEs around which are relevant to kolla-ansible docker
integration. They are not fixed in here to not alter behavior.

[1] https://review.opendev.org/667363

Change-Id: I1c9764fb9bbda08a71186091aced67433ad4e3d6
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-07-18 13:39:06 +00:00
Zuul
dc1bce370e Merge "Fix glance with file backend" 2019-07-14 12:17:46 +00:00
Mark Goddard
d5e5e885d1 During deploy, always sync DB
A common class of problems goes like this:

* kolla-ansible deploy
* Hit a problem, often in ansible/roles/*/tasks/bootstrap.yml
* Re-run kolla-ansible deploy
* Service fails to start

This happens because the DB is created during the first run, but for some
reason we fail before performing the DB sync. This means that on the second run
we don't include ansible/roles/*/tasks/bootstrap_service.yml because the DB
already exists, and therefore still don't perform the DB sync. However this
time, the command may complete without apparent error.

We should be less careful about when we perform the DB sync, and do it whenever
it is necessary. There is an argument for not doing the sync during a
'reconfigure' command, although we will not change that here.

This change only always performs the DB sync during 'deploy' and
'reconfigure' commands.

Change-Id: I82d30f3fcf325a3fdff3c59f19a1f88055b566cc
Closes-Bug: #1823766
Closes-Bug: #1797814
2019-07-12 08:56:54 +00:00
Mark Goddard
602f89baa6 Fix glance with file backend
Since https://review.opendev.org/647699/, we lost the logic to only
deploy glance-api on a single host when using the file backend.

This code was always a bit custom, and would be better supported by
using the 'host_in_groups' pattern we have in a few other places where a
single group name does not describe the placement of containers for a
service.

Change-Id: I21ce4a3b0beee0009ac69fecd0ce24efebaf158d
Closes-Bug: #1836151
2019-07-11 09:25:46 +01:00
Mark Goddard
de00bf491d Simplify handler conditionals
Currently, we have a lot of logic for checking if a handler should run,
depending on whether config files have changed and whether the
container configuration has changed. As rm_work pointed out during
the recent haproxy refactor, these conditionals are typically
unnecessary - we can rely on Ansible's handler notification system
to only trigger handlers when they need to run. This removes a lot
of error prone code.

This patch removes conditional handler logic for all services. It is
important to ensure that we no longer trigger handlers when unnecessary,
because without these checks in place it will trigger a restart of the
containers.

Implements: blueprint simplify-handlers

Change-Id: I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
2019-06-27 15:57:19 +00:00
Mark Goddard
b123bf6621 Use become for all docker tasks
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
2019-06-06 19:04:58 +01:00
binhong.hua
12ff28a693 Make kolla-ansible support extra volumes
When integrating 3rd party component into openstack with kolla-ansible,
maybe have to mount some extra volumes to container.

Change-Id: I69108209320edad4c4ffa37dabadff62d7340939
Implements: blueprint support-extra-volumes
2019-05-17 11:55:04 +08:00
Krzysztof Klimonda
f5b5260866 Check if glance_upgrading is defined before using it
glance_upgrading variable is defined only during rolling upgrades, but
its value is checked on every call to "Restart glance-api container" handler.

Closes-Bug: #1826511
Change-Id: Idc95306c50a09666ac0a12e975307a42aef9e352
2019-04-26 06:58:08 +00:00
Zuul
ed5588c934 Merge "Don't pull images during upgrade" 2019-03-28 12:41:22 +00:00
Scott Solkhon
c70d806666 Add mising handlers for external Ceph.
When Nova, Glance, or Cinder are deployed alongside an external Ceph deployment
handlers will fail to trigger if keyring files are updated, which results in the
containers not being restarted.

This change adds the missing 'when' conditions for nova-libvirt, nova-compute,
cinder-volume, cinder-backup, and glance-api containers.

Change-Id: I8e183aac9a72e7a7210f7edc7cdcbaedd4fbcaa9
2019-03-22 11:20:34 +00:00
Mark Goddard
58d6dc3bcf Don't pull images during upgrade
When adding the rolling upgrade support, some upgrade procedures were
modified to pull images explicitly. This is done inconsistently between
services, and is a change in behaviour from Rocky and earlier releases.

This change removes all image pulling from upgrade tasks.

Change-Id: Id0fed17714235e1daed60b83b1f30620f097eb97
2019-03-20 18:51:45 +00:00
Jim Rollenhagen
2e4e60503a Use keystone_*_url var in all configs
We're duplicating code to build the keystone URLs in nearly every
config, where we've already done it in group_vars. Replace the
redundancy with a variable that does the same thing.

Change-Id: I207d77870e2535c1cdcbc5eaf704f0448ac85a7a
2019-03-06 15:08:26 -05:00
Jim Rollenhagen
a819ef1215 Allow glance services to use independent hostnames
This allows glance service endpoints to use custom hostnames, and adds the
following variables:

* glance_internal_fqdn
* glance_external_fqdn

These default to the old values of kolla_internal_fqdn or
kolla_external_fqdn.

This also adds a glance_api_listen_port option, which defaults to
glance_api_port for backward compatibility.

This option allow the user to differentiate between the port the
service listens on, and the port the service is reachable on. This is
useful for external load balancers which live on the same host as the
service itself.

Change-Id: Icb91f728533e2db1908b23dabb0501cf9f8a2b75
Implements: blueprint service-hostnames
2019-02-08 10:25:02 -05:00
Zuul
d7a905eb9e Merge "Fix glance configuration when using external ceph" 2018-12-06 09:51:19 +00:00
Jeffrey Zhang
6f020a0483 Fix glance configuration when using external ceph
When using external ceph, enable_ceph=no and glance_backend_ceph=yes,
glance.conf should enable rbd store.

Change-Id: Ia09cd57c829b00f28674cddf44fb55583e193d0f
2018-12-05 12:05:29 +08:00
Zuul
63eccc6372 Merge "Remove the deprecate the Glance Registry" 2018-12-01 07:08:36 +00:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
caoyuan
03fd9715c5 Remove the deprecate the Glance Registry
A spec to Deprecate the Glance Registry Service[0] was accepted in Newton,
but it contained the ambiguous statement, "Mark the service as deprecated
and ready for removal in the Q release." kolla-ansible disable the
glance-registry in Q release[1], and since we are in S now,
remove glance-registry is safe.

[0]: http://specs.openstack.org/openstack/glance-specs/specs/newton/approved/glance/deprecate-registry.html
[1]: https://review.openstack.org/#/c/566804/

Change-Id: I48f794029e97aa6f76bbd500e33f28f51a3f2ac4
2018-11-21 20:51:51 +08:00
Eduardo Gonzalez
cc9dae4dee Add glance-cache support
Glance cache is used to keep a locally cache image
in the glance_api service.
Is an usefull service when an image is commonly used
to speed times between pulling from storage backend
and send to nova.

Change-Id: I8e684cc10e4fee1cb52c17a126e3b11f69576cf6
2018-11-21 10:15:48 +01:00
Mark Goddard
242625dff4 Generate Ceph configuration during upgrade
If upgrading the nova, cinder or manila services via 'kolla-ansible
upgrade', the Ceph config files are not generated. Users will expect
that these files are generated, to pull in any changes from their
configuration or the base kolla configuration.

This change moves Ceph tasks inside config.yml to ensure that they are
performed during deploy, reconfigure and upgrade. This has been done for
nova, cinder, gnocchi and manila - glance already does this.

Change-Id: Ic75692c2bcba9b81dee922ff6fbbccd160e7fa19
Closes-Bug: #1794275
2018-10-10 10:48:55 +01:00
Mark Goddard
b50917fe2c Fix non-root reconfiguration with ceph enabled
Various ceph-related tasks were missing a 'become' that would allow them
to work as a non-root user. This seems to only cause a problem after an
initial deployment, perhaps due to the recursive ownership & permissions
changes at the end of the ceph.yml and external_ceph.yml files.

This change adds the necessary becomes.

Change-Id: I887c7b3bdef49db1dd1bf9e5bdbf5dc47b7f41af
Closes-Bug: #1795125
2018-10-02 09:16:34 +01:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
jiangpch
d57c7019a9 Make haproxy proxy to the right glance_api backend
Since glance_api only start one container when using file
backend, the haproxy should follow this rule.

See: https://review.openstack.org/#/c/448654

Closes-Bug: #1722422

Change-Id: Id3519581e0f54509dacd24d0dd542c630342c771
2018-09-13 15:24:02 +01:00
Zuul
cde23811f7 Merge "Support checkout dedicated version from git with dev mode" 2018-08-29 05:53:04 +00:00
Zuul
3288e0c4e3 Merge "Update usage of "|" to "is"" 2018-08-23 10:23:54 +00:00
MinSun
12f4554330 Support checkout dedicated version from git with dev mode
Now kolla dev mode only support clone master branch from git,
add version tag to support clone dedicated branch.

Change-Id: I88de238e5dc7461ba0662a3ecea9a2d80fd0db60
2018-08-14 16:06:00 +08:00
caoyuan
471985dc2c Update usage of "|" to "is"
With the more recent versions of ansible, we should now use
"is" instead of the "|"

This should update it.

Change-Id: I6fba56fca182349972e8b0ee5452b37aa4090e0c
2018-08-13 12:40:10 +05:30
ZhongShengping
ae246945a6 Deprecate auth_uri option
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.

[1]https://review.openstack.org/#/c/508522/

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ifd8527d404f1df807ae8196eac2b3849911ddc26
Closes-Bug: #1761907
2018-08-07 11:58:23 +05:30
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Zuul
d1e427b7c1 Merge "Apply Resource Constraints to Openstack Services" 2018-07-26 14:18:34 +00:00
Lakshmi Prasanna Goutham Pratapa
76210a2d85 Apply Resource Constraints to Openstack Services
This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa
2018-07-25 17:05:04 +01:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Joshua Harlow
07dfc20292 Allow externally managed swift as glance backend
It is possible to have an accessible swift API that is not managed by
kolla-ansible -- for example, ceph exposes a swift API, and using that
requires setting swift as the glance backend.

So, we should loosen the requirement that using the swift backend for
glance requires swift be enabled in kolla-ansible.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>

Change-Id: I17076d5412d2b1e1f13bb0badceaca85a5cee108
2018-07-20 08:55:57 +00:00
Adam Harwell
8cc97c60d3 Fix glance config copy when a service is disabled
In this patch, the glance-registry service was disabled:
https://review.openstack.org/#/c/566804/

However, the config task still tries to copy files for it, which will
break due to path errors.

Change-Id: If39bb12bf830e6559342037ae2a2b99a784ee503
2018-07-18 14:08:40 -07:00
Zuul
66205ab1fa Merge "Migrate ceph keyring creation to kolla_ceph_keyring module" 2018-07-05 07:29:37 +00:00
Jeffrey Zhang
3397668d10 Migrate ceph keyring creation to kolla_ceph_keyring module
In this way, keyring caps is updatable.

Change-Id: Idf7f222645b5073e2c72d59eecf3d47b3f1dc6ba
2018-07-02 09:49:48 +08:00
Mark Duggan
87043bc918 config proxy settings inside glance_api container
Adding the capability to configure settings for http_proxy https_proxy
and no_proxy inside glance_api container. This is required when import
images from external http resources behind a proxy.

Co-Authored-By: Paul Bourke <paul.bourke@oracle.com>
Closes-bug: #1775815
Change-Id: I85661f04311d5671adecc84c470f2e4db07cfe1d
2018-06-24 11:51:32 +00:00
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
Zhangfei Gao
ce809aea23 osprofiler support redis
Currently osprofiler only choose elasticsearch,
which is only supported on x86.
On other platform like aarch64 osprofiler can
not be used since no elasticsearch package.

Enable osprofiler by enable_osprofiler: "yes",
which choose elasticsearch by default.
Choose redis by enable_redis: "yes" & osprofiler_backend: "redis"
On platform without elasticsearch support like aarch64
set enable_elasticsearch: "no"

Change-Id: I68fe7a33e11d28684962fc5d0b3d326e90784d78
2018-06-01 09:34:04 +08:00
Zuul
06b46533a4 Merge "Fix ansible warning" 2018-05-11 04:13:44 +00:00
Zuul
10b3e38961 Merge "Apply Glance database migration" 2018-05-11 02:55:14 +00:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Duong Ha-Quang
365e3d3a3f Apply Glance database migration
Apply Glance database upgrade procedure:

https://docs.openstack.org/glance/latest/admin/zero-downtime-db-upgrade.html

Rolling upgrade in Glance requires that glance-registry is not deployed and run

Depends-On: https://review.openstack.org/532128
Depends-On: https://review.openstack.org/566804

Change-Id: Ia5d0cee2385facefda9d423051b74ab51d075f13
Implements: blueprint apply-service-upgrade-procedure
2018-05-09 16:29:27 +00:00
Duong Ha-Quang
a155e796af Disable glance-registry
As described in [1], [2], glance-registry was deprecated from Queens and will be
removed in S, so we should disable glance registry in Queens but keep the image
and deploy logic if user need this (but it's low chance due to glance-registry
almost do nothing currently).

This patch only disable glance registry microservice.

[1]: http://specs.openstack.org/openstack/glance-specs/specs/newton/approved/glance/deprecate-registry.html
[2]: https://blueprints.launchpad.net/glance/+spec/deprecate-registry

Change-Id: I6667325711d1c54cf5233ad6973d89ea396a46b5
Closes-bug: #1769837
2018-05-08 14:43:30 +07:00
Zuul
2b8bc555cb Merge "Fix SSL api for multiple services" 2018-04-27 08:55:48 +00:00
caoyuan
1822d4890d Avoid creating an empty volume
Avoid creating an empty volume[0] when not use dev mode
for glance.

[0]: https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/glance/defaults/main.yml#L14

Change-Id: Ib27489abe20fb562acbc28263f7680e39d9dab44
2018-04-22 21:11:06 +08:00