51 Commits

Author SHA1 Message Date
Radosław Piliszek
3411b9e420 Performance: optimize genconfig
Config plays do not need to check containers. This avoids skipping
tasks during the genconfig action.

Ironic and Glance rolling upgrades are handled specially.

Swift and Bifrost do not use the handlers at all.

Partially-Implements: blueprint performance-improvements
Change-Id: I140bf71d62e8f0932c96270d1f08940a5ba4542a
2020-10-12 19:30:06 +02:00
Mark Goddard
b685ac44e0 Performance: replace unconditional include_tasks with import_tasks
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. For unconditionally included tasks, switching to
import_tasks provides a clear benefit.

Benchmarking of include vs. import is available at [1].

This change switches from include_tasks to import_tasks where there is
no condition applied to the include.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md#task-include-and-import

Partially-Implements: blueprint performance-improvements

Change-Id: Ia45af4a198e422773d9f009c7f7b2e32ce9e3b97
2020-08-28 16:12:03 +00:00
Mark Goddard
9702d4c3c3 Performance: use import_tasks for check-containers.yml
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the check-containers.yml include, the
included file only has a single task, so the overhead of skipping this
task will not be greater than the overhead of the task import. It
therefore makes sense to switch to use import_tasks there.

Partially-Implements: blueprint performance-improvements

Change-Id: I65d911670649960708b9f6a4c110d1a7df1ad8f7
2020-07-28 12:10:59 +01:00
wu.chunyang
3e9a648601 permission denied when enable_kolla_dev_mod
non-root user has no permission to create directory under /opt
directory. use "become: true" to resolve it.

Change-Id: I155efc4b1e0691da0aaf6ef19ca709e9dc2d9168
2020-06-07 19:36:42 +08:00
Zuul
87984f5425 Merge "Add Ansible group check to prechecks" 2020-04-16 15:33:46 +00:00
James Kirsch
4d155d69cd Refactor copy certificates task
Refactor service configuration to use the copy certificates task. This
reduces code duplication and simplifies implementing encrypting backend
HAProxy traffic for individual services.

Change-Id: I0474324b60a5f792ef5210ab336639edf7a8cd9e
2020-04-14 17:26:19 +00:00
Zuul
2a2ce059dc Merge "Add notify restart container when cert changed" 2020-03-10 12:12:55 +00:00
yj.bai
d3cc2f670e Add notify restart container when cert changed
When change the cert file in /etc/kolla/certificate/.
The certificate in the container has not changed.
So I think can use kolla-ansible deploy when certificate is
changed. restart <container>

Partially-Implements: blueprint custom-cacerts

Change-Id: Iaac6f37e85ffdc0352e8062ae5049cc9a6b3db26
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2020-03-10 16:23:09 +08:00
Radosław Piliszek
266fd61ad7 Use "name:" instead of "role:" for *_role modules
Both include_role and import_role expect role's name to be given
via "name" param instead of "role".
This worked but caused errors with ansible-lint.
See: https://review.opendev.org/694779

Change-Id: I388d4ae27111e430d38df1abcb6c6127d90a06e0
2020-03-02 10:01:17 +01:00
Mark Goddard
49fb55f182 Add Ansible group check to prechecks
We assume that all groups are present in the inventory, and quite obtuse
errors can result if any are not.

This change adds a precheck that checks for the presence of all expected
groups in the inventory for each service. It also introduces a common
service-precheck role that we can use for other common prechecks.

Change-Id: Ia0af1e7df4fff7f07cd6530e5b017db8fba530b3
Partially-Implements: blueprint improve-prechecks
2020-02-28 16:23:14 +00:00
James Kirsch
511ba9f6a2 Copy CA into containers.
When kolla_copy_ca_into_containers is set to "yes", the Certificate
Authority in /etc/kolla/certificates will be copied into service
containers to enable trust for that CA. This is especially useful when
the CA is self signed, and would not be trusted by default.

Partially-Implements: blueprint custom-cacerts

Change-Id: I4368f8994147580460ebe7533850cf63a419d0b4
2020-01-28 14:03:32 -08:00
Mark Goddard
fe217e98c0 Ansible lint: whitespace
Co-Authored-By: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>

Change-Id: I65d9604d8522f0a60fbfeea718a63866410768b6
2020-01-13 10:38:04 +00:00
Michal Nasiadka
1009931162 Change local_action to delegate_to: localhost
As part of the effort to implement Ansible code linting in CI
(using ansible-lint) - we need to implement recommendations from
ansible-lint output [1].

One of them is to stop using local_action in favor of delegate_to -
to increase readability and and match the style of typical ansible
tasks.

[1]: https://review.opendev.org/694779/

Partially implements: blueprint ansible-lint

Change-Id: I46c259ddad5a6aaf9c7301e6c44cd8a1d5c457d3
2019-11-22 15:04:44 +00:00
Kris Lindgren
2fe0d98ebb Add a job that *only* deploys updated containers
Sometimes as cloud admins, we want to only update code that is running
in a cloud.  But we dont need to do anything else.  Make an action in
kolla-ansible that allows us to do that.

Change-Id: I904f595c69f7276e71692696471e32fd1f88e6e8
Implements: blueprint deploy-containers-action
2019-09-26 17:51:14 +01:00
Mark Goddard
3522d235bd Refactor service, endpoint and user registration
Use upstream Ansible modules for registration of services, endpoints,
users, projects, roles, and role grants.

Change-Id: I7c9138d422cc91c177fd8992347176bb54156b5a
2019-09-17 10:13:56 -07:00
Scott Solkhon
09e02ef8f1 Support configuration of trusted CA certificate file
This commit adds the functionality for an operator to specify
their own trusted CA certificate file for interacting with the
Keystone API.

Implements: blueprint support-trusted-ca-certificate-file
Change-Id: I84f9897cc8e107658701fb309ec318c0f805883b
2019-08-16 12:47:42 +00:00
Mark Goddard
de00bf491d Simplify handler conditionals
Currently, we have a lot of logic for checking if a handler should run,
depending on whether config files have changed and whether the
container configuration has changed. As rm_work pointed out during
the recent haproxy refactor, these conditionals are typically
unnecessary - we can rely on Ansible's handler notification system
to only trigger handlers when they need to run. This removes a lot
of error prone code.

This patch removes conditional handler logic for all services. It is
important to ensure that we no longer trigger handlers when unnecessary,
because without these checks in place it will trigger a restart of the
containers.

Implements: blueprint simplify-handlers

Change-Id: I4f1aa03e9a9faaf8aecd556dfeafdb834042e4cd
2019-06-27 15:57:19 +00:00
Mark Goddard
b123bf6621 Use become for all docker tasks
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
2019-06-06 19:04:58 +01:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
MinSun
12f4554330 Support checkout dedicated version from git with dev mode
Now kolla dev mode only support clone master branch from git,
add version tag to support clone dedicated branch.

Change-Id: I88de238e5dc7461ba0662a3ecea9a2d80fd0db60
2018-08-14 16:06:00 +08:00
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Lakshmi Prasanna Goutham Pratapa
14bf524756 Apply Resource Constraints to Services.
This commit is to apply resource-constraints to a few more OpenStack services.
Commit to  apply constraints to the last set of services will be made in
the upcoming commit.

Depends-on: Icafa54baca24d2de64238222a5677b9d8b90e2aa
Change-Id: I39004f54281f97d53dfa4b1dbcf248650ad6f186
2018-07-26 11:35:28 +00:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Jeffrey Zhang
84ade4e149 Remove useless module_extra_vars in registry.yml
- remove uesless module_extra_vars, this is a historical issue. In the
  past, we use 'docker exec kolla_toolbox ansible xxx' to run module on
  target node, so complex data have to pass through extra_vars. Now we
  are using kolla_toolbox module, no need to use extra_vars anymore.
- Remove some useless until.

Change-Id: I72ed28001202917f9a82a1c3ea33cd6319911ec8
2018-04-16 09:14:54 +08:00
Duong Ha-Quang
9965cc46ff Specify 'become' for only neccesary tasks (all other roles)
Add become to only neccesary tasks in roles:

- aodh
- barbican
- bifrost
- ceilometer
- ceph
- chrony
- cinder
- cloudkitty
- collectd
- congress
- designate
- elasticsearch
- etcd
- freezer
- gnocchi
- grafana
- influxdb
- ironic
- iscsi
- karbor
- kibana
- kuryr
- magnum
- manila
- mistral
- mongodb
- multipathd
- murano
- octavia
- panko
- qdrouterd
- rally
- sahara
- searchlight
- senlin
- skydive
- solum
- swift
- swift
- tacker
- telegraf
- tempest
- trove
- vmtp
- watcher
- zun

Change-Id: I6e32d94d4172dd96d09d8609e8a5221ab5586a31
Partial-Implements: blueprint ansible-specific-task-become
2018-03-12 09:37:43 +07:00
Dai Dang Van
4f8eb892c7 Support policy.yaml file [part 5]
- Freezer
- Gnocchi
- Kuryr
- Murano
- Panko

This will copy only yaml or json policy file if they exist.

Change-Id: I5450839cb06c515f2be445883421f8f987ca834d
Implements: blueprint support-custom-policy-yaml
Co-authored-By: Duong Ha-Quang <duonghq@vn.fujitsu.com>
2018-01-22 01:57:05 +00:00
Michal Rostecki
e4c7b50320 dev mode: Add support for kuryr
Provide support for kolla dev mode in Kuryr. When
'kolla_dev_mode' or 'kuryr_dev_mode' variables are
enabled, source code of Kuryr project is cloned
and bindmounted.

Partially implements: blueprint mount-sources

Change-Id: I74c1ea70b4220a0e91f19403971303ca55652371
2017-12-19 13:26:55 +01:00
Eduardo Gonzalez
56374206be Fix deployment with public TLS enabled
When deploying with tls enabled in public
endpoints, ansible modules fails due SSL certificates
are self-signed.

This change adds a new variable to allow customization
on which endpoints ansible should connect.
Defaults to admin because admin auth parameters defaults
to admin endpoint.

Change-Id: Ic3ed58cf9c9579cae08a11bbfe6fce983b5a9cbc
Closes-Bug: #1720995
2017-10-05 08:36:34 +00:00
Bertrand Lallau
a3dfdfc085 Simplify Openstack services configuration possibilities
Actually Openstack services configuration can be overriden using many
files:
- /etc/kolla/config/<< service name >>/<< config file >>
- /etc/kolla/config/<< service name >>/<<host>>/<< config file >>
- /etc/kolla/config/global.conf
- /etc/kolla/config/database.conf
- /etc/kolla/config/messaging.conf

Only per-service configuration is actually documented here:
https://github.com/openstack/kolla-ansible/blob/master/doc/advanced-configuration.rst#L164

Allowing to globally modify service configuration can be perform too,
but it can be done in 3 different manners, all not documented:
- /etc/kolla/config/global.conf
- /etc/kolla/config/database.conf
- /etc/kolla/config/messaging.conf

database.conf and messaging.conf seems redundant with global.conf.
In order to simplify codebase it seems logical to remove them.

Documentation has been added for overriding configuration globally and
release note has been added too.

Closes-Bug: #1682479
Change-Id: I5d922dfc0d938173bad34ac64e490b78db1b7e31
2017-09-13 17:47:36 +02:00
Jenkins
a71aa61ea7 Merge "Use unified vars in task" 2017-07-12 01:56:12 +00:00
Bertrand Lallau
a9113fc466 Avoid launching "local" actions too many times
In order to speed up deployment time some "local" actions should be run
only once using 'run_once: True'.
This will decrease deployment time in case of multihost configuration.

Change-Id: I6015d772d35c15e96c52f577013b6e41197cb41a
2017-07-05 15:55:39 +02:00
Jeffrey Zhang
03e1b05f99 Use unified vars in task
Ansible task support vars directive, no need implement another one in
merge_config. This patch remove the vars directive in merge_config
action plugin.

Change-Id: I33648a2b6e39b4d49ce76eb66fbf2522721f8c68
2017-06-15 22:37:28 +08:00
Eduardo Gonzalez
f2fd735d51 Fix kuryr ansible deployment
Missing config options in kuryr.conf
Missing kuryr logging
Remove useless bootstrap task, justs create a directory
Wrong user permissions in config.json file.
Missing volumes
Missing container capabilities

Depends-On: I4c08a21df263fdefe1fe991cb7ad41cfee65019d
Change-Id: Id4577a78ebf3f1cda5ee36d14e2cc017f38e0f07
Closes-bug: #1697480
2017-06-13 13:47:29 +02:00
Bertrand Lallau
afdd11b9a2 Generalize api_interface_address variable usage
Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455
2017-05-23 08:35:15 +00:00
Jeffrey Zhang
b1f015616c Add timeout parameter for precheck tasks
wait_for module waits 300 seconds for the port started or stopped.  This
is meaningless and useless in precheck. This patch change timeout to 1
seconds.

Change-Id: I9b251ec4ba17ce446655917e8ef5e152ef947298
Closes-Bug: #1688152
2017-05-04 09:02:25 +08:00
Paul Bourke
5418ada148 Enable sanity checks from kolla-ansible
Add a new subcommand 'check' to kolla-ansible, used to run the
smoke/sanity checks.

Add stub files to all services that don't currently have checks.

Change-Id: I9f661c5fc51fd5b9b266f23f6c524884613dee48
Partially-implements: blueprint sanity-check-container
2017-03-09 10:37:06 +00:00
Jeffrey Zhang
38ad05a872 Remove unnecessary until in register.yml file
Change-Id: I1577cc3afef4dadd3a188c8ba749c9cdfad313ae
2017-02-04 14:38:54 +08:00
Jenkins
ac6822e8ea Merge "Optimize reconfiguration for kuryr" 2017-01-27 15:21:17 +00:00
Jeffrey Zhang
8155d74d8d Refactor register.yml files
Co-Authored-By: Mauricio Lima <mauriciolimab@gmail.com>
Change-Id: I9a4a6b6523dee4b388513386b7d85d421f2b7b89
2017-01-26 17:10:27 +00:00
Mauricio Lima
f12ecdc424 Optimize reconfiguration for kuryr
Co-Authored-By: caoyuan <cao.yuan@99cloud.net>
Change-Id: I6f14860090765d3fb515238dc1525c5fc718f803
Partially-implements: blueprint better-reconfigure
2017-01-26 03:17:50 +00:00
caoyuan
901d63f5a5 Move kuryr precheck into its own role
Change-Id: I0ae528e614aec65d4fee3044fff7036717a50376
Partially-implements: blueprint condition-pre-check
2017-01-12 22:46:00 +08:00
Jenkins
dbc881456b Merge "Custom policy files" 2016-11-04 13:42:55 +00:00
Jeffrey Zhang
fa45801946 Move precheck into its own role
* Merge prechecks.yml and site.yml playbook
* Create empty precheck.yml into all roles.

Change-Id: I8a138558a26c0a2a66c5fd48ed37be657c99c1dd
Implements: blueprint condition-pre-check
2016-11-03 14:48:58 +08:00
Eduardo Gonzalez
ef1fd6b8c0 Custom policy files
Allow operators to use their custom policy files.
Avoid maintain policy files in kolla repos, only copying
the files when an operator add their custom config.

Implements: blueprint custom-policies
Change-Id: Icf3c961b87cbc7a1f1dd2ffbfffcf271d151d862
2016-10-28 10:47:05 +01:00
Jeffrey Zhang
ce5a11a374 Fix stale namespace removal issue
When using both /run and /run/netns in docker container, mount
propagation won't work when delete namespace after the containers are
restarted. This PS makes /run mount as shared.

Closes-Bug: #1616268
Change-Id: Ie16699e1b193b2bf1263d89ea634e89ea69add9e
2016-10-15 01:49:13 +08:00
Jenkins
2abef796d6 Merge "Mount host localtime for containers" 2016-10-13 14:06:20 +00:00
Jeffrey Zhang
16a1f9f006 Mount host localtime for containers
Add /etc/localtime:/etc/localtime:ro to volume for aodh, barbican, etcd,
gnocchi, kuryr and sahara.
All the containers are added in Netwon cycle, so no need to backport

Closes-Bug: #1633049
Change-Id: I9cdba54cf730af44fb1a9ff6f2c936d23dadbe9a
2016-10-13 11:35:03 +00:00
Steven Dake
86fe4c99da Make Kuryr upgrade not crater the upgrade
TrivialFix

Co-Authored-By: zhubingbing <zhubingbing10@gmail.com>

Change-Id: Idcd4c0024f93266746a00a2320980c08eca8b7e2
2016-10-11 12:43:53 +00:00