141 Commits

Author SHA1 Message Date
Raimund Hook
84ea42bd7c Updating Jinja filters to conform to Ansible 2.5+
Since Ansible 2.5, the use of jinja tests as filters has been
deprecated.

I've run the script provided by the ansible team to 'fix' the
jinja filters to conform to the newer syntax.

This fixes the deprecation warnings.

Change-Id: I844ecb7bec94e561afb09580f58b1bf83a6d00bd
Closes-bug: #1827370
2019-05-02 14:58:09 +01:00
Zuul
d9a0734fc5 Merge "Use database_address and database_port var for mariadb check" 2019-04-04 07:35:08 +00:00
Jim Rollenhagen
524f969bfc Use database_address and database_port var for mariadb check
This is how services reach mariadb; verify it that way.

Closes-Bug: #1823005
Change-Id: I9924ad050118b8a853e2309654a089f65178cd77
2019-04-03 12:54:21 +00:00
Mark Goddard
a4bb8567da Fix up config file permissions on the host
Several config file permissions are incorrect on the host. In general,
files should be 0660, and directories and executables 0770.

Change-Id: Id276ac1864f280554e98b937f2845bb424d521de
Closes-Bug: #1821579
2019-04-02 17:23:31 +01:00
Mark Goddard
b25c0ee477 Fix MariaDB 10.3 upgrade
Upgrading MariaDB from Rocky to Stein currently fails, with the new
container left continually restarting. The problem is that the Rocky
container does not shutdown cleanly, leaving behind state that the new
container cannot recover. The container does not shutdown cleanly
because we run dumb-init with a --single-child argument, causing it to
forward signals to only the process executed by dumb-init. In our case
this is mysqld_safe, which ignores various signals, including SIGTERM.
After a (default 10 second) timeout, Docker then kills the container.

A Kolla change [1] removes the --single-child argument from dumb-init
for the MariaDB container, however we still need to support upgrading
from Rocky images that don't have this change. To do that, we add new
handlers to execute 'mysqladmin shutdown' to cleanly shutdown the
service.

A second issue with the current upgrade approach is that we don't
execute mysql_upgrade after starting the new service. This can leave the
database state using the format of the previous release. This patch also
adds handlers to execute mysql_upgrade.

[1] https://review.openstack.org/644244

Depends-On: https://review.openstack.org/644244
Depends-On: https://review.openstack.org/645990
Change-Id: I08a655a359ff9cfa79043f2166dca59199c7d67f
Closes-Bug: #1820325
2019-03-23 10:21:37 +00:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
Nick Jones
f704a78029 Add new option to perform an on-demand backup of MariaDB
blueprint database-backup-recovery

Introduce a new option, mariadb_backup, which takes a backup of all
databases hosted in MariaDB.

Backups are performed using XtraBackup, the output of which is saved to
a dedicated Docker volume on the target host (which defaults to the
first node in the MariaDB cluster).

It supports either full (the default) or incremental backups.

Change-Id: Ied224c0d19b8734aa72092aaddd530155999dbc3
2018-11-22 09:20:59 +00:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
caoyuan
471985dc2c Update usage of "|" to "is"
With the more recent versions of ansible, we should now use
"is" instead of the "|"

This should update it.

Change-Id: I6fba56fca182349972e8b0ee5452b37aa4090e0c
2018-08-13 12:40:10 +05:30
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Lakshmi Prasanna Goutham Pratapa
14bf524756 Apply Resource Constraints to Services.
This commit is to apply resource-constraints to a few more OpenStack services.
Commit to  apply constraints to the last set of services will be made in
the upcoming commit.

Depends-on: Icafa54baca24d2de64238222a5677b9d8b90e2aa
Change-Id: I39004f54281f97d53dfa4b1dbcf248650ad6f186
2018-07-26 11:35:28 +00:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
MinSun
bee02d36a0 Mariadb bootstrap_cluster enhancement
Using mariadb service defined in default when boot bootstrap_mariadb
Not a bug here, just an enhancement.

Change-Id: I1f8b51fb6177a8524483e600701924dbfc3403cb
2018-05-30 10:49:38 +08:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Eduardo Gonzalez
8a63c801c6 Fix mariadb recover seqnum regex
Regex used to find the recover seqnum partition is not
returning the real num id rather a None.
Task fails due seqnum[0] is not iterable.

Change-Id: I1be55b6ebfc17c6d423e638662ec2a9f4b9b49a2
Closes-Bug: #1752128
2018-04-12 09:03:34 +02:00
Eduardo Gonzalez
ea1a1dee0d Verify YAML syntax in gates
This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799
2018-03-26 17:56:22 +02:00
Alexandru Bogdan Pica
465bc9ee1c Improve mariadb_recovery
The purpose of this change is to improve upon
https://review.openstack.org/#/c/531122/

- Moved vars inside the defaults/main.yml file
- Made the regex for the lineinfile safer

Change-Id: Id581c0b36f3d4bd61d3627b8364b79296b967387
Closes-Bug: 1746567
Related-Bug: 1682153
2018-02-01 00:01:44 +02:00
Marian Tudosoiu
cead8ec623 Rework mariadb recovery tasks
In recover_cluster.yaml playbook the task to find the highest
seqno/Global Transaction ID is no longer relying only on grastate.dat
Instead it now follows the recommendations from galera cluster website
http://galeracluster.com/documentation-webpages/restartingcluster.html

Closes-Bug: 1682153

Change-Id: I5fc3eaa8baee659576c4c39aef9cfd351c8e9af7
2018-01-31 01:09:09 +02:00
Benjamin Diaz
6f64549e1b Set bash as shell when executing mariadb recovery task
Added 'executable' argument to the shell action in the
'Comparing seqno value' task in the cluster recovery playbook.

Change-Id: I3e96a4a76b44ffb558b9a41cde16e66a8d0fab1a
Closes-Bug: #1729603
2017-11-27 15:07:38 -03:00
Duong Ha-Quang
26b2c2d9e9 Specify 'become' to necessary tasks (general roles)
Add config_owner_user and config_owner_group to group_vars/all,
which is user and group of Kolla configuration files in /etc/kolla.

Add become to post-deploy playbook.

Add become to only neccesary tasks in roles:
- certificate
- common
- destroy
- haproxy
- mariadb
- memcached
- rabbitmq

Change-Id: I2aba745a6e3928c52642f64551470fd08cbfd058
Partial-Implements: blueprint ansible-specific-task-become
2017-09-25 06:52:04 +00:00
caoyuan
5cd55bf236 Optimize reconfiguration for mariadb
Change-Id: I278609f9832955849bc9381120a1b260f5a03f1b
Partially-implements: blueprint better-reconfigure
2017-07-22 08:50:08 +08:00
Jenkins
a71aa61ea7 Merge "Use unified vars in task" 2017-07-12 01:56:12 +00:00
Jeffrey Zhang
03e1b05f99 Use unified vars in task
Ansible task support vars directive, no need implement another one in
merge_config. This patch remove the vars directive in merge_config
action plugin.

Change-Id: I33648a2b6e39b4d49ce76eb66fbf2522721f8c68
2017-06-15 22:37:28 +08:00
Duong Ha-Quang
41686edba9 Replace always_run by check_mode
always_run is deprecated and removed in Ansible 2.4
check_mode is introduced in Ansible 2.2 and Kolla-ansible bump Ansible to
2.2.0 so it's safe to replace always_run by check_mode now.

Change-Id: Id1028d38b7bde30a6afe17b319dcdc77907914ab
Closes-Bug: #1643633
Implements: blueprint migrate-to-ansible-2-2-0
2017-06-15 08:10:33 +00:00
Bertrand Lallau
afdd11b9a2 Generalize api_interface_address variable usage
Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455
2017-05-23 08:35:15 +00:00
Jeffrey Zhang
b1f015616c Add timeout parameter for precheck tasks
wait_for module waits 300 seconds for the port started or stopped.  This
is meaningless and useless in precheck. This patch change timeout to 1
seconds.

Change-Id: I9b251ec4ba17ce446655917e8ef5e152ef947298
Closes-Bug: #1688152
2017-05-04 09:02:25 +08:00
Jeffrey Zhang
430a9dedf3 Use new bootstrap and register method
Change-Id: I61aa70affc8ff98c7e57160de8da0df3d6be5929
2017-02-05 20:45:46 +08:00
Mauricio Lima
d53f27caea Move prechecks into its own role
To avoid many merge conflicts and re-works
* Panko
* Octavia
* Ceph
* MariaDB

Co-Authored-By: caoyuan <cao.yuan@99cloud.net>
Partially-implements: blueprint condition-pre-check
Change-Id: I3d2766d0d9f01ad4052731c7219a2d0635527ac5
2017-01-13 12:43:49 -03:00
Eduardo Gonzalez
f56d8e58ba Revert "updating-deprecated-ansible-modules"
check_mode option is included in Ansible 2.2.
Using in our playbooks mean that any other version before
Ansible 2.2 can be used

This reverts commit 529f202d00e4615caa1fcb447aaa47fde6aa1417.

Change-Id: I3af96290443d760346264e6d994fd2a44de65543
Closes-Bug: #1644828
2016-11-25 14:41:38 +00:00
gardlt
529f202d00 updating-deprecated-ansible-modules
* update ceph tasks
* update mariadb tasks
Closes-Bug: #1643633

Change-Id: Ib81789574843edba6e33394a7f66a2e8077075eb
2016-11-21 21:52:26 -06:00
Jeffrey Zhang
fa45801946 Move precheck into its own role
* Merge prechecks.yml and site.yml playbook
* Create empty precheck.yml into all roles.

Change-Id: I8a138558a26c0a2a66c5fd48ed37be657c99c1dd
Implements: blueprint condition-pre-check
2016-11-03 14:48:58 +08:00
Jenkins
e87f095085 Merge "Update the upgrade.yml for mariadb roles" 2016-10-21 16:19:33 +00:00
Jenkins
45b0f2cf63 Merge "Remove do_reconfigure.yml file for all roles" 2016-10-13 16:20:30 +00:00
Jeffrey Zhang
1bcb139392 Choose node with largest seqno number for mariadb recovery
When all mariadb nodes are stopped gracefully, mariadb galera will
write it's last executed position into the grastate.dat file. Need find
the node with largest seqno number in that file and recovery from that
node.

Closes-Bug: #1627717
Change-Id: I6e97c190eec99c966bffde0698f783e519ba14bd
2016-10-09 12:08:58 +00:00
caoyuan
9f6b6db302 Update the upgrade.yml for mariadb roles
beacause upgrade is not required to recreate the database
TrivialFix

Change-Id: I50bbb1592dd1fd5262642077bbe3d9bd32503033
2016-10-02 15:49:26 +08:00
Jeffrey Zhang
fa856556d5 Remove do_reconfigure.yml file for all roles
do_reconfigure.yml is introduced to use serial directive. But we use
it in wrong. Now serial has moved to playbook file. So it is time to
remove the do_reconfigure.yml file

Closes-Bug: #1628152
Change-Id: I8d42d27e6bc302a0e575b0353956eaef9b2ca9fd
2016-09-27 15:04:00 +00:00
Jenkins
6d03a05819 Merge "Fix permissions of tmp files" 2016-09-13 16:44:10 +00:00
Christian Berendt
924fbb6f7a Fix permissions of tmp files
When running kolla-ansible as non-root users the tmp files will
be created with become user as owner and mode 0600. This way the user
running kolla-ansible is not able to read the generated files. This
is necessary because of the use of the lookup method, which will
not run as become user.

TrivialFix

Change-Id: If86d114c05a9b2e7fa240718611d78e93100ebdf
2016-09-13 13:30:25 +02:00
Mick Thompson
4c1482b999 Added serial option to site.yml.
Useful for upgrade etc., which is preferablly done serially.

Example usage: tools/kolla-ansible deploy OR tools/kolla-ansible upgrade

Closes-Bug: #1576708

DocImpact

Change-Id: I34b2e16f8ce53e472a4682a4738c4ac0f5abf00c
2016-09-13 09:26:51 +00:00
Duong Ha-Quang
34975e291e Port with_ loop variable syntax to Ansbile 2
Migrate to full variable syntax in with_ loop
instead of bare variables for:
- ceph
- ceilometer
- glance
- heat
- horizon
- keystone
- mariadb
- memcached
- neutron
- nova
- rabbitmq

Change-Id: Ib925b4ecea47ac758a77739c9c7e140f06933b08
Partial-Implements: blueprint ansible2
2016-08-09 23:23:21 +07:00
Ken Wronkiewicz
1d74db7df0 Config files having underscores breaks Kolla-Kubernetes.
I just thought I'd dash off a quick patch and change the underscore
to a dash.

Change-Id: Ib34cfc8039de01be7e37176648482f9815ac3848
Closes-Bug: #1589734
2016-06-06 16:58:58 -07:00
Ettore Simone
b4759b280c Enable HAProxy consider MariaDB wsrep_local_state
This patch enable wsrep_notify_cmd to rename haproxy user in haproxy_blocked
when the node is not ready to serve and restore it when ready.

Change-Id: I4f49960d7ff2fa689d6ea730b2574f16f083edc1
Closes-Bug: 1578752
Closes-Bug: 1587752
2016-06-03 01:30:00 +02:00
Jenkins
6f5239ce40 Merge "Bump the ansible to 2" 2016-05-26 10:57:27 +00:00
Jenkins
6f17a71d7f Merge "add support for non-default database_port in kolla/ansible" 2016-05-26 09:24:56 +00:00
Jeffrey Zhang
491822c510 Bump the ansible to 2
DocImpact

Change-Id: I3cdfbf84919de80f535c030bd146787ecda40dec
partial-Implements: blueprint ansible2
2016-05-24 19:53:31 +08:00
Ryan Hallisey
2da010a7b9 Make configurable the location where config files are merged
An operator may want to specify the location of custom config
files so that kolla can detect their location and merge
them with the default configs generated.

Partially implements: blueprint multi-project-config

Change-Id: Ibfb38d07a36dfa7fe25381adc34cc1d3cbe7d1e1
2016-05-23 03:23:06 -04:00
Waldemar Znoinski
a2c2eba34f add support for non-default database_port in kolla/ansible
This change makes each step of the kolla deployment aware
of the port database was configured to listen on.
It defaults mariadb_port to database_port.

Change-Id: I8e85d5732015afc0a5481cb33e0b629fdfa84a1b
Closes-Bug: #1576151
DocImpact
2016-05-13 12:12:25 +00:00
Jeffrey Zhang
0fcee87549 map the host localtime to the container
Closes-Bug: #1577148
Change-Id: I636cefc63cf532434a41af3898b63dffa711e280
2016-05-03 09:27:51 +08:00
Steven Dake
ba4bcdbbe8 Change waiting on mariadb.pid
Previously the code looked at mariadb.pid, but this seemed
flakey in function.  It seemed racey and prone to failure on
slower connections to a registry.  The original task was
extremely complex and it didnt really verify that the MariaDB
was ready to serve connections.  Use wait_for with a regex
instead.

Change-Id: I3aafac04f03639b08e0ef4d6a9c9e1a4499f000c
Closes-Bug: #1564278
2016-04-05 12:02:01 -04:00