107 Commits

Author SHA1 Message Date
Mark Goddard
cc555c4196 Add <project>_install_type for all projects
This allows the install type for the project to be different than
kolla_install_type

This can be used to avoid hitting bug 1786238, since kuryr only supports
the source type.

Change-Id: I2b6fc85bac092b1614bccfd22bee48442c55dda4
Closes-Bug: #1786238
2019-09-23 10:23:54 +00:00
Doug Szumski
339ea2bdeb Support namespacing RabbitMQ logs
The RabbitMQ role supports namespacing the service via the
project_name. For example, if you change the project_name, the
container name and config directory will be renamed accordingly. However
the log folder is currently fixed, even though the service tries to
write to one named after the project_name. This change fixes that.

Whilst you might generally use vhosts, running multiple RabbitMQ
services on a single node is useful at the very least for testing,
or for running 'outward RabbitMQ' on the same node.

This change is part of the work to support Cells v2.

Partially Implements: blueprint support-nova-cells
Change-Id: Ied2c24c01571327ea532ba0aaf2fc5e89de8e1fb
2019-08-08 16:46:32 +00:00
Zuul
8f70bc22d6 Merge "Add extra volumes support for services that were not previously supported" 2019-08-05 09:02:04 +00:00
Radosław Piliszek
6a737b1968 Fix handling of docker restart policy
Docker has no restart policy named 'never'. It has 'no'.
This has bitten us already (see [1]) and might bite us again whenever
we want to change the restart policy to 'no'.

This patch makes our docker integration honor all valid restart policies
and only valid restart policies.
All relevant docker restart policy usages are patched as well.

I added some FIXMEs around which are relevant to kolla-ansible docker
integration. They are not fixed in here to not alter behavior.

[1] https://review.opendev.org/667363

Change-Id: I1c9764fb9bbda08a71186091aced67433ad4e3d6
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-07-18 13:39:06 +00:00
ZijianGuo
e610a73e98 Add extra volumes support for services that were not previously supported
We don't add extra volumes support for all services in patch [1].
In order to unify the management of the volume, so we need add extra volumes
support for these services.

[1] 12ff28a693

Change-Id: Ie148accdd8e6c60df6b521d55bda12b850c0d255
Partially-Implements: blueprint support-extra-volumes
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-06-27 18:32:15 +08:00
Mark Goddard
b123bf6621 Use become for all docker tasks
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
2019-06-06 19:04:58 +01:00
Raimund Hook
84ea42bd7c Updating Jinja filters to conform to Ansible 2.5+
Since Ansible 2.5, the use of jinja tests as filters has been
deprecated.

I've run the script provided by the ansible team to 'fix' the
jinja filters to conform to the newer syntax.

This fixes the deprecation warnings.

Change-Id: I844ecb7bec94e561afb09580f58b1bf83a6d00bd
Closes-bug: #1827370
2019-05-02 14:58:09 +01:00
Mark Goddard
a4bb8567da Fix up config file permissions on the host
Several config file permissions are incorrect on the host. In general,
files should be 0660, and directories and executables 0770.

Change-Id: Id276ac1864f280554e98b937f2845bb424d521de
Closes-Bug: #1821579
2019-04-02 17:23:31 +01:00
Mark Goddard
1e2a1a8fe1 Fix rabbitmq reconfigure, simplify role
Since Id724b44a3edd951fa8b06c9f2c347e9ed8c5ffd9, there is a reference to a
non-existent variable, rabbitmq_confs, that causes deployment to fail if
rabbitmq configuration other than config.json is changed.

I'm taking this opportunity to simplify the role, since we can use the Ansible
handler notification system to determine when handlers need to run, without
registering and checking variables. This simpler approach was used in the
haproxy refactor.

Change-Id: Ibe0e7fda93afff741243ff9c350db1c8c6e1e6d3
Closes-Bug: #1816053
2019-02-15 14:59:26 +00:00
Benjamin Diaz
0729472256 Make Rabbitmq config files overridable
Rabbitmq config files can be overrided in node_custom_config directory.

Change-Id: Id724b44a3edd951fa8b06c9f2c347e9ed8c5ffd9
Closes-Bug: #1674446
2018-12-14 12:28:31 -03:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
Paul Bourke
b163cb02d1 Update rabbitmq to use new conf & clustering
Depends-On: I75e00312b36e1678b90a42cf58d24652323eff27
Change-Id: Ia716fabffca41eff816e59bbf9f4cab79ee8b72f
2018-09-21 16:47:59 +00:00
Paul Bourke
0d03fc2789 Temporarily remove the rabbitmq clusterer plugin
In order to migrate to the latest release of rabbitmq (3.7), we need to
first remove this deprecated plugin which is no longer supported (the
problems it solved are now addressed in rabbitmq itself).

This avoids a circular dependency in CI where the new images depend on
the new clustering and the new clustering depends on the new images.

Change-Id: I921459f3e40b9e0d4af9497384e49aabf0abe79b
2018-08-21 11:24:09 +00:00
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Lakshmi Prasanna Goutham Pratapa
9f0db30fd1 Apply Resource-Constraints to all services.
This commit is the final commit to apply resource-constraints
to all OpenStack services.

Depends-on: I39004f54281f97d53dfa4b1dbcf248650ad6f186
Change-Id: I072d69be9698be54775cb0ae286ea2b6ed78776c
Implements: blueprint resource-constraints
2018-07-23 19:07:05 +05:30
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
MinSun
ddc564d21f Rabbitmq bootstrap enhancement
Using rabbitmq service defined in default when
boot rabbitmq_bootstrap.
Not a bug here, just an enhancement.

Change-Id: I79f0f7efe3308ed4eb898b85a6370be1bd637d9a
2018-06-01 13:49:11 +08:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Eduardo Gonzalez
ea1a1dee0d Verify YAML syntax in gates
This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799
2018-03-26 17:56:22 +02:00
Zuul
3a16dd574d Merge "Add configurable rabbitmq monitoring user" 2017-12-15 03:05:34 +00:00
Taeha Kim
fed4c3c615 Add missing hostname to nodename in RabbitMQ for multinode deployment
When a RabbitMQ node in multiple RabbitMQ nodes is started during
multinode deployment, it is required to communicate with each other
to be clustered. However, RabbitMQ nodes cannot communicate between
them due to missing host name in the nodename environment variable
of RabbitMQ. Subsequently, all of RabbitMQs cannot be started and
it will give rise to a deployment failure.

Change-Id: I7b4ba76807750db4a14d859454ba650bdaaf23ca
Signed-off-by: Taeha Kim <kthguru@gmail.com>
2017-12-14 00:43:50 +00:00
Zuul
0c915c908f Merge "Revert "Omit outward_rabbitmq from check for upgrade"" 2017-11-15 05:11:53 +00:00
Doug Szumski
245cd42694 Add configurable rabbitmq monitoring user
As an operator I want to be able to monitor the status
of RabbitMQ by collecting metrics such as queue length,
message rates (globally and per channel), and information
about resource usage on the host, such as memory use,
open file descriptors and the state of the cluster. Whilst
it is possible to gather all of this information using
the OpenStack RabbitMQ user configured by Kolla Ansible,
this user has write access to the OpenStack vhost. This
feature adds a monitoring user which has access to all of
the information described above, but does not have write
access. An example of a service which may use the
monitoring user is the RabbitMQ plugin for the Monasca
Agent. As not all users will configure monitoring, by
default the monitoring user is disabled. To create it,
the user should override the rabbitmq_monitoring_user
variable.

Implements: blueprint add-monitoring-user-for-rabbit
Change-Id: Ie895ddc59dda1c38faab6305163d9bed6710ff9d
2017-11-07 09:52:10 +00:00
Eduardo Gonzalez
50be17292e Revert "Omit outward_rabbitmq from check for upgrade"
This reverts commit c618dbcd0efb20d5ece50d1b034eb0aba13bb31c.

Pike is already released

Change-Id: Id54a90e1c1d60f2be02ff4af3b648671d053c6a6
2017-10-27 05:43:43 +00:00
Jenkins
109328be3e Merge "Specify 'become' to necessary tasks (general roles)" 2017-10-03 20:01:04 +00:00
Christian Berendt
108d60cb05 Support HiPE (High-Performance Erlang) in RabbitMQ role
Change-Id: I6394475f14bb4875be6d924591e4aea533ac2eb6
Closes-bug: #1621404
Depends-on: Ifc4f1d0f1b59b1789011e33c649033eb3cf5b010
2017-09-27 09:23:23 +02:00
Duong Ha-Quang
26b2c2d9e9 Specify 'become' to necessary tasks (general roles)
Add config_owner_user and config_owner_group to group_vars/all,
which is user and group of Kolla configuration files in /etc/kolla.

Add become to post-deploy playbook.

Add become to only neccesary tasks in roles:
- certificate
- common
- destroy
- haproxy
- mariadb
- memcached
- rabbitmq

Change-Id: I2aba745a6e3928c52642f64551470fd08cbfd058
Partial-Implements: blueprint ansible-specific-task-become
2017-09-25 06:52:04 +00:00
John Garbutt
7e9fa0cd41 Make outward_rabbit pre-checks idempotent
Copy the patterns from the rabbit checks, skip some pre-checks when the
container has already been started. Without this change the pre-checks
fail when you re-run the deploy, i.e. the port is not free because
rabbit is already running on that port.

This bug was triggered because murano is enabled, and this change has
been added to add the extra rabbitmq instance by default:
d8fe3ea780c188b6e937ab6f08a8475d2330a9fa

Closes-Bug: #1715135

Change-Id: I0eb8785e7cd4eadfa792ea14a27f54a891b2bf02
2017-09-05 15:48:13 +01:00
Mick Thompson
c618dbcd0e Omit outward_rabbitmq from check for upgrade
Upgrade fails as outward_rabbitmq does not exist
 and cannot therefore be upgraded. Omit it from the
 upgrade check and bootstrap it after rabbitmq upgrade.

 Remove jinja2 from 'Find gospel node' task; removes warnings.

Change-Id: I3766271c62779c8dbd31e7cf2300473815bbbe68
2017-08-21 11:30:59 +01:00
Jeffrey Zhang
cacf08f0a6 Remove all kolla-kubernetes configurations
kolla-kubernetes is using its own configuration generation[0], so it is
time for kolla-ansible to remove the related code to simplify the
logical.

[0] https://github.com/openstack/kolla-kubernetes/tree/master/ansible

Change-Id: I7bb0b7fe3b8eea906613e936d5e9d19f4f2e80bb
Implements: blueprint clean-k8s-config
2017-07-18 22:00:58 +08:00
Paul Bourke
aff43f73e3 Add vhost to outward rabbitmq for Murano
Change-Id: I09654f29b59e0327ee1a7961e0990e4c6927e8fc
Closes-Bug: #1620374
2017-06-15 14:19:33 +01:00
Paul Bourke
d8fe3ea780 Add a new 'outward' rabbitmq instance
Certain services such as Murano and trove require access to a rabbitmq
instance from tenant networks. [0]

Exposing the internal rabbitmq to end users is a security hole, hence
there are two options, 1) use vhosts in the existing rabbitmq, or two a
separate rabbitmq instances. Given the importance of rabbitmq to the
OpenStack deployment, we have decided to go with a separate instance.
Refer to [1] for more detail on the various options.

This change makes the rabbitmq role generic so that it can be reused, in
this case to start 'outward_rabbitmq'. It needs to be exposed via
haproxy both for network isolation and also because this is what Murano
configuration requires.

Follow on patches will be added to add a vhost in this outward instance
for Murano and other services which require access.

Based on the original work by bdaca[2]

[0] http://murano.readthedocs.io/en/stable-liberty/intro/architecture.html
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-December/109091.html
[2] https://review.openstack.org/#/c/374525

Change-Id: Ib2bcc7ed4bf4f883a7cd1dfad3db89201e3cfd8d
Partial-Bug: #1620374
Depends-On: I020eb6219f89a310451becde41f6f1c7f54baadd
Co-Authored-By: Bartłomiej Daca <bartek.daca@gmail.com>
2017-06-15 11:12:22 +00:00
Duong Ha-Quang
4738c75ab3 Fix rabbitmq prechecks broken due to Ansible upgrade
In Ansible 2.3.0 when statements should not include jinja2 templating
delimiters such as {{ }} or {% %}, and gate is broken with Ansible 2.3.1.

This patchset rewrite when statement in rabbitmq precheck task to not use
string interpolation.

Change-Id: Ie2f1666cc8ced7cf20ceba40c7c7aaec750778f9
Closes-Bug: #1695111
2017-06-02 07:02:45 +00:00
Jenkins
57f0a3c4a7 Merge "Optimize reconfigure action for rabbitmq" 2017-05-23 11:24:04 +00:00
Bertrand Lallau
afdd11b9a2 Generalize api_interface_address variable usage
Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455
2017-05-23 08:35:15 +00:00
Jeffrey Zhang
b1f015616c Add timeout parameter for precheck tasks
wait_for module waits 300 seconds for the port started or stopped.  This
is meaningless and useless in precheck. This patch change timeout to 1
seconds.

Change-Id: I9b251ec4ba17ce446655917e8ef5e152ef947298
Closes-Bug: #1688152
2017-05-04 09:02:25 +08:00
Paul Bourke
5418ada148 Enable sanity checks from kolla-ansible
Add a new subcommand 'check' to kolla-ansible, used to run the
smoke/sanity checks.

Add stub files to all services that don't currently have checks.

Change-Id: I9f661c5fc51fd5b9b266f23f6c524884613dee48
Partially-implements: blueprint sanity-check-container
2017-03-09 10:37:06 +00:00
caoyuan
d56c58685e Optimize reconfigure action for rabbitmq
Change-Id: I14e1619f8fd33be5b0652982131c2a4a4b3e1292
Partially-implements: blueprint better-reconfigure
2017-02-17 11:10:04 +08:00
caoyuan
2646650f35 Move rabbitmq precheck into its own role
Change-Id: Iab182674f17f21322c88e6f7a728fb534168eb25
Partially-implements: blueprint condition-pre-check
2017-01-12 22:43:20 +08:00
Jeffrey Zhang
fa45801946 Move precheck into its own role
* Merge prechecks.yml and site.yml playbook
* Create empty precheck.yml into all roles.

Change-Id: I8a138558a26c0a2a66c5fd48ed37be657c99c1dd
Implements: blueprint condition-pre-check
2016-11-03 14:48:58 +08:00
Jenkins
45b0f2cf63 Merge "Remove do_reconfigure.yml file for all roles" 2016-10-13 16:20:30 +00:00
Jeffrey Zhang
4c207bc039 Fix rabbitmq upgrade permission issue
During the upgrade from Mitaka to Newton, the uid/gid may change for the
same image. Especially on Ubuntu, we moved to Ubuntu Xenial in Newton
and it added systemd related user which break all the uid/gid during an
upgrade. It will the permissions in all docker named volumes.

This fix extends set_config.py to set the proper permission during
container start. This is super light then add commands in
extend_start.sh file  or add ansible tasks.

This patch just fixes rabbitmq case. Other services will be fixed in
following patches.

Partial-Bug: #1631503
Change-Id: Ib17027b97abbc9bf4e3cd503601b8010325b5c5b
2016-10-13 01:19:05 +08:00
Jeffrey Zhang
fa856556d5 Remove do_reconfigure.yml file for all roles
do_reconfigure.yml is introduced to use serial directive. But we use
it in wrong. Now serial has moved to playbook file. So it is time to
remove the do_reconfigure.yml file

Closes-Bug: #1628152
Change-Id: I8d42d27e6bc302a0e575b0353956eaef9b2ca9fd
2016-09-27 15:04:00 +00:00
Jeffrey Zhang
7608f1ee65 Revert "Bind EPMD to api interface address"
This issue still exsits when disable ipv6 feature.

This reverts commit 5480bd9b1d3a9efcd3618ddf12718d2621ceeb47.

Change-Id: I1e6c6bff5585cf5a49890668203d6971112c32f1
2016-09-24 11:25:28 +08:00
Jeffrey Zhang
5480bd9b1d
Bind EPMD to api interface address
Closes-Bug: #1562701
Change-Id: Ica68bdee81223232995bc21ad5e5d5fbf9e8b05f
2016-09-14 11:35:10 +08:00
Mick Thompson
4c1482b999 Added serial option to site.yml.
Useful for upgrade etc., which is preferablly done serially.

Example usage: tools/kolla-ansible deploy OR tools/kolla-ansible upgrade

Closes-Bug: #1576708

DocImpact

Change-Id: I34b2e16f8ce53e472a4682a4738c4ac0f5abf00c
2016-09-13 09:26:51 +00:00
Serguei Bezverkhi
134089d6cd Adding required check for Orchestration Kubernetes
Thgis PS add missing check for Kubernetes orchestration engine,
without these changes, running kolla-ansible genconfig fails
for Kubernetes environment.

TrivialFix

Change-Id: I25eeb7ae3ddba7c924f6d48aa24afdbe74227170
2016-09-12 15:32:42 -04:00
Paul Bourke
6c238ca3bf Move rabbitmq precheck from start
rabbitmq's start task contains a precheck. This should be part of the
other prechecks for consistency

TrivialFix

Change-Id: I7728ec3f5be3248424d74a4387925b72114b8943
2016-08-26 16:18:04 +01:00