246 Commits

Author SHA1 Message Date
Mark Goddard
365bb5177d Create cells before starting nova services
Nova services may reasonably expect cell databases to exist when they
start. The current cell setup tasks in kolla run after the nova
containers have started, meaning that cells may or may not exist in the
database when they start, depending on timing. In particular, we are
seeing issues in kolla CI currently with jobs timing out waiting for
nova compute services to start. The following error is seen in the nova
logs of these jobs, which may or may not be relevant:

No cells are configured, unable to continue

This change creates the cell0 and cell1 databases prior to starting nova
services.

In order to do this, we must create new containers in which to run the
nova-manage commands, because the nova-api container may not yet exist.
This required adding support to the kolla_docker module for specifying a
command for the container to run that overrides the image's command.

We also add the standard output and error to the module's result when a
non-detached container is run. A secondary benefit of this is that the
output of bootstrap containers is now displayed in the Ansible output if
the bootstrapping command fails, which will help with debugging.

Change-Id: I2c1e991064f9f588f398ccbabda94f69dc285e61
Closes-Bug: #1808575
2018-12-14 19:26:42 +00:00
Paul Bourke
a16d78711f Allow operators to customise Nova vendor info
Nova allows customisation of various metadata passed through to VMs via
a 'release' file[0]. Allow operators to make use of this.

[0] https://github.com/openstack/nova/blob/master/etc/nova/release.sample

Change-Id: I71569314c8e64320f8ffad79b9273f4d6d903bb6
2018-11-30 09:48:28 +00:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
Christian Berendt
03788e17d4 Set "no_log" for "databases user and setting permissions" tasks
At the moment the "databases user and setting permissions" task for
designate and nova leaks the database_password because of the use
of with_items:

---snip---
TASK [nova : Creating Nova databases user and setting permissions] *********************************************************
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova', u'database_username': u'nova'})
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova_cell0', u'database_username': u'nova'})
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova_api', u'database_username': u'nova_api'})
---snap---

Change-Id: I141e4153223c8772c82a31d81e58057ce266c0b9
Co-authored-by: Bernd Müller <mueller@b1-systems.de>
2018-11-19 11:10:41 +00:00
Zuul
c0435b833a Merge "Generate Ceph configuration during upgrade" 2018-10-26 06:33:36 +00:00
Christian Berendt
864e589803 nova: add support for a dedicated migration network
Two new parameters (migration_interface, migration_interface_address) to make
the use of a dedicated migration network possible.

Change-Id: I723c9bea9cf1881e02ba39d5318c090960c22c47
2018-10-23 18:37:28 +02:00
Mark Goddard
242625dff4 Generate Ceph configuration during upgrade
If upgrading the nova, cinder or manila services via 'kolla-ansible
upgrade', the Ceph config files are not generated. Users will expect
that these files are generated, to pull in any changes from their
configuration or the base kolla configuration.

This change moves Ceph tasks inside config.yml to ensure that they are
performed during deploy, reconfigure and upgrade. This has been done for
nova, cinder, gnocchi and manila - glance already does this.

Change-Id: Ic75692c2bcba9b81dee922ff6fbbccd160e7fa19
Closes-Bug: #1794275
2018-10-10 10:48:55 +01:00
Zuul
2a1c6349ab Merge "Git checkout version support for several projects" 2018-10-09 03:05:50 +00:00
howardlee
8b292c99bf Git checkout version support for several projects
Kolla-ansible provides support for the dev mode for some projects
of openstack, but there are still some projects that do not yet
support specific release tag. This patch will implement this function
for these project.

Change-Id: I917b27dd61295b542457a21b240afe2cd4e83e58
2018-10-03 15:30:06 +08:00
Mark Goddard
b50917fe2c Fix non-root reconfiguration with ceph enabled
Various ceph-related tasks were missing a 'become' that would allow them
to work as a non-root user. This seems to only cause a problem after an
initial deployment, perhaps due to the recursive ownership & permissions
changes at the end of the ceph.yml and external_ceph.yml files.

This change adds the necessary becomes.

Change-Id: I887c7b3bdef49db1dd1bf9e5bdbf5dc47b7f41af
Closes-Bug: #1795125
2018-10-02 09:16:34 +01:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
Duong Ha-Quang
c79c5d1f36 Apply Nova rolling upgrade logic
This patchset apply Nova rolling upgrade logic [1]

[1] https://docs.openstack.org/nova/latest/user/upgrade.html

Implements: blueprint apply-service-upgrade-procedure

Co-Authored-By: Ha Manh Dong <donghm@vn.fujitsu.com>
Depends-On: https://review.openstack.org/#/c/558765/
Change-Id: I20531a25dc7f5b05dc70ba771cf4c4222ade33e3
2018-09-19 16:25:51 +07:00
Zuul
e29f5c84d9 Merge "Provide support for nova dev mode in kolla" 2018-09-18 09:56:09 +00:00
zhulingjie
7f1f0178c7 Remove the unncessary register variable defined
when creating Nova databases user and setting permissions,
no need to register the database_user_create, and it used
nowhere, remove it is safe

Change-Id: If456b7c2ed25aa729be7d98ef875230c66581d65
2018-08-28 23:01:25 -04:00
Zuul
874cde1e35 Merge "Allow nova to use cephfs cinder volumes without nova cephfs backend" 2018-08-27 07:42:51 +00:00
caoyuan
0b0b2b39d7 Provide support for nova dev mode in kolla
Add a possibility to mount sources as volumes to containers,
in "more than documentation" way. That will let us to use kolla
as a replacement for devstack.

Partially implements: blueprint mount-sources
Co-Authored-By: zhulingjie <easyzlj@gmail.com>

Change-Id: I10677e5ad22f2107a0657feeeaf32287ab9f8e28
2018-08-24 10:50:45 +05:30
caoyuan
471985dc2c Update usage of "|" to "is"
With the more recent versions of ansible, we should now use
"is" instead of the "|"

This should update it.

Change-Id: I6fba56fca182349972e8b0ee5452b37aa4090e0c
2018-08-13 12:40:10 +05:30
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Zuul
d1e427b7c1 Merge "Apply Resource Constraints to Openstack Services" 2018-07-26 14:18:34 +00:00
Lakshmi Prasanna Goutham Pratapa
76210a2d85 Apply Resource Constraints to Openstack Services
This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa
2018-07-25 17:05:04 +01:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Mark Goddard
83595f78dd Fix typo in nova rp_filter sysctl
Fixes a typo introduced in I93e53bad9727beb786b00bd7fcd6d78785c619c2.

Change-Id: I9fd6587913cccd5a29b3fc012b4ddeac8859a0ff
Related-Bug: #1782799
TrivialFix
2018-07-23 08:22:25 +01:00
Doug Szumski
264866ca2b Support setting rp_filter mode
Enables setting rp_filter mode on Neutron L3 agent and Nova compute
hosts whilst maintaining the default that it is disabled.

Closes-Bug: #1782799
Change-Id: I93e53bad9727beb786b00bd7fcd6d78785c619c2
2018-07-20 16:41:00 +01:00
Jeffrey Zhang
3397668d10 Migrate ceph keyring creation to kolla_ceph_keyring module
In this way, keyring caps is updatable.

Change-Id: Idf7f222645b5073e2c72d59eecf3d47b3f1dc6ba
2018-07-02 09:49:48 +08:00
Will Szumski
19b1e9c08d Allow nova to use cephfs cinder volumes without nova cephfs backend
It is not necessary to enable the nova cephfs backend to make use
of cinder cephfs volumes.

Change-Id: I35c3d2e49769962e5c47f585d91d1efd492a53d6
Closes-Bug: #1778107
2018-06-21 18:31:32 +01:00
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
Christian Berendt
ad9dc76112 nova: make libvirt port configurable and check if the port is free
Change-Id: I7dbb5a33ef78dc4ecd338ffbd59a29c0fe3be9c5
2018-05-18 10:26:39 +02:00
Zuul
06b46533a4 Merge "Fix ansible warning" 2018-05-11 04:13:44 +00:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Zuul
d3f496ec7f Merge "List and map compute hosts by service instead of by compute node." 2018-05-10 08:53:03 +00:00
Zuul
b60468f06e Merge "Do not create nova db user if database unchanged" 2018-05-09 16:32:36 +00:00
Jorge Niedbalski
b0ba58d536 List and map compute hosts by service instead of by compute node.
This commit adds the --by-service flag in the discover_hosts
command of the nova cell subsystem, using this flag,
hosts are mapped based on the existing nova-compute services instead
of using the compute node reference.

In this way, we can ensure that the host mappings exists
for this new host (and create it if not) preventing the
HostMappingNotFound exception to loop over and over.

Bug: #1768094
Change-Id: I948004031bc3e125f929ed3e6c670aea11e73a31
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@linaro.org>
2018-05-09 10:02:53 -03:00
Jeffrey Zhang
84ade4e149 Remove useless module_extra_vars in registry.yml
- remove uesless module_extra_vars, this is a historical issue. In the
  past, we use 'docker exec kolla_toolbox ansible xxx' to run module on
  target node, so complex data have to pass through extra_vars. Now we
  are using kolla_toolbox module, no need to use extra_vars anymore.
- Remove some useless until.

Change-Id: I72ed28001202917f9a82a1c3ea33cd6319911ec8
2018-04-16 09:14:54 +08:00
Doug Szumski
67d4fdd2af Do not create nova db user if database unchanged
This change prevents the task running if the database
has not been changed when a preconfigured external
database is not in use. This was the behaviour before
the preconfigured database feature was introduced.

Close-Bug: 1763375
Change-Id: I4f68905732dac75bc0deddab88047c9c0d7d65b5
2018-04-12 14:51:50 +01:00
Jeffrey Zhang
b0b51e83ae Refresh nova cell cache in nova-scheduler process
Base on patch https://review.openstack.org/550527 , after nova cell is
created, need send SIGHUP to nova-scheduler process to refresh nova
cell cache.

Change-Id: Ic1f52b2ca736b593b70fc355a057923b01e0b3ed
2018-04-09 09:40:53 +08:00
Zuul
5e2814a139 Merge "XenAPI: use dom0's hostname for nova-compute" 2018-03-28 07:25:55 +00:00
Eduardo Gonzalez
ea1a1dee0d Verify YAML syntax in gates
This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799
2018-03-26 17:56:22 +02:00
Zuul
37f6e92bb1 Merge "Configure application for ceph pool" 2018-03-26 02:51:11 +00:00
Jeffrey Zhang
6e377622be Configure application for ceph pool
pool application is mandatory since ceph Luminous.

Change-Id: I8c124899b46b1388ab3c2957b4ece0f6aab1378b
Closes-Bug: #1757957
2018-03-22 15:13:31 +08:00
wu.chunyang
5e440d7939 Remove the duplicate "become" for config-nova-fake.yml
Change-Id: I109d431eefa8725540b893b4dc61525b46b29c6a
Co-Authored-By: pippo <huang.zhiping@99cloud.net>
2018-03-22 02:55:59 +00:00
Jianghua Wang
3b89a7d466 XenAPI: use dom0's hostname for nova-compute
When the virt dirver is xenapi, we should explicitly specify host
with dom0's hostname for nova-compute. The host field should be same
as neutron-openvswitch-agent-xenapi.

In order to use dom0's hostname, we need pull the bootstrap_xenapi
to before running nova config tasks. The dom0's hostname will be
included in the XenAPI facts.

Change-Id: Ibfc964bbd6c569062ad33c1399f9a4612baf5b92
blueprint: xenserver-support
2018-03-20 08:54:50 +00:00
Zuul
fde4f2112d Merge "Handle external ceph configs as templates" 2018-03-13 07:42:28 +00:00
Zuul
9275ba878f Merge "FIX inject password error" 2018-03-12 14:27:32 +00:00
Christian Berendt
fcf755c8ce Handle external ceph configs as templates
Change-Id: Ice5b058fe17ad7d648f21d1ddacaa339c6f7102f
2018-03-06 18:47:42 +01:00
Jianghua Wang
65fa37fb84 XenAPI: centralize the facts files
When bootstrap compute hosts for XenAPI, it will generate a facts
file for each compute node. It contains some XenAPI specific variables
for both the compute host and the XenServer where the compute host
run on. This commit is to fetch the facts file into deployment host
and put it under a centralized directory - each compute host will
have a separate sub-dir which is named with its *inventory_hostname*.

In this way, the following tasks can use proper variable from the
proper facts file which exactly belongs to the host they running on.

Change-Id: I68d1a2d098d38c8e6bf4db76cdaf1f0465831822
blueprint: xenserver-support
2018-03-05 13:57:15 +00:00
Dai Dang Van
d77930373e Support policy.yaml file [part 2]
- Keystone
- Glance
- Nova
- Cinder

This will copy only yaml or json policy file if they exist.

Change-Id: I4a9415d82322aed68c9b7650bdf346f58fa49e2a
Implements: blueprint support-custom-policy-yaml
Co-authored-By: Duong Ha-Quang <duonghq@vn.fujitsu.com>
2018-01-29 13:35:59 +00:00
Alexandru Bogdan Pica
8e3b79440c Implement external MariaDB and pre-configured Databases support
This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121
2018-01-23 13:07:40 +00:00
Jianghua Wang
b2c7269ee7 XenAPI: Support compute node bootstrap
This is to add a bootstrap task for XenAPI to bootstrap the compute
node. One compute node is composed with a XenServer and a DomU VM
running on the XenServer. Before it's ready for deploying OpenStack
sevices on it, we need bootstrap it firstly. The package of
python-os-xenapi contains a tool and some utils which were developed
to finish all of the needed bootstrap tasks
e.g.:
1. configure himn(host internal management network)
2. configure ipatables to allow traffic
3. install xapi plugins to dom0
4. gather XenAPI facts and save them into a file
...

The task added in this commit is to invoke that tool so that the
bootstrap tasks can be completed automatically during kolla deployment.

Depends-On: Ie2d7d40f2755580aac4a10f3d302190a8bd4fe6f
Change-Id: I1500535a1c9a085bcbfec5db5fbddcf040e6892d
blueprint: xenserver-support
2018-01-19 10:59:11 +00:00
Zhaokun Fu
93fcfe7134 FIX inject password error
The envirenment variable LIBGUESTFS_BACKEND = direct
is not enabled by default in docker container.
Without it, GuestFS() init failed.

Closes-Bug: #1742029

Change-Id: I24330502df7abc8e8f952ebb41bd9ae5e4ba1168
2018-01-12 07:54:59 +00:00
Zuul
89d5a24ad1 Merge "Allow empty compute group when ironic is in use" 2017-12-29 08:20:19 +00:00