9614 Commits

Author SHA1 Message Date
Mark Goddard
56b4352f9e Fix fact gathering with --limit
Prior to this change, when the --limit argument is used, each host in the
limit gathers facts for every other host. This is clearly unnecessary, and
can result in up to (N-1)^2 fact gathers.

This change gathers facts for each host only once. Hosts that are not in
the limit are divided between those that are in the limit, and facts are
gathered via delegation.

This change also factors out the fact gathering logic into a separate
playbook that is imported where necessary.

Change-Id: I923df5af41a7f1b7b0142d0da185a9a0979be543
2018-12-07 14:49:27 +00:00
Mark Goddard
5d8403bdc8 Scalability improvements for disabled services
Currently, every service has a play in site.yml that is executed, and
the role is skipped if the service is disabled. This can be slow,
particularly with many hosts, since each play takes time to setup, and
evaluate.

This change creates various Ansible groups for hosts with services
enabled at the beginning of the playbook. If a service is disabled, this
new group will have no hosts, and the play for that service will be a
noop.

I have tested this on a laptop using an inventory with 12 hosts (each
pointing to my laptop via SSH), and a config file that disables every
service. Time taken to run 'kolla-ansible deploy':

Before change: 2m30s
After change: 0m14s

During development I also tried an approach using an 'include_role' task
for each service. This was not as good, taking 1m00s.

The downsides to this patch are that there is a large number of tasks at
the beginning of the playbook to perform the grouping, and every play
for a disabled service now outputs this warning message:

[WARNING]: Could not match supplied host pattern, ignoring: enable_foo_True

This is because if the service is disabled, there are no hosts in the
group. This seems like a reasonable tradeoff.

Change-Id: Ie56c270b26926f1f53a9582d451f4bb2457fbb67
2018-12-07 14:48:53 +00:00
Zuul
29a2cda220 Merge "change the state value from installed to present" 2018-12-07 11:33:58 +00:00
Zuul
f1be7033b0 Merge "Allow set tty for containers" 2018-12-07 09:30:57 +00:00
XiaojueGuan
077cc3eeb2 change the state value from installed to present
refer:
https://docs.ansible.com/ansible/2.5/modules/package_module.html

Change-Id: I68a0eb64a61bc6c0f77cbae7e8b4f4c7143202c5
2018-12-06 14:35:04 +00:00
Zuul
d7a905eb9e Merge "Fix glance configuration when using external ceph" 2018-12-06 09:51:19 +00:00
Eduardo Gonzalez
846c15d8e1 Allow set tty for containers
This change adds support to comfigure tty,
it was enabled by default but a recent patch
removed it. Some services such as Karaf in opendaylight
requires a TTY during startup.

Closes-Bug: #1806662
Change-Id: Ia4335523b727d0e45505cbb1efb40ccf04c27db7
2018-12-05 11:44:37 +00:00
Zuul
ee9ff448aa Merge "Fix fluentd input log parser for OpenDaylight" 2018-12-05 10:35:05 +00:00
Zuul
daec9a177d Merge "Allow operators to customise Nova vendor info" 2018-12-05 10:35:04 +00:00
Jeffrey Zhang
6f020a0483 Fix glance configuration when using external ceph
When using external ceph, enable_ceph=no and glance_backend_ceph=yes,
glance.conf should enable rbd store.

Change-Id: Ia09cd57c829b00f28674cddf44fb55583e193d0f
2018-12-05 12:05:29 +08:00
zhangbailin
ef3df55228 Change openstack-dev to openstack-discuss
Mailinglists have been updated. Openstack-discuss replaces
openstack-dev.

Change-Id: I848598c6aa4fb40f759302da1a830d1f19ac06a2
2018-12-03 22:39:21 -05:00
Paul Bourke
a16d78711f Allow operators to customise Nova vendor info
Nova allows customisation of various metadata passed through to VMs via
a 'release' file[0]. Allow operators to make use of this.

[0] https://github.com/openstack/nova/blob/master/etc/nova/release.sample

Change-Id: I71569314c8e64320f8ffad79b9273f4d6d903bb6
2018-11-30 09:48:28 +00:00
Zuul
e59867e951 Merge "Add ironic_dnsmasq_interface parameter to globals.yml" 2018-11-28 10:26:01 +00:00
ya.wang
1795ea8959 Add ironic_dnsmasq_interface parameter to globals.yml
Specifying a nic for dnsmasq when using ironic inspector,
by default is network_interface

Change-Id: I579c667c40b14030f340c1f3d7f57fe37613ee1a
2018-11-27 18:58:15 +08:00
Zuul
7bf519cd43 Merge "Remove the memcache container restart" 2018-11-27 10:58:13 +00:00
Farid Da Encarnacao
209d9c7680 Fix karbor upgrade
Remove mode "0660" because mode it's not a supported parameters for kolla_docker

Change-Id: I1e3d690eb3cb5d61b1c88f6da2f9b10e2c5f3603
Closes-Bug: #1804702
2018-11-26 16:21:17 -05:00
caoyuan
d154919865 Remove the memcache container restart
the bug[0] has beed fix, restart container is unncessary, remove it
is safe.

[0]: https://bugs.launchpad.net/oslo.cache/+bug/1590779

Change-Id: Iadfb3666bad8160b5c071fa1a512e2dcdda84043
2018-11-26 23:43:39 +08:00
Zuul
c6e1ae982d Merge "Add neutron metering role into kolla-ansible" 2018-11-26 11:45:45 +00:00
Zuul
0ca3adc8d9 Merge "Support stop specific containers" 2018-11-26 11:26:00 +00:00
Zuul
7ec4a2cfbf Merge "Add missing steps to Vagrant instructions" 2018-11-26 09:46:03 +00:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
Zuul
23415bf8bb Merge "Rework documentation into more specific folder" 2018-11-26 05:36:55 +00:00
Eduardo Gonzalez
02295e7283 Rework documentation into more specific folder
Change index to ease identify what service want to look.
Split docs into more specific folder such as networking
and storage.

Change-Id: Ic7ac12b3dd555fa5c018eeb897ccd4a5a2dfe8f3
2018-11-23 10:27:12 +00:00
Nick Jones
f704a78029 Add new option to perform an on-demand backup of MariaDB
blueprint database-backup-recovery

Introduce a new option, mariadb_backup, which takes a backup of all
databases hosted in MariaDB.

Backups are performed using XtraBackup, the output of which is saved to
a dedicated Docker volume on the target host (which defaults to the
first node in the MariaDB cluster).

It supports either full (the default) or incremental backups.

Change-Id: Ied224c0d19b8734aa72092aaddd530155999dbc3
2018-11-22 09:20:59 +00:00
Zuul
86f29f7c11 Merge "Load configfs kernel module for iscsid" 2018-11-21 16:08:57 +00:00
Zuul
9ca1039e26 Merge "Support Ironic Inspector dnsmasq PXE filter" 2018-11-21 15:10:44 +00:00
Zuul
bf15842e70 Merge "Add YAML format into docs for policy file" 2018-11-21 15:10:43 +00:00
Zuul
11dedb653a Merge "Set "no_log" for "databases user and setting permissions" tasks" 2018-11-21 15:10:41 +00:00
Zuul
62290431a1 Merge "Document glance service" 2018-11-21 15:10:40 +00:00
Zuul
2d3e47759c Merge "Add glance-cache support" 2018-11-21 15:10:39 +00:00
jacky06
377222bb00 Add YAML format into docs for policy file
kolla-ansible support yaml format for policy file[1] too, but the
docs missing it, this ps to add it.

[1]: https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L393
Closes-Bug: #1804455

Change-Id: I44eb1d64f9299ccaf99972c8b5354683a3501f6b
2018-11-21 08:34:12 -05:00
Eduardo Gonzalez
a86834e01b Document glance service
Add documentation about glance backends,
upgrades and futher configuration.

Change-Id: Ibb15804fa3f38abab855084dd7d7b83fd4a54b8f
2018-11-21 11:14:13 +00:00
Eduardo Gonzalez
cc9dae4dee Add glance-cache support
Glance cache is used to keep a locally cache image
in the glance_api service.
Is an usefull service when an image is commonly used
to speed times between pulling from storage backend
and send to nova.

Change-Id: I8e684cc10e4fee1cb52c17a126e3b11f69576cf6
2018-11-21 10:15:48 +01:00
Zuul
399f452ec7 Merge "Suppress log copy output in gates" 2018-11-20 18:21:42 +00:00
Mark Goddard
5753a569c7 Load configfs kernel module for iscsid
The configfs kernel module is not mounted by default in Ubuntu 16.04,
leading to the iscsid container failing to start because it bind mounts
/sys/kernel/config. The issue does not apply to Ubuntu 18.04, or other
distros (AFAIK), which load configfs by default.

This change loads the configfs module when the iscsid container is in
use.

Change-Id: I5b521ddca24b919658d2664ede2d878507d6d106
Closes-Bug: #1631072
2018-11-20 15:52:56 +00:00
caoyuan
a5badc89f2 Add neutron metering role into kolla-ansible
Refer to neutron docs [1][2]
[1]: https://docs.openstack.org/neutron/pike/admin/archives/adv-config.html#l3-metering-service-driver
[2]: https://wiki.openstack.org/wiki/Neutron/Metering/Bandwidth
Co-Authored-By: ZhijunWei <wzj334965317@outlook.com>

Change-Id: I4a676d041bc2a86497cb139d9347365738b156df
2018-11-20 23:36:45 +08:00
Mark Goddard
4418c1641b Support Ironic Inspector dnsmasq PXE filter
The dnsmasq PXE filter [1] provides far better scalability than the
iptables filter typically used. Inspector manages files in a dhcp-hostsdir
directory that is watched by dnsmasq via inotify. Dnsmasq then either
whitelists or blacklists MAC addresses based on the contents of these
files.

This change adds a new variable, ironic_inspector_pxe_filter, that can
be used to configure the PXE filter for ironic inspector. Currently
supported values are 'iptables' and 'dnsmasq', with 'iptables' being the
default for backwards compatibility.

[1]
https://docs.openstack.org/ironic-inspector/latest/admin/dnsmasq-pxe-filter.html

Implements: blueprint ironic-inspector-dnsmasq-pxe-filter
Change-Id: I73cae9c33b49972342cf1984372a5c784df5cbc2
2018-11-20 14:01:15 +00:00
Eduardo Gonzalez
76482c9a68 Fix fluentd input log parser for OpenDaylight
OpenDaylight logs have different format than openstack,
is a karaf log with java error traces.
This PS add required config to make fluentd parse properly
ODL logs.

Change-Id: I34fb96c8a424679b3b618f2ff6a840b8dc165bec
2018-11-20 14:24:25 +01:00
Eduardo Gonzalez
a7dbc39240 Suppress log copy output in gates
Change-Id: I01e58d3548d6adc4a2d6f1088773df7941da3865
2018-11-20 11:14:24 +00:00
Zuul
7598edca5b Merge "Fix "Error: Connection activation failed: Failed to find a compatible device for this connection"" 2018-11-20 10:41:15 +00:00
Zuul
445e4f7640 Merge "Support custom monasca-notification templates" 2018-11-20 09:41:21 +00:00
Dat Le
5bbecaa3ed Fix "Error: Connection activation failed: Failed to find a compatible device for this connection"
Change-Id: I7887ff3317f9eac068fee117b493db10b0e97862
2018-11-20 15:00:17 +07:00
Doug Szumski
205df6949c Add missing steps to Vagrant instructions
Add a couple of missing steps to complete a Vagrant deployment. In
the case of the multi-node deployment we could go one step further
and ensure that the supplied inventory matches the default set of
nodes created by Vagrant.

Change-Id: Iee878e26989e92e4de06c071704a6794011b6e58
2018-11-19 18:23:15 +00:00
Zuul
52319dabfb Merge "Set ramdisk logs path for ironic inspector" 2018-11-19 12:23:47 +00:00
Zuul
d0bb2fdcba Merge "Don't allocate pseudo-TTY when creating containers" 2018-11-19 12:18:15 +00:00
Christian Berendt
03788e17d4 Set "no_log" for "databases user and setting permissions" tasks
At the moment the "databases user and setting permissions" task for
designate and nova leaks the database_password because of the use
of with_items:

---snip---
TASK [nova : Creating Nova databases user and setting permissions] *********************************************************
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova', u'database_username': u'nova'})
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova_cell0', u'database_username': u'nova'})
ok: [x -> y] => (item={u'database_password': u'password', u'database_name': u'nova_api', u'database_username': u'nova_api'})
---snap---

Change-Id: I141e4153223c8772c82a31d81e58057ce266c0b9
Co-authored-by: Bernd Müller <mueller@b1-systems.de>
2018-11-19 11:10:41 +00:00
Zuul
b5cf164cfb Merge "add monitoring of zookeeper" 2018-11-19 03:59:51 +00:00
jacky06
1f30414807 Update the openstack_release
Change-Id: Ia1b29ca85e8ddb4d9bc740888d92bb2378a44fbf
Closes-Bug: #1803874
2018-11-18 09:07:57 -05:00
zhulingjie
c645841958 update gather_facts to false for storm
add gather_facts to false to avoid duplicated

Change-Id: I633fc9042e7df24f5cdf3a62fd7ec619270469d5
Closes-Bug: #1806443
2018-11-17 14:42:38 -05:00
Zuul
5281e9e114 Merge "Use openstack command in CI" 2018-12-03 10:17:45 +00:00