335 Commits

Author SHA1 Message Date
Vikram Hosakote
7af0ab9ba9 Make heka collect libvirtd.log in nova_libvirt container
Test results of this patch set are at:
http://paste.openstack.org/show/491382/

Change-Id: I1bc8bdb3180f30dc69af28a35fa057ac801bde0f
Closes-Bug: #1560298
2016-03-22 03:23:41 +00:00
SamYaple
dcc15b9568 Wrong variable name for spicehtml5proxy
Reconfigure was hard broken with spiceproxy

Change-Id: I80b5fcc16bcc0b090807843f7a59f0ae9ea27aec
Closes-Bug: #1560175
2016-03-21 19:15:56 +00:00
Jenkins
b198da6318 Merge "Enabled nova->glance snapshots" 2016-03-21 12:33:38 +00:00
SamYaple
9fbcd6c2ba Enabled nova->glance snapshots
Thanks to the related-id commit we can do nova->glance snapshots. This
updates the keyrings to reflect that change.

Change-Id: I02f083aec0255e9d681bd225a11ead6f5a379366
Related-Id: If13d25b6c94e247d2505583b524ae3af9c34b5dc
2016-03-18 14:55:32 +00:00
Jenkins
f5bdcfbb60 Merge "Allow per node configuration with augments" 2016-03-18 03:45:06 +00:00
Jenkins
9c235f6b91 Merge "Change libvirt volume name to libvirtd" 2016-03-18 02:06:19 +00:00
SamYaple
1f371e2717 Allow per node configuration with augments
This type of per node configuration is required to support things like
availability zones for nova. As always, if this file doesnt exist it
doesnt get used so this change is safe.

TrivialFix

Change-Id: Iff8172af522c2c96e5f2c173b24a5dfd4d522ed2
2016-03-17 17:18:00 +00:00
SamYaple
b98bc3a458 Change libvirt volume name to libvirtd
This change is a "futureproofing" thing. It has already been discussed
that libvirt should not be a child of nova and should be removed out
to the base docker directory (just like openvswitch isn't a child of
neutron). That is not going to happen this cycle but when it does we
can't change the name of the volume. This updates the volumes to the
proper name of libvirtd. This is in contrast with the libvirtlogd
volume that will be needed in newton due to libvirt 1.3

Of note, the container can remain named nova_libvirt since we can
change that on the fly later without breaking instances.

This wont break liberty as named_volumes are not backported yet.

TrivialFix

Change-Id: I16cf9e1b1dbba9b5a9f5cc883494580e276d4f72
2016-03-15 22:41:00 +00:00
SamYaple
976c9c93c7 Switch to with items for sysctl
This will give more info to the user about the specific values being
set and allows us to grow this list easier.

TrivialFix

Change-Id: Iaf22bf2c9f4ab294bec0cca17699d461852109ac
2016-03-15 14:57:06 +00:00
Jeffrey Zhang
8170ea6f9c Fix the nova reconfig not work issue
The nova_compute nova_compute_ironic, nova_novncproxy
and nova_spicehtml5proxy do not be reconfigured due to
the wrong when condition

TrivialFix

Change-Id: Id27828b151301244da5d327b2b656780f283c4b4
2016-03-13 19:29:39 +08:00
Vikram Hosakote
1c33173927 Reconfigure nova service
This patch set implements reconfiguring the nova service.

Change-Id: If078efd9ca98f815ca7830de912a4bc8916ff403
Partially-implements: bp kolla-reconfig
2016-03-03 20:48:44 +00:00
SamYaple
57124620ab Fix Keystone v3 and Horizon
After our switch to keystone-manage bootstrap Horizon is not happy
due to v3 not being setup correctly. This patch fixes that

This also includes removal of unused variables (transforms them into
endpoint url variables)

TrivialFix
Change-Id: I1e04db8c24049f80e974c063f03068a2ab32a563
2016-03-03 15:44:53 +00:00
Allen Gao
1eeaf60497 Remove the role path from the template src
The path of the template file under the same role
can easily be omitted, and we are using this omitting
in most places except those this commit is fixing.

TrivialFix
Change-Id: I6d1563e235151669d9d9268d69555aae15e31926
2016-02-29 10:42:01 +08:00
SamYaple
d3cfb2052a Change kolla_internal_address variable
Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.

This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.

kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn

The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.

Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla
2016-02-26 20:00:09 +00:00
Jeffrey Zhang
d26c259754 Add the default empty reconfigure.yml file to all roles
DocImpact

Partially-implements: bp kolla-reconfig
Change-Id: I9738e80960bcfbef18d1ef1b7942f81c45684e85
2016-02-25 09:32:53 +08:00
Éric Lemoine
511a758982 Make Heka collect Nova logs
Partially implements: blueprint heka
Change-Id: I67dfc5d762d74454e4a852dd61c57fe9cd4b78c8
2016-02-19 21:49:18 +00:00
Michal Jastrzebski
ce4230fab1 Playbook for nova upgrade
This creates tree and playbook for nova upgrade. Also other service
upgrades will follow standard setup here.

Change-Id: Ic31759efaee4986eb87b9ff0968f13189d130d48
Partially-Implements: blueprint upgrade-kolla
Implements: blueprint upgrade-nova
2016-02-19 13:58:18 -06:00
Jenkins
6e622fa534 Merge "Move socket binding to named_volume" 2016-02-17 02:06:26 +00:00
SamYaple
690e6853de Move socket binding to named_volume
The extend_start.sh script for rsyslog is removed as it is no longer
needed. Docker no longer binds to /dev/log or /run/kolla/log

Closes-Bug: #1544545
Change-Id: Ic0a323a26ee4e9e15baf4598285844a8a4955f23
2016-02-16 14:42:41 +00:00
Dave McCowan
1cedf77f19 Use variables to specify http or https when constructing URLs
To allow for TLS to protect the service endpoints, the protocol
in the URLs for the endpoints will be either http or https.

This patch removes the hardcoded values of http and replaces them
with variables that can be adjusted accordingly in future patches.

Change-Id: Ibca6f8aac09c65115d1ac9957410e7f81ac7671e
Partially-implements: blueprint ssl-kolla
2016-02-15 09:48:58 -05:00
SamYaple
ddb7cd88c5 Convert Neutron to thin containers
*** Requires Docker 1.10 which is released ***

Documentation will be in the next patch. You must set the following
in your docker.service daemon control file for propogation to work:

    [Service]
    MountFlags=shared

======================================================================

Thanks to mount propagation in Docker 1.10 we can use thin containers
finally! This is extremely useful to operators since now they can
access the network namespaces from the hosts (outside the neutron
container). But additionally it allows us to implement the VPN agent
and other services easier.

Neutron containers and the neutron role are brought into the standards
of the new Kolla. Completely with drop-root and ansible formating
updates.

The ip_wrapper.py script was (thankfully) not needed so it has been
removed from the repo.

Partially-Implements: blueprint upgrade-neutron
Change-Id: Iaf5555283240457e1912459f397a6393d886fba1
2016-02-06 07:13:47 +00:00
SamYaple
e8d66766f0 Fix gate for docker 1.10
Docker 1.10 has broken the gate and this patch will correct that
breakage.

The issue comes with rsyslog. Due to a commit in Docker 1.10 [1] we
must change the way we get the log socket for rsyslog. The /dev/
folder will no longer populate as we used it. So instead we simply
make a new socket in a path we control and share that to the correct
location in the containers.

Additionally, adjust the gate for new Docker daemon.

[1] https://github.com/docker/docker/pull/16639
Partially-Implements: blueprint kolla-upgrade
Change-Id: I881a2ecdf6d7b35991e1d38a3f3e60d022d6577f
2016-02-06 06:48:53 +00:00
SamYaple
508f3863cc Fix two missing toolbox references
Two patches merged that did not have teh correct name for
kolla_toolbox

TrivialFix

Change-Id: I09e230e29cff496247c32bf227715bfb7caa0a95
2016-02-05 17:24:39 +00:00
Jenkins
b891afd64c Merge "Rename kolla_ansible to kolla_toolbox" 2016-02-05 15:04:37 +00:00
SamYaple
1ca35e688f Make libvirt reestablish connection with qemu
Libvirt stores some information in /run at runtime that is needed to
automatically reestablish a connection with the VM when a new
container is created. Without this information a long (and manual)
process is needed to redefine the running vms and reattach to the
running qemu process.

This mountpoint was removed as "unneeded" in the past, but it does
exist in Liberty branch enabling a no-vm-downtime upgrade.

TrivialFix

Change-Id: I2eb31c602d8d17cbd6a8e405daf4123070794843
2016-02-04 12:15:36 +00:00
SamYaple
e7dfd63bfc Create nova-api database
This is a required patch.

Kilo introduced a nova-api database but didn't use it. [1]
Mitaka now uses it and has broken booting instances. [2]

[1] https://wiki.openstack.org/wiki/ReleaseNotes/Kilo#OpenStack_Compute_.28Nova.29
[2] df0fca62cf/releasenotes/notes/request-spec-api-db-b9cc6e0624d563c5.yaml

Change-Id: If0a7f4ba4937fab8d60f5236ca00db06faa768c7
Closes-Bug: #1540579
2016-02-02 21:56:54 +00:00
SamYaple
80b7266ed1 Rename kolla_ansible to kolla_toolbox
This change is needed for clarity. We have a kolla-ansible script.
We have a kolla-mesos repo. We plan to have a kolla-ansible repo.
Already we have had far too much confusion about whether we are
talking about the container or the project. Naming this kolla-toolbox
eliminates all of that confusion and its probably a bit more accurate
of a name too.

Closes-Bug: #1541053
Change-Id: I8fd1f49d5a22b36ede5b10f46b9fe02ddda9007e
2016-02-02 18:12:15 +00:00
SamYaple
6d8959a962 Update all bootstraps to use Label
Add bootstrap label to all bootstrap containers to ensure that when
the a new container is launched a difference is seen between it and
the bootstrap container since we cannot rely on ENV variables for
this. This only affects mariadb at this stage, but it is needed to
ensure rabbitmq works when we switch to named volumes.

Change-Id: Ia022af26212d2e5445c06149848831037a508407
Closes-Bug: #1538136
2016-01-28 20:26:41 +00:00
Michal Jastrzebski
375965ddfb Add stub upgrade.yml
After introduction of pull action and turing every main.yml into
{{action}}.yml we lost ability to perform upgrade

Change-Id: Ie9fa2cd083b061033abc733fba53d54f9c55e393
Fixes-Bug: #1538210
2016-01-26 11:52:19 -06:00
Artur Zarzycki
ef971bff51 Fix permissions to /var/lib/nova/
Due to changes with the drop-root work, we lost the ability to write
to /var/lib/nova/*. This fixes those permissions and ensures cross
container talk works properly between nova_libvirt and nova_compute

Additionally, this fixes another issue introduced which saw that
nova-compute could not run sudo commands as it did not have a proper
sudoers entry

Testing from previous deploys means you need a fresh environment. You
have to remove all of the named volumes that kolla created in docker.
Check these with `docker volume ls`

Signed-off-by: Hui Kang <kangh@us.ibm.com>
Signed-off-by: Artur Zarzycki <azarzycki@mirantis.com>
Co-Authored-By: Sam Yaple <sam@yaple.net>
Co-Authored-By: Hui Kang <kangh@us.ibm.com>
Closes-Bug: #1533350
Change-Id: I7f864c448a2414e0b5d89f48337be411b891df35
2016-01-20 16:58:01 +00:00
SamYaple
575bc72f7b Allow nova role to pull images
Change-Id: Ic20c31efea461f30917a94546db6e261a8b5166f
Partially-Implements: blueprint pre-pull-images
2016-01-06 04:23:28 +00:00
SamYaple
8af400733d Convert to named volumes instead of data container
This intentionally leaves out rabbitmq from this patchset. It will
require additional work to remove its data container

UpgradeImpact
Partially-Implements: blueprint docker-named-volumes

Change-Id: Id68b8e43a3c077ef4f4f4d67ea34d0692e66eef7
2016-01-03 16:44:44 +00:00
SamYaple
57f6e3204d Convert Nova to kolla_docker
Change-Id: I0c6df8e7b732a37ff3cabff8d588f8b7d15df192
Partially-Implements: blueprint kolla-docker-module
2015-12-31 16:38:56 +00:00
Hui Kang
047dbf12e0 Enable nova fake driver in nova containers
- See instructions in doc/nova-fake-driver.rst

Implements: blueprint nova-fake-driver

Change-Id: I553a40c2df39bdcc391eb1b8b2b8fd5f4ed48c33
Signed-off-by: Hui Kang <kangh@us.ibm.com>
Signed-off-by: Marcio D. Silva <marcios@us.ibm.com>
2015-12-29 06:39:35 -05:00
Andrey Shestakov
c779511511 Add container for nova-compute-ironic
Implements blueprint ironic-compute-container

Change-Id: I110bf3aa94bdfd33038ed39d78bd40ea30b13e57
2015-12-25 17:09:00 +00:00
SamYaple
ed82afa8e9 Simplify config creation
Convert config creation from a playbook to an action_plugin. This
reduces the complexity and confusion while retaining the same augment
structure and flexibility.

This allows us to remove the 0-byte files as requirements. They will
still be used if they are present (this means we require additional
documentation around them).

DocImpact
Closes-Bug: #1528430
Change-Id: I2c789f6be9f195c7771ca093a6d59499564b4740
2015-12-22 04:28:53 +00:00
Jenkins
359f8e83d6 Merge "Remove the two different auth profiles" 2015-11-26 11:51:05 +00:00
Sam Yaple
970617b80a Make the database json variable more readable
Additionally remove tty from the container cleanup docker section. It
was added in a sed in a previous patchset by serves no purpose.

Change-Id: Ib617870616bca687f72ffaa44b2e9a3a11ef1011
Partially-Implements: blueprint cleanup-playbooks
2015-11-23 17:24:22 +00:00
Hui Kang
790bf336d5 Fix bootstrap for services
The bootstrap tasks could be started on any node listed in the
site.yml

The issue is that all the tasks must run on the same node, and the
only node that all the tasks can run on is the 'api' node due to
needing to start the bootstrap container which binds in the configs.

delegate_to is required to ensure that the proper node gets the task.

Co-Authored-By: Sam Yaple <sam@yaple.net>
Change-Id: Ica04ab366777a571a92212ac22c482f1881d4ecd
Closes-Bug: #1513598
Related-Bug: #1513439
2015-11-23 17:09:36 +00:00
Sam Yaple
8c1e7061f8 Remove the two different auth profiles
Ceph can function just fine generating the keys before the pools have
been created so we can apply the proper permissions to the auth string
ahead of time. This allows us to not require additional steps to add a
cache tier on the fly in the future.

Change-Id: I8214c567fb7c337f95d908c5699d1da922bfa1a6
Closes-Bug: #1518475
2015-11-20 23:24:53 +00:00
Sam Yaple
42b9660c7c Only start nova-data on the compute hosts
Change-Id: Icd7aa39556c50a3367c3b9630b24b7195612ba70
Closes-Bug: #1517914
2015-11-20 22:04:54 +00:00
Sam Yaple
564f8f2bd4 Move data container to start.yml
The original reason for having the data container in the bootstrap.yml
was to keep consistency with rabbitmq and mariadb, but in light of
this bug we need to move the data container out of the bootstrap.yml
role since we will be limiting the hosts that those tasks can run on.

Change-Id: I7f2f5979b01807275908699d1243756cb97d3588
Partial-Bug: #1513598
2015-11-09 10:47:16 +00:00
Sam Yaple
6b28343017 Incorrect parsed variable name
database_user_create was not correctly referenced when parsing the
variable names. This could never actually lead to a situation that
reported a false change, but it could break an operation if you were
using the --step option with ansible and skipped the database create
task.

TrivialFix
Backport: Liberty

Change-Id: Idf69fffcc3814f509448ccea11b7d175f074ccf1
2015-11-09 10:47:15 +00:00
Jenkins
44b5a8867e Merge "Finish implementation of spice" 2015-11-06 21:57:08 +00:00
Sam Yaple
99932d650d Finish implementation of spice
Due bad rebases there is a huge section of the spice patch missing
from the implementation unfortunately. This patch finishes the rest
of this patch out properly.

Change-Id: I693c6745e9594fd91eb6453f6de9dfcbd410e89c
Paritally-Implements: blueprint nova-proxies
2015-11-06 03:48:07 +00:00
Jenkins
037bdca42d Merge "bootstrap takes place on the wrong node" 2015-11-06 03:32:36 +00:00
Jenkins
58a9759b84 Merge "Allow disabling of sysctl values" 2015-11-06 03:31:22 +00:00
Sam Yaple
e2bc9dcf79 bootstrap takes place on the wrong node
The bootstrap must occur on the nova-api node due to binding in the
nova-api directory (same goes for all other services)

Closes-Bug: #1513439
Backport: Liberty
Change-Id: Iab88b49712828085e4d7e7f85e6d8f0b7999a9bf
2015-11-05 16:30:44 +00:00
Sam Yaple
1efdf4574d Allow disabling of sysctl values
The main reason for this change is to allow the DinD stuff to work. It
has limited use outside of that use case, but it may still be useful
to others in the future.

Change-Id: Ib3a4639cfb3fc0d378d33fc8b9ff8eb597f818ab
Partially-Implements: blueprint multinode-gate
2015-11-05 16:29:25 +00:00
Sam Yaple
347730cec8 Remove vip for rabbitmq
Adjust all the configs to list all the rabbitmq hosts rather than
running rabbitmq through the VIP. This is made possible by clusterer
which has already merged.

Change-Id: I5db48f5f10ec68f4c8863a29bc13984f6845a4f9
Partially-Implements: blueprint rabbitmq-clusterer
2015-11-05 15:43:20 +00:00