103 Commits

Author SHA1 Message Date
Ethan Lynn
e237319965 Always check enable_* variables first
Check enable_* variables first, then check inventory_host in
group, will help to avoid configuration errors.

Change-Id: Icdb1f50e5c911203b92ac431723620756b15f3c6
Closes-Bug: #1648376
2016-12-08 17:59:48 +08:00
Eduardo Gonzalez
f56d8e58ba Revert "updating-deprecated-ansible-modules"
check_mode option is included in Ansible 2.2.
Using in our playbooks mean that any other version before
Ansible 2.2 can be used

This reverts commit 529f202d00e4615caa1fcb447aaa47fde6aa1417.

Change-Id: I3af96290443d760346264e6d994fd2a44de65543
Closes-Bug: #1644828
2016-11-25 14:41:38 +00:00
gardlt
529f202d00 updating-deprecated-ansible-modules
* update ceph tasks
* update mariadb tasks
Closes-Bug: #1643633

Change-Id: Ib81789574843edba6e33394a7f66a2e8077075eb
2016-11-21 21:52:26 -06:00
Jeffrey Zhang
fa45801946 Move precheck into its own role
* Merge prechecks.yml and site.yml playbook
* Create empty precheck.yml into all roles.

Change-Id: I8a138558a26c0a2a66c5fd48ed37be657c99c1dd
Implements: blueprint condition-pre-check
2016-11-03 14:48:58 +08:00
Jeffrey Zhang
fa856556d5 Remove do_reconfigure.yml file for all roles
do_reconfigure.yml is introduced to use serial directive. But we use
it in wrong. Now serial has moved to playbook file. So it is time to
remove the do_reconfigure.yml file

Closes-Bug: #1628152
Change-Id: I8d42d27e6bc302a0e575b0353956eaef9b2ca9fd
2016-09-27 15:04:00 +00:00
Jenkins
6d03a05819 Merge "Fix permissions of tmp files" 2016-09-13 16:44:10 +00:00
Christian Berendt
924fbb6f7a Fix permissions of tmp files
When running kolla-ansible as non-root users the tmp files will
be created with become user as owner and mode 0600. This way the user
running kolla-ansible is not able to read the generated files. This
is necessary because of the use of the lookup method, which will
not run as become user.

TrivialFix

Change-Id: If86d114c05a9b2e7fa240718611d78e93100ebdf
2016-09-13 13:30:25 +02:00
Mick Thompson
4c1482b999 Added serial option to site.yml.
Useful for upgrade etc., which is preferablly done serially.

Example usage: tools/kolla-ansible deploy OR tools/kolla-ansible upgrade

Closes-Bug: #1576708

DocImpact

Change-Id: I34b2e16f8ce53e472a4682a4738c4ac0f5abf00c
2016-09-13 09:26:51 +00:00
Paul Bourke
ec40ed3cbb Add a check before wiping Ceph disks in bootstrap
An unwitting user may apply the KOLLA_CEPH_OSD[_CACHE]_BOOTSTRAP label
to a partition assuming it will only use that partition for Ceph, and
end up wiping out their disk.

This change adds a layer of checking to this scenario to try and help
avoid a disaster scenario.

Closes-Bug: 1599103
DocImpact

Change-Id: Ibb9fb42f87a76bc02165ec0b93b60234bad8747a
2016-08-29 14:34:01 +01:00
Duong Ha-Quang
34975e291e Port with_ loop variable syntax to Ansbile 2
Migrate to full variable syntax in with_ loop
instead of bare variables for:
- ceph
- ceilometer
- glance
- heat
- horizon
- keystone
- mariadb
- memcached
- neutron
- nova
- rabbitmq

Change-Id: Ib925b4ecea47ac758a77739c9c7e140f06933b08
Partial-Implements: blueprint ansible2
2016-08-09 23:23:21 +07:00
Paul Bourke
a479afac4a Use inventory_hostname for Ceph bootstrap
The inventory_hostname is more reliable than ansible_hostname for the
way we use it here. If the deploy node cannot resolve the hostname of a
target node (the operator is using IP addresses in their inventory), the
bootstrap will fail with:

"Host: x.x.x.x, Task: ceph : Fetching Ceph keyrings, Status:
unreachable, Message: SSH Error: data could not be sent to the remote
host.  Make sure this host can be reached over ssh"

Closes-Bug: 1599138
Change-Id: Ic76e4baf8c642c59bbe81e6671ac97bcbb57d470
2016-07-10 05:16:10 +00:00
Stephen Hindle
cadf8afdcf Read GPT label using sgdisk rather than udev
The Ceph osd bootstrap/startup logic depends upon reading/writing
partition names.  Some older versions of udev have trouble reading these
(/dev/disk/by-label is missing).

To work around this, we shell out to sgdisk and scrape the partition
name data directly.

Co-authored-by: Paul Bourke <paul.bourke@oracle.com>
Closes-Bug: 1585185
Change-Id: I362b3f8e91de79687fc84e256996fbcaf303b6af
2016-07-07 10:26:34 +01:00
Paul Bourke
2d4161de63 Improve task names for Ceph
The task names while bootstrapping Ceph OSDs and cache OSDs were
identical making them harder to distinguish in the Ansible output.

TrivialFix

Change-Id: I0228221a09bdaaba46b6280b6e1be9920ad13598
2016-07-05 16:06:40 +01:00
Christian Berendt
2726f6edf2 Do not use bare variables in the ceph role
This will resolve the following deprecation warning:

Using bare variables is deprecated. Update your playbooks so
that the environment value uses the full variable syntax. This
feature will be removed in a future release.

Partial-bug: #1586404
Change-Id: I9086d3f9e7904836e72e6df2fefaa084e5229f7e
2016-05-27 13:55:37 +00:00
Ryan Hallisey
2da010a7b9 Make configurable the location where config files are merged
An operator may want to specify the location of custom config
files so that kolla can detect their location and merge
them with the default configs generated.

Partially implements: blueprint multi-project-config

Change-Id: Ibfb38d07a36dfa7fe25381adc34cc1d3cbe7d1e1
2016-05-23 03:23:06 -04:00
Francesco Vollero
a7d330d9e7 Fixed typos in Ansible files
Files affected:
- ansible/roles/ceph/tasks/bootstrap_mons.yml
- ansible/roles/neutron/tasks/deploy.yml
- ansible/roles/neutron/tasks/start.yml

TrivialFix
Change-Id: Ic4b624cfe0ae4d73b49e616319e9db2229931249
2016-05-18 08:33:15 +02:00
Mauricio Lima
1a2fe93a95 Make Heka collect Ceph logs
Change-Id: Ibaa5f8ba600afafa111d2ef204ee9192b8babdfc
Closes-Bug: #1570529
2016-05-05 13:26:44 -04:00
Jeffrey Zhang
0fcee87549 map the host localtime to the container
Closes-Bug: #1577148
Change-Id: I636cefc63cf532434a41af3898b63dffa711e280
2016-05-03 09:27:51 +08:00
Jeffrey Zhang
f7066db4f0 Fix the Ceph reconfigure
Closes-Bug: #1563251
Change-Id: I0072c92d6c8160f868b4ab0ec08b11541d3f3401
2016-03-29 18:05:48 +08:00
SamYaple
5250a00781 Allow external ceph journals and fix bootstrap
This allows us to specify external journals for osds which can greatly
improve performance when the external journals are on the solid-state
drives.

The new lookup and startup methods fix the previous races we had
preventing osds from being created properly.

This retains the same functionality as before and is completely
compatible with the previous method and labels, however this does set
new labels for all new bootstrap OSDs. This was due to a limitation
in the length of the name of a GPT partition.

Closes-Bug: #1558853
DocImpact
Partially-Implements: blueprint ceph-improvements
Change-Id: I61fd10cb35c67dabc53bd82270f26909ef51fc38
2016-03-19 22:36:30 +00:00
SamYaple
6d0cd535d3 Allow user to choose filesystem for osd
Change-Id: I9c770a08219d2eead5ece8e3462157359cd920cb
Partially-Implements: blueprint ceph-improvements
2016-03-19 22:36:30 +00:00
SamYaple
1f371e2717 Allow per node configuration with augments
This type of per node configuration is required to support things like
availability zones for nova. As always, if this file doesnt exist it
doesnt get used so this change is safe.

TrivialFix

Change-Id: Iff8172af522c2c96e5f2c173b24a5dfd4d522ed2
2016-03-17 17:18:00 +00:00
SamYaple
7b6fbdf41d Remove storage_hostname
This was an attempt to get storage_interface to work properly but that
work will not be completed and functional this cycle. There are design
topics that need to be discussed about it that were brought to light
by the RAX gate failing for it.

TrivialFix

Change-Id: I65579f9e0e0dcf3fa51c0ea031ff474145457c40
2016-03-09 12:08:15 +00:00
Jenkins
eacaf76241 Merge "Move hostname registration to common" 2016-03-07 19:28:45 +00:00
Vikram Hosakote
efd1099335 Reconfigure ceph service
This patch set implements reconfiguring the ceph service.

Change-Id: I904d5e4b33c82e20a8e7e84308b62573f59de714
Partially-implements: bp kolla-reconfig
2016-03-04 18:06:50 +00:00
SamYaple
7f6de0a7e0 Move hostname registration to common
The storage hostname is used in many conf files all over the place and
should be registered upon each run for every service

TrivialFix

Change-Id: I1d5a20cb6d51cee9f529cf2e1fb144158f7718a9
2016-03-03 15:18:30 +00:00
SamYaple
e76f862e99 Add upgrade play for ceph
Ceph is pretty easy to work with. Upgrade mons, then osds, then rgws

We want to eventually make these serial values configurable, but for
now due to cephs delicate distributed network nature it is safest to
only run 1 change at a time.

Change-Id: Icc721ab3651379c28fee853ca95f9e3ddf102998
Partially-Implements: blueprint upgrade-kolla
Implements: blueprint upgrade-ceph
2016-02-26 15:45:43 +00:00
Jenkins
c79174012c Merge "Add the default empty reconfigure.yml file to all roles" 2016-02-25 21:58:56 +00:00
Jeffrey Zhang
d26c259754 Add the default empty reconfigure.yml file to all roles
DocImpact

Partially-implements: bp kolla-reconfig
Change-Id: I9738e80960bcfbef18d1ef1b7942f81c45684e85
2016-02-25 09:32:53 +08:00
SamYaple
1e2dfad50b Switch ceph to use storage network
Ceph was not properly using storage network.

TrivialFix

Change-Id: Ibf3da5d19cd2ca874d251b455a7eb856154fc3f7
2016-02-24 17:19:52 +00:00
SamYaple
80b7266ed1 Rename kolla_ansible to kolla_toolbox
This change is needed for clarity. We have a kolla-ansible script.
We have a kolla-mesos repo. We plan to have a kolla-ansible repo.
Already we have had far too much confusion about whether we are
talking about the container or the project. Naming this kolla-toolbox
eliminates all of that confusion and its probably a bit more accurate
of a name too.

Closes-Bug: #1541053
Change-Id: I8fd1f49d5a22b36ede5b10f46b9fe02ddda9007e
2016-02-02 18:12:15 +00:00
Jenkins
5fc2794eb0 Merge "Update ceph-mon keyrings" 2016-01-29 18:47:57 +00:00
SamYaple
6d8959a962 Update all bootstraps to use Label
Add bootstrap label to all bootstrap containers to ensure that when
the a new container is launched a difference is seen between it and
the bootstrap container since we cannot rely on ENV variables for
this. This only affects mariadb at this stage, but it is needed to
ensure rabbitmq works when we switch to named volumes.

Change-Id: Ia022af26212d2e5445c06149848831037a508407
Closes-Bug: #1538136
2016-01-28 20:26:41 +00:00
SamYaple
33f3e1d8ca Update ceph-mon keyrings
Since the fetch script fetched _all_ keyrings from the ceph-mon
container, the ceph-mon container must contain all keyrings. This
setup works AIO but was broken on multinode because the ceph-mon
container did not have the radosgw keyring. This issue affects every
multinode install regardless of using the radosgw or not.

TrivialFix

Change-Id: Ie416de1a5275862da6d77ef0dd174e85e499fc0f
2016-01-28 17:47:30 +00:00
Michal Jastrzebski
375965ddfb Add stub upgrade.yml
After introduction of pull action and turing every main.yml into
{{action}}.yml we lost ability to perform upgrade

Change-Id: Ie9fa2cd083b061033abc733fba53d54f9c55e393
Fixes-Bug: #1538210
2016-01-26 11:52:19 -06:00
Jenkins
bb12f82c70 Merge "Rename ceph-mon bootstrap container to avoid conflict" 2016-01-21 19:05:13 +00:00
Jenkins
0416913fb5 Merge "Fix ceph-mon existing cluster detection logic" 2016-01-19 17:48:36 +00:00
SamYaple
af0d73ea38 Rename ceph-mon bootstrap container to avoid conflict
Ceph-mon bootstrap container shares the same name of `ceph_mon` with
real ceph-mon service. When bootstrapping, it will sleep forever for the
purpose of distribute rings. Later on, when starting real `ceph_mon`
service, kolla_docker module will take bootstrap container as real one
because bootstrap container has a super set of env variables and
kolla_docker doesn't do strict comparing of env variable items.

This change follows the pattern of other bootstrap containers and rename
to use `bootstrap_ceph_mon` to avoid conflict.

Co-Authored-By: Sam Yaple <sam@yaple.net>

Change-Id: I0082820900928c42f9cc778ffd705980fea9ad1c
Closes-Bug: #1535140
2016-01-17 23:20:54 +00:00
Qiu Yu
8c125f01fd Fix pid leftover after converting ceph to kolla_docker
Related-Id: Icb1e81dc8acabe51740edc5658957022ddd13cd7
TrivialFix

Change-Id: Icf7cca554a972370bb109530cce3a0d451c95ae1
2016-01-18 02:32:47 +08:00
Qiu Yu
20c0f4c3f1 Fix ceph-mon existing cluster detection logic
After moving from data container to named volumes, ceph monmap detection
is broken. This change fix the logic by leverage kolla_docker's
idempotency creating a named volume.

Change-Id: Iabc2e2fe5f526e1a6fc2e1b1158d1702aaf4bdf9
Related-Bug: #1533368
2016-01-16 06:17:44 +08:00
SamYaple
cba75b4084 Allow ceph role to pull images
Change-Id: I7176bdc897e07f28684eedffb3c59e407eba322e
Partially-Implements: blueprint pre-pull-images
2016-01-04 19:49:27 +00:00
SamYaple
8af400733d Convert to named volumes instead of data container
This intentionally leaves out rabbitmq from this patchset. It will
require additional work to remove its data container

UpgradeImpact
Partially-Implements: blueprint docker-named-volumes

Change-Id: Id68b8e43a3c077ef4f4f4d67ea34d0692e66eef7
2016-01-03 16:44:44 +00:00
SamYaple
b018c1c23f Convert Ceph to kolla_docker
Change-Id: Icb1e81dc8acabe51740edc5658957022ddd13cd7
Partially-Implements: blueprint kolla-docker-module
2015-12-31 16:54:40 +00:00
Andrey Shestakov
1a368c9134 Add RadosGW container
Implements blueprint radosgw-container

Change-Id: Idc88a67a0979be626d3eaa9b2b9a527010aa2006
2015-12-29 12:49:03 +00:00
SamYaple
ed82afa8e9 Simplify config creation
Convert config creation from a playbook to an action_plugin. This
reduces the complexity and confusion while retaining the same augment
structure and flexibility.

This allows us to remove the 0-byte files as requirements. They will
still be used if they are present (this means we require additional
documentation around them).

DocImpact
Closes-Bug: #1528430
Change-Id: I2c789f6be9f195c7771ca093a6d59499564b4740
2015-12-22 04:28:53 +00:00
Sam Yaple
dd16395ae1 Switch find_disks to use a decent python lib
Now /dev/loop devices will work!

Change-Id: Icb8efe4f9bc3a21a72eb04bfd03452c26f13fd70
Closes-Bug: #1518438
2015-11-20 23:24:53 +00:00
Sam Yaple
b27880aa77 Automate setting up a cache tier
Introduces a new flag to bootstrap cache devices

DocImpact
Partially-Implements: blueprint ceph-improvements

Change-Id: I09b5a0d5c61b3465237e5f01dc10120725561cd3
2015-10-22 10:51:44 +00:00
Sam Yaple
b2ac6e80cd Add tty to containers
sudo requires a tty to function by default on centos. Instead of
tweaking the sudo conf we can just add a tty. This has the added
advantage of making the containers more friendly if you have to
`docker exec -i <container> bash` into them.

Change-Id: If97a02ca1d37c243a787d98ade54bde8d641aecd
backport: liberty
Partially-Implements: blueprint functional-testing-gate
2015-10-14 10:54:33 +00:00
Sam Yaple
c5be228be8 Bump ansible to 1.9.2
Ansible 1.9.2 contains the fix needed for docker-py >1.2.0

This is needed for some gate reasons, but it is also a good version
bump because it fixes a few issues with delegate_to.

Change-Id: Iafbabb3b0232620849d0548c5cd9d8d316c2b0f3
backport: liberty
Partially-Implements: blueprint functional-testing-gate
2015-10-13 15:13:34 +00:00
Sam Yaple
94162bd08f Bring Kolla inline with FHS
This brings Kolla images inline with FHS and should make finding
locations of things more consistent and reliable with the linux world
at large.

Change-Id: Iece5b4da4bace0fb8b1f41a65ab2c852ec73e6f8
Closes-Bug: #1485742
2015-10-06 03:30:53 +00:00