90 Commits

Author SHA1 Message Date
Serhat Demircan
c4ebd99030 link kolla_logs volume to docker_runtime_directory if docker_runtime_directory variable exists
Change-Id: I053c4a47e657effe25e8d75a3ef55477635729d2
2019-05-16 14:22:37 +03:00
ZijianGuo
6532c62c0d Install a list of packages to speed up package installation
Change-Id: I89313d7c279f95e80c593cf02309c8a9f8e6e318
Signed-off-by: ZijianGuo <guozijn@gmail.com>
2019-05-09 21:45:42 +08:00
Mark Goddard
88bf4075bd Don't install empty package name on old Ubuntu
Follow up to https://review.openstack.org/#/c/651136/, which fails on
Ubuntu versions prior to 18.04.

Change-Id: I74a85dec95ceed7c6eeafb9eb1920c7c30b3e64a
Related-Bug: #1813492
2019-04-10 12:07:33 +01:00
Mark Goddard
c1c02f67dc Don't use easy_install on Ubuntu 18+
Currently easy_install is not available on Ubuntu 18+ due to [1]. In
that case, install pip via apt rather than easy_install.

[1]
https://bugs.launchpad.net/ubuntu/+source/python-setuptools/+bug/1774419

Change-Id: Id358fdb655d71490b0915680dff131cfe33f4a40
Closes-Bug: #1813492
2019-04-09 09:28:42 +01:00
Mark Goddard
a4bb8567da Fix up config file permissions on the host
Several config file permissions are incorrect on the host. In general,
files should be 0660, and directories and executables 0770.

Change-Id: Id276ac1864f280554e98b937f2845bb424d521de
Closes-Bug: #1821579
2019-04-02 17:23:31 +01:00
Mark Goddard
6b0be5c5ba Remove recurse: yes for owner/perms on /etc/kolla
When kolla-ansible bootstrap-servers is run, it executes one of the
following two tasks:

- name: Ensure node_config_directory directory exists for user kolla
  file:
    path: "{{ node_config_directory }}"
    state: directory
    recurse: true
    owner: "{{ kolla_user }}"
    group: "{{ kolla_group }}"
    mode: "0755"
  become: True
  when: create_kolla_user | bool

- name: Ensure node_config_directory directory exists
  file:
    path: "{{ node_config_directory }}"
    state: directory
    recurse: true
    mode: "0755"
  become: True
  when: not create_kolla_user | bool

On the first run, normally node_config_directory (/etc/kolla/) doesn't
exist, so it is created with kolla:kolla ownership and 0755 permissions.

If we then run 'kolla-ansible deploy', config files are created for
containers in this directory, e.g. /etc/kolla/nova-compute/. Permissions
for those files should be set according to 'config_owner_user' and
'config_owner_group'.

If at some point we again run kolla-ansible bootstrap-servers, it will
recursively set the ownership and permissions of all files in /etc/kolla
to kolla:kolla / 0755.

The solution is to change bootstrap-servers to not set the owner and
permissions recursively. It's also arguable that /etc/kolla should be
owned by 'config_owner_user' and 'config_owner_group', rather than
kolla:kolla, although that's a separate issue.

Change-Id: I24668914a9cedc94d5a6cb835648740ce9ce6e39
Closes-Bug: #1821599
2019-03-25 15:39:21 +00:00
Mark Goddard
1b5353593c Fix bootstrap-servers on Ansible 2.6+
Recently as part of adding support for Docker CE we added the following
task to the baremetal role:

- name: Update yum cache
  yum:
    update_cache: yes
  become: True
  when: ansible_os_family == 'RedHat'

This works fine on Ansible 2.5, but no longer works on Ansible
2.6, which complains that either the 'name' or 'list' argument
is mandatory for the yum module.

This change updates the cache later on, when installing packages.

Change-Id: I1a158bda52c4e362cb12d361d7f961cfc699b385
Closes-Bug: #1819173
2019-03-08 14:36:08 +00:00
Mark Goddard
48aea5637f Support Docker CE in bootstrap-servers
Kolla Ansible's bootstrap-servers command provides support for
installing the Docker engine. This is currently done using the packages
at https://apt.dockerproject.org and https://yum.dockerproject.org.
These packages are outdated, with the most recent packages from May 2017
- docker-engine-17.05.

The source for up to date docker packages is
https://download.docker.com, which was introduced with the move to
Docker Community Edition (CE) and Docker Enterprise Edition (EE).

This change adds support to bootstrap-servers for Docker CE for CentOS
and Ubuntu.

It also adds a new variable, 'enable_docker_repo', which controls
whether a package repository for Docker will be enabled.

It also adds a new variable, 'docker_legacy_packages', which controls
whether the legacy packages at dockerproject.org will be used or the
newer packages at docker.com. The default value for this variable is
'false', meaning to use Docker CE.

Upgrading from docker-engine to docker-ce has been tested on CentOS 7.5
and Ubuntu 16.04, by running 'kolla-ansible bootstrap-servers' with
'docker_legacy_packages' set to 'false'. The upgrades were successful,
but result in all containers being stopped. For this reason, the
bootstrap-servers command checks running containers prior to upgrading
packages, and ensures they are running after the package upgrade is
complete.

As mentioned in the release note, care should be taken when upgrading
Docker with clustered services, which could lose quorum. To avoid this,
use --serial or --limit to apply the change in batches.

Change-Id: I6dfd375c868870f8646ef1a8f02c70812e8f6271
Implements: blueprint docker-ce
2018-12-17 14:04:43 +00:00
XiaojueGuan
077cc3eeb2 change the state value from installed to present
refer:
https://docs.ansible.com/ansible/2.5/modules/package_module.html

Change-Id: I68a0eb64a61bc6c0f77cbae7e8b4f4c7143202c5
2018-12-06 14:35:04 +00:00
Doug Szumski
bd54b99132 Constrain the size of Docker logs
Even though Kolla services are configured to log output to file rather than
stdout, some stdout still occurs when for example the container re(starts).
Since the Docker logs are not constrained in size, they can fill up the
docker volumes drive and bring down the host. One example of when this is
particularly problematic is when Fluentd cannot parse a log message. The
warning output is written to the Docker log and in production we have seen
it eat 100GB of disk space in less than a day. We could configure Fluentd
not to do this, but the problem may still occur via another mechanism.

Change-Id: Ia6d3935263a5909c71750b34eb69e72e6e558b7a
Closes-Bug: #1794249
2018-10-11 13:31:47 +01:00
caoyuan
747e8f2057 Disable the ntp service when bootstrap-servers
since we use chrony container to adjust time by default, we no need
to enable ntp service, this ps to disable it.

Change-Id: I2f1fd9269c9f8cfd0c98e0e903ba69de692473a0
2018-08-03 08:48:05 +00:00
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Mark Goddard
0cc2daa57d Fix sudoers in bootstrap-servers when user != group
In some cases we may want a configuration in which the kolla user's
primary group name is not the same as their username. Doing this
currently breaks the sudoers configuration, since user entries should
reference a user, or a group prefixed with a '%'.

There does not seem to be a good reason to give root privileges to the
entire group (which sometimes may be a shared group), so let's revert to
giving only the user root privileges.

See kayobe CI test [1] in which a different user and group were
configured, leading to permission denied when using kolla ansible.

[1] http://logs.openstack.org/53/581053/2/check/kayobe-overcloud-centos/a70168e

TrivialFix

Change-Id: I677778ebd0de58df0adfa2a8705f161ec5552283
2018-07-10 12:45:23 +01:00
Mark Goddard
8ec92df8e3 Make sudoers config optional in bootstrap-servers
In some environments it may not be desirable to modify the sudoers
configuration. This change makes this part of bootstrap-servers
optional, based on the create_kolla_user_sudoers variable.

Change-Id: I653403bfc5431741807edef57df58e05e679900b
2018-07-09 19:10:39 +01:00
Mark Goddard
2a69050e37 Append to kolla user groups in bootstrap-servers
This makes the bootstrap-servers command more idempotent, since without
the append argument set the kolla user will be removed from the docker
group before being added to it again in a later task.

TrivialFix

Change-Id: Iab0f6b5e18a103e9140631ee3ebbbb48c490bc24
2018-07-09 19:05:24 +01:00
Mark Goddard
4283dff9b4 Fix ownership of virtualenv in bootstrap-servers
In I86bf5e1df3d6568c4f1ca6f4757f08a3dd22754d, creation of the kolla user
was moved to after package installation to ensure the sudo package is
installed when required. This change does not work when python
dependencies are installed in a virtual environment however - when the
virtualenv variable is set.

This change moves the ownership change of the virtualenv to after the
kolla user has been created. It also uses the kolla_user and kolla_group
variables to set the user and group appropriately.

Change-Id: I320e5d611099ad162945a98d5505a79606da0eba
TrivialFix
2018-07-09 18:58:53 +01:00
Zuul
17cc67816f Merge "Option for enable SSL verification on docker registry" 2018-07-04 15:02:55 +00:00
Kevin Tibi
acfc4fd26a Option for enable SSL verification on docker registry
By default, kolla configure docker to use an insecure connection
with the private registry. If we want to use SSL verification we need
to add an option.

Change-Id: Id1805c9cfeb499da9bb56c70028f14c6f8bb20b6
2018-07-02 17:01:29 +02:00
fan_guiju
9ff5d5483e Add sudo package for bootstrap-servers
sudo package is required when we use ubuntu base on centos to deploy.

The following tasks belong to the environment check after
installation of environment-related software packages.
So, move to the post-install module.

    Create kolla user
    Add public key to kolla user authorized keys
    Grant kolla user passwordless sudo
    Ensure node_config_directory directory exists for user kolla
    Ensure node_config_directory directory exists

Change-Id: I86bf5e1df3d6568c4f1ca6f4757f08a3dd22754d
Closes-Bug: #1777571
2018-06-25 16:22:11 +08:00
Liping Mao (limao)
01be6a3b4e remove unused template file in baremetal role
Change-Id: I5f8598368ad93530d159816c004f27765b165afc
2018-06-07 16:37:24 +08:00
Kevin Tibi
b6bab5b931 Add custom option for docker
Some options can't be add in the daemon.json
The only way to configure daemon is to add option in the service file

Change-Id: I88697951ed6518f62bca30bb52288ad0e487ec03
2018-05-30 17:45:27 +02:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Eduardo Gonzalez
ea1a1dee0d Verify YAML syntax in gates
This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799
2018-03-26 17:56:22 +02:00
Zuul
2381fa89f6 Merge "Support virtualenv installation in baremetal role" 2018-03-24 15:00:01 +00:00
Zuul
7dc385577f Merge "Add a configuration about docker runtime directory" 2018-03-14 07:14:46 +00:00
Zuul
83573ca16e Merge "Use kolla_user variable to setting kolla user name" 2018-03-06 13:27:27 +00:00
zhubingbing
83fe717837 Use kolla_user variable to setting kolla user name
Change-Id: I653e16270aab539edcec6d8622f0a1c06d04b492
2018-02-27 15:39:27 +08:00
Zuul
dd694b2362 Merge "Fix the warning when reload the docker" 2018-02-26 09:46:17 +00:00
caoyuan
37ef38b6a3 Fix the warning when reload the docker
use systemd module to reload the docker service, rather than
command

Change-Id: I2e3b2299365701a08de4025d6ad34cb492b33fda
Closes-Bug: #1751690
2018-02-26 03:24:24 +00:00
Zuul
151d7fff55 Merge "Update task about selinux set." 2018-02-13 07:24:52 +00:00
David Rabel
12e0c5ae8e Use apt Ansible module instead of apt-get to update cache
Ansible recommends to use apt module instead of apt-get.
This patch fixes install.yml and pre-install.yml accordingly.

Change-Id: I3241ce332e7cf522786e78280643440a30a23875
Closes-Bug: #1747436
2018-02-05 15:38:17 +01:00
pengdake
4f98f08ffa Update task about selinux set.
1.Fix the invalid value about selinux policy
2.Update description of task about selinux.The permissive mode
need enable selinux.The parameter named "disable_selinux" is not good.
In order to customize selinux modes, we need a new
parameter named "selinux_state".

Closes-Bug: #1749046
Change-Id: I20c084cf2e46cc0de149afbd34c6dcb77a1051f4
2018-02-03 22:30:24 +08:00
Mark Goddard
69979efc2e Support virtualenv installation in baremetal role
Installing python packages directly to the system site-packages can
cause various problems, in particular when pip overwrites a system
package. Python virtualenvs are one solution to this issue, as they
allow python packages to be installed in an isolated environment.

This change adds support to the baremetal role for installing python
dependencies in a virtualenv. Typically we will need to enable use of
system site-packages from within this virtualenv, to support the use of
modules such as yum, apt, and selinux, which are not available on PyPI.

The path to the virtualenv is configured via the 'virtualenv' variable,
and access to site-packages is controlled via
'virtualenv_site_packages'.

When executing other kolla-ansible commands, the variable
'ansible_python_interpreter' should be set to the python interpreter
installed in 'virtualenv'. Note that this variable cannot be templated.

Change-Id: I0741923065246f9c5b168059fcd66504f2753c41
Related-Bug: #1731026
2018-01-21 08:59:49 +00:00
pengdake
f3e19ecf7b Add a configuration about docker runtime directory
Control the disk space used for Docker images, containers and
volumes is important for us.We need add configuration of
docker daemon to control the disk spaces used for docker images,
containers and volumes.

Change-Id: I3cf99f4e3f62c8406d37201b1cc24a83c68e3b27
Signed-off-by: pengdake <19921207pq@gmail.com>
2018-01-13 09:51:08 +08:00
Tone.Zhang
3886918c1f Change the docker daemon name in Debian 9 on Arm
Enable docker-ce on AArch64 platform in Debian.
Set the docker daemon name according to the existing of "dockerd" binary.

Closes-Bug: #1734255

Co-authored-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>

Change-Id: Ie0b3bac72b408dccf3fb99912b4fe2d4788705e5
Signed-off-by: Tone.Zhang <tone.zhang@arm.com>
2018-01-09 15:02:40 +08:00
Zuul
68d0a213b0 Merge "Unify global syntax in baremetal role" 2017-10-24 09:17:38 +00:00
Jenkins
002addc6da Merge "Added open-iscsi to default removals" 2017-09-25 09:27:37 +00:00
Eduardo Gonzalez
fa06dcd37b Fix ci gate failure
This patch includes three unrelated fixes.

Make qemu use nova user in centos
Libvirt 3.2.0 (latest version in centos) seems to
have changed behavior of dynamic_ownership.

Pin ansible to <2.4 to make ara work in gates
ARA does not work yet with ansible 2.4, this change
pins to lower version to make gates work.
Revert once ara works with 2.4

Disable selinux for oraclelinux and centos.

Co-Authored-By: wanghongxu <wang19930902@gmail.com>
Co-Authored-By: Jeffrey Zhang <jeffrey.zhang@99cloud.net>
Change-Id: Iac8bec19437192cd198d58f71c6ed0a65a76f820
Closes-bug: #1718541
2017-09-23 09:03:52 +08:00
Eduardo Gonzalez
aa0209c9a4 Unify global syntax in baremetal role
Baremetal roles does not keep common sytax used
in rest of the roles, even it baremetal role some
tasks differs in syntax with each others.

Change-Id: Ib04fe123501e2a3e829176953f20719a253999d3
2017-09-19 11:23:20 +02:00
James Benson
8e1940d47a ntpd behind proxy fails
Change-Id: I0447e90f53f9b04315bc4e6618e528351521e05d
Closes-Bug: #1705558
2017-07-23 03:09:00 +00:00
Jenkins
7333ae330f Merge "Upgrade from docker-py to docker" 2017-07-19 03:19:33 +00:00
Bertrand Lallau
8b5b3cef4e Clean some Boolean condition checks
* "bool" filter is removed when not required
* 'not' is used instead of '== False' check

Change-Id: I85a5bb9a5ea874ac1c397cbf8de416147d2424c3
2017-07-10 15:00:28 +00:00
Hongbin Lu
3daeea37ea Upgrade from docker-py to docker
The pypi package 'docker-py' [1] has been renamed to 'docker' [2].
It is better to move to the new 'docker' package because the old
package will be deprecated and all the new features will go into
the new package only.

Package 'docker' has been added to requirements [3]. The old
package 'docker-py' is still allowed to be in the global requirements
during the transition period but it should be removed after all or
most of the projects finsih the migration.

[1] https://pypi.python.org/pypi/docker-py
[2] https://pypi.python.org/pypi/docker
[3] https://review.openstack.org/#/c/423715/

Change-Id: Ibcd5a57a1fbf55dcc5a690e41f20917f95b63da0
2017-07-10 14:19:28 +00:00
Michel Rode
82fe7bc07a Added open-iscsi to default removals
The bootstrapping process should remove the open-iscsi package.
Otherwise the iscsid startup will failed.

Change-Id: I0c31c84f4486bd44c467fbaf7a26563e4dbc5ccf
2017-07-10 12:15:13 +00:00
Jenkins
8bcbf493f1 Merge "Adds NTP sync process to account for ceph time sync requirements. Removes precheck portion of NTP. Corrects for redhat Fixes typo" 2017-07-07 03:11:13 +00:00
Jenkins
9f80f2d8d7 Merge "Fix bootstrap-servers with invalid api_interface" 2017-07-05 00:21:09 +00:00
Bertrand Lallau
6c4d100911 Remove Ubuntu Trusty 14.04 related code
Removed code try to install Wily(15.10) kernel in case of Ubuntu
Trusty(14.04).
Last Openstack version supported on Ubuntu 14.04 is Mitaka.
Hence Ubuntu 14.04 related code can be safely remove from Ansible tasks
since Kolla Newton release.

Co-Authored-By: Duong Ha-Quang <duonghq@vn.fujitsu.com>
Change-Id: Ieca7975a69fb0ba8b49cc522f05e4beca1c2f526
2017-06-28 11:09:10 +02:00
James Benson
844ba6d4ba Adds NTP sync process to account for ceph time sync requirements.
Removes precheck portion of NTP.
Corrects for redhat
Fixes typo

Change-Id: Ic8d2cd3c2ba02f9f672db862a74950dc73753f2d
Closes-Bug: #1700121
2017-06-26 16:17:22 -05:00
Michal (inc0) Jastrzebski
2e3b6639fe Fix baremetal role idempotency
When you add new nodes to existing cluster, docker will restart
all anyway and that will break a lot, including mariadb.

Change-Id: Ie46f99a141f99480a87218ead4b76ba65f2edae9
Closes-Bug: #1699335
2017-06-21 11:04:19 -07:00