194 Commits

Author SHA1 Message Date
Mark Goddard
242625dff4 Generate Ceph configuration during upgrade
If upgrading the nova, cinder or manila services via 'kolla-ansible
upgrade', the Ceph config files are not generated. Users will expect
that these files are generated, to pull in any changes from their
configuration or the base kolla configuration.

This change moves Ceph tasks inside config.yml to ensure that they are
performed during deploy, reconfigure and upgrade. This has been done for
nova, cinder, gnocchi and manila - glance already does this.

Change-Id: Ic75692c2bcba9b81dee922ff6fbbccd160e7fa19
Closes-Bug: #1794275
2018-10-10 10:48:55 +01:00
Mark Goddard
b50917fe2c Fix non-root reconfiguration with ceph enabled
Various ceph-related tasks were missing a 'become' that would allow them
to work as a non-root user. This seems to only cause a problem after an
initial deployment, perhaps due to the recursive ownership & permissions
changes at the end of the ceph.yml and external_ceph.yml files.

This change adds the necessary becomes.

Change-Id: I887c7b3bdef49db1dd1bf9e5bdbf5dc47b7f41af
Closes-Bug: #1795125
2018-10-02 09:16:34 +01:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
Mick Thompson
331d2f053f Add swift as gnocchi storage option
- Uses swift if swift is enabled.
 - Uses ceph if ceph is enabled.
 - Defaults to file if swift and ceph are enabled.
   Explicitly set to swift or ceph when both are enabled.
 - Include swift client detail in storage section of gnocchi conf

Change-Id: I78df9a2fbe546038e1d6df350d8db0fd9b6f6d49
2018-09-24 10:42:05 +01:00
ZhongShengping
ae246945a6 Deprecate auth_uri option
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.

[1]https://review.openstack.org/#/c/508522/

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ifd8527d404f1df807ae8196eac2b3849911ddc26
Closes-Bug: #1761907
2018-08-07 11:58:23 +05:30
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Lakshmi Prasanna Goutham Pratapa
76210a2d85 Apply Resource Constraints to Openstack Services
This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa
2018-07-25 17:05:04 +01:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Jeffrey Zhang
3397668d10 Migrate ceph keyring creation to kolla_ceph_keyring module
In this way, keyring caps is updatable.

Change-Id: Idf7f222645b5073e2c72d59eecf3d47b3f1dc6ba
2018-07-02 09:49:48 +08:00
Zuul
f4796819e8 Merge "Fix copy external ceph files from custome/gnocchi in gnocchi" 2018-06-15 06:51:58 +00:00
Jeffrey Zhang
dfb5ddaad6 Fix copy external ceph files from custome/gnocchi in gnocchi
There is no need to load custom ceph.conf and keyring from different
folder for gnocchi components. Just load the files from
node_custom_config/gnocchi folder.

Depends-On: I379ff17856509c9321b86c13a72eacc18f5c1202
Change-Id: Id553dff274a29d82fb1b743437e3656d6f817e52
2018-06-12 11:02:44 +08:00
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
Zuul
06b46533a4 Merge "Fix ansible warning" 2018-05-11 04:13:44 +00:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
wu.chunyang
bd708d9433 Make the workers configurable for gnocchi metricd
By default, the workers for gnocchi_metricd is 5[0], it's too
small for production environment, this PS to make it configurable,
in this way, user can change it according to actual needs.

[0]: https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L356

Change-Id: Ibf4dec793f02c407547c77963cc13799d044b90f
2018-05-09 21:45:18 +08:00
Kevin TIBI
a81a5d5d5d Fix SSL api for multiple services
If SSL is enabled, api of multiple services returns
wrong external URL without https prefix.

Removal of condition for deletion of http  header.

Change-Id: I4264e04d0d6b9a3e11ef7dd7add6c5e166cf9fb4
Closes-Bug: #1749155
Closes-Bug: #1717491
2018-04-18 17:20:27 +02:00
Jeffrey Zhang
84ade4e149 Remove useless module_extra_vars in registry.yml
- remove uesless module_extra_vars, this is a historical issue. In the
  past, we use 'docker exec kolla_toolbox ansible xxx' to run module on
  target node, so complex data have to pass through extra_vars. Now we
  are using kolla_toolbox module, no need to use extra_vars anymore.
- Remove some useless until.

Change-Id: I72ed28001202917f9a82a1c3ea33cd6319911ec8
2018-04-16 09:14:54 +08:00
Zuul
24ee2e348a Merge "Use redis as gnocchi incoming storage and tooz backend when enabled" 2018-03-28 07:06:59 +00:00
Jeffrey Zhang
fca572506e Use redis as gnocchi incoming storage and tooz backend when enabled
- redis is much faster than ceph as incoming storage
- redis tooz backend is full implemented

Change-Id: Ia0b4bc65ea11e504c82fc8a5ad8aa447bee00630
2018-03-27 02:04:31 +00:00
Zuul
37f6e92bb1 Merge "Configure application for ceph pool" 2018-03-26 02:51:11 +00:00
Jeffrey Zhang
6e377622be Configure application for ceph pool
pool application is mandatory since ceph Luminous.

Change-Id: I8c124899b46b1388ab3c2957b4ece0f6aab1378b
Closes-Bug: #1757957
2018-03-22 15:13:31 +08:00
Zuul
a4991c6973 Merge "Security reinforce for apache server" 2018-03-16 08:06:29 +00:00
Jeffrey Zhang
f8cb527f78 Security reinforce for apache server
Disable ServerSignature and Hide apache related infromation.

Change-Id: I9188ddb85988539087c922117bb9f53454b7507c
2018-03-14 18:14:26 +08:00
Zuul
fde4f2112d Merge "Handle external ceph configs as templates" 2018-03-13 07:42:28 +00:00
Duong Ha-Quang
9965cc46ff Specify 'become' for only neccesary tasks (all other roles)
Add become to only neccesary tasks in roles:

- aodh
- barbican
- bifrost
- ceilometer
- ceph
- chrony
- cinder
- cloudkitty
- collectd
- congress
- designate
- elasticsearch
- etcd
- freezer
- gnocchi
- grafana
- influxdb
- ironic
- iscsi
- karbor
- kibana
- kuryr
- magnum
- manila
- mistral
- mongodb
- multipathd
- murano
- octavia
- panko
- qdrouterd
- rally
- sahara
- searchlight
- senlin
- skydive
- solum
- swift
- swift
- tacker
- telegraf
- tempest
- trove
- vmtp
- watcher
- zun

Change-Id: I6e32d94d4172dd96d09d8609e8a5221ab5586a31
Partial-Implements: blueprint ansible-specific-task-become
2018-03-12 09:37:43 +07:00
Christian Berendt
fcf755c8ce Handle external ceph configs as templates
Change-Id: Ice5b058fe17ad7d648f21d1ddacaa339c6f7102f
2018-03-06 18:47:42 +01:00
caoyuan
c1ad9a95ab Add the missing permission setting for external-ceph
when non-root (become), the permission changed is necessary,
but some roles is missing, the patch to add it.

Change-Id: I987becde489f4b509b32edc1deceabfa2d2e1b74
2018-01-30 14:24:58 +08:00
Dai Dang Van
3d9fd47e96 Support policy.yaml file [Final]
- Ceilometer
- Gnocchi
- Rally

This will copy only yaml or json policy file if they exist.

Change-Id: I59f3376ab9fb6fb83577465a6c9096764b9f19c0
Implements: blueprint support-custom-policy-yaml
Co-authored-By: Duong Ha-Quang <duonghq@vn.fujitsu.com>
2018-01-26 04:37:53 +00:00
Alexandru Bogdan Pica
8e3b79440c Implement external MariaDB and pre-configured Databases support
This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121
2018-01-23 13:07:40 +00:00
Zuul
7f4769d61c Merge "Remove the unnecessary var defined" 2018-01-22 07:31:18 +00:00
caoyuan
bcafbb05dd Remove the unnecessary var defined
Change-Id: I673c70ad2c0465babd7a39bf3d5fb5fb8252f1e0
2018-01-18 17:39:49 +08:00
caoyuan
b63ecd7e3e Update kolla_internal_fqdn to database_address since it defined in all.yml
through the database_address has beed defined in groups_vars/all.yml, we should
better use it, this way, if we want to use external database, we just need to
redefined in all.yml

refer to https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L83
Co-Authored-By: chenqiaomin <chen.qiaomin@99cloud.net>

Change-Id: Ie559301451954e16347ceaabf02f594c5c5cbe56
2018-01-17 13:01:48 +08:00
Zuul
a9b8a392ce Merge "Restart services after a change in the external ceph configuration" 2017-12-13 17:58:43 +00:00
Jeffrey Zhang
d832139a09 Use openstack_service_workers variable for gnocchi metricd workers
Change-Id: Ibfa777065291c2e9c2ae00db5288afc8db91e055
Closes-Bug: #1735095
2017-11-29 16:17:19 +08:00
Christian Berendt
6039ad46f9 Restart services after a change in the external ceph configuration
Change-Id: I6b6114e85a4df82458c441f471929513c2526c23
2017-11-02 08:43:43 +01:00
Eduardo Gonzalez
56374206be Fix deployment with public TLS enabled
When deploying with tls enabled in public
endpoints, ansible modules fails due SSL certificates
are self-signed.

This change adds a new variable to allow customization
on which endpoints ansible should connect.
Defaults to admin because admin auth parameters defaults
to admin endpoint.

Change-Id: Ic3ed58cf9c9579cae08a11bbfe6fce983b5a9cbc
Closes-Bug: #1720995
2017-10-05 08:36:34 +00:00
Jenkins
9cdbae707b Merge "Simplify Openstack services configuration possibilities" 2017-09-26 15:49:22 +00:00
caoyuan
8fed5e2e62 Make the pg and pgp value configurable
Change-Id: I99cca8c3830077736f71a7ab1abd3818688f9bb1
2017-09-22 03:21:52 +00:00
Bertrand Lallau
a3dfdfc085 Simplify Openstack services configuration possibilities
Actually Openstack services configuration can be overriden using many
files:
- /etc/kolla/config/<< service name >>/<< config file >>
- /etc/kolla/config/<< service name >>/<<host>>/<< config file >>
- /etc/kolla/config/global.conf
- /etc/kolla/config/database.conf
- /etc/kolla/config/messaging.conf

Only per-service configuration is actually documented here:
https://github.com/openstack/kolla-ansible/blob/master/doc/advanced-configuration.rst#L164

Allowing to globally modify service configuration can be perform too,
but it can be done in 3 different manners, all not documented:
- /etc/kolla/config/global.conf
- /etc/kolla/config/database.conf
- /etc/kolla/config/messaging.conf

database.conf and messaging.conf seems redundant with global.conf.
In order to simplify codebase it seems logical to remove them.

Documentation has been added for overriding configuration globally and
release note has been added too.

Closes-Bug: #1682479
Change-Id: I5d922dfc0d938173bad34ac64e490b78db1b7e31
2017-09-13 17:47:36 +02:00
Jenkins
e5d9295441 Merge "enable CORS for Gnocchi and Keystone for grafana datasource" 2017-08-15 09:15:46 +00:00
zhubingbing
18b1737d58 Add gnocchi log on fluentd
Change-Id: I43fec45a49198daedddbb7c5d477690bbe3864e8
2017-08-03 16:34:54 +08:00
Tatsuma Matsuki
db0f469614 Unified Apache access log format
Apache access log formats are modified to be consistent with
the format defined in wsgi-keystone.conf, which includes
the response time (%D) and X-forwarder-For fields.

Change-Id: I02aa5eb106fb894196dfb6e22daf2968e27ed3cb
Closes-Bug: #1703571
2017-07-26 04:11:54 +00:00
Narasimha SV
4566d1cfc9 enable CORS for Gnocchi and Keystone for grafana datasource
adding cors section to keystone and gnocchi templates when
grafana is enabled

Change-Id: I54f4c06ed64254df0f9481e461c9393a399212a3
Closes-bug: #1651796
2017-07-26 09:52:56 +08:00
Jeffrey Zhang
f5dd178fc5 Disable trace for all containers running httpd
Trace method is enabled in default for httpd. There is security risk
with trace enabled. So disable it in default. more info please check[0].

[0] https://security.stackexchange.com/a/7711

Change-Id: I4496a6d058d88e1abfb210085f189e7a610e0362
Closes-Bug: #1705160
2017-07-19 10:52:41 +08:00
Jeffrey Zhang
65ebba8b1d Fix wsgi file path and log-file parameter
* use gnocchi-api script for wsgi based on this commit[0].
* logfile is deprecated. Use log-file instead[1].

[0] c8717e8b71
[1] https://github.com/gnocchixyz/gnocchi/blob/master/gnocchi/opts.py#L77,L82

Change-Id: I8fbe8425e43d82e0f9601ba6303dd7f24142a2e6
2017-07-17 10:02:28 +08:00
Jenkins
a71aa61ea7 Merge "Use unified vars in task" 2017-07-12 01:56:12 +00:00
Jenkins
0572a394be Merge "Avoid launching "local" actions too many times" 2017-07-10 13:40:49 +00:00
Bertrand Lallau
372e991bec Standardize Keystone domain variables
As described here:
https://github.com/openstack/keystone/blob/master/keystone/resource/core.py#L841
https://github.com/openstack/keystone/blob/master/keystone/conf/identity.py#L21

* default project domain name MUST be named 'Default'
* default project domain id MUST be named 'default'
* default project user name MUST be named 'Default'
* default project user id MUST be named 'default'

Change-Id: I610a0416647fdea31bb04889364da5395d8c8d74
2017-07-06 14:34:11 +00:00
Bertrand Lallau
a9113fc466 Avoid launching "local" actions too many times
In order to speed up deployment time some "local" actions should be run
only once using 'run_once: True'.
This will decrease deployment time in case of multihost configuration.

Change-Id: I6015d772d35c15e96c52f577013b6e41197cb41a
2017-07-05 15:55:39 +02:00
caoyuan
36c530f9ed Fix the database user name for all roles
when create database user, it should use
database_user, not database_name.

Change-Id: I4dfa01d1a5a46c5c58f1fc47b0be71b186462764
Closes-Bug: #1698762
2017-06-20 21:19:31 +08:00