152 Commits

Author SHA1 Message Date
Mark Goddard
b123bf6621 Use become for all docker tasks
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
2019-06-06 19:04:58 +01:00
Mark Goddard
a4bb8567da Fix up config file permissions on the host
Several config file permissions are incorrect on the host. In general,
files should be 0660, and directories and executables 0770.

Change-Id: Id276ac1864f280554e98b937f2845bb424d521de
Closes-Bug: #1821579
2019-04-02 17:23:31 +01:00
Zuul
ed5588c934 Merge "Don't pull images during upgrade" 2019-03-28 12:41:22 +00:00
Mark Goddard
5629b5aad9 Fix swift reconfigure
Reconfiguring Swift currently fails to restart containers if
configuration changes. This is because kolla_set_configs is executed in
the containers as the default swift user, which does not have permission
to access all necessary files.

This change uses the root user to execute the command instead, which
allows it to exit with the correct status of 1 if the config files
differ.

Change-Id: I2a2363c71430a7173bb5253662412ae5dba09654
2019-03-20 18:59:28 +00:00
Mark Goddard
58d6dc3bcf Don't pull images during upgrade
When adding the rolling upgrade support, some upgrade procedures were
modified to pull images explicitly. This is done inconsistently between
services, and is a change in behaviour from Rocky and earlier releases.

This change removes all image pulling from upgrade tasks.

Change-Id: Id0fed17714235e1daed60b83b1f30620f097eb97
2019-03-20 18:51:45 +00:00
Scott Solkhon
a781c64319 Support separate Swift storage networks
Adds support to seperate Swift access and replication traffic from other storage traffic.

In a deployment where both Ceph and Swift have been deployed,
this changes adds functionalality to support optional seperation
of storage network traffic. This adds two new network interfaces
'swift_storage_interface' and 'swift_replication_interface' which maintain
backwards compatibility.

The Swift access network interface is configured via 'swift_storage_interface',
which defaults to 'storage_interface'. The Swift replication network
interface is configured via 'swift_replication_interface', which
defaults to 'swift_storage_interface'.

If a separate replication network is used, Kolla Ansible now deploys separate
replication servers for the accounts, containers and objects, that listen on
this network. In this case, these services handle only replication traffic, and
the original account-, container- and object- servers only handle storage
user requests.

Change-Id: Ib39e081574e030126f2d08f51de89641ddb0d42e
2019-03-14 14:00:18 +00:00
Jim Rollenhagen
31ed556e67 Allow swift proxy server to use independent hostnames
This allows swift service endpoints to use custom hostnames, and adds the
following variables:

* swift_internal_fqdn
* swift_external_fqdn

These default to the old values of kolla_internal_fqdn or
kolla_external_fqdn.

This also adds a swift_proxy_server_listen_port option, which defaults to
swift_proxy_server_port for backward compatibility.

This option allow the user to differentiate between the port the
service listens on, and the port the service is reachable on. This is
useful for external load balancers which live on the same host as the
service itself.

While we're in here, use the ``internal_protocol`` variable for the swift
endpoint in cinder's swift backup driver configuration, instead of hardcoding
to ``http``.

Change-Id: Ibc01618383c26e16c0067f7f6b9cf5160d968d1e
Implements: blueprint service-hostnames
2019-03-06 15:08:28 -05:00
Jim Rollenhagen
2e4e60503a Use keystone_*_url var in all configs
We're duplicating code to build the keystone URLs in nearly every
config, where we've already done it in group_vars. Replace the
redundancy with a variable that does the same thing.

Change-Id: I207d77870e2535c1cdcbc5eaf704f0448ac85a7a
2019-03-06 15:08:26 -05:00
Zuul
f5429e42ff Merge "Apply Swift rolling upgrade" 2018-11-30 11:29:27 +00:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
Ha Manh Dong
4e5ae36c5b Apply Swift rolling upgrade
Apply Swift rolling upgrade based on recommendations from Swift PTL John
Dickinson at [1]

[1] https://www.swiftstack.com/blog/2013/12/20/upgrade-openstack-swift-no-downtime/

Co-Authored-By: Surya Prakash <singh.surya64mnnit@gmail.com>
Change-Id: I99f505438916be2f89b24df20506339604e5bd6e
Implements: blueprint apply-service-upgrade-procedure
2018-10-31 09:13:18 +00:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
Ha Manh Dong
5bc3e19288 Remove when condition in roles/swift/tasks/deploy.yml
Because all task in deploy.yml need run in all swift host,
so we don't need the when condition.

Change-Id: Ibd61ce2615985cbad70c13a37c863fc6e5e858e4
2018-07-30 08:51:18 +07:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Zuul
003a6a12b1 Merge "remove uid, gid from rsyncd.conf for swift user" 2018-07-20 08:43:56 +00:00
Zuul
57c9155004 Merge "Added delay_auth_decision config option for swift" 2018-07-18 17:07:01 +00:00
Minho Ban
ecc1e4a6eb remove uid, gid from rsyncd.conf for swift user
The rsync prior to v3.1.0 the uid/gid parameter have no effect at
all if it runs as normal(non-root) user.

Since v3.1.0 these parameter are problematic for normal user
because now rsync, regardless of root or non-root, if the
parameters are given then it just tries to call setgroups() which
is not possible for normal user so errors may occur.

    swift-object-replicator: @ERROR: setgroups failed\u0000
    swift-object-replicator: rsync error: error starting
    client-server protocol (code 5) at main.c(1648)
    [sender=3.1.2]\u0000

Either way, these parameters are not needed for swift-rsync
container.

Change-Id: Ia7fe9f06d7a21a55f52b90c2cc1b2498300e6532
Signed-off-by: Minho Ban <mhban@samsung.com>
2018-07-18 09:17:12 +09:00
fan_guiju
420b523f31 Added delay_auth_decision config option for swift
The authtoken config variable delay_auth_decision must be set to True.
The default is False, but that breaks public access, StaticWeb, FormPost,
TempURL, and authenticated capabilities requests (using Discoverability).

Change-Id: I420a95f5f9fda3321a4acfc5846e40294a8bd588
Closes-Bug: #1768795
2018-07-04 23:11:40 -04:00
tone.zhang
cebb77d7e2 Make Swift log_level changeable
The log_level in Swift is fixed to INFO. The patch make it changeable
according to the value of "openstack_logging_debug".

When "openstack_logging_debug" is "False", the log_level is set to
"INFO". It is default value. Otherwise, the log_level is set to
"DEBUG".

Closes-Bug: #1777982

Change-Id: I62f430abd8f332cc2ece56a6733776fa03b10f77
Signed-off-by: tone.zhang <tone.zhang@arm.com>
2018-06-21 14:02:04 +08:00
Ha Manh Dong
30be04ea91 Specify 'become' for all tasks that use kolla_docker module
Add become to all tasks that use the module "kolla_docker"

Change-Id: I4309c4011687b88ec31d739fd8f834fe2326ff10
Partial-Implements: blueprint ansible-specific-task-become
2018-06-08 12:39:24 +00:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
James McCarthy
cac5c8954b Update swift_devices_mount_point propagation settings
The mount is "rprivate" by default, change this to "shared"
such that mount points will dynamically propagate.

Closes-Bug: #1767828

Change-Id: If3b99bd1626a44aa7b41cf7d50d67b4fc9634462
2018-04-30 09:23:50 +01:00
caoyuan
d6a6486886 Add owner and group when create directories
Change-Id: Idb429823d48c4f40cf391151e8bb7478cc7d3631
2018-04-20 23:16:08 +08:00
Jeffrey Zhang
84ade4e149 Remove useless module_extra_vars in registry.yml
- remove uesless module_extra_vars, this is a historical issue. In the
  past, we use 'docker exec kolla_toolbox ansible xxx' to run module on
  target node, so complex data have to pass through extra_vars. Now we
  are using kolla_toolbox module, no need to use extra_vars anymore.
- Remove some useless until.

Change-Id: I72ed28001202917f9a82a1c3ea33cd6319911ec8
2018-04-16 09:14:54 +08:00
Vladislav Belogrudov
d5fd744409 Swift configuration misses quotas to pass DefCore
Swift configuration needs container and account quotas to
pass 'OpenStack Powered' certification test suite

Co-Authored-By: Alexander Reunov <alexander.reunov@oracle.com>
Change-Id: I7a31315754cbb46257d3d98379daa39d32205480
Closes-Bug: #1757451
2018-03-21 14:47:00 +00:00
Duong Ha-Quang
9965cc46ff Specify 'become' for only neccesary tasks (all other roles)
Add become to only neccesary tasks in roles:

- aodh
- barbican
- bifrost
- ceilometer
- ceph
- chrony
- cinder
- cloudkitty
- collectd
- congress
- designate
- elasticsearch
- etcd
- freezer
- gnocchi
- grafana
- influxdb
- ironic
- iscsi
- karbor
- kibana
- kuryr
- magnum
- manila
- mistral
- mongodb
- multipathd
- murano
- octavia
- panko
- qdrouterd
- rally
- sahara
- searchlight
- senlin
- skydive
- solum
- swift
- swift
- tacker
- telegraf
- tempest
- trove
- vmtp
- watcher
- zun

Change-Id: I6e32d94d4172dd96d09d8609e8a5221ab5586a31
Partial-Implements: blueprint ansible-specific-task-become
2018-03-12 09:37:43 +07:00
Zuul
4df5fe5033 Merge "Add rsync env for account & container replicators" 2018-01-08 07:37:49 +00:00
Paul Bourke
859e88ce4c Add tempurl to swift pipeline
This is required for some tempest tests and in turn to achieve 100%
refstack certification for clouds deployed by Kolla. tempurl is default in
Swift[0] but we're missing as we override the pipeline.

[0] https://github.com/openstack/swift/blob/\
    b86bf15a644db4438770801a312fe074a09c91ef/\
    etc/proxy-server.conf-sample#L97

Change-Id: I0e36fcd7a785f878005d51159eb51725c284229c
2017-12-13 12:21:40 +00:00
Paul Bourke
414d1dbb4b Add rsync env for account & container replicators
In the previous patch to address broken replication in Swift the account
and container replicators were missed.

Change-Id: I34a58b137dbba99c66e7680546fe85e2ef6131e9
Co-Authored-By: Thorsten Frueauf <thorsten.frueauf@oracle.com>
Partial-Bug: #1733851
2017-12-12 13:45:25 +00:00
Zuul
822a3994fd Merge "Update swift templates for rsync_module" 2017-12-07 10:41:28 +00:00
Zuul
7576dcc44c Merge "Update swift for replication issues" 2017-11-29 11:13:41 +00:00
James McCarthy
3245201af1 Update swift for replication issues
There are corresponding image changes to go with these ansible
changes - changes in rsyncd template, and add
environment settings for RSYNC_CONNECT_PROG (i.e. nc)

nc allows for rsync replication to easily target {{ swift_rsync_port }}

update the lock file and chroot settings in template - see bug
for more details.

Change-Id: Ic81b7de8fad8aec9416e4e27e8ffda6d03be293c
Closes-Bug: #1733851
2017-11-24 14:49:38 +00:00
Andrew Smith
fd1d3af0df Add support for hybrid messaging backends
This commit separates the messaging rpc and notify transports in order
to support separate and different oslo.messaging backends

This patch:
* add rpc and notify variables
* update service role conf templates
* add example to globals.yaml
* add release note

Implements: blueprint hybrid-messaging
Change-Id: I34691c2895c8563f1f322f0850ecff98d11b5185
2017-11-22 14:09:40 -05:00
James McCarthy
4cac847f6c Update swift templates for rsync_module
With the current templates, the rsync_module is left as the default value,
which has not {meta} included. Setting sync_module has no effect.

References:
https://github.com/openstack/swift/blob/master/etc/container-server.conf-sample
https://github.com/openstack/swift/blob/master/swift/obj/replicator.py#89

Change these templates to set rsync_module instead.

Change-Id: I14773a09013b90dd2dd68110173eab1d3ccec3de
2017-11-20 18:17:03 +00:00
Zuul
bf0e8a8d52 Merge "Fix ansible running issue on Debian" 2017-11-15 11:44:35 +00:00
Vladislav Belogrudov
33513db407 Allow object versioning by default
Object versioning is necessary to pass DefCore test suite,
i.e. allows Kolla version of OpenStack to be certified
as "OpenStack Powered Platform".

Change-Id: Id5003f7fe2aebdeffe1cf7ce1b6177a6bca8f5b6
Co-Authored-By: Alexander Reunov <alexander.reunov@oracle.com>
Closes-Bug: #1729583
2017-11-02 14:48:14 +03:00
Xinliang Liu
df60b76552 Fix ansible running issue on Debian
When deploying on debian, it reports error:
stat /usr/bin/ansible: no such file or directory

That's because on Debian and Ubuntu pip install ansible to
/usr/local/bin/ansible, whereas on CentOS the location is
/usr/bin/ansible.

Change to ansible to handle both cases.

Closes-Bug: #1729216
Depends-On: I2b57403128bc103148ae696c219df52590214adc

Change-Id: I025037cf48596450e6479ab7ff6425c48ac73aad
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
2017-11-02 09:46:38 +01:00
Eduardo Gonzalez
56374206be Fix deployment with public TLS enabled
When deploying with tls enabled in public
endpoints, ansible modules fails due SSL certificates
are self-signed.

This change adds a new variable to allow customization
on which endpoints ansible should connect.
Defaults to admin because admin auth parameters defaults
to admin endpoint.

Change-Id: Ic3ed58cf9c9579cae08a11bbfe6fce983b5a9cbc
Closes-Bug: #1720995
2017-10-05 08:36:34 +00:00
Tatsuma Matsuki
03354bc99e Add fluentd enable option
This change adds enable_fluentd option and enables some other log shippers
to be integrated. When enable_fluentd is "no", syslog server is also disabled.
Then, this change also adds syslog parameters to use a syslog server
prepared by users.

Change-Id: I7c83ef7fe30a6b9ab7385bcee953ad07e96b0a83
Implements: blueprint fluentd-enable-option
2017-09-28 04:36:36 +00:00
Bertrand Lallau
191e268575 Fix warning on {failed|changed}_when statements
Ansible 2.3.0 does not like delimiters in when statement.

Related link: https://github.com/ansible/ansible/issues/22397

Closes-Bug: #1714349
Change-Id: I973cc6537c4c1374546b5cddb4ce713a553b92f4
2017-08-31 23:16:26 +02:00
guochao
b63d71607a Add swift ring files check in prechecks
Precheck will fail when swift ring files do not exists.

Change-Id: I20e7c9dbe527015ac2b01e85907637feb0deacbc
2017-08-22 10:55:35 +00:00
Jenkins
a71aa61ea7 Merge "Use unified vars in task" 2017-07-12 01:56:12 +00:00
Jenkins
0572a394be Merge "Avoid launching "local" actions too many times" 2017-07-10 13:40:49 +00:00
Bertrand Lallau
372e991bec Standardize Keystone domain variables
As described here:
https://github.com/openstack/keystone/blob/master/keystone/resource/core.py#L841
https://github.com/openstack/keystone/blob/master/keystone/conf/identity.py#L21

* default project domain name MUST be named 'Default'
* default project domain id MUST be named 'default'
* default project user name MUST be named 'Default'
* default project user id MUST be named 'default'

Change-Id: I610a0416647fdea31bb04889364da5395d8c8d74
2017-07-06 14:34:11 +00:00
Bertrand Lallau
a9113fc466 Avoid launching "local" actions too many times
In order to speed up deployment time some "local" actions should be run
only once using 'run_once: True'.
This will decrease deployment time in case of multihost configuration.

Change-Id: I6015d772d35c15e96c52f577013b6e41197cb41a
2017-07-05 15:55:39 +02:00
Vladislav Belogrudov
0ffa770b83 Always create ResellerAdmin role for Swift
ResellerAdmin role should be created always when Swift is enabled
and not only for Ceilometer. The role is needed for normal users
to get administration rights for their Swift projects and is
required to pass DefCore (OpenStack Powered) certification.

Change-Id: I4faa63b8fae1814e382de2794301248cc0f4a90a
Closes-Bug: #1700729
2017-06-27 12:34:15 +03:00
Jeffrey Zhang
03e1b05f99 Use unified vars in task
Ansible task support vars directive, no need implement another one in
merge_config. This patch remove the vars directive in merge_config
action plugin.

Change-Id: I33648a2b6e39b4d49ce76eb66fbf2522721f8c68
2017-06-15 22:37:28 +08:00
Jenkins
2830541dd9 Merge "Mixing binary and source images for S* projects" 2017-06-06 08:47:59 +00:00
Jenkins
3a68aee3ad Merge "Use auth vars for admin user" 2017-06-04 15:05:29 +00:00
Dai Dang Van
74242ee904 Mixing binary and source images for S* projects
In case Kolla's users want to deploy with both of
binary and source image, we should have a variable
install type that define install type for each project.

We also add specific image tag for each Openstack project.

This commit is implemented for Sahara, Searchlight,
Senlin, Solum and Swift projects.

Change-Id: I964796b2f9e3eae69d7eccf68e9428ce9390010c
Implements: blueprint mixing-binary-and-source-image
2017-06-03 16:17:18 +07:00