85 Commits

Author SHA1 Message Date
xu.hongzhi
857a90cc31 Move swift precheck into its own role
Co-Authored-By: caoyuan <cao.yuan@99cloud.net>

Change-Id: If3a0a849e551b2b8973a343d1ea603ebb5840e28
Partially-implements: blueprint condition-pre-check
2017-01-16 21:22:17 +08:00
Eduardo Gonzalez
a18174b3b4 Unify jinja syntax
Change-Id: I63197f8c5646e44a9a7287e644c904a1e227af23
2017-01-12 10:52:44 +00:00
Vladislav Belogrudov
25fcdfb986 Use default user group consistently
Default user group should be set much earlier in deployment
and should be used consistently accross all projects.

Change-Id: Id399f9ddebc903bb9c3eeb5a0ff6f33ca6d6828c
Closes-Bug: #1650501
2016-12-19 11:43:23 +03:00
James McCarthy
66e17f6a4b Swift data transfer should not be using the API Network
The account, container, and object templates were incorrectly
using the api_interface as their bind_ip configuration setting.

Updated these templates to instead use the storage_interface.

Change-Id: I683102096cd6aa3c77a7900f5a1a248cdcddba42
Note: Rings must be generated accordingly.
2016-12-06 21:05:55 +00:00
Eduardo Gonzalez
775d8019b6 Add custom policies in service.json
Include custom policy.json files in service-api.json.j2 files

Change-Id: Ic55bfc6f61131aa72c3497ce8b2282056bcc7f92
Partially-Implements: blueprint custom-policies
2016-12-02 16:22:17 +00:00
Duong Ha-Quang
b5d1e4b457 Fix placement of policy.json
Currently, policy.json is put in
"{{ node_config_directory }}/{{ service_name }}"
in target nodes.

Relocation policy.json to "{{ node_config_directory }}/{{ item }}"
with item is corresponding service compoment config directory.

Currently, the policy.json is copied to all services, but it
should be reviewed and left only in neccesary service
(at many cases, only API service needs that).

Redundant files will be removed in follow up patchset.

Change-Id: I0e997dccf4ec438c9c0436db71ec2fd06650f50d
Closes-Bug: #1639686
2016-11-29 16:00:09 +07:00
Eduardo Gonzalez
13e98d8e1a Pull swift-object-expirer image
Add pull task for object-expirer image.

Change-Id: Icc3cb3111e3b78545678e011f7321ba623ec85c2
Closes-Bug: #1643741
2016-11-22 17:41:00 +00:00
Jenkins
04ba40cd63 Merge "Adjust object-replicator heading within object.conf.j2 template." 2016-11-06 20:40:25 +00:00
Jenkins
dbc881456b Merge "Custom policy files" 2016-11-04 13:42:55 +00:00
Jeffrey Zhang
fa45801946 Move precheck into its own role
* Merge prechecks.yml and site.yml playbook
* Create empty precheck.yml into all roles.

Change-Id: I8a138558a26c0a2a66c5fd48ed37be657c99c1dd
Implements: blueprint condition-pre-check
2016-11-03 14:48:58 +08:00
James McCarthy
790c6b2a9f Adjust object-replicator heading within object.conf.j2 template.
The swift-object-auditor logs many complaints of missing
object-replicator configuration, as this section is undefined.

Move the header outside of the == 'swift-object-replicator' check,
so other services will get an empty section to stop complaints.

TrivialFix
Closes-Bug: #1637561
Change-Id: I9c8d960b0c31e448afdeaac5f3d38e1be5ff965d
2016-10-28 16:44:14 +01:00
Eduardo Gonzalez
ef1fd6b8c0 Custom policy files
Allow operators to use their custom policy files.
Avoid maintain policy files in kolla repos, only copying
the files when an operator add their custom config.

Implements: blueprint custom-policies
Change-Id: Icf3c961b87cbc7a1f1dd2ffbfffcf271d151d862
2016-10-28 10:47:05 +01:00
Jeffrey Zhang
3e89f987c0 Add validate no new line at end of file
TrivialFix

Change-Id: I817aa52caf56b7d54b266e553db6db1ceb38b773
Signed-off-by: Jeffrey Zhang <zhang.lei.fly@gmail.com>
2016-10-19 17:17:06 +00:00
Jenkins
57203424c2 Merge "ansible: swift: Fix swift-object-expirer restart loop" 2016-10-14 18:08:12 +00:00
Otavio Salvador
9eb38405b7 ansible: swift: Fix swift-object-expirer restart loop
The swift-object-expirer is provided by the 'openstack-swift-proxy'
package and thus it is unavailable on swift-object image. This change
adds a new Docker image to fulfill this requirement and stop using
swift-object image in this case.

This image is needed while RDO does not fix the packaging. The issue
is being tracked in:

  https://bugzilla.redhat.com/show_bug.cgi?id=1382921

Change-Id: Idc7ee92d756d8923da2198ede33abf5ed1142041
Closes-Bug: 1630425
2016-10-11 14:56:18 -03:00
Jeffrey Zhang
fa856556d5 Remove do_reconfigure.yml file for all roles
do_reconfigure.yml is introduced to use serial directive. But we use
it in wrong. Now serial has moved to playbook file. So it is time to
remove the do_reconfigure.yml file

Closes-Bug: #1628152
Change-Id: I8d42d27e6bc302a0e575b0353956eaef9b2ca9fd
2016-09-27 15:04:00 +00:00
Bartłomiej Daca
27621351ac Repair swift reconfiguration
Change the conditions order in when statement.

TrivialFix

Change-Id: Ied9907ff25b66de3fd43502a920a88af8ff0cacd
Closes-Bug: #1625256
2016-09-19 18:46:13 +02:00
Mick Thompson
4c1482b999 Added serial option to site.yml.
Useful for upgrade etc., which is preferablly done serially.

Example usage: tools/kolla-ansible deploy OR tools/kolla-ansible upgrade

Closes-Bug: #1576708

DocImpact

Change-Id: I34b2e16f8ce53e472a4682a4738c4ac0f5abf00c
2016-09-13 09:26:51 +00:00
Duong Ha-Quang
5e35fc54a6 Port with_ loop variable syntax to Ansbile 2 (finish)
Migrate to full variable syntax in with_ loop
instead of bare variables for:

- cinder
- haproxy
- ironic
- magnum
- mistral
- mongodb
- murano
- swift
- watcher

TrivialFix
Change-Id: I3ef2e79053cf609aaa710e43ffd0adbc5a97565b
2016-08-18 17:45:52 +07:00
Stephen Hindle
cadf8afdcf Read GPT label using sgdisk rather than udev
The Ceph osd bootstrap/startup logic depends upon reading/writing
partition names.  Some older versions of udev have trouble reading these
(/dev/disk/by-label is missing).

To work around this, we shell out to sgdisk and scrape the partition
name data directly.

Co-authored-by: Paul Bourke <paul.bourke@oracle.com>
Closes-Bug: 1585185
Change-Id: I362b3f8e91de79687fc84e256996fbcaf303b6af
2016-07-07 10:26:34 +01:00
Ryan Hallisey
2da010a7b9 Make configurable the location where config files are merged
An operator may want to specify the location of custom config
files so that kolla can detect their location and merge
them with the default configs generated.

Partially implements: blueprint multi-project-config

Change-Id: Ibfb38d07a36dfa7fe25381adc34cc1d3cbe7d1e1
2016-05-23 03:23:06 -04:00
Jeffrey Zhang
0fcee87549 map the host localtime to the container
Closes-Bug: #1577148
Change-Id: I636cefc63cf532434a41af3898b63dffa711e280
2016-05-03 09:27:51 +08:00
Serguei Bezverkhi
d553514cb7 Configures swift-rsyncd to use non-default port
This PS configures swift-rsyncd process to use non-default port
from the range above 1024.

Change-Id: I7c37c548a5185a2ffac789383fe012619e401131
Closes-Bug: #1573137
2016-04-21 12:54:30 -04:00
SamYaple
d4535b6dc3 Add memcached_servers to keystone_auth section
The in-process cache for keystone tokens has been deprecated due to
"incosistent results and high memory usage" with the expectation we
switch to memcached_servers if we want to stay performant.

Add memcache_servers [cache] section to the appropriate servers as the
[DEFAULT]\memcache_servers options was deprecated.

TrivialFix
Related-Id: Ied2b88c8cefe5655a88d0c2f334de04e588fa75a

Change-Id: Ic971bdddc0be3338b15924f7cc0f97d4a3ad2440
2016-03-19 21:53:03 +00:00
Serguei Bezverkhi
05b6995f63 Fixes incorrect swift service endpoints
This bugfix fixes incorrect swift proxy-server endpoint.

Closes-Bug: #1559207

Change-Id: Idcdbaedbbde4922b0d5e31557aaf5cfcfdff8636
2016-03-18 12:42:23 -04:00
SamYaple
1f371e2717 Allow per node configuration with augments
This type of per node configuration is required to support things like
availability zones for nova. As always, if this file doesnt exist it
doesnt get used so this change is safe.

TrivialFix

Change-Id: Iff8172af522c2c96e5f2c173b24a5dfd4d522ed2
2016-03-17 17:18:00 +00:00
Serguei Bezverkhi
514da17605 Reconfigure for Swift
This implements reconfigure for Swift service. Fixes swift-rsyncd in
All-in-One deployment

Partially-Implements: blueprint kolla-reconfig

Change-Id: I82bfa3c73b7365bc3a0ea35d3fc102a3525ebd8c
2016-03-05 10:24:13 -05:00
Jenkins
91f3bb0395 Merge "Reconfigure for Swift" 2016-03-04 15:44:41 +00:00
Serguei Bezverkhi
bb3079898b Reconfigure for Swift
This implements reconfigure for Swift service.

Change-Id: Ifa4f02fc39ca79a886d6fd85f87073c897a777eb
Partially-Implements: blueprint kolla-reconfig
2016-03-04 08:01:31 -05:00
SamYaple
57124620ab Fix Keystone v3 and Horizon
After our switch to keystone-manage bootstrap Horizon is not happy
due to v3 not being setup correctly. This patch fixes that

This also includes removal of unused variables (transforms them into
endpoint url variables)

TrivialFix
Change-Id: I1e04db8c24049f80e974c063f03068a2ab32a563
2016-03-03 15:44:53 +00:00
Eugene Bagdasaryan
790675be4f Sanity check for swift
This runs first sanity check for swift. Once
swift is deployed it checks list()

Change-Id: I613bf9f2893d66814863893ec5acde5aa252548d
Partially-Implements: blueprint sanity-check-container
2016-02-29 18:23:13 +03:00
SamYaple
d3cfb2052a Change kolla_internal_address variable
Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.

This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.

kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn

The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.

Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla
2016-02-26 20:00:09 +00:00
Jeffrey Zhang
d26c259754 Add the default empty reconfigure.yml file to all roles
DocImpact

Partially-implements: bp kolla-reconfig
Change-Id: I9738e80960bcfbef18d1ef1b7942f81c45684e85
2016-02-25 09:32:53 +08:00
Jenkins
2b0a070ee5 Merge "Fix Swift logging" 2016-02-22 21:49:42 +00:00
Éric Lemoine
e6a9b9627a Fix Swift logging
Swift uses Syslog, but it uses a custom log format.  So this commit
adds a specific Heka decoder for Swift.

It also increases the log level from "warning" to "info" to make
Swift more verbose.  Note that "info" is the default log level in
Swift.

And it disables the Heka configuration for Swift when "enable_swift"
is set to "no".  This prevents Heka from creating 15 empty Swift log
files in the logs volume.

Partially implements: blueprint heka

Change-Id: If7a7d0707e71be2957178e2d45b5de51b788232e
2016-02-22 08:53:20 -08:00
Paul Bourke
e09375a4ed Upgrade task for Swift
Based on the Nova upgrade patch and recommendations from Swift PTL John
Dickinson at
https://swiftstack.com/blog/2013/12/20/upgrade-openstack-swift-no-downtime/

Notes:

As part of this upgrade I have chosen to *not* migrate any data from the
old style swift_data container. This is because it was never intended to
be used in production; this fact is made clear in the docs.

In regards to testing, as of this patch we do not yet have an upgrade
task for the common containers (rsyslog and kolla-toolbox), so
attempting to upgrade swift will result in it failing to find the
kolla-toolbox. This will be true of any other upgrade until upgrade for
common is added. It can be worked around by deploying another role such
as keystone which will drag in the common role and start up
kolla-toolbox, after which Swift can be successfully upgraded.

Change-Id: I138556932e9bddcd595d94a3dcb69603268880ff
Partially-Implements: blueprint upgrade-kolla
Implements: blueprint upgrade-swift
2016-02-21 10:12:58 +00:00
Éric Lemoine
88633684a1 Make Heka collect Swift logs
Partially implements: blueprint heka
Change-Id: I893a0c4a4cab9d4d98821634ddd2ff67015c4e3f
2016-02-19 21:56:05 +00:00
Jenkins
6e622fa534 Merge "Move socket binding to named_volume" 2016-02-17 02:06:26 +00:00
Jenkins
c6da765b49 Merge "Use variables to specify http or https when constructing URLs" 2016-02-16 22:37:27 +00:00
SamYaple
690e6853de Move socket binding to named_volume
The extend_start.sh script for rsyslog is removed as it is no longer
needed. Docker no longer binds to /dev/log or /run/kolla/log

Closes-Bug: #1544545
Change-Id: Ic0a323a26ee4e9e15baf4598285844a8a4955f23
2016-02-16 14:42:41 +00:00
Jenkins
84f43146dc Merge "Use uppercase 'S' in word "OpenStack"" 2016-02-15 16:23:37 +00:00
Dave McCowan
1cedf77f19 Use variables to specify http or https when constructing URLs
To allow for TLS to protect the service endpoints, the protocol
in the URLs for the endpoints will be either http or https.

This patch removes the hardcoded values of http and replaces them
with variables that can be adjusted accordingly in future patches.

Change-Id: Ibca6f8aac09c65115d1ac9957410e7f81ac7671e
Partially-implements: blueprint ssl-kolla
2016-02-15 09:48:58 -05:00
venkatamahesh
70de590a11 Use uppercase 'S' in word "OpenStack"
Change-Id: I27ca0ea75f3d6a4371c91b3cb2c7a999ec90fbc4
2016-02-15 12:37:30 +00:00
SamYaple
e8d66766f0 Fix gate for docker 1.10
Docker 1.10 has broken the gate and this patch will correct that
breakage.

The issue comes with rsyslog. Due to a commit in Docker 1.10 [1] we
must change the way we get the log socket for rsyslog. The /dev/
folder will no longer populate as we used it. So instead we simply
make a new socket in a path we control and share that to the correct
location in the containers.

Additionally, adjust the gate for new Docker daemon.

[1] https://github.com/docker/docker/pull/16639
Partially-Implements: blueprint kolla-upgrade
Change-Id: I881a2ecdf6d7b35991e1d38a3f3e60d022d6577f
2016-02-06 06:48:53 +00:00
SamYaple
508f3863cc Fix two missing toolbox references
Two patches merged that did not have teh correct name for
kolla_toolbox

TrivialFix

Change-Id: I09e230e29cff496247c32bf227715bfb7caa0a95
2016-02-05 17:24:39 +00:00
Jenkins
b891afd64c Merge "Rename kolla_ansible to kolla_toolbox" 2016-02-05 15:04:37 +00:00
SamYaple
80b7266ed1 Rename kolla_ansible to kolla_toolbox
This change is needed for clarity. We have a kolla-ansible script.
We have a kolla-mesos repo. We plan to have a kolla-ansible repo.
Already we have had far too much confusion about whether we are
talking about the container or the project. Naming this kolla-toolbox
eliminates all of that confusion and its probably a bit more accurate
of a name too.

Closes-Bug: #1541053
Change-Id: I8fd1f49d5a22b36ede5b10f46b9fe02ddda9007e
2016-02-02 18:12:15 +00:00
Qiu Yu
b209da1306 Fix Swift for multinode P+ACO deployment
Current Swift playbook is based on the preassumption of AIO setup.
However, if one goes with default multinode setup
(ansible/inventory/multinode), it follows the P + ACO deployment model,
which proxy-server runs on controller nodes where ACO
(account/container/object services) run on storage nodes.

It will break because swift proxy-server no longer has access (it
shouldn't have) to /srv/node path. This change ensure disk mounting part
only happens on storage node. It also moves chown from proxy-server
Dockerfile to rsyncd because no matter with PACO, P+ACO or P+A+C+O
model, rsyncd is always running on each storage node.

Change-Id: I3aa20454902caa9c84d3901bb91e4e4c93ac5f34
Partially-Implements: blueprint swift-physical-disk
Closes-Bug: #1537544
2016-02-02 02:14:31 +08:00
Jenkins
c755a62ee0 Merge "Fix missing swift proxy server image in pull action" 2016-02-01 09:41:32 +00:00
Qiu Yu
d1d8308b51 Fix missing swift proxy server image in pull action
Add back missing swift proxy server image which was removed in Swift
shared image change.

TrivialFix

Change-Id: Icf13d4a1550192f73e266a6c6aa74f604ee4e77a
2016-01-30 06:00:59 +08:00