9543 Commits

Author SHA1 Message Date
Doug Szumski
516b0dd34e Support deploying an Apache Storm cluster
Storm is required for running the Monasca thresholder component for
generating alerts.

Change-Id: I5e1ef74dc55a787293abbb3e629b5ab1ce5f4bbb
Partially-Implements: blueprint monasca-roles
2018-09-26 10:54:32 +00:00
Mark Goddard
8e635db8f1 Allow fluentd to read barbican-api.log
Barbican API uses uWSGI, which by default writes out log files using
0640 permissions and default ownership for the user. This means that the
log file in /var/log/kolla/barbican/barbican-api.log is not readable by
fluentd.

This was tested via the following command on a queens deployment:

$ docker exec -it fluentd bash
find /var/log/kolla/ -type f | while read f; do test -r $f || echo
"Cannot read $f"; done
Cannot read /var/log/kolla/barbican/barbican-api.log

Generally there are a few ways in which access is provided to log file
for fluentd:

1. Set log file ownership to $USER:kolla, permissions to 0640.
2. Set log file ownership to $USER:$USER, permissions to 0644.
3. MariaDB is a special case, and uses 0640 with the fluentd user added
to the mysql group.

Of these, 1. seems the most secure.

This change uses the --logfile-chmod argument to set the log file
permissions to 644, since it does not appear possible to specify a group
to change ownership to using --logfile-chown. We use command line
arguments since putting the option in the config file does not seem to
work. Perhaps it is an ordering issue.

Change-Id: If98ca7cd9630b5622132a00718cb09304b8285b3
Closes-Bug: #1794472
2018-09-26 10:43:08 +00:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
Zuul
921a6d8762 Merge "Support deploying Monasca Log Metrics" 2018-09-26 01:07:34 +00:00
Zuul
fdc6152508 Merge "Support configuring Monasca log pipeline performance" 2018-09-26 01:07:33 +00:00
Zuul
a883cbc69b Merge "Improve scalability of Kafka" 2018-09-26 01:07:29 +00:00
Zuul
6a9f050025 Merge "Remove Kafka partitions config option" 2018-09-26 01:07:27 +00:00
Zuul
858d27cb35 Merge "Provide support for freezer dev mode in kolla" 2018-09-25 17:31:08 +00:00
Doug Szumski
1ae10375f7 Support deploying Monasca Log Metrics
The log metrics service generates metrics from log messages
which allows further analysis and alerting to be performed
on them. Basic configuration is provided so that metrics
are generated for high level warning logs such as error, or
warning.

Change-Id: I45cc17817c716296451f620f304c0b1108162a56
Partially-Implements: blueprint monasca-roles
2018-09-25 16:36:14 +00:00
Zuul
da0904c28d Merge "Add swift as gnocchi storage option" 2018-09-25 14:36:14 +00:00
Doug Szumski
01da938412 Support configuring Monasca log pipeline performance
Change-Id: Id8948fcf2d165f8285c7562e7aebd4145c4ff0db
Partially-Implements: blueprint monasca-roles
2018-09-25 11:41:29 +00:00
Doug Szumski
2b34f6d2fd Improve scalability of Kafka
Set sensible defaults for replica counts and minimum insync replicas
as a function of the number of nodes in the Kafka cluster.

Partially-Implements: blueprint monasca-roles
Change-Id: Icf1dddb7dd6a64f4e5efb7dffa5ffdf0880f891f
2018-09-25 11:41:00 +00:00
Doug Szumski
0a90020289 Remove Kafka partitions config option
This option doesn't actually do anything and a bug to remove
it from the Monasca API config file has been raised.

Partially-Implements: blueprint monasca-roles
Change-Id: I7ec1786b5828ab0135ca86ec040f83a6f4c78d9f
2018-09-25 11:40:54 +00:00
Ha Manh Dong
79da68fab6 Fix missing slash at mount volumes for prometheus-cadvisor
Change-Id: I0444b23aee900d028c879ec64d153d59a18ff504
2018-09-25 10:41:42 +07:00
Zuul
4431b1d0fa Merge "Update rabbitmq to use new conf & clustering" 2018-09-24 15:07:28 +00:00
Zuul
3c1616ffbc Merge "Switch to stestr" 2018-09-24 11:07:48 +00:00
Mick Thompson
331d2f053f Add swift as gnocchi storage option
- Uses swift if swift is enabled.
 - Uses ceph if ceph is enabled.
 - Defaults to file if swift and ceph are enabled.
   Explicitly set to swift or ceph when both are enabled.
 - Include swift client detail in storage section of gnocchi conf

Change-Id: I78df9a2fbe546038e1d6df350d8db0fd9b6f6d49
2018-09-24 10:42:05 +01:00
Mark Goddard
652b78fc51 Fix POST_FAILURE in CI when ARA DB doesn't exist
If the main CI job fails before generating an ARA report, the SQLite
database file will not exist. This cases the job to fail with
POST_FAILURE, rather than FAILURE, and the following is seen in the
logs:

rsync: change_dir "/home/zuul/.ara" failed: No such file or directory
rsync error: some files/attrs were not transferred (see previous errors)
(code 23) at main.c(1655) [Receiver=3.1.1]
rsync: [Receiver] write error: Broken pipe (32)

This change fixes this by checking for an SQLite database file, and only
intiating the rsync transfer if it exists.

Change-Id: I370e5bc9f137abe552918a3215a025fa61e3a0ca
2018-09-24 09:32:02 +01:00
Zuul
0b6e401c4f Merge "Add support Octavia network interface" 2018-09-23 05:25:53 +00:00
Zuul
0c03f6041e Merge "Remove compute hosts from etcd group" 2018-09-23 05:16:28 +00:00
confi-surya
dfd372b6e2 Revert commit of fixing ceph gates
This patch is to test and revert the change

Change-Id: If930306a357e40ad02c4d7f71779701a9a52b94a
revert: Ie8feb03e9a07a6486a8b375d3304c9e6f1f8b890
2018-09-23 04:29:47 +00:00
Zuul
4ceb58bad0 Merge "Fix aodh task name" 2018-09-21 18:34:40 +00:00
Zuul
cbfcded0a7 Merge "Allow custom fluentd input configurations" 2018-09-21 17:29:02 +00:00
Paul Bourke
b163cb02d1 Update rabbitmq to use new conf & clustering
Depends-On: I75e00312b36e1678b90a42cf58d24652323eff27
Change-Id: Ia716fabffca41eff816e59bbf9f4cab79ee8b72f
2018-09-21 16:47:59 +00:00
Zuul
a247e6495e Merge "Remove when condition in roles/swift/tasks/deploy.yml" 2018-09-21 15:59:12 +00:00
Zuul
2ca6f89d8e Merge "Move the ironic-check action into prechecks.yml" 2018-09-21 15:59:11 +00:00
Eduardo Gonzalez
fdb5e33482 Fix aodh task name
Fix a typo in the policy task name

Change-Id: I43b9c1ea1e3ff4bef1713c889e2cb885207abb0e
2018-09-21 17:16:33 +02:00
caoyuan
e50d18e4fb Provide support for freezer dev mode in kolla
Add a possibility to mount sources as volumes to containers,
in "more than documentation" way. That will let us to use kolla
as a replacement for devstack.

Partially implements: blueprint mount-sources
Co-Authored-By: wu.chunyang <wu.chunyang@99cloud.net>

Change-Id: Ia3112a0d297ac777e1f25d3ddecced7746ca6418
2018-09-21 15:06:56 +00:00
Zuul
d03b6daf77 Merge "Make haproxy proxy to the right glance_api backend" 2018-09-21 15:00:56 +00:00
Zuul
dd1b1b7da0 Merge "Update the inner-compute name" 2018-09-21 10:18:40 +00:00
Zuul
a722986fa2 Merge "Apply Nova rolling upgrade logic" 2018-09-21 08:50:40 +00:00
caoyuan
ebf4afdad0 Move the ironic-check action into prechecks.yml
Change-Id: Ia1fa141acd7233f0c9dd30dd3d13e31cf1a2fb8e
2018-09-21 12:44:29 +08:00
Zuul
62d17e48d5 Merge "Use alembic migrations to bootstrap Monasca database" 2018-09-20 09:05:04 +00:00
Zuul
95135e7f03 Merge "Temporarily remove the rabbitmq clusterer plugin" 2018-09-20 07:43:17 +00:00
Doug Szumski
4c0656e10f Use alembic migrations to bootstrap Monasca database
Alembic uses the monasca-agent config file for the DB
credentials. These are therefore no longer required.

Partially-Implements: blueprint monasca-roles
Change-Id: Id076e7a0b838888314159dd9e76697f89adecb5e
2018-09-19 17:52:23 +01:00
Zuul
b31d6ed6e9 Merge "Increase CI job timeout to 2 hours" 2018-09-19 15:47:16 +00:00
Mark Goddard
ed8a7b7b89 Increase CI job timeout to 2 hours
If there is a change to kolla then images are built, which can push the
job over the existing 1.5 hour timeout. These jobs are also used in the
kolla CI.

Change-Id: I88a28745cf834376163ced36984c317fce831346
2018-09-19 10:54:51 +01:00
Duong Ha-Quang
c79c5d1f36 Apply Nova rolling upgrade logic
This patchset apply Nova rolling upgrade logic [1]

[1] https://docs.openstack.org/nova/latest/user/upgrade.html

Implements: blueprint apply-service-upgrade-procedure

Co-Authored-By: Ha Manh Dong <donghm@vn.fujitsu.com>
Depends-On: https://review.openstack.org/#/c/558765/
Change-Id: I20531a25dc7f5b05dc70ba771cf4c4222ade33e3
2018-09-19 16:25:51 +07:00
Dai Dang Van
008d45b8de Add support Octavia network interface
For now, we use api interface/network for Octavia.

This change will make more flexible for Octavia deployment
with Kolla when we want to use another network for managing
amphora instances (config, health check, clean up)

Change-Id: Ief12f1f8b6c7d3974932e6320af95bb58d46bdb9
Co-Authored-By: Duc Nguyen Cong <ducnc7@viettel.com.vn>
Closes-Bug: #1791207
2018-09-19 01:04:29 +00:00
Zuul
b46efe5dc4 Merge "Rework ceph-guide bluestore/filestore sections" 2018-09-18 11:26:44 +00:00
Zuul
0c0d4a2994 Merge "Add missing enable_horizon_fwaas variable" 2018-09-18 11:26:43 +00:00
Zuul
e29f5c84d9 Merge "Provide support for nova dev mode in kolla" 2018-09-18 09:56:09 +00:00
ZhijunWei
2d711bc860 Update the inner-compute name
Change-Id: I5bc56c843e1d8986b1c7bf649a6f8dafc0928e0b
Closes-Bug: #1793637
2018-09-18 03:36:38 -04:00
Zuul
2866704aa0 Merge "Remove_images in kolla_docker module" 2018-09-17 15:53:24 +00:00
ZhijunWei
6d5cf09f7e Fix the zookeeper group missing when run tests
Change-Id: I05960661b8efb32a3d57686dc3780d4138309ef5
Closes-Bug: #1793262
2018-09-17 10:01:41 -04:00
Michal Nasiadka
55455cdde2 Rework ceph-guide bluestore/filestore sections
Create bluestore/filestore subsections in ceph-guide and add note
about using bluestore by default - for better readability.

Change-Id: Ib51706838ecf35ac274053b916b3a17f0de90758
2018-09-17 10:35:47 +02:00
Eduardo Gonzalez
292f7c22e0 Allow configure java params for ODL karaf
Allow configure memory limits as variables in ODL karaf
java instance.
Allows us to define smaller memory settings for CI
and bigger for prod environments.

Change-Id: I8cafaa543ad09572bdbb6af1bbbd65f55fe42f8c
2018-09-14 19:29:08 +02:00
Eduardo Gonzalez
fffdc6b3e5 Remove_images in kolla_docker module
Add support to remove images from kolla_docker
ansible module.

Change-Id: Ib6bcb98b5b295a2d590df3013188913d1f7f3584
2018-09-14 17:14:25 +02:00
jiangpch
d57c7019a9 Make haproxy proxy to the right glance_api backend
Since glance_api only start one container when using file
backend, the haproxy should follow this rule.

See: https://review.openstack.org/#/c/448654

Closes-Bug: #1722422

Change-Id: Id3519581e0f54509dacd24d0dd542c630342c771
2018-09-13 15:24:02 +01:00
Kuo-tung Kao
34474decb1 Change the owner of the wsgi-zun.conf to zun
Change the owner of the wsgi-zun.conf to zun instead of root.

Change-Id: Ib0455210f3251965547ce3449fa4b72e28b32d11
Closes-Bug: #1792089
2018-09-12 08:05:22 +00:00