9301 Commits

Author SHA1 Message Date
Doug Szumski
380418f351 Update Monasca Log Persister config
* Make number of threads configurable
* Improve naming
* Update path and log options for Logstash 2.4
* Use the correct logs directory

Partially-Implements: blueprint monasca-roles
Change-Id: I09388de4d31740f2786393c957201bd037572226
2018-10-09 16:04:51 +01:00
Doug Szumski
b1dee2d85b Increase maximum payload size for Monasca Log API
Increase the maximum payload size so that it slightly exceeds the default
maximum chunk size (8MB) in Fluentd. This prevents the Monasca Log API
dropping logs when under heavy load.

Partially-Implements: blueprint monasca-roles
Change-Id: I13020abde7de1c92acc07352466b814b4a6ce5bc
2018-10-09 16:04:51 +01:00
Doug Szumski
0cae7d9b34 Prevent Monasca Notification using YAML mode
If the config option is not specified Monasca Notification defaults
to YAML config parsing and fails to load.

Partially-Implements: blueprint monasca-roles
Change-Id: Id85d9765df525b34d43a77a9e4045e0de19aef99
2018-10-09 16:02:47 +01:00
Doug Szumski
2af1d1d95e Improve registration of Monasca InfluxDB database
Monasca is not yet compatible with InfluxDB > 1.1.10, which means
that the official Ansible modules for managing InfluxDB don't work [1].
We therefore fall back to manual commands to register the database
and a default retention policy.

[1] https://github.com/influxdata/influxdb-python
    #influxdb-pre-v110-users

Partially-Implements: blueprint monasca-roles
Change-Id: I59ceda1e7a6e945b13872089011045db04548b94
2018-09-26 10:54:43 +00:00
Doug Szumski
645853bce9 Add region to Monasca Log API
This is required for upcoming log query support and it also
causes an error in the Keystone middleware if it's missing.

Partially-Implements: blueprint monasca-roles
Change-Id: I2bcb32bc0c079c799d2b0e45a97b454d38896986
2018-09-26 10:54:43 +00:00
Doug Szumski
9c2e0b81d5 Support configuring Monasca Persister performance
On a single node deployment, the Monasca persister can
limit the rate at which Monasca can persist metrics to
InfluxDB. Increasing the thread count can remove this
bottle neck.

Partially-Implements: blueprint monasca-roles
Change-Id: I763a5ae6aa8c8ab3bf766ab5b58c386da74a188b
2018-09-26 10:54:43 +00:00
Doug Szumski
6b4cd657ad Add missing Monasca persister and notification dimensions
Partially-Implements: blueprint monasca-roles
Change-Id: I2fb43387bd1bc3b70f5c23d5268a7a7154ec31f0
2018-09-26 10:54:43 +00:00
Doug Szumski
fddbbbbdc4 Support deploying Monasca Persister
The Monasca Persister reads metrics from Kafka and stores them
in a configurable time series database.

Change-Id: I8166b32bfb1583098ab8318a5f38d25bddb81e89
Partially-Implements: blueprint monasca-roles
2018-09-26 10:54:43 +00:00
Doug Szumski
c502e0b17a Add some missing parameters for Monasca Notification
Partially-Implements: blueprint monasca-roles
Change-Id: I21de7748156f8d3689ebfc29f2fc4dc5f7f36ddf
2018-09-26 10:54:42 +00:00
Doug Szumski
da1fa3f578 Support deploying Monasca Notification engine
The Monasca Notification engine generates alerts such as Slack
notifications from alerts.

Change-Id: I84861d5feefe6b6f38acc4dd71e94c386d40b562
Partially-Implements: blueprint monasca-roles
2018-09-26 10:54:42 +00:00
Doug Szumski
b6cce3e3f3 Support deploying Monasca Thresh
Monasca Thresh is a Storm topology which generates alerts from
metric streams according to alarms defined via the Monasca API.

This change runs the thresholder in local mode, which means that
the log output for the topology is directed to stdout and the
topology is restarted if the container is restarted. A future
change will improve the log collection and introduce a better
way of the checking the topology is running for multi-node
clusters.

Change-Id: I063dca5eead15f3cec009df62f0fc5d857dd4bb0
Partially-Implements: blueprint monasca-roles
2018-09-26 10:54:37 +00:00
Doug Szumski
516b0dd34e Support deploying an Apache Storm cluster
Storm is required for running the Monasca thresholder component for
generating alerts.

Change-Id: I5e1ef74dc55a787293abbb3e629b5ab1ce5f4bbb
Partially-Implements: blueprint monasca-roles
2018-09-26 10:54:32 +00:00
Doug Szumski
1ae10375f7 Support deploying Monasca Log Metrics
The log metrics service generates metrics from log messages
which allows further analysis and alerting to be performed
on them. Basic configuration is provided so that metrics
are generated for high level warning logs such as error, or
warning.

Change-Id: I45cc17817c716296451f620f304c0b1108162a56
Partially-Implements: blueprint monasca-roles
2018-09-25 16:36:14 +00:00
Doug Szumski
01da938412 Support configuring Monasca log pipeline performance
Change-Id: Id8948fcf2d165f8285c7562e7aebd4145c4ff0db
Partially-Implements: blueprint monasca-roles
2018-09-25 11:41:29 +00:00
Doug Szumski
2b34f6d2fd Improve scalability of Kafka
Set sensible defaults for replica counts and minimum insync replicas
as a function of the number of nodes in the Kafka cluster.

Partially-Implements: blueprint monasca-roles
Change-Id: Icf1dddb7dd6a64f4e5efb7dffa5ffdf0880f891f
2018-09-25 11:41:00 +00:00
Doug Szumski
0a90020289 Remove Kafka partitions config option
This option doesn't actually do anything and a bug to remove
it from the Monasca API config file has been raised.

Partially-Implements: blueprint monasca-roles
Change-Id: I7ec1786b5828ab0135ca86ec040f83a6f4c78d9f
2018-09-25 11:40:54 +00:00
chenxing
8219542f89 Fix the syntax issues of ovs-dpdk
backport: rocky

Change-Id: Id437c26376f80462369db1fb3888db2098278f16
Closes-Bug: #1791686
2018-09-11 10:47:03 +08:00
Zuul
e7958d1435 Merge "Add check.yml for prometheus and vitrage" 2018-09-06 11:34:09 +00:00
Zuul
00f26ad0a2 Merge "Update reno for stable/rocky" 2018-09-06 07:20:11 +00:00
Zuul
2ff923c695 Merge "fix condition of kibana pull action" 2018-09-05 15:28:15 +00:00
f7c208084c Update reno for stable/rocky
Change-Id: I6f14d42362fc2364d967b0e2a4ebc57700fab200
2018-09-04 16:24:24 +00:00
Mark Goddard
354894e2e9 Add check.yml for prometheus and vitrage
Without this, kolla-ansible check fails with the following error:

Unable to retrieve file contents
Could not find or access '/path/to/kolla-ansible/ansible/check.yml'"}

Also adds the check command to the CI tests, to ensure that it does not
break again.

Change-Id: I9fc2f9999f55cb742ac3ac38579dcf26524a9fc7
Closes-Bug: #1790653
2018-09-04 15:36:34 +01:00
Zuul
4dcb4ce741 Merge "Run etcd on controller node only" 2018-09-03 08:59:38 +00:00
Zuul
6fca49ab51 Merge "Fix bluestore disk naming format in kolla-ansible" 2018-09-03 06:00:30 +00:00
Hongbin Lu
d29ecbd64c Run etcd on controller node only
In before, etcd container is launched in compute node if kuryr is
enabled. This logic breaks the all-in-one setup in which the etcd
cluster is configured with two members (controller and compute)
but it actually has one member only.

However, even if kuryr is enabled, running etcd in compute node
is not the common practice and kuryr has changed its devstack
plugin to run etcd on controller only [1].

[1] https://review.openstack.org/#/c/514815/

Closes-Bug: #1790415
Change-Id: I701d495675178c3ed8ec1f00b31d09f198b38a6f
2018-09-02 22:30:50 +00:00
binhong.hua
1e26b05fe0 fix condition of kibana pull action
Change-Id: Ia459d8b9925b22b23c90a33364da6ef40f4b1a70
2018-08-31 14:39:33 +08:00
Victor Morales
cc5c347217 Fix Copying 10-rest-connector.xml task
The destination folder for this task requires root permissions.

Change-Id: Icdc37cde7ba7b6971cf36632a9b6071b3cf600c2
2018-08-30 21:08:53 -07:00
wangwei
4e5e28fff5 Fix bluestore disk naming format in kolla-ansible
The current bluestore disk label naming is inconsistent with the
filestore. The filestore naming format is that the disk prefixes
belonging to the same osd are the same and the suffixes are
different.

This patch keeps the bluestore's disk naming as well.

Change-Id: I71dda29fc4a6765300ce7bb173d2c448c24f6eca
2018-08-31 09:55:09 +09:00
Zuul
3f14a99f2a Merge "[prometheus] Allow custom alert rules to be configured." 2018-08-30 11:57:11 +00:00
Zuul
f16a467758 Merge "Improve ODL configuration" 2018-08-30 09:44:37 +00:00
Zuul
cfee876895 Merge "[prometheus] Enable ceph mgr exporter" 2018-08-30 07:09:48 +00:00
Zuul
a9e02d6264 Merge "Remove usage of nova's RamFilter" 7.0.0.0rc1 2018-08-29 08:53:56 +00:00
Zuul
fd8fbafe2a Merge "Remove the unncessary register variable defined" 2018-08-29 08:50:31 +00:00
Zuul
cde23811f7 Merge "Support checkout dedicated version from git with dev mode" 2018-08-29 05:53:04 +00:00
zhulingjie
7f1f0178c7 Remove the unncessary register variable defined
when creating Nova databases user and setting permissions,
no need to register the database_user_create, and it used
nowhere, remove it is safe

Change-Id: If456b7c2ed25aa729be7d98ef875230c66581d65
2018-08-28 23:01:25 -04:00
Matt Riedemann
e4a9d48f20 Remove usage of nova's RamFilter
As of https://review.openstack.org/596502/ in Stein
the RamFilter in nova is deprecated. It is not needed
when using the filter_scheduler driver (which is the
default scheduler driver).

Change-Id: Iee3dab7b5dc7074e9840826afe58ce2bfa40a3df
2018-08-28 14:20:39 -04:00
Zuul
874cde1e35 Merge "Allow nova to use cephfs cinder volumes without nova cephfs backend" 2018-08-27 07:42:51 +00:00
Zuul
140180178b Merge "Include default_docker_volume_type for magnum.conf" 2018-08-24 05:36:04 +00:00
Zuul
6dd6fe8028 Merge "Fix redis sentinel change myid every time when it restarted" 2018-08-23 10:23:55 +00:00
Zuul
3288e0c4e3 Merge "Update usage of "|" to "is"" 2018-08-23 10:23:54 +00:00
qingszhao
b7ea7f5761 add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: Ife0cb63cf4e5c1cf2ce43510bbf9ff130fdfc1f7
Story: #2002586
Task: #24305
2018-08-23 07:47:23 +00:00
qingszhao
f326f6fd19 switch documentation job to new PTI
This is a mechanically generated patch to switch the documentation
jobs to use the new PTI versions of the jobs as part of the
python3-first goal.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: I2e313bb43d3a03a0d516cd87a77b3911363de7ee
Story: #2002586
Task: #24305
2018-08-23 07:46:52 +00:00
qingszhao
d272aa1306 import zuul job settings from project-config
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.

Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.

Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: Iddd03da96d104a4c44c811b2dffd36a2d5b5d14e
Story: #2002586
Task: #24305
2018-08-23 07:40:05 +00:00
Liping Mao (limao)
735d085bc7 Fix redis sentinel change myid every time when it restarted
If you use COPY_ALWAYS, kolla will overwrite /etc/redis/redis.conf.
But Redis Sentinel will store myid in this file. So when it restart,
Redis Sentinel will regenerate new myid every time. Redis Sentinel
cluster will see more and more sentinel, This patch use hash value
with hostname as sentinel myid.

Change-Id: I6d9403231e41baac177f29c58e67ba605795385d
Closes-bug: #1788179
2018-08-23 14:34:28 +08:00
Zuul
eb249b4fea Merge "add trove policy file customization" 2018-08-20 18:08:00 +00:00
Zuul
511e3d2389 Merge "Make source jobs voting and gating" 2018-08-20 09:38:56 +00:00
Zuul
1a6d28e756 Merge "Explicitly specify endpoint type during Monasca registration" 2018-08-20 08:43:39 +00:00
wu.chunyang
69bc7b4ee2 add trove policy file customization
now,the trove does not support policy customization.
this ps to add it.

Change-Id: I32d62d0293ddf63572cf754a7108ed74f94b216a
2018-08-20 04:28:06 +00:00
Doug Szumski
ea5a7dd300 Explicitly specify endpoint type during Monasca registration
Monasca defaults to using the public endpoint to register
Monasca endpoints which is bad practice and doesn't work on
some deployments. This change uses the admin interface by
default.

Change-Id: Ia9c5e041e71867fe72aad43e1344cd2107652d7d
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Closes-Bug: #1787610
2018-08-17 16:46:01 +01:00
Eduardo Gonzalez
f5e83cedb4 Temporary ignore B303 in bandit tests
Revert this change once the insecure usages are fixed.

Change-Id: Id7e8c9fe72db1b6f123eca540e09f77ddee2f242
2018-08-17 15:04:18 +00:00