878 Commits

Author SHA1 Message Date
Zuul
98c9b148f3 Merge "Nagios: Update ceph_health check" 2018-11-09 03:24:23 +00:00
Zuul
a90ebb784c Merge "Prometheus: Update discovery configuration for ceph-mgr services" 2018-11-09 01:01:54 +00:00
Zuul
77772547e2 Merge "RGW: Fix multinode deploy for ceph rgw" 2018-11-08 22:54:01 +00:00
Zuul
d530635348 Merge "Do not use OSH_INFRA_PATH in osh-infra" 2018-11-08 22:54:00 +00:00
Meg Heisler
774e0cb654 RGW: Fix multinode deploy for ceph rgw
Change deployment script for rgw to not use the docker
bridge for public and cluster network overrides. Instead,
calculate network values in same way as other ceph multinodes
deployment steps

Change-Id: I2bacd1af1cc331d76a5d61f3b589ca6ef80b1b2e
2018-11-08 11:39:23 -06:00
Zuul
7274c5f95f Merge "Revert "Fix rally deployment config to rally 1.2.0"" 2018-11-07 22:26:22 +00:00
Zuul
47d49bcfd4 Merge "prometheus ceph.rules changes" 2018-11-07 20:51:42 +00:00
Pete Birley
b7e77dfea0 Revert "Fix rally deployment config to rally 1.2.0"
This reverts commit 5c2859c3e9026e464bf0c35b591aaae810ff2a1c.

This commit breaks the ability to declare users to use with rally/helm test - and needs to be refactored to match the commit message's intent.

Change-Id: I2bc66ef40694c277058b4324b8a3528f4f25d1d1
2018-11-07 19:31:49 +00:00
Zuul
b28aed8331 Merge "Fix rally deployment config to rally 1.2.0" 2018-11-07 14:12:32 +00:00
Steve Wilkerson
ba22b0e726 Nagios: Update ceph_health check
The ceph_health check in Nagios incorrectly sets the warning and
error level to 0. The ceph_health_status metric's value of 0
indicates the cluster is healthy, while 1 indicates a warning and
2 indicates an error state. The Nagios check for ceph_health is
updated to reflect these values

Change-Id: Iffe80f1c34f6edee6370dd7e707e5f55f83f1ec1
2018-11-06 14:51:40 -06:00
Steve Wilkerson
e0f2d66ee3 Prometheus: Update discovery configuration for ceph-mgr services
This updates the Prometheus scrape configuration to use the
service based discovery mechanism instead of endpoints. This
removes issues associated with multiple ceph-mgr replicas deployed

Change-Id: I2c557af0c7200d0c4aea646c5f9ecd1a070db33e
2018-11-06 13:56:37 -06:00
Jean-Philippe Evrard
ff1f75fc45 Do not use OSH_INFRA_PATH in osh-infra
If OSH_INFRA_PATH is never used in the openstack-helm-infra repository,
as all the references are using relative paths.

The keystone script is not using a relative path, and relies on
OSH_INFRA_PATH to be defined to work.

This is a problem, because when it is not defined, the expected path
for ldap chart is /ldap, which is an incorrect path.

This fixes the problem by ensuring the path is relative.

Change-Id: I04a8d5c074b7c1e6fa66617bbb907f2ad4dcb3af
2018-11-05 13:36:03 +00:00
Zuul
fca344900f Merge "Enable the mgr balancer module by default." 2018-11-02 22:36:13 +00:00
Steve Wilkerson
69196031cd Nagios: Ensure processes are reaped
This moves Nagios to run as child processes of either
the pause container or use the hosts init system (for k8s <1.10)
to prevent defunct process sprawl

Change-Id: I6a93d446577674b0b012f9567d5e6a5794ebc44b
2018-11-02 08:12:24 -05:00
Matthew Heler
a79562a28b Enable the mgr balancer module by default.
The balancer module will distribute PGs more evenly across OSDs. 
While CRUSH does a good job at this, it is not perfect and hot spots
(where an OSD has more PGs then it's peers) can occur.

Change-Id: Ic45a6bf745bdd09a3f5782e9e8bda89c3d3da2aa
2018-11-01 15:52:51 +00:00
inspurericzhang
f1c2bf976f [Trivial Fix] modify spelling error of "resource"
Although it is spelling mistakes, it affects reading.

Change-Id: I75a1f66002ec46fe206f31fec02fbd47f9cee443
2018-11-01 09:52:04 +08:00
kranthi guttikonda
fac358a575 prometheus ceph.rules changes
With new ceph luminous ceph.rules are obsolete.

Added a new rule for ceph-mgr count

Changed ceph_monitor_quorum_count to ceph_mon_quorum_count

Updated ceph_cluster_usage_highas ceph_cluster_used_bytes,
ceph_cluster_capacity_bytes aren't valid

Updated ceph_placement_group_degrade_pct_high as
ceph_degraded_pgs, ceph_total_pgs aren't valid

Updated ceph_osd_down_pct_high as ceph_osds_down,
ceph_osds_up aren't available, ceph_osd_up is
available but ceph_osd_down isn't. Need to
calculate the down based on count(ceph_osd_up==0)
and total osd using count(ceph_osd_metadata)

Removed ceph_monitor_clock_skew_high as the metric
ceph_monitor_clock_skew_seconds isn't  valid anymore

Added new alarms ceph_osd_down, ceph_osd_out

Implements: prometheus ceph.rules changes with new valid metrics
Closes-Bug: #1800548
Change-Id: Id68e64472af12e8dadffa61373c18bbb82df96a3
Signed-off-by: Kranthi Guttikonda <kranthi.guttikonda@b-yond.com>
2018-10-31 10:23:11 -04:00
Matthew Heler
3e7ba37290 Ensure latest Ceph packages during deployment
Change-Id: Ia5bc0802577e2b72a1de078085f5fe7e60f63604
2018-10-31 02:16:50 -05:00
Tin Lam
5730631ba6 Clean-up script
This patch set cleans up the script to be consistent with other OSH
installation scripts.

Change-Id: I212cd0cf0e818f1fc924b9b690d18f5d107b850b
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-30 16:22:45 +00:00
Zuul
31a9bb6ad4 Merge "[gate] Use Kubernetes 1.10.9" 2018-10-30 08:05:08 +00:00
Steve Wilkerson
45da8c2b69 Ceph: Update log directory host mount path
This updates the ceph-mon and ceph-osd charts to use the release
name for the hostpath defined for mounting the /var/log/ceph
directories to. This gives us a mechanism for creating unique log
directories for multiple releases of the same chart without the
need for specifying an override for each deployment of that chart

Change-Id: Ie6e05b99c32f24440fbade02d59c7bb14d8aa4c8
2018-10-29 13:05:46 -05:00
Chris Wedgwood
b10ebbb63a [gate] Use Kubernetes 1.10.9
Change-Id: I5bb951f455fa6d7d344a264336a2a9b985fd85f4
2018-10-29 15:10:35 +00:00
Matthew Heler
6ef48d3706 Further performance tuning changes for Ceph
- Throttle down snap trimming as to lessen it's performance impact
(Setting just osd_snap_trim_priority isn't effective enough to throttle
down the impact)
osd_snap_trim_sleep: 0.1 (default 0)
osd_pg_max_concurrent_snap_trims: 1 (default 2)

- Align filestore_merge_threshold with upstream Ceph values
(A negative number disables this function, no change in behavior)
filestore_merge_threshold: -10 (formerly -50, default 10)

- Increase RGW pool thread size for more concurrent connections
rgw_thread_pool_size: 512 (default 100)

- Disable in-memory logs for the ms subsytem.
debug_ms: 0/0 (default 0/5)

- Formating cleanups

Change-Id: I4aefcb6e774cb3e1252e52ca6003cec495556467
2018-10-26 15:10:50 +00:00
Zuul
62f49e7c74 Merge "Define OSH_PATH by default" 2018-10-26 11:35:11 +00:00
Zuul
3e62b48036 Merge "Node-Exporter: allows to set collectors enable/disable" 2018-10-26 08:01:40 +00:00
Zuul
2e239086e4 Merge "Restrict libvirt Ceph access scope to what is needed only." 2018-10-26 06:36:49 +00:00
Zuul
1ca39def84 Merge "ceph: make log directory configurable" 2018-10-26 02:41:15 +00:00
Jawon Choo
b4dfb27f0c Node-Exporter: allows to set collectors enable/disable
This PS allows to set collectors enable/disable using values.
_node-exporter.sh.tpl makes collectors-list from values.yaml.

Change-Id: Iba2cf4d8304f2405db394fbb6fee58119eab13fc
2018-10-26 01:15:15 +00:00
Jean-Charles Lopez
566a489bbe Restrict libvirt Ceph access scope to what is needed only.
Change-Id: I78bffe6764e9cbb16b2a615be766c910ba5d4e48
2018-10-26 01:15:12 +00:00
Jean-Philippe Evrard
52f41c0af0 Define OSH_PATH by default
OSH_PATH is not defined by default outside OpenStack's CI.

This is a problem if a user wants to run scripts manually on its
machine for local testing.

This fixes it by having, by default, the OSH_PATH defined
in the scripts using OSH relatively to current folder.

For user experience, the script returns to the same path after
running.

Change-Id: I915e7d3c945f2002a2008b2b033a2b7725320b17
2018-10-26 01:15:08 +00:00
Zuul
4835aa637a Merge "MariaDB: Galera cluster refactor" 2018-10-25 16:32:56 +00:00
Chinasubbareddy M
a1b8f394b2 ceph: make log directory configurable
this is make log directory configurable incase if  another mon or
osd running on same host can point to other directory

Change-Id: I2db6dffd45599386f8082db8f893c799d139aba3
2018-10-25 14:34:14 +00:00
Zuul
1ec8981aa4 Merge "MariaDB: Move to use mariabackup instead of xtrabackup-v2" 2018-10-25 13:44:16 +00:00
Matthew Heler
f8ac6c3f21 ceph co-location journal and permission fixes
Support co-located journals with Ceph helm chart
Ensure proper ownership set on OSD/Journal disks

Change-Id: Ic954d75c8bd7532991dc9b3184ad6d74b97855d1
2018-10-25 08:21:31 +00:00
Pete Birley
f6e84fe15f MariaDB: Galera cluster refactor
This PS updates the MariaDB chart to better support clustering,
using a configmap to track cluster state.

Change-Id: Ifd9c3d63353a9b587384b6f13c0863ecc4fbd956
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-25 06:21:01 +00:00
Pete Birley
8bc03bf88c MariaDB: Move to use mariabackup instead of xtrabackup-v2
This PS moves to use mariabackup instead of xtrabackup-v2, for info
see:
 * https://mariadb.com/kb/en/library/upgrading-from-mariadb-102-to-mariadb-103/#mariadb-backup-and-percona-xtrabackup
 * https://mariadb.com/kb/en/library/mariabackup-overview/#about-mariabackup

Additionally the readyness script is updated to match the order of
validation tests described in the mariadb/galera documentation.

Change-Id: I031c63d6305f1514ffdd53d77d621bc7edc0e68c
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-25 05:43:59 +00:00
Steve Wilkerson
8d6cfd72d0 Nagios: Remove Nagios log monitors
This removes the checks for Nagios to query Elasticsearch for
logged events. The current plugin in the image is resulting in
unstable behavior, and should be removed until this plugins been
improved

Change-Id: If1bdd954956f063ac1eebbb94d1128df8b8d2695
2018-10-25 05:21:22 +00:00
Tin Lam
653b84a2e1 Fix k8s-auth job
This patch set addresses a cross-repo conflict with the enablement of
network policy in gate script override.

Change-Id: I284d6b04940424a87e5b239ccc9d30ae01075f38
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-24 20:49:17 -05:00
Pete Birley
1144ccbbb2 Ceph: Update MGR check to allow use on hosts with fqdns defined
This PS updates the mgr check to allow use on hosts with fqdns
defined.

Change-Id: If1cb740e8093fbcafce846234c96db931409b436
Signed-off-by: Pete Birley <pete@port.direct>
2018-10-24 00:57:12 +00:00
Zuul
eabf53253f Merge "ceph-mgr: make prometheus module port configurable" 2018-10-23 23:19:09 +00:00
Chinasubbareddy M
e23e372120 ceph-mgr: make prometheus module port configurable
this is to give example for prometheus module port configurable

Change-Id: I66844bb8ee59a58f7bfd3e3002a183779810e881
2018-10-23 15:40:43 -05:00
Zuul
5c446bb2d3 Merge "Use supplied HELM variable for dep up in Makefile" 2018-10-23 20:36:22 +00:00
Zuul
f49461acc4 Merge "cronjob-checkPGs failure fix" 2018-10-23 20:21:46 +00:00
Zuul
860a897aee Merge "[gate] allow pip caching" 2018-10-23 18:30:20 +00:00
Zuul
4c4e947e17 Merge "Ceph: A script to check object replication across the hosts" 2018-10-23 18:25:43 +00:00
Zuul
1e3693f1a3 Merge "[gate] Put nfs-provisioner in it's own namespace (docker-registry)" 2018-10-23 18:22:01 +00:00
Zuul
bad8427b21 Merge "[gate] Put nfs-provisioner in it's own namespace" 2018-10-23 18:22:00 +00:00
Zuul
11ec46bdce Merge "Prometheus kubelet.rules change" 2018-10-23 17:57:26 +00:00
Bryan Strassner
dacb01c82a Use supplied HELM variable for dep up in Makefile
Updates the helm dep up command to use the $(HELM) variable instead of
the locally istalled helm for the host machine. This bring this line of
code in alignment with the other uses of helm in the same Makefile.

Change-Id: I91bfdceedd3bac0ac49daf5b9410c05e0e840168
2018-10-23 11:26:16 -05:00
Zuul
a0d58decff Merge "[Calico] Allow resource configuration using chart (overrides)" 2018-10-22 22:49:08 +00:00