This adds a liveness probe to the fluentd chart. This probe will
simply perform a tcpSocket check on the same port the readiness
probe executes the check on.
Change-Id: I768b23d36d50d6f6938f5588bea71e97aeb624b9
This updates the Prometheus pod container status alerts. This
ensures there are alerts defined for ImagePullBackOff,
ErrImagePull, and CreateContainerConfigError errors.
This also updates the Nagios service checks to include correct
checks for those alerts
Change-Id: I91544e7dff8c6aac8c79cd8aa7d8f7bc03adaa9a
Task: 29045
Story: 2004843
This patch pins pip to 18.1 as the latest pip 19.0 has a problem with
--no-cache-dir option. This problem is causing the build of docker
images of mariadb and kubeadm-aio to fail when they upgrade the
setuptools package.
Change-Id: If2b76249eeacec519a6a76605607ba6f3f81ac7d
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
Create a cron and associated script to run monthly OSD defrags.
When the script runs it will switch the OSD disk to the CFQ I/O
scheduler to ensure that this is a non-blocking operation for ceph.
While this cron job will run monthly, it will only execute on OSDs
that are HDD based with Filestore.
Change-Id: I06a4679e0cbb3e065974d610606d232cde77e0b2
This adds exception handling to the Kibana Selenium tests
to address the test failures due to TimeoutExceptions when
the dashboard loads slowly. Only TimeoutExceptions are handled
so if there is an issue with the page itself an error will still
cause the gate to fail as intended. When a TimeoutException
occurs an error message is logged and a screenshot is taken
of the current page.
Change-Id: I16cd3a61ffce2e5fdc39bd7731cc068b8a6ec41f
This updates the Ceph dashboards for Grafana, as some of the ceph
metrics have changed with the Mimic release. This fixes issues
with the ceph OSD metrics that broke some Grafana panels, and also
removes the Ceph panel for displaying the number of monitors in
quorum, as that metric has been removed in Mimic
Change-Id: If6cbbfa7d2972ddd0e44b29a6c8277188d2d9ff0
Under some conditions udev may not trigger correctly and create
the proper uuid symlinks required by Ceph. In order to work around
this we manually create the symlinks.
Change-Id: Icadce2c005864906bcfdae4d28117628c724cc1c
This PS updates the ks user job script to not display the password
on stdout.
Change-Id: I3c11601a409d6d5993c351170c7057217cfabd8a
Signed-off-by: Pete Birley <pete@port.direct>
PS provides possibility to use TLS in etcd (for Calico).
The ansible scripts were updated as well.
Change-Id: I522a78043a125660153aaa60f13d61ba8e325e75
This temporarily adds a "|| true" suffix to the kibana
selenium script execution, as we've noticed rare cases where the
tests fail due to the paths not being ready in time. Once we have
a path forward for waiting to ensure the path is ready,
we should allow for periodic failures of the kibana selenium tests
Change-Id: I6c406ad8907cc87425562dee56eec6b8a0502142
This PS updates the Keystone user job script to not
set the user password upon potential creation, falling back
to the set password command later in the script. This is both
slighly cleaner, and avoids potential race conditions when
running multiple keystone servers.
Change-Id: Ibe775df23fe7b747aea5137ca85975e067b8cea3
Signed-off-by: Pete Birley <pete@port.direct>
This helps verify Kibana is working properly by using
Selenium Webdriver to navigate to different index dashboards
and takes screenshot of each one. It also add the scripts to
the gates for single and multinode deployments.
Change-Id: Ic2c91734d1eaac0ea4e7985bf69082942166715d
This updates the Nagios chart configuration to not use syslog for
logging, removes the logging of notifications, and drastically
increases the number of concurrent checks executed.
This also removes the hostPath for Nagios logs, as it seems to add
no value over what's already reported to the console. Finally, as
Nagios's log file has the potential to grow very rapidly while the
service has no means to disable logging to disk, this adds a
readiness probe that both checks whether Nagios's endpoint is
being served and clears out the log file by redirecting the
no-op commands output to the nagios log file.
Change-Id: I81151c48ef4e0b7877f595c271f55b8fd479e8c1
If the source chart does not declare a vHost value, or uses the value of
"/", the script would fail upon trying to declare the vhost. This change
avoids the declaration of the "/" vhost, and continues with setting the
specified user with permissions to "/"
Change-Id: I28619c0aef22049c632c92a2f9a9d3831f8c284c
This updates the Elasticsearch health status expressions used in
Prometheus, Nagios and Grafana. The previous Prometheus rule
defined for Elasticsearch health checked for a status that was
> 0 to trigger an alarm for a green health status. The correct
returned values are: 1 for green, 0 for both red and yellow. This
changes the expression to use arithmetic operators to give us a
result that maps to: 2 for green, 1 for yellow, 0 for red.
This also updates the Elasticsearch dashboard in Grafana to add a
new mapping for the updated 2g,1y,0r scale.
Finally, this also updates the Nagios service check to be a bit
more verbose in its output.
For reference, see:
https://github.com/justwatchcom/elasticsearch_exporter/issues/120
Change-Id: I6ef2a7c308c6ebfdb693b46127a285bceb6ba872
This fixes the Nagios volume mount for the Elasticsearch query
file. Previously, the check for adding the volumemount to the
pod definition was incorrect. This fixes the conditional check,
and also adds the same conditional check to the configuration
secret
This adds a simple check to the monitoring and multinode jobs to
validate the resulting json gets mounted into the pod successfully
Change-Id: I2af289ccc4e1cff1669cb5e6e829514781b14dd3
This updates the helm-toolkit s3 bucket creation script and job
manifest to account for situations where the radosgw endpoint
might require the --no-ssl flag. The update checks for the
radosgw endpoint scheme to determine whether to use the flag in
order to preserve previous behavior
Change-Id: I75f441f55ca29b7864c09c70d875e48b366ebf52
When ceph-osd journal as a directory and data as
a block device ceph-osd fails to deploy while
waiting for the journal file in
/var/lib/ceph/journal/journal.<id>
Added the condition before checking bluestore for
directory and removed the same later in the script
Closes-Bug: #1811154
Change-Id: Ibd4cf0be5ed90dfc4de5ffab554a91da1b62e5f4
Signed-off-by: Kranthi Guttikonda <kranthi.guttikonda@b-yond.com>
Signed-off-by: kranthi guttikonda <kranthi.guttikonda9@gmail.com>
This updates the fluentd configuration to use 8 threads for the
Elasticsearch output configuration by default. This uses the
correct buffer output settings for the fluent-elasticsearch
plugin
This also updates the buffer output settings to the defaults used
for fluentd
Change-Id: I976cddaa973e850dabe4de495cd3bf1a4acdd4e7
This creates a new section in calico/values.yaml that enables
BGP communities to be applied to a cidr by using the bird_ipam
templates.
Change-Id: I4dbbc8d8e761e0484eeb7c8bf0fefa28d29493e5
Using a node selector can not run the prometheus-process-exporter
on the master node. So, This PS changes the scheduling to use
either taint/toleration or the node selector.
Change-Id: Ie84b2d2e0354fa927c1010c18392667dad171483
