125 Commits

Author SHA1 Message Date
KHIYANI, RAHUL (rk0850)
a43f479e6c Fix application name for grafana session sync
Implement helm-toolkit snippet for grafana add-home-dashboard
which adds security context template at pod/container

Change-Id: I12a5fd6c5043079f830eb36043f5b0ca495a3e93
2020-07-07 16:50:41 -05:00
Andrii Ostapenko
824f168efc Undo octal-values restriction together with corresponding code
Unrestrict octal values rule since benefits of file modes readability
exceed possible issues with yaml 1.2 adoption in future k8s versions.
These issues will be addressed when/if they occur.

Also ensure osh-infra is a required project for lint job, that matters
when running job against another project.

Change-Id: Ic5e327cf40c4b09c90738baff56419a6cef132da
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-07 15:42:53 +00:00
Andrii Ostapenko
83e27e600c Enable key-duplicates and octal-values yamllint checks
With corresponding code changes.

Change-Id: I11cde8971b3effbb6eb2b69a7d31ecf12140434e
2020-06-17 13:14:30 -05:00
Andrii Ostapenko
8f24a74bc7 Introduces templates linting
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.

Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.

Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
2020-06-11 23:29:42 -05:00
Andrii Ostapenko
731a6b4cfa Enable yamllint checks
- document-end
- document-start
- empty-lines
- hyphens
- indentation
- key-duplicates
- new-line-at-end-of-file
- new-lines
- octal-values

with corresponding code adjustment.

Change-Id: I92d6aa20df82aa0fe198f8ccd535cfcaf613f43a
2020-05-29 19:49:05 +00:00
diwakar thyagaraj
163c5aa780 Enable Apparmor to all osh-infra test pods
Also Changed container names to static.

Change-Id: I51f53b480d18aaa38a9707429f01052ee122e7e9
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-19 15:36:07 +00:00
Gage Hugo
d14d826b26 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
2020-05-07 02:11:15 +00:00
diwakar thyagaraj
aaeb0b1abb Enable Apparmor to Grafana Completed pods
This also adds init containers.

Change-Id: Ia70db208a1583b9a44a32d9a3d485ca7dc8a3ce2
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-05 15:59:22 +00:00
Radhika Pai
20aad64409 [Update] Grafana: add home dashboard script
Adding a loop to wait for the grafana dashboard to be up and contain the
OSH Home dashboard before running the script.
This should resolve the job completing before the OSH Home dashboard is
in the grafana pod.

Change-Id: I7ab20fad3ce7f7216e2b2679d863f02f97ef1ff4
2020-03-12 14:08:35 -05:00
Radhika Pai
dc9e435abb Grafana: Change to import the dashboards in json raw format
This code change is to enable grafana to use the raw json format
dashboards . This is to avoid the conversion of dashboards from json to
yaml and back to json during which the format is encountering issue.
Also this will help in adding new dashboards and maintaining the old
ones.
All the exisiting dashboards under values_override folder are updated to
use raw json format.

Change-Id: I48a7db1514857e082cecbb3b57deff9174509601
2020-03-04 09:52:36 -06:00
Radhika Pai
b0bb378a3c Grafana: Provision to add customized HomePage
This code will help to add any customized dashboard as a Home Page for
Grafana. The add_home_dashboard script will be executed after the
Grafana is deployed which sets a new Dashboard(OSH Home) as a landing
Page for a specific Organization.

Change-Id: I32b6b9cad4eaefe7d153cae797d3b3143be5c49b
2020-02-26 10:00:21 -06:00
dt241s@att.com
cc392aaa85 Add Apparmor to Grafana
Added apparmor feature gate and Zuul Gate Job

Change-Id: I9ce522f77447b1cb3f189ab7023c5c711e577618
2020-02-25 19:25:12 +00:00
Pai, Radhika (rp592h)
370111ee48 Grafana: Updated the ceph OSD expression
In Ceph Cluster Dashboard the OSDs In, OSDs Out, OSDs Down Panel was
showing wrong values. Updated
the expression from "count" to "sum" to show the correct values.

Change-Id: I1959eeb445bf297c1ec696f3867315f05552b03e
2020-01-10 04:45:27 +00:00
Steve Wilkerson
3a6df3b544 Grafana: Remove default dashboards from chart
This removes the default dashboards from the Grafana chart and
instead places them in the values_overrides directory, similar to
what was done for the Prometheus rules. As Grafana dashboards
will likely be heavily dependent upon end-user needs, the old
default dashboard configs should only be used as a reference
instead of opinionated defaults that are difficult to override.
The previous defaults made using specialized labels for dashboard
variables difficult, as they were making dangerous assumptions
about deployed namespaces and host fqdns. By removing the defaults
entirely, end users can define their own dashboards to meet their
specialized needs

Change-Id: I7def8df68371deda0b75a685363c8a73b818dd45
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-09 13:39:13 +00:00
Steve Wilkerson
97e029e606 Grafana: Support multiple datasources
This updates the Grafana chart to support the definition of
multiple datasources. This moves to defining a template in the
chart's values.yaml file that allows for inline gotpl for
defining an arbitrary number of datasources. This also updates the
grafana dashboards to include a selector for the Prometheus
datasource to use via a drop down selector. This is vetted out in
the federated monitoring job

Change-Id: I55171fed5c2b343130d135d0b42bc96ff11c4712
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-22 14:45:04 +00:00
Zuul
2b3d3ef131 Merge "Move charts off using the :latest built tags" 2019-11-21 21:34:04 +00:00
Steve Wilkerson
cbeb7f149b Move charts off using the :latest built tags
This updates charts that consume images built from osh-images to
use tags other than the :latest tags. This will be followed up
with the definition of jobs to allow for vetting out of updated
images, as reliance on :latest tags assumes any change merged into
osh-images will result in functionally correct behavior (which has
shown to not be the case traditionally)

Change-Id: I181aa56ed187604dc7583d8081e53cc69eb27310
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-21 19:57:07 +00:00
Steve Wilkerson
a4816feda2 Grafana: Add support for arbitrary environment variables
This updates the Grafana chart to support the definition of
arbitrary environment variables to support scenarios where
additional information may be required at runtime for things like
datasource and dashboard provisioning

Change-Id: I95e4abe9030116a440c6d78a1d14dbcaaf743b40
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-21 12:40:04 +00:00
Zuul
3e18a436d2 Merge "Grafana: Update version" 2019-11-19 20:50:37 +00:00
Zuul
029f94a776 Merge "Grafana - Update cadvisor labels for k8s 1.16" 2019-11-18 20:55:10 +00:00
Steve Wilkerson
1bfa091203 Grafana: Update version
This updates the Grafana version deployed by default from 5.0.0 to
6.2.0

Change-Id: I39b5405cc3f3fe7754ed6544a8388ff912a4ef58
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-18 08:49:38 -06:00
Bharat Khare
ab95e311a3 Grafana - Update cadvisor labels for k8s 1.16
This patch set will implement the grafana metrics related changes
required for kubernetes version upgrade to 1.16. Updates are mostly
specific to cadvisor metric labels. It is to make sure all
existing metrics are scraped and available in Prometheus so that
these can be consumed by Grafana & Nagios.

Change-Id: I74369ac49dd3f7d9f3682dd5318a3818a4d3f178
2019-11-11 17:57:09 +00:00
Zuul
c90ffb11f9 Merge "Grafana gridPos y key resolves true in chart json" 2019-11-11 09:42:13 +00:00
Bharat Khare
020a9d9593 Grafana gridPos y key resolves true in chart json
This change is to fix the issue where Kubernetes Calico dashboard
is not loading in grafana dashboard.
Found the behavior that YAML reserved keys like yes/no/y/n evaluate
to true and false in Ruby hash,so the gridPos key like 'y:' converts
to 'true:' in charts json causing the dashbaord not loading.
Hence updating gridPos panel key (y:) by wrapping it with quotes
like ('y':) to resolve the issue.
There's actually a long list of YAML reserved words with this
behavior like below-
y|Y|yes|Yes|YES|n|N|no|No|NO
|true|True|TRUE|false|False|FALSE
|on|On|ON|off|Off|OFF

Change-Id: I56b01f1312efa8de90def349518c618affb5de85
2019-10-21 14:58:58 +00:00
Steve Wilkerson
b50fae62a4 Update kubernetes-entrypoint image reference
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintained

Change-Id: I5bfdc156ae228ab16da57569ac6b05a9a125cb6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-10-18 18:20:11 +00:00
Steven Fitzpatrick
998885c330 Update Grafana Helm test to use python3
This change updates the selenium_tests container image
to one which installs python3.

The selenium-test.py template file has been refactored
to match the structure of the selenium tests in
/tools/gate/selenium

Depends on: https://review.opendev.org/688436
Change-Id: I4ece5c71df18c21f0cdff536140f63881ff24e30
2019-10-17 11:39:26 -05:00
Pai, Radhika (rp592h)
3c2c2d78d0 Grafana:Updated ES unassigned shards query
The elasticsearch dashboard was not showing correct value for the
unassigned shards panel. Updated the query.

Change-Id: I60056daede2957a45958769dd6b723f54cc54ed7
2019-09-19 19:58:34 +00:00
Zuul
c9d6676bf9 Merge "Grafana: Updated the Ceph-Cluster variable sorting" 2019-09-16 17:15:12 +00:00
Steve Wilkerson
d52fd14373 Use internal endpoint lookups for selenium helm tests
This updates the grafana and nagios helm test pod templates to
use the internal endpoints for their selenium tests instead of the
public endpoints when defined

Change-Id: I1138cb29a808894d3339bc1b07c3a60804b9546f
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-08-28 15:06:55 -05:00
Pai, Radhika (rp592h)
f6ff42061f Grafana: Updated the Ceph-Cluster variable sorting
Earlier the query used to sort variables in asc alphabetical order,
updated to sort the cluster in desc alphabetical order

Change-Id: I8f08a44b05d0159ad1a043f052751d44b4625f1d
2019-08-16 16:07:47 +00:00
Pai, Radhika (rp592h)
a37925c7e8 Grafana: Code for Calico Dashboard
Appended the code that will add the calico dashboard to the Grafana. This will
display the felix metrics which are collected by the prometheus.

Change-Id: If18a18949f8093747b3f9ba819e036778c40b84e
2019-07-31 20:53:55 +00:00
Pai, Radhika (rp592h)
891e259d66 Updated the CEPH Cluster Health Panel values
Change-Id: Id4016d1ce6c0e2acadef31496102667ee79f030f
2019-07-11 15:25:39 +00:00
Sreejith Punnapuzha
7314edc57d Fix Grafana helm test exception error
Grafana helm test is failing with the below error
"NameError: name 'exception' is not defined"
This is because exception is defined in smaller case. changing
exception to Exception fixes this issue

Change-Id: I533ae822babb4f063242fee1cd42b5b821519b5f
Signed-off-by: Sreejith Punnapuzha <Sreejith.Punnapuzha@outlook.com>
2019-07-08 14:02:42 +00:00
Zuul
819cf51083 Merge "Grafana: Disable analytics settings" 2019-07-04 18:36:59 +00:00
Zuul
e45a7def59 Merge "Add OpenStack instance panel to grafana dashboard" 2019-07-04 00:09:45 +00:00
Zuul
ebd59710e9 Merge "Grafana: Remove tests that query API" 2019-07-02 15:47:18 +00:00
John Lawrence
27ef45017b Adding hostname to the node details
Change-Id: I558c100cbb70475d55122b194ef18287c76a68b3
2019-06-28 13:20:06 +00:00
Steve Wilkerson
c7290b7ffe Grafana: Remove tests that query API
This removes the tests that query the Grafana API for checking
whether the prometheus datasource has been provisioned and for
checking the number of active dashboards against the number of
expected dashboards determined via the chart's values.yaml.

The reason for removing these is that Grafana can be configured
to use data source types beyond just Prometheus and additional
dashboards can be added to Grafana via the Grafana UI.  In cases
where dashboards are added via the Grafana UI, they are persisted
in the grafana database which will cause helm test failures during
upgrade scenarios.  Now that we have selenium tests executed as
part of the Grafana helm tests that validate Grafana is
functional, these API tests add little value

Change-Id: I9f20ca28e9c840fb3f4fa0707a43c9419fafa2c1
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-26 09:25:47 -05:00
Steve Wilkerson
4b8fdc390a Grafana: Disable analytics settings
This disables the analytics settings for Grafana that will check
grafana.com for plugin/dashboard updates every 10 minutes and for
sending anonymous usage statistics

Change-Id: I0f5283a8a54b563199528bb612aa0cdc6cf238e2
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-26 09:18:36 -05:00
Hemant
79a2b09d4b Add OpenStack instance panel to grafana dashboard
Add new openstack instance panel to grafana dashboard to see the status
of instances.

Change-Id: I42c62bca73dd3d257154a3141f8902d607789ac3
2019-06-20 12:56:00 +02:00
Steve Wilkerson
b4b1dd9528 Add missing affinity keys to chart pod specs
This adds the affinity key to the pod spec for the grafana,
nagios, kube-state-metrics, and openstack-exporter charts as it
was previously missed

Change-Id: Ifefa88d7f33607b4d595effa5fbf72f3387e5081
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-13 19:15:42 +00:00
Steve Wilkerson
25e105f26a Grafana: Add Selenium tests to helm test pod
This adds selenium tests for the grafana chart to the helm test
pod to help ensure the Grafana deployment is functional and
accessible

Change-Id: Idc8d97e5111628d1ed4f25145086d54c5e0136e7
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-05 16:09:04 +00:00
caoyuan
040edeb79a Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I0e3af4a3385f5b2a7705bc19b775863b16c2e08e
2019-05-31 01:52:10 +00:00
Tin Lam
6bae1b020d Fix python template
This patch set removes an unused import that is not python3 compatible.

Change-Id: I360989c8eb23065d8e655d4583eb97338244412d
Signed-off-by: Tin Lam <tin@irrational.io>
2019-05-26 21:43:19 +00:00
Jean-Philippe Evrard
5f5e988fb3 Point to OSH-images images
We now have a process for OSH-images image building,
using Zuul, so we should point the images by default to those
images, instead of pointing to stale images.

Without this, the osh-images build process is completely not
in use (and completely opaque to deployers), and updating the
osh-images process or patching its code has no impact on OSH.

This should fix it.

Change-Id: Ic00bd98c151669dc2485cd88e0e8c2ab05445959
2019-05-17 08:17:32 +00:00
Zuul
b69584bd65 Merge "Expose Anti-Affinity Weight Setting" 2019-05-16 17:17:03 +00:00
Roy Tang (rt7380)
85bd731562 Expose Anti-Affinity Weight Setting
This ps exposes the anti-affinity weight value, including
default, that will be consumed by the updated htk function.

Change-Id: Id8eb303674764ef8b0664f62040723aaf77e0a54
2019-05-14 17:04:52 -05:00
Zuul
9388c2ba5a Merge "Grafana: Add security context to chart and read-only-fs" 2019-05-13 21:45:01 +00:00
Meg Heisler
e1f2a3cf78 Fix broken network policy check/gate
This adds a basic egress policy to the charts run by the
network-policy check. A change was recently merged requiring
the eggress tag to be in the chart but did not add it, this
addresses that

Change-Id: I60669c9351db7854cba8c69723eb783a966d2a56
2019-05-10 05:55:22 +00:00
Rahul Khiyani
d8ca55c685 Logs format standardization for LMA component
Added file name, line number and function name to logging message format
for troubleshooting purpose
    - This change is related to Grafana's session-db-sync job

Change-Id: Iaadbedfda0fd9cd7fe4b5c09fc05cb6181c400d1
2019-05-06 21:13:27 +00:00