Skip to content

Commit 3c63204

Browse files
authored
Improve dashboard upload script (opea-project#625)
Single label is enough, and Grafana does not need to be restarted, one just needs to wait enough for it to check+process new configMaps, *and* reload of Grafana page. Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
1 parent fe94599 commit 3c63204

File tree

2 files changed

+10
-26
lines changed

2 files changed

+10
-26
lines changed

kubernetes-addons/Observability/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -122,16 +122,16 @@ You can either:
122122

123123
- Import them manually to Grafana,
124124
- Use [`update-dashboards.sh`](./update-dashboards.sh) script to add them to Kubernetes as (more persistent) Grafana dashboard `configMap`s
125-
- Script uses `$USER-<file name>` as dashboard `configMap` names, overwrites any pre-existing `configMap` with the same name, and _restarts Grafana_ so that it notices `configMap` updates
125+
- Script uses `$USER-<file name>` as dashboard `configMap` names, and overwrites any pre-existing `configMap` with the same name
126126
- Or create your own dashboards based on them
127127

128128
When dashboard is imported to Grafana, you can directly save changes to it, but such dashboards go away if Grafana is removed / re-installed. When dashboard is in `configMap`, Grafana saves its changes to a (selected) file, but you need to re-apply those files to Kubernetes with the script, for your changes to be there when that Grafana dashboard page is reloaded in browser.
129129

130130
Gotchas for dashboard `configMap` script usage:
131131

132132
- If you change dashboard file name, you need to change also its 'uid' field (at end of the file), otherwise Grafana will see multiple `configMap`s for the same dashboard ID
133-
- If there's no `uid` specified for the dashboard, Grafana will generate one on `configMap` load. Meaning that dashboard ID, and Grafana URL to it, will change on every reload
134-
- Script assumes Prometheus / Grafana to be installed according to above instructions. If not, list of `labels` within script need to be updated to match Prometheus / Grafana installation
133+
- If there's no `uid` specified for the dashboard, Grafana will generate one on `configMap` load. Meaning that dashboard ID, and Grafana URL to it, will change on every reload
134+
- Script assumes default Prometheus / Grafana install (`monitoring` namespace, `grafana_dasboard=1` label identifying dashboard `configMap`s)
135135

136136
NOTE: Services provide metrics only after they have processed at least one query, before that dashboards can be empty!
137137

kubernetes-addons/Observability/update-dashboards.sh

Lines changed: 7 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -15,14 +15,15 @@ selector="app.kubernetes.io/name=grafana"
1515
grafana="kubectl -n $ns get pod --selector $selector --field-selector=status.phase=Running -o name"
1616

1717
# Labels needed in configMap to get (Helm installed) Grafana to load it as dashboard
18-
labels="grafana_dashboard=1 release=prometheus-stack app=kube-prometheus-stack-grafana"
18+
labels="grafana_dashboard=1"
1919

2020
error_exit ()
2121
{
2222
name=${0##*/}
2323
echo
24-
echo "Create/update dashboards specified in given JSON files to Grafana."
25-
echo "Names for the created configMaps will be in '$USER-<filename>' format."
24+
echo "Create/update dashboards specified in given JSON files for Grafana."
25+
echo "Names for the created configMaps will be in '$USER-<filename>' format"
26+
echo "and they go to (Grafana) '$ns' namespace."
2627
echo
2728
echo "usage: $name <JSON files>"
2829
echo
@@ -44,11 +45,12 @@ for file in "$@"; do
4445
if [ ! -f "$file" ]; then
4546
error_exit "JSON file '$file' does not exist"
4647
fi
47-
# Dashboard 'uid' is optional, but it should have a title...
48+
# Dashboard 'uid' is optional as Grafana can generate one...
4849
uid=$(jq .uid "$file" | tail -1)
4950
if [ -z "$uid" ]; then
5051
error_exit "'$file' dashboard has invalid JSON"
5152
fi
53+
# ...but it should have a title.
5254
title=$(jq .title "$file" | tail -1)
5355
if [ "$title" = "null" ]; then
5456
error_exit "'$file' dashboard has no 'title' field"
@@ -103,22 +105,4 @@ done
103105

104106
rm $tmp
105107

106-
echo
107-
echo "Restarting Grafana so that it notices updated dashboards..."
108-
pod=$($grafana)
109-
echo "kubectl -n $ns delete $pod"
110-
kubectl -n "$ns" delete "$pod"
111-
112-
echo
113-
echo "Waiting until new Grafana instance is running..."
114-
while true; do
115-
sleep 2
116-
pod=$($grafana)
117-
if [ -n "$pod" ]; then
118-
break
119-
fi
120-
done
121-
echo "kubectl -n $ns wait $pod --for=condition=Ready"
122-
kubectl -n "$ns" wait "$pod" --for=condition=Ready
123-
124-
echo "DONE!"
108+
echo "DONE! => Dashboard(s) should appear in Grafana after 1 min wait *and* Grafana page reload."

0 commit comments

Comments
 (0)