Script for indexing micro benchmark results #128557
Conversation
idegtiarenko
added
:Delivery/Build
|
Pinging @elastic/es-delivery (Team:Delivery) |
| #!/bin/bash | ||
|
|
||
| METRICS_HOST=$(vault read -field=es_host /secret/performance/employees/cloud/esbench-metrics) | ||
| METRICS_INDEX_NAME="dummy-micro-benchmarks" |
There was a problem hiding this comment.
The index name is temporary and will be replaced with a data stream soon
There was a problem hiding this comment.
These are Infra Vault paths. We need to copy the necessary fields into esbench-repo Elasticsearch repo location in CI Vault. I suggest to use micro-benchmarks in the name of the secret. We will upate its contents (host, username, password) once this moves into "production". @breskeby Can you help with creating the necessary CI Vault secret?
|
I love it, but get someone who knows more about this to approve it for real. |
|
|
||
| jq -c '.[]' "benchmarks/build/result.json" | while read -r doc; do | ||
| curl -s -X POST "https://$METRICS_HOST/$METRICS_INDEX_NAME/_doc" \ | ||
| -u "$METRICS_USERNAME:$METRICS_PASSWORD" \ |
There was a problem hiding this comment.
Do we leak the credentials here to console log? @brianseeders
There was a problem hiding this comment.
It doesn't look like it, but the vault read commands above need to happen inside of .buildkite/hooks/pre-command. If they happen there, then they will get automatically redacted if they are accidentally printed.
Take a look at the other vault commands in that script, there's a few different ways to do it. I usually prefer introducing an env var like USE_ES_BENCH_CREDENTIALS. You can also do it based on something like the step label periodic-micro-benchmarks, but then it will break if you rename the step.
@gbanasiak @idegtiarenko I've copied those credentials to |
# Conflicts: # .buildkite/pipelines/periodic-micro-benchmarks.yml
This change adds a bash script that indexes micro benchmarks results into elasticsearch so that we could visualize them.