[8.x] [ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)

[kibanamachine]

Backport

This will backport the following commits from main to 8.x:

• [ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)

Questions ?

Please refer to the Backport tool documentation

APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com//issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n

\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n

\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n

\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n

\n\n- ⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu \nCo-authored-by: Cauê Marcondes <55978943+cauemarcondes@users.noreply.github.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:prev-minor","backport:prev-major","Team:obs-ux-infra_services","v9.1.0"],"title":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path","number":211822,"url":"https://github.com//pull/211822","mergeCommit":{"message":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)\n\nCloses #211783\nPart of https://github.com//issues/195857\n\n## Summary\n\nThis PR expands the logic to get the dashboard files based on the agent.\nWe have many different ways to ingest data so we want to add more\nmetrics dashboards to the APM metrics tab. The different ingest paths we\nhave:\nClassic APM Agent --> APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com//issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n

\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n

\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n

\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n

\n\n- ⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu \nCo-authored-by: Cauê Marcondes <55978943+cauemarcondes@users.noreply.github.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com//pull/211822","number":211822,"mergeCommit":{"message":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)\n\nCloses #211783\nPart of https://github.com//issues/195857\n\n## Summary\n\nThis PR expands the logic to get the dashboard files based on the agent.\nWe have many different ways to ingest data so we want to add more\nmetrics dashboards to the APM metrics tab. The different ingest paths we\nhave:\nClassic APM Agent --> APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com//issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n

\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n

\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n

\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n

\n\n- ⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu \nCo-authored-by: Cauê Marcondes <55978943+cauemarcondes@users.noreply.github.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0"}}]}] BACKPORT-->

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)