[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different i…

…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`) ![image](https://github.com/user-attachments/assets/caef88ea-2603-41ad-b815-f4c0c3647809) - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service) ![image](https://github.com/user-attachments/assets/4cb6cca3-240e-422b-9288-701ef080f9cc) - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> - ⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <sergi.romeu@elastic.co> Co-authored-by: Cauê Marcondes <55978943+cauemarcondes@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit f195570)
kibanamachine · Mar 6, 2025 · 4179f28 · 4179f28
1 parent 6fd617c
commit 4179f28
Show file tree

Hide file tree

Showing 17 changed files with 590 additions and 61 deletions.
diff --git a/src/platform/packages/shared/kbn-apm-synthtrace-client/src/lib/otel/index.ts b/src/platform/packages/shared/kbn-apm-synthtrace-client/src/lib/otel/index.ts
@@ -132,11 +132,13 @@ class Otel extends Serializable<OtelDocument> {
       },
       resource: {
         attributes: {
-          'agent.name': 'otlp',
+          'agent.name': 'opentelemetry/nodejs',
           'agent.version': '1.28.0',
           'service.instance.id': '89117ac1-0dbf-4488-9e17-4c2c3b76943a',
           'service.name': 'sendotlp-synth',
           'metricset.interval': '10m',
+          'telemetry.sdk.name': 'opentelemetry',
+          'telemetry.sdk.language': 'nodejs',
         },
         dropped_attributes_count: 0,
       },

diff --git a/src/platform/packages/shared/kbn-elastic-agent-utils/index.ts b/src/platform/packages/shared/kbn-elastic-agent-utils/index.ts
@@ -32,6 +32,10 @@ export {
   AGENT_NAMES,
 } from './src/agent_names';
 
+export { getIngestionPath } from './src/agent_ingestion_path';
+
+export { getSdkNameAndLanguage } from './src/agent_sdk_name_and_language';
+
 export type {
   ElasticAgentName,
   OpenTelemetryAgentName,

diff --git a/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_guards.test.ts b/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_guards.test.ts
@@ -13,6 +13,7 @@ import {
   isAndroidAgentName,
   isAWSLambdaAgentName,
   isAzureFunctionsAgentName,
+  isElasticAgentName,
   isIosAgentName,
   isJavaAgentName,
   isJRubyAgentName,
@@ -44,6 +45,17 @@ describe('Agents guards', () => {
     expect(isOpenTelemetryAgentName('not-an-agent')).toBe(false);
   });
 
+  it('isElasticAgentName should guard if the passed agent is an APM agent one.', () => {
+    expect(isElasticAgentName('nodejs')).toBe(true);
+    expect(isElasticAgentName('iOS/swift')).toBe(true);
+    expect(isElasticAgentName('java')).toBe(true);
+    expect(isElasticAgentName('rum-js')).toBe(true);
+    expect(isElasticAgentName('android/java')).toBe(true);
+    expect(isElasticAgentName('node-js')).toBe(false);
+    expect(isElasticAgentName('opentelemetry/nodejs/elastic')).toBe(false);
+    expect(isElasticAgentName('not-an-agent')).toBe(false);
+  });
+
   it('isJavaAgentName should guard if the passed agent is an Java one.', () => {
     expect(isJavaAgentName('java')).toBe(true);
     expect(isJavaAgentName('otlp/java')).toBe(true);

diff --git a/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_guards.ts b/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_guards.ts
@@ -9,6 +9,7 @@
 
 import {
   ANDROID_AGENT_NAMES,
+  ELASTIC_AGENT_NAMES,
   IOS_AGENT_NAMES,
   JAVA_AGENT_NAMES,
   OPEN_TELEMETRY_AGENT_NAMES,
@@ -17,13 +18,16 @@ import {
 
 import type {
   AndroidAgentName,
+  ElasticAgentName,
   IOSAgentName,
   JavaAgentName,
   OpenTelemetryAgentName,
   RumAgentName,
   ServerlessType,
 } from './agent_names';
 
+const ElasticAgentNamesSet = new Set(ELASTIC_AGENT_NAMES);
+
 export function getAgentName(
   agentName: string | null,
   telemetryAgentName: string | null,
@@ -57,6 +61,9 @@ export function isOpenTelemetryAgentName(agentName: string): agentName is OpenTe
   );
 }
 
+export const isElasticAgentName = (agentName: string): agentName is ElasticAgentName =>
+  ElasticAgentNamesSet.has(agentName as ElasticAgentName);
+
 export function isJavaAgentName(agentName?: string): agentName is JavaAgentName {
   return (
     hasOpenTelemetryPrefix(agentName, 'java') ||

diff --git a/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_ingestion_path.ts b/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_ingestion_path.ts
@@ -0,0 +1,11 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the "Elastic License
+ * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
+ * Public License v 1"; you may not use this file except in compliance with, at
+ * your election, the "Elastic License 2.0", the "GNU Affero General Public
+ * License v3.0 only", or the "Server Side Public License, v 1".
+ */
+
+export const getIngestionPath = (hasOpenTelemetryFields: boolean) =>
+  hasOpenTelemetryFields ? 'otel_native' : 'classic_apm';
diff --git a/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.test.ts b/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.test.ts
@@ -0,0 +1,61 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the "Elastic License
+ * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
+ * Public License v 1"; you may not use this file except in compliance with, at
+ * your election, the "Elastic License 2.0", the "GNU Affero General Public
+ * License v3.0 only", or the "Server Side Public License, v 1".
+ */
+
+import { getSdkNameAndLanguage } from './agent_sdk_name_and_language';
+
+describe('getSdkNameAndLanguage', () => {
+  it.each([
+    {
+      agentName: 'java',
+      result: { sdkName: 'apm', language: 'java' },
+    },
+    {
+      agentName: 'iOS/swift',
+      result: { sdkName: 'apm', language: 'iOS/swift' },
+    },
+    {
+      agentName: 'android/java',
+      result: { sdkName: 'apm', language: 'android/java' },
+    },
+    {
+      agentName: 'opentelemetry/java/test/elastic',
+      result: { sdkName: 'edot', language: 'java' },
+    },
+    {
+      agentName: 'opentelemetry/java/elastic',
+      result: { sdkName: 'edot', language: 'java' },
+    },
+    {
+      agentName: 'otlp/nodejs',
+      result: { sdkName: 'otel_other', language: 'nodejs' },
+    },
+    {
+      agentName: 'otlp',
+      result: { sdkName: 'otel_other', language: undefined },
+    },
+    {
+      agentName: 'test/test/test/something-else/elastic',
+      result: { sdkName: undefined, language: undefined },
+    },
+    {
+      agentName: 'test/java/test/something-else/',
+      result: { sdkName: undefined, language: undefined },
+    },
+    {
+      agentName: 'elastic',
+      result: { sdkName: undefined, language: undefined },
+    },
+    {
+      agentName: 'my-awesome-agent/otel',
+      result: { sdkName: undefined, language: undefined },
+    },
+  ])('for the agent name $agentName returns $result', ({ agentName, result }) => {
+    expect(getSdkNameAndLanguage(agentName)).toStrictEqual(result);
+  });
+});
diff --git a/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.ts b/src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.ts
@@ -0,0 +1,36 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the "Elastic License
+ * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
+ * Public License v 1"; you may not use this file except in compliance with, at
+ * your election, the "Elastic License 2.0", the "GNU Affero General Public
+ * License v3.0 only", or the "Server Side Public License, v 1".
+ */
+
+import { isElasticAgentName, isOpenTelemetryAgentName } from './agent_guards';
+
+interface SdkNameAndLanguage {
+  sdkName?: 'apm' | 'edot' | 'otel_other';
+  language?: string;
+}
+
+const LANGUAGE_INDEX = 1;
+
+export const getSdkNameAndLanguage = (agentName: string): SdkNameAndLanguage => {
+  if (isElasticAgentName(agentName)) {
+    return { sdkName: 'apm', language: agentName };
+  }
+  const agentNameParts = agentName.split('/');
+
+  if (isOpenTelemetryAgentName(agentName)) {
+    if (agentNameParts[agentNameParts.length - 1] === 'elastic') {
+      return { sdkName: 'edot', language: agentNameParts[LANGUAGE_INDEX] };
+    }
+    return {
+      sdkName: 'otel_other',
+      language: agentNameParts[LANGUAGE_INDEX],
+    };
+  }
+
+  return { sdkName: undefined, language: undefined };
+};
diff --git a/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/index.test.tsx b/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/index.test.tsx
@@ -0,0 +1,155 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+import type { CoreStart } from '@kbn/core/public';
+import { createKibanaReactContext } from '@kbn/kibana-react-plugin/public';
+import { render } from '@testing-library/react';
+import { createMemoryHistory } from 'history';
+import React from 'react';
+import type { ApmPluginContextValue } from '../../../context/apm_plugin/apm_plugin_context';
+import {
+  MockApmPluginContextWrapper,
+  mockApmPluginContextValue,
+} from '../../../context/apm_plugin/mock_apm_plugin_context';
+import * as useApmServiceContext from '../../../context/apm_service/use_apm_service_context';
+import type { ServiceEntitySummary } from '../../../context/apm_service/use_service_entity_summary_fetcher';
+import * as useApmDataViewHook from '../../../hooks/use_adhoc_apm_data_view';
+import { FETCH_STATUS } from '../../../hooks/use_fetcher';
+import { fromQuery } from '../../shared/links/url_helpers';
+import { Metrics } from '.';
+import type { DataView } from '@kbn/data-views-plugin/common';
+
+const KibanaReactContext = createKibanaReactContext({
+  settings: { client: { get: () => {} } },
+} as unknown as Partial<CoreStart>);
+
+function MetricsWithWrapper() {
+  jest
+    .spyOn(useApmDataViewHook, 'useAdHocApmDataView')
+    .mockReturnValue({ dataView: { id: 'id-1', name: 'apm-data-view' } as DataView });
+
+  const history = createMemoryHistory();
+  history.replace({
+    pathname: '/services/testServiceName/metrics',
+    search: fromQuery({
+      rangeFrom: 'now-15m',
+      rangeTo: 'now',
+    }),
+  });
+
+  return (
+    <KibanaReactContext.Provider>
+      <MockApmPluginContextWrapper
+        history={history}
+        value={mockApmPluginContextValue as unknown as ApmPluginContextValue}
+      >
+        <Metrics />
+      </MockApmPluginContextWrapper>
+    </KibanaReactContext.Provider>
+  );
+}
+
+describe('Metrics', () => {
+  describe('render the correct metrics content for', () => {
+    describe('APM agent / server service', () => {
+      beforeEach(() => {
+        jest.spyOn(useApmServiceContext, 'useApmServiceContext').mockReturnValue({
+          agentName: 'java',
+          serviceName: 'testServiceName',
+          transactionTypeStatus: FETCH_STATUS.SUCCESS,
+          transactionTypes: [],
+          fallbackToTransactions: true,
+          serviceAgentStatus: FETCH_STATUS.SUCCESS,
+          serviceEntitySummaryStatus: FETCH_STATUS.SUCCESS,
+          serviceEntitySummary: {
+            dataStreamTypes: ['metrics'],
+          } as unknown as ServiceEntitySummary,
+        });
+      });
+
+      it('shows java dashboard content', () => {
+        const result = render(<MetricsWithWrapper />);
+        // Check that the other content is not rendered as we don't have test id in the dashboard rendering component
+        const loadingBar = result.queryByRole('progressbar');
+        expect(loadingBar).toBeNull();
+        expect(result.queryByTestId('apmMetricsNoDashboardFound')).toBeNull();
+        expect(result.queryByTestId('apmAddApmCallout')).toBeNull();
+      });
+    });
+
+    describe('APM agent / EDOT sdk with dashboard', () => {
+      beforeEach(() => {
+        jest.spyOn(useApmServiceContext, 'useApmServiceContext').mockReturnValue({
+          agentName: 'opentelemetry/nodejs/elastic',
+          serviceName: 'testServiceName',
+          transactionTypeStatus: FETCH_STATUS.SUCCESS,
+          transactionTypes: [],
+          fallbackToTransactions: true,
+          serviceAgentStatus: FETCH_STATUS.SUCCESS,
+          serviceEntitySummaryStatus: FETCH_STATUS.SUCCESS,
+          serviceEntitySummary: {
+            dataStreamTypes: ['metrics'],
+          } as unknown as ServiceEntitySummary,
+        });
+      });
+
+      it('shows nodejs dashboard content', () => {
+        const result = render(<MetricsWithWrapper />);
+        // Check that the other content is not rendered as we don't have test id in the dashboard rendering component
+        const loadingBar = result.queryByRole('progressbar');
+        expect(loadingBar).toBeNull();
+        expect(result.queryByTestId('apmMetricsNoDashboardFound')).toBeNull();
+        expect(result.queryByTestId('apmAddApmCallout')).toBeNull();
+      });
+    });
+
+    describe('APM agent / otel sdk with no dashboard', () => {
+      beforeEach(() => {
+        jest.spyOn(useApmServiceContext, 'useApmServiceContext').mockReturnValue({
+          agentName: 'opentelemetry/go',
+          serviceName: 'testServiceName',
+          transactionTypeStatus: FETCH_STATUS.SUCCESS,
+          transactionTypes: [],
+          fallbackToTransactions: true,
+          serviceAgentStatus: FETCH_STATUS.SUCCESS,
+          serviceEntitySummaryStatus: FETCH_STATUS.SUCCESS,
+          serviceEntitySummary: {
+            dataStreamTypes: ['metrics'],
+          } as unknown as ServiceEntitySummary,
+        });
+      });
+
+      it('shows "no dashboard found" message', () => {
+        const result = render(<MetricsWithWrapper />);
+        const apmMetricsNoDashboardFound = result.getByTestId('apmMetricsNoDashboardFound');
+        expect(apmMetricsNoDashboardFound).toBeInTheDocument();
+      });
+    });
+
+    describe('Logs signals', () => {
+      beforeEach(() => {
+        jest.spyOn(useApmServiceContext, 'useApmServiceContext').mockReturnValue({
+          agentName: 'java',
+          serviceName: 'testServiceName',
+          transactionTypeStatus: FETCH_STATUS.SUCCESS,
+          transactionTypes: [],
+          fallbackToTransactions: true,
+          serviceAgentStatus: FETCH_STATUS.SUCCESS,
+          serviceEntitySummaryStatus: FETCH_STATUS.SUCCESS,
+          serviceEntitySummary: {
+            dataStreamTypes: ['logs'],
+          } as unknown as ServiceEntitySummary,
+        });
+      });
+
+      it('shows service from logs metrics content', () => {
+        const result = render(<MetricsWithWrapper />);
+        const apmAddApmCallout = result.getByTestId('apmAddApmCallout');
+        expect(apmAddApmCallout).toBeInTheDocument();
+      });
+    });
+  });
+});