question: applying user.id when using opentelemetry api with Elastic

[peterabbott]

We use opentelemetry API to capture our trace data. Have been attempting to apply user context where previously we would have used setUserContext, but not getting anything coming through when checking through Kibana. It always comes through as a label.

After reading through the code and finding some older tickets. Even tried different casing.

 span.setAttributes({
                  [SemanticAttributes.ENDUSER_ID]: userHash,
                  ['user.id']: userHash,
                  ['user.Id']: userHash,
                  ['User.Id']: userHash,
                  ['_user.id']: userHash,
                  ['user.Hash']: userHash,
                  ['user.name']: userHash,
                  ['userId']: userHash,
                  ['UserId']: userHash,
                  ['id']: userHash
                })
              }

Any thoughts or tips? For reasons I can't go into our stack is still on 7.14 (trying to upgrade) so it may very well be the version we are on.

[trentm]

Hi @peterabbott,

First, to clarify:

• Are you using this APM agent (npm package elastic-apm-node) with its OpenTelemetry Bridge (https://www.elastic.co/guide/en/apm/agent/nodejs/master/opentelemetry-bridge.html) which supports calling the @opentelemetry/api?
• Or, are you using the OpenTelemetry SDK packages directly? (E.g. @opentelemetry/sdk-node, @opentelemetry/instrumentation, @opentelemetry/instrumentation-*)

For now, I'm assuming the latter.

For reasons I can't go into our stack is still on 7.14 (trying to upgrade) so it may very well be the version we are on.

I believe that may be part of it. My limited understanding is that APM server version 7.16 is when some more effort was put into translating some OTel span attributes into our span/transaction data model. My understanding is based on when support was added to support our OpenTelemetry Bridge work that I mentioned above. Its spec includes this section that refers to optional fallback handling for APM server versions earlier than 7.16: https://github.com/elastic/apm/blob/main/specs/agents/tracing-api-otel.md#attributes-mapping

It always comes through as a label.

Unfortunately, having said that, I think even with a more recent version -- perhaps you've tried a small test Elastic deployment on cloud.elastic.co? -- any user.* span attributes will still come through as labels.

Current efforts in mapping fields are in merging Elastic's ECS data specs (e.g. here are the user.* fields: https://www.elastic.co/guide/en/ecs/current/ecs-user.html) into OpenTelemetry's semantic conventions. This is a long process. I don't see anything user-related in the current OpenTelemetry semconv: https://opentelemetry.io/docs/specs/semconv/general/trace/

The current code (again my possibly incomplete understanding) for translating OTLP fields into Elastic's documents is here: https://github.com/elastic/apm-data/tree/main/input/otlp/
I don't see any special handling of user.* fields there, hence I expect them to end up as labels.

Are you wanting user.id as is for your own queries and dashboards, or is this breaking/limiting something in the curated APM app in Kibana?

You might have more luck asking in the apm-server repo, to get more authoritative answers on the translations of OTel span attributes it is doing.

[peterabbott]

thanks for the response. It was with both scenarios using the elastic-apm-node directrly and more recently the @opentelemetry api's.

After poking around the agent code, wondered if I was doing something wrong or not. Sounds like there is a combination of things working against me here, was really wanting to popluate the User section when looking via Kibana but querying by the label is just as easy.

I'd bee through that attribute mapping code many times, helped me discover I can set type attribute on the OTel root span to change the Transaction Type that shows up on the APM dashboard side.

Thanks for the detailed response.