OpenTelemetry - root span duration lower than process span duration

[yann-combarnous]

We setup OpenTelemetry for FastStream, and observe one strange behavior.

It seems that for each message, FastStream creates 2 spans:

• <topic_name> create parent span
• process child span

While process span represents the entire duration of the handler function for the message, the parent span is almost closing a few microseconds after start, maybe just after the ACK and before calling handler function. Example:

• <topic_name> create parent span starts at time X, finishes at X + 50us
• process child span starts at time X, starts at X + 50us, finishes at X + 250ms

This creates strange rendering in some viewers, would there be any way to have <topic_name> create end just after process span?

Rendering example (= represent span duration):

= <topic_name> create
================================================================= process

[yann-combarnous]

After looking back in the documentation and the example screenshot, it seems to be the expected behavior.

I will raise a support ticket with Datadog and see what they say, as they present trace duration as duration of first span only.

[draincoder]

Hi, root span in asynchronous services does not make sense to extend the entire length of the trace, since we do not know where the endpoint of event processing will be, in this regard, tracing asynchronous applications differs from synchronous applications, where we immediately expect a response. In this case, the create root span is just a starting point.

[bradydean]

Is there any otel documentation around this @draincoder?

I'm wondering why there needs to be a root span that ends in a few microseconds at all. An event makes more sense - faststream/kafka receiving an event is a point in time operation.

[draincoder]

In this case, the create root span is just a starting point.

If the consumer receives a message without a trace context in the headers, we create a create root span to indicate the beginning of the message processing chain.

Almost a year ago, this was reflected in the experimental documentation of OpenTelemetry for messaging systems. Now it has been updated and create has a different meaning there, other details have also changed.

Other team members and I are discussing the stabilization and improvement of the tracing implementation and are open to suggestions. Do you have any ideas?

[bradydean]

I think I'd just be fine with the processing span but whatever matches the otel spec is fine too.