Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Telemetry (#905) #918

Merged
merged 46 commits into from
Feb 6, 2025
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
4fcfcb2
Telemetry refresh and element/pipeline telemetry testing for init and…
wende Dec 5, 2024
328d30a
Tests async false. Telemetry captured other tests' processes
wende Dec 5, 2024
e599fcb
Unnecessary code trim
wende Dec 5, 2024
e60b127
Telemetry spans for init and setup
wende Dec 6, 2024
42f099e
Telemetry for spans, metrics and events for elements and pipelines
wende Dec 12, 2024
56890e0
Report playing and links
wende Dec 16, 2024
3cb267e
CQ and cleanup
wende Dec 16, 2024
7a69d1e
Cleanup of old telemetry and more universal method of measuring spans…
wende Jan 7, 2025
a77d6c0
Tests for handlers throwing exceptions and proper santization of comp…
wende Jan 9, 2025
3861859
Proper naming, dead code cleanup, no unnecessary optimization, crash …
wende Jan 13, 2025
66eea5a
CQ
wende Jan 16, 2025
0e95d43
Comments by @FelonEkonom adressed
wende Jan 20, 2025
f6b5ce1
Comments by @FelonEkonom adressed
wende Jan 24, 2025
db19995
Old metrics revamped and adjusted to new telemetry implementation
wende Jan 24, 2025
a442758
Small changes to Telemetry.report_buffer/2
wende Jan 27, 2025
f970d20
Legacy telemetry revived and rerouted based on config usage
wende Jan 28, 2025
2b1a474
Tests and documentation for measured events and their metadata
wende Jan 28, 2025
b00b8c9
Final touches
wende Jan 28, 2025
e508240
Update lib/membrane/telemetry.ex
wende Jan 30, 2025
9c65c71
Update lib/membrane/telemetry.ex
wende Jan 30, 2025
d53708b
Update lib/membrane/telemetry.ex
wende Jan 30, 2025
ec80e2a
PR and CI Comments addressed
wende Jan 30, 2025
f910961
Credo changes
wende Jan 30, 2025
083f65e
Unnecessary element and bin state fields removed
wende Jan 30, 2025
95bdab1
Credo satisfied
wende Jan 30, 2025
c905ec5
Changed component context and added tests reflecting the changes
wende Jan 30, 2025
b1a9627
LegacyTelemetry sometimes didn't work properly
wende Jan 30, 2025
8a01d9b
Dialyzer fix
wende Jan 30, 2025
fe9e359
Dialyzer satisfied
wende Jan 31, 2025
6c7b091
Appeasing the dialyzer
wende Jan 31, 2025
4bc4c25
Credo
wende Jan 31, 2025
d7f083a
Context instead of state
wende Jan 31, 2025
66cdefd
Simpler(?) telemetry private API
wende Feb 3, 2025
24f6cb6
Documentation of metadata passed to and from telemetry modules
wende Feb 3, 2025
7041bf9
Satisfy FilterAggregator.Context
wende Feb 3, 2025
72ca1be
Documentation of metadata passed to and from telemetry modules
wende Feb 3, 2025
fe86693
Event -> Datapoint
wende Feb 3, 2025
17dfe27
Update lib/membrane/telemetry.ex
wende Feb 4, 2025
cf8efa1
Comments addressed
wende Feb 4, 2025
fe9d0eb
Track callback handler name
wende Feb 4, 2025
70cc188
Config cleanup
wende Feb 4, 2025
e846d7a
Faulty f+r
wende Feb 5, 2025
9583a5d
Better public API
wende Feb 6, 2025
9894437
Update lib/membrane/telemetry.ex
wende Feb 6, 2025
94ba04a
Documentation changes
wende Feb 6, 2025
dc707ac
faulty merge
wende Feb 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/config.exs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ if config_env() == :test do
bin: :all,
pipeline: :all
],
events: :all
datapoints: :all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, this enables the old, deprecated metrics. Why do we add a new config key for a deprecated feature?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not for deprecated metrics. It's for those that were renewed. So:
link, buffer, queue_len, stream_format, event, store and take (no bitrate, demand, init or terminate)

And it's only for those to be tested


# config to test legacy version of telemetry
# config :membrane_core, :telemetry_flags, [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove the legacy config

Expand Down
6 changes: 4 additions & 2 deletions lib/membrane/bin/callback_context.ex
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,15 @@ defmodule Membrane.Bin.CallbackContext do
`c:Membrane.Bin.handle_crash_group_down/3`.
"""
@type t :: %{
:children => %{Membrane.Child.name() => Membrane.ChildEntry.t()},
:clock => Membrane.Clock.t(),
:parent_clock => Membrane.Clock.t(),
:pads => %{Membrane.Pad.ref() => Membrane.Bin.PadData.t()},
:module => module(),
:name => Membrane.Bin.name(),
:children => %{Membrane.Child.name() => Membrane.ChildEntry.t()},
:pads => %{Membrane.Pad.ref() => Membrane.Bin.PadData.t()},
:playback => Membrane.Playback.t(),
:resource_guard => Membrane.ResourceGuard.t(),
:setup_incomplete? => boolean(),
:utility_supervisor => Membrane.UtilitySupervisor.t(),
optional(:pad_options) => map(),
optional(:members) => [Membrane.Child.name()],
Expand Down
4 changes: 3 additions & 1 deletion lib/membrane/core/bin/callback_context.ex
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,15 @@ defmodule Membrane.Core.Bin.CallbackContext do
def from_state(state, optional_fields \\ []) do
Map.new(optional_fields)
|> Map.merge(%{
children: state.children,
clock: state.synchronization.clock,
parent_clock: state.synchronization.parent_clock,
pads: state.pads_data,
module: state.module,
name: state.name,
children: state.children,
playback: state.playback,
resource_guard: state.resource_guard,
setup_incomplete?: state.setup_incomplete?,
utility_supervisor: state.subprocess_supervisor
})
end
Expand Down
90 changes: 49 additions & 41 deletions lib/membrane/core/callback_handler.ex
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ defmodule Membrane.Core.CallbackHandler do
use Bunch

alias Membrane.CallbackError
alias Membrane.ComponentPath

require Membrane.Logger
require Membrane.Core.Telemetry, as: Telemetry
Expand Down Expand Up @@ -126,7 +127,7 @@ defmodule Membrane.Core.CallbackHandler do
end

@spec exec_callback(callback :: atom, args :: list, handler_params, state) ::
{list, internal_state}
{list, internal_state} | no_return()
defp exec_callback(
callback,
args,
Expand All @@ -136,50 +137,57 @@ defmodule Membrane.Core.CallbackHandler do
context = context_fun.(state)
args = args ++ [context, internal_state]

callback_result =
try do
Telemetry.component_span(
state.__struct__,
callback,
fn ->
res = {_actions, new_internal_state} = apply(module, callback, args)

Telemetry.state_result(res, %{
args: args,
module: module,
internal_state_before: internal_state,
internal_state_after: new_internal_state,
component_context: context
})
try do
fn ->
apply(module, callback, args)
|> validate_callback_result!(module, callback)
end
|> report_telemetry(callback, args, state, context)
Copy link
Member

@FelonEkonom FelonEkonom Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API of Telemetry module should allow on sth like

Suggested change
|> report_telemetry(callback, args, state, context)
|> Telemetry.track_callback_handler(callback, args, context, state)

and the creation of event metadata should be fully hidden inside Telemetry module

rescue
e in UndefinedFunctionError ->
_ignored =
with %{module: ^module, function: ^callback, arity: arity} <- e do
reraise CallbackError,
[
kind: :not_implemented,
callback: {module, callback},
arity: arity,
args: args
],
__STACKTRACE__
end
)
rescue
e in UndefinedFunctionError ->
_ignored =
with %{module: ^module, function: ^callback, arity: arity} <- e do
reraise CallbackError,
[
kind: :not_implemented,
callback: {module, callback},
arity: arity,
args: args
],
__STACKTRACE__
end

reraise e, __STACKTRACE__
end
reraise e, __STACKTRACE__
end
end

case callback_result do
{actions, _state} when is_list(actions) ->
callback_result
defp validate_callback_result!({actions, _state} = callback_result, _module, _cb)
when is_list(actions) do
callback_result
end

_result ->
raise CallbackError,
kind: :bad_return,
callback: {module, callback},
value: callback_result
end
defp validate_callback_result!(callback_result, module, callback) do
raise CallbackError,
kind: :bad_return,
callback: {module, callback},
value: callback_result
end

defp report_telemetry(f, callback, args, state, context) do
Telemetry.span_component_callback(
f,
state.__struct__,
callback,
%{
callback: callback,
callback_args: args,
callback_context: context,
component_path: ComponentPath.get(),
component_type: state.module,
internal_state_before: state.internal_state,
internal_state_after: nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move creating this map into Telemetry.span_component_callback/4

)
end

@spec handle_callback_result(
Expand Down
2 changes: 2 additions & 0 deletions lib/membrane/core/element/callback_context.ex
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@ defmodule Membrane.Core.Element.CallbackContext do
pads: state.pads_data,
clock: state.synchronization.clock,
parent_clock: state.synchronization.parent_clock,
module: state.module,
name: state.name,
playback: state.playback,
resource_guard: state.resource_guard,
setup_incomplete?: state.setup_incomplete?,
utility_supervisor: state.subprocess_supervisor
})
end
Expand Down
2 changes: 2 additions & 0 deletions lib/membrane/core/filter_aggregator/context.ex
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,14 @@ defmodule Membrane.Core.FilterAggregator.Context do
|> Map.new(fn {name, description} -> {name, build_pad_data(description)} end)

%{
module: module,
pads: pads_data,
clock: nil,
name: name,
parent_clock: nil,
playback: :stopped,
resource_guard: agg_ctx.resource_guard,
setup_incomplete?: true,
utility_supervisor: agg_ctx.utility_supervisor
}
end
Expand Down
2 changes: 2 additions & 0 deletions lib/membrane/core/pipeline/callback_context.ex
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,10 @@ defmodule Membrane.Core.Pipeline.CallbackContext do
|> Map.merge(%{
clock: state.synchronization.clock_proxy,
children: state.children,
module: state.module,
playback: state.playback,
resource_guard: state.resource_guard,
setup_incomplete?: state.setup_incomplete?,
utility_supervisor: state.subprocess_supervisor
})
end
Expand Down
Loading
Loading