Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Livebooks 1.0 #267

Merged
merged 8 commits into from
Jan 29, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions livebooks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Livebook examples

This folder contains interactive livebook examples. To launch them you need to install livebook first.

## Installation

1. Install Livebook

It is recommended to install Livebook via command line ([see official installation guide](https://github.com/livebook-dev/livebook#escript)).

If livebook was installed directly from the official page, one should add `$PATH` variable to the Livebook environment:
![Setting path](./assets/path_set.png "Title")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this in guide? 1. in line 7 looks little bit weird

Binary file added livebooks/assets/path_set.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 13 additions & 0 deletions livebooks/audio_mixer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Membrane audio mixer

This livebook shows how to mix a beep sound into background music over a period of time.

To run the demo, [install Livebook](https://github.com/livebook-dev/livebook#escript) and open the `audio_mixer.livemd` file there.

## Copyright and License

Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

[![Software Mansion](https://docs.membrane.stream/static/logo/swm_logo_readme.png)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

Licensed under the [Apache License, Version 2.0](LICENSE)
22 changes: 11 additions & 11 deletions livebooks/audio_mixer/audio_mixer.livemd
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ File.cd(__DIR__)
Logger.configure(level: :error)

Mix.install([
{:membrane_core, "~> 0.11.2"},
{:membrane_audio_mix_plugin, "~> 0.12.0"},
{:membrane_file_plugin, "~> 0.13.0"},
{:membrane_mp3_mad_plugin, "~> 0.14.0"},
{:membrane_ffmpeg_swresample_plugin, "~> 0.16.1"},
{:membrane_aac_fdk_plugin, "~> 0.14.0"},
{:membrane_kino_plugin, github: "membraneframework-labs/membrane_kino_plugin"},
{:membrane_tee_plugin, "~> 0.10.1"}
{:membrane_core, "~> 1.0"},
{:membrane_audio_mix_plugin, "~> 0.16.0"},
{:membrane_file_plugin, "~> 0.16.0"},
{:membrane_mp3_mad_plugin, "~> 0.18.0"},
{:membrane_ffmpeg_swresample_plugin, "~> 0.19.0"},
{:membrane_aac_fdk_plugin, "~> 0.18.0"},
{:membrane_kino_plugin, github: "membraneframework-labs/membrane_kino_plugin", tag: "v0.3.1"},
{:membrane_tee_plugin, "~> 0.12.0"}
])
```

Expand Down Expand Up @@ -111,10 +111,10 @@ structure = beeps_split ++ [beep_audio_input, background_audio_input, mixer_outp
## Playing audio

```elixir
alias Membrane.RemoteControlled, as: RC
alias Membrane.RCPipeline, as: RC

pipeline = RC.Pipeline.start!()
RC.Pipeline.exec_actions(pipeline, spec: structure, playback: :playing)
pipeline = RC.start!()
RC.exec_actions(pipeline, spec: structure)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
RC.exec_actions(pipeline, spec: structure)
RC.exec_actions(pipeline, spec: spec)


kino
```
13 changes: 13 additions & 0 deletions livebooks/messages_source_and_sink/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Membrane messages source and sink

This livebook shows how to setup a simple pipeline and send messages through it.

To run the demo, [install Livebook](https://github.com/livebook-dev/livebook#escript) and open the `messages_source_and_sink.livemd` file there.

## Copyright and License

Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

[![Software Mansion](https://docs.membrane.stream/static/logo/swm_logo_readme.png)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

Licensed under the [Apache License, Version 2.0](LICENSE)
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ File.cd(__DIR__)
Logger.configure(level: :error)

Mix.install([
{:membrane_core, "~> 0.12.7"}
{:membrane_core, "~> 1.0"}
])
```

Expand All @@ -18,7 +18,7 @@ defmodule MessageSource do
require Membrane.Logger

def_output_pad(:output,
mode: :push,
flow_control: :push,
accepted_format: _any
)

Copy link
Member

@FelonEkonom FelonEkonom Jan 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the brackets from all pad definitions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did it, but Livebook does some kind of auto-formatting and adds these brackets every time you just open the file in Livebook.

Expand Down Expand Up @@ -75,7 +75,7 @@ end
defmodule MessageSink do
use Membrane.Sink

def_input_pad(:input, mode: :push, accepted_format: _any)
def_input_pad(:input, flow_control: :push, accepted_format: _any)
def_options(receiver: [description: "PID of the process that will
receive messages from the sink", spec: pid()])

Expand Down Expand Up @@ -111,7 +111,7 @@ defmodule MyPipeline do
end
end

{:ok, _supervisor, pipeline} = MyPipeline.start(receiver: self())
{:ok, _supervisor, pipeline} = Membrane.Pipeline.start(MyPipeline, receiver: self())
payloads = 1..10

Task.async(fn ->
Expand Down
13 changes: 13 additions & 0 deletions livebooks/playing_mp3_file/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Membrane playing mp3 file demo

This livebook shows how to load `MP3` audio from the file, transcode it to the `AAC` codec, and play it.

To run the demo, [install Livebook](https://github.com/livebook-dev/livebook#escript) and open the `playing_mp3_file.livemd` file there.

## Copyright and License

Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

[![Software Mansion](https://docs.membrane.stream/static/logo/swm_logo_readme.png)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

Licensed under the [Apache License, Version 2.0](LICENSE)
20 changes: 10 additions & 10 deletions livebooks/playing_mp3_file/playing_mp3_file.livemd
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ File.cd(__DIR__)
Logger.configure(level: :error)

Mix.install([
{:membrane_core, "~> 0.11.2"},
{:membrane_file_plugin, "~> 0.13.0"},
{:membrane_mp3_mad_plugin, "~> 0.14.0"},
{:membrane_ffmpeg_swresample_plugin, "~> 0.16.1"},
{:membrane_aac_fdk_plugin, "~> 0.14.0"},
{:membrane_kino_plugin, github: "membraneframework-labs/membrane_kino_plugin"}
{:membrane_core, "~> 1.0"},
{:membrane_file_plugin, "~> 0.16.0"},
{:membrane_mp3_mad_plugin, "~> 0.18.0"},
{:membrane_ffmpeg_swresample_plugin, "~> 0.19.0"},
{:membrane_aac_fdk_plugin, "~> 0.18.0"},
{:membrane_kino_plugin, github: "membraneframework-labs/membrane_kino_plugin", tag: "v0.3.1"}
])
```

Expand Down Expand Up @@ -54,7 +54,7 @@ alias Membrane.{
audio_path = "./assets/sample.mp3"
kino = Membrane.Kino.Player.new(audio: true)

structure =
spec =
child(:file_source, %File.Source{location: audio_path})
|> child(:decoder_mp3, MP3.MAD.Decoder)
|> child(:converter, %FFmpeg.SWResample.Converter{
Expand All @@ -73,10 +73,10 @@ structure =
Run pipeline:

```elixir
alias Membrane.RemoteControlled, as: RC
alias Membrane.RCPipeline, as: RC

pipeline = RC.Pipeline.start!()
RC.Pipeline.exec_actions(pipeline, spec: structure, playback: :playing)
pipeline = RC.start!()
RC.exec_actions(pipeline, spec: spec)

kino
```
15 changes: 15 additions & 0 deletions livebooks/rtmp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Membrane RMTP demo

Sender livebook shows how to download video and audio from the web using the `Hackney` plugin, and stream it via `RTMP` to the other, receiver livebook.

Receiver livebook shows how to receive `RTMP` stream mentioned above and play it in the livebook.

To run the demo, [install Livebook](https://github.com/livebook-dev/livebook#escript) and open both `rtmp_sender.livemd` and `rtmp_sender.livemd` files there.

## Copyright and License

Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

[![Software Mansion](https://docs.membrane.stream/static/logo/swm_logo_readme.png)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

Licensed under the [Apache License, Version 2.0](LICENSE)
21 changes: 11 additions & 10 deletions livebooks/rtmp/rtmp_receiver.livemd
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ File.cd(__DIR__)
Logger.configure(level: :error)

Mix.install([
{:membrane_kino_plugin, github: "membraneframework-labs/membrane_kino_plugin"},
{:membrane_core, "~>0.11.2"},
{:membrane_realtimer_plugin, "~>0.6.1"},
{:membrane_rtmp_plugin, "~>0.11.0"}
{:membrane_core, "~> 1.0"},
{:membrane_realtimer_plugin, "~> 0.9.0"},
{:membrane_rtmp_plugin, "~> 0.19.0"},
{:membrane_kino_plugin, github: "membraneframework-labs/membrane_kino_plugin", tag: "v0.3.1"}
])
```

Expand Down Expand Up @@ -45,7 +45,6 @@ defmodule RTMP.Receiver.Pipeline do
get_child(:source)
|> via_out(:audio)
|> child(:audio_parser, %Membrane.AAC.Parser{
in_encapsulation: :none,
out_encapsulation: :ADTS
})
|> via_in(:audio)
Expand All @@ -54,16 +53,17 @@ defmodule RTMP.Receiver.Pipeline do
playing_video =
get_child(:source)
|> via_out(:video)
|> child(:video_parser, %Membrane.H264.FFmpeg.Parser{
framerate: {25, 1}
|> child(:video_parser, %Membrane.H264.Parser{
generate_best_effort_timestamps: %{framerate: {25, 1}},
output_stream_structure: :annexb
})
|> via_in(:video)
|> get_child(:player)

player = child(:player, %Membrane.Kino.Player.Sink{kino: kino})

structure = [source, playing_audio, playing_video, player]
{[spec: structure, playback: :playing], %{}}
{[spec: structure], %{}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{[spec: structure], %{}}
{[spec: spec], %{}}

You can rename structure to spec in similar places as well

end

# Once the source initializes, we grant it the control over the tcp socket
Expand Down Expand Up @@ -133,7 +133,9 @@ defmodule RTMP.Receiver do
],
socket_handler: fn socket ->
# On new connection a pipeline is started
{:ok, _supervisor, pipeline} = RTMP.Receiver.Pipeline.start(socket: socket, kino: kino)
{:ok, _supervisor, pipeline} =
Membrane.Pipeline.start(RTMP.Receiver.Pipeline, socket: socket, kino: kino)

send(parent, {:pipeline_spawned, pipeline})
{:ok, pipeline}
end
Expand Down Expand Up @@ -171,6 +173,5 @@ port = 1942

kino = Membrane.Kino.Player.new(video: true, audio: true)
Kino.render(kino)

RTMP.Receiver.run(port: port, kino: kino)
```
30 changes: 15 additions & 15 deletions livebooks/rtmp/rtmp_sender.livemd
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ File.cd(__DIR__)
Logger.configure(level: :error)

Mix.install([
{:membrane_core, "~>0.11.2"},
{:membrane_realtimer_plugin, "~>0.6.1"},
{:membrane_hackney_plugin, "~>0.9.0"},
{:membrane_rtmp_plugin, "~>0.11.0"}
{:membrane_core, "~> 1.0"},
{:membrane_realtimer_plugin, "~> 0.9.0"},
{:membrane_hackney_plugin, "~> 0.11.0"},
{:membrane_rtmp_plugin, "~> 0.21.0"}
])
```

Expand Down Expand Up @@ -40,22 +40,20 @@ defmodule RTMP.Sender.Pipeline do
location: @video_url,
hackney_opts: [follow_redirect: true]
})
|> child(:video_parser, %Membrane.H264.FFmpeg.Parser{
framerate: {25, 1},
alignment: :au,
attach_nalus?: true,
skip_until_keyframe?: true
|> child(:video_parser, %Membrane.H264.Parser{
output_alignment: :au,
skip_until_keyframe: true,
generate_best_effort_timestamps: %{framerate: {25, 1}}
})
|> child(:video_realtimer, Membrane.Realtimer)
|> child(:video_payloader, Membrane.MP4.Payloader.H264)
|> child(:video_payloader, %Membrane.H264.Parser{output_stream_structure: :avc1})

audio_source =
child(:audio_source, %Membrane.Hackney.Source{
location: @audio_url,
hackney_opts: [follow_redirect: true]
})
|> child(:audio_parser, %Membrane.AAC.Parser{
in_encapsulation: :ADTS,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not a specialist, when it comes to AAC, but make sure, that it is safe, to remove this option

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In current version of acc parser this field no longer exists. Documentation says, correct acc stream format is supplied with the stream. And it works just fine

out_encapsulation: :ADTS
})
|> child(:audio_realtimer, Membrane.Realtimer)
Expand All @@ -68,15 +66,15 @@ defmodule RTMP.Sender.Pipeline do

structure = [
video_source
|> via_in(:video)
|> via_in(Pad.ref(:video, 0))
|> get_child(:rtmp_sink),
audio_source
|> via_in(:audio)
|> via_in(Pad.ref(:audio, 0))
|> get_child(:rtmp_sink),
rtmp_sink
]

{[spec: structure, playback: :playing], %{streams_to_end: 2}}
{[spec: structure], %{streams_to_end: 2}}
end

# The rest of the example module is only used for self-termination of the pipeline after processing finishes
Expand Down Expand Up @@ -116,7 +114,9 @@ defmodule RTMP.Sender do
end

defp start_tcp_client(destination_url) do
{:ok, _supervisor, pipeline} = RTMP.Sender.Pipeline.start(destination: destination_url)
{:ok, _supervisor, pipeline} =
Membrane.Pipeline.start(RTMP.Sender.Pipeline, destination: destination_url)

{:ok, pipeline}
end

Expand Down
13 changes: 13 additions & 0 deletions livebooks/soundwave/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Membrane Soundwave demo

This livebook example shows how to perform real-time soundwave plotting with the use of the [Membrane Framework](https://github.com/membraneframework) and [Vega-Lite](https://vega.github.io/vega-lite/).

To run the demo, [install Livebook](https://github.com/livebook-dev/livebook#escript) and open the `soundwave.livemd` file there.

## Copyright and License

Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

[![Software Mansion](https://docs.membrane.stream/static/logo/swm_logo_readme.png)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)

Licensed under the [Apache License, Version 2.0](LICENSE)
32 changes: 15 additions & 17 deletions livebooks/soundwave/soundwave.livemd
Original file line number Diff line number Diff line change
@@ -1,29 +1,27 @@
# Soundwave plotting example

## Introduction

This livebook example shows how to perform real-time soundwave plotting with the use of the [Membrane Framework](https://github.com/membraneframework) and [Vega-Lite](https://vega.github.io/vega-lite/).

By following that example you will learn how to read the audio from the microphone, how is audio represented, and how to create your custom Membrane element that plots the soundwave with the use of the elixir bindings to the Vega-Lite.

## Installation

You need to install the following `mix` dependencies:

```elixir
File.cd(__DIR__)
Logger.configure(level: :error)

Mix.install([
{:membrane_core, "~> 0.12.7"},
{:membrane_raw_audio_parser_plugin, "~> 0.2.0"},
{:membrane_portaudio_plugin, "~>0.16.0"},
{:vega_lite, "~> 0.1.7"},
{:kino_vega_lite, "~> 0.1.8"}
{:membrane_core, "~> 1.0"},
{:membrane_raw_audio_parser_plugin, "~> 0.4.0"},
{:membrane_portaudio_plugin, "~> 0.18.0"},
{:vega_lite, "~> 0.1.8"},
{:kino_vega_lite, "~> 0.1.11"}
])
```

Furthermore, you need to have `FFmpeg` installed. For installation details take a look [here](https://www.ffmpeg.org/).
## Introduction

This livebook example shows how to perform real-time soundwave plotting with the use of the [Membrane Framework](https://github.com/membraneframework) and [Vega-Lite](https://vega.github.io/vega-lite/).

By following that example you will learn how to read the audio from the microphone, how is audio represented, and how to create your custom Membrane element that plots the soundwave with the use of the elixir bindings to the Vega-Lite.

## Installation

You need to have `FFmpeg` installed. For installation details take a look [here](https://www.ffmpeg.org/).

## Soundwave plotting sink

Expand Down Expand Up @@ -172,7 +170,7 @@ defmodule Visualizer do
x =
(pts + (chunk_i * samples_per_point + sample_i) * sample_duration -
state.initial_pts)
|> Membrane.Time.round_to_milliseconds()
|> Membrane.Time.as_milliseconds(:round)

%{x: x, y: value}
end)
Expand Down
Binary file removed livebooks/video_compositor/res/input.h264
Binary file not shown.
Loading