Merge pull request #268 from membraneframework/livebooks-1.0

Livebooks from membrane guide

.gitignore

@@ -0,0 +1 @@
+.DS_Store

README.md

@@ -12,13 +12,20 @@ In the subdirectories of this repository you can find some examples of using the
 - [rtp_to_hls](https://github.com/membraneframework/membrane_demo/tree/master/rtp_to_hls) - receiving RTP stream and broadcasting it via HLS
 - [rtsp_to_hls](https://github.com/membraneframework/membrane_demo/tree/master/rtsp_to_hls) - receiving RTSP stream and converting it to HLS
 - [video_mixer](https://github.com/membraneframework/membrane_demo/tree/master/video_mixer) - how to mix audio and video files
-- [speech_to_text](https://github.com/membraneframework/membrane_demo/tree/master/speech_to_text) - real-time speech recognition using [Whisper](https://github.com/openai/whisper) in [Livebook]
 - [webrtc_to_hls](https://github.com/jellyfish-dev/membrane_rtc_engine/tree/master/examples/webrtc_to_hls) - converting WebRTC stream into HLS
 - [webrtc_videoroom](https://github.com/jellyfish-dev/membrane_rtc_engine/tree/master/examples/webrtc_videoroom) - basic example of [Membrane RTC Engine](https://github.com/jellyfish-dev/membrane_rtc_engine.git). It's as simple as possible just to show you how to use our API.
-- 
+
+Also there are some livebook examples located in [livebooks](https://github.com/membraneframework/membrane_demo/tree/master/livebooks) directory:
+
+- [speech_to_text](https://github.com/membraneframework/membrane_demo/tree/master/livebooks/speech_to_text) - real-time speech recognition using [Whisper](https://github.com/openai/whisper) in [Livebook]
+- [audio_mixer](https://github.com/membraneframework/membrane_demo/tree/master/livebooks/audio_mixer) - mix a beep sound into background music
+- [messages_source_and_sink](https://github.com/membraneframework/membrane_demo/tree/master/livebooks/messages_source_and_sink) - setup a simple pipeline and send messages through it
+- [playing_mp3_file](https://github.com/membraneframework/membrane_demo/tree/master/livebooks/playing_mp3_file) - read mp3 file, transcode to acc and play
+- [rtmp](https://github.com/membraneframework/membrane_demo/tree/master/livebooks/rtmp) - send and receive `RTMP` stream
+- [soundwave](https://github.com/membraneframework/membrane_demo/tree/master/livebooks/soundwave) - plot live audio amplitude on a graph
 ## Copyright and License
 
-Copyright 2018, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
+Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
 
 [![Software Mansion](https://logo.swmansion.com/logo?color=white&variant=desktop&width=200&tag=membrane-github)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
 

livebooks/README.md

@@ -0,0 +1,10 @@
+# Livebook examples 
+
+This folder contains interactive livebook examples. To launch them you need to install livebook first.
+
+## Installation
+
+It is recommended to install Livebook via command line ([see official installation guide](https://github.com/livebook-dev/livebook#escript)). 
+
+If livebook was installed directly from the official page, one should add `$PATH` variable to the Livebook environment:
+![Setting path](./assets/path_set.png "Title")

livebooks/audio_mixer/README.md

@@ -0,0 +1,13 @@
+# Membrane audio mixer
+
+This livebook shows how to mix a beep sound into background music over a period of time.
+
+To run the demo, [install Livebook](https://github.com/livebook-dev/livebook#escript) and open the `audio_mixer.livemd` file there.
+
+## Copyright and License
+
+Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
+
+[![Software Mansion](https://docs.membrane.stream/static/logo/swm_logo_readme.png)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
+
+Licensed under the [Apache License, Version 2.0](LICENSE)

livebooks/audio_mixer/audio_mixer.livemd

@@ -0,0 +1,120 @@
+# Mixing audio files
+
+```elixir
+File.cd(__DIR__)
+Logger.configure(level: :error)
+
+Mix.install([
+  {:membrane_core, "~> 1.0"},
+  {:membrane_audio_mix_plugin, "~> 0.16.0"},
+  {:membrane_file_plugin, "~> 0.16.0"},
+  {:membrane_mp3_mad_plugin, "~> 0.18.0"},
+  {:membrane_ffmpeg_swresample_plugin, "~> 0.19.0"},
+  {:membrane_aac_fdk_plugin, "~> 0.18.0"},
+  {:membrane_kino_plugin, github: "membraneframework-labs/membrane_kino_plugin", tag: "v0.3.1"},
+  {:membrane_tee_plugin, "~> 0.12.0"}
+])
+```
+
+## Installation
+
+To run this demo one needs to install native dependencies:
+
+1. [MP3 MAD](https://github.com/membraneframework/membrane_mp3_mad_plugin/tree/v0.14.0#installation)
+2. [AAC FDK](https://github.com/membraneframework/membrane_aac_fdk_plugin#installation)
+3. [SWResample FFmpeg](https://github.com/membraneframework/membrane_ffmpeg_swresample_plugin#installation)
+
+## Description
+
+This is an example of mixing multiple short "beep" sound into background music, one by one, every second.
+
+## Pipeline definition
+
+Define all constants.
+
+```elixir
+n_beeps = 30
+beep_filepath = "./assets/beep.aac"
+background_filepath = "./assets/sample_music_short.mp3"
+:ok
+```
+
+The file's "beep" sound input is decoded from `AAC` and split into separate inputs using the `Tee` element. These inputs are then filled into the mixer with corresponding time offsets.
+
+```elixir
+import Membrane.ChildrenSpec
+
+alias Membrane.{File, AAC, Tee, Time}
+
+beep_audio_input =
+  child({:file_source, :beep}, %File.Source{location: beep_filepath})
+  |> child({:decoder_aac, :beep}, AAC.FDK.Decoder)
+  |> child(:beeps, Tee.PushOutput)
+
+beeps_split =
+  for i <- 1..n_beeps do
+    get_child(:beeps)
+    |> via_in(:input, options: [offset: Time.seconds(i)])
+    |> get_child(:mixer)
+  end
+
+:ok
+```
+
+The background music is loaded from a file and then decoded from the `MP3` format to the appropriate `Raw Audio` format.
+
+All mixer inputs must be of the same format.
+
+```elixir
+import Membrane.ChildrenSpec
+
+alias Membrane.{File, RawAudio, MP3}
+alias Membrane.FFmpeg.SWResample.Converter
+
+background_audio_input =
+  child(:file_source, %File.Source{location: background_filepath})
+  |> child(:decoder_mp3, MP3.MAD.Decoder)
+  |> child(:converter, %Converter{
+    input_stream_format: %RawAudio{channels: 2, sample_format: :s24le, sample_rate: 44_100},
+    output_stream_format: %RawAudio{channels: 1, sample_format: :s16le, sample_rate: 44_100}
+  })
+  |> get_child(:mixer)
+
+:ok
+```
+
+Mixer is created and directly connected to audio input of the player.
+
+```elixir
+import Membrane.ChildrenSpec
+
+alias Membrane.{AudioMixer, AAC, Kino}
+
+kino = Membrane.Kino.Player.new(audio: true)
+
+mixer_output =
+  child(:mixer, Membrane.AudioMixer)
+  |> child(:encoder_aac, AAC.FDK.Encoder)
+  |> via_in(:audio)
+  |> child(:player, %Kino.Player.Sink{kino: kino})
+
+:ok
+```
+
+Whole pipeline structure.
+
+```elixir
+spec = beeps_split ++ [beep_audio_input, background_audio_input, mixer_output]
+:ok
+```
+
+## Playing audio
+
+```elixir
+alias Membrane.RCPipeline, as: RC
+
+pipeline = RC.start!()
+RC.exec_actions(pipeline, spec: spec)
+
+kino
+```

livebooks/messages_source_and_sink/README.md

@@ -0,0 +1,13 @@
+# Membrane messages source and sink
+
+This livebook shows how to setup a simple pipeline and send messages through it.
+
+To run the demo, [install Livebook](https://github.com/livebook-dev/livebook#escript) and open the `messages_source_and_sink.livemd` file there.
+
+## Copyright and License
+
+Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
+
+[![Software Mansion](https://docs.membrane.stream/static/logo/swm_logo_readme.png)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
+
+Licensed under the [Apache License, Version 2.0](LICENSE)

livebooks/messages_source_and_sink/messages_source_and_sink.livemd

@@ -0,0 +1,140 @@
+# Messages source and sink
+
+```elixir
+File.cd(__DIR__)
+Logger.configure(level: :error)
+
+Mix.install([
+  {:membrane_core, "~> 1.0"}
+])
+```
+
+## Erlang messages driven source
+
+```elixir
+defmodule MessageSource do
+  use Membrane.Source
+
+  require Membrane.Logger
+
+  def_output_pad :output,
+    flow_control: :push,
+    accepted_format: _any
+
+  def_options register_name: [
+                description: "The name under which the element's process will be registered",
+                spec: atom()
+              ]
+
+
+  @impl true
+  def handle_init(_ctx, opts) do
+    Process.register(self(), opts.register_name)
+    {[], %{buffered: []}}
+  end
+
+  @impl true
+  def handle_playing(_ctx, state) do
+    {actions, state} = send_buffers(state)
+    {[stream_format: {:output, %Membrane.RemoteStream{type: :bytestream}}] ++ actions, state}
+  end
+
+  @impl true
+  def handle_info({:message, message}, ctx, state) do
+    state = %{state | buffered: state.buffered ++ [message]}
+
+    if ctx.playback == :playing do
+      send_buffers(state)
+    else
+      {[], state}
+    end
+  end
+
+  @impl true
+  def handle_info(msg, _ctx, state) do
+    Membrane.Logger.warning("Unknown message received: #{inspect(msg)}")
+    {[], state}
+  end
+
+  defp send_buffers(state) do
+    actions =
+      Enum.map(state.buffered, fn message ->
+        {:buffer, {:output, %Membrane.Buffer{payload: message}}}
+      end)
+
+    {actions, %{state | buffered: []}}
+  end
+end
+```
+
+## Erlang messages driven sink
+
+```elixir
+defmodule MessageSink do
+  use Membrane.Sink
+
+  def_input_pad :input,
+    flow_control: :push,
+    accepted_format: _any
+
+  def_options receiver: [
+                description: "PID of the process that will receive messages from the sink",
+                spec: pid()
+              ]
+
+  @impl true
+  def handle_init(_ctx, opts) do
+    {[], %{receiver: opts.receiver}}
+  end
+
+  @impl true
+  def handle_buffer(:input, buffer, _ctx, state) do
+    send(state.receiver, {:message, self(), buffer.payload})
+    {[], state}
+  end
+end
+```
+
+## Pipeline definition and startup
+
+```elixir
+alias Membrane.RCPipeline
+import Membrane.ChildrenSpec
+
+defmodule MyPipeline do
+  use Membrane.Pipeline
+
+  @impl true
+  def handle_init(_ctx, opts) do
+    spec =
+      child(:source, %MessageSource{register_name: :messages_source})
+      |> child(:sink, %MessageSink{receiver: Keyword.get(opts, :receiver)})
+
+    {[spec: spec], nil}
+  end
+end
+
+{:ok, _supervisor, pipeline} = Membrane.Pipeline.start(MyPipeline, receiver: self())
+payloads = 1..10
+
+Task.async(fn ->
+  Enum.each(
+    payloads,
+    &send(:messages_source, {:message, &1})
+  )
+end)
+
+:ok
+```
+
+## Printing of the messages received and pipeline termination
+
+```elixir
+for _i <- 1..10 do
+  receive do
+    {:message, _pid, _value} = msg -> IO.inspect(msg)
+  end
+end
+
+RCPipeline.terminate(pipeline)
+```

livebooks/playing_mp3_file/README.md

@@ -0,0 +1,13 @@
+# Membrane playing mp3 file demo
+
+This livebook shows how to load `MP3` audio from the file, transcode it to the `AAC` codec, and play it.
+
+To run the demo, [install Livebook](https://github.com/livebook-dev/livebook#escript) and open the `playing_mp3_file.livemd` file there.
+
+## Copyright and License
+
+Copyright 2024, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
+
+[![Software Mansion](https://docs.membrane.stream/static/logo/swm_logo_readme.png)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane)
+
+Licensed under the [Apache License, Version 2.0](LICENSE)