Using pyannote in Rust

[thewh1teagle]

I would like to use pyannote from Rust with whisper-rs which uses whisper.cpp
I'm wondering about rewriting pyannote in Rust for using in Vibe. How hard should it be to rewrite this? any thoughts?

Update:
created pyannote-rs

[altunenes]

I'm looking for something like that with Rust right now :-) what do you think is the best approach so far?

[thewh1teagle]

I think that using the onnx runtime is the most promising approach, but the thing is that pyannote-audio is not a simple solution. They use multiple models to get the job done, such as segmenting (finding speech timings) and then voice embedding on each of those segments.

So, it will require some work. Initially, I thought to use the segments returned from Whisper itself, but it turned out that it doesn't really split segments across speakers, so a segments (sentences) from whisper.cpp can contain multiple speakers.

Join the research in vibe/issues/74 :)

[altunenes]

thank you so much for the detailed message, the best is to continue in the Vibe!