Improvement of Speaker Diarization pipeline performance for Indian languages

[mukherjeesougata]

I am using the Emilia-Pipe pipeline to process in-the-wild raw audio containing Indian languages and extract clean speech audio with segregated speakers. This pipeline utilizes the PyAnnote Speaker Diarization 3.1 pretrained model from Hugging Face, accessed via Hugging Face tokens. However, in many cases, the pipeline fails to segregate speakers accurately. I have a few queries regarding this issue:

• Should I pretrain the pipeline from scratch with Indian languages, or would fine-tuning be sufficient to achieve optimal performance?
• If in case I have to do pretraining how to do the pretraining and minimum how much duration of data is required?
• I found this Colab notebook for adapting a pretrained pipeline to custom data. Here the adaptation is given for 2.1 version, so will it work for 3.1 version also? How can I access and integrate the adapted pipeline into Emilia-Pipe?
• I would also like to know what are the modifications and developments done in Speaker-diarization-3.1 pipeline in comparison to the one that is mentioned in this paper.

Looking forward to your guidance. Thank you!

[mukherjeesougata]

@hbredin can you kindly please respond to this query?

[mukherjeesougata]

@mogwai can you kindly please reply to this query?