YourTTS and voice cloning

[ROBERT-MCDOWELL]

I'm using tts_with_vc() with yourtts like this:
tts.tts_with_vc(
text='hello world'],
speaker='male-en-2',
speaker_wav=voice_clone_path,
language='en'
)
I had to add speaker= to make it work, although with fairseq I don't need to add it. is it correct? btw the result is not really the cloned voice but a voice sounding like the builtin voice with a very little sound from the voice_clone. is it normal? or does it need more settings? thanks

[eginhard]

Yes, for multi-speaker models you always need to specify a speaker to do TTS, Coqui doesn't have a concept of a default speaker for a model. The Fairseq models are single-speaker, so it's not needed there.

But note that YourTTS specifically also supports voice cloning directly and you will probably achieve better results like this instead of separately doing voice conversion afterwards. And in that case there's no need to specify a speaker:

from TTS.api import TTS

tts = TTS('tts_models/multilingual/multi-dataset/your_tts')
tts.tts_to_file('hello world', speaker_wav='reference.wav', language='en')

[ROBERT-MCDOWELL]

ok got it thanks!

[ROBERT-MCDOWELL]

Could you confirm yourtts is only eng, fr-fr, pt-br? I saw somewhere it was supporting more languages, but how?

[eginhard]

Yes, this model has only these languages. There is another YourTTS model trained on other languages - the training recipe (https://github.com/idiap/coqui-ai-TTS/blob/dev/recipes/multilingual/cml_yourtts/train_yourtts.py) is in the repo. The model itself is not fully integrated, but you can download it and run manually: coqui-ai#2735

[ROBERT-MCDOWELL]

thanks eginhard!, to load it, should I use tts() or else? because for now I have only the xttsv2 load checkpoint as an example to load custom models

[ROBERT-MCDOWELL]

btw thee are the datasets, but how to get the config.json and model.pth already done?

[eginhard]

There's a Colab notebook in the linked PR that has some usage examples for the model.

[ROBERT-MCDOWELL]

ok thanks