Skip to content

Create fine-tune-csm #2860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions _blog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6005,3 +6005,20 @@
- community
- announcement
- transformers

- local: fine-tune-csm
title: "Fine-Tune CSM For Conversational Speech LLMs with 🤗 Transformers"
thumbnail: /blog/assets/112_fine_tune_csm/thumbnail.jpg
authors:
- user: eustlb
- user: reach-vb
- user: Steveeeeeeen
date: TBD
tags:
- csm
- conversational
- audio
- speech
- text-to-speech
- llm
- fine-tuning
38 changes: 38 additions & 0 deletions fine-tune-csm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
title: "Fine-Tune CSM For Conversational Speech LLMs with 🤗 Transformers"
thumbnail: /blog/assets/112_fine_tune_csm/thumbnail.jpg
authors:
- user: eustlb
- user: reach-vb
- user: Steveeeeeeen
---

# Fine-Tune CSM For Conversational Speech LLMs with 🤗 Transformers

# To be updated
<a target="_blank" href="">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
</a>


In this blog, we present a step-by-step guide on fine-tuning CSM
for any conversational speech LLM dataset using Hugging Face 🤗 Transformers. This blog
provides in-depth explanations of the CSM model, the dataset and
the theory behind fine-tuning, with accompanying code cells to execute the data
preparation and fine-tuning steps. For a more streamlined version of the notebook
with fewer explanations but all the code, see the accompanying [Google Colab]().


## Table of Contents

1. [Introduction](#introduction)
2. [Fine-tuning CSM in a Google Colab](#fine-tuning-csm-in-a-google-colab)
1. [Prepare Environment](#prepare-environment)
2. [Load Dataset](#load-dataset)
3. [Prepare Feature Extractor, Tokenizer and Data](#prepare-feature-extractor-tokenizer-and-data)
4. [Training and Evaluation](#training-and-evaluation)
5. [Building a Demo](#building-a-demo)
3. [Closing Remarks](#closing-remarks)

## Introduction

Loading