diff --git a/fern/docs.yml b/fern/docs.yml index c5eb56f7..f44fa7be 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -242,6 +242,8 @@ navigation: path: tools-calling.mdx - page: Prompting Guide path: prompting-guide.mdx + - page: OpenAI Realtime + path: openai-realtime.mdx - section: Glossary contents: - page: Definitions diff --git a/fern/openai-realtime.mdx b/fern/openai-realtime.mdx new file mode 100644 index 00000000..e82ac8be --- /dev/null +++ b/fern/openai-realtime.mdx @@ -0,0 +1,16 @@ +--- +title: OpenAI Realtime +subtitle: You can use OpenAI's newest speech-to-speech model with your Vapi assistants. +slug: openai-realtime +--- + + + The Realtime API is currently in beta, and not recommended for production use by OpenAI. We're excited to have you try this new feature and welcome your [feedback](https://discord.com/invite/pUFNcf2WmH) as we continue to refine and improve the experience. + + +OpenAI’s Realtime API enables developers to use a native speech-to-speech model. Unlike other Vapi configurations which orchestrate a transcriber, model and voice API to simulate speech-to-speech, OpenAI’s Realtime API natively processes audio in and audio out. + +To start using it with your Vapi assistants, select `gpt-4o-realtime-preview-2024-10-01` as your model. +- Please note that only OpenAI voices may be selected while using this model. The voice selection will not act as a TTS (text-to-speech) model, but rather as the voice used within the speech-to-speech model. +- Also note that we don’t currently support Knowledge Bases with the Realtime API. +- Lastly, note that our Realtime integration still retains the rest of Vapi's orchestration layer such as the endpointing and interruption models to enable a reliable conversational flow. \ No newline at end of file