Skip to content

Commit 6b0133d

Browse files
rmcwhorterrmcwhorter
authored
VAP-5778 websocket docs (#299)
* websocket docs * sidebar * gpt-4.5'ed the docs * strip the <important> tags
1 parent 8f479bc commit 6b0133d

File tree

2 files changed

+186
-0
lines changed

2 files changed

+186
-0
lines changed

‎fern/calls/websocket-transport.mdx

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
---
2+
title: WebSocket Transport
3+
description: Stream audio directly via WebSockets for real-time, bidirectional communication
4+
slug: calls/websocket-transport
5+
---
6+
7+
# WebSocket Transport
8+
9+
Vapi's WebSocket transport enables real-time, bidirectional audio communication directly between your application and Vapi's AI assistants. Unlike traditional phone or web calls, this transport method lets you stream raw audio data instantly with minimal latency.
10+
11+
## Key Benefits
12+
13+
- **Low Latency**: Direct streaming ensures minimal delays.
14+
- **Bidirectional Streaming**: Real-time audio flow in both directions.
15+
- **Easy Integration**: Compatible with any environment supporting WebSockets.
16+
- **Flexible Audio Formats**: Customize audio parameters such as sample rate.
17+
- **Automatic Sample Rate Conversion**: Seamlessly handles various audio rates.
18+
19+
## Creating a WebSocket Call
20+
21+
To initiate a call using WebSocket transport:
22+
23+
```bash
24+
curl 'https://api.vapi.ai/call' \
25+
-H 'authorization: Bearer YOUR_API_KEY' \
26+
-H 'content-type: application/json' \
27+
--data-raw '{
28+
"assistant": { "assistantId": "YOUR_ASSISTANT_ID" },
29+
"transport": {
30+
"provider": "vapi.websocket",
31+
"audioFormat": {
32+
"format": "pcm_s16le",
33+
"container": "raw",
34+
"sampleRate": 16000
35+
}
36+
}
37+
}'
38+
```
39+
40+
### Sample API Response
41+
42+
```json
43+
{
44+
"id": "7420f27a-30fd-4f49-a995-5549ae7cc00d",
45+
"assistantId": "5b0a4a08-133c-4146-9315-0984f8c6be80",
46+
"type": "vapi.websocketCall",
47+
"createdAt": "2024-09-10T11:14:12.339Z",
48+
"updatedAt": "2024-09-10T11:14:12.339Z",
49+
"orgId": "eb166faa-7145-46ef-8044-589b47ae3b56",
50+
"cost": 0,
51+
"status": "queued",
52+
"transport": {
53+
"provider": "vapi.websocket",
54+
"websocketCallUrl": "wss://api.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/transport"
55+
}
56+
}
57+
```
58+
59+
## Audio Format Configuration
60+
61+
When creating a WebSocket call, the audio format can be customized:
62+
63+
| Parameter | Description | Default |
64+
|-------------|-------------------------|---------------------|
65+
| `format` | Audio encoding format | `pcm_s16le` (16-bit PCM) |
66+
| `container` | Audio container format | `raw` (Raw PCM) |
67+
| `sampleRate`| Sample rate in Hz | `16000` (16kHz) |
68+
69+
Currently, Vapi supports only raw PCM (`pcm_s16le` with `raw` container). Additional formats may be supported in future updates.
70+
71+
<Note>
72+
Vapi automatically converts sample rates as needed. You can stream audio at 8kHz, 44.1kHz, etc., and Vapi will handle conversions seamlessly.
73+
</Note>
74+
75+
## Connecting to the WebSocket
76+
77+
Use the WebSocket URL from the response to establish a connection:
78+
79+
```javascript
80+
const socket = new WebSocket("wss://api.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/transport");
81+
82+
socket.onopen = () => console.log("WebSocket connection opened.");
83+
socket.onclose = () => console.log("WebSocket connection closed.");
84+
socket.onerror = (error) => console.error("WebSocket error:", error);
85+
```
86+
87+
## Sending and Receiving Data
88+
89+
The WebSocket supports two types of messages:
90+
91+
- **Binary audio data** (PCM, 16-bit signed little-endian)
92+
- **Text-based JSON control messages**
93+
94+
### Sending Audio Data
95+
96+
```javascript
97+
function sendAudioChunk(audioBuffer) {
98+
if (socket.readyState === WebSocket.OPEN) {
99+
socket.send(audioBuffer);
100+
}
101+
}
102+
103+
navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
104+
const audioContext = new AudioContext();
105+
const source = audioContext.createMediaStreamSource(stream);
106+
const processor = audioContext.createScriptProcessor(1024, 1, 1);
107+
108+
processor.onaudioprocess = (event) => {
109+
const pcmData = event.inputBuffer.getChannelData(0);
110+
const int16Data = new Int16Array(pcmData.length);
111+
112+
for (let i = 0; i < pcmData.length; i++) {
113+
int16Data[i] = Math.max(-32768, Math.min(32767, pcmData[i] * 32768));
114+
}
115+
116+
sendAudioChunk(int16Data.buffer);
117+
};
118+
119+
source.connect(processor);
120+
processor.connect(audioContext.destination);
121+
});
122+
```
123+
124+
### Receiving Data
125+
126+
```javascript
127+
socket.onmessage = (event) => {
128+
if (event.data instanceof Blob) {
129+
event.data.arrayBuffer().then(buffer => {
130+
const audioData = new Int16Array(buffer);
131+
playAudio(audioData);
132+
});
133+
} else {
134+
try {
135+
const message = JSON.parse(event.data);
136+
handleControlMessage(message);
137+
} catch (error) {
138+
console.error("Failed to parse message:", error);
139+
}
140+
}
141+
};
142+
```
143+
144+
### Sending Control Messages
145+
146+
```javascript
147+
function sendControlMessage(messageObj) {
148+
if (socket.readyState === WebSocket.OPEN) {
149+
socket.send(JSON.stringify(messageObj));
150+
}
151+
}
152+
153+
// Example: hangup call
154+
function hangupCall() {
155+
sendControlMessage({ type: "hangup" });
156+
}
157+
```
158+
159+
## Ending the Call
160+
161+
To gracefully end the WebSocket call:
162+
163+
```javascript
164+
sendControlMessage({ type: "hangup" });
165+
socket.close();
166+
```
167+
168+
## Comparison: WebSocket Transport vs. Call Listen Feature
169+
170+
Vapi provides two WebSocket options:
171+
172+
| WebSocket Transport | Call Listen Feature |
173+
|-------------------------------------|------------------------------------|
174+
| Primary communication method | Secondary, monitoring-only channel |
175+
| Bidirectional audio streaming | Unidirectional (listen-only) |
176+
| Replaces phone/web as transport | Supplements existing calls |
177+
| Uses `provider: "vapi.websocket"` | Accessed via `monitor.listenUrl` |
178+
179+
Refer to [Live Call Control](/calls/call-features) for more on the Call Listen feature.
180+
181+
<Warning>
182+
When using WebSocket transport, phone-based parameters (`phoneNumber` or `phoneNumberId`) are not permitted. These methods are mutually exclusive.
183+
</Warning>
184+

‎fern/docs.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -362,6 +362,8 @@ navigation:
362362
path: calls/call-handling-with-vapi-and-twilio.mdx
363363
- page: Voicemail Detection
364364
path: calls/voicemail-detection.mdx
365+
- page: WebSocket Transport
366+
path: calls/websocket-transport.mdx
365367

366368
- section: SDKs
367369
path: sdks.mdx

0 commit comments

Comments
 (0)