Skip to content

Commit 7d2095b

Browse files
bajajcodesbajajcodes
authored
feat: add silent transfer and voice mail detection docs (#167)
lgtm
1 parent fd89348 commit 7d2095b

File tree

3 files changed

+372
-0
lines changed

3 files changed

+372
-0
lines changed

‎fern/calls/voice-mail-detection.mdx

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
Voicemail is basically a digital answering machine. When you can’t pick up, callers can leave a message so you don’t miss anything important. It’s especially handy if you’re in a meeting, driving, or just can’t get to the phone in time.
2+
3+
### **The Main Problem**
4+
5+
If a lot of your calls are landing in voicemail, you could be spending too much time and money on calls that never connect to a real person. This leads to wasted resources, and sometimes missed business opportunities.
6+
7+
### **The Solution: Early Voicemail Detection**
8+
9+
By detecting voicemail right away, your VAPI Assistant can either hang up (if leaving a message isn’t necessary) or smoothly play a recorded message. This cuts down on useless call time and makes your entire call flow more efficient.
10+
11+
## **Two Ways to Detect Voicemail**
12+
13+
### **1. Using Twilio’s Voicemail Detection**
14+
15+
Twilio has built-in features to detect when a machine picks up. You configure these settings in your VAPI Assistant so it knows when a voicemail system has answered instead of a live person.
16+
17+
```jsx
18+
voicemailDetection: {
19+
provider: "twilio",
20+
voicemailDetectionTypes: [
21+
"machine_start",
22+
"machine_end_beep",
23+
"machine_end_silence",
24+
"unknown",
25+
"machine_end_other"
26+
],
27+
enabled: true,
28+
machineDetectionTimeout: 15,
29+
machineDetectionSpeechThreshold: 2500,
30+
machineDetectionSpeechEndThreshold: 2050,
31+
machineDetectionSilenceTimeout: 2000
32+
}
33+
34+
```
35+
36+
- **provider**: Tells VAPI to use Twilio’s system.
37+
- **voicemailDetectionTypes**: Defines the events that mean “voicemail.”
38+
- **machineDetectionTimeout**: How many seconds to wait to confirm a machine.
39+
- The other settings let you fine-tune how quickly or accurately Twilio identifies a machine based on speech or silence.
40+
41+
#### Quick Reference
42+
43+
| Setting | Type | Valid Range | Default |
44+
| ---------------------------------- | ------ | ------------- | -------- |
45+
| machineDetectionTimeout | number | 3 – 59 (sec) | 30 (sec) |
46+
| machineDetectionSpeechThreshold | number | 1000–6000 ms | 2400 ms |
47+
| machineDetectionSpeechEndThreshold | number | 500–5000 ms | 1200 ms |
48+
| machineDetectionSilenceTimeout | number | 2000–10000 ms | 5000 ms |
49+
50+
### **2. Using VAPI’s Built-In Voicemail Tool**
51+
52+
VAPI also has an LLM-powered tool that listens for typical voicemail greetings or prompts in the call’s audio transcription. If you prefer an approach that relies more on phrasing and context clues, this is a great option.
53+
54+
```jsx
55+
{
56+
...yourExistingSettings,
57+
"model": {
58+
"tools": [{ type: "voicemail" }]
59+
}
60+
}
61+
62+
```
63+
64+
Here, `tools: [{ type: "voicemail" }]` signals that your VAPI Assistant should look for keywords or patterns indicating a voicemail greeting.
65+
66+
## **Combining Both Approaches**
67+
68+
For the best of both worlds, you can enable Twilio’s detection **and** the built-in voicemail tool at the same time:
69+
70+
```jsx
71+
{
72+
...yourExistingSettings,
73+
voicemailDetection: {
74+
provider: "twilio",
75+
voicemailDetectionTypes: [
76+
"machine_start",
77+
"machine_end_beep",
78+
"unknown"
79+
],
80+
enabled: true,
81+
machineDetectionTimeout: 15
82+
},
83+
model: {
84+
tools: [{ type: "voicemail" }]
85+
}
86+
}
87+
88+
```
89+
90+
When one method doesn’t catch it, the other might—boosting your overall detection accuracy.
91+
92+
## **Tips for Better Voicemail Handling**
93+
94+
1. **Adjust Detection Timing**
95+
96+
Lower `machineDetectionTimeout` (e.g., to 5 seconds) if you want the system to decide faster. But remember, shorter timeouts can lead to occasional false positives.
97+
98+
2. **Fine-Tune Speech and Silence Thresholds**
99+
100+
For example:
101+
102+
```jsx
103+
{
104+
"provider": "twilio",
105+
"enabled": true,
106+
"machineDetectionTimeout": 5,
107+
"machineDetectionSpeechThreshold": 2400,
108+
"machineDetectionSpeechEndThreshold": 1000,
109+
"machineDetectionSilenceTimeout": 3000
110+
}
111+
112+
```
113+
114+
These values tweak how quickly Twilio “listens” for human speech or background silence.
115+
116+
3. **Think Through Your Call Flow**
117+
- **Give It Time**: If you’re leaving a message, you might want to increase `startSpeakingPlan.waitSeconds` so the detection has enough time before the tone.
118+
- **firstMessageMode**: Setting it to `assistant-waits-for-user` can also give you smoother call handling—your assistant won’t barge in if someone unexpectedly picks up late
119+
120+
## **What Happens When a Call Ends?**
121+
122+
- **Detected Voicemail + No Message**: The call will end, and you’ll see a reason like `customer-did-not-answer`.
123+
- **Detected Voicemail + Have a Message**: Your assistant leaves the recorded message, and the call ends with a reason like `voicemail`.
124+
125+
## **Testing and Next Steps**
126+
127+
1. **Make a Test Call**: Dial a known voicemail number and watch how quickly (and accurately) your VAPI Assistant identifies the machine.
128+
2. **Tweak Settings**: Adjust your timeout and threshold values based on real-world performance.
129+
3. **Repeat**: Keep testing until you’re confident your configuration is catching voicemail reliably without cutting off real people.
130+
131+
By following these steps, you’ll save time, improve call-handling efficiency, and ensure your system feels more professional. If you need to fine-tune or add new features later, you can always revisit these settings and make quick adjustments.

‎fern/docs.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -238,6 +238,8 @@ navigation:
238238
path: squads.mdx
239239
- page: Example
240240
path: squads-example.mdx
241+
- page: Silent Transfers
242+
path: squads/silent-transfers.mdx
241243
- section: Advanced Concepts
242244
contents:
243245
- section: Calls
@@ -250,6 +252,8 @@ navigation:
250252
path: calls/call-ended-reason.mdx
251253
- page: Live Call Control
252254
path: calls/call-features.mdx
255+
- page: Voice Mail Detection
256+
path: calls/voice-mail-detection.mdx
253257
- section: SIP
254258
contents:
255259
- page: SIP Introduction

‎fern/squads/silent-transfers.mdx

Lines changed: 237 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,237 @@
1+
- **The Problem**: In traditional AI call flows, when transferring from one agent to another, announcing the transfer verbally can confuse or annoy callers and disrupt the conversation's flow.
2+
- **The Solution**: Silent transfers keep the call experience _uninterrupted_, so the user doesn’t know multiple assistants are involved. The conversation flows more naturally, boosting customer satisfaction.
3+
4+
If you want to allow your call flow to move seamlessly from one assistant to another _without_ the caller hearing `Please hold while we transfer you` here’s what to do:
5+
6+
1. **Update the Destination Assistant’s First Message**
7+
- Set the assistant’s `firstMessage` to an _empty string_.
8+
- Make sure the `firstMessageMode` is set to `assistant-speaks-first-with-model-generated-message`.
9+
2. **Trigger the Transfer from the Source Assistant**
10+
11+
- In that assistant’s prompt, include a line instructing it to transfer to the desired assistant:
12+
13+
```json
14+
trigger the transferCall tool with 'assistantName' Assistant.
15+
```
16+
17+
- Replace `'assistantName'` with the exact name of the next assistant.
18+
19+
3. **Direct the Destination Assistant’s Behavior**
20+
- In that assistant’s prompt, include a line instructing it to _`Proceed directly to the Task section without any greetings or small talk.`_
21+
- This ensures there’s no awkward greeting or “Hello!” when the next assistant begins speaking.
22+
23+
### **Example Usage Scenario**
24+
25+
- **HPMA (Main Assistant)** is talking to the customer. They confirm the order details and then quietly passes the conversation to **HPPA (Payment Assistant)**.
26+
- **HPPA** collects payment details without the customer ever hearing, `We’re now transferring you to the Payment Assistant.` It feels like one continuous conversation.
27+
- Once payment is done, **HPPA** transfers the call again—this time to **HPMA-SA (Main Sub Assistant)**—which takes over final shipping arrangements.
28+
29+
Everything happens smoothly behind the scenes!
30+
31+
## **Squad and Assistant Configurations**
32+
33+
Below are the key JSON examples you’ll need. These show how to structure your assistants and squads so they work together for silent transfers.
34+
35+
### **HP Payment Squad With SubAgent**
36+
37+
```json
38+
{
39+
"members": [
40+
{
41+
"assistantId": "2d8e0d13-1b3c-4358-aa72-cf6204d6244e",
42+
"assistantDestinations": [
43+
{
44+
"message": " ",
45+
"description": "Transfer call to the payment agent",
46+
"type": "assistant",
47+
"assistantName": "HPPA"
48+
}
49+
]
50+
},
51+
{
52+
"assistantId": "ad1c5347-bc32-4b31-8bb7-6ff5fcb131f4",
53+
"assistantDestinations": [
54+
{
55+
"message": " ",
56+
"description": "Transfer call to the main sub agent",
57+
"type": "assistant",
58+
"assistantName": "HPMA-SA"
59+
}
60+
]
61+
},
62+
{
63+
"assistantId": "f1c258bc-4c8b-4c51-9b44-883ab5e40b2f",
64+
"assistantDestinations": []
65+
}
66+
],
67+
"name": "HP Payment Squad With SubAgent"
68+
}
69+
```
70+
71+
### **HPMA Assistant (Main Assistant)**
72+
73+
```json
74+
{
75+
"name": "HPMA",
76+
"voice": {
77+
"voiceId": "248be419-c632-4f23-adf1-5324ed7dbf1d",
78+
"provider": "cartesia",
79+
"fillerInjectionEnabled": false
80+
},
81+
"createdAt": "2024-11-04T17:15:08.980Z",
82+
"updatedAt": "2024-11-30T13:04:58.401Z",
83+
"model": {
84+
"model": "gpt-4o",
85+
"messages": [
86+
{
87+
"role": "system",
88+
"content": "[Identity]\nYou are the Main Assistant..."
89+
}
90+
],
91+
"provider": "openai",
92+
"maxTokens": 50,
93+
"temperature": 0.3
94+
},
95+
"firstMessage": "",
96+
"transcriber": {
97+
"model": "nova-2",
98+
"language": "en",
99+
"provider": "deepgram"
100+
},
101+
"backchannelingEnabled": false,
102+
"backgroundDenoisingEnabled": false,
103+
"isServerUrlSecretSet": false
104+
}
105+
```
106+
107+
(Similar JSON information for the HPPA and HPMA-SA assistants can follow, just like in the original text.)
108+
109+
## **Assistant Prompts (In Plain Text)**
110+
111+
Each assistant has its own system prompt outlining identity, context, style, and tasks. These prompts ensure the conversation is smooth, customer-centric, and aligned with your call flow needs. Here’s a streamlined version for reference:
112+
113+
### **HPMA (Main Assistant Prompt)**
114+
115+
```
116+
[Identity]
117+
You are the Main Assistant, a friendly and helpful agent assisting customers
118+
in purchasing widgets over the phone.
119+
120+
[Context]
121+
You're engaged with the customer to book an appointment.
122+
Stay focused on this context and provide relevant information.
123+
Once connected to a customer, proceed to the Task section.
124+
Do not invent information not drawn from the context.
125+
Answer only questions related to the context.
126+
127+
[Style]
128+
- Be polite and professional.
129+
- Use a conversational and engaging tone.
130+
- Keep responses concise and clear.
131+
132+
[Response Guidelines]
133+
- Ask one question at a time and wait for the customer's response before
134+
proceeding.
135+
- Confirm the customer's responses when appropriate.
136+
- Use simple language that is easy to understand.
137+
- Never say the word 'function' nor 'tools' nor the name of the
138+
Available functions.
139+
- Never say ending the call.
140+
- Never say transferring.
141+
142+
[Task]
143+
1.Greet the customer and ask if they are interested in purchasing widgets.
144+
- Wait for the customer's response.
145+
2. If the customer is interested, ask for their name.
146+
- Wait for the customer's response.
147+
3.Ask how many widgets the customer would like to purchase.
148+
- Wait for the customer's response.
149+
4.Confirm the order details with the customer.
150+
- trigger the transferCall tool with Payment `HPPA` Assistant.
151+
```
152+
153+
### **HPPA (Payment Assistant Prompt)**
154+
155+
```
156+
[Identity]
157+
You are the Payment Assistant, operating in secure mode to collect payment information from customers safely and confidentially.
158+
159+
[Context]
160+
You're engaged with the customer to collect payment details. Stay focused
161+
on this context and provide relevant information.
162+
Do not invent information not drawn from the context.
163+
Answer only questions related to the context.
164+
Once connected to a customer, proceed to the Task section without
165+
any greetings or small talk.
166+
167+
[Style]
168+
- Be professional and reassuring.
169+
- Maintain confidentiality at all times.
170+
- Speak clearly and calmly.
171+
172+
[Response Guidelines]
173+
- Collect the customer's credit card number, expiration date, and CVV.
174+
- Confirm each piece of information after it is provided.
175+
- Ensure the customer feels secure during the transaction.
176+
- Do not record or log any information.
177+
- Never say the word 'function' nor 'tools' nor the name of the
178+
Available functions.
179+
- Never say ending the call.
180+
- Never say transferring.
181+
182+
[Task]
183+
1. Ask for the credit card number.
184+
- Wait for the customer's response.
185+
2. Ask for the expiration date of the card.
186+
- Wait for the customer's response.
187+
3. Ask for the CVV number.
188+
- Wait for the customer's response.
189+
4. Confirm that the payment has been processed successfully.
190+
- trigger the transferCall tool with Payment `HPMA-SA` Assistant.
191+
```
192+
193+
### **HPMA-SA (Main Sub Assistant Prompt)**
194+
195+
```
196+
[Identity]
197+
You are the Main Assistant, a friendly and helpful agent assisting customers
198+
in purchasing widgets over the phone.
199+
200+
[Context]
201+
You're engaged with the customer to book an appointment.
202+
Stay focused on this context and provide relevant information.
203+
Do not invent information not drawn from the context.
204+
Answer only questions related to the context.
205+
Once connected to a customer, proceed to the Task section without any greetings
206+
or small talk.
207+
208+
[Style]
209+
- Be professional and reassuring.
210+
- Maintain confidentiality at all times.
211+
- Speak clearly and calmly.
212+
213+
[Response Guidelines]
214+
- Collect the customer's credit card number, expiration date, and CVV.
215+
- Confirm each piece of information after it is provided.
216+
- Ensure the customer feels secure during the transaction.
217+
- Do not record or log any information.
218+
- Never say the word 'function' nor 'tools' nor the name of the
219+
Available functions.
220+
- Never say ending the call.
221+
- Never say transferring.
222+
223+
[Task]
224+
1.Ask for the customer's shipping address to deliver the widgets.
225+
- Wait for the customer's response.
226+
2.Confirm the shipping address and provide an estimated delivery date.
227+
3.Ask if the customer has any additional questions or needs further assistance.
228+
- Wait for the customer's response.
229+
4.Provide any additional information or assistance as needed.
230+
5.Thank the customer for their purchase and end the call politely.
231+
```
232+
233+
## **Conclusion**
234+
235+
By following these steps and examples, you can configure your call system to conduct **silent transfers** ensuring that callers experience a single, uninterrupted conversation. Each assistant does its job smoothly, whether it’s capturing payment, finalizing a shipping address, or collecting basic info.
236+
237+
Enjoy setting up your silent transfers!

0 commit comments

Comments
 (0)