Skip to content

Commit 21f0ca6

Browse files
committed
smart endpointing plan update
1 parent 6fdb955 commit 21f0ca6

File tree

2 files changed

+8
-23
lines changed

2 files changed

+8
-23
lines changed

fern/customization/speech-configuration.mdx

Lines changed: 8 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -16,35 +16,20 @@ This plan defines the parameters for when the assistant begins speaking after th
1616
- **Wait Time Before Speaking**: You can set how long the assistant waits before speaking after the customer finishes. The default is 0.4 seconds, but you can increase it if the assistant is speaking too soon, or decrease it if there’s too much delay.
1717
**Example:** For tech support calls, set `waitSeconds` for the assistant to more than 1.0 seconds to give customers time to complete their thoughts, even if they have some pauses in between.
1818

19-
- **Smart Endpointing**: This feature uses advanced processing to detect when the customer has truly finished speaking, especially if they pause mid-thought. It’s off by default but can be turned on if needed. **Example:** In insurance claims, `smartendpointingEnabled` helps avoid interruptions while customers think through responses and as they formulate responses. Example: The assistant mentions "do you want a loan," triggering the system to check the customer's response. If the customer responds with "yes" (matching the CustomerRegex for "yes|no"), the system waits for 1.1 seconds before proceeding, allowing time for further clarification. For responses requiring number sequences like "What’s your account number?", set longer timeouts like 5 seconds or more to accommodate pauses between digits.
19+
- **Smart Endpointing Plan**: This feature uses advanced processing to detect when the customer has truly finished speaking, especially if they pause mid-thought. It can be configured in three ways:
20+
- **Off**: Disabled by default
21+
- **LiveKit**: Recommended for English conversations as it provides the most sophisticated solution for detecting natural speech patterns and pauses. LiveKit can be fine-tuned using the `waitFunction` parameter to adjust response timing based on the probability that the user is still speaking.
22+
- **Vapi**: Recommended for non-English conversations or as an alternative when LiveKit isn't suitable
23+
24+
![LiveKit Smart Endpointing Configuration](../static/images/advanced-tab/livekit-smart-endpointing.png)
25+
26+
**Example:** In insurance claims, Vapi's smart endpointing helps avoid interruptions while customers think through complex responses. For instance, when the assistant asks "do you want a loan," the system can intelligently wait for the complete response rather than interrupting after the initial "yes" or "no." For responses requiring number sequences like "What's your account number?", the system can detect natural pauses between digits without prematurely ending the customer's turn to speak.
2027

2128
- **Transcription-Based Detection**: Customize how the assistant determines that the customer has stopped speaking based on what they’re saying. This offers more control over the timing. **Example:** When a customer says, "My account number is 123456789, I want to transfer $500."
2229
- The system detects the number "123456789" and waits for 0.5 seconds (`WaitSeconds`) to ensure the customer isn't still speaking.
2330
- If the customer were to finish with an additional line, "I want to transfer $500.", the system uses `onPunctuationSeconds` to confirm the end of the speech and then proceed with the request processing.
2431
- In a scenario where the customer has been silent for a long and has already finished speaking but the transcriber is not confident to punctuate the transcription, `onNoPunctuationSeconds` is used for 1.5 seconds.
2532

26-
Here's a code snippet for Start Speaking Plan -
27-
28-
```json
29-
"startSpeakingPlan": {
30-
"waitSeconds": 0.4,
31-
"smartEndpointingEnabled": false,
32-
"customEndpointingRules": [
33-
{
34-
"type": "both",
35-
"assistantRegex": "customEndpointingRules",
36-
"customerRegex": "customEndpointingRules",
37-
"timeoutSeconds": 1.1
38-
}
39-
],
40-
"transcriptionEndpointingPlan": {
41-
"onPunctuationSeconds": 0.1,
42-
"onNoPunctuationSeconds": 1.5,
43-
"onNumberSeconds": 0.5
44-
}
45-
}
46-
```
47-
4833

4934
## Stop Speaking Plan
5035
The Stop Speaking Plan defines when the assistant stops talking after detecting customer speech.
Loading

0 commit comments

Comments
 (0)