Skip to content

Commit 73905c9

Browse files
committed
chore: update docs for language detection
1 parent 0970249 commit 73905c9

File tree

1 file changed

+39
-5
lines changed

1 file changed

+39
-5
lines changed

README.md

+39-5
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ expo-speech-recognition implements the iOS [`SFSpeechRecognizer`](https://develo
2424
- [Polyfilling the Web SpeechRecognition API](#polyfilling-the-web-speechrecognition-api)
2525
- [Muting the beep sound on Android](#muting-the-beep-sound-on-android)
2626
- [Improving accuracy of single-word prompts](#improving-accuracy-of-single-word-prompts)
27+
- [Language Detection](#language-detection)
2728
- [Platform Compatibility Table](#platform-compatibility-table)
2829
- [Common Troubleshooting issues](#common-troubleshooting-issues)
2930
- [Android issues](#android-issues)
@@ -36,14 +37,14 @@ expo-speech-recognition implements the iOS [`SFSpeechRecognizer`](https://develo
3637
- [getPermissionsAsync()](#getpermissionsasync-promisepermissionresponse)
3738
- [getStateAsync()](#getstateasync-promisespeechrecognitionstate)
3839
- [addSpeechRecognitionListener()](#addspeechrecognitionlistener)
39-
- [getSupportedLocales()](#getsupportedlocales-promise-locales-string-installedlocales-string-)
40+
- [getSupportedLocales()](#getsupportedlocales)
4041
- [getSpeechRecognitionServices()](#getspeechrecognitionservices-string-android-only)
4142
- [getDefaultRecognitionService()](#getdefaultrecognitionservice--packagename-string--android-only)
4243
- [getAssistantService()](#getassistantservice--packagename-string--android-only)
4344
- [isRecognitionAvailable()](#isrecognitionavailable-boolean)
4445
- [supportsOnDeviceRecognition()](#supportsondevicerecognition-boolean)
4546
- [supportsRecording()](#supportsrecording-boolean)
46-
- [androidTriggerOfflineModelDownload()](#androidtriggerofflinemodeldownload-locale-string--promise-status-opened_dialog--download_success--download_canceled-message-string-)
47+
- [androidTriggerOfflineModelDownload()](#androidtriggerofflinemodeldownload)
4748
- [setCategoryIOS()](#setcategoryios-void-ios-only)
4849
- [getAudioSessionCategoryAndOptionsIOS()](#getaudiosessioncategoryandoptionsios-ios-only)
4950
- [setAudioSessionActiveIOS()](#setaudiosessionactiveiosvalue-boolean-options--notifyothersondeactivation-boolean--void)
@@ -334,7 +335,7 @@ Events are largely based on the [Web Speech API](https://developer.mozilla.org/e
334335
| `speechend` | Fired when speech recognized by the speech recognition service has stopped being detected. | Not supported yet on iOS |
335336
| `start` | Speech recognition has started | Use this event to indicate to the user when to speak. |
336337
| `volumechange` | Fired when the input volume changes. | Returns a value between -2 and 10 indicating the volume of the input audio. Consider anything below 0 to be inaudible. |
337-
| `languagedetection` | Called when the language detection (and switching) results are available. | Android 14+ only. Enabled with `EXTRA_ENABLE_LANGUAGE_DETECTION` in the `androidIntent` option when starting. Also can be called multiple times by enabling `EXTRA_ENABLE_LANGUAGE_SWITCH`. |
338+
| `languagedetection` | Called when the language detection (and switching) results are available. | Android 14+ only with `com.google.android.as`. Enabled with `EXTRA_ENABLE_LANGUAGE_DETECTION` in the `androidIntent` option when starting. Also can be called multiple times by enabling `EXTRA_ENABLE_LANGUAGE_SWITCH`. |
338339

339340
## Handling Errors
340341

@@ -697,6 +698,39 @@ You may notice that after saying short syllables, words, letters, or numbers (e.
697698
- For both platforms, you also may want to consider using on-device recognition. On Android this seems to work well for single-word prompts.
698699
- Alternatively, you may want to consider recording the recognized audio and sending it to an external service for further processing. See [Persisting Audio Recordings](#persisting-audio-recordings) for more information. Note that some services (such as the Google Speech API) may require an audio file with a duration of at least 3 seconds.
699700

701+
## Language Detection
702+
703+
> [!NOTE]
704+
> This feature is currently only available on Android 14+ using the `com.google.android.as` service package.
705+
706+
You can use the `languagedetection` event to get the detected language and confidence level. This feature has a few requirements:
707+
708+
- Android 14+ only.
709+
- The `com.google.android.as` (on-device recognition) service package must be selected. This seems to be the only service that supports language detection as of writing this.
710+
- You must enable `EXTRA_ENABLE_LANGUAGE_DETECTION` in the `androidIntentOptions` when starting the recognition.
711+
- Optional: You can enable `EXTRA_ENABLE_LANGUAGE_SWITCH` to allow the user to switch languages, however **keep in mind that you need the language model to be downloaded for this to work**. Refer to [androidTriggerOfflineModelDownload()](#androidtriggerofflinemodeldownload) to download a model, and [getSupportedLocales()](#getsupportedlocales) to get the list of downloaded on-device locales.
712+
713+
Example:
714+
715+
```tsx
716+
import { useSpeechRecognitionEvent } from "expo-speech-recognition";
717+
718+
useSpeechRecognitionEvent("languagedetection", (event) => {
719+
console.log("Language detected:", event.detectedLanguage); // e.g. "en-us"
720+
console.log("Confidence:", event.confidence); // A value between 0.0 and 1.0
721+
console.log("Top locale alternatives:", event.topLocaleAlternatives); // e.g. ["en-au", "en-gb"]
722+
});
723+
724+
// Start recognition
725+
ExpoSpeechRecognitionModule.start({
726+
androidIntentOptions: {
727+
EXTRA_ENABLE_LANGUAGE_DETECTION: true,
728+
EXTRA_ENABLE_LANGUAGE_SWITCH: true,
729+
},
730+
androidRecognitionServicePackage: "com.google.android.as", // or set "requiresOnDeviceRecognition" to true
731+
});
732+
```
733+
700734
## Platform Compatibility Table
701735

702736
As of 7 Aug 2024, the following platforms are supported:
@@ -853,7 +887,7 @@ const listener = addSpeechRecognitionListener("result", (event) => {
853887
listener.remove();
854888
```
855889

856-
### `getSupportedLocales(): Promise<{ locales: string[]; installedLocales: string[] }>`
890+
### `getSupportedLocales()`
857891

858892
> [!NOTE]
859893
> Not supported on Android 12 and below
@@ -967,7 +1001,7 @@ const available = supportsRecording();
9671001
console.log("Recording available:", available);
9681002
```
9691003

970-
### `androidTriggerOfflineModelDownload({ locale: string }): Promise<{ status: "opened_dialog" | "download_success" | "download_canceled", message: string }>`
1004+
### `androidTriggerOfflineModelDownload()`
9711005

9721006
Users on Android devices will first need to download the offline model for the locale they want to use in order to use on-device speech recognition (i.e. the `requiresOnDeviceRecognition` setting in the `start` options).
9731007

0 commit comments

Comments
 (0)