React Native Sherpa ONNX

https://github.com/xdcobra/react-native-sherpa-onnx
Admin
A React Native SDK providing offline and streaming speech processing capabilities including...
Tokens:273,468
Snippets:1,393
Trust Score:5.6
License:MIT
Update:4 days ago
Show doc for...
Tokens:
Raw
### Start the packager

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/CONTRIBUTING.md

Starts the Metro server for the example application.

```sh
yarn example start
```

--------------------------------

### Run the Example Application

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/README.md

Commands to navigate to the example directory, install dependencies, and launch the application on Android or iOS.

```sh
cd example
yarn install
yarn android  # or yarn ios
```

--------------------------------

### downloadModel usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example of starting a model download with a progress callback.

```ts
await downloadModel(ModelCategory.Tts, 'vits-piper-en_US-lessac-medium', {
  onProgress: (p) => console.log(p.bytesProcessed, p.totalBytes),
});
```

--------------------------------

### Build and run iOS app with bundle install, yarn ios, and yarn build:ios

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/example/README.md

Run these commands from the example directory. First install Ruby gems (CocoaPods) with bundle install, then run on simulator with yarn ios (triggers pod install if needed), or build only with yarn build:ios for CI.

```sh
# One-time: install Ruby gems (CocoaPods) if you haven’t already
bundle install

# Run on simulator (triggers pod install if needed)
yarn ios

# Or build only (e.g. for CI), output in ios/build
yarn build:ios
```

--------------------------------

### Install react-native-sherpa-onnx

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/README.md

Standard installation command for the SDK.

```sh
npm install react-native-sherpa-onnx
```

--------------------------------

### Run example app on Android

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/CONTRIBUTING.md

Builds and runs the example application on an Android device or emulator.

```sh
yarn example android
```

--------------------------------

### Run example app on iOS

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/CONTRIBUTING.md

Builds and runs the example application on an iOS device or simulator.

```sh
yarn example ios
```

--------------------------------

### TtsEngine.startPcmPlayer()

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/tts.md

Starts the native PCM player for real-time audio playback.

```APIDOC
## TtsEngine.startPcmPlayer()

### Description
Starts the native PCM player for real-time audio playback.

### Method
startPcmPlayer

### Parameters
- **sampleRate** (number) - Required - The sample rate of the PCM audio.

### Returns
- **void**
```

--------------------------------

### resumeDownload usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example of resuming a previously paused download.

```ts
await resumeDownload(ModelCategory.Tts, 'vits-piper-en_US-lessac-medium');
```

--------------------------------

### Install iOS Pods

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/README.md

Commands to install iOS dependencies and trigger the framework download script.

```sh
cd your-app/ios
bundle install
bundle exec pod install
```

--------------------------------

### Start Metro with npm or Yarn

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/example/README.md

Run these commands from the project root to start the Metro dev server. Use either npm or Yarn.

```sh
# Using npm
npm start

# OR using Yarn
yarn start
```

--------------------------------

### configureBackgroundDownloader usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Demonstrates enabling notifications for the background downloader.

```ts
configureBackgroundDownloader({ showNotificationsEnabled: true });
```

--------------------------------

### getIncompleteDownloads usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example of retrieving a list of pending downloads for a specific category.

```ts
const pending = await getIncompleteDownloads(ModelCategory.Stt);
```

--------------------------------

### Install with Yarn Environment Variable

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/README.md

Alternative method to set the node linker during installation.

```sh
YARN_NODE_LINKER=node-modules yarn install
```

--------------------------------

### PcmLiveStreamHandle.start

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/pcm-live-stream.md

Starts native capture. Ensure permission is granted first.

```APIDOC
## PcmLiveStreamHandle.start

### Description
Starts native capture. Ensure permission is granted first.

### Method Signature
```ts
start: () => Promise<void>;
```

### Parameters
None.

### Return Value
Returns a Promise that resolves when capture starts.

### Example
```ts
await stream.start();
```
```

--------------------------------

### getStorageBasePath usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Demonstrates retrieving the storage base path.

```ts
const basePath = await getStorageBasePath();
```

--------------------------------

### tts.startPcmPlayer()

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/tts.md

Starts the PCM audio player for the TTS engine.

```APIDOC
## tts.startPcmPlayer()

### Description
Starts the PCM audio player for the TTS engine.

### Method
JS: `tts.startPcmPlayer()`

### Native Method
`startTtsPcmPlayer(instanceId, sampleRate, channels)`

### Parameters
- **instanceId** (string) - Required - The instance ID of the TTS engine.
- **sampleRate** (number) - Required - The sample rate of the audio.
- **channels** (number) - Required - The number of audio channels.
```

--------------------------------

### Install coreutils for stdbuf (optional)

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/third_party/ffmpeg_prebuilt/BUILD_MSYS2.md

Run in the MSYS2 shell (not MinGW64) to install coreutils, which provides stdbuf for line-by-line configure/make output. The build script uses stdbuf automatically if present.

```bash
pacman -S --noconfirm coreutils
```

--------------------------------

### getProtectedKeys usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Demonstrates how to retrieve the set of protected keys.

```ts
const protectedKeys = await getProtectedKeys();
```

--------------------------------

### Quick Start: Selecting an Execution Provider

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/execution-providers.md

Demonstrates how to check for available hardware acceleration providers on Android and iOS and pass the selected provider to the engine creation function.

```typescript
import {
  getQnnSupport,
  getNnapiSupport,
  getXnnpackSupport,
  getCoreMlSupport,
  getAvailableProviders,
} from 'react-native-sherpa-onnx';
import { Platform } from 'react-native';

// Pick the best provider for this device
let provider = 'cpu';

if (Platform.OS === 'android') {
  const qnn = await getQnnSupport();
  if (qnn.canInit) {
    provider = 'qnn';
  } else {
    const nnapi = await getNnapiSupport();
    if (nnapi.canInit) provider = 'nnapi';
  }
} else {
  const cml = await getCoreMlSupport();
  if (cml.canInit) provider = 'coreml';
}

// Pass to engine creation
const stt = await createSTT({
  modelPath: { type: 'asset', path: 'models/my-stt-model' },
  provider,
});
```

--------------------------------

### Set SHERPA_ONNX_DISABLE_LIBARCHIVE for pod install

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/disable-libarchive.md

Export this environment variable before running pod install to disable libarchive for iOS. The setup script will skip downloading libarchive.xcframework and the podspec will ignore existing frameworks.

```bash
export SHERPA_ONNX_DISABLE_LIBARCHIVE=1
cd ios && pod install
```

--------------------------------

### Multi-speaker usage with getNumSpeakers and generateSpeech

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/example/android/app/src/main/assets/models/sherpa-onnx-vits-piper-en_US-libritts_r-medium/README.md

TypeScript example showing how to get the number of speakers (904) and generate speech with a specific speaker ID (0-903) using the generateSpeech function.

```typescript
// Get number of speakers
const numSpeakers = await getNumSpeakers(); // Returns 904

// Generate with different speakers
const audio = await generateSpeech('Hello!', { sid: 42 }); // Speaker ID 0-903
```

--------------------------------

### Quick Start Model Loading

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/model-setup.md

Demonstrates the workflow for discovering bundled models, detecting their type, and initializing the STT engine.

```typescript
import {
  assetModelPath,
  listAssetModels,
  resolveModelPath,
} from 'react-native-sherpa-onnx';
import { createSTT, detectSttModel } from 'react-native-sherpa-onnx/stt';

// 1) Discover bundled models
const models = await listAssetModels();
// [{ folder: 'sherpa-onnx-whisper-tiny-en', hint: 'stt' }, ...]

// 2) Detect model type before loading
const modelPath = assetModelPath('models/sherpa-onnx-whisper-tiny-en');
const detection = await detectSttModel(modelPath);
console.log(detection.modelType); // 'whisper'

// 3) Create engine
const stt = await createSTT({
  modelPath,
  modelType: 'auto', // uses detected type
});
```

--------------------------------

### getSampleRate, startPcmPlayer, writePcmChunk, stopPcmPlayer

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/tts.md

Native PCM playback methods to minimize JS roundtrips. Use getSampleRate to get the sample rate, startPcmPlayer to start playback, writePcmChunk to feed audio data, and stopPcmPlayer to stop playback.

```APIDOC
## getSampleRate

### Description
Returns the sample rate of the TTS engine.

### Method
Async method on TTS engine

### Returns
- **sampleRate** (number) - The sample rate

## startPcmPlayer

### Description
Starts the native PCM player with the given sample rate and number of channels.

### Method
Async method on TTS engine

### Parameters
- **sampleRate** (number) - Required - Sample rate for playback
- **channels** (number) - Required - Number of channels (e.g., 1 for mono)

## writePcmChunk

### Description
Writes a PCM chunk to the native player. Expects float PCM in [-1.0, 1.0].

### Method
Method on TTS engine

### Parameters
- **samples** (Float32Array) - Required - PCM samples in [-1.0, 1.0]

## stopPcmPlayer

### Description
Stops the native PCM player.

### Method
Method on TTS engine

### Example
```typescript
const sampleRate = await tts.getSampleRate();
await tts.startPcmPlayer(sampleRate, 1); // mono

const controller = await tts.generateSpeechStream(text, undefined, {
  onChunk: (chunk) => {
    if (chunk.samples.length > 0) tts.writePcmChunk(chunk.samples);
  },
  onEnd: () => tts.stopPcmPlayer(),
  onError: () => tts.stopPcmPlayer(),
});
```
```

--------------------------------

### clearModelsCache usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example call to clearModelsCache for the speech-to-text category.

```TypeScript
await clearModelsCache(ModelCategory.Stt);
```

--------------------------------

### getModelsCacheStatus usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example call to getModelsCacheStatus for the speech-to-text category.

```TypeScript
const status = await getModelsCacheStatus(ModelCategory.Stt);
```

--------------------------------

### Quick Start STT Engine Usage

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/stt.md

Demonstrates the basic workflow of creating an STT engine, transcribing a WAV file, and cleaning up native resources.

```typescript
import { createSTT } from 'react-native-sherpa-onnx/stt';

// 1) Create an STT engine (auto-detects model type from files)
const stt = await createSTT({
  modelPath: { type: 'asset', path: 'models/sherpa-onnx-whisper-tiny-en' },
  modelType: 'auto',
  preferInt8: true,
});

// 2) Transcribe a WAV file (16 kHz mono recommended)
const result = await stt.transcribeFile('/path/to/audio.wav');
console.log('Text:', result.text);

// 3) Cleanup
await stt.destroy();
```

--------------------------------

### Quick Start Streaming STT Implementation

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/stt-streaming.md

Demonstrates the complete lifecycle of a streaming STT session: initializing the engine, creating a stream, processing audio chunks, and releasing resources.

```typescript
import { createStreamingSTT } from 'react-native-sherpa-onnx/stt';

// 1) Create streaming engine
const engine = await createStreamingSTT({
  modelPath: { type: 'asset', path: 'models/streaming-zipformer-en' },
  modelType: 'transducer', // or 'auto' to detect
});

// 2) Create a stream (one per recognition session)
const stream = await engine.createStream();

// 3) Feed audio and get results
const { result, isEndpoint } = await stream.processAudioChunk(samples, 16000);
console.log('Partial:', result.text);
if (isEndpoint) console.log('Utterance ended');

// 4) Cleanup
await stream.release();
await engine.destroy();
```

--------------------------------

### getModelById usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example call to getModelById for a speech-to-text model.

```TypeScript
const model = await getModelById(ModelCategory.Stt, 'sherpa-onnx-whisper-tiny');
```

--------------------------------

### listModels usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example call to listModels for the Alignment category.

```TypeScript
const models = await listModels(ModelCategory.Alignment);
```

--------------------------------

### deleteIncompleteDownload usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example of deleting partial download artifacts and state.

```ts
await deleteIncompleteDownload(ModelCategory.Stt, 'sherpa-onnx-whisper-tiny');
```

--------------------------------

### Run the build script

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/third_party/shine_prebuilt/README.md

Set the Android NDK path and API level, then execute the build script from the third_party/shine_prebuilt directory.

```bash
export ANDROID_NDK_ROOT="C:/path/to/android-ndk"
export ANDROID_API=24
cd third_party/shine_prebuilt
./build_shine_msys2.sh
```

--------------------------------

### Initialize FFmpeg submodule

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/third_party/ffmpeg_prebuilt/BUILD_MSYS2.md

Initialize and update the FFmpeg submodule if not already done, then return to the ffmpeg_prebuilt directory.

```bash
cd /c/path/to/react-native-sherpa-onnx-core
git submodule update --init third_party/ffmpeg
cd third_party/ffmpeg_prebuilt
```

--------------------------------

### pauseDownload usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example of pausing an active download while retaining partial bytes.

```ts
await pauseDownload(ModelCategory.Tts, 'vits-piper-en_US-lessac-medium');
```

--------------------------------

### refreshModels usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Example call to refreshModels for a text-to-speech category with forceRefresh enabled.

```TypeScript
await refreshModels(ModelCategory.Tts, { forceRefresh: true });
```

--------------------------------

### onProgress usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Demonstrates subscribing to progress updates and logging the phase and percentage.

```ts
const unsubscribe = onProgress((category, modelId, progress) => {
  console.log(category, modelId, progress.phase, progress.percent);
});
```

--------------------------------

### Install dependencies

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/CONTRIBUTING.md

Run this command in the root directory to install dependencies for all packages in the Yarn workspace.

```sh
yarn
```

--------------------------------

### checkDiskSpace usage example

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/download-manager.md

Demonstrates checking disk space and handling a failure result.

```ts
const disk = await checkDiskSpace(750 * 1024 * 1024);
if (!disk.success) console.log(disk.message);
```

--------------------------------

### Quick Start TTS Implementation

Source: https://github.com/xdcobra/react-native-sherpa-onnx/blob/main/docs/tts.md

A complete workflow for initializing the TTS engine, generating speech from text, saving the output to a file, and cleaning up resources.

```typescript
import { createTTS, saveAudioToFile } from 'react-native-sherpa-onnx/tts';

// 1) Create TTS engine
const tts = await createTTS({
  modelPath: { type: 'asset', path: 'models/sherpa-onnx-vits-piper-en' },
  modelType: 'auto',
  numThreads: 2,
});

// 2) Generate speech
const audio = await tts.generateSpeech('Hello, world!');
console.log('sampleRate:', audio.sampleRate, 'samples:', audio.samples.length);

// 3) Save to file
await saveAudioToFile(audio, '/path/to/output.wav');

// 4) Cleanup
await tts.destroy();
```
Tokens:
Raw