### Start Example App Packager Source: https://github.com/mybigday/whisper.rn/blob/main/CONTRIBUTING.md Start the Metro server for the example application to test your changes. ```sh yarn example start ``` -------------------------------- ### Install iOS Pods for Example Source: https://github.com/mybigday/whisper.rn/blob/main/AGENTS.md Install CocoaPods dependencies for the example application. ```bash yarn example pods ``` -------------------------------- ### Setup Project Source: https://github.com/mybigday/whisper.rn/blob/main/AGENTS.md Execute this command to set up the project, including installing dependencies and building the whisper.cpp submodule. ```bash yarn bootstrap ``` -------------------------------- ### Run Example App on iOS Source: https://github.com/mybigday/whisper.rn/blob/main/CONTRIBUTING.md Install iOS dependencies and run the example application on an iOS simulator or device. ```sh yarn example pods yarn example ios ``` -------------------------------- ### Run Example App Source: https://github.com/mybigday/whisper.rn/blob/main/CLAUDE.md Runs the example application on Android or iOS. ```bash yarn example android ``` ```bash yarn example ios ``` -------------------------------- ### Install Dependencies Source: https://github.com/mybigday/whisper.rn/blob/main/AGENTS.md Run this command to install project dependencies. ```bash yarn ``` -------------------------------- ### start Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/realtime_transcription.AudioStreamInterface.md Starts the audio stream recording. This method should be called after initialization. ```APIDOC ## start ### Description Starts the audio stream recording. This method should be called after initialization. ### Method start ### Returns - Promise ### Defined in realtime-transcription/types.ts:23 ``` -------------------------------- ### Run Example App on Android Source: https://github.com/mybigday/whisper.rn/blob/main/CONTRIBUTING.md Build and run the example application on an Android device or emulator. ```sh yarn example android ``` -------------------------------- ### Install Dependencies Source: https://github.com/mybigday/whisper.rn/blob/main/CONTRIBUTING.md Run these commands in the root directory to install project dependencies and bootstrap the packages. ```sh yarn yarn bootstrap ``` -------------------------------- ### Run Example App in Release Mode Source: https://github.com/mybigday/whisper.rn/blob/main/CONTRIBUTING.md To test performance, run the example app in Release mode for iOS or Android. ```sh yarn example ios --mode Release ``` ```sh yarn example android --mode release ``` -------------------------------- ### start Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/realtime_transcription.RealtimeTranscriber.md Starts the real-time transcription process. ```APIDOC ## start ### Description Start realtime transcription. ### Returns `Promise` - A promise that resolves when transcription has started. ``` -------------------------------- ### initialize Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/realtime_transcription.AudioStreamInterface.md Initializes the audio stream with the provided configuration. This method should be called before starting the recording. ```APIDOC ## initialize ### Description Initializes the audio stream with the provided configuration. ### Method initialize ### Parameters #### Path Parameters - **config** (AudioStreamConfig) - Required - Configuration object for the audio stream. ### Returns - Promise ### Defined in realtime-transcription/types.ts:22 ``` -------------------------------- ### onSpeechStart Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/realtime_transcription.RingBufferVad.md Registers a callback function to be called when speech starts. ```APIDOC ## onSpeechStart ### Description Registers a callback function to be called when speech starts. ### Parameters #### Path Parameters - **callback** ((confidence: number, data: Uint8Array) => void) - Required - The callback function to execute when speech starts. ### Returns `void` ### Implementation of RealtimeVadContextLike.onSpeechStart ### Defined in realtime-transcription/RingBufferVad.ts:86 ``` -------------------------------- ### installJsi Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/modules/index.md Installs the JavaScript Interface (JSI) for Whisper.rn. This is an asynchronous operation. ```APIDOC ## installJsi ### Description Installs the JSI for Whisper.rn. ### Returns `Promise` - A Promise that resolves when the JSI is installed. ``` -------------------------------- ### Install whisper.rn Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Install the whisper.rn package using npm. This is the initial step for integrating the library into your React Native project. ```sh npm install whisper.rn ``` -------------------------------- ### JSI and Logging Functions Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/modules/index.md Functions for installing the JavaScript Interface (JSI) and managing native logging. ```APIDOC ## installJsi ### Description Installs the JavaScript Interface (JSI) for Whisper.rn, enabling communication between JavaScript and native modules. ### Function Signature `installJsi(): void` --- ## addNativeLogListener ### Description Adds a listener for native log messages. This is useful for debugging. ### Function Signature `addNativeLogListener(listener: (message: string) => void): void` ### Parameters #### Parameters - **listener** (function) - A callback function that will be called with each log message received from the native side. --- ## toggleNativeLog ### Description Toggles the native logging feature on or off. ### Function Signature `toggleNativeLog(enabled: boolean): void` ### Parameters #### Parameters - **enabled** (boolean) - Set to `true` to enable native logging, `false` to disable it. ``` -------------------------------- ### Initialize Whisper Context for Transcription Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Initialize the Whisper context by providing the path to the Whisper model file. This setup is required before performing any transcription tasks. ```javascript import { initWhisper } from 'whisper.rn' const whisperContext = await initWhisper({ filePath: 'file://.../ggml-tiny.en.bin', }) ``` -------------------------------- ### Access whisper.rn Runtime Constants Source: https://context7.com/mybigday/whisper.rn/llms.txt Demonstrates how to access read-only module-level constants like `libVersion`, `isUseCoreML`, and `isCoreMLAllowFallback` to get information about the loaded native runtime. ```typescript import { libVersion, isUseCoreML, isCoreMLAllowFallback } from 'whisper.rn' console.log('whisper.cpp version:', libVersion) // => whisper.cpp version: 1.8.4 console.log('Core ML active:', isUseCoreML) // => Core ML active: true (iOS with Core ML model present) console.log('Core ML CPU fallback allowed:', isCoreMLAllowFallback) // => Core ML CPU fallback allowed: true ``` -------------------------------- ### Manual iOS Framework Build Source: https://github.com/mybigday/whisper.rn/blob/main/AGENTS.md Manually build the iOS framework using the provided script. ```bash ./scripts/build-ios.sh ``` -------------------------------- ### Build iOS Frameworks Source: https://github.com/mybigday/whisper.rn/blob/main/AGENTS.md Build the rnwhisper.xcframework for iOS and tvOS. ```bash yarn build:ios-frameworks ``` -------------------------------- ### Generate API Documentation Source: https://github.com/mybigday/whisper.rn/blob/main/AGENTS.md Generate API documentation using typedoc. ```bash yarn docgen ``` -------------------------------- ### Initialize Whisper with Core ML Support (iOS) Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Demonstrates how to initialize the Whisper context, specifying Core ML model assets for iOS. This requires providing the filename and individual asset paths for the Core ML model components. ```javascript const whisperContext = await initWhisper({ filePath: require('../assets/ggml-tiny.en.bin') coreMLModelAsset: Platform.OS === 'ios' ? { filename: 'ggml-tiny.en-encoder.mlmodelc', assets: [ require('../assets/ggml-tiny.en-encoder.mlmodelc/weights/weight.bin'), require('../assets/ggml-tiny.en-encoder.mlmodelc/model.mil'), require('../assets/ggml-tiny.en-encoder.mlmodelc/coremldata.bin'), ], } : undefined, }) ``` -------------------------------- ### constructor Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/realtime_transcription.RealtimeTranscriber.md Initializes a new instance of the RealtimeTranscriber class. ```APIDOC ## constructor ### Description Initializes a new instance of the RealtimeTranscriber class. ### Parameters #### Path Parameters - **dependencies** (RealtimeTranscriberDependencies) - Required - Dependencies for the transcriber. - **options** (RealtimeOptions) - Optional - Configuration options for the transcriber. - **callbacks** (RealtimeTranscriberCallbacks) - Optional - Callback functions for transcription events. ``` -------------------------------- ### Process Transcription Segments Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Iterates through transcription segments and logs their start time, end time, and duration. Assumes 'segments' is an array of segment objects with 't0' and 't1' properties. ```typescript segments.forEach((segment, index) => { console.log( `Segment ${index + 1}: ${segment.t0.toFixed(2)}s - ${segment.t1.toFixed( 2, )}s`, ) console.log(`Duration: ${(segment.t1 - segment.t0).toFixed(2)}s`) }) ``` -------------------------------- ### Initialize and Use RealtimeTranscriber Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Sets up and manages a realtime transcription session using `RealtimeTranscriber`. Requires Whisper and VAD contexts, an audio stream adapter, and a filesystem module. Configure transcription options and event handlers for transcription, VAD events, status changes, and errors. ```javascript // If your RN packager is not enable package exports support, use whisper.rn/src/realtime-transcription import { RealtimeTranscriber } from 'whisper.rn/realtime-transcription' import { AudioPcmStreamAdapter } from 'whisper.rn/realtime-transcription/adapters' import RNFS from 'react-native-fs' // or any compatible filesystem // Dependencies const whisperContext = await initWhisper({ /* ... */ }) const vadContext = await initWhisperVad({ /* ... */ }) const audioStream = new AudioPcmStreamAdapter() // requires @fugood/react-native-audio-pcm-stream // Create transcriber const transcriber = new RealtimeTranscriber( { whisperContext, vadContext, audioStream, fs: RNFS }, { audioSliceSec: 30, vadPreset: 'default', autoSliceOnSpeechEnd: true, transcribeOptions: { language: 'en' }, }, { onTranscribe: (event) => console.log('Transcription:', event.data?.result), onVad: (event) => console.log('VAD:', event.type, event.confidence), onStatusChange: (isActive) => console.log('Status:', isActive ? 'ACTIVE' : 'INACTIVE'), onError: (error) => console.error('Error:', error), }, ) // Start/stop transcription await transcriber.start() await transcriber.stop() ``` -------------------------------- ### Use VAD Presets with whisper.rn Source: https://context7.com/mybigday/whisper.rn/llms.txt Demonstrates how to import and use pre-configured VAD presets like 'default' or 'meeting' with `initWhisperVad` and `detectSpeech`. Ensure the VAD model file is correctly specified. ```typescript import { VAD_PRESETS } from 'whisper.rn/realtime-transcription' console.log(VAD_PRESETS.default) // => { threshold: 0.5, minSpeechDurationMs: 250, minSilenceDurationMs: 100, // maxSpeechDurationS: 30, speechPadMs: 30, samplesOverlap: 0.1 } // Available presets and their intended use cases: const presets = { 'default': 'Balanced — general purpose', 'sensitive': 'Quiet environments, soft speech', 'very-sensitive': 'Whisper detection, very quiet rooms', 'conservative': 'Reduces false positives in moderate noise', 'very-conservative':'Only clear, loud speech passes', 'continuous': 'Presentations, lectures (60s max segments)', 'meeting': 'Multi-speaker meetings (45s max segments)', 'noisy': 'High-noise environments (0.75 threshold)', } // Use a preset directly with initWhisperVad + detectSpeech import { initWhisperVad } from 'whisper.rn' const vadCtx = await initWhisperVad({ filePath: 'file:///path/to/ggml-silero-v6.2.0.bin' }) const segments = await vadCtx.detectSpeech('file:///audio.wav', VAD_PRESETS.meeting) console.log(`Detected ${segments.length} speech segments in meeting audio`) ``` -------------------------------- ### initWhisperVad(options) Source: https://context7.com/mybigday/whisper.rn/llms.txt Initializes a Voice Activity Detection (VAD) context by loading the Silero VAD model. This context can be used for detecting speech segments in audio. ```APIDOC ## initWhisperVad(options) — Initialize a VAD context Loads the Silero VAD model and returns a `WhisperVadContext` for detecting speech segments in audio. The VAD context can be used standalone for speech detection or paired with `RealtimeTranscriber` for stream-based speech segmentation. ### Parameters - **options** (object) - Configuration options for initializing the VAD context. - **filePath** (string | number) - Path to the VAD model file. Can be a file path, an asset reference, or a URL. - **useGpu** (boolean) - Whether to use GPU acceleration (iOS only). Defaults to `true`. - **nThreads** (number) - The number of threads to use. Defaults to auto-detected based on CPU cores. - **isBundleAsset** (boolean) - Indicates if the `filePath` points to a bundled asset. Defaults to `false`. ### Returns - **Promise** - A promise that resolves with the initialized `WhisperVadContext` object. ``` -------------------------------- ### Run Unit Tests Source: https://github.com/mybigday/whisper.rn/blob/main/CONTRIBUTING.md Execute the project's unit tests using Jest. ```sh yarn test ``` -------------------------------- ### Core ML Model File Structure Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Lists the required and optional files within a Core ML model directory (`.mlmodelc`). Ensure `model.mil`, `coremldata.bin`, and `weights/weight.bin` are present for Core ML functionality. ```json5 [ 'model.mil', 'coremldata.bin', 'weights/weight.bin', // Not required: // 'metadata.json', 'analytics/coremldata.bin', ] ``` -------------------------------- ### Initialize Silero VAD Context Source: https://context7.com/mybigday/whisper.rn/llms.txt Use `initWhisperVad` to load the Silero VAD model for speech detection. Supports GPU acceleration on iOS and configurable thread counts. ```typescript import { initWhisperVad } from 'whisper.rn' const vadContext = await initWhisperVad({ filePath: require('./assets/ggml-silero-v6.2.0.bin'), useGpu: true, // GPU acceleration (iOS only), default: true nThreads: 4, // Thread count (default: auto based on CPU cores) isBundleAsset: false, }) console.log('VAD GPU active:', vadContext.gpu) ``` -------------------------------- ### constructor Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/realtime_transcription.RingBufferVad.md Initializes a new instance of the RingBufferVad class. ```APIDOC ## constructor ### Description Initializes a new instance of the RingBufferVad class. ### Parameters #### Path Parameters - **vadContext** (WhisperVadContextLike) - Required - The VAD context. - **options** (RingBufferVadOptions) - Optional - The options for the RingBufferVad. ### Defined in realtime-transcription/RingBufferVad.ts:51 ``` -------------------------------- ### initWhisper(options) Source: https://context7.com/mybigday/whisper.rn/llms.txt Loads a GGML Whisper model and initializes a WhisperContext for transcription. Supports GPU acceleration and Core ML on iOS. ```APIDOC ## initWhisper(options) ### Description Loads a GGML Whisper model from disk or a bundled asset and returns a `WhisperContext` instance ready for transcription. GPU acceleration and Core ML are enabled by default where available. ### Parameters #### Options Object - **filePath** (string | number) - Required - Path to the GGML model file or a require() reference to a bundled asset. - **useGpu** (boolean) - Optional - Enables GPU acceleration. Defaults to true. (iOS only) - **useCoreMLIos** (boolean) - Optional - Enables Core ML encoder on iOS. Defaults to true. - **useFlashAttn** (boolean) - Optional - Enables Flash attention if GPU is available. Defaults to false. - **coreMLModelAsset** (object) - Optional - Configuration for bundled Core ML model assets on iOS. - **filename** (string) - Required - The filename of the Core ML model. - **assets** (array) - Required - An array of require() references to the model's asset files. ### Returns - **WhisperContext** - An object with methods for transcription and properties indicating GPU status. - **gpu** (boolean) - Indicates if GPU acceleration is active. ``` -------------------------------- ### libVersion, isUseCoreML, isCoreMLAllowFallback Source: https://context7.com/mybigday/whisper.rn/llms.txt Read-only module-level constants that provide version and capability information about the loaded native whisper.cpp runtime. ```APIDOC ## libVersion, isUseCoreML, isCoreMLAllowFallback ### Description Read-only module-level constants providing version and capability information about the loaded native runtime. ### Usage ```typescript import { libVersion, isUseCoreML, isCoreMLAllowFallback } from 'whisper.rn' console.log('whisper.cpp version:', libVersion) // => whisper.cpp version: 1.8.4 console.log('Core ML active:', isUseCoreML) // => Core ML active: true (iOS with Core ML model present) console.log('Core ML CPU fallback allowed:', isCoreMLAllowFallback) // => Core ML CPU fallback allowed: true ``` ``` -------------------------------- ### Initialization Functions Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/modules/index.md Functions for initializing and releasing Whisper and Whisper VAD contexts. ```APIDOC ## initWhisper ### Description Initializes the Whisper context. This function must be called before any transcription operations. ### Function Signature `initWhisper(options?: ContextOptions): Promise` ### Parameters #### Parameters - **options** (ContextOptions) - Optional. Configuration options for the Whisper context, including model paths and GPU/Core ML usage. ### Returns A Promise that resolves when the Whisper context is successfully initialized. --- ## initWhisperVad ### Description Initializes the Whisper VAD (Voice Activity Detection) context. This function must be called before using VAD-related features. ### Function Signature `initWhisperVad(options?: VadOptions): Promise` ### Parameters #### Parameters - **options** (VadOptions) - Optional. Configuration options for the Whisper VAD context. ### Returns A Promise that resolves when the Whisper VAD context is successfully initialized. --- ## releaseAllWhisper ### Description Releases all initialized Whisper contexts, freeing up resources. ### Function Signature `releaseAllWhisper(): Promise` ### Returns A Promise that resolves when all Whisper contexts have been released. --- ## releaseAllWhisperVad ### Description Releases all initialized Whisper VAD contexts, freeing up resources. ### Function Signature `releaseAllWhisperVad(): Promise` ### Returns A Promise that resolves when all Whisper VAD contexts have been released. ``` -------------------------------- ### Build TypeScript Library Source: https://github.com/mybigday/whisper.rn/blob/main/AGENTS.md Build the TypeScript library using react-native-builder-bob. ```bash yarn build ``` -------------------------------- ### onData Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/realtime_transcription.AudioStreamInterface.md Registers a callback function to be executed when new audio data is available. ```APIDOC ## onData ### Description Registers a callback function to be executed when new audio data is available. ### Method onData ### Parameters #### Path Parameters - **callback** (data: AudioStreamData) => void - Required - The callback function to handle incoming audio data. ### Returns - void ### Defined in realtime-transcription/types.ts:26 ``` -------------------------------- ### Lint Files Source: https://github.com/mybigday/whisper.rn/blob/main/AGENTS.md Run this command to lint files using ESLint. Add --fix to automatically fix lint errors. ```bash yarn lint ``` ```bash yarn lint --fix ``` -------------------------------- ### Configure Jest Mock for whisper.rn Source: https://context7.com/mybigday/whisper.rn/llms.txt Provides instructions on setting up the Jest mock for `whisper.rn` in `jest.config.js` or inline within a test file. This allows testing code that imports `whisper.rn` without needing a real device or native bridge. ```javascript // jest.config.js or package.json { "jest": { "moduleNameMapper": { "whisper.rn": "/node_modules/whisper.rn/jest-mock" } } } // Or inline in a test file: jest.mock('whisper.rn', () => require('whisper.rn/jest-mock')) // Now whisper.rn APIs are available as Jest mocks import { initWhisper } from 'whisper.rn' test('transcribes audio', async () => { const ctx = await initWhisper({ filePath: 'file:///dummy.bin' }) const { promise } = ctx.transcribe('file:///dummy.wav', { language: 'en' }) const { result } = await promise expect(typeof result).toBe('string') }) ``` -------------------------------- ### Publish New Versions Source: https://github.com/mybigday/whisper.rn/blob/main/CONTRIBUTING.md Use this command to publish new versions of the package to npm. It handles version bumping, tagging, and release creation. ```sh yarn release ``` -------------------------------- ### Verify Code Quality Source: https://github.com/mybigday/whisper.rn/blob/main/CONTRIBUTING.md Run these commands to ensure your code adheres to TypeScript types and ESLint rules. ```sh yarn typecheck yarn lint ``` -------------------------------- ### Enable and Listen to Native Logs in whisper.rn Source: https://context7.com/mybigday/whisper.rn/llms.txt Shows how to enable forwarding of whisper.cpp logs to JavaScript using `toggleNativeLog` and subscribe to these messages with `addNativeLogListener`. Remember to unsubscribe when finished and disable logging to conserve resources. ```typescript import { toggleNativeLog, addNativeLogListener } from 'whisper.rn' // Enable native log forwarding await toggleNativeLog(true) // Subscribe to log messages const subscription = addNativeLogListener((level, text) => { console.log(`[whisper.cpp][${level}] ${text}`) // => [whisper.cpp][info] whisper_init_from_file_with_params_no_state: loading model from '/path/to/model.bin' // => [whisper.cpp][info] whisper_model_load: model size = 77.11 MB }) const ctx = await initWhisper({ filePath: 'file:///path/to/ggml-tiny.en.bin' }) // Logs will appear above during model loading // Unsubscribe when done subscription.remove() // Disable log forwarding await toggleNativeLog(false) ``` -------------------------------- ### RingBuffer Constructor Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/realtime_transcription.RingBuffer.md Initializes a new RingBuffer with a specified maximum capacity. ```APIDOC ## new RingBuffer(maxBytes) ### Description Creates a new RingBuffer instance. ### Parameters #### Path Parameters - **maxBytes** (number) - Required - Maximum buffer size in bytes ``` -------------------------------- ### WhisperContext.reasonNoGPU Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/index.WhisperContext.md A string property providing the reason why GPU is not available, if applicable. Defaults to an empty string. ```APIDOC ## reasonNoGPU ### Description Provides the reason why GPU is not available, if applicable. ### Property - **reasonNoGPU** (string) - Defaults to `''`. ### Defined in index.ts:233 ``` -------------------------------- ### Initialize Whisper with Asset Model Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Initializes the Whisper context using a model file located in the application's assets. This requires the model file to be correctly referenced using `require`. ```javascript import { initWhisper } from 'whisper.rn' const whisperContext = await initWhisper({ filePath: require('../assets/ggml-tiny.en.bin'), }) ``` -------------------------------- ### initWhisper Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/modules/index.md Initializes a whisper context using a provided GGML model file. This function is asynchronous and returns a Promise that resolves to a WhisperContext instance. ```APIDOC ## initWhisper ### Description Initialize a whisper context with a GGML model file. ### Parameters #### Parameters - **options** ([`ContextOptions`](index.md#contextoptions)) - Options for configuring the Whisper context. ### Returns `Promise` - A Promise that resolves to a WhisperContext instance. ``` -------------------------------- ### RingBuffer Methods Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/realtime_transcription.RingBuffer.md Provides methods to manage and query the state of the RingBuffer. ```APIDOC ## clear() ### Description Clears all data from the buffer and resets its internal indices. ### Returns `void` ## getCapacity() ### Description Retrieves the maximum capacity of the buffer in bytes. ### Returns `number` - The maximum buffer size in bytes. ## getFillRatio() ### Description Calculates and returns the current fill ratio of the buffer. ### Returns `number` - A value between 0 and 1, indicating how full the buffer is. ## getLength() ### Description Returns the current number of bytes stored in the buffer. ### Returns `number` - The number of bytes currently in the buffer. ## isEmpty() ### Description Checks if the buffer currently contains any data. ### Returns `boolean` - `true` if the buffer is empty, `false` otherwise. ## isFull() ### Description Checks if the buffer has reached its maximum capacity. ### Returns `boolean` - `true` if the buffer is full, `false` otherwise. ## read() ### Description Reads all available data from the buffer in the correct chronological order without clearing it. Use `clear()` to empty the buffer. ### Returns `Uint8Array` - A Uint8Array containing the buffered data in order. ## write(data) ### Description Writes audio data into the buffer. If the incoming data exceeds the buffer's capacity, the oldest data in the buffer will be overwritten. ### Parameters #### Path Parameters - **data** (Uint8Array) - Required - The audio data to write to the buffer. ### Returns `void` ``` -------------------------------- ### Initialize Whisper Context Source: https://context7.com/mybigday/whisper.rn/llms.txt Loads a GGML Whisper model. GPU acceleration and Core ML are enabled by default where available. Ensure bundled assets are correctly configured in metro.config.js. ```typescript import { initWhisper } from 'whisper.rn' import { Platform } from 'react-native' // From a local file path const whisperContext = await initWhisper({ filePath: 'file:///path/to/ggml-tiny.en.bin', useGpu: true, // GPU acceleration (iOS only), default: true useCoreMLIos: true, // Use Core ML encoder on iOS, default: true useFlashAttn: false, // Flash attention (only if GPU is available), default: false }) // From a bundled asset (requires metro.config.js to include 'bin' extensions) const whisperContextFromAsset = await initWhisper({ filePath: require('./assets/ggml-tiny.en.bin'), }) // With bundled Core ML model assets (iOS only) const whisperContextWithCoreML = await initWhisper({ filePath: require('./assets/ggml-tiny.en.bin'), coreMLModelAsset: Platform.OS === 'ios' ? { filename: 'ggml-tiny.en-encoder.mlmodelc', assets: [ require('./assets/ggml-tiny.en-encoder.mlmodelc/weights/weight.bin'), require('./assets/ggml-tiny.en-encoder.mlmodelc/model.mil'), require('./assets/ggml-tiny.en-encoder.mlmodelc/coremldata.bin'), ], } : undefined, }) console.log('GPU active:', whisperContextWithCoreML.gpu) // => GPU active: true (if Metal/CoreML is available) ``` -------------------------------- ### onStatusChange Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/realtime_transcription.AudioStreamInterface.md Registers a callback function to be executed when the recording status of the audio stream changes. ```APIDOC ## onStatusChange ### Description Registers a callback function to be executed when the recording status of the audio stream changes. ### Method onStatusChange ### Parameters #### Path Parameters - **callback** (isRecording: boolean) => void - Required - The callback function to handle status changes. ### Returns - void ### Defined in realtime-transcription/types.ts:28 ``` -------------------------------- ### TranscribeFileOptions Properties Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/index.TranscribeFileOptions.md This interface defines the configuration options available when transcribing an audio file. It inherits properties from TranscribeOptions and adds specific callbacks for progress and new segment detection. ```APIDOC ## Interface: TranscribeFileOptions ### Properties - **beamSize** (`number`): Optional. Beam size for beam search. - **bestOf** (`number`): Optional. Number of best candidates to keep. - **duration** (`number`): Optional. Duration of audio to process in milliseconds. - **language** (`string`): Optional. Spoken language (Default: 'auto' for auto-detect). - **maxContext** (`number`): Optional. Maximum number of text context tokens to store. - **maxLen** (`number`): Optional. Maximum segment length in characters. - **maxThreads** (`number`): Optional. Number of threads to use during computation (Default: 2 for 4-core devices, 4 for more cores). - **nProcessors** (`number`): Optional. Number of processors to use for parallel processing with whisper_full_parallel (Default: 1 to use whisper_full). - **offset** (`number`): Optional. Time offset in milliseconds. - **onNewSegments** (`(result: TranscribeNewSegmentsResult) => void`): Optional. Callback when new segments are transcribed. - **onProgress** (`(progress: number) => void`): Optional. Progress callback, the progress is between 0 and 100. - **prompt** (`string`): Optional. Initial Prompt. - **tdrzEnable** (`boolean`): Optional. Enable token-based timestamp log removal. - **temperature** (`number`): Optional. Temperature for sampling (Default: 0.0). - **temperatureInc** (`number`): Optional. Temperature increment for contrastive search (Default: 0.2). - **tokenTimestamps** (`boolean`): Optional. Enable token-level timestamps. - **translate** (`boolean`): Optional. Translate the audio into English. - **wordThold** (`number`): Optional. Word timestamp threshold (Default: 0.5). ``` -------------------------------- ### VAD_PRESETS Source: https://context7.com/mybigday/whisper.rn/llms.txt Provides pre-configured Voice Activity Detection (VAD) parameter sets for various acoustic environments and use cases. These presets can be directly used with `initWhisperVad` and `detectSpeech`. ```APIDOC ## VAD_PRESETS ### Description A collection of ready-made `VadOptions` configurations tuned for different acoustic environments and use cases. ### Usage ```typescript import { VAD_PRESETS } from 'whisper.rn/realtime-transcription' console.log(VAD_PRESETS.default) // => { threshold: 0.5, minSpeechDurationMs: 250, minSilenceDurationMs: 100, // maxSpeechDurationS: 30, speechPadMs: 30, samplesOverlap: 0.1 } // Available presets and their intended use cases: const presets = { 'default': 'Balanced — general purpose', 'sensitive': 'Quiet environments, soft speech', 'very-sensitive': 'Whisper detection, very quiet rooms', 'conservative': 'Reduces false positives in moderate noise', 'very-conservative':'Only clear, loud speech passes', 'continuous': 'Presentations, lectures (60s max segments)', 'meeting': 'Multi-speaker meetings (45s max segments)', 'noisy': 'High-noise environments (0.75 threshold)', } // Use a preset directly with initWhisperVad + detectSpeech import { initWhisperVad } from 'whisper.rn' const vadCtx = await initWhisperVad({ filePath: 'file:///path/to/ggml-silero-v6.2.0.bin' }) const segments = await vadCtx.detectSpeech('file:///audio.wav', VAD_PRESETS.meeting) console.log(`Detected ${segments.length} speech segments in meeting audio`) ``` ``` -------------------------------- ### Transcribe Audio from Asset Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Initiates transcription for an audio file located in the application's assets. The audio file must be referenced using `require`. ```javascript const { stop, promise } = whisperContext.transcribe( require('../assets/sample.wav'), options, ) ``` -------------------------------- ### getSliceForTranscription Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/realtime_transcription.SliceManager.md Retrieves the next available audio slice that is ready for transcription. ```APIDOC ## getSliceForTranscription() ### Description Get a slice for transcription. ### Returns ``null`` | `AudioSlice` - The next AudioSlice ready for transcription, or null if none are available. ``` -------------------------------- ### WhisperContext Constructor Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/index.WhisperContext.md Initializes a new WhisperContext instance. It requires a NativeWhisperContext object. ```APIDOC ## new WhisperContext(«destructured») ### Description Initializes a new WhisperContext instance. ### Parameters #### Path Parameters - **«destructured»** (NativeWhisperContext) - Required - Description of the destructured parameter. #### Defined in index.ts:235 ``` -------------------------------- ### RealtimeOptions Properties Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/realtime_transcription.RealtimeOptions.md Configuration options for realtime transcription. ```APIDOC ## Interface: RealtimeOptions ### Properties - **audioMinSec** (`Optional` `number`): Minimum duration in seconds for audio slices. - **audioOutputPath** (`Optional` `string`): Path to save the transcribed audio output. - **audioSliceSec** (`Optional` `number`): Duration in seconds for audio slices. - **audioStreamConfig** (`Optional` `AudioStreamConfig`): Configuration for the audio stream. - **initRealtimeAfterMs** (`Optional` `number`): Delay in milliseconds before initializing realtime transcription. - **initialPrompt** (`Optional` `string`): An initial prompt to guide the transcription. - **logger** (`Optional` `(message: string) => void`): A function for logging messages. - **maxSlicesInMemory** (`Optional` `number`): Maximum number of audio slices to keep in memory. - **promptPreviousSlices** (`Optional` `boolean`): Whether to use previous slices as context for prompting. - **realtimeProcessingPauseMs** (`Optional` `number`): Pause duration in milliseconds for realtime processing. - **transcribeOptions** (`Optional` `TranscribeOptions`): Options for the transcription process. ``` -------------------------------- ### Modify NDK Build Script for Apple Silicon Source: https://github.com/mybigday/whisper.rn/blob/main/docs/TROUBLESHOOTING.md When encountering 'Unknown host CPU architecture: arm64' on Apple Silicon Macs, modify the ndk-build script to prepend 'arch -x86_64 /bin/bash'. This ensures compatibility with the NDK build process. ```bash #!/bin/sh DIR="$(cd "$(dirname "$0")" && pwd)" arch -x86_64 /bin/bash $DIR/build/ndk-build "$@" ``` -------------------------------- ### WhisperContext.transcribe(filePathOrBase64, options) Source: https://context7.com/mybigday/whisper.rn/llms.txt Transcribes an audio file from a given path, asset, or base64 string. Provides progress updates and allows for cancellation. ```APIDOC ## WhisperContext.transcribe(filePathOrBase64, options) ### Description Transcribes an audio file given a local file path, a bundled asset reference, or a base64-encoded WAV string. Returns an object with a `stop()` function for cancellation and a `promise` that resolves to the full transcription result. ### Parameters #### filePathOrBase64 - **filePathOrBase64** (string | number) - Required - Path to the audio file, a require() reference to a bundled audio asset, or a base64-encoded WAV string. #### Options Object - **language** (string) - Optional - Source language for transcription ('auto' for auto-detect). Defaults to 'auto'. - **translate** (boolean) - Optional - If true, translates the transcription to English. Defaults to false. - **maxThreads** (number) - Optional - Number of CPU threads to use for transcription. - **tokenTimestamps** (boolean) - Optional - If true, enables word-level timestamps. Defaults to false. - **prompt** (string) - Optional - An initial prompt to guide the transcription style. - **onProgress** (function) - Optional - Callback function that receives transcription progress percentage. - **onNewSegments** (function) - Optional - Callback function that receives new transcription segments as they are decoded. - **result** (string) - The current full transcription result. - **segments** (array) - An array of newly decoded segments. ### Returns - **object** - An object containing: - **stop** (function) - A function to cancel the transcription. - **promise** (Promise) - A promise that resolves with the transcription result. ### Transcription Result Object (from promise) - **result** (string) - The full transcribed text. - **segments** (array) - An array of transcribed segments, each with `t0`, `t1`, and `text` properties. - **language** (string) - The detected language of the source audio. - **isAborted** (boolean) - True if the transcription was cancelled. ``` -------------------------------- ### Initialize VAD Context Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Initializes the Voice Activity Detection (VAD) context with a specified model file. This function is asynchronous and returns a context object for VAD operations. ```APIDOC ## initWhisperVad ### Description Initializes the Voice Activity Detection (VAD) context with a specified model file. ### Method `initWhisperVad(options: { filePath: string | number, useGpu?: boolean, nThreads?: number }) => Promise` ### Parameters #### Path Parameters - **filePath** (string | number) - Required - The path to the VAD model file (e.g., require('./assets/ggml-silero-v6.2.0.bin')). - **useGpu** (boolean) - Optional - Whether to use GPU acceleration (iOS only). - **nThreads** (number) - Optional - The number of threads to use for processing. ``` -------------------------------- ### AudioStreamConfig Properties Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/realtime_transcription.AudioStreamConfig.md This interface outlines the configurable properties for an audio stream used in real-time transcription. All properties are optional. ```APIDOC ## Interface: AudioStreamConfig ### Properties - **sampleRate** (number) - Optional - The sample rate of the audio stream. - **channels** (number) - Optional - The number of audio channels. - **bufferSize** (number) - Optional - The size of the audio buffer. - **bitsPerSample** (number) - Optional - The number of bits per sample. - **audioSource** (number) - Optional - The audio source identifier. ``` -------------------------------- ### exists Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/realtime_transcription.WavFileWriterFs.md Checks if a file exists at the specified file path. ```APIDOC ## exists ### Description Checks if a file exists at the specified file path. ### Method Signature `exists(filePath: string): Promise` ### Parameters - **filePath** (string) - The path to the file to check. ### Returns A Promise that resolves with a boolean indicating whether the file exists. ``` -------------------------------- ### addAudioData Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/realtime_transcription.SliceManager.md Adds audio data to the current slice. If the slice is full, it may be finalized. ```APIDOC ## addAudioData(audioData) ### Description Add audio data to the current slice. ### Parameters #### Path Parameters - **audioData** (Uint8Array) - Required - The audio data to add. ### Returns `Object` - An object that may contain the finalized slice. #### Success Response - **slice** (AudioSlice) - Optional - The finalized audio slice, if any. ``` -------------------------------- ### detectSpeech Method Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/index.WhisperVadContext.md Detects speech segments in an audio file, which can be provided as a file path or a base64 encoded WAV string. Options for detection can be passed as an argument. ```APIDOC ## Method ### detectSpeech(filePathOrBase64, options?) Detect speech segments in audio file (path or base64 encoded wav file) base64: need add `data:audio/wav;base64,` prefix #### Parameters - **filePathOrBase64** (string | number) - The path to the audio file or base64 encoded audio data. - **options** (VadOptions) - Optional configuration for speech detection. #### Returns `Promise` - A promise that resolves to an array of detected speech segments. ``` -------------------------------- ### JSI Bindings - Detect Speech Source: https://github.com/mybigday/whisper.rn/blob/main/CLAUDE.md High-performance JSI function for detecting speech in audio data using the VAD model. Enables efficient, direct memory transfer. ```cpp whisperVadDetectSpeech ``` -------------------------------- ### TranscribeFileOptions Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/index.TranscribeFileOptions.md Options for configuring file transcription. ```APIDOC ## TranscribeFileOptions ### Description Options for configuring file transcription. ### Properties #### `translate` (boolean) - Optional Translate from source language to english (Default: false) #### `tdrzEnable` (boolean) - Optional Enable tinydiarize (requires a tdrz model) #### `wordThold` (number) - Optional Word timestamp probability threshold #### `tokenTimestamps` (boolean) - Optional Enable token-level timestamps #### `temperature` (number) - Optional Initial decoding temperature #### `temperatureInc` (number) - Optional Temperature fallback increment applied between decoding retries ``` -------------------------------- ### WhisperContext.bench(maxThreads) Source: https://context7.com/mybigday/whisper.rn/llms.txt Benchmarks the loaded model's encoder and decoder performance on the device, returning timing metrics. This is useful for determining optimal thread counts for specific devices. ```APIDOC ## WhisperContext.bench(maxThreads) — Benchmark model performance Runs a benchmark of the loaded model's encoder and decoder on the device and returns timing metrics. Useful for choosing optimal thread counts for a specific device. ### Parameters - **maxThreads** (number) - The maximum number of threads to use for benchmarking. ### Returns - **Promise** - A promise that resolves with an object containing benchmark results: - **config** (string) - The model configuration name. - **nThreads** (number) - The number of threads used. - **encodeMs** (number) - Average time in milliseconds for encoder processing. - **decodeMs** (number) - Average time in milliseconds for decoder processing. - **batchMs** (number) - Average time in milliseconds for batch processing. - **promptMs** (number) - Average time in milliseconds for prompt processing. ``` -------------------------------- ### initWhisperVad Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/modules/index.md Initializes a Voice Activity Detection (VAD) context. This function is asynchronous and returns a Promise that resolves to a WhisperVadContext instance. ```APIDOC ## initWhisperVad ### Description Initialize a VAD context for voice activity detection. ### Parameters #### Parameters - **options** ([`VadContextOptions`](index.md#vadcontextoptions)) - Options for configuring the VAD context. ### Returns `Promise` - A Promise that resolves to a WhisperVadContext instance. ``` -------------------------------- ### WhisperVadContext Constructor Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/classes/index.WhisperVadContext.md Initializes a new instance of the WhisperVadContext class. It requires a destructured object of type NativeWhisperVadContext. ```APIDOC ## Constructor ### new WhisperVadContext(«destructured») #### Parameters - **«destructured»** (NativeWhisperVadContext) - Description of the destructured parameter. ``` -------------------------------- ### RealtimeVadEvent Properties Source: https://github.com/mybigday/whisper.rn/blob/main/docs/API/interfaces/realtime_transcription.RealtimeVadEvent.md This snippet details the properties of the RealtimeVadEvent interface, which represents events detected by the Voice Activity Detector. ```APIDOC ## Interface: RealtimeVadEvent Represents an event detected by the Voice Activity Detector (VAD) during realtime transcription. ### Properties - **timestamp** (number) - The timestamp of the event. - **lastSpeechDetectedTime** (number) - The timestamp of the last detected speech segment. - **confidence** (number) - The confidence score of the VAD detection. - **duration** (number) - The duration of the detected audio segment (e.g., speech or silence). - **sliceIndex** (number) - The index of the current audio slice. - **type** (string) - The type of VAD event. Possible values are: "speech_start", "speech_end", "speech_continue", "silence". - **analysis** (Object) - Optional. Contains detailed audio analysis for the event. - **averageAmplitude** (number) - The average amplitude of the audio segment. - **peakAmplitude** (number) - The peak amplitude of the audio segment. - **spectralCentroid?** (number) - Optional. The spectral centroid of the audio segment. - **zeroCrossingRate?** (number) - Optional. The zero-crossing rate of the audio segment. - **currentThreshold** (number) - Optional. The current VAD threshold. - **environmentNoise** (number) - Optional. Estimated level of background noise. ``` -------------------------------- ### RingBufferVad: VAD Wrapper with Pre-recording Source: https://context7.com/mybigday/whisper.rn/llms.txt The `RingBufferVad` wraps a `WhisperVadContext` to provide the `RealtimeVadContextLike` interface. It handles audio accumulation, VAD inference, and speech event firing, including pre-speech audio via its internal `RingBuffer`. ```typescript import { initWhisperVad } from 'whisper.rn' import { RingBufferVad, VAD_PRESETS } from 'whisper.rn/realtime-transcription' const vadCtx = await initWhisperVad({ filePath: 'file:///path/to/ggml-silero-v6.2.0.bin' }) const ringVad = new RingBufferVad(vadCtx, { vadPreset: 'sensitive', // Pre-defined preset (overrides vadOptions if set) vadOptions: VAD_PRESETS.sensitive, // Manual VAD parameters preRecordingBufferMs: 1000, // Pre-speech audio retained in ms (default: 1000) sampleRate: 16000, // Must match your audio stream (default: 16000) inferenceIntervalMs: 96, // VAD inference batch size in ms (default: 96ms) logger: (msg) => console.debug('[VAD]', msg), }) // Register event handlers ringVad.onSpeechStart((confidence, preRollData) => { console.log(`Speech started (confidence=${confidence.toFixed(2)})`) console.log(`Pre-roll audio included: ${preRollData.length} bytes`) }) ringVad.onSpeechContinue((confidence, data) => { console.log(`Speech continues (confidence=${confidence.toFixed(2)})`) }) ringVad.onSpeechEnd((confidence) => { console.log(`Speech ended (confidence=${confidence.toFixed(2)})`) }) ringVad.onError((err) => console.error('VAD error:', err)) // Feed raw PCM audio from your microphone stream ringVad.processAudio(new Uint8Array(3200)) // 100ms chunk at 16kHz // Flush remaining buffered audio at end of stream await ringVad.flush() // Reset for reuse await ringVad.reset() ``` -------------------------------- ### Initialize VAD Context for Speech Detection Source: https://github.com/mybigday/whisper.rn/blob/main/README.md Initialize the Whisper VAD context, specifying the path to the VAD model file and optional parameters like GPU usage and number of threads. This context is used for detecting speech segments. ```typescript import { initWhisperVad } from 'whisper.rn' const vadContext = await initWhisperVad({ filePath: require('./assets/ggml-silero-v6.2.0.bin'), // VAD model file useGpu: true, // Use GPU acceleration (iOS only) nThreads: 4, // Number of threads for processing }) ```