### Quick Start with Script Tags for Browser VAD Source: https://docs.vad.ricky0123.com/user-guide/browser This example demonstrates how to quickly integrate the voice activity detector (VAD) into a web page using script tags. It loads the necessary ONNX Runtime Web and VAD web bundles from CDNs. The `onSpeechEnd` callback captures audio data when speech is detected, and `start()` initiates the VAD process. Ensure correct paths for `onnxWASMBasePath` and `baseAssetPath` are provided. ```javascript ``` -------------------------------- ### Install @ricky0123/vad-web via NPM Source: https://docs.vad.ricky0123.com/user-guide/browser This command installs the `@ricky0123/vad-web` package using NPM, a common package manager for Node.js projects. This allows for managing the VAD library as a project dependency. ```bash npm i @ricky0123/vad-web ``` -------------------------------- ### Webpack Configuration for Serving VAD Assets Source: https://docs.vad.ricky0123.com/user-guide/browser This Webpack configuration example utilizes the `copy-webpack-plugin` to ensure that necessary VAD and ONNX Runtime Web files are copied to the output directory during the build process. This is crucial when you choose to serve these assets locally instead of relying on CDNs. Ensure `copy-webpack-plugin` is installed (`npm i -D copy-webpack-plugin`). ```javascript const CopyPlugin = require("copy-webpack-plugin") module.exports = { // ... plugins: [ // ... new CopyPlugin({ patterns: [ // ... { from: "node_modules/@ricky0123/vad-web/dist/vad.worklet.bundle.min.js", to: "[name][ext]", }, { from: "node_modules/@ricky0123/vad-web/dist/*.onnx", to: "[name][ext]", }, { from: "node_modules/onnxruntime-web/dist/*.wasm", to: "[name][ext]" }, { from: "node_modules/onnxruntime-web/dist/*.mjs", to: "[name][ext]" }, ], }), ], } ``` -------------------------------- ### Install Vad-React Package Source: https://docs.vad.ricky0123.com/user-guide/react Install the Vad-React package using npm. This is the first step to integrate voice activity detection into your React application. ```bash npm i @ricky0123/vad-react ``` -------------------------------- ### Install Dependencies and Build Project with npm Source: https://docs.vad.ricky0123.com/developer-guide/hacking Commands to set up the development environment by installing project dependencies and building all packages using npm. These commands should be executed from the root of the repository. ```shell npm install npm run build ``` -------------------------------- ### Initialize and Use MicVAD with Callbacks (JavaScript) Source: https://docs.vad.ricky0123.com/user-guide/api Demonstrates how to initialize the MicVAD API with custom callbacks for speech events and start processing audio. It requires the '@ricky0123/vad-web' package and returns audio data on speech end. ```javascript import { MicVAD } from "@ricky0123/vad-web" const myvad = await MicVAD.new({ onSpeechEnd: (audio) => { // do something with `audio` (Float32Array of audio samples at sample rate 16000)... }, }) myvad.start() ``` -------------------------------- ### Customize Audio Stream Constraints with MicVAD Source: https://docs.vad.ricky0123.com/user-guide/browser This example demonstrates how to override the default microphone stream acquisition in MicVAD by providing a custom 'getStream' function. This allows for specific audio constraints such as channel count, sample rate, and disabling features like echo cancellation, auto gain control, and noise suppression. The onSpeechEnd callback is also shown for processing audio data upon speech detection. ```javascript import { MicVAD } from "@ricky0123/vad-web" const myvad = await MicVAD.new({ getStream: async () => { return await navigator.mediaDevices.getUserMedia({ audio: { channelCount: 2, // Stereo instead of mono echoCancellation: false, // Disable echo cancellation autoGainControl: false, // Disable auto gain control noiseSuppression: false, // Disable noise suppression sampleRate: 48000, // Custom sample rate }, }) }, onSpeechEnd: (audio) => { // do something with `audio` (Float32Array of audio samples at sample rate 16000)... }, }) myvad.start() ``` -------------------------------- ### Configure Browser VAD with Custom Asset Paths Source: https://docs.vad.ricky0123.com/user-guide/browser This JavaScript code snippet shows how to initialize the VAD when serving model and runtime files locally. By setting `baseAssetPath` and `onnxWASMBasePath`, you can specify the locations from which the VAD and ONNX Runtime Web assets should be loaded. The `onSpeechEnd` callback is used to process detected speech audio. ```javascript import { MicVAD } from "@ricky0123/vad-web" const myvad = await MicVAD.new({ baseAssetPath: "/", // or whatever you want onnxWASMBasePath: "/", // or whatever you want onSpeechEnd: (audio) => { // do something with `audio` (Float32Array of audio samples at sample rate 16000)... }, }) myvad.start() ``` -------------------------------- ### Custom Audio Configuration with useMicVAD Source: https://docs.vad.ricky0123.com/user-guide/react Configure custom audio stream settings for the useMicVAD hook. This allows for advanced control over audio constraints like channel count, echo cancellation, and noise suppression. ```javascript import { useMicVAD } from "@ricky0123/vad-react" const MyComponent = () => { const vad = useMicVAD({ getStream: async () => { return await navigator.mediaDevices.getUserMedia({ audio: { channelCount: 2, // Stereo echoCancellation: false, autoGainControl: false, noiseSuppression: false, }, }) }, onSpeechEnd: (audio) => { console.log("User stopped talking") }, }) return
{vad.userSpeaking && "User is speaking"}
} ``` -------------------------------- ### Use useMicVAD Hook in React Source: https://docs.vad.ricky0123.com/user-guide/react Utilize the useMicVAD hook from the Vad-React package to enable voice activity detection. This hook provides the `userSpeaking` state and an `onSpeechEnd` callback for handling speech events. ```javascript import { useMicVAD } from "@ricky0123/vad-react" const MyComponent = () => { const vad = useMicVAD({ onSpeechEnd: (audio) => { console.log("User stopped talking") }, }) return
{vad.userSpeaking && "User is speaking"}
} ``` -------------------------------- ### Configure Vite for @ricky0123/vad-web Assets Source: https://docs.vad.ricky0123.com/user-guide/browser This configuration uses the vite-plugin-static-copy to copy necessary VAD assets (worklet, ONNX models, onnxruntime-web WASM and MJS files) from the node_modules directory to the build output. The 'dest' path is relative to the root directory after the build. Adjust 'modelURL' and 'workletURL' when initializing MicVad.new() if 'dest' is changed. ```javascript import { defineConfig } from 'vite'; import viteStaticCopy from 'vite-plugin-static-copy'; export default defineConfig({ plugins: [ viteStaticCopy({ targets: [ { src: 'node_modules/@ricky0123/vad-web/dist/vad.worklet.bundle.min.js', dest: './', }, { src: 'node_modules/@ricky0123/vad-web/dist/silero_vad_v5.onnx', dest: './', }, { src: 'node_modules/@ricky0123/vad-web/dist/silero_vad_legacy.onnx', dest: './', }, { src: 'node_modules/onnxruntime-web/dist/*.wasm', dest: './', }, { src: 'node_modules/onnxruntime-web/dist/*.mjs', dest: './', }, ], }), ], }); ``` -------------------------------- ### Non-Real-Time VAD with vad-node Source: https://docs.vad.ricky0123.com/user-guide/api This code snippet demonstrates how to use the vad-node library for non-real-time voice activity detection. It initializes a NonRealTimeVAD instance, processes audio data, and identifies speech segments. The output includes audio segments, start times, and end times. ```javascript const vad = require("@ricky0123/vad-node") // or @ricky0123/vad-web const options: Partial = { /* ... */ } const myvad = await vad.NonRealTimeVAD.new(options) const audioFileData, nativeSampleRate = ... // get audio and sample rate from file or something for await (const {audio, start, end} of myvad.run(audioFileData, nativeSampleRate)) { // do stuff with // audio (float32array of audio) // start (milliseconds into audio where speech starts) // end (milliseconds into audio where speech ends) } ``` -------------------------------- ### React Hook for Real-Time VAD with vad-react Source: https://docs.vad.ricky0123.com/user-guide/api This code snippet shows how to integrate the MicVAD hook from the vad-react library into a React component for real-time voice activity detection from a microphone input. It provides options for customization, including callbacks for speech start and end events. ```typescript import { useMicVAD } from "@ricky0123/vad-react" const MyComponent = () => { const vad = useMicVAD({ startOnLoad: true, onSpeechEnd: (audio) => { console.log("User stopped speaking") }, }) return
User speaking: {vad.userSpeaking}
} ``` -------------------------------- ### Interact with ONNX Runtime VAD Model in Browser Console Source: https://docs.vad.ricky0123.com/developer-guide/hacking JavaScript code to load the ONNX Runtime Web library, fetch the Silero VAD model, initialize the session, and run inference with sample audio data directly in the browser's developer console. This enables interactive experimentation with the VAD model. ```javascript script = this.document.createElement("script") script.src = "https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/ort.js" document.body.appendChild(script) // wait a few seconds modelarraybuffer = await fetch(`${location}silero_vad_v5.onnx`).then((model) => model.arrayBuffer()) session = await ort.InferenceSession.create(modelarraybuffer) state_zeroes = Array(2 * 128).fill(0) state = new this.ort.Tensor("float32", state_zeroes, [2, 1, 128]) // https://github.com/snakers4/silero-vad/blob/fdbb0a3a81e0f9d95561d6b388d67dce5d9e3f1b/utils_vad.py#L58 audio_zeros = Array(512).fill(0) audio = new this.ort.Tensor("float32", audio_zeros, [1, audio_zeros.length]) sr = new this.ort.Tensor("int64", [16000n]) inputs = { sr, state, input: audio } out = await session.run(inputs) ``` -------------------------------- ### Run Local Development Server for VAD Testing Source: https://docs.vad.ricky0123.com/developer-guide/hacking Command to run the VAD test site locally for manual testing with different parameters. This allows for interactive testing before deploying changes. ```shell npm run dev ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.