### Start Transcription Server with Defaults Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Run the `wlk serve` command with default settings to start the transcription server. ```bash wlk ``` -------------------------------- ### Install Production ASGI Server and Run Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Install uvicorn and gunicorn, then run the application using gunicorn with multiple Uvicorn workers for production. ```bash pip install uvicorn gunicorn gunicorn -k uvicorn.workers.UvicornWorker -w 4 your_app:app ``` -------------------------------- ### WebSocket Connection Example with Query Parameters Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Example of connecting to the WebSocket API with language and mode query parameters specified. ```bash ws://localhost:8000/asr?language=fr&mode=diff ``` -------------------------------- ### Use Voxtral Backend with Different Installations Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Installs and runs the server using the Voxtral backend. Supports native Apple Silicon (MLX) or GPU acceleration via HuggingFace. Ensure correct installation packages are used. ```bash pip install "whisperlivekit[voxtral-mlx]" wlk --backend voxtral-mlx ``` ```bash pip install "whisperlivekit[cu129,voxtral-hf]" wlk --backend voxtral ``` -------------------------------- ### Install WhisperLiveKit Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Install the WhisperLiveKit package using pip. This is the first step to using the library. ```bash pip install whisperlivekit ``` -------------------------------- ### Install WhisperLiveKit for Development Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/CLAUDE.md Install the package in editable mode with development and testing dependencies. ```sh pip install -e ".[test]" ``` -------------------------------- ### Install Test Dependencies and Run Unit Tests Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Install the necessary dependencies for testing and then run the unit tests using pytest. ```bash pip install -e ".[test]" pytest tests/ -v ``` -------------------------------- ### Install Optional Dependencies with pip install -e Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Use `pip install -e` with the appropriate extras to install optional dependencies for features like MLX Whisper, Voxtral, CPU PyTorch, CUDA PyTorch, translation, sentence tokenization, and speaker diarization. ```bash pip install -e ".[mlx-whisper]" ``` ```bash pip install -e ".[voxtral-mlx]" ``` ```bash pip install -e ".[cpu]" ``` ```bash pip install -e ".[cu129]" ``` ```bash pip install -e ".[translation]" ``` ```bash pip install -e ".[sentence_tokenizer]" ``` ```bash pip install -e ".[voxtral-hf]" ``` ```bash pip install -e ".[diarization-sortformer]" ``` ```bash pip install -e ".[diarization-diart]" ``` -------------------------------- ### Run Server with Qwen3-ASR Model Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Start the server and load the 'Qwen3-ASR' model. ```bash wlk run qwen3:1.7b ``` -------------------------------- ### Start WhisperLiveKit Server Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/technical_integration.md Use this command to start the WhisperLiveKit server with specified model and language. Ensure the host and port are accessible. ```bash wlk --model small --language en --host 0.0.0.0 --port 9000 ``` -------------------------------- ### Start WhisperLiveKit Server Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Start the WhisperLiveKit server with a specified model and language. Access the interface at http://localhost:8000. ```bash wlk --model base --language en ``` -------------------------------- ### Start Transcription Server with Specific Backend Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Start the transcription server using a specific backend, such as 'voxtral', and a particular model. ```bash wlk --backend voxtral --model base ``` -------------------------------- ### Run Server with Extra Options Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Start the server, auto-pulling the model, and pass additional server configuration options like language and port. ```bash wlk run voxtral --lan fr --port 9000 ``` -------------------------------- ### Run Server with Specific Backend and Model Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Start the server using a precise backend and model combination, such as 'faster-whisper:base'. ```bash wlk run faster-whisper:base ``` -------------------------------- ### Run Server with Specific Model Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Start the server and ensure a specific model, like 'large-v3', is downloaded and used. ```bash wlk run large-v3 ``` -------------------------------- ### Install Voxtral Backend Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Install the Voxtral backend for WhisperLiveKit. Use `.[voxtral-mlx]` for Apple Silicon with native MLX, or `transformers torch` with `.[voxtral]` for Linux/GPU using HuggingFace transformers. ```bash # Apple Silicon (native MLX, recommended) pip install -e ".[voxtral-mlx]" wlk --backend voxtral-mlx ``` ```bash # Linux/GPU (HuggingFace transformers) pip install transformers torch wlk --backend voxtral ``` -------------------------------- ### wlk models Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md List available backends, installation status, and downloaded models. ```APIDOC ## `wlk models` ### Description List available backends, installation status, and downloaded models. ### Usage ```bash wlk models ``` ``` -------------------------------- ### Run WhisperLiveKit with Auto-Model Pull Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Automatically pull a specified Whisper model and start the server. Useful for quick setup and testing. ```bash wlk run whisper:tiny ``` -------------------------------- ### Command Line Usage Examples for Translation Target Language Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/supported_languages.md Examples demonstrating how to set the target language for translation using the command line interface. ```APIDOC ### Command Line ```bash # Using language name whisperlivekit-server --target-language "French" # Using ISO code whisperlivekit-server --target-language fr # Using NLLB code whisperlivekit-server --target-language fra_Latn ``` ``` -------------------------------- ### wlk run Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Automatically pulls a model if not downloaded, then starts the server. ```APIDOC ## `wlk run` ### Description Auto-pull model if not downloaded, then start the server. ### Usage ```bash wlk run voxtral # Pull voxtral + start server wlk run large-v3 # Pull large-v3 + start server wlk run faster-whisper:base # Specific backend + model wlk run qwen3:1.7b # Qwen3-ASR wlk run voxtral --lan fr --port 9000 # Extra server options passed through ``` ``` -------------------------------- ### wlk version Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Print the installed version. ```APIDOC ## `wlk version` ### Description Print the installed version. ### Usage ```bash wlk version ``` ``` -------------------------------- ### List WhisperLiveKit Models with CLI Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md List available backends, installation status, and downloaded models using the `wlk models` command. ```bash wlk models ``` -------------------------------- ### Start Transcription Server with Explicit Serve Command Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Use the explicit `serve` command to start the transcription server, specifying the port and language. ```bash wlk serve --port 9000 --lan fr ``` -------------------------------- ### Install Optional Dependencies with uv sync Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Use `uv sync` with the `--extra` flag to install specific optional dependencies for features like MLX Whisper, Voxtral, CPU PyTorch, CUDA PyTorch, translation, sentence tokenization, and speaker diarization. ```bash uv sync --extra mlx-whisper ``` ```bash uv sync --extra voxtral-mlx ``` ```bash uv sync --extra cpu ``` ```bash uv sync --extra cu129 ``` ```bash uv sync --extra translation ``` ```bash uv sync --extra sentence_tokenizer ``` ```bash uv sync --extra voxtral-hf ``` ```bash uv sync --extra diarization-sortformer ``` ```bash uv sync --extra diarization-diart ``` -------------------------------- ### JSON Response Format Example Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Example of the JSON response format for audio transcriptions. ```json {"text": "Hello world, how are you?"} ``` -------------------------------- ### Install GPU Profiles with uv sync Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Install specific GPU profiles using `uv sync` with multiple `--extra` flags. Profile A includes Sortformer diarization, while Profile B includes Voxtral HF and translation. Note that `voxtral-hf` and `diarization-sortformer` are incompatible and require separate environments. ```bash # Profile A: Sortformer diarization uv sync --extra cu129 --extra diarization-sortformer ``` ```bash # Profile B: Voxtral HF + translation uv sync --extra cu129 --extra voxtral-hf --extra translation ``` -------------------------------- ### wlk serve Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Starts the WhisperLiveKit transcription server. ```APIDOC ## `wlk serve` ### Description Start the transcription server. ### Usage ```bash wlk # Start with defaults wlk --backend voxtral --model base # Specific backend wlk serve --port 9000 --lan fr # Explicit serve command ``` ``` -------------------------------- ### Start Server with Auto-Downloaded Model Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Automatically downloads a specified model (e.g., Whisper tiny, large-v3, Voxtral, Qwen3) if it's not already cached, then starts the transcription server. Additional server options can be passed through. ```bash wlk run whisper:tiny ``` ```bash wlk run large-v3 ``` ```bash wlk run voxtral ``` ```bash wlk run qwen3:1.7b ``` ```bash wlk run faster-whisper:base --lan fr --port 9000 ``` -------------------------------- ### AudioProcessor Setup and WebSocket Endpoint Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Details the setup of `AudioProcessor` for per-connection audio pipelines and the creation of a FastAPI WebSocket endpoint for handling audio streams. ```APIDOC ## Python API — `AudioProcessor` Per-connection audio pipeline. Create one per WebSocket client; reuse the shared `TranscriptionEngine`. ### FastAPI WebSocket Endpoint Setup ```python import asyncio from contextlib import asynccontextmanager from fastapi import FastAPI, WebSocket, WebSocketDisconnect from whisperlivekit import AudioProcessor, TranscriptionEngine, parse_args transcription_engine = None @asynccontextmanager async def lifespan(app: FastAPI): global transcription_engine # Load models once at startup transcription_engine = TranscriptionEngine(model_size="medium", diarization=True, lan="en") yield app = FastAPI(lifespan=lifespan) async def handle_results(websocket: WebSocket, results_gen): async for response in results_gen: await websocket.send_json(response.to_dict()) await websocket.send_json({"type": "ready_to_stop"}) @app.websocket("/asr") async def websocket_endpoint(websocket: WebSocket): # Per-session language from query string session_language = websocket.query_params.get("language", None) # One AudioProcessor per connection processor = AudioProcessor( transcription_engine=transcription_engine, language=session_language, # per-session override ) results_gen = await processor.create_tasks() results_task = asyncio.create_task(handle_results(websocket, results_gen)) await websocket.accept() try: while True: message = await websocket.receive_bytes() await processor.process_audio(message) # stream bytes here except WebSocketDisconnect: pass finally: await processor.cleanup() # always clean up ``` ``` -------------------------------- ### Install GPU-enabled PyTorch Wheels Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/troubleshooting.md Installs or upgrades PyTorch, torchvision, and torchaudio with CUDA support. Replace 'cu130' with your CUDA toolkit version. ```bash pip install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu130 ``` -------------------------------- ### Install WhisperLiveKit Optional Dependencies Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Install WhisperLiveKit with optional dependencies for specific backends like MLX-Whisper, Voxtral MLX, CUDA, translation, diarization, or CPU support. ```bash pip install "whisperlivekit[mlx-whisper]" ``` ```bash pip install "whisperlivekit[voxtral-mlx]" ``` ```bash pip install "whisperlivekit[cu129]" ``` ```bash pip install "whisperlivekit[translation]" ``` ```bash pip install "whisperlivekit[diarization-sortformer]" ``` ```bash pip install "whisperlivekit[cpu]" ``` -------------------------------- ### Run Transcription Server with Options Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Examples of running the WhisperLiveKit transcription server from the command line. Options include specifying the model, source and target languages for translation, host and port for the server, and enabling diarization. ```bash # Large model and translate from french to danish wlk --model large-v3 --language fr --target-language da ``` ```bash # Diarization and server listening on */80 wlk --host 0.0.0.0 --port 80 --model medium --diarization --language fr ``` -------------------------------- ### Python API Usage Examples for Language Information Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/supported_languages.md Examples showing how to retrieve language information using the Python API, with different methods of specifying the language. ```APIDOC ### Python API ```python from nllw.translation import get_language_info # Get language information by name lang_info = get_language_info("French") print(lang_info) # {'name': 'French', 'nllb': 'fra_Latn', 'language_code': 'fr'} # Get language information by ISO code lang_info = get_language_info("fr") # Get language information by NLLB code lang_info = get_language_info("fra_Latn") # All three return the same result ``` ``` -------------------------------- ### Install CUDA and cuDNN on Arch/CachyOS Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/troubleshooting.md Installs CUDA and cuDNN packages and updates the dynamic linker cache. Ensure these match your PyTorch build. ```bash sudo pacman -S cuda cudnn sudo ldconfig ``` -------------------------------- ### Print WhisperLiveKit Version with CLI Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Print the installed version of WhisperLiveKit using the `wlk version` command. ```bash wlk version ``` -------------------------------- ### Install CTranslate2 CUDA Wheel Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/troubleshooting.md Installs the CUDA-enabled CTranslate2 wheel. Replace '4.5.0' and 'cu130' with your desired version and CUDA toolkit version. ```bash pip install ctranslate2==4.5.0 -f https://opennmt.net/ctranslate2/whl/cu130 ``` -------------------------------- ### Configure Server with Model, Language, and Diarization Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Starts the server with a specified model (e.g., large-v3), source language (e.g., French), and target language for translation (e.g., Danish). Diarization can be enabled, and the server can listen on all interfaces with a custom port. ```bash wlk --model large-v3 --language fr --target-language da ``` ```bash wlk --host 0.0.0.0 --port 80 --model medium --diarization ``` -------------------------------- ### Docker deployment for WhisperLiveKit (GPU) Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Build and run the Docker image for GPU acceleration. Ensure the NVIDIA Container Toolkit is installed for GPU support. ```bash # GPU (recommended) — build and run docker build -t wlk . docker run --gpus all -p 8000:8000 wlk # GPU with custom args docker run --gpus all -p 8000:8000 wlk --model large-v3 --language fr ``` -------------------------------- ### Verbose JSON Response Format Example Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Example of the verbose JSON response format, including detailed timing and segment information for transcriptions. ```json { "task": "transcribe", "language": "en", "duration": 7.16, "text": "Hello world", "words": [{"word": "Hello", "start": 0.0, "end": 0.5}, ...], "segments": [{"id": 0, "start": 0.0, "end": 3.5, "text": "Hello world"}] } ``` -------------------------------- ### Run Server and Auto-Pull Model Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Start the transcription server and automatically download the specified model if it's not already present. ```bash wlk run voxtral ``` -------------------------------- ### GET /v1/models Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Lists the currently loaded model on the server. ```APIDOC ## GET /v1/models ### Description List the currently loaded model. ### Method GET ### Endpoint /v1/models ### Parameters None ### Request Example ```bash curl http://localhost:8000/v1/models ``` ### Response (Response structure not explicitly defined in source, but implies a list of models) ``` -------------------------------- ### Test Harness for Real-time Transcription Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/CLAUDE.md Use the TestHarness to perform end-to-end testing with real audio files. This example demonstrates feeding audio, managing silence, and evaluating transcription results. ```python import asyncio from whisperlivekit import TestHarness async def main(): async with TestHarness(model_size="base", lan="en", diarization=True) as h: await h.feed("audio.wav", speed=1.0) # feed at real-time await h.drain(2.0) # let ASR catch up h.print_state() # see current output await h.silence(7.0, speed=1.0) # 7s silence await h.wait_for_silence() # verify detection result = await h.finish() print(f"WER: {result.wer('expected text'):.2%}") print(f"Speakers: {result.speakers}") print(f"Text at 3s: {result.text_at(3.0)}") asyncio.run(main()) ``` -------------------------------- ### Example 1: STT Punctuation and Diarization at Prediction 't' Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/alignement_principles.md Demonstrates the alignment where STT punctuation and speaker changes from diarization both occur within the current prediction 't'. ```text punctuations_segments : __#_______.__________________!____ diarization_segments: SPK1 __#____________ SPK2 # ___________________ --> ALIGNED SPK1 __#_______. ALIGNED SPK2 # __________________!____ t-1 output: SPK1: __# SPK2: NO DIARIZATION BUFFER: NO t output: SPK1: __#__. SPK2: __________________!____ DIARIZATION BUFFER: No ``` -------------------------------- ### Live Microphone Transcription in Terminal Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Performs live microphone transcription directly in the terminal without starting a server. Requires the `sounddevice` package. Options include specifying language, enabling diarization, choosing a backend, and saving output. ```bash wlk listen ``` ```bash wlk listen --language fr ``` ```bash wlk listen --diarization ``` ```bash wlk listen --backend voxtral ``` ```bash wlk listen -o transcript.txt ``` -------------------------------- ### TranscriptionEngine Initialization Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Shows how to initialize the `TranscriptionEngine` using keyword arguments or a configuration object. ```APIDOC ## Python API — `TranscriptionEngine` Thread-safe singleton. Initialize once at application startup; all `AudioProcessor` instances share it. ### Initialization with Keyword Arguments ```python from whisperlivekit import TranscriptionEngine engine = TranscriptionEngine( model_size="large-v3", lan="fr", diarization=True, diarization_backend="sortformer", target_language="en", backend_policy="simulstreaming", backend="faster-whisper", ) ``` ### Initialization with Config Object ```python from whisperlivekit import TranscriptionEngine, WhisperLiveKitConfig config = WhisperLiveKitConfig( model_size="base", lan="auto", backend="auto", diarization=False, pcm_input=True, frame_threshold=25, beams=1, ) engine = TranscriptionEngine(config=config) ``` ### Resetting the Singleton (for testing only) ```python # For testing only — reset the singleton between test runs TranscriptionEngine.reset() ``` ``` -------------------------------- ### Docker deployment for WhisperLiveKit (CPU) Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Build and run the Docker image for CPU-only execution. Use the Dockerfile.cpu and specify 'cpu' for the build argument. ```bash # CPU only docker build -f Dockerfile.cpu -t wlk-cpu --build-arg EXTRAS="cpu" . docker run -p 8000:8000 wlk-cpu ``` -------------------------------- ### Build and Run Docker Image with GPU Acceleration Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Build the Docker image and run it with GPU acceleration enabled, mapping port 8000. ```bash docker build -t wlk . docker run --gpus all -p 8000:8000 --name wlk wlk ``` -------------------------------- ### GET /health Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Checks the health status of the server. ```APIDOC ## GET /health ### Description Server health check. ### Method GET ### Endpoint /health ### Parameters None ### Request Example ```bash curl http://localhost:8000/health ``` ### Response (Response structure not explicitly defined in source, but implies a health status indicator) ``` -------------------------------- ### Run Quick Benchmark with CLI Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Execute a quick benchmark using the WhisperLiveKit CLI. Options include specifying a backend, model, and outputting results to JSON. ```bash wlk bench ``` ```bash wlk bench --backend faster-whisper --model large-v3 ``` ```bash wlk bench --languages all --json results.json ``` -------------------------------- ### Initialize TranscriptionEngine (Keyword Arguments) Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Initialize the thread-safe TranscriptionEngine using keyword arguments for model size, language, diarization, and backend policy. ```python from whisperlivekit import TranscriptionEngine, WhisperLiveKitConfig # Keyword-argument style engine = TranscriptionEngine( model_size="large-v3", lan="fr", diarization=True, diarization_backend="sortformer", target_language="en", backend_policy="simulstreaming", backend="faster-whisper", ) ``` -------------------------------- ### Deepgram Client Initialization and Connection Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Demonstrates how to initialize the Deepgram client to point to a local server and establish a WebSocket connection for live transcription. ```APIDOC ## Deepgram Client Initialization and Connection ### Description Initialize the Deepgram client to connect to a local server and set up a WebSocket connection for live transcription. This includes defining a message handler and starting the transcription with specified options. ### Usage ```python from deepgram import DeepgramClient, LiveTranscriptionEvents, LiveOptions # Point the SDK at the local server (api_key is unused) deepgram = DeepgramClient(api_key="unused", config={"url": "localhost:8000"}) connection = deepgram.listen.websocket.v("1") def on_message(result, **kwargs): sentence = result.channel.alternatives[0].transcript if result.is_final: print(f"Final: {sentence}") else: print(f"Partial: {sentence}", end="\r") connection.on(LiveTranscriptionEvents.Transcript, on_message) connection.start(LiveOptions( model="nova-2", language="en", interim_results=True, vad_events=True, )) # Stream audio frames as bytes with open("audio.wav", "rb") as f: while chunk := f.read(4096): connection.send(chunk) # Flush and close connection.send(json.dumps({"type": "Finalize"})) connection.finish() ``` ### Supported client messages Binary audio, `{"type": "KeepAlive"}`, `{"type": "CloseStream"}`, `{"type": "Finalize"}`. ``` -------------------------------- ### Benchmark WhisperLiveKit with CLI Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Benchmark speed and accuracy using the `wlk bench` command. Specify backend, model, or output format. ```bash wlk bench # Benchmark with defaults ``` ```bash wlk bench --backend faster-whisper # Specific backend ``` ```bash wlk bench --model large-v3 # Larger model ``` ```bash wlk bench --json results.json # Export results ``` -------------------------------- ### Transcribe Audio File Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Transcribe a local audio file without starting a server. Supports various audio formats. ```bash wlk transcribe meeting.wav ``` -------------------------------- ### Check WhisperLiveKit System Dependencies with CLI Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Verify system dependencies such as Python, ffmpeg, and torch using the `wlk check` command. ```bash wlk check ``` -------------------------------- ### Live Microphone Transcription Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Transcribe audio directly from the microphone using the `wlk listen` command. Requires the `sounddevice` library to be installed. ```bash wlk listen ``` -------------------------------- ### Initialize TranscriptionEngine (Config Object) Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Initialize the TranscriptionEngine using a WhisperLiveKitConfig object for type-checked configuration. The singleton can be reset for testing. ```python # Config-object style (full type-checked config) config = WhisperLiveKitConfig( model_size="base", lan="auto", backend="auto", diarization=False, pcm_input=True, frame_threshold=25, beams=1, ) engine = TranscriptionEngine(config=config) # For testing only — reset the singleton between test runs TranscriptionEngine.reset() ``` -------------------------------- ### wlk check Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Verify system dependencies (Python, ffmpeg, torch, etc.). ```APIDOC ## `wlk check` ### Description Verify system dependencies (Python, ffmpeg, torch, etc.). ### Usage ```bash wlk check ``` ``` -------------------------------- ### Manage Models and System Dependencies Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Commands for managing models, including listing available backends and cached models, pulling new models, removing existing ones, and checking system dependencies like Python, ffmpeg, and PyTorch. Also includes a command to diagnose pipeline issues. ```bash wlk models ``` ```bash wlk pull large-v3 ``` ```bash wlk pull faster-whisper:large-v3 ``` ```bash wlk pull voxtral ``` ```bash wlk pull qwen3:1.7b ``` ```bash wlk rm large-v3 ``` ```bash wlk check ``` ```bash wlk version ``` ```bash wlk diagnose audio.wav ``` ```bash wlk diagnose audio.wav --backend voxtral --speed 0 --probe-interval 1 ``` -------------------------------- ### Connect to WebSocket API with Python Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Example of connecting to the Deepgram-compatible WebSocket API using the Deepgram Python client. Specify model and language options for live transcription. ```python from deepgram import DeepgramClient, LiveOptions deepgram = DeepgramClient(api_key="unused", config={"url": "localhost:8000"}) connection = deepgram.listen.websocket.v("1") connection.start(LiveOptions(model="nova-2", language="en")) ``` -------------------------------- ### Simplified Transcription with WhisperLiveKit TestHarness Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Use the `TestHarness` for simpler programmatic transcription in Python. It handles setup and teardown, allowing direct audio feeding and result retrieval. ```python import asyncio from whisperlivekit import TestHarness async def main(): async with TestHarness(model_size="base", lan="en") as h: await h.feed("audio.wav", speed=0) result = await h.finish() print(result.text) asyncio.run(main()) ``` -------------------------------- ### Download WhisperLiveKit Models with CLI Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Download models for offline use with `wlk pull`. Specify backend and model, or use defaults. ```bash wlk pull base # Download for best available backend ``` ```bash wlk pull faster-whisper:large-v3 # Specific backend + model ``` ```bash wlk pull voxtral # Voxtral HF model ``` ```bash wlk pull qwen3:1.7b # Qwen3-ASR 1.7B ``` -------------------------------- ### Create WhisperLiveKitConfig from Keyword Arguments Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Build a WhisperLiveKitConfig object from keyword arguments, allowing for flexible configuration of transcription parameters. Unknown keys are logged and ignored. ```python from whisperlivekit import WhisperLiveKitConfig # Build from kwargs (unknown keys are logged and ignored) cfg = WhisperLiveKitConfig.from_kwargs( model_size="medium", lan="de", diarization=True, backend_policy="simulstreaming", frame_threshold=30, target_language="en", nllb_size="1.3B", ssl_certfile="/etc/ssl/cert.pem", ssl_keyfile="/etc/ssl/key.pem", ) ``` -------------------------------- ### Serving WhisperLiveKit web interface Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Generate HTML for the bundled browser frontend. Use get_inline_ui_html for a single-file deployment or get_web_interface_html for external asset references. ```python from fastapi.responses import HTMLResponse from whisperlivekit import get_inline_ui_html, get_web_interface_html # All assets inlined (single-file, no CDN) @app.get("/") async def index(): return HTMLResponse(get_inline_ui_html()) # References external assets (suitable for static hosting) @app.get("/") async def index(): return HTMLResponse(get_web_interface_html()) ``` -------------------------------- ### Example 3: STT Punctuation at 't-1', Diarization at 't' Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/alignement_principles.md Shows the alignment where STT punctuation is from the previous prediction 't-1', while speaker changes from diarization occur in the current prediction 't'. ```text punctuations_segments : ___.__#__________ diarization_segments: SPK1 ______#__ SPK2 # ________ --> ALIGNED SPK1 ___. # ALIGNED SPK2 __#__________ t-1 output: SPK1: ___. # SPK2: DIARIZATION BUFFER: __# t output: SPK1: # SPK2: __#___________ DIARIZATION BUFFER: NO ``` -------------------------------- ### Example 2: STT Punctuation at 't', Diarization at 't-1' Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/alignement_principles.md Illustrates the alignment where STT punctuation falls into the current prediction 't', but speaker changes from diarization occurred in the previous prediction 't-1'. ```text punctuations_segments : _____#__.___________ diarization_segments: SPK1 ___ # SPK2 __#______________ --> ALIGNED SPK1 _____#__. ALIGNED SPK2 # ___________ t-1 output: SPK1: ___ # SPK2: DIARIZATION BUFFER: __# t output: SPK1: __#__. SPK2: ___________ DIARIZATION BUFFER: No ``` -------------------------------- ### Enable Raw PCM Input Mode Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Configures the server to accept raw PCM audio input, typically used with AudioWorklet in the browser instead of MediaRecorder. Requires specifying the model, language, and other relevant options. ```bash wlk --pcm-input --model base --language en ``` -------------------------------- ### Get Top 2 Speaker Predictions Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/DEV_NOTES.md Extracts the indices of the top two speaker predictions from a numpy array. This is a preliminary step in constraining a diarization model from predicting 4 speakers to 2. ```python top_2_speakers = np.argsort(preds_np, axis=1)[:, -2:] ``` -------------------------------- ### WebSocket Transcription Update Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt This is an example of a WebSocket message sent from the server to the client, providing real-time transcription updates in full mode. It includes speaker information, text segments, timestamps, and translation details. ```APIDOC ## WebSocket Message Reference ### Server → Client: Transcription update (full mode) ```json { "status": "active_transcription", "lines": [ { "speaker": 1, "text": "Hello world, how are you?", "start": "0:00:00", "end": "0:00:03", "translation": "Bonjour monde, comment allez-vous?", "detected_language": "en" }, { "speaker": -2, "text": null, "start": "0:00:10", "end": "0:00:15" } ], "buffer_transcription": "And you", "buffer_diarization": "", "buffer_translation": "", "remaining_time_transcription": 1.2, "remaining_time_diarization": 0.5 } ``` - `speaker: -2` with `text: null` = silence segment (only generated for pauses > 5 seconds) - `status: "no_audio_detected"` = no speech has been detected yet - `type: "ready_to_stop"` = all audio processed after client sends `b""` ``` -------------------------------- ### Get Language Information via Python API Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/supported_languages.md Shows how to retrieve language details using the get_language_info function from the nllw.translation module. This function accepts language name, ISO code, or NLLB code. ```python from nllw.translation import get_language_info # Get language information by name lang_info = get_language_info("French") print(lang_info) # {'name': 'French', 'nllb': 'fra_Latn', 'language_code': 'fr'} # Get language information by ISO code lang_info = get_language_info("fr") # Get language information by NLLB code lang_info = get_language_info("fra_Latn") # All three return the same result ``` -------------------------------- ### WhisperLiveKitConfig Usage Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Illustrates how to create and use `WhisperLiveKitConfig` objects, including building from keyword arguments or a namespace, and accessing key fields. ```APIDOC ## Python API — `WhisperLiveKitConfig` Typed dataclass holding all pipeline configuration. Factory methods make it easy to build from CLI args or kwargs. ### Building from kwargs ```python from whisperlivekit import WhisperLiveKitConfig # Build from kwargs (unknown keys are logged and ignored) cfg = WhisperLiveKitConfig.from_kwargs( model_size="medium", lan="de", diarization=True, backend_policy="simulstreaming", frame_threshold=30, target_language="en", nllb_size="1.3B", ssl_certfile="/etc/ssl/cert.pem", ssl_keyfile="/etc/ssl/key.pem", ) ``` ### Building from argparse Namespace ```python import argparse from whisperlivekit import WhisperLiveKitConfig ns = argparse.Namespace(model_size="small", lan="en", diarization=False) cfg = WhisperLiveKitConfig.from_namespace(ns) ``` ### Key Fields and Defaults ```python # Assuming cfg is a WhisperLiveKitConfig object print(cfg.model_size) # Example: "medium" print(cfg.backend_policy) # Example: "simulstreaming" print(cfg.backend) # Example: "auto" print(cfg.vac) # Example: True (Voice Activity Controller) print(cfg.vad) # Example: True (Voice Activity Detection) print(cfg.pcm_input) # Example: False (MediaRecorder WebM → FFmpeg decode) print(cfg.frame_threshold) # Example: AlignAtt threshold (default 25) ``` ``` -------------------------------- ### Run Docker Container with Custom Model and Language Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Run the Docker container with GPU acceleration, specifying a custom model and language. ```bash docker run --gpus all -p 8000:8000 --name wlk wlk --model large-v3 --language fr ``` -------------------------------- ### Configure Server for HTTPS and Reverse Proxy Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Enables HTTPS for microphone access over non-localhost domains by specifying certificate and key files. Allows the server to be run behind a reverse proxy by configuring allowed client IP addresses. ```bash wlk --ssl-certfile cert.pem --ssl-keyfile key.pem ``` ```bash wlk --forwarded-allow-ips "10.0.0.0/8" ``` -------------------------------- ### Build and Run Docker Image with CPU Only Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Build a Docker image specifically for CPU usage using a separate Dockerfile and run it, mapping port 8000. ```bash docker build -f Dockerfile.cpu -t wlk --build-arg EXTRAS="cpu" . docker run -p 8000:8000 --name wlk wlk ``` -------------------------------- ### Generate Benchmark Samples and Run Scatter Plot Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Scripts to generate test samples and run scatter plot benchmarks for speed vs. accuracy, with options for different languages. ```python python scripts/create_long_samples.py # generate ~90s test samples (cached) ``` ```python python scripts/run_scatter_benchmark.py # English (both modes) ``` ```python python scripts/run_scatter_benchmark.py --lang fr # French ``` -------------------------------- ### Server to Client WebSocket Transcription Update Source: https://context7.com/quentinfuxa/whisperlivekit/llms.txt Example of a WebSocket message from the server to the client containing transcription updates. It includes speaker information, text, timestamps, translations, and buffer status. Silence segments are indicated by speaker -2 and null text. ```json { "status": "active_transcription", "lines": [ { "speaker": 1, "text": "Hello world, how are you?", "start": "0:00:00", "end": "0:00:03", "translation": "Bonjour monde, comment allez-vous?", "detected_language": "en" }, { "speaker": -2, "text": null, "start": "0:00:10", "end": "0:00:15" } ], "buffer_transcription": "And you", "buffer_diarization": "", "buffer_translation": "", "remaining_time_transcription": 1.2, "remaining_time_diarization": 0.5 } ``` -------------------------------- ### Run WhisperLiveKit with Voxtral Backend Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Use this command to run WhisperLiveKit with the Voxtral multilingual backend, which automatically detects the language. ```bash wlk --backend voxtral-mlx ``` -------------------------------- ### OpenAI Python Client Usage Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/CLAUDE.md Demonstrates how to use the OpenAI Python client library to interact with the Whisper LiveKit's OpenAI-compatible API. ```APIDOC ## OpenAI Python Client Example ### Description This example shows how to integrate with the Whisper LiveKit's transcription endpoint using the official OpenAI Python client. ### Code ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused") result = client.audio.transcriptions.create(model="whisper-1", file=open("audio.mp3", "rb")) print(result.text) ``` ``` -------------------------------- ### Docker Compose Up Commands Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/README.md Commands to bring up Docker Compose services for different WhisperLiveKit profiles, including GPU and CPU options. ```bash docker compose up --build wlk-gpu-sortformer ``` ```bash docker compose up --build wlk-gpu-voxtral ``` ```bash docker compose up --build wlk-cpu ``` -------------------------------- ### Live Microphone Transcription with Specific Backend Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Perform live microphone transcription using a specified backend, such as 'voxtral'. ```bash wlk listen --backend voxtral ``` -------------------------------- ### wlk listen Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Performs live microphone transcription using the command line. ```APIDOC ## `wlk listen` ### Description Live microphone transcription. Requires `sounddevice` (`pip install sounddevice`). ### Usage ```bash wlk listen # Transcribe from microphone wlk listen --backend voxtral # Use specific backend wlk listen --language fr # Force French wlk listen --diarization # With speaker identification wlk listen -o transcript.txt # Save to file on exit ``` Committed lines print as they are finalized. The current buffer (partial transcription) is shown in gray and updates in-place. Press Ctrl+C to stop; remaining audio is flushed before exit. ``` -------------------------------- ### Diagnose WhisperLiveKit Pipeline with CLI Source: https://github.com/quentinfuxa/whisperlivekit/blob/main/docs/API.md Run pipeline diagnostics on an audio file using `wlk diagnose`. Specify backend, speed, or probe interval. Can use a built-in test sample. ```bash wlk diagnose audio.wav # Diagnose with default backend ``` ```bash wlk diagnose audio.wav --backend voxtral # Diagnose specific backend ``` ```bash wlk diagnose --speed 0 --probe-interval 1 # Instant feed, probe every 1s ``` ```bash wlk diagnose # Use built-in test sample ```