### Install Whisper-Timestamped Backend

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs the alternative whisper-timestamped backend, which is less restrictive but slower than faster-whisper.

```bash
pip install git+https://github.com/linto-ai/whisper-timestamped
```

--------------------------------

### Install Audio Processing Library

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs the librosa and soundfile libraries for audio processing.

```bash
pip install librosa soundfile
```

--------------------------------

### Install Faster-Whisper Backend

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs the recommended faster-whisper backend, which supports GPU acceleration. Ensure NVIDIA libraries (e.g., CUDNN 8.5.0, CUDA 11.7) are set up before installation.

```bash
pip install faster-whisper
```

--------------------------------

### Install Voice Activity Controller Dependencies

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs PyTorch and torchaudio, which are optional but highly recommended for the voice activity controller.

```bash
pip install torch torchaudio
```

--------------------------------

### Install OpenAI Backend

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs the openai library for using the OpenAI Whisper API. Requires Python 3.8+.

```bash
pip install openai
```

--------------------------------

### Install Opus-Fast-Mosestokenizer for Sentence Segmentation

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs the opus-fast-mosestokenizer for sentence segmentation in various languages. This is an optional component.

```bash
pip install opus-fast-mosestokenizer
```

--------------------------------

### Install Tokenize-UK for Ukrainian Sentence Segmentation

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs the tokenize_uk library specifically for Ukrainian sentence segmentation. This is an optional component.

```bash
pip install tokenize_uk
```

--------------------------------

### Install Whisper MLX Backend

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs the Whisper MLX library, optimized for Apple Silicon (M1, M2, etc.), providing fast transcription without a dedicated GPU.

```bash
pip install mlx-whisper
```

--------------------------------

### Install Wtpsplit for Multilingual Sentence Segmentation

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Installs PyTorch and wtpsplit for a multilingual sentence segmentation model. The model is downloaded automatically on first use.

```bash
pip install torch wtpsplit
```

--------------------------------

### Start whisper_online_server.py TCP streaming server

Source: https://context7.com/ufal/whisper_streaming/llms.txt

A standalone TCP server that listens for raw PCM audio, transcribes it in real time using `OnlineASRProcessor`, and streams back timestamped text lines. One client is served per connection.

```bash
# Start server on default port 43007 with faster-whisper large-v2,
# English, VAC enabled, and a warmup file to pre-load the model
python3 whisper_online_server.py \
    --backend faster-whisper \
    --model large-v2 \
    --lan en \
    --vac \
    --warmup-file /path/to/warmup.wav \
    --host 0.0.0.0 \
    --port 43007 \
    --log-level INFO

# Client: stream live mic audio via arecord + netcat
arecord -f S16_LE -c1 -r 16000 -t raw -D default | nc localhost 43007

# The server writes transcript lines to stdout in the format:
# <beg_ms> <end_ms> <text>
# Example server stdout:
```

--------------------------------

### Real-time Audio Streaming via Netcat

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Example of streaming raw audio from a microphone using `arecord` to a netcat client listening on a specific host and port. This is useful for testing real-time audio input to a server. Ensure `arecord` is configured with the correct audio format (S16_LE, 16000Hz, mono).

```bash
arecord -f S16_LE -c1 -r 16000 -t raw -D default | nc localhost 43001
```

--------------------------------

### Whisper Streaming Output Format

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Example of the output format from `whisper_online.py`. Each line contains a timestamp, start and end times for a segment, and the transcribed text. This format is useful for analyzing transcription results.

```text
2691.4399 300 1380 Chairman, thank you.
6914.5501 1940 4940 If the debate today had a
9019.0277 5160 7160 the subject the situation in
10065.1274 7180 7480 Gaza
11058.3558 7480 9460 Strip, I might
12224.3731 9460 9760 have
13555.1929 9760 11060 joined Mrs.
14928.5479 11140 12240 De Kaiser and all the
16588.0787 12240 12560 other
18324.9285 12560 14420 colleagues across the
```

--------------------------------

### Process Audio Buffer and Get Committed Text with process_iter

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Run Whisper inference on the current audio buffer, identify newly committed words using LocalAgreement-2, and return committed text with timestamps. Returns (None, None, '') if no new text is committed.

```python
from whisper_online import FasterWhisperASR, OnlineASRProcessor, load_audio_chunk

asr = FasterWhisperASR(lan="en", modelsize="large-v2")
online = OnlineASRProcessor(asr)

# Feed 3 seconds of audio then call process_iter
chunk = load_audio_chunk("speech.wav", 0, 3)
online.insert_audio_chunk(chunk)

beg, end, text = online.process_iter()
if beg is not None:
    print(f"Committed: [{beg*1000:.0f}ms – {end*1000:.0f}ms] '{text}'")
    # Example: Committed: [120ms – 2400ms] 'Good morning everyone.'
else:
    print("No committed text yet — need more audio context.")
```

--------------------------------

### WhisperTimestampedASR

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Implementation of the ASR interface using the `whisper_timestamped` library, suitable for environments with simpler GPU installation requirements.

```APIDOC
## WhisperTimestampedASR

### Description
An alternative ASR backend using the `whisper_timestamped` library. It offers simpler GPU installation compared to `faster-whisper` and is compatible with the `OnlineASRProcessor` interface.

### Initialization
```python
from whisper_online import WhisperTimestampedASR, load_audio_chunk

# Loads an OpenAI Whisper model with timestamped word output
asr = WhisperTimestampedASR(lan="fr", modelsize="medium")
```

### Optional Configurations
```python
asr.set_translate_task()  # translate French → English
asr.use_vad()             # use VAD via whisper_timestamped
```

### Transcribe Audio

#### Method
`transcribe(audio_chunk, init_prompt="")`

#### Parameters
- **audio_chunk** (numpy.ndarray) - A numpy float32 audio array.
- **init_prompt** (string, optional) - An initial prompt to guide transcription.

#### Returns
- Transcription result object.

### Get Timestamped Words

#### Method
`ts_words(result)`

#### Parameters
- **result** (object) - The transcription result object returned by `transcribe`.

#### Returns
- List of tuples: `[(start_sec, end_sec, "word"), ...]`

### Example
```python
from whisper_online import WhisperTimestampedASR, load_audio_chunk

asr = WhisperTimestampedASR(lan="fr", modelsize="medium")
# asr.set_translate_task()  # translate French → English
# asr.use_vad()             # use VAD via whisper_timestamped

audio_chunk = load_audio_chunk("french_speech.wav", 0, 15)
result = asr.transcribe(audio_chunk, init_prompt="Bonjour,")

words = asr.ts_words(result)
for start, end, word in words[:3]:
    print(f"[{start:.2f}s – {end:.2f}s] {word!r}")
```
```

--------------------------------

### WhisperTimestampedASR Backend for Transcription

Source: https://context7.com/ufal/whisper_streaming/llms.txt

An alternative backend using the whisper_timestamped library. It is slower than faster-whisper but has simpler GPU installation. Compatible with the OnlineASRProcessor interface.

```python
from whisper_online import WhisperTimestampedASR, load_audio_chunk

# Loads an OpenAI Whisper model with timestamped word output
asr = WhisperTimestampedASR(lan="fr", modelsize="medium")
# asr.set_translate_task()  # translate French → English
# asr.use_vad()             # use VAD via whisper_timestamped

audio_chunk = load_audio_chunk("french_speech.wav", 0, 15)
result = asr.transcribe(audio_chunk, init_prompt="Bonjour,")

words = asr.ts_words(result)
for start, end, word in words[:3]:
    print(f"[{start:.2f}s – {end:.2f}s] {word!r}")
# Output:
# [0.20s – 0.55s] ' Bonjour'
# [0.56s – 0.90s] ' tout'
# [0.91s – 1.10s] ' le'
```

--------------------------------

### Initialize and Process Audio Stream with Whisper

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Demonstrates initializing the ASR processor, inserting audio chunks, and processing the stream iteratively. Use this for real-time transcription where audio is received incrementally. Ensure the audio source is properly configured and the loop continues until audio ends.

```python
from whisper_online import *

src_lan = "en"  # source language
tgt_lan = "en"  # target language  -- same as source for ASR, "en" if translate task is used

asr = FasterWhisperASR(lan, "large-v2")  # loads and wraps Whisper model
# set options:
# asr.set_translate_task()  # it will translate from lan into English
# asr.use_vad()  # set using VAD

online = OnlineASRProcessor(asr)  # create processing object with default buffer trimming option

while audio_has_not_ended:   # processing loop:
	a = # receive new audio chunk (and e.g. wait for min_chunk_size seconds first, ...)
	online.insert_audio_chunk(a)
	o = online.process_iter()
	print(o) # do something with current partial output
# at the end of this audio processing
o = online.finish()
print(o)  # do something with the last output


online.init()  # refresh if you're going to re-use the object for the next audio
```

--------------------------------

### Whisper Online Script Help

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Displays the help message for the `whisper_online.py` script, outlining all available arguments and their descriptions. Use this to understand the full range of options for real-time simulation.

```bash
whisper_online.py -h
```

--------------------------------

### Real-time ASR with Voice Activity Control (VACOnlineASRProcessor)

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Wraps `OnlineASRProcessor` with Silero VAD to detect speech/silence in real time. It triggers `OnlineASRProcessor` only during detected speech and forces a `finish()` call at the end of each utterance (500 ms of silence), reducing unnecessary inference and latency.

```python
import torch
import numpy as np
from whisper_online import FasterWhisperASR, VACOnlineASRProcessor, load_audio_chunk

asr = FasterWhisperASR(lan="en", modelsize="large-v2")

# online_chunk_size: seconds of voice-active audio to accumulate before running Whisper
online = VACOnlineASRProcessor(
    online_chunk_size=0.04,  # run Whisper every ~40ms of voice
    asr=asr,
    tokenizer=None,
    buffer_trimming=("segment", 15)
)

# Stream 40ms frames (VAC processes them internally)
FRAME = int(0.04 * 16000)
audio_path = "speech_with_pauses.wav"
full_audio = load_audio_chunk(audio_path, 0, 60)

for i in range(0, len(full_audio), FRAME):
    chunk = full_audio[i:i+FRAME]
    if len(chunk) < FRAME:
        break
    online.insert_audio_chunk(chunk)
    beg, end, text = online.process_iter()
    if text:
        print(f"[{beg:.2f}s – {end:.2f}s] {text}")
    # VAC suppresses output during silence; only prints during/after speech

online.finish()
online.init()  # reset for next utterance
```

--------------------------------

### Run Real-time Simulation

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Executes the `whisper_online.py` script to simulate real-time processing from an audio file. Specify the input audio path, language, and minimum chunk size. Output is redirected to `out.txt`.

```bash
python3 whisper_online.py en-demo16.wav --language en --min-chunk-size 1 > out.txt
```

--------------------------------

### Instantiate ASR and processor from CLI args using asr_factory

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Factory function used by CLI scripts to build `ASRBase` and `OnlineASRProcessor` (or `VACOnlineASRProcessor`) from parsed arguments. Handles backend selection, VAD activation, task setting, and tokenizer creation.

```python
import argparse
from whisper_online import asr_factory, add_shared_args

parser = argparse.ArgumentParser()
add_shared_args(parser)

# Simulate: faster-whisper, English, large-v2, with VAC
args = parser.parse_args([
    "--backend", "faster-whisper",
    "--model", "large-v2",
    "--lan", "en",
    "--vac",
    "--vac-chunk-size", "0.04",
    "--buffer_trimming", "segment",
    "--buffer_trimming-sec", "15",
    "--log-level", "INFO"
])

asr, online = asr_factory(args)
# asr   → FasterWhisperASR instance
# online → VACOnlineASRProcessor instance (because --vac was set)
print(type(asr).__name__, type(online).__name__)
# Output: FasterWhisperASR VACOnlineASRProcessor
```

--------------------------------

### OpenaiApiASR

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Sends audio chunks to the OpenAI /audio/transcriptions or /audio/translations API. Requires OPENAI_API_KEY in the environment. No local GPU needed; costs accrue per second of audio processed.

```APIDOC
## OpenaiApiASR

### Description
Sends audio chunks to the OpenAI `/audio/transcriptions` or `/audio/translations` API. Requires `OPENAI_API_KEY` in the environment. No local GPU needed; costs accrue per second of audio processed.

### Usage
```python
import os
from whisper_online import OpenaiApiASR, load_audio_chunk

os.environ["OPENAI_API_KEY"] = "sk-"

# Auto language detection
asr = OpenaiApiASR(lan="auto")
# asr.set_translate_task()  # use translations endpoint instead

audio_chunk = load_audio_chunk("speech.wav", 0, 10)
result = asr.transcribe(audio_chunk, prompt="Previous context here.")

words = asr.ts_words(result)
for start, end, word in words[:3]:
    print(f"[{start:.2f}s – {end:.2f}s] {word!r}")

print(f"Total API seconds billed so far: {asr.transcribed_seconds}")
```

### Attributes
- `transcribed_seconds` (float): Total API seconds billed so far.
```

--------------------------------

### OnlineASRProcessor.process_iter()

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Runs Whisper on the current audio buffer, applies LocalAgreement-2 to find newly committed words, optionally trims the buffer, and returns a `(beg_timestamp, end_timestamp, text)` tuple. Returns `(None, None, "")` when no new text has been committed yet.

```APIDOC
## OnlineASRProcessor.process_iter()

### Description
Runs Whisper on the current audio buffer, applies LocalAgreement-2 to find newly committed words, optionally trims the buffer, and returns a `(beg_timestamp, end_timestamp, text)` tuple. Returns `(None, None, "")` when no new text has been committed yet.

### Returns
- **tuple**: A tuple containing `(beg_timestamp, end_timestamp, text)`. Returns `(None, None, "")` if no new text has been committed.

### Usage
```python
from whisper_online import FasterWhisperASR, OnlineASRProcessor, load_audio_chunk

asr = FasterWhisperASR(lan="en", modelsize="large-v2")
online = OnlineASRProcessor(asr)

# Feed 3 seconds of audio then call process_iter
chunk = load_audio_chunk("speech.wav", 0, 3)
online.insert_audio_chunk(chunk)

beg, end, text = online.process_iter()
if beg is not None:
    print(f"Committed: [{beg*1000:.0f}ms – {end*1000:.0f}ms] '{text}'")
    # Example: Committed: [120ms – 2400ms] 'Good morning everyone.'
else:
    print("No committed text yet — need more audio context.")
```
```

--------------------------------

### Feed Raw Audio to OnlineASRProcessor

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Append raw audio data (numpy float32 array, 16 kHz mono) to the internal buffer of OnlineASRProcessor. This should be called before each process_iter() call.

```python
import numpy as np
from whisper_online import FasterWhisperASR, OnlineASRProcessor

asr = FasterWhisperASR(lan="de", modelsize="small")
online = OnlineASRProcessor(asr)

# Simulate receiving 40ms frames from a microphone at 16kHz
FRAME_SIZE = int(0.04 * 16000)  # 640 samples = 40ms

for i in range(50):  # 50 frames = 2 seconds
    frame = np.zeros(FRAME_SIZE, dtype=np.float32)  # replace with real mic data
    online.insert_audio_chunk(frame)

o = online.process_iter()
print(o)  # (None, None, '') if nothing committed yet, or (beg, end, "text")
```

--------------------------------

### Set OpenAI API Key

Source: https://github.com/ufal/whisper_streaming/blob/main/README.md

Sets the OpenAI API key as an environment variable. Replace 'sk-xxx' with your actual API key.

```bash
export OPENAI_API_KEY=sk-xxx
```

--------------------------------

### asr_factory(args)

Source: https://context7.com/ufal/whisper_streaming/llms.txt

A factory function to instantiate ASR processors and online processors from command-line arguments. It handles backend selection, VAD activation, task settings, and tokenizer creation.

```APIDOC
### `asr_factory(args)` — Instantiate ASR + processor from CLI args

Factory function used by both `whisper_online.py` and `whisper_online_server.py` to build the correct `ASRBase` subclass and `OnlineASRProcessor` (or `VACOnlineASRProcessor`) from a parsed `argparse.Namespace`. Handles backend selection, VAD activation, task setting, and tokenizer creation.

```python
import argparse
from whisper_online import asr_factory, add_shared_args

parser = argparse.ArgumentParser()
add_shared_args(parser)

# Simulate: faster-whisper, English, large-v2, with VAC
args = parser.parse_args([
    "--backend", "faster-whisper",
    "--model", "large-v2",
    "--lan", "en",
    "--vac",
    "--vac-chunk-size", "0.04",
    "--buffer_trimming", "segment",
    "--buffer_trimming-sec", "15",
    "--log-level", "INFO"
])

asr, online = asr_factory(args)
# asr   → FasterWhisperASR instance
# online → VACOnlineASRProcessor instance (because --vac was set)
print(type(asr).__name__, type(online).__name__)
# Output: FasterWhisperASR VACOnlineASRProcessor
```
```

--------------------------------

### Core Streaming Processor with OnlineASRProcessor

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Utilize OnlineASRProcessor for real-time transcription, managing audio buffers and committing stable words using the LocalAgreement-2 policy. Supports segment-based buffer trimming.

```python
from whisper_online import FasterWhisperASR, OnlineASRProcessor, load_audio_chunk

# Set up backend
asr = FasterWhisperASR(lan="en", modelsize="large-v2")

# Create processor: "segment" trimming at 15s is the recommended default
online = OnlineASRProcessor(
    asr,
    tokenizer=None,                    # required only for buffer_trimming="sentence"
    buffer_trimming=("segment", 15),   # trim audio buffer at confirmed segment boundaries once > 15s
    logfile=open("transcript.log", "w")
)

# Simulate real-time streaming from a file in 1-second chunks
import numpy as np
audio_path = "long_speech.wav"
chunk_size = 1.0  # seconds
duration = 60.0   # process 60 seconds

beg = 0.0
while beg < duration:
    end = min(beg + chunk_size, duration)
    chunk = load_audio_chunk(audio_path, beg, end)

    online.insert_audio_chunk(chunk)         # feed new audio
    result = online.process_iter()           # get committed output (if any)

    beg_ts, end_ts, text = result
    if text:
        print(f"[{beg_ts:.2f}s – {end_ts:.2f}s] {text}")
    # Example output:
    # [0.12s – 2.40s] Hello and welcome to today's session.
    # [2.41s – 5.10s] We're going to cover streaming transcription.

    beg = end

# Flush any remaining uncommitted words at the end of the stream
final_beg, final_end, final_text = online.finish()
if final_text:
    print(f"[FINAL] [{final_beg:.2f}s – {final_end:.2f}s] {final_text}")

# To reuse the processor for a new audio stream:
online.init()
```

--------------------------------

### load_audio and load_audio_chunk

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Functions to load audio files. `load_audio` loads the entire file at 16 kHz, and results are cached. `load_audio_chunk` loads a specific portion of the audio file.

```APIDOC
## load_audio(fname)

### Description
Loads a WAV (or any librosa-compatible) audio file resampled to 16 kHz mono float32. Results are LRU-cached.

### Parameters
- **fname** (string) - Path to the audio file.

### Returns
- numpy.ndarray: Audio data as a float32 numpy array.

### Example
```python
from whisper_online import load_audio, load_audio_chunk
import numpy as np

# Load the entire file (cached after first call)
audio = load_audio("speech.wav")
print(f"Loaded {len(audio)/16000:.2f} seconds of audio, dtype={audio.dtype}")

# Load only seconds 5.0 – 10.0
chunk = load_audio_chunk("speech.wav", beg=5.0, end=10.0)
print(f"Chunk length: {len(chunk)} samples ({len(chunk)/16000:.2f}s)")
```

## load_audio_chunk(fname, beg, end)

### Description
Loads a specific chunk of an audio file, resampled to 16 kHz mono float32.

### Parameters
- **fname** (string) - Path to the audio file.
- **beg** (float) - Start time in seconds.
- **end** (float) - End time in seconds.

### Returns
- numpy.ndarray: Audio data for the specified chunk.

### Example
```python
from whisper_online import load_audio_chunk

chunk = load_audio_chunk("speech.wav", beg=5.0, end=10.0)
print(f"Chunk length: {len(chunk)} samples ({len(chunk)/16000:.2f}s)")
```
```

--------------------------------

### Manual Model Path Configuration for MLXWhisper

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Specify a custom directory for MLXWhisper models to override the default modelsize.

```python
asr_custom = MLXWhisper(lan="en", model_dir="/models/my-custom-mlx-whisper")
```

--------------------------------

### Load Audio Files with Whisper Streaming

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Loads a WAV audio file resampled to 16 kHz mono float32. Results are LRU-cached for efficiency. Can also load specific chunks of audio.

```python
from whisper_online import load_audio, load_audio_chunk
import numpy as np

# Load the entire file (cached after first call)
audio = load_audio("speech.wav")
print(f"Loaded {len(audio)/16000:.2f} seconds of audio, dtype={audio.dtype}")
# Output: Loaded 42.30 seconds of audio, dtype=float32

# Load only seconds 5.0 – 10.0
chunk = load_audio_chunk("speech.wav", beg=5.0, end=10.0)
print(f"Chunk length: {len(chunk)} samples ({len(chunk)/16000:.2f}s)")
# Output: Chunk length: 80000 samples (5.00s)
```

--------------------------------

### create_tokenizer(lan)

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Creates a language-specific sentence tokenizer for buffer trimming. It supports over 100 languages using various underlying libraries like opus-fast-mosestokenizer, tokenize_uk, or wtpsplit.

```APIDOC
### `create_tokenizer(lan)` — Sentence segmenter for buffer trimming

Returns a language-specific sentence tokenizer with a `.split(text)` method, required when using `buffer_trimming="sentence"` mode. Supports 100+ languages via `opus-fast-mosestokenizer`, `tokenize_uk` (Ukrainian), or `wtpsplit` (multilingual neural fallback).

```python
from whisper_online import create_tokenizer

# English — uses MosesTokenizer
tok = create_tokenizer("en")
sentences = tok.split("Hello world. How are you? I am fine.")
print(sentences)
# Output: ['Hello world.', 'How are you?', 'I am fine.']

# Ukrainian — uses tokenize_uk
tok_uk = create_tokenizer("uk")
sentences_uk = tok_uk.split("Привіт світ. Як справи?")
print(sentences_uk)
# Output: ['Привіт світ.', 'Як справи?']

# Japanese (not in Moses/uk) — falls back to wtpsplit neural model
tok_ja = create_tokenizer("ja")
```
```

--------------------------------

### Create language-specific sentence tokenizer with create_tokenizer

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Returns a language-specific sentence tokenizer with a `.split(text)` method, required for `buffer_trimming="sentence"` mode. Supports 100+ languages via various backends.

```python
from whisper_online import create_tokenizer

# English — uses MosesTokenizer
tok = create_tokenizer("en")
sentences = tok.split("Hello world. How are you? I am fine.")
print(sentences)
# Output: ['Hello world.', 'How are you?', 'I am fine.']

# Ukrainian — uses tokenize_uk
tok_uk = create_tokenizer("uk")
sentences_uk = tok_uk.split("Привіт світ. Як справи?")
print(sentences_uk)
# Output: ['Привіт світ.', 'Як справи?']

# Japanese (not in Moses/uk) — falls back to wtpsplit neural model
tok_ja = create_tokenizer("ja")
```

--------------------------------

### VACOnlineASRProcessor

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Voice Activity Control wrapper that integrates Silero VAD with OnlineASRProcessor. It triggers transcription only during detected speech and finalizes utterances after silence, optimizing inference and latency.

```APIDOC
### `VACOnlineASRProcessor` — Voice Activity Control wrapper

Wraps `OnlineASRProcessor` with Silero VAD to detect speech/silence in real time. It triggers `OnlineASRProcessor` only during detected speech and forces a `finish()` call at the end of each utterance (500 ms of silence), reducing unnecessary inference and latency.

```python
import torch
import numpy as np
from whisper_online import FasterWhisperASR, VACOnlineASRProcessor, load_audio_chunk

asr = FasterWhisperASR(lan="en", modelsize="large-v2")

# online_chunk_size: seconds of voice-active audio to accumulate before running Whisper
online = VACOnlineASRProcessor(
    online_chunk_size=0.04,  # run Whisper every ~40ms of voice
    asr=asr,
    tokenizer=None,
    buffer_trimming=("segment", 15)
)

# Stream 40ms frames (VAC processes them internally)
FRAME = int(0.04 * 16000)
audio_path = "speech_with_pauses.wav"
full_audio = load_audio_chunk(audio_path, 0, 60)

for i in range(0, len(full_audio), FRAME):
    chunk = full_audio[i:i+FRAME]
    if len(chunk) < FRAME:
        break
    online.insert_audio_chunk(chunk)
    beg, end, text = online.process_iter()
    if text:
        print(f"[{beg:.2f}s – {end:.2f}s] {text}")
    # VAC suppresses output during silence; only prints during/after speech

online.finish()
online.init()  # reset for next utterance
```
```

--------------------------------

### Robust Silero VAD Iterator

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Handles arbitrary audio chunk sizes by buffering internally. Use when dealing with variable-length audio inputs for Silero VAD.

```python
import torch
import numpy as np
from silero_vad_iterator import FixedVADIterator

# Load Silero VAD model
model, _ = torch.hub.load(repo_or_dir='snakers4/silero-vad', model='silero_vad')

# Create iterator with 500ms silence threshold, 100ms speech padding
vac = FixedVADIterator(
    model,
    threshold=0.5,
    sampling_rate=16000,
    min_silence_duration_ms=500,
    speech_pad_ms=100
)

# Process any-length chunk (e.g. 40ms = 640 samples)
audio_chunk = np.random.randn(640).astype(np.float32) * 0.01  # near-silence
result = vac(audio_chunk)
print(result)  # None (no speech event detected)

# Simulate speech start
loud_chunk = np.random.randn(640).astype(np.float32) * 0.5
result = vac(loud_chunk)
if result and 'start' in result:
    print(f"Speech started at sample {result['start']}")
elif result and 'end' in result:
    print(f"Speech ended at sample {result['end']}")

# Reset for new audio stream
vac.reset_states()
```

--------------------------------

### whisper_online_server.py

Source: https://context7.com/ufal/whisper_streaming/llms.txt

A standalone TCP server for real-time audio transcription. It accepts raw PCM audio, processes it using OnlineASRProcessor, and streams back timestamped text lines to clients.

```APIDOC
### `whisper_online_server.py` — TCP streaming server

A standalone TCP server that listens for raw PCM audio (16 kHz, mono, S16_LE) from a client, transcribes it in real time using `OnlineASRProcessor`, and streams back timestamped text lines. One client is served per connection; a new `ServerProcessor` is created per connection.

```bash
# Start server on default port 43007 with faster-whisper large-v2,
# English, VAC enabled, and a warmup file to pre-load the model
python3 whisper_online_server.py \
    --backend faster-whisper \
    --model large-v2 \
    --lan en \
    --vac \
    --warmup-file /path/to/warmup.wav \
    --host 0.0.0.0 \
    --port 43007 \
    --log-level INFO

# Client: stream live mic audio via arecord + netcat
arecord -f S16_LE -c1 -r 16000 -t raw -D default | nc localhost 43007

# The server writes transcript lines to stdout in the format:
# <beg_ms> <end_ms> <text>
# Example server stdout:

```

--------------------------------

### OnlineASRProcessor.finish()

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Forces all pending words in the hypothesis buffer to be emitted. This should be called once at the very end of the audio stream to ensure all transcribed text is captured.

```APIDOC
## `OnlineASRProcessor.finish()` — Flush remaining transcript

Forces all words still pending in the hypothesis buffer to be emitted, regardless of whether LocalAgreement-2 has been satisfied. Call once at the very end of the audio stream.

```python
from whisper_online import FasterWhisperASR, OnlineASRProcessor, load_audio_chunk

asr = FasterWhisperASR(lan="en", modelsize="large-v2")
online = OnlineASRProcessor(asr)

audio = load_audio_chunk("speech.wav", 0, 30)
online.insert_audio_chunk(audio)
online.process_iter()  # normal streaming update

# End of stream — flush remainder
beg, end, text = online.finish()
print(f"Final segment: [{beg:.2f}s – {end:.2f}s] '{text}'")
# Example: Final segment: [27.40s – 29.88s] 'Thank you for listening.'
```
```

--------------------------------

### MLXWhisper

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Implementation of the ASR interface optimized for Apple Silicon (M1/M2/M3) using the `mlx-whisper` library.

```APIDOC
## MLXWhisper

### Description
An ASR backend specifically optimized for Apple Silicon (M1/M2/M3) chips, utilizing the `mlx-whisper` library. It automatically maps standard Whisper model names to their HuggingFace MLX variants.

### Initialization
```python
from whisper_online import MLXWhisper, load_audio_chunk

# Automatically maps "large-v3-turbo" → "mlx-community/whisper-large-v3-turbo"
asr = MLXWhisper(lan="en", modelsize="large-v3-turbo")
```

### Transcribe Audio

#### Method
`transcribe(audio_chunk)`

#### Parameters
- **audio_chunk** (numpy.ndarray) - A numpy float32 audio array.

#### Returns
- List of segments representing the transcription results.

### Get Timestamped Words

#### Method
`ts_words(segments)`

#### Parameters
- **segments** (List) - The list of segments returned by `transcribe`.

#### Returns
- List of tuples: `[(start_sec, end_sec, "word"), ...]`

### Example
```python
from whisper_online import MLXWhisper, load_audio_chunk

asr = MLXWhisper(lan="en", modelsize="large-v3-turbo")

audio_chunk = load_audio_chunk("speech.wav", 0, 10)
segments = asr.transcribe(audio_chunk)
words = asr.ts_words(segments)
print(words[:3])
```
```

--------------------------------

### MLXWhisper Backend for Apple Silicon

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Optimized for Apple M1/M2/M3 chips using the mlx-whisper library. It automatically maps standard Whisper model names to their HuggingFace MLX variants.

```python
from whisper_online import MLXWhisper, load_audio_chunk

# Automatically maps "large-v3-turbo" → "mlx-community/whisper-large-v3-turbo"
asr = MLXWhisper(lan="en", modelsize="large-v3-turbo")

audio_chunk = load_audio_chunk("speech.wav", 0, 10)
segments = asr.transcribe(audio_chunk)
words = asr.ts_words(segments)
print(words[:3])
# Output: [(0.10, 0.42, ' Hello'), (0.43, 0.71, ' world'), (0.72, 1.00, '.')]
```

--------------------------------

### OnlineASRProcessor.insert_audio_chunk(audio)

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Appends a numpy float32 array (16 kHz mono) to the internal audio buffer. Call this in a loop before each `process_iter()` call.

```APIDOC
## OnlineASRProcessor.insert_audio_chunk(audio)

### Description
Appends a numpy float32 array (16 kHz mono) to the internal audio buffer. Call this in a loop before each `process_iter()` call.

### Parameters
- **audio** (np.ndarray): A numpy float32 array representing audio data (16 kHz mono).

### Usage
```python
import numpy as np
from whisper_online import FasterWhisperASR, OnlineASRProcessor

asr = FasterWhisperASR(lan="de", modelsize="small")
online = OnlineASRProcessor(asr)

# Simulate receiving 40ms frames from a microphone at 16kHz
FRAME_SIZE = int(0.04 * 16000)  # 640 samples = 40ms

for i in range(50):  # 50 frames = 2 seconds
    frame = np.zeros(FRAME_SIZE, dtype=np.float32)  # replace with real mic data
    online.insert_audio_chunk(frame)

o = online.process_iter() # This call is just to show the effect, typically called after inserting chunks
print(o)  # (None, None, '') if nothing committed yet, or (beg, end, "text")
```
```

--------------------------------

### Flush remaining transcript with OnlineASRProcessor.finish()

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Call `finish()` once at the end of the audio stream to emit any remaining words in the hypothesis buffer. This ensures all pending audio is processed.

```python
from whisper_online import FasterWhisperASR, OnlineASRProcessor, load_audio_chunk

asr = FasterWhisperASR(lan="en", modelsize="large-v2")
online = OnlineASRProcessor(asr)

audio = load_audio_chunk("speech.wav", 0, 30)
online.insert_audio_chunk(audio)
online.process_iter()  # normal streaming update

# End of stream — flush remainder
beg, end, text = online.finish()
print(f"Final segment: [{beg:.2f}s – {end:.2f}s] '{text}'")
# Example: Final segment: [27.40s – 29.88s] 'Thank you for listening.'
```

--------------------------------

### FasterWhisperASR

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Implementation of the ASR interface using the `faster-whisper` library, optimized for GPU inference with CTranslate2.

```APIDOC
## FasterWhisperASR

### Description
The recommended backend for Whisper Streaming, wrapping the `faster-whisper` library for efficient CUDA FP16 inference. It supports transcription and translation tasks and can produce word-level timestamps.

### Initialization
```python
from whisper_online import FasterWhisperASR, load_audio_chunk

# Initialize: loads large-v2 onto CUDA with float16
asr = FasterWhisperASR(lan="en", modelsize="large-v2")
```

### Optional Configurations
```python
# Optional: switch to translation (output is always English)
asr.set_translate_task()

# Optional: enable built-in VAD filter
asr.use_vad()
```

### Transcribe Audio

#### Method
`transcribe(audio_chunk, init_prompt="")`

#### Parameters
- **audio_chunk** (numpy.ndarray) - A numpy float32 audio array.
- **init_prompt** (string, optional) - An initial prompt to guide transcription.

#### Returns
- List of segments representing the transcription results.

### Get Timestamped Words

#### Method
`ts_words(segments)`

#### Parameters
- **segments** (List) - The list of segments returned by `transcribe`.

#### Returns
- List of tuples: `[(start_sec, end_sec, "word"), ...]`

### Get Segment End Timestamps

#### Method
`segments_end_ts(segments)`

#### Parameters
- **segments** (List) - The list of segments returned by `transcribe`.

#### Returns
- List of floats representing the end timestamps of each segment.

### Example
```python
from whisper_online import FasterWhisperASR, load_audio_chunk

asr = FasterWhisperASR(lan="en", modelsize="large-v2")
audio_chunk = load_audio_chunk("speech.wav", 0, 10)
segments = asr.transcribe(audio_chunk, init_prompt="")

words = asr.ts_words(segments)
for start, end, word in words[:5]:
    print(f"[{start:.2f}s – {end:.2f}s] {word!r}")

ends = asr.segments_end_ts(segments)
print("Segment ends:", ends)
```
```

--------------------------------

### OpenAI Whisper API Transcription with OpenaiApiASR

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Send audio chunks to the OpenAI API for transcription. Requires OPENAI_API_KEY. Auto language detection is supported.

```python
import os
from whisper_online import OpenaiApiASR, load_audio_chunk

os.environ["OPENAI_API_KEY"] = "sk-"

# Auto language detection
asr = OpenaiApiASR(lan="auto")
# asr.set_translate_task()  # use translations endpoint instead

audio_chunk = load_audio_chunk("speech.wav", 0, 10)
result = asr.transcribe(audio_chunk, prompt="Previous context here.")

words = asr.ts_words(result)
for start, end, word in words[:3]:
    print(f"[{start:.2f}s – {end:.2f}s] {word!r}")

print(f"Total API seconds billed so far: {asr.transcribed_seconds}")
# Output: Total API seconds billed so far: 10
```

--------------------------------

### FasterWhisperASR GPU Backend for Transcription

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Uses the faster-whisper library with CTranslate2 for efficient CUDA FP16 inference. Supports transcription and translation, and can filter high-no-speech-probability segments. Requires initialization with language and model size.

```python
from whisper_online import FasterWhisperASR, load_audio_chunk

# Initialize: loads large-v2 onto CUDA with float16
asr = FasterWhisperASR(lan="en", modelsize="large-v2")

# Optional: switch to translation (output is always English)
# asr.set_translate_task()

# Optional: enable built-in VAD filter
# asr.use_vad()

# Transcribe a numpy float32 audio array
audio_chunk = load_audio_chunk("speech.wav", 0, 10)
segments = asr.transcribe(audio_chunk, init_prompt="")

# Extract timestamped words: [(start_sec, end_sec, "word"), ...]
words = asr.ts_words(segments)
for start, end, word in words[:5]:
    print(f"[{start:.2f}s – {end:.2f}s] {word!r}")
# Output:
# [0.12s – 0.45s] ' Hello'
# [0.46s – 0.78s] ' and'
# [0.79s – 1.10s] ' welcome'
# [1.11s – 1.35s] ' to'
# [1.36s – 1.72s] ' the'

# Get segment end timestamps (used for buffer trimming)
ends = asr.segments_end_ts(segments)
print("Segment ends:", ends)
# Output: Segment ends: [4.82, 9.61]
```

--------------------------------

### OnlineASRProcessor

Source: https://context7.com/ufal/whisper_streaming/llms.txt

The central object for real-time transcription. Maintains an audio buffer and hypothesis buffer, runs Whisper on each new chunk, applies the LocalAgreement-2 policy to commit stable words, and trims the audio buffer at confirmed boundaries to keep latency and memory bounded.

```APIDOC
## OnlineASRProcessor

### Description
The central object for real-time transcription. Maintains an audio buffer and hypothesis buffer, runs Whisper on each new chunk, applies the LocalAgreement-2 policy to commit stable words, and trims the audio buffer at confirmed boundaries to keep latency and memory bounded.

### Initialization
```python
from whisper_online import FasterWhisperASR, OnlineASRProcessor

# Set up backend
asr = FasterWhisperASR(lan="en", modelsize="large-v2")

# Create processor: "segment" trimming at 15s is the recommended default
online = OnlineASRProcessor(
    asr,
    tokenizer=None,                    # required only for buffer_trimming="sentence"
    buffer_trimming=("segment", 15),   # trim audio buffer at confirmed segment boundaries once > 15s
    logfile=open("transcript.log", "w")
)
```

### Usage Example
```python
from whisper_online import FasterWhisperASR, OnlineASRProcessor, load_audio_chunk
import numpy as np

# Set up backend
asr = FasterWhisperASR(lan="en", modelsize="large-v2")

# Create processor
online = OnlineASRProcessor(
    asr,
    buffer_trimming=("segment", 15)
)

# Simulate real-time streaming from a file in 1-second chunks
audio_path = "long_speech.wav"
chunk_size = 1.0  # seconds
duration = 60.0   # process 60 seconds

beg = 0.0
while beg < duration:
    end = min(beg + chunk_size, duration)
    chunk = load_audio_chunk(audio_path, beg, end)

    online.insert_audio_chunk(chunk)         # feed new audio
    result = online.process_iter()           # get committed output (if any)

    beg_ts, end_ts, text = result
    if text:
        print(f"[{beg_ts:.2f}s – {end_ts:.2f}s] {text}")

    beg = end

# Flush any remaining uncommitted words at the end of the stream
final_beg, final_end, final_text = online.finish()
if final_text:
    print(f"[FINAL] [{final_beg:.2f}s – {final_end:.2f}s] {final_text}")

# To reuse the processor for a new audio stream:
online.init()
```
```

--------------------------------

### Socket Text Transport Utilities

Source: https://context7.com/ufal/whisper_streaming/llms.txt

Utilities for sending and receiving newline-terminated text lines over TCP sockets using fixed-size packets. Useful for server-client communication of transcript lines.

```python
import socket
import line_packet

# Server side: send a transcript line
def handle_client(conn):
    line_packet.send_one_line(conn, "1200 3400 Hello from the server")

# Client side: receive lines non-blocking
def read_results(conn):
    lines = line_packet.receive_lines(conn)  # returns [] on no data, None on closed
    if lines is None:
        print("Connection closed")
        return
    for line in lines:
        if line.strip():
            beg_ms, end_ms, *words = line.strip().split()
            text = " ".join(words)
            print(f"[{beg_ms}ms – {end_ms}ms] {text}")

# Minimal client connecting to the server:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.connect(("localhost", 43007))
    s.setblocking(False)
    # send raw PCM audio bytes...
    # s.sendall(raw_pcm_bytes)
    lines = line_packet.receive_lines(s)
    if lines:
        for l in lines:
            print(l)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.