### Install Faster Qwen3 TTS with Demo

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Install the package with demo extras. This command installs the necessary dependencies to run the example server.

```bash
pip install "faster-qwen3-tts[demo]"
```

--------------------------------

### Demo UI Setup and Launch

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Install the demo dependencies and launch the web UI. Access the UI via http://localhost:7860 in your browser.

```bash
pip install -e ".[demo]"
python demo/server.py
```

--------------------------------

### Install sounddevice for Local Playback

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Install the sounddevice library for local speaker playback from a repository checkout using the example helper.

```bash
pip install sounddevice
```

--------------------------------

### Run Windows Setup Script

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/WINDOWS_SETUP_GUIDE.md

Execute the main setup script for Windows. This script handles environment creation, dependency installation, and model pre-downloading.

```batch
setup_windows.bat
```

--------------------------------

### Install and Run Locally

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/README.md

Install the necessary package and run the demo locally. Ensure you have Python installed. The server can be accessed at http://localhost:7860.

```bash
pip install "faster-qwen3-tts[demo]"
python server.py --model Qwen/Qwen3-TTS-12Hz-0.6B-Base
# open http://localhost:7860
```

--------------------------------

### Clone Repository and Setup for Linux/macOS/WSL

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Clones the project repository and runs the setup script. This is the initial step for benchmarking on Linux-based systems.

```bash
git clone https://github.com/andimarafioti/faster-qwen3-tts
cd faster-qwen3-tts
./setup.sh
```

--------------------------------

### Start OpenAI-Compatible Server

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Installs the necessary package and starts the OpenAI-compatible server with a specified model and reference audio. This server can be used with various HTTP clients that support the OpenAI TTS protocol.

```bash
pip install "faster-qwen3-tts[demo]"
python examples/openai_server.py \
    --model Qwen/Qwen3-TTS-12Hz-1.7B-Base \
    --ref-audio ref_audio.wav \
    --ref-text "I'm confused why some people have super short timelines..." \
    --language English \
    --port 8000
```

--------------------------------

### Clone Repository and Setup for Windows

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Clones the project repository and runs the setup batch script for Windows. This is the initial step for benchmarking on native Windows.

```cmd
git clone https://github.com/andimarafioti/faster-qwen3-tts
cd faster-qwen3-tts
setup_windows.bat
```

--------------------------------

### Install Dependencies

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/WINDOWS_SETUP_GUIDE.md

Upgrade pip and install project dependencies, including PyTorch. Use this if not using the setup script.

```shell
pip install --upgrade pip
```

```shell
pip install -e .
```

--------------------------------

### Install Flash Attention (Optional)

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/WINDOWS_SETUP_GUIDE.md

Install the flash-attn library, which is optional and may require a compiler.

```shell
pip install flash-attn
```

--------------------------------

### Clone Faster Qwen3-TTS Repository and Setup

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/BLOG.md

Clones the faster-qwen3-tts repository, sets up the environment with dependencies and models, and runs a streaming benchmark. This is the initial setup for deploying the TTS model.

```bash
git clone https://github.com/andimarafioti/faster-qwen3-tts
cd faster-qwen3-tts
./setup.sh       # creates venv with uv, installs deps, downloads models
./benchmark.sh   # runs streaming benchmark, saves JSON + audio samples
```

--------------------------------

### Install Faster Qwen3-TTS

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Install the library using pip. Ensure you have Python 3.10+ and PyTorch 2.5.1+ with a compatible NVIDIA GPU.

```bash
pip install faster-qwen3-tts
```

--------------------------------

### Run with Docker

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/README.md

Build the Docker image and run the demo container. This method requires Docker to be installed and configured to use GPUs.

```bash
docker build -t faster-qwen3-tts-demo .
docker run --gpus all -p 7860:7860 faster-qwen3-tts-demo
```

--------------------------------

### Run Windows Benchmark Script

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/WINDOWS_SETUP_GUIDE.md

Execute the benchmark script after setup. Allows specifying which model size or mode to benchmark.

```batch
benchmark_windows.bat
```

```batch
benchmark_windows.bat 0.6B
```

```batch
benchmark_windows.bat 1.7B
```

```batch
benchmark_windows.bat custom
```

```batch
benchmark_windows.bat both
```

--------------------------------

### CLI Quick Start: VoiceDesign

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Generate speech based on instructional prompts using the VoiceDesign model. Specify the model, instruction, text, language, and output file.

```bash
faster-qwen3-tts design \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign \
  --instruct "Warm, confident narrator with slight British accent" \
  --text "Welcome to the show." \
  --language English \
  --output out.wav
```

--------------------------------

### CLI Quick Start: Streaming Generation to WAV

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Perform streaming generation and save the output directly to a WAV file. The command prints the Real-Time Factor (RTF) after the write is complete.

```bash
faster-qwen3-tts custom \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --speaker aiden \
  --text "What do you mean that I'm not real?" \
  --language English \
  --output out.wav \
  --streaming
```

--------------------------------

### Run OpenAI-compatible API Server

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Starts the OpenAI-compatible TTS API server. Requires reference audio and text for voice cloning. The server listens on the specified port.

```bash
python examples/openai_server.py \
    --ref-audio ref_audio.wav \
    --ref-text "I'm confused why some people have super short timelines, yet at the same time are bullish on scaling up reinforcement learning atop LLMs. If we're actually close to a human-like learner, then this whole approach of training on verifiable outcomes is doomed." \
    --language English --port 8000
```

--------------------------------

### Python Quick Start: Voice Cloning with Streaming and Non-Streaming

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Demonstrates voice cloning using FasterQwen3TTS. The streaming option yields audio chunks during generation, while the non-streaming option returns all audio at once. Requires a reference audio file and text.

```python
from examples.audio import StreamPlayer  # helper from this repo's examples/ 
from faster_qwen3_tts import FasterQwen3TTS

model = FasterQwen3TTS.from_pretrained("Qwen/Qwen3-TTS-12Hz-0.6B-Base")
ref_audio = "ref_audio.wav"
ref_text = (
    "I'm confused why some people have super short timelines, yet at the same time are bullish on scaling up "
    "reinforcement learning atop LLMs. If we're actually close to a human-like learner, then this whole approach "
    "of training on verifiable outcomes is doomed."
)

# Streaming — yields audio chunks during generation
play = StreamPlayer()
try:
    for audio_chunk, sr, timing in model.generate_voice_clone_streaming(
        text="What do you mean that I'm not real?", language="English",
        ref_audio=ref_audio, ref_text=ref_text,
        chunk_size=8,  # 8 steps ≈ 667ms of audio per chunk
    ):
        play(audio_chunk, sr)
finally:
    play.close()

# Non-streaming — returns all audio at once
audio_list, sr = model.generate_voice_clone(
    text="Hello world!", language="English",
    ref_audio=ref_audio, ref_text=ref_text,
)
```

--------------------------------

### CLI Quick Start: Voice Cloning

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Perform voice cloning using the command-line interface. Requires specifying the model, text, language, reference audio, and reference text.

```bash
faster-qwen3-tts clone \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-Base \
  --text "What do you mean that I'm not real?" \
  --language English \
  --ref-audio ref_audio.wav \
  --ref-text "I'm confused why some people have super short timelines, yet at the same time are bullish on scaling up reinforcement learning atop LLMs. If we're actually close to a human-like learner, then this whole approach of training on verifiable outcomes is doomed." \
  --output out.wav
```

--------------------------------

### CLI Quick Start: CustomVoice

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Utilize predefined speaker IDs for voice generation with the CustomVoice model. Use `--list-speakers` to see available options.

```bash
faster-qwen3-tts custom --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --list-speakers
```

```bash
faster-qwen3-tts custom \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --speaker aiden \
  --text "What do you mean that I'm not real?" \
  --language English \
  --output out.wav
```

--------------------------------

### CLI Quick Start: Server Mode

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Run the TTS model in server mode for persistent inference. The model remains loaded for subsequent requests. Stop the server by typing 'exit'.

```bash
faster-qwen3-tts serve \
  --mode custom \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --speaker aiden \
  --language English \
  --streaming
```

--------------------------------

### Create Python Virtual Environment

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/WINDOWS_SETUP_GUIDE.md

Manually create a Python virtual environment named '.venv'. Ensure Python 3.10+ is installed and in your PATH.

```shell
python -m venv .venv
```

--------------------------------

### Start Audio Recording

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Initiates audio recording using the browser's MediaDevices API. Includes checks for secure context and user media support. Sets up audio processing graph.

```javascript
async function startRec() { if (!navigator.mediaDevices?.getUserMedia) { showMsg('err', 'Recording not supported. Use HTTPS or localhost.'); return; } // Secure-context guard — 0.0.0.0 will silently fail without this if (!window.isSecureContext) { const { hostname, port } = window.location; if (hostname !== 'localhost' && hostname !== '127.0.0.1') { showMsg('err', `Recording requires HTTPS or localhost. Try http://localhost:${port}/`); return; } } hideMsg(); recChunks = []; recPeak = 0; recHasSignal = false; recSignalTs = null; recSignalCount = 0; recActive = true; $('micBtn').classList.add('rec'); $('recStatus').textContent = 'Starting mic…'; $('recBarWrap').style.display = 'block'; $('recPreview').style.display = 'none'; try { const constraints = { audio: { channelCount: 1, echoCancellation: false, noiseSuppression: false, autoGainControl: false } }; if (recDeviceId) constraints.audio.deviceId = { exact: recDeviceId }; recStream = await navigator.mediaDevices.getUserMedia(constraints); await loadMics(); // refresh labels now that permission is granted recCtx = new (window.AudioContext || window.webkitAudioContext)({ latencyHint: 'interactive' }); if (recCtx.state === 'suspended') await recCtx.resume(); recSource = recCtx.createMediaStreamSource(recStream); recAnalyser = recCtx.createAnalyser(); recAnalyser.fftSize = 2048; recGain = recCtx.createGain(); recGain.gain.value = 0.0001; // near-silent, keeps graph alive recProc = recCtx.createScriptProcessor(4096, 1, 1); recProc.onaudioprocess = e => { if (!recActive) return; const inp = e.inputBuffer.getChannelData(0); const copy = new Float32Array(inp.length); for (let i = 0; i < inp.length; i++) { copy[i] = inp[i]; const a = Math.abs(inp[i]); if (a > recPeak) recPeak = a; } recChunks.push(copy); }; recSource.connect(recAnalyser); recAnalyser.connect(recProc); recProc.connect(recGain); recGain.connect(recCtx.destination); recMeterBuf = new Uint8Array(recAnalyser.fftSize); const tick = () => { if (!recActive) return; recAnalyser.getByteTimeDomainData(recMeterBuf); let peak = 0; for (let i = 0; i < recMeterBuf.length; i++) { const v = Math.abs(recMeterBuf[i] - 128); if (v > peak) peak = v; } const norm = Math.min(1, peak / 64); $('recBar').style.width = Math.round(norm * 100) + '%'; if (norm > 0.02) recSignalCount++; else recSignalCount = 0; if (!recHasSignal && recSignalCount >= 4) { recHasSignal = true; recSignalTs = performance.now(); } $('recStatus').textContent = recHasSignal ? 'Recording…' : 'Waiting for signal…'; if (recHasSignal) { const s = Math.floor((performance.now() - recSignalTs) / 1000); $('recTime').textContent = Math.floor(s / 60) + ':' + String(s % 60).padStart(2, '0'); } recMeterRaf = requestAnimationFrame(tick); }; recMeterRaf = requestAnimationFrame(tick); } catch (e) { showMsg('err', 'Mic failed: ' + e.message); cleanupRec(); } }
```

--------------------------------

### Handle Keyboard Shortcuts

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Listens for keyboard events to trigger actions. Ctrl+Enter or Meta+Enter starts generation, and Escape can close overlays.

```javascript
document.addEventListener('keydown', e => {
  if ((e.metaKey || e.ctrlKey) && e.key === 'Enter') {
    e.preventDefault();
    // Pick best mode based on what's filled in
    const mode = refFile ? 'voice_clone' : $('speakerId').value ? 'custom' : 'voice_design';
    generate(mode);
  }
  if (e.key === 'Escape') {
    if ($('loaderOv').classList.contains('open')) $('loaderOv').classList.remove('open');
    else if ($('settingsOv').classList.contains('open')) closeSettings();
  }
});
```

--------------------------------

### CLI: Voice Cloning (ICL Mode)

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Perform voice cloning using the `faster-qwen3-tts clone` command. This example demonstrates the default ICL mode, specifying model, text, language, reference audio, reference text, and output file.

```bash
# Voice cloning (ICL mode, default)
faster-qwen3-tts clone \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-Base \
  --text "What do you mean that I'm not real?" \
  --language English \
  --ref-audio ref_audio.wav \
  --ref-text "I'm confused why some people have super short timelines..." \
  --output out_clone.wav
```

--------------------------------

### Stream Audio Playback with StreamPlayer

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Utilizes `StreamPlayer` from `examples.audio` for gap-free audio playback of streaming TTS output. It enqueues audio chunks and plays them from a separate audio thread, preventing gaps. Ensure `sounddevice` is installed for this functionality.

```python
from examples.audio import StreamPlayer
from faster_qwen3_tts import FasterQwen3TTS

model = FasterQwen3TTS.from_pretrained("Qwen/Qwen3-TTS-12Hz-0.6B-Base")

player = StreamPlayer(
    channels=1,           # mono output (matches Qwen3-TTS 24kHz mono)
    dtype="float32",
    max_queue_chunks=0,   # unbounded queue (set >0 for backpressure)
)

try:
    for audio_chunk, sr, timing in model.generate_voice_clone_streaming(
        text="Faster than real time, smoother than ever.",
        language="English",
        ref_audio="ref_audio.wav",
        ref_text="I'm confused why some people have super short timelines...",
        chunk_size=8,
    ):
        player(audio_chunk, sr)   # non-blocking enqueue; plays from audio thread
finally:
    player.close(wait=True, timeout=10.0)  # drain remaining audio before exit
```

--------------------------------

### Install PyTorch with CUDA Support

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/WINDOWS_SETUP_GUIDE.md

Manually install the CUDA-enabled version of PyTorch if the automatic setup fails or CUDA is not detected. Ensure your CUDA version matches the wheel (e.g., cu128 for CUDA 12.8).

```shell
.venv\Scripts\pip install "torch>=2.7.0" --index-url https://download.pytorch.org/whl/cu128
```

--------------------------------

### Initialization Logic for TTS Demo

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Handles the window load event to initialize the theme, fetch status, load the default model, and load microphones. Also sets up an audio warm-up listener.

```javascript
window.addEventListener('load', async () => {
  initTheme();
  autoGrow($('voiceInstr'));
  await fetchStatus();
  if (!loadedModel && availableModels.length > 0) {
    // Prefer CustomVoice as default (richer demo), fall back to first available
    const preferred = availableModels.find(m => m.includes('CustomVoice')) || availableModels[0];
    $('modelSel').value = preferred;
    loadModel();
  }
  loadMics();
  populateSpeakers([]);
  const warm = () => {
    warmAudio();
    window.removeEventListener('pointerdown', warm);
  };
  window.addEventListener('pointerdown', warm, { passive: true });
});
```

--------------------------------

### CLI: Serve Mode for Continuous Generation

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Run the `faster-qwen3-tts serve` command to keep the model loaded in memory for continuous generation. This mode accepts text input line-by-line from stdin, ideal for batch or interactive use without repeated model loading.

```bash
# Serve mode — keep model hot, generate multiple requests
faster-qwen3-tts serve \
  --mode custom \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --speaker aiden \
  --language English \
  --streaming
# Then type lines of text at the prompt; files are written to ./outputs/
```

--------------------------------

### Reset Performance Metrics (JavaScript)

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Resets the displayed performance metrics on the UI to their default state. Call this function before starting a new generation to ensure accurate measurements.

```javascript
// ── Metrics ───────────────────────────────────────────────────────────────────
function resetMetrics() {
  ['mTTFA', 'mClient', 'mRTF', 'mDur', 'mBuf', 'mClone'].forEach(id => $(id).innerHTML = '&mdash;');
  $('mCloneWrap').style.display = 'none';
}
```

--------------------------------

### Generate MP3 Audio via OpenAI API

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Generates speech in MP3 format by sending a POST request to the OpenAI-compatible server. This is a non-streaming method and requires `pydub` to be installed.

```bash
curl http://localhost:8000/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{"model": "tts-1", "input": "Hello world.", "voice": "alloy", "response_format": "mp3"}' \
    --output speech.mp3
```

--------------------------------

### Select Preset Reference

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Handles the selection of a preset reference audio. It fetches audio data, creates a Blob and File object, sets the reference text, and updates the preview and UI elements. Errors during loading are reported.

```javascript
async function selectPresetRef(id) {
  try {
    const d = await fetch(`/preset_ref/${id}`).then(r => r.json());
    const bytes = Uint8Array.from(atob(d.audio_b64), c => c.charCodeAt(0));
    const blob = new Blob([bytes], { type: 'audio/wav' });
    const file = new File([blob], d.filename || `${id}.wav`, { type: 'audio/wav' });
    presetRefId = id;
    refFile = null; // avoid uploading when preset is selected
    setXvec(false); // presets have verified transcripts — use ICL mode
    $('refLabel').textContent = d.label || d.filename || 'Reference audio';
    $('refChip').classList.add('has');
    const prev = $('recPreview');
    if (prev.src) URL.revokeObjectURL(prev.src);
    prev.src = URL.createObjectURL(file);
    prev.style.display = 'block';
    $('refTextIn').value = d.ref_text || '';
    autoGrow($('refTextIn'));
    updatePresetButtons();
  } catch {
    showMsg('err', 'Failed to load preset audio.');
  }
}
```

--------------------------------

### faster-qwen3-tts CLI

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Command-line interface for Faster-Qwen3-TTS, supporting voice cloning, custom voice, voice design, and serving modes.

```APIDOC
## CLI — `faster-qwen3-tts` command-line interface

### Description
The installed `faster-qwen3-tts` CLI exposes four subcommands: `clone`, `custom`, `design`, and `serve`. The `serve` subcommand keeps the model loaded in memory and accepts text from stdin line-by-line, useful for batch or interactive use without repeated model loading overhead.

### Subcommands

#### `clone`
Voice cloning.

**Usage:**
```bash
faster-qwen3-tts clone \
  --model <model_name> \
  --text <text_to_clone> \
  --language <language> \
  --ref-audio <reference_audio_path> \
  --ref-text <reference_text> \
  --output <output_path> \
  [--streaming] \
  [--chunk-size <chunk_size>]
```

**Example (non-streaming):**
```bash
faster-qwen3-tts clone \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-Base \
  --text "What do you mean that I\'m not real?" \
  --language English \
  --ref-audio ref_audio.wav \
  --ref-text "I\'m confused why some people have super short timelines..." \
  --output out_clone.wav
```

**Example (streaming):**
```bash
faster-qwen3-tts clone \
  --model Qwen/Qwen3-TTS-12Hz-0.6B-Base \
  --text "Hello from the streaming path!" \
  --language English \
  --ref-audio ref_audio.wav \
  --ref-text "I\'m confused why some people have super short timelines..." \
  --output out_stream.wav \
  --streaming \
  --chunk-size 8
```

#### `custom`
Custom voice synthesis.

**Usage:**
```bash
faster-qwen3-tts custom \
  --model <model_name> \
  --speaker <speaker_name> \
  --text <text_to_synthesize> \
  --language <language> \
  --output <output_path> \
  [--streaming]
```

**Example (list speakers):**
```bash
faster-qwen3-tts custom --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --list-speakers
```

**Example (generate):**
```bash
faster-qwen3-tts custom \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --speaker aiden \
  --text "Streaming is the future." \
  --language English \
  --output out_custom.wav \
  --streaming
```

#### `design`
Instruction-based voice design.

**Usage:**
```bash
faster-qwen3-tts design \
  --model <model_name> \
  --instruct <voice_instructions> \
  --text <text_to_synthesize> \
  --language <language> \
  --output <output_path>
```

**Example:**
```bash
faster-qwen3-tts design \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign \
  --instruct "Warm, confident narrator with slight British accent" \
  --text "Welcome to the show." \
  --language English \
  --output out_design.wav
```

#### `serve`
Starts a model serving process.

**Usage:**
```bash
faster-qwen3-tts serve \
  --mode <mode> \
  --model <model_name> \
  [--speaker <speaker_name>] \
  [--language <language>] \
  [--streaming]
```

**Example:**
```bash
faster-qwen3-tts serve \
  --mode custom \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --speaker aiden \
  --language English \
  --streaming
```

**Note:** In serve mode, type lines of text at the prompt; files are written to `./outputs/`.
```

--------------------------------

### Initialize Audio Input Selection

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Sets up the audio input selection dropdown based on available devices. Handles default device selection and previous selections.

```javascript
function loadMics() { try { const inputs = await navigator.mediaDevices.enumerateDevices(); const sel = $('micSel'); sel.innerHTML = ''; let defaultId = null; let recDeviceId = null; const prevSelection = recDeviceId; inputs.filter(d => d.kind === 'audioinput').forEach(d => { const opt = document.createElement('option'); opt.value = d.deviceId; opt.textContent = d.label || d.deviceId; sel.appendChild(opt); if (!defaultId && (d.deviceId === 'default' || d.label.includes('default'))) { defaultId = d.deviceId; } }); sel.onchange = () => { recDeviceId = sel.value; }; if (prevSelection && inputs.some(d => d.deviceId === prevSelection)) { recDeviceId = prevSelection; sel.value = prevSelection; } else if (defaultId) { recDeviceId = defaultId; sel.value = defaultId; } else if (inputs.length) { recDeviceId = inputs[0].deviceId; sel.value = inputs[0].deviceId; } } catch {} }
```

--------------------------------

### Initialize Web Audio Context and ScriptProcessor

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Initializes the Web Audio context and a ScriptProcessorNode for audio processing. Handles resuming the context if suspended.

```javascript
async function initAudio(sr) {
    rawPcmSr = sr || 24000;
    pcmQueue = [];
    rawPcmParts = [];
    chunkQ = Promise.resolve();
    dlBlob = null;
    firstChunkAt = null;
    firstAudioAt = null;
    lastBufS = 0;
    if (actx) {
        if (actx.state === 'suspended') await actx.resume();
        return;
    }
    actx = new (window.AudioContext || window.webkitAudioContext)({ sampleRate: rawPcmSr });
    sproc = actx.createScriptProcessor(256, 0, 1);
    sproc.onaudioprocess = e => {
        const out = e.outputBuffer.getChannelData(0);
        let i = 0, wrote = false;
        while (i < out.length) {
            if (!pcmQueue.length) {
                out.fill(0, i);
                break;
            }
            const seg = pcmQueue[0];
            const take = Math.min(out.length - i, seg.data.length - seg.pos);
            out.set(seg.data.subarray(seg.pos, seg.pos + take), i);
            seg.pos += take;
            i += take;
            wrote = true;
            if (seg.pos >= seg.data.length) pcmQueue.shift();
        }
        if (wrote && firstAudioAt == null) {
            firstAudioAt = performance.now();
            pushClientMetrics();
        }
    };
    if (actx.state === 'suspended') await actx.resume();
    sproc.connect(actx.destination);
}
```

--------------------------------

### Voice Cloning with Precomputed Prompt Items

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Demonstrates how to use precomputed prompt items for voice cloning. This involves computing the prompt items once from reference audio and then passing them to the generation API.

```python
import torch
from faster_qwen3_tts import FasterQwen3TTS

model = FasterQwen3TTS.from_pretrained("Qwen/Qwen3-TTS-12Hz-1.7B-Base")

# 1) Compute prompt_items once from reference audio
prompt_items = model.model.create_voice_clone_prompt(
    ref_audio="voice.wav",
    ref_text="",
    x_vector_only_mode=True,
)

# 2) You can pass prompt_items directly
audio_list, sr = model.generate_voice_clone(
    text="Hello world!",
    language="English",
    voice_clone_prompt=prompt_items,
)
```

--------------------------------

### Send Request to OpenAI TTS API

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Example using curl to send a speech synthesis request to the running OpenAI-compatible API server. The request specifies model, input text, voice, and output format.

```bash
curl http://localhost:8000/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{"model": "tts-1", "input": "Hello world.", "voice": "alloy", "response_format": "wav"}' \
    --output speech.wav
```

--------------------------------

### Append Silence to Reference Audio for Voice Cloning

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/BLOG.md

Appends 0.5 seconds of silence to the reference audio before encoding to prevent phoneme artifacts at the start of generated speech. This is crucial for clean turn boundaries in conversational applications.

```python
audio, sr = sf.read(ref_audio_path, dtype="float32", always_2d=False)
silence = np.zeros(int(0.5 * sr), dtype=np.float32)
ref_audio_input = (np.concatenate([audio, silence]), sr)
```

--------------------------------

### FasterQwen3TTS.from_pretrained

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Loads a Qwen3-TTS model from HuggingFace Hub or a local path, builds CUDA-graph objects, and returns a ready-to-use FasterQwen3TTS instance. CUDA graph capture is deferred to the first generation call.

```APIDOC
## FasterQwen3TTS.from_pretrained — Load model and build CUDA graphs

### Description
Loads a Qwen3-TTS model from HuggingFace Hub or a local path, builds the `PredictorGraph` and `TalkerGraph` CUDA-graph objects, and returns a ready-to-use `FasterQwen3TTS` instance. CUDA graph capture itself is deferred to the first generation call (warmup). Requires an NVIDIA GPU with CUDA; raises `ValueError` on CPU-only environments.

### Method Signature
```python
FasterQwen3TTS.from_pretrained(
    pretrained_model_name_or_path: str,
    *, 
    device: str = "cuda",
    dtype: torch.dtype = torch.bfloat16,
    attn_implementation: str = "sdpa",
    max_seq_len: int = 2048,
    **kwargs
)
```

### Parameters
- **pretrained_model_name_or_path** (str) - Required - Path to the model on HuggingFace Hub or a local directory.
- **device** (str) - Optional - The device to load the model on (e.g., "cuda"). Defaults to "cuda".
- **dtype** (torch.dtype) - Optional - The data type for the model weights (e.g., `torch.bfloat16`). `bfloat16` is recommended.
- **attn_implementation** (str) - Optional - Attention implementation to use (e.g., "sdpa" or "flash_attention_2"). Defaults to "sdpa".
- **max_seq_len** (int) - Optional - The maximum sequence length for the static KV cache. Defaults to 2048.

### Request Example
```python
from faster_qwen3_tts import FasterQwen3TTS
import torch

model = FasterQwen3TTS.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
    device="cuda",
    dtype=torch.bfloat16,
    attn_implementation="sdpa",
    max_seq_len=2048,
)
print(f"Sample rate: {model.sample_rate} Hz")
```

### Response
- **FasterQwen3TTS instance** - An initialized FasterQwen3TTS model ready for inference.
- **sample_rate** (int) - The sample rate of the generated audio (e.g., 24000 Hz).
```

--------------------------------

### Run Benchmarks on Linux/macOS/WSL

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/README.md

Executes the benchmarking script to evaluate performance. Can be run for all models or a specific one.

```bash
./benchmark.sh # or ./benchmark.sh 0.6B or ./benchmark.sh 1.7B for a single model
```

--------------------------------

### Push Performance Metrics (JavaScript)

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Updates the UI with performance metrics obtained during audio generation. This includes metrics like Time To First Byte (TTFA), Real-Time Factor (RTF), duration, and voice cloning time. Use this to display generation performance.

```javascript
function pushMetrics(ttfa, rtf, dur, cloneMs) {
  if (ttfa != null) $('mTTFA').textContent = Math.round(ttfa) + 'ms';
  if (rtf != null) $('mRTF').textContent = rtf.toFixed(2) + 'x';
  if (dur != null) $('mDur').textContent = dur.toFixed(1) + 's';
  if (clo
```

--------------------------------

### Initialize DOM Elements and Auto-Grow Function

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Sets up shorthand for DOM element selection and an auto-grow function for textareas. Useful for initializing UI elements and dynamic text input.

```javascript
const $ = id => document.getElementById(id);
function autoGrow(el) {
  el.style.height = 'auto';
  el.style.height = el.scrollHeight + 'px';
}
function setPlayBtns(disabled) {
  document.querySelectorAll('.play-btn').forEach(b => b.disabled = disabled);
}
```

--------------------------------

### Download Qwen3-TTS Models

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/WINDOWS_SETUP_GUIDE.md

Manually download the Qwen3-TTS models using a Python script. This requires the huggingface_hub library.

```python
from huggingface_hub import snapshot_download; [snapshot_download(f'Qwen/{m}') for m in ['Qwen3-TTS-12Hz-0.6B-Base', 'Qwen3-TTS-12Hz-1.7B-Base']]
```

--------------------------------

### Run Streaming Audio Generation (JavaScript)

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Handles the streaming audio generation process by fetching audio chunks from the server. It decodes base64 encoded audio, initializes audio playback, and enqueues audio chunks. Use this for real-time audio feedback.

```javascript
async function runStream(fd) {
  clientT0 = performance.now();
  firstChunkAt = null;
  firstAudioAt = null;
  lastBufS = 0;
  firstServerWall = null;

  const res = await fetch('/generate/stream', {
    method: 'POST',
    body: fd
  });

  if (!res.ok) {
    const e = await res.json();
    throw new Error(e.detail || 'Request failed');
  }

  const reader = res.body.getReader();
  const dec = new TextDecoder();
  let buf = '', audioInited = false;

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buf += dec.decode(value, { stream: true });

    const lines = buf.split('\n');
    buf = lines.pop();

    for (const line of lines) {
      if (!line.startsWith('data: ')) continue;
      const d = JSON.parse(line.slice(6));

      if (d.type === 'queued') {
        const bar = $('queueBar');
        bar.textContent = d.position === 1 ? 'Waiting... (1 request ahead of you)' : `Waiting... (${d.position} requests ahead of you)`;
        bar.classList.add('show');
      } else if (d.type === 'chunk') {
        $('queueBar').className = 'queue-bar';
        if (!audioInited) {
          await initAudio(d.sample_rate);
          audioInited = true;
        }
        // Capture server wall time on first chunk
        if (firstChunkAt == null && d.elapsed_ms != null) firstServerWall = d.elapsed_ms;
        pushMetrics(d.ttfa_ms, d.rtf, d.total_audio_s, d.voice_clone_ms);
        enqueueChunk(d.audio_b64);
      } else if (d.type === 'done') {
        $('queueBar').className = 'queue-bar';
        pushMetrics(d.ttfa_ms, d.rtf, d.total_audio_s, d.voice_clone_ms);
        await chunkQ;
        setDone();
        const blob = buildFinalWav();
        if (blob) setPlayer(blob);
      } else if (d.type === 'error') {
        $('queueBar').className = 'queue-bar';
        throw new Error(d.message);
      }
    }
  }
}
```

--------------------------------

### Theme Toggling and Application

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Functions to initialize, toggle, and apply themes (light/dark) based on user preference or system settings. Updates the UI and local storage.

```javascript
function initTheme() {
  const t = localStorage.getItem('theme') || (matchMedia('(prefers-color-scheme: light)').matches ? 'light' : 'dark');
  applyTheme(t);
}
function toggleTheme() {
  applyTheme((document.documentElement.dataset.theme || 'dark') === 'dark' ? 'light' : 'dark');
}
function applyTheme(t) {
  document.documentElement.dataset.theme = t;
  localStorage.setItem('theme', t);
  $('themeIcon').innerHTML = t === 'dark' ? '<circle cx="12" cy="12" r="5"/><line x1="12" y1="1" x2="12" y2="3"/><line x1="12" y1="21" x2="12" y2="23"/><line x1="4.22" y1="4.22" x2="5.64" y2="5.64"/><line x1="18.36" y1="18.36" x2="19.78" y2="19.78"/><line x1="1" y1="12" x2="3" y2="12"/><line x1="21" y1="12" x2="23" y2="12"/><line x1="4.22" y1="19.78" x2="5.64" y2="18.36"/><line x1="18.36" y1="5.64" x2="19.78" y2="4.22"/>' : '<path d="M21 12.79A9 9 0 1111.21 3 7 7 0 0021 12.79z"/>';
}
```

--------------------------------

### CLI: CustomVoice - List Speakers and Generate

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Interact with the CustomVoice model via the CLI. First, list available speakers using `--list-speakers`, then generate audio with a specified speaker, text, language, and optional streaming output.

```bash
# CustomVoice — list speakers then generate
faster-qwen3-tts custom --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --list-speakers
faster-qwen3-tts custom \
  --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --speaker aiden \
  --text "Streaming is the future." \
  --language English \
  --output out_custom.wav \
  --streaming
```

--------------------------------

### Switch Model and Generate

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Handles switching to a new model and then generating audio. It prevents multiple concurrent operations by setting a 'busy' flag and ensures UI updates.

```javascript
async function switchAndGen(target) {
  if (busy) return;
  const mode = pendingMode;
  busy = true;
  hideMsg();
  setPlayBtns(true);
  try {
    await loadModel();
  } finally {
    busy = false;
  }
  setPlayBtns(false);
  generate(mode);
}
```

--------------------------------

### Load Model and Build CUDA Graphs with FasterQwen3TTS

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Loads a Qwen3-TTS model and prepares it for inference by building CUDA graph objects. CUDA graph capture is deferred to the first generation call. Requires an NVIDIA GPU with CUDA.

```python
from faster_qwen3_tts import FasterQwen3TTS
import torch

# Load the 0.6B base model (voice cloning)
model = FasterQwen3TTS.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
    device="cuda",
    dtype=torch.bfloat16,       # bfloat16 recommended
    attn_implementation="sdpa", # "sdpa" (default) or "flash_attention_2"
    max_seq_len=2048,           # static KV cache size
)
print(f"Sample rate: {model.sample_rate} Hz")  # 24000
```

--------------------------------

### CLI: Streaming Voice Cloning

Source: https://context7.com/andimarafioti/faster-qwen3-tts/llms.txt

Execute streaming voice cloning with the `faster-qwen3-tts clone` command using the `--streaming` and `--chunk-size` flags. This offers lower Time-To-First-Audio (TTFA) while maintaining final audio quality.

```bash
# Voice cloning — streaming output (lower TTFA, same final quality)
faster-qwen3-tts clone \
  --model Qwen/Qwen3-TTS-12Hz-0.6B-Base \
  --text "Hello from the streaming path!" \
  --language English \
  --ref-audio ref_audio.wav \
  --ref-text "I'm confused why some people have super short timelines..." \
  --output out_stream.wav \
  --streaming \
  --chunk-size 8
```

--------------------------------

### Run Non-Streaming Audio Generation (JavaScript)

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Handles the non-streaming audio generation process by fetching the complete audio response from the server. It decodes the base64 encoded audio and sets up the audio player. Use this for generating complete audio files at once.

```javascript
async function runNonStream(fd) {
  $('waveInd').classList.add('off');
  const res = await fetch('/generate', {
    method: 'POST',
    body: fd
  });

  if (!res.ok) {
    const e = await res.json();
    throw new Error(e.detail || 'Request failed');
  }

  const d = await res.json();
  const m = d.metrics;
  pushMetrics(m.total_ms, m.rtf, m.audio_duration_s, m.voice_clone_ms);
  setDone();

  const bytes = Uint8Array.from(atob(d.audio_b64), c => c.charCodeAt(0));
  const blob = new Blob([bytes], { type: 'audio/wav' });
  setPlayer(blob);
  $('player').play().catch(() => {}); // gracefully handle autoplay block
}
```

--------------------------------

### Set Reference Audio File

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Handles the selection of a reference audio file and updates the UI to display the file name and a preview.

```javascript
function pickRef(input) { const f = input.files[0]; if (f) setRef(f); }
function setRef(f) { clearPresetSelection(); refFile = f; $('refLabel').textContent = f.name; $('refChip').classList.add('has'); const prev = $('recPreview'); if (prev.src) URL.revokeObjectURL(prev.src); prev.src = URL.createObjectURL(f); prev.style.display = 'block'; transcribeRef(); }
```

--------------------------------

### Build Final WAV Blob from Raw Parts

Source: https://github.com/andimarafioti/faster-qwen3-tts/blob/main/demo/index.html

Concatenates all raw PCM data parts into a single WAV Blob. Requires that audio chunks have already been enqueued and parsed.

```javascript
function buildFinalWav() {
    if (!rawPcmParts.length) return null;
    const totalPcm = rawPcmParts.reduce((s, p) => s + p.length, 0);
    const ab = new ArrayBuffer(44 + totalPcm);
    const v = new DataView(ab);
    const ws = (o, s) => [...s].forEach((c, i) => v.setUint8(o + i, c.charCodeAt(0)));
    ws(0, 'RIFF');
    v.setUint32(4, 36 + totalPcm, true);
    ws(8, 'WAVE');
    ws(12, 'fmt ');
    v.setUint32(16, 16, true);
    v.setUint16(20, 1, true);
    v.setUint16(22, 1, true);
    v.setUint32(24, rawPcmSr, true);
    v.setUint32(28, rawPcmSr * 2, true);
    v.setUint16(32, 2, true);
    v.setUint16(34, 16, true);
    ws(36, 'data');
    v.setUint32(40, totalPcm, true);
    const out = new Uint8Array(ab, 44);
    let offset = 0;
    for (const part of rawPcmParts) {
        out.set(part, offset);
        offset += part.length;
    }
    return new Blob([ab], { type: 'audio/wav' });
}
```