### Full VibeVoice Setup on GPU

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

This multi-step script sets up the VibeVoice ASR demo on a specified GPU (defaulting to 0) and port (defaulting to 6001). It includes starting the server, installing dependencies, launching the Gradio interface, and retrieving the public link.

```bash
# 1. Start server
docker run -d --gpus '"device=0"' --name vibevoice-asr-demo \
  --ipc=host -p 6001:6001 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --port 6001"

# 2. Wait for startup (~2 min), then verify
docker logs -f vibevoice-asr-demo  # wait for "Application startup complete."
curl http://localhost:6001/v1/models

# 3. Install tmux and launch Gradio
docker exec vibevoice-asr-demo apt-get install -y tmux
docker exec vibevoice-asr-demo bash -c \
  "PYTHONUNBUFFERED=1 tmux new-session -d -s gradio \
  'PYTHONUNBUFFERED=1 python3 /app/vllm_plugin/scripts/gradio_asr_demo_api_video.py \
  --api_url http://localhost:6001 --share \
  2>&1 | tee /tmp/gradio.log'"

# 4. Get the public link
sleep 20 && docker exec vibevoice-asr-demo cat /tmp/gradio.log
```

--------------------------------

### Setup Environment and Download Model

Source: https://github.com/microsoft/vibevoice/blob/main/demo/vibevoice_realtime_colab.ipynb

This snippet checks for a T4 GPU, clones the VibeVoice repository, installs project dependencies, and downloads the VibeVoice-Realtime model. It also includes instructions for changing the runtime to T4 GPU if not detected.

```python
# Check for T4 GPU
import torch
if torch.cuda.is_available() and "T4" in torch.cuda.get_device_name(0):
    print("✅ T4 GPU detected")
else:
    print("\n    ⚠️ WARNING: T4 GPU not detected\n\n    The recommended runtime for this Colab notebook is \"T4 GPU\".\n\n    To change the runtime type:\n\n        1. Click on \"Runtime\" in the top navigation menu\n        2. Click on \"Change runtime type\"\n        3. Select \"T4 GPU\"\n        4. Click \"OK\" if a \"Disconnect and delete runtime\" window appears\n        5. Click on \"Save\"\n\n    ")

# Clone the VibeVoice repository
![ -d /content/VibeVoice ] || git clone --quiet --branch main --depth 1 https://github.com/microsoft/VibeVoice.git /content/VibeVoice
print("✅ Cloned VibeVoice repository")

# Install project dependencies
!uv pip --quiet install --system -e /content/VibeVoice[streamingtts]
!wget -q https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64 -O cloudflared && chmod +x cloudflared
print("✅ Installed dependencies")

# Download model
from huggingface_hub import snapshot_download
snapshot_download("microsoft/VibeVoice-Realtime-0.5B", local_dir="/content/models/VibeVoice-Realtime-0.5B")
print("✅ Downloaded model: microsoft/VibeVoice-Realtime-0.5B")

```

--------------------------------

### Launch Gradio Demo

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-asr.md

Starts the VibeVoice ASR Gradio demo. Requires ffmpeg to be installed. The --share flag makes the demo publicly accessible via a temporary URL.

```bash
apt update && apt install ffmpeg -y # for demo

python demo/vibevoice_asr_gradio_demo.py --model_path microsoft/VibeVoice-ASR --share
```

--------------------------------

### Install Flash Attention (if needed)

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-asr.md

Installs Flash Attention if it's not included in your Docker environment. Refer to the official Flash Attention GitHub repository for detailed installation instructions.

```bash
# pip install flash-attn --no-build-isolation
```

--------------------------------

### Start Single GPU ASR Server with Docker

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

Launches a Docker container for the vLLM ASR server on a single GPU. Ensure Docker with GPU support is installed.

```bash
docker run -d --gpus '"device=0"' --name vibevoice-asr-demo \
  --ipc=host \
  -p 6001:6001 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --port 6001"
```

--------------------------------

### Start Audio Generation and Playback

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Initiates the audio generation process when the 'Start' button is clicked. It retrieves user inputs, resets the state, creates the audio chain, and prepares for streaming.

```javascript
const start = () => {
  if (isPlaying) {
    return;
  }
  const textValue = promptInput?.value || '';
  const cfgValue = Number(cfgSelect.value);
  const stepsValue = Number(stepsSelect.value);
  const voiceValue = voiceSelect.value || '';
  clearLogs();
  const cfgDisplay = Number.isFinite(cfgValue) ? cfgValue.toFixed(3) : 'default';
  const stepsDisplay = Number.isFinite(stepsValue) ? stepsValue : 'default';
  appendLog(`[Frontend] Start button clicked, CFG=${cfgDisplay}, Steps=${stepsDisplay}, Speaker=${voiceValue || 'default'}`);
  setModelGenerated(0);
  setPlaybackElapsed(0);
  resetState(true);
  clearRecordedChunks();
  isPlaying = true;
  previewActive = true;
  updateStreamingPreview();
  updateButtonLabel();
  createAudioChain();
  const params = new URLSearchParams();
  params.set('text', textValue);
  if (!Number.isNaN(cfgValue)) {
    params.set('cfg', cfgValue.toFixed(3));
  }
  if (!Number.isNaN(stepsValue)) {

```

--------------------------------

### Clone VibeVoice Repository and Install

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-asr.md

Clones the VibeVoice repository from GitHub and installs it using pip. This command should be run after cloning the repository.

```bash
git clone https://github.com/microsoft/VibeVoice.git
cd VibeVoice

pip install -e .
```

--------------------------------

### Install Dependencies for LoRA Fine-tuning

Source: https://github.com/microsoft/vibevoice/blob/main/finetuning-asr/README.md

Installs the VibeVoice package and the PEFT library required for LoRA fine-tuning.

```bash
pip install -e .

pip install peft
```

--------------------------------

### Clone VibeVoice Repository and Install

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-realtime-0.5b.md

Clones the VibeVoice repository from GitHub and installs the package with streaming TTS dependencies. This command should be run after navigating into the cloned directory.

```bash
git clone https://github.com/microsoft/VibeVoice.git
cd VibeVoice/

pip install -e .[streamingtts]
```

--------------------------------

### Check FFmpeg Installation

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Verify that FFmpeg is installed on your system, which is required for audio decoding. Run this command in your terminal.

```bash
ffmpeg -version
```

--------------------------------

### Basic LoRA Fine-tuning (Specific GPUs)

Source: https://github.com/microsoft/vibevoice/blob/main/finetuning-asr/README.md

Starts LoRA fine-tuning on specified GPUs, allowing for distributed training.

```bash
# Specific GPUs (e.g., GPU 0,1,2,3)
CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node=4 lora_finetune.py \
    --model_path microsoft/VibeVoice-ASR \
    --data_dir ./toy_dataset \
    --output_dir ./output \
    --num_train_epochs 3 \
    --per_device_train_batch_size 1 \
    --learning_rate 1e-4 \
    --bf16 \
    --report_to none
```

--------------------------------

### Install NVIDIA PyTorch Container

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-asr.md

Launches the NVIDIA PyTorch Docker container, recommended for managing CUDA environments. Ensure you have Docker installed and NVIDIA Container Toolkit configured.

```bash
sudo docker run --privileged --net=host --ipc=host --ulimit memlock=-1:-1 --ulimit stack=-1:-1 --gpus all --rm -it  nvcr.io/nvidia/pytorch:25.12-py3
```

--------------------------------

### Verify Vibevoice Installation

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Check if the Vibevoice package is installed correctly. Use this command to confirm its presence.

```bash
pip show vibevoice
```

--------------------------------

### Install tmux in Docker Container

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

Install the tmux package inside the vibevoice-asr-demo Docker container to manage the Gradio session.

```bash
docker exec vibevoice-asr-demo apt-get install -y tmux
```

--------------------------------

### Run VibeVoice with vLLM using Hybrid Parallelism (DP x TP)

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Combines Data Parallel and Tensor Parallelism to create a hybrid configuration. This example uses 2 Data Parallel replicas, each split across 2 GPUs, totaling 4 GPUs. The total number of GPUs required is dp * tp.

```bash
docker run -d --gpus '"device=0,1,2,3"' --name vibevoice-vllm \
  --ipc=host \
  -p 8000:8000 \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --dp 2 --tp 2"
```

--------------------------------

### Download Model (after login)

Source: https://github.com/microsoft/vibevoice/blob/main/demo/vibevoice_realtime_colab.ipynb

This snippet downloads the VibeVoice-Realtime model after logging into Hugging Face. It should be used if the initial download fails or gets stuck.

```python
snapshot_download("microsoft/VibeVoice-Realtime-0.5B", local_dir="/content/models/VibeVoice-Realtime-0.5B")
print("✅ Downloaded model: microsoft/VibeVoice-Realtime-0.5B")

```

--------------------------------

### Start Gradio Demo in tmux

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

Launch the Gradio ASR demo within a tmux session inside the Docker container. This command sets the PYTHONUNBUFFERED environment variable and redirects output to a log file.

```bash
docker exec vibevoice-asr-demo bash -c \
  "PYTHONUNBUFFERED=1 tmux new-session -d -s gradio \
  'PYTHONUNBUFFERED=1 python3 /app/vllm_plugin/scripts/gradio_asr_demo_api_video.py \
  --api_url http://localhost:6001 --share \
  2>&1 | tee /tmp/gradio.log'"
```

--------------------------------

### Start Multi-GPU ASR Server with Docker (Data Parallel)

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

Launches a Docker container for the vLLM ASR server across multiple GPUs using data parallelism for load balancing. Use `--dp N` for N-way data parallel or `--tp N` for tensor parallel.

```bash
docker run -d --gpus '"device=0,1,2,3"' --name vibevoice-asr-demo \
  --ipc=host \
  -p 6001:6001 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --port 6001 --dp 4"
```

--------------------------------

### Retrieve Gradio Share Link

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

After starting the Gradio demo, retrieve the public share link by viewing the content of the gradio.log file within the Docker container.

```bash
docker exec vibevoice-asr-demo cat /tmp/gradio.log
```

--------------------------------

### Inference from Files

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-realtime-0.5b.md

Performs speech synthesis inference directly from text files using the VibeVoice model. Example scripts are provided for demonstration.

```bash
# We provide some example scripts under demo/text_examples/ for demo
python demo/realtime_model_inference_from_file.py --model_path microsoft/VibeVoice-Realtime-0.5B --txt_path demo/text_examples/1p_vibevoice.txt --speaker_name Carter
```

--------------------------------

### Control Button Event Listener

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Attaches an event listener to a control button to toggle between starting and stopping the WebSocket stream or playback.

```javascript
controlBtn.addEventListener('click', () => { if (isPlaying) { stop(); } else { start(); } });
```

--------------------------------

### Manage Playback Timer

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Controls the interval timer for updating the playback elapsed time. It ensures only one timer is active at a time by clearing any existing timer before starting a new one.

```javascript
const stopPlaybackTimer = () => {
  if (playbackTimer) {
    clearInterval(playbackTimer);
    playbackTimer = null;
  }
};
const startPlaybackTimer = () => {
  stopPlaybackTimer();
  playbackTimer = setInterval(() => {
    setPlaybackElapsed(playbackSamples / SAMPLE_RATE);
  }, 250);
};
```

--------------------------------

### Quick Test with Audio File

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

Perform a quick test of the VibeVoice ASR system using a sample audio file via Docker exec.

```bash
docker exec -it vibevoice-asr-demo \
  python3 /app/vllm_plugin/tests/test_api.py /app/en-Alice_woman.wav \
  --url http://localhost:6001
```

--------------------------------

### Launch Real-time Websocket Demo

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-realtime-0.5b.md

Launches the real-time websocket demo for VibeVoice. Note that real-time performance may vary based on hardware capabilities and network latency.

```bash
python demo/vibevoice_realtime_demo.py --model_path microsoft/VibeVoice-Realtime-0.5B
```

--------------------------------

### Initialize Audio Playback Context

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Sets up the Web Audio API context, script processor node, and WebSocket connection for real-time audio streaming. Handles buffer management and playback state.

```javascript
(() => {
  const SAMPLE_RATE = 24000;
  const BUFFER_SIZE = 2048;
  const PREBUFFER_SEC = 0.1;
  let audioCtx = null;
  let scriptNode = null;
  let socket = null;
  let buffer = new Float32Array(0);
  let isPlaying = false;
  let hasStartedPlayback = false;
  let silentFrameCount = 0;
  const promptInput = document.getElementById('prompt');
  const streamingPreview = document.getElementById('streamingPreview');
  const controlBtn = document.getElementById('playback');
  const cfgSelect = document.getElementById('cfgScale');
  const stepsSelect = document.getElementById('inferenceSteps');
  const voiceSelect = document.getElementById('voiceSelect');
  const cfgValueLabel = document.getElementById('cfgValue');
  const stepsValueLabel = document.getElementById('stepsValue');
  const modelGeneratedLabel = document.getElementById('modelGenerated');
  const playbackElapsedLabel = document.getElementById('playbackElapsed');
  const logOutput = document.getElementById('logOutput');
  const resetBtn = document.getElementById('resetControls');
  const saveBtn = document.getElementById('saveAudio');
  let playbackTimer = null;
  let lastPlaybackElapsed = 0;
  let playbackSamples = 0;
  let modelGeneratedTotal = 0;
  let firstBrowserChunkLogged = false;
  let playbackStartedLogged = false;
  const logEntries = [];
  let logSequence = 0;
  let recordedChunks = [];
  let recordedSamples = 0;
  let recordingComplete = false;
  let downloadUrl = null;
  const revokeDownloadUrl = () => {
    if (downloadUrl) {
      URL.revokeObjectURL(downloadUrl);
      downloadUrl = null;
    }
  };
  const updateSaveButtonState = () => {
    if (!saveBtn) {
      return;
    }
    saveBtn.disabled = recordedSamples === 0 || !recordingComplete;
  };
  const clearRecordedChunks = () => {
    recordedChunks = [];
    recordedSamples = 0;
    recordingComplete = false;
    revokeDownloadUrl();
    updateSaveButtonState();
  };
  const createWavBlob = () => {
    if (!recordedSamples) {
      return null;
    }
    const wavBuffer = new ArrayBuffer(44 + recordedSamples * 2);
    const view = new DataView(wavBuffer);
    const writeString = (offset, str) => {
      for (let i = 0; i < str.length; i += 1) {
        view.setUint8(offset + i, str.charCodeAt(i));
      }
    };
    writeString(0, 'RIFF');
    view.setUint32(4, 36 + recordedSamples * 2, true);
    writeString(8, 'WAVE');
    writeString(12, 'fmt ');
    view.setUint32(16, 16, true);
    view.setUint16(20, 1, true);
    view.setUint16(22, 1, true);
    view.setUint32(24, SAMPLE_RATE, true);
    view.setUint32(28, SAMPLE_RATE * 2, true);
    view.setUint16(32, 2, true);
    view.setUint16(34, 16, true);
    writeString(36, 'data');
    view.setUint32(40, recordedSamples * 2, true);
    const pcmData = new Int16Array(wavBuffer, 44, recordedSamples);
    let offset = 0;
    recordedChunks.forEach(chunk => {
      const chunkData = new Int16Array(chunk);
      pcmData.set(chunkData, offset);
      offset += chunkData.length;
    });
    return new Blob([wavBuffer], { type: 'audio/wav' });
  };
  const updateCfgDisplay = () => {
    cfgValueLabel.textContent = Number(cfgSelect.value).toFixed(2);
  };
  const updateStepsDisplay = () => {
    stepsValueLabel.textContent = Number(stepsSelect.value).toString();
  };
  cfgSelect.addEventListener('input', updateCfgDisplay);
  stepsSelect.addEventListener('input', updateStepsDisplay);
  updateCfgDisplay();
  updateStepsDisplay();
  const pad2 = value => value.toString().padStart(2, '0');
  const pad3 = value => value.toString().padStart(3, '0');
  const formatLocalTimestamp = () => {
    const d = new Date();
    const year = d.getFullYear();
    const month = pad2(d.getMonth() + 1);
    const day = pad2(d.getDate());
    const hours = pad2(d.getHours());
    const minutes = pad2(d.getMinutes());
    const seconds = pad2(d.getSeconds());
    const millis = pad3(d.getMilliseconds());
    return `${year}-${month}-${day} ${hours}:${minutes}:${seconds}.${millis}`;
  };
  const formatSeconds = raw => {
    const value = Number(raw);
    return Number.isFinite(value) ? value.toFixed(2) : '0.00';
  };
  const parseTimestamp = value => {
    if (!value) {
      return new Date();
    }
    if (/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3}/.test(value)) {
      return new Date(value.replace(' ', 'T'));
    }
    return new Date(value);
  };
  const setModelGenerated = value => {
    const numeric = Number(value);
    if (!Number.isFinite(numeric)) {
      return;
    }
    modelGeneratedTotal = Math.max(0, numeric);

```

--------------------------------

### Check Vibevoice Entry Point

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Inspect the entry points for the Vibevoice package to ensure plugins are correctly configured. This command lists package files and their associated entry points.

```bash
pip show -f vibevoice | grep entry
```

--------------------------------

### Run VibeVoice with vLLM using Data Parallel (4 GPUs)

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Launches vLLM with Data Parallelism for 4 GPUs, enabling automatic load balancing behind an nginx reverse proxy for optimal throughput. Ensure the correct number of GPUs are specified in the --gpus flag.

```bash
docker run -d --gpus '"device=0,1,2,3"' --name vibevoice-vllm \
  --ipc=host \
  -p 8000:8000 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --dp 4"
```

--------------------------------

### Launch VibeVoice vLLM Server with Docker

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Launch the VibeVoice ASR server using the official vLLM Docker image. This command configures GPU usage, port mapping, environment variables for concurrency and memory allocation, and mounts the local VibeVoice directory into the container.

```bash
docker run -d --gpus all --name vibevoice-vllm \
  --ipc=host \
  -p 8000:8000 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py"
```

--------------------------------

### Create and Configure Audio Chain

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Initializes the Web Audio API components, including the AudioContext and ScriptProcessorNode. Sets up the audio processing logic to pull audio from the buffer and play it.

```javascript
const createAudioChain = () => {
  teardownAudio();
  resetPlaybackFlags();
  audioCtx = new (window.AudioContext || window.webkitAudioContext)({
    sampleRate: SAMPLE_RATE
  });
  scriptNode = audioCtx.createScriptProcessor(BUFFER_SIZE, 0, 1);
  const minBufferSamples = Math.floor(audioCtx.sampleRate * PREBUFFER_SEC);
  scriptNode.onaudioprocess = event => {
    const output = event.outputBuffer.getChannelData(0);
    const needPrebuffer = !hasStartedPlayback;
    const socketClosed = !socket || socket.readyState === WebSocket.CLOSED || socket.readyState === WebSocket.CLOSING;
    if (needPrebuffer) {
      if (buffer.length >= minBufferSamples || socketClosed) {
        hasStartedPlayback = true;
        if (!playbackStartedLogged) {
          playbackStartedLogged = true;
          appendLog('[Frontend] Browser started to play audio');
          startPlaybackTimer();
        }
      } else {
        output.fill(0);
        return;
      }
    }
    const chunk = pullAudio(output.length);
    output.set(chunk);
    if (hasStartedPlayback) {
      playbackSamples += output.length;
    }
    if (socketClosed && buffer.length === 0 && chunk.every(sample => sample === 0)) {
      silentFrameCount += 1;
      if (silentFrameCount >= 4) {
        stop();
      }
    } else {
      silentFrameCount = 0;
    }
  };
  scriptNode.connect(audioCtx.destination);
};
```

--------------------------------

### Basic LoRA Fine-tuning (1 GPU)

Source: https://github.com/microsoft/vibevoice/blob/main/finetuning-asr/README.md

Initiates LoRA fine-tuning on a single GPU using default settings.

```bash
# 1 GPU
torchrun --nproc_per_node=1 lora_finetune.py \
    --model_path microsoft/VibeVoice-ASR \
    --data_dir ./toy_dataset \
    --output_dir ./output \
    --num_train_epochs 3 \
    --per_device_train_batch_size 1 \
    --learning_rate 1e-4 \
    --bf16 \
    --report_to none
```

--------------------------------

### Launch VibeVoice-Realtime Demo and Cloudflare Tunnel

Source: https://github.com/microsoft/vibevoice/blob/main/demo/vibevoice_realtime_colab.ipynb

This snippet launches the VibeVoice-Realtime demo server and a Cloudflare tunnel to expose it publicly. It continuously monitors the server and tunnel output to provide the public URL.

```python
import subprocess, re, time, threading

srv = subprocess.Popen(
    "python /content/VibeVoice/demo/vibevoice_realtime_demo.py --model_path /content/models/VibeVoice-Realtime-0.5B --port 8000",
    shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, bufsize=1, universal_newlines=True,
)
cf = subprocess.Popen(
    "./cloudflared tunnel --url http://localhost:8000 --no-autoupdate",
    shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, bufsize=1, universal_newlines=True,
)

public_url = None
server_ready = False
url_pattern  = re.compile(r"(https://[a-z0-9-]+\.trycloudflare.com)")

def read_srv():
    global server_ready
    for ln in srv.stdout:
        print(ln.strip())
        if "Uvicorn running on" in ln:
            server_ready = True

def read_cf():
    global public_url
    for ln in cf.stdout:
        m = url_pattern.search(ln)
        if m:
            public_url = m.group(1)
            break

threading.Thread(target=read_srv, daemon=True).start()
threading.Thread(target=read_cf,  daemon=True).start()


while True:
    if server_ready and public_url:
        print(f"✅ Public URL: {public_url}\n");
        public_url = None
    time.sleep(0.25)

```

--------------------------------

### Clone VibeVoice Repository

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Clone the VibeVoice repository to your local machine. This is the first step before launching the server.

```bash
git clone https://github.com/microsoft/VibeVoice.git
cd VibeVoice
```

--------------------------------

### Download Experimental Voices

Source: https://github.com/microsoft/vibevoice/blob/main/demo/vibevoice_realtime_colab.ipynb

This command downloads experimental voices for VibeVoice. It is an optional step.

```bash
!bash /content/VibeVoice/demo/download_experimental_voices.sh

```

--------------------------------

### Check ASR Server Logs

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

Follow the logs of the running ASR server Docker container to monitor its status. Wait for 'Application startup complete.' to confirm readiness.

```bash
docker logs -f vibevoice-asr-demo
```

--------------------------------

### Initial UI Updates and Beforeunload Handler

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Performs initial updates to the button label and save button state. Also sets up a 'beforeunload' event listener to reset state and clean up resources when the user navigates away.

```javascript
updateButtonLabel(); updateSaveButtonState(); window.addEventListener('beforeunload', () => { resetState(); clearPreviewTimer(); revokeDownloadUrl(); }); })();
```

--------------------------------

### LoRA Fine-tuning with Full Options

Source: https://github.com/microsoft/vibevoice/blob/main/finetuning-asr/README.md

Executes LoRA fine-tuning with a comprehensive set of hyperparameters, including LoRA-specific settings and training arguments.

```bash
torchrun --nproc_per_node=4 lora_finetune.py \
    --model_path microsoft/VibeVoice-ASR \
    --data_dir ./toy_dataset \
    --output_dir ./output \
    --lora_r 16 \
    --lora_alpha 32 \
    --lora_dropout 0.05 \
    --num_train_epochs 3 \
    --per_device_train_batch_size 1 \
    --gradient_accumulation_steps 4 \
    --learning_rate 1e-4 \
    --warmup_ratio 0.1 \
    --weight_decay 0.01 \
    --max_grad_norm 1.0 \
    --logging_steps 10 \
    --save_steps 100 \
    --gradient_checkpointing \
    --bf16 \
    --report_to none
```

--------------------------------

### Handle Audio Save Request

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Initiates the download of recorded audio as a WAV file. It checks if audio data exists, creates a WAV blob, generates a download URL, and triggers the download.

```javascript
const handleSaveClick = () => {
  if (!recordedSamples) {
    appendLog('[Frontend] Save requested but no audio received yet');
    return;
  }
  const wavBlob = createWavBlob();
  if (!wavBlob) {
    appendLog('[Error] Failed to assemble WAV data for download');
    return;
  }
  revokeDownloadUrl();
  downloadUrl = URL.createObjectURL(wavBlob);
  const link = document.createElement('a');
  const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
  link.href = downloadUrl;
  link.download = `vibevoice_realtime_audio_${timestamp}.wav`;
  document.body.appendChild(link);
  link.click();
  document.body.removeChild(link);
  appendLog('[Frontend] Audio download triggered');
};
```

--------------------------------

### Load Voice Presets

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Asynchronously fetches available voice presets from the '/config' endpoint. It populates a dropdown select element with the voices and handles potential errors during the fetch or parsing process.

```javascript
const loadVoices = async () => {
  try {
    voiceSelect.disabled = true;
    const response = await fetch('/config');
    if (!response.ok) {
      throw new Error(`Failed to fetch config: ${response.status}`);
    }
    const data = await response.json();
    const voices = Array.isArray(data.voices) ? data.voices : [];
    voiceSelect.innerHTML = '';
    if (voices.length === 0) {
      const option = document.createElement('option');
      option.value = '';
      option.textContent = 'No voices available';
      voiceSelect.appendChild(option);
      voiceSelect.disabled = true;
      appendLog('[Error] No voice presets available');
      return;
    }
    voices.forEach(voice => {
      const option = document.createElement('option');
      option.value = voice;
      option.textContent = voice;
      voiceSelect.appendChild(option);
    });
    if (data.default_voice && voices.includes(data.default_voice)) {
      voiceSelect.value = data.default_voice;
    }
    voiceSelect.disabled = false;
    appendLog(`[Frontend] Loaded ${voices.length} voice presets`);
  } catch (err) {
    console.error('Failed to load voices', err);
    voiceSelect.innerHTML = '';
    const option = document.createElement('option');
    option.value = '';
    option.textContent = 'Load failed';
    voiceSelect.appendChild(option);
    voiceSelect.disabled = true;
    appendLog('[Error] Failed to load voice presets');
  }
};
```

--------------------------------

### Run VibeVoice with vLLM using Data Parallel (8 GPUs)

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Deploys vLLM with Data Parallelism across all 8 available GPUs. This configuration is recommended for scaling throughput. The container runs in detached mode and exposes port 8000.

```bash
docker run -d --gpus all --name vibevoice-vllm \
  --ipc=host \
  -p 8000:8000 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --dp 8"
```

--------------------------------

### Verify VibeVoice Model Loading

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

Check if the VibeVoice model is successfully loaded by querying the local API endpoint.

```bash
# Check the model is loaded
curl http://localhost:6001/v1/models
```

--------------------------------

### Launch NVIDIA PyTorch Docker Container

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-realtime-0.5b.md

Launches a privileged NVIDIA PyTorch Docker container. Ensure you have the correct version (e.g., 24.07) or a later compatible version. The --privileged flag is required for certain operations.

```bash
sudo docker run --privileged --net=host --ipc=host --ulimit memlock=-1:-1 --ulimit stack=-1:-1 --gpus all --rm -it  nvcr.io/nvidia/pytorch:24.07-py3
```

--------------------------------

### Run VibeVoice with vLLM using Tensor Parallel (2 GPUs)

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Configures vLLM with Tensor Parallelism to split a single model across 2 GPUs. This is useful when a model's memory requirements exceed a single GPU's capacity. Ensure the --gpus flag correctly specifies the devices.

```bash
docker run -d --gpus '"device=0,1"' --name vibevoice-vllm \
  --ipc=host \
  -p 8000:8000 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --tp 2"
```

--------------------------------

### Manage Streaming Text Preview

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Handles the logic for displaying a real-time text preview as it's being generated. It schedules ticks to append tokens at a defined interval.

```javascript
const STREAMING_WPM = 180;
const STREAMING_INTERVAL_MS = 60000 / STREAMING_WPM;
let previewTimeoutId = null;
let previewTokens = [];
let previewIndex = 0;
let previewActive = false;
const clearPreviewTimer = () => {
  if (previewTimeoutId) {
    clearTimeout(previewTimeoutId);
    previewTimeoutId = null;
  }
};
const setPreviewIdle = message => {
  if (!streamingPreview) {
    return;
  }
  streamingPreview.classList.remove('streaming-active');
  streamingPreview.textContent = message;
};
const schedulePreviewTick = () => {
  if (!streamingPreview) {
    return;
  }
  if (previewIndex >= previewTokens.length) {
    streamingPreview.classList.remove('streaming-active');
    return;
  }
  streamingPreview.classList.add('streaming-active');
  streamingPreview.textContent += previewTokens[previewIndex];
  previewIndex += 1;
  previewTimeoutId = setTimeout(schedulePreviewTick, STREAMING_INTERVAL_MS);
};
const updateStreamingPreview = () => {
  if (!streamingPreview) {
    return;
  }
  clearPreviewTimer();
  previewIndex = 0;
  const source = (promptInput?.value || '').trimEnd();
  streamingPreview.textContent = '';
  previewTokens = source.match(/\S+\s
omino/g) || [];
  schedulePreviewTick();
};
```

--------------------------------

### WebSocket Connection and Message Handling

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Establishes a WebSocket connection, sends parameters, and handles incoming messages, including audio data and log messages. It processes audio as Float32Array and logs errors.

```javascript
params.set('steps', stepsValue.toString()); } if (voiceValue) { params.set('voice', voiceValue); } const wsUrl = `${location.origin.replace(/^http/, 'ws')}/stream?${params.toString()}`; socket = new WebSocket(wsUrl); socket.binaryType = 'arraybuffer'; socket.onmessage = event => { if (typeof event.data === 'string') { handleLogMessage(event.data); return; } if (!(event.data instanceof ArrayBuffer)) { return; } const rawBuffer = event.data.slice(0); const view = new DataView(rawBuffer); const floatChunk = new Float32Array(view.byteLength / 2); for (let i = 0; i < floatChunk.length; i += 1) { floatChunk[i] = view.getInt16(i * 2, true) / 32768; } appendAudio(floatChunk); recordedChunks.push(rawBuffer); recordedSamples += floatChunk.length; updateSaveButtonState(); if (!firstBrowserChunkLogged) { firstBrowserChunkLogged = true; appendLog('[Frontend] Received first audio chunk'); } }; socket.onerror = err => { console.error('WebSocket error', err); appendLog(`[Error] WebSocket error: ${err?.message || err}`); stop(); }; socket.onclose = () => { socket = null; if (recordedSamples > 0) { recordingComplete = true; updateSaveButtonState(); } };
```

--------------------------------

### View vLLM Container Logs

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Follows the logs of the running 'vibevoice-vllm' Docker container in real-time. This is useful for monitoring the service's status and debugging issues.

```bash
docker logs -f vibevoice-vllm
```

--------------------------------

### Basic API Transcription

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Use this script for basic audio transcription. Ensure the audio file is accessible within the container.

```bash
docker exec -it vibevoice-vllm python3 vllm_plugin/tests/test_api.py /app/audio.wav
```

--------------------------------

### Inference with Fine-tuned LoRA Model

Source: https://github.com/microsoft/vibevoice/blob/main/finetuning-asr/README.md

Performs inference using a fine-tuned LoRA model on a specified audio file, with optional context information.

```bash
python inference_lora.py \
    --base_model microsoft/VibeVoice-ASR \
    --lora_path ./output \
    --audio_file ./toy_dataset/0.mp3 \
    --context_info "Tea Brew, Aiden Host"
```

--------------------------------

### Inference from Audio Files

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-asr.md

Performs ASR inference directly from specified audio files using the VibeVoice model. Replace '[add an audio path here]' with the actual path to your audio file(s).

```bash
python demo/vibevoice_asr_inference_from_file.py --model_path microsoft/VibeVoice-ASR --audio_files [add an audio path here]
```

--------------------------------

### Login to Hugging Face

Source: https://github.com/microsoft/vibevoice/blob/main/demo/vibevoice_realtime_colab.ipynb

This snippet is used to log in to Hugging Face, which may be necessary if the model download is interrupted or stuck.

```python
from huggingface_hub import login, snapshot_download
login()

```

--------------------------------

### Reset Controls to Defaults

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Resets the configuration and steps select elements to their default values and updates the display accordingly. It also logs this action.

```javascript
resetBtn.addEventListener('click', () => {
  cfgSelect.value = '1.5';
  stepsSelect.value = '5';
  updateCfgDisplay();
  updateStepsDisplay();
  appendLog('[Frontend] Controls reset to defaults (CFG=1.5, Steps=5)');
});
```

--------------------------------

### API Transcription with Auto-Recovery and Hotwords

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Combine auto-recovery for long audio with hotwords for improved recognition of specific terms. This offers robust transcription for complex audio.

```bash
docker exec -it vibevoice-vllm python3 vllm_plugin/tests/test_api_auto_recover.py /app/audio.wav --hotwords "Microsoft,VibeVoice"
```

--------------------------------

### JSON Label Format for Training Data

Source: https://github.com/microsoft/vibevoice/blob/main/finetuning-asr/README.md

Defines the structure for JSON label files, including audio duration, path, segments with speaker, text, start/end times, and optional customized context.

```json
{
  "audio_duration": 351.73,
  "audio_path": "0.mp3",
  "segments": [
    {
      "speaker": 0,
      "text": "Hey everyone, welcome back...",
      "start": 0.0,
      "end": 38.68
    },
    {
      "speaker": 1,
      "text": "Thanks for having me...",
      "start": 38.75,
      "end": 77.88
    }
  ],
  "customized_context": ["Tea Brew", "Aiden Host", "The property is near Meter Street."]  // optional, domain-specific terms or context sentences
}
```

--------------------------------

### Stop and Remove Docker Container

Source: https://github.com/microsoft/vibevoice/blob/main/docs/setup_gradio_demo.md

Stops and removes the VibeVoice ASR demo Docker container. Use this to completely shut down the service and free up resources.

```bash
docker stop vibevoice-asr-demo
docker rm vibevoice-asr-demo
```

--------------------------------

### API Transcription with Auto-Recovery

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Enable auto-recovery from repetition loops for long audio files. This script helps manage extended audio inputs.

```bash
docker exec -it vibevoice-vllm python3 vllm_plugin/tests/test_api_auto_recover.py /app/audio.wav
```

--------------------------------

### Reset All State

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Performs a comprehensive reset of the application state, including closing the WebSocket, tearing down audio, and resetting playback flags and UI elements.

```javascript
const resetState = (resetSamples = true) => {
  closeSocket();
  teardownAudio();
  resetPlaybackFlags(resetSamples);
  isPlaying = false;
  stopPlaybackTimer();
};
```

--------------------------------

### API Transcription with Hotwords

Source: https://github.com/microsoft/vibevoice/blob/main/docs/vibevoice-vllm-asr.md

Enhance recognition accuracy for specific terms by providing hotwords. Separate hotwords with commas.

```bash
docker exec -it vibevoice-vllm python3 vllm_plugin/tests/test_api.py /app/audio.wav --hotwords "Microsoft,VibeVoice"
```

--------------------------------

### Handle Backend Events

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Processes incoming events from the backend, such as request status, audio chunks, and errors. Appends logs and updates UI states based on event types.

```javascript
const { event, data = {}, timestamp } = payload;
switch (event) {
  case 'backend_request_received': {
    const cfg = typeof data.cfg_scale === 'number' ? data.cfg_scale.toFixed(3) : data.cfg_scale;
    const steps = data.inference_steps ?? 'default';
    const voice = data.voice || 'default';
    const textLength = data.text_length ?? 0;
    appendLog(`[Backend] Received request`, timestamp);
    break;
  }
  case 'backend_first_chunk_sent':
    appendLog('[Backend] Sent first audio chunk', timestamp);
    break;
  case 'model_progress':
    if (typeof data.generated_sec !== 'undefined') {
      const generated = Number(data.generated_sec);
      if (Number.isFinite(generated)) {
        setModelGenerated(generated);
      }
    }
    return;
  case 'generation_error':
    appendLog(`[Error] Generation error: ${data.message || 'Unknown error'}`, timestamp);
    break;
  case 'backend_error':
    appendLog(`[Error] Backend error: ${data.message || 'Unknown error'}`, timestamp);
    break;
  case 'client_disconnected':
    appendLog('[Frontend] Client disconnected', timestamp);
    break;
  case 'backend_stream_complete':
    appendLog('[Backend] Backend finished', timestamp);
    recordingComplete = true;
    updateSaveButtonState();
    break;
  default:
    appendLog(`[Log] Event ${event}`, timestamp);
    break;
}
```

--------------------------------

### Append Audio Chunk to Buffer

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Merges a new audio chunk into an existing buffer, creating a new Float32Array to hold the combined audio data.

```javascript
const appendAudio = chunk => {
  const merged = new Float32Array(buffer.length + chunk.length);
  merged.set(buffer, 0);
  merged.set(chunk, buffer.length);
  buffer = merged;
};
```

--------------------------------

### Update Streaming Preview on Input

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Triggers an update of the streaming text preview when the prompt input value changes, but only if the preview is currently active.

```javascript
if (promptInput) {
  promptInput.addEventListener('input', () => {
    if (previewActive) {
      updateStreamingPreview();
    }
  });
}
```

--------------------------------

### Pull Audio Frames from Buffer

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Retrieves a specified number of audio frames from the buffer. If the buffer is empty or has fewer frames than requested, it returns available frames or a padded array of zeros.

```javascript
const pullAudio = frameCount => {
  const available = buffer.length;
  if (available === 0) {
    return new Float32Array(frameCount);
  }
  if (available <= frameCount) {
    const chunk = buffer;
    buffer = new Float32Array(0);
    if (chunk.length < frameCount) {
      const padded = new Float32Array(frameCount);
      padded.set(chunk, 0);
      return padded;
    }
    return chunk;
  }
  const chunk = buffer.subarray(0, frameCount);
  buffer = buffer.subarray(frameCount);
  return chunk;
};
```

--------------------------------

### Merge LoRA Weights into Base Model

Source: https://github.com/microsoft/vibevoice/blob/main/finetuning-asr/README.md

Loads a base model and applies LoRA weights, then merges them for faster inference and saves the combined model.

```python
from peft import PeftModel

# Load base model + LoRA
model = VibeVoiceASRForConditionalGeneration.from_pretrained("microsoft/VibeVoice-ASR", ...)
model = PeftModel.from_pretrained(model, "./output")

# Merge and save
model = model.merge_and_unload()
model.save_pretrained("./merged_model")
```

--------------------------------

### Tear Down Audio Context and Node

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Disconnects the audio script node and closes the audio context to release audio resources. Handles potential errors during disconnection.

```javascript
const teardownAudio = () => {
  if (scriptNode) {
    try {
      scriptNode.disconnect();
    } catch (err) {
      console.warn('disconnect error', err);
    }
    scriptNode.onaudioprocess = null;
  }
  if (audioCtx) {
    try {
      audioCtx.close();
    } catch (err) {
      console.warn('audioCtx.close error', err);
    }
  }
  audioCtx = null;
  scriptNode = null;
};
```

--------------------------------

### Reset Playback State Flags

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Resets various flags and counters related to audio playback, including the audio buffer, playback samples, and elapsed time. Optionally resets sample counts.

```javascript
const resetPlaybackFlags = (resetSamples = true) => {
  buffer = new Float32Array(0);
  if (resetSamples) {
    playbackSamples = 0;
    setPlaybackElapsed(0);
  }
  hasStartedPlayback = false;
  silentFrameCount = 0;
  firstBrowserChunkLogged = false;
  playbackStartedLogged = false;
};
```

--------------------------------

### Format Seconds to MM:SS String

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Converts a total number of seconds into a human-readable MM:SS string format. Used for displaying elapsed time.

```javascript
const formatSeconds = totalSeconds => {
  const minutes = Math.floor(totalSeconds / 60);
  const seconds = Math.floor(totalSeconds % 60);
  return `${String(minutes).padStart(2, '0')}:${String(seconds).padStart(2, '0')}`;
};
```

--------------------------------

### Stop Function for WebSocket Stream

Source: https://github.com/microsoft/vibevoice/blob/main/demo/web/index.html

Handles stopping the current playback or recording process. It resets the state, updates UI elements, and manages the save button state based on recorded samples.

```javascript
const stop = () => { if (!isPlaying) { resetState(false); updateButtonLabel(); return; } resetState(false); setPlaybackElapsed(Math.min(lastPlaybackElapsed, modelGeneratedTotal)); appendLog('[Frontend] Playback stopped'); if (recordedSamples > 0) { recordingComplete = true; updateSaveButtonState(); } previewActive = false; clearPreviewTimer(); streamingPreview?.classList.remove('streaming-active'); updateButtonLabel(); };
```