### Run Qwen3-ASR Docker Container

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

This command pulls the latest Qwen3-ASR Docker image and starts a container, mounting a local workspace, exposing a port, and enabling GPU access. Ensure the NVIDIA Container Toolkit is installed.

```bash
LOCAL_WORKDIR=/path/to/your/workspace
HOST_PORT=8000
CONTAINER_PORT=80
docker run --gpus all --name qwen3-asr \
    -v /var/run/docker.sock:/var/run/docker.sock -p $HOST_PORT:$CONTAINER_PORT \
    --mount type=bind,source=$LOCAL_WORKDIR,target=/data/shared/Qwen3-ASR \
    --shm-size=4gb \
    -it qwenllm/qwen3-asr:latest
```

--------------------------------

### Install qwen-asr Package (vLLM Backend)

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Install the 'qwen-asr' Python package with the '[vllm]' extra, enabling the vLLM backend for faster inference and streaming support. This installation is recommended for performance-critical applications.

```bash
pip install -U qwen-asr[vllm]
```

--------------------------------

### Install qwen-asr Package (Transformers Backend)

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Install the 'qwen-asr' Python package with minimal dependencies, providing support for the transformers backend. This is the basic installation for using the Qwen3-ASR models.

```bash
pip install -U qwen-asr
```

--------------------------------

### Start vLLM Server for Qwen3-ASR

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Launches a vLLM server for Qwen3-ASR using the `qwen-asr-serve` command. This command is a wrapper around `vllm serve` and accepts arguments compatible with `vllm serve` for server configuration.

```bash
qwen-asr-serve Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000
```

--------------------------------

### Install Qwen-ASR and Datasets Packages

Source: https://github.com/qwenlm/qwen3-asr/blob/main/finetuning/README.md

Installs the 'qwen-asr' and 'datasets' Python packages required for fine-tuning. This is the first step in setting up the environment for Qwen3-ASR fine-tuning.

```bash
pip install -U qwen-asr datasets
```

--------------------------------

### Run Qwen3-ASR Streaming Demo

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

This command starts a Flask-based web UI demo for Qwen3-ASR's streaming transcription. It captures microphone audio, resamples it, and sends it to the model. Ensure the Qwen/Qwen3-ASR-1.7B model is accessible.

```bash
qwen-asr-demo-streaming \
  --asr-model-path Qwen/Qwen3-ASR-1.7B \
  --gpu-memory-utilization 0.9 \
  --host 0.0.0.0 \
  --port 8000
```

--------------------------------

### Install qwen-asr from Source (Editable Mode)

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Clone the Qwen3-ASR repository and install the package in editable mode for local development or modification. This allows changes to the code to be reflected immediately without reinstallation.

```bash
git clone https://github.com/QwenLM/Qwen3-ASR.git
cd Qwen3-ASR
pip install -e .
# support vLLM backend
# pip install -e ".[vllm]"
```

--------------------------------

### Install Qwen3-ASR with Transformers and vLLM Backends

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

Installs the Qwen3-ASR package with support for the Transformers backend for basic inference. Includes an option to install with the vLLM backend for enhanced performance and streaming capabilities. FlashAttention 2 can be optionally installed for improved memory efficiency.

```bash
# Basic installation with Transformers backend
pip install -U qwen-asr

# With vLLM backend for faster inference and streaming
pip install -U qwen-asr[vllm]

# Optional: FlashAttention 2 for memory efficiency
pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Install vLLM for Qwen3-ASR

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Installs the nightly version of vLLM with audio dependencies, recommended for Qwen3-ASR. It uses `uv` as the environment manager and specifies CUDA versions. Ensure you have a compatible CUDA toolkit installed.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --pre \
    --extra-index-url https://wheels.vllm.ai/nightly/cu129 \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --index-strategy unsafe-best-match
uv pip install "vllm[audio]"
```

--------------------------------

### Initialize Qwen3-ASR with vLLM Backend for Fast Inference

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Initializes the Qwen3-ASR model using the vLLM backend for accelerated inference. Requires `qwen-asr[vllm]` installation. FlashAttention is recommended for timestamp output. Ensure code is wrapped in `if __name__ == '__main__':` to prevent multiprocessing errors.

```python
import torch
from qwen_asr import Qwen3ASRModel

if __name__ == '__main__':
    model = Qwen3ASRModel.LLM(
        model="Qwen/Qwen3-ASR-1.7B",
        gpu_memory_utilization=0.7,
        max_inference_batch_size=128, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
        max_new_tokens=4096, # Maximum number of tokens to generate. Set a larger value for long audio input.
        forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
        forced_aligner_kwargs=dict(
            dtype=torch.bfloat16,
            device_map="cuda:0",
            # attn_implementation="flash_attention_2",
        ),
    )

    results = model.transcribe(
        audio=[
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
        ],
        language=["Chinese", "English"], # can also be set to None for automatic language detection
        return_time_stamps=True,
    )

    for r in results:
        print(r.language, r.text, r.time_stamps[0])
```

--------------------------------

### Deploy Qwen3-ASR as vLLM OpenAI-Compatible API Server

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

Instructions for deploying the Qwen3-ASR model as an API server using vLLM, making it compatible with OpenAI's API standards. The server exposes endpoints for chat completions and audio transcription. Example commands for starting the server and making requests via Python `requests` and `curl` are provided.

```bash
# Start the server
qwen-asr-serve Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000
```

--------------------------------

### Start and Execute in Existing Qwen3-ASR Container

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Commands to start a stopped Qwen3-ASR Docker container and then execute a bash shell within it. This is useful for re-entering a previously created container.

```bash
docker start qwen3-asr
docker exec -it qwen3-asr bash
```

--------------------------------

### Run Qwen3-ASR with Transformers Backend

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

This command starts the Qwen3-ASR demo using the Transformers backend. It specifies the ASR checkpoint, backend type, and network configuration for the server. This is suitable for quick prototyping.

```bash
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --backend transformers \
  --cuda-visible-devices 0 \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### Initialize Qwen3ASRModel with vLLM Backend for High Throughput

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

Initializes the Qwen3ASRModel using the vLLM backend, optimized for high-throughput inference and streaming. This requires the `qwen-asr[vllm]` installation and must be placed within an `if __name__ == '__main__':` block. It supports similar configuration options as the Transformers backend, including forced aligner integration.

```python
import torch
from qwen_asr import Qwen3ASRModel

if __name__ == '__main__':
    model = Qwen3ASRModel.LLM(
        model="Qwen/Qwen3-ASR-1.7B",
        gpu_memory_utilization=0.8,
        max_inference_batch_size=128,
        max_new_tokens=4096,
        forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
        forced_aligner_kwargs=dict(
            dtype=torch.bfloat16,
            device_map="cuda:0",
        ),
    )

    # Use same transcribe() API as Transformers backend
    results = model.transcribe(
        audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
        language=None,
    )
    print(results[0].language, results[0].text)
```

--------------------------------

### Deploy Qwen3-ASR with vLLM

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Deploys the Qwen3-ASR model using vLLM for efficient online serving. This command starts an inference server that can be accessed via an API. The model path 'Qwen/Qwen3-ASR-1.7B' must be valid.

```bash
vllm serve Qwen/Qwen3-ASR-1.7B
```

--------------------------------

### Install FlashAttention 2 for Performance

Source: https://github.com/qwenlm/qwen3-asr/blob/main/finetuning/README.md

Installs FlashAttention 2 to optimize GPU memory usage and training speed. This is recommended for faster training but requires compatible hardware and specific PyTorch data types (float16 or bfloat16).

```bash
pip install -U flash-attn --no-build-isolation
```

```bash
MAX_JOBS=4 pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Launch Qwen3-ASR Demo Without Forced Aligner

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Launches the Qwen3-ASR demo without a forced aligner, which means timestamps will not be available. This is a simpler setup if timestamp information is not required.

```bash
qwen-asr-demo --asr-checkpoint Qwen/Qwen3-ASR-1.7B
```

--------------------------------

### Prepare JSONL Training Data for Fine-tuning

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

This section provides an example format for preparing training data in JSONL format for fine-tuning Qwen3-ASR. Each line should contain an audio file path and its corresponding transcript, potentially with language tags.

```bash
# Prepare JSONL training data
# train.jsonl format:
# {"audio":"/data/wavs/utt0001.wav","text":"language English<asr_text>This is a test sentence."}
# {"audio":"/data/wavs/utt0002.wav","text":"language Chinese<asr_text>Chinese transcript here."}
```

--------------------------------

### Install FlashAttention 2 with Limited Jobs

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Install FlashAttention 2 on machines with less than 96GB of RAM and many CPU cores, by limiting the number of parallel jobs during the build process. This can help prevent build failures on resource-constrained systems.

```bash
MAX_JOBS=4 pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Download Model Weights using Hugging Face CLI

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Manually download Qwen3-ASR model weights to a local directory using the Hugging Face CLI. This method is an alternative to ModelScope. It requires the 'huggingface_hub' package to be installed.

```bash
pip install -U "huggingface_hub[cli]"
huggingface-cli download Qwen/Qwen3-ASR-1.7B --local-dir ./Qwen3-ASR-1.7B
huggingface-cli download Qwen/Qwen3-ASR-0.6B --local-dir ./Qwen3-ASR-0.6B
huggingface-cli download Qwen/Qwen3-ForcedAligner-0.6B --local-dir ./Qwen3-ForcedAligner-0.6B
```

--------------------------------

### Install FlashAttention 2

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Install FlashAttention 2 for reduced GPU memory usage and accelerated inference speed, especially for long inputs and large batch sizes. This requires compatible hardware and specific PyTorch versions.

```bash
pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Download Model Weights using ModelScope

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Manually download Qwen3-ASR model weights to a local directory using the ModelScope CLI. This is recommended for users in Mainland China. It requires the 'modelscope' package to be installed.

```bash
pip install -U modelscope
modelscope download --model Qwen/Qwen3-ASR-1.7B  --local_dir ./Qwen3-ASR-1.7B
modelscope download --model Qwen/Qwen3-ASR-0.6B --local_dir ./Qwen3-ASR-0.6B
modelscope download --model Qwen/Qwen3-ForcedAligner-0.6B --local_dir ./Qwen3-ForcedAligner-0.6B
```

--------------------------------

### Run Qwen3-ASR Streaming Demo

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

This command starts a streaming demo for Qwen3-ASR, allowing for real-time audio transcription. It specifies the ASR model path and GPU memory utilization, along with network host and port.

```bash
qwen-asr-demo-streaming \
  --asr-model-path Qwen/Qwen3-ASR-1.7B \
  --gpu-memory-utilization 0.9 \
  --host 0.0.0.0 --port 8000
```

--------------------------------

### Override Transformers Backend Initialization Arguments

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Demonstrates how to override default initialization arguments for the Transformers backend using the `--backend-kwargs` option. This example shows how to enable flash attention for potentially faster inference.

```bash
--backend-kwargs '{"device_map":"cuda:0","dtype":"bfloat16","attn_implementation":"flash_attention_2"}'
```

--------------------------------

### Offline Inference with vLLM and Qwen3-ASR

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

This Python script demonstrates offline inference using vLLM with the Qwen3-ASR model. It loads an audio asset, constructs a conversation with the audio, and runs inference. Ensure vLLM and its audio dependencies are installed.

```python
from vllm import LLM, SamplingParams
from vllm.assets.audio import AudioAsset
import base64
import requests

# Initialize the LLM
llm = LLM(
    model="Qwen/Qwen3-ASR-1.7B"
)

# Load audio
audio_asset = AudioAsset("winning_call")

# Create conversation with audio content
conversation = [
    {
        "role": "user",
        "content": [
            {
                "type": "audio_url",
                "audio_url": {"url": audio_asset.url}
            }
        ]
    }
]

sampling_params = SamplingParams(temperature=0.01, max_tokens=256)

# Run inference using .chat()
outputs = llm.chat(conversation, sampling_params=sampling_params)
print(outputs[0].outputs[0].text)
```

--------------------------------

### One-Click Shell Script for Qwen3-ASR Fine-tuning

Source: https://github.com/qwenlm/qwen3-asr/blob/main/finetuning/README.md

A comprehensive bash script for fine-tuning Qwen3-ASR, incorporating multi-GPU setup with `torchrun` and various training parameters. It includes model and data paths, output directory, batching, learning rate, epochs, logging, saving strategies, and worker configurations.

```bash
#!/usr/bin/env bash
set -e

export CUDA_VISIBLE_DEVICES=0,1

MODEL_PATH="Qwen/Qwen3-ASR-1.7B"
TRAIN_FILE="./train.jsonl"
EVAL_FILE="./eval.jsonl"
OUTPUT_DIR="./qwen3-asr-finetuning-out"

torchrun --nproc_per_node=2 qwen3_asr_sft.py \
  --model_path ${MODEL_PATH} \
  --train_file ${TRAIN_FILE} \
  --eval_file ${EVAL_FILE} \
  --output_dir ${OUTPUT_DIR} \
  --batch_size 32 \
  --grad_acc 4 \
  --lr 2e-5 \
  --epochs 1 \
  --log_steps 10 \
  --save_strategy steps \
  --save_steps 200 \
  --save_total_limit 5 \
  --num_workers 2 \
  --pin_memory 1 \
  --persistent_workers 1 \
  --prefetch_factor 2
```

--------------------------------

### Qwen3-ASR Input JSONL Format Example

Source: https://github.com/qwenlm/qwen3-asr/blob/main/finetuning/README.md

Demonstrates the required JSONL format for training data. Each line is a JSON object containing the 'audio' file path and its corresponding 'text' transcript, optionally prefixed with language information.

```jsonl
{"audio":"/data/wavs/utt0001.wav","text":"language English<asr_text>This is a test sentence."}
{"audio":"/data/wavs/utt0002.wav","text":"language English<asr_text>Another example."}
{"audio":"/data/wavs/utt0003.wav","text":"language English<asr_text>Fine-tuning data line."}
```

--------------------------------

### Launch Qwen3-ASR Gradio Demo with vLLM + Forced Aligner

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Launches the Qwen3-ASR web UI demo using the vLLM backend, which is known for its high throughput. This configuration also includes a forced aligner for timestamp generation and allows for specific vLLM and aligner backend configurations.

```bash
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B \
  --backend vllm \
  --cuda-visible-devices 0 \
  --backend-kwargs '{"gpu_memory_utilization":0.7,"max_inference_batch_size":8,"max_new_tokens":2048}' \
  --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### Launch Qwen3-ASR Gradio Demo with Transformers + Forced Aligner

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Launches the Qwen3-ASR web UI demo with the Transformers backend and enables timestamps by including a forced aligner checkpoint. It allows for detailed configuration of both the ASR and aligner backends, including device mapping and data types.

```bash
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B \
  --backend transformers \
  --cuda-visible-devices 0 \
  --backend-kwargs '{"device_map":"cuda:0","dtype":"bfloat16","max_inference_batch_size":8,"max_new_tokens":256}' \
  --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### vLLM Backend Initialization and Transcription

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Initialize the Qwen3ASRModel with the vLLM backend for optimized inference speed and transcribe audio files. This method supports batch processing and language specification.

```APIDOC
## Initialize Qwen3ASRModel with vLLM Backend

### Description
Initializes the Qwen3ASRModel using the vLLM backend for accelerated inference. This requires installing `qwen-asr[vllm]` and optionally `flash-attn` for improved forced aligner performance. Ensure code is wrapped in `if __name__ == '__main__':` to prevent multiprocessing issues.

### Method
`Qwen3ASRModel.LLM(...)`

### Parameters
- **model** (str) - Required - The name or path of the Qwen3-ASR model.
- **gpu_memory_utilization** (float) - Optional - Fraction of GPU memory to use for inference.
- **max_inference_batch_size** (int) - Optional - Maximum batch size for inference. `-1` for unlimited.
- **max_new_tokens** (int) - Optional - Maximum number of tokens to generate. Adjust for long audio.
- **forced_aligner** (str) - Optional - Path or name of the forced aligner model.
- **forced_aligner_kwargs** (dict) - Optional - Keyword arguments for the forced aligner initialization, such as `dtype` and `device_map`.

### Request Example
```python
import torch
from qwen_asr import Qwen3ASRModel

if __name__ == '__main__':
    model = Qwen3ASRModel.LLM(
        model="Qwen/Qwen3-ASR-1.7B",
        gpu_memory_utilization=0.7,
        max_inference_batch_size=128,
        max_new_tokens=4096,
        forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
        forced_aligner_kwargs=dict(
            dtype=torch.bfloat16,
            device_map="cuda:0",
        ),
    )
```

## Transcribe Audio using vLLM Backend

### Description
Transcribes one or more audio files using the initialized vLLM backend model. Supports specifying the language for each audio file or automatic detection. Can return timestamps.

### Method
`model.transcribe(...)`

### Parameters
- **audio** (list[str] or str) - Required - A list of audio file paths/URLs or a single path/URL.
- **language** (list[str] or str or None) - Optional - A list of languages corresponding to the audio files, or a single language, or `None` for auto-detection.
- **return_time_stamps** (bool) - Optional - Whether to return timestamps for the transcription.

### Request Example
```python
    results = model.transcribe(
        audio=[
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
        ],
        language=["Chinese", "English"],
        return_time_stamps=True,
    )

    for r in results:
        print(r.language, r.text, r.time_stamps[0])
```

### Response
#### Success Response (200)
- **results** (list[dict]) - A list of transcription results, where each result contains:
  - **language** (str) - The detected or specified language.
  - **text** (str) - The transcribed text.
  - **time_stamps** (list[dict]) - List of timestamp objects if `return_time_stamps` is True.

#### Response Example
```json
[
  {
    "language": "Chinese",
    "text": "你好世界",
    "time_stamps": [{"start_time": 0.1, "end_time": 0.5, "text": "你好"}]
  }
]
```
```

--------------------------------

### Qwen3-ASR Chat Completion with OpenAI SDK

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Demonstrates how to use the OpenAI Python SDK to interact with a vLLM-served Qwen3-ASR model for chat completions, including audio input. Requires the vLLM server to be running on http://localhost:8000/v1.

```python
import base64
import httpx
from openai import OpenAI

# Initialize client
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY"
)

# Create multimodal chat completion request
response = client.chat.completions.create(
    model="Qwen/Qwen3-ASR-1.7B",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "audio_url",
                    "audio_url": {
                        {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"}
                    }
                }
            ]
        }
    ],
)

print(response.choices[0].message.content)
```

--------------------------------

### Initialize Qwen3ASRModel with Transformers Backend

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

Loads the Qwen3ASRModel using the HuggingFace Transformers library. This initialization supports specifying data type, device mapping, maximum inference batch size, and new tokens. It also allows for the optional inclusion of a forced aligner for generating timestamps.

```python
import torch
from qwen_asr import Qwen3ASRModel

# Basic initialization
model = Qwen3ASRModel.from_pretrained(
    "Qwen/Qwen3-ASR-1.7B",
    dtype=torch.bfloat16,
    device_map="cuda:0",
    # attn_implementation="flash_attention_2",  # Enable for faster inference
    max_inference_batch_size=32,  # Limit batch size to avoid OOM
    max_new_tokens=256,  # Increase for longer audio
)

# With forced aligner for timestamps
model_with_timestamps = Qwen3ASRModel.from_pretrained(
    "Qwen/Qwen3-ASR-1.7B",
    dtype=torch.bfloat16,
    device_map="cuda:0",
    forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
    forced_aligner_kwargs=dict(
        dtype=torch.bfloat16,
        device_map="cuda:0",
    ),
    max_inference_batch_size=32,
    max_new_tokens=256,
)
```

--------------------------------

### Launch Qwen3-ASR Gradio Demo with HTTPS

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Launches the Qwen3-ASR Gradio demo over HTTPS using the generated SSL certificate and key. This ensures secure communication and is particularly useful when accessing the demo remotely or through modern browsers.

```bash
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --backend transformers \
  --cuda-visible-devices 0 \
  --ip 0.0.0.0 --port 8000 \
  --ssl-certfile cert.pem \
  --ssl-keyfile key.pem \
  --no-ssl-verify
```

--------------------------------

### Launch Qwen3-ASR Demo With Forced Aligner for Timestamps

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Launches the Qwen3-ASR demo including a forced aligner, enabling the generation of timestamps. This configuration is necessary for applications that require precise timing information for transcribed speech.

```bash
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B
```

--------------------------------

### Select Specific CUDA GPU for Demo

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Illustrates how to select a specific GPU for the demo using the `--cuda-visible-devices` environment variable. This is crucial for managing multiple GPUs or when vLLM's default device selection is not desired.

```bash
# Use GPU 0
--cuda-visible-devices 0

# Use GPU 1
--cuda-visible-devices 1
```

--------------------------------

### Run Qwen3-ASR with vLLM Backend and Forced Aligner

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

This command launches the Qwen3-ASR demo using the vLLM backend, which is optimized for high-throughput serving. It also includes a forced aligner checkpoint and specific backend configurations for GPU memory utilization and data types.

```bash
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B \
  --backend vllm \
  --cuda-visible-devices 0 \
  --backend-kwargs '{"gpu_memory_utilization":0.7}' \
  --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### Override vLLM Backend Initialization Arguments

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Shows how to customize vLLM backend initialization by specifying GPU memory utilization via the `--backend-kwargs` option. This allows for fine-tuning resource allocation.

```bash
--backend-kwargs '{"gpu_memory_utilization":0.65}'
```

--------------------------------

### Basic Audio Transcription with Qwen3-ASR (Transformers Backend)

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

This code snippet demonstrates how to perform basic audio transcription using the Qwen3-ASR package with the transformers backend. It loads a pre-trained model and transcribes a single audio file from a URL, printing the detected language and transcribed text. Dependencies include `torch` and `qwen_asr`.

```python
import torch
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "Qwen/Qwen3-ASR-1.7B",
    dtype=torch.bfloat16,
    device_map="cuda:0",
    # attn_implementation="flash_attention_2",
    max_inference_batch_size=32, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
    max_new_tokens=256, # Maximum number of tokens to generate. Set a larger value for long audio input.
)

results = model.transcribe(
    audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
    language=None, # set "English" to force the language
)

print(results[0].language)
print(results[0].text)
```

--------------------------------

### Word-Level Timestamp Alignment with Qwen3ForcedAligner

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

Utilizes the Qwen3ForcedAligner model to align text transcripts with audio, providing word or character-level timestamps. It supports multiple languages and can perform alignment on single samples or batches. The alignment results include the text, start time, and end time for each aligned segment.

```python
import torch
from qwen_asr import Qwen3ForcedAligner

aligner = Qwen3ForcedAligner.from_pretrained(
    "Qwen/Qwen3-ForcedAligner-0.6B",
    dtype=torch.bfloat16,
    device_map="cuda:0",
)

# Single sample alignment
results = aligner.align(
    audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
    text="transcript text here",
    language="Chinese",
)

for item in results[0]:
    print(f"'{item.text}': {item.start_time}s -> {item.end_time}s")

# Batch alignment with multiple languages
results = aligner.align(
    audio=[
        "https://example.com/audio_zh.wav",
        "https://example.com/audio_en.wav",
    ],
    text=[
        "Chinese transcript",
        "English transcript goes here",
    ],
    language=["Chinese", "English"],
)

for i, result in enumerate(results):
    print(f"Sample {i}: {len(result)} aligned items")
```

--------------------------------

### Initialize Qwen3ForcedAligner for Text-Speech Alignment

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Initializes the `Qwen3ForcedAligner` model for aligning text with speech and obtaining word or character-level timestamps. Requires PyTorch and the `qwen-asr` library. The model can be loaded with specified data types and device mapping.

```python
import torch
from qwen_asr import Qwen3ForcedAligner

model = Qwen3ForcedAligner.from_pretrained(
    "Qwen/Qwen3-ForcedAligner-0.6B",
    dtype=torch.bfloat16,
    device_map="cuda:0",
    # attn_implementation="flash_attention_2",
)

results = model.align(
    audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
    text="甚至出现交易几乎停滞的情况。",
    language="Chinese",
)

print(results[0])
print(results[0][0].text, results[0][0].start_time, results[0][0].end_time)
```

--------------------------------

### Qwen3-ASR Transcription with OpenAI SDK

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Shows how to perform audio transcription using the OpenAI Python SDK with a vLLM-served Qwen3-ASR model. It fetches an audio file and sends it for transcription. The vLLM server must be accessible at http://localhost:8000/v1.

```python
import httpx
from openai import OpenAI

# Initialize client
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY"
)
audio_url = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"
audio_file = httpx.get(audio_url).content

transcription = client.audio.transcriptions.create(
    model="Qwen/Qwen3-ASR-1.7B",
    file=audio_file,
)

print(transcription.text)
```

--------------------------------

### vLLM Server Deployment

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Deploy a vLLM server for Qwen3-ASR using the `qwen-asr-serve` command. This allows for remote inference requests via an API.

```APIDOC
## Deploy vLLM Server

### Description
Starts a vLLM inference server for Qwen3-ASR. This command is a wrapper around `vllm serve` and accepts arguments compatible with it.

### Method
`qwen-asr-serve` (command-line tool)

### Parameters
- **[MODEL_NAME_OR_PATH]** (str) - Required - The name or path of the Qwen3-ASR model.
- **--gpu-memory-utilization** (float) - Optional - Fraction of GPU memory to use.
- **--host** (str) - Optional - Host address for the server.
- **--port** (int) - Optional - Port for the server.
- **[Other vLLM serve arguments]** - Any arguments supported by `vllm serve` can be passed.

### Request Example
```bash
qwen-asr-serve Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000
```

## Send Request to vLLM Server

### Description
Sends an audio transcription request to a running vLLM server using the OpenAI-compatible API format.

### Method
`POST`

### Endpoint
`/v1/chat/completions` (relative to the server address)

### Parameters
#### Request Body
- **messages** (list) - Required - A list of message objects. For ASR, the user role should contain an audio URL.
  - **role** (str) - Must be "user".
  - **content** (list) - Required - List of content parts.
    - **type** (str) - Must be "audio_url".
    - **audio_url** (dict) - Contains the audio URL.
      - **url** (str) - Required - The URL of the audio file.

### Request Example
```python
import requests

url = "http://localhost:8000/v1/chat/completions"
headers = {"Content-Type": "application/json"}

data = {
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "audio_url",
                    "audio_url": {
                        "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"
                    },
                }
            ],
        }
    ]
}

response = requests.post(url, headers=headers, json=data, timeout=300)
response.raise_for_status()
content = response.json()['choices'][0]['message']['content']
print(content)

from qwen_asr import parse_asr_output
language, text = parse_asr_output(content)
print(language)
print(text)
```

### Response
#### Success Response (200)
- **choices** (list) - List of response choices.
  - **message** (dict)
    - **content** (str) - The transcribed text, potentially including language information.

#### Response Example
```json
{
  "id": "chatcmpl-xxxxxxxxxxxxxxxxxxxx",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "Qwen/Qwen3-ASR-1.7B",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\"language\": \"English\", \"text\": \"Hello world.\", \"time_stamps\": []}"
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 1,
    "completion_tokens": 1,
    "total_tokens": 2
  }
}
```
```

--------------------------------

### Gradio Web Demo

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

Launch a local web UI for interactive transcription with optional timestamp visualization.

```APIDOC
## Gradio Web Demo

### Description
Launch a local web UI for interactive transcription with optional timestamp visualization.

### Method
N/A (Command Line Interface)

### Endpoint
N/A (Local web UI)

### Parameters
None

### Request Example
```bash
# Command to launch the Gradio demo (specific command not provided in source)
# Example placeholder:
# gradio app.py
```

### Response
N/A (Launches a web interface)

### Response Example
N/A
```

--------------------------------

### Real-time Streaming Transcription with Qwen3ASRModel

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

Demonstrates how to perform real-time streaming transcription using the Qwen3ASRModel. It initializes the model, sets up streaming state with parameters for context, language, and chunk handling, and processes audio chunks incrementally. The state object is updated with intermediate results and finalized at the end.

```python
import numpy as np
from qwen_asr import Qwen3ASRModel

if __name__ == '__main__':
    model = Qwen3ASRModel.LLM(
        model="Qwen/Qwen3-ASR-1.7B",
        gpu_memory_utilization=0.8,
        max_new_tokens=32,  # Small value for streaming
    )

    # Initialize streaming state
    state = model.init_streaming_state(
        context="",
        language=None,  # Or "English" to force language
        unfixed_chunk_num=2,  # Reset prefix for first N chunks
        unfixed_token_num=5,  # Rollback N tokens for jitter reduction
        chunk_size_sec=2.0,  # Chunk duration
    )

    # Simulate streaming audio (16kHz mono PCM)
    audio_16k = np.random.randn(80000).astype(np.float32)  # 5 seconds
    step_samples = 8000  # 0.5 second steps

    for i in range(0, len(audio_16k), step_samples):
        chunk = audio_16k[i:i + step_samples]
        model.streaming_transcribe(chunk, state)
        print(f"Partial: {state.language} - {state.text}")

    # Finalize transcription
    model.finish_streaming_transcribe(state)
    print(f"Final: {state.language} - {state.text}")
```

--------------------------------

### Transcribe Audio with Qwen3ASRModel

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

Demonstrates the `transcribe` method of Qwen3ASRModel for processing single or batched audio inputs. Supports various audio formats including URLs, local files, base64 strings, and NumPy arrays. Optional parameters include language forcing, context hints, and detailed timestamp generation.

```python
import torch
import numpy as np
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "Qwen/Qwen3-ASR-1.7B",
    dtype=torch.bfloat16,
    device_map="cuda:0",
    forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
    forced_aligner_kwargs=dict(dtype=torch.bfloat16, device_map="cuda:0"),
)

# Single audio from URL (auto language detection)
results = model.transcribe(
    audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
    language=None,
    return_time_stamps=False,
)
print(f"Language: {results[0].language}")
print(f"Text: {results[0].text}")

# Batch transcription with mixed inputs
results = model.transcribe(
    audio=[
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
        "data:audio/wav;base64,UklGRi...",  # Base64 encoded audio
        (np.random.randn(16000).astype(np.float32), 16000),  # NumPy array with sample rate
    ],
    context=["", "key words hint", ""],  # Context hints for better recognition
    language=["Chinese", None, "English"],  # Force language or auto-detect
    return_time_stamps=True,  # Get word-level timestamps
)

for i, r in enumerate(results):
    print(f"Sample {i}: {r.language} - {r.text}")
    if r.time_stamps:
        for ts in r.time_stamps[:3]:  # First 3 timestamps
            print(f"  '{ts.text}': {ts.start_time}s -> {ts.end_time}s")
```

--------------------------------

### Batch Audio Transcription with Timestamps using Qwen3-ASR (Transformers Backend)

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

This code snippet shows how to perform batch audio transcription and obtain timestamps using the Qwen3-ASR package with the transformers backend. It initializes the model with a forced aligner and processes multiple audio files, printing language, text, and the first timestamp for each. Dependencies include `torch` and `qwen_asr`.

```python
import torch
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "Qwen/Qwen3-ASR-1.7B",
    dtype=torch.bfloat16,
    device_map="cuda:0",
    # attn_implementation="flash_attention_2",
    max_inference_batch_size=32, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
    max_new_tokens=256, # Maximum number of tokens to generate. Set a larger value for long audio input.
    forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
    forced_aligner_kwargs=dict(
        dtype=torch.bfloat16,
        device_map="cuda:0",
        # attn_implementation="flash_attention_2",
    ),
)

results = model.transcribe(
    audio=[
      "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
      "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
    ],
    language=["Chinese", "English"], # can also be set to None for automatic language detection
    return_time_stamps=True,
)

for r in results:
    print(r.language, r.text, r.time_stamps[0])
```

--------------------------------

### Load Fine-tuned Qwen3-ASR Model and Transcribe Audio

Source: https://context7.com/qwenlm/qwen3-asr/llms.txt

This Python script shows how to load a fine-tuned Qwen3-ASR model from a specified checkpoint directory. It then uses the loaded model to transcribe an audio file and prints the resulting text.

```python
# Load fine-tuned model
import torch
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "./qwen3-asr-finetuned/checkpoint-200",
    dtype=torch.bfloat16,
    device_map="cuda:0",
)

results = model.transcribe(audio="./test_audio.wav")
print(results[0].text)
```

--------------------------------

### Qwen3-ASR Quick Inference Test

Source: https://github.com/qwenlm/qwen3-asr/blob/main/finetuning/README.md

A Python script demonstrating how to load a fine-tuned Qwen3-ASR model and perform transcription on an audio file. It shows how to specify the model path, data type, device, and how to access the transcription results.

```python
import torch
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "qwen3-asr-finetuning-out/checkpoint-200",
    dtype=torch.bfloat16,
    device_map="cuda:0",
)

results = model.transcribe(
    audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
)

print(results[0].language)
print(results[0].text)
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/qwenlm/qwen3-asr/blob/main/README.md

Create a new isolated Conda environment named 'qwen3-asr' with Python 3.12 and activate it. This is recommended to avoid dependency conflicts.

```bash
conda create -n qwen3-asr python=3.12 -y
conda activate qwen3-asr
```