### Quickstart Inference with Transformers

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/7

This snippet demonstrates how to perform inference using the Qwen3-ASR-1.7B model with the transformers library. Ensure the transformers library is installed.

```python
from transformers import pipeline


pipe = pipeline("automatic-speech-recognition", model="Qwen/Qwen3-ASR-1.7B")


# Example usage:
# result = pipe("audio.wav")
# print(result["text"])
```

--------------------------------

### Initialize Qwen3-ASR with vLLM Backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blame/main/README.md

Use Qwen3ASRModel.LLM(...) for the fastest inference speed. Install with `pip install -U qwen-asr[vllm]`. For timestamps, install FlashAttention. Wrap code in `if __name__ == '__main__':` to avoid multiprocessing errors.

```python
import torch
from qwen_asr import Qwen3ASRModel

if __name__ == '__main__':
    model = Qwen3ASRModel.LLM(
        model="Qwen/Qwen3-ASR-1.7B",
        gpu_memory_utilization=0.7,
        max_inference_batch_size=128, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
        max_new_tokens=4096, # Maximum number of tokens to generate. Set a larger value for long audio input.
        forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
        forced_aligner_kwargs=dict(
            dtype=torch.bfloat16,
            device_map="cuda:0",
            # attn_implementation="flash_attention_2",
        ),
    )

    results = model.transcribe(
        audio=[
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
        ],
        language=["Chinese", "English"], # can also be set to None for automatic language detection
        return_time_stamps=True,
    )

    for r in results:
        print(r.language, r.text, r.time_stamps[0])
```

--------------------------------

### Install Qwen3-ASR from Source

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md?code=true

Clone the repository and install the package in editable mode for local development. The commented line shows how to include vLLM support.

```bash
git clone https://github.com/QwenLM/Qwen3-ASR.git
cd Qwen3-ASR
pip install -e .
# support vLLM backend
# pip install -e ".[vllm]"
```

--------------------------------

### Install Qwen-ASR with Transformers Backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Install the minimal 'qwen-asr' package with support for the transformers backend. This is the recommended minimal installation.

```bash
pip install -U qwen-asr
```

--------------------------------

### Run Qwen3-ASR Docker Container

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B

Pulls the latest qwenllm/qwen3-asr Docker image and starts a container with GPU access, volume mounting, and port forwarding. Ensure NVIDIA Container Toolkit is installed and replace `/path/to/your/workspace` with your actual local directory.

```bash
LOCAL_WORKDIR=/path/to/your/workspace
HOST_PORT=8000
CONTAINER_PORT=80
docker run --gpus all --name qwen3-asr \
    -v /var/run/docker.sock:/var/run/docker.sock -p $HOST_PORT:$CONTAINER_PORT \
    --mount type=bind,source=$LOCAL_WORKDIR,target=/data/shared/Qwen3-ASR \
    --shm-size=4gb \
    -it qwenllm/qwen3-asr:latest

```

--------------------------------

### Install Qwen-ASR from Source

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Clone the Qwen3-ASR repository and install the package in editable mode for local development or modification. This requires Git.

```bash
git clone https://github.com/QwenLM/Qwen3-ASR.git
cd Qwen3-ASR
pip install -e .
```

--------------------------------

### Install vLLM for Qwen3-ASR

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/resolve/main/README.md?download=true

Installs the nightly version of vLLM with audio dependencies using uv. Recommends using uv as the environment manager for installation.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --pre \
    --extra-index-url https://wheels.vllm.ai/nightly/cu129 \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --index-strategy unsafe-best-match
uv pip install "vllm[audio]"
```

--------------------------------

### Transcribe Audio with Qwen3-ASR-1.7B

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/7/files

Example of how to load the Qwen3-ASR-1.7B model and transcribe an audio file. Ensure you have the necessary libraries installed.

```python
import torch
from transformers import pipeline

# Load the ASR pipeline with the Qwen3-ASR-1.7B model
asr_pipeline = pipeline("audio-classification", model="Qwen/qwen3-asr-1.7b")

# Example audio file path (replace with your actual file)
audio_file = "path/to/your/audio.wav"

# Perform transcription
results = asr_pipeline(audio_file)

# Print the transcribed text
print(results[0].text)
```

--------------------------------

### Get Help for Qwen3-ASR Demo

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Displays help information for launching the Qwen3-ASR web UI gradio demo.

```bash
qwen-asr-demo --help
```

--------------------------------

### Get Help for Qwen-ASR Demo

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B

Use this command to see all available options for the qwen-asr-demo.

```bash
qwen-asr-demo --help

```

--------------------------------

### Launch Gradio Demo with Transformers Backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blame/main/README.md

Launches the Gradio demo using the Transformers backend. Ensure the 'qwen-asr' package is installed.

```bash
# Transformers backend
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --backend transformers \
  --cuda-visible-devices 0 \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### Install vLLM Nightly with Audio Dependencies

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Install the nightly version of vLLM, including necessary audio dependencies, using `uv` as the environment manager. This is required for deploying Qwen3-ASR with vLLM.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --pre \
    --extra-index-url https://wheels.vllm.ai/nightly/cu129 \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --index-strategy unsafe-best-match
uv pip install "vllm[audio]" # For additional audio dependencies
```

--------------------------------

### Install vLLM Nightly for Qwen3-ASR

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md?code=true

Installs the nightly version of vLLM with necessary audio dependencies. Recommended for Qwen3-ASR support. Ensure you are using 'uv' as your environment manager.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm \
 --pre \
 --extra-index-url https://wheels.vllm.ai/nightly/cu129 \
 --extra-index-url https://download.pytorch.org/whl/cu129 \
 --index-strategy unsafe-best-match
uv pip install "vllm[audio]" # For additional audio dependencies
```

--------------------------------

### Serve Qwen3-ASR model with vLLM

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

This command starts the vLLM serving process for the Qwen3-ASR model. It specifies the model name and resource utilization. This is an alternative to using qwen-asr-serve directly.

```bash
vllm serve Qwen/Qwen3-ASR-1.7B --served-model-name qwen-asr
```

--------------------------------

### Download Models with Hugging Face CLI

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Use the Hugging Face CLI to download Qwen3-ASR models. Ensure you have the huggingface_hub package installed.

```bash
pip install -U "huggingface_hub[cli]"
huggingface-cli download Qwen/Qwen3-ASR-1.7B --local-dir ./Qwen3-ASR-1.7B
huggingface-cli download Qwen/Qwen3-ASR-0.6B --local-dir ./Qwen3-ASR-0.6B
huggingface-cli download Qwen/Qwen3-ForcedAligner-0.6B --local-dir ./Qwen3-ForcedAligner-0.6B
```

--------------------------------

### Interact with vLLM Served Model using OpenAI SDK

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/7/files

Example of interacting with a vLLM-served Qwen3-ASR model using Python and the httpx library, simulating OpenAI SDK usage.

```python
import base64
import httpx
```

--------------------------------

### Offline Inference Setup with vLLM

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Provides the initial import statements for using vLLM for offline inference with audio assets. This snippet is a starting point for more complex offline processing pipelines.

```python
from vllm import LLM, SamplingParams
from vllm.assets.audio import AudioAsset
import base64
import requests
```

--------------------------------

### Run Qwen3-ASR-1.7B with Transformers Backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Launches the ASR demo using the Transformers backend. Ensure CUDA is available and visible.

```bash
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --backend transformers \
  --cuda-visible-devices 0 \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### Start vLLM Server for Qwen3-ASR

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B

Launch a vLLM server for Qwen3-ASR using the `qwen-asr-serve` command. This command is a wrapper for `vllm serve`, allowing you to pass various arguments supported by `vllm serve` to configure your server, such as GPU memory utilization, host, and port.

```shell
qwen-asr-serve Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000

```

--------------------------------

### Launch Qwen-ASR Demo with Transformers Backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/resolve/main/README.md?download=true

Launches the Qwen3-ASR web UI demo using the Transformers backend. Ensure the specified checkpoint is available.

```bash
# Transformers backend
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --backend transformers \
  --cuda-visible-devices 0 \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### Install qwen-asr with vLLM support

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

Installs the qwen-asr package with vLLM dependencies. This command may also install torch and vllm, but manual verification and potential reinstallation of CUDA-enabled versions are recommended.

```bash
pip install -U qwen-asr[vllm]
```

--------------------------------

### Launch Qwen-ASR Gradio Demo with Transformers Backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B

Launches the Gradio demo using the Transformers backend. Ensure CUDA is available and visible.

```bash
# Transformers backend
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --backend transformers \
  --cuda-visible-devices 0 \
  --ip 0.0.0.0 --port 8000

```

--------------------------------

### Running Qwen-ASR-Serve with Specific Parameters

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

This command initiates the vLLM API server for Qwen-ASR. Ensure the model path is correctly specified and GPU memory utilization is set appropriately.

```bash
qwen-asr-serve "/root/autodl-tmp/models/asr/1.7B/Qwen/Qwen3-ASR-1.7B" --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000
```

--------------------------------

### Install Specific Transformers Version

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

Attempt to resolve the model recognition error by installing a specific version of the Transformers library.

```bash
pip install transformers==4.57.6
```

--------------------------------

### Serve Qwen3-ASR model with specific path and resources

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

This command starts the online service for a Qwen3-ASR model located at a specific path, using vLLM for acceleration. It configures GPU memory utilization, host, and port.

```bash
qwen-asr-serve /root/autodl-tmp/models/asr/0.6B/Qwen/Qwen3-ASR-0.6B --gpu-memory-utilization 0.6 --host 0.0.0.0 --port 8000
```

--------------------------------

### Install flash-attn

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

Installs the flash-attn library, which may require a long compilation time. This is a crucial step for performance optimization with certain hardware.

```bash
pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Run Qwen-ASR Server

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

This command starts the Qwen-ASR server, similar to 'vllm serve'. It wraps vLLM for accelerated inference. Specify the model path, GPU memory utilization, host, and port.

```bash
qwen-asr-serve Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000
```

--------------------------------

### Install FlashAttention 2

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md?code=true

Install FlashAttention 2 to reduce GPU memory usage and accelerate inference. Use MAX_JOBS for machines with limited RAM.

```bash
pip install -U flash-attn --no-build-isolation
```

```bash
MAX_JOBS=4 pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Install Transformers from Source

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/18

Install the most up-to-date code for the Transformers library directly from its GitHub repository. This is useful if the latest release version does not yet support a new model.

```bash
pip install git+https://github.com/huggingface/transformers.git
```

--------------------------------

### Start and Re-enter Existing Qwen3-ASR Container

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B

Starts a stopped qwen3-asr container and re-enters its bash shell. This is used to resume work within an already configured container.

```bash
docker start qwen3-asr
docker exec -it qwen3-asr bash

```

--------------------------------

### Install vllm-omni

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

Installs the vllm-omni package, which has specific version requirements for vLLM and CUDA. Note that CUDA 12.8 may work even if CUDA 12.9 is stated as a requirement.

```bash
pip install vllm-omni==0.14.0
```

--------------------------------

### Install torch and torchaudio with CUDA support

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/discussions/5

After installing qwen-asr, it's recommended to uninstall existing torch, torchvision, and torchaudio, then reinstall CUDA-enabled versions to ensure compatibility with vLLM.

```bash
pip install torch==2.9.1+cu128 torchaudio==2.9.1+cu128
```

--------------------------------

### Initialize Qwen3-ASR with vLLM Backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/resolve/main/README.md?download=true

Use this snippet to initialize the Qwen3-ASR model with the vLLM backend for optimal inference speed. Ensure vLLM and FlashAttention are installed for enhanced performance and timestamping. Wrap code in `if __name__ == '__main__':` to prevent multiprocessing errors.

```python
import torch
from qwen_asr import Qwen3ASRModel

if __name__ == '__main__':
    model = Qwen3ASRModel.LLM(
        model="Qwen/Qwen3-ASR-1.7B",
        gpu_memory_utilization=0.7,
        max_inference_batch_size=128, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
        max_new_tokens=4096, # Maximum number of tokens to generate. Set a larger value for long audio input.
        forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
        forced_aligner_kwargs=dict(
            dtype=torch.bfloat16,
            device_map="cuda:0",
            # attn_implementation="flash_attention_2",
        ),
    )

    results = model.transcribe(
        audio=[
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
        "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
        ],
        language=["Chinese", "English"], # can also be set to None for automatic language detection
        return_time_stamps=True,
    )

    for r in results:
        print(r.language, r.text, r.time_stamps[0])
```

--------------------------------

### Launch Qwen3-ASR Gradio demo with Transformers backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md?code=true

Launches the Gradio demo using the Transformers backend. Ensure the Qwen/Qwen3-ASR-1.7B checkpoint is accessible.

```bash
# Transformers backend
qwen-asr-demo \
 --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
 --backend transformers \
 --cuda-visible-devices 0 \
 --ip 0.0.0.0 --port 8000
```

--------------------------------

### Install FlashAttention 2 with MAX_JOBS

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Install FlashAttention 2 with a specified number of jobs, recommended for machines with limited RAM and many CPU cores. Ensure hardware compatibility and use with float16 or bfloat16.

```bash
MAX_JOBS=4 pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Run Qwen3-ASR Docker Container

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

This command pulls the qwenllm/qwen3-asr Docker image and starts a container, mounting a local workspace and mapping host and container ports. It's designed for GPU acceleration and easy access to the model.

```bash
LOCAL_WORKDIR=/path/to/your/workspace
HOST_PORT=8000
CONTAINER_PORT=80
docker run --gpus all --name qwen3-asr \
    -v /var/run/docker.sock:/var/run/docker.sock -p $HOST_PORT:$CONTAINER_PORT \
    --mount type=bind,source=$LOCAL_WORKDIR,target=/data/shared/Qwen3-ASR \
    --shm-size=4gb \
    -it qwenllm/qwen3-asr:latest
```

--------------------------------

### Launch Qwen-ASR Demo with vLLM Backend and Forced Aligner

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/resolve/main/README.md?download=true

Launches the Qwen3-ASR web UI demo using the vLLM backend and a forced aligner for timestamp generation. This configuration is optimized for performance with vLLM.

```bash
# vLLM backend + Forced Aligner (enable timestamps)
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B \
  --backend vllm \
  --cuda-visible-devices 0 \
  --backend-kwargs '{"gpu_memory_utilization":0.7,"max_inference_batch_size":8,"max_new_tokens":2048}' \
  --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### Start and Execute Commands in Existing Qwen3-ASR Container

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

These commands are used to start a previously stopped Docker container named 'qwen3-asr' and then attach to its bash shell. This allows resuming work within the containerized environment.

```bash
docker start qwen3-asr
docker exec -it qwen3-asr bash
```

--------------------------------

### Transcribe Audio with Qwen3-ASR using vLLM Backend

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md?code=true

Use this snippet to transcribe audio files with Qwen3-ASR for fast inference. Ensure vLLM is installed and consider installing FlashAttention for improved forced aligner performance. Wrap code in `if __name__ == '__main__':` to prevent multiprocessing errors.

```python
import torch
from qwen_asr import Qwen3ASRModel

if __name__ == '__main__':
 model = Qwen3ASRModel.LLM(
 model="Qwen/Qwen3-ASR-1.7B",
 gpu_memory_utilization=0.7,
 max_inference_batch_size=128, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
 max_new_tokens=4096, # Maximum number of tokens to generate. Set a larger value for long audio input.
 forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
 forced_aligner_kwargs=dict(
 dtype=torch.bfloat16,
 device_map="cuda:0",
 # attn_implementation="flash_attention_2",
 ),
 )

 results = model.transcribe(
 audio=[
 "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
 "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
 ],
 language=["Chinese", "English"], # can also be set to None for automatic language detection
 return_time_stamps=True,
 )

 for r in results:
 print(r.language, r.text, r.time_stamps[0])
```

--------------------------------

### Run Qwen3-ASR-1.7B with vLLM Backend and Forced Aligner

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Launches the ASR demo with the vLLM backend and a forced aligner for timestamps. Configures vLLM-specific initialization parameters.

```bash
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B \
  --backend vllm \
  --cuda-visible-devices 0 \
  --backend-kwargs '{"gpu_memory_utilization":0.7,"max_inference_batch_size":8,"max_new_tokens":2048}' \
  --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' \
  --ip 0.0.0.0 --port 8000
```

--------------------------------

### Launch Qwen-ASR Gradio Demo with vLLM Backend and Forced Aligner

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B

Launches the Gradio demo using the vLLM backend and a forced aligner for timestamps. Configures vLLM and aligner initialization parameters.

```bash
# vLLM backend + Forced Aligner (enable timestamps)
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B \
  --backend vllm \
  --cuda-visible-devices 0 \
  --backend-kwargs '{"gpu_memory_utilization":0.7,"max_inference_batch_size":8,"max_new_tokens":2048}' \
  --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' \
  --ip 0.0.0.0 --port 8000

```

--------------------------------

### Override Transformers Backend Initialization Arguments

Source: https://huggingface.co/Qwen/Qwen3-ASR-1.7B/blob/main/README.md

Example of overriding default Transformers initialization arguments, such as disabling flash attention.

```bash
--backend-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}'
```