### Install Qwen Omni Utilities

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Install the qwen-omni-utils package for handling various audio and visual input types. Ensure 'ffmpeg' is installed on your system.

```bash
pip install qwen-omni-utils -U
```

--------------------------------

### Start vLLM Serve for Qwen3-Omni

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Command to start vLLM serving for the Qwen3-Omni model. Note that vLLM serve currently only supports the thinker model and does not support the `use_audio_in_video` parameter.

```bash
python -m vllm.entrypoints.openai.api_server \
    --model Qwen/Qwen3-Omni-110B-Chat \
    --tensor-parallel-size 4 \
    --trust-remote-code
```

--------------------------------

### Install vLLM and Qwen-Omni Utilities

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Installs the vLLM library for efficient inference and the qwen-omni-utils package. It is recommended to create a new Python environment before installation.

```bash
pip install vllm
pip install qwen-omni-utils -U
```

--------------------------------

### Launch Qwen3 Omni Web Demo with vLLM backend

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Start the Gradio web demo using the vLLM backend for text-only output. This is the default and fastest option.

```bash
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Instruct
```

--------------------------------

### Install Web UI Dependencies

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Install necessary Python packages for the web UI demo. Ensure you have followed the vLLM or Transformers installation guide beforehand.

```bash
pip install gradio==5.44.1 gradio_client==1.12.1 soundfile==0.13.1
```

--------------------------------

### Launch Captioner Web Demo with vLLM backend

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Start the dedicated Captioner Gradio demo using the vLLM backend, which is recommended for optimal performance.

```bash
python web_demo_captioner.py -c Qwen/Qwen3-Omni-30B-A3B-Captioner
```

--------------------------------

### Install model download tools

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Install the necessary command-line tools for downloading models from ModelScope and Hugging Face.

```bash
pip install -U modelscope "huggingface_hub[cli]"
```

--------------------------------

### Image Question Answering Example

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/image_question.ipynb

Provides an example of how to ask a question about an image using the Qwen3 Omni model. It includes displaying the image, running the model, and printing the response. The `run_model` function is assumed to be defined elsewhere.

```python
image_path = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/cookbook/2621.jpg"

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image_path},
            {"type": "text", "text": "What style does this image depict?"},
        ]
    }
]

display(Image(image_path, width=640, height=360))

response, audio = run_model(model=model, messages=messages, processor=processor, return_audio=RETURN_AUDIO, use_audio_in_video=USE_AUDIO_IN_VIDEO)

print(response)
if audio is not None:
    display(Audio(audio, rate=24000))
```

--------------------------------

### Launch Qwen3 Omni Web Demo with public share and custom port

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Start the Gradio web demo with a public share link and specify a custom server port and host address.

```bash
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Instruct \
    --share --server-port 7860 --server-name 0.0.0.0
```

--------------------------------

### Launch Web Demo inside Docker container

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

After starting the Docker container, use this command inside the container to launch the web demo, ensuring it's accessible externally.

```bash
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Instruct \
    --server-port 80 --server-name 0.0.0.0
```

--------------------------------

### Prepare Audio Captioning Input

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/audio_caption.ipynb

Sets up the audio file path and constructs the messages payload for the model, including the audio source and a text prompt. This example uses a direct URL for the audio file.

```python
audio_path = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/cookbook/caption1.mp3"

messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": audio_path},
            {"type": "text", "text": "Give the detailed description of the audio."},
        ]
    }
]
```

--------------------------------

### Serve Qwen3-Omni-30B-A3B-Thinking on Multi-GPU

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Serve the Thinking model across multiple GPUs, increasing the maximum model length for longer sequences. Example shown for 4 GPUs.

```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Thinking --port 8901 --host 127.0.0.1 --dtype bfloat16 --max-model-len 65536 --allowed-local-media-path / -tp 4
```

--------------------------------

### Audio Captioning with Second Audio File

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/audio_caption.ipynb

This example demonstrates generating a detailed description for a different audio file. It follows the same pattern of preparing messages and running the model.

```python
audio_path = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/cookbook/caption3.mp3"

messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": audio_path},
            {"type": "text", "text": "Give a thorough description of the audio."},
        ]
    }
]

display(Audio(librosa.load(audioread.ffdec.FFmpegAudioFile(audio_path), sr=16000)[0], rate=16000))

response, audio = run_model(model=model, messages=messages, processor=processor, return_audio=RETURN_AUDIO, use_audio_in_video=USE_AUDIO_IN_VIDEO)

print(response)
if audio is not None:
    display(Audio(audio, rate=24000))
```

--------------------------------

### Launch Captioner Web Demo with public share and custom port

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Start the Captioner Gradio demo, making it publicly accessible via a share link and specifying a custom server port and host.

```bash
python web_demo_captioner.py -c Qwen/Qwen3-Omni-30B-A3B-Captioner \
    --share --server-port 7860 --server-name 0.0.0.0
```

--------------------------------

### Single-Turn Multimodal Inference with Qwen3-Omni

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Demonstrates how to perform a single-turn inference using Qwen3-Omni with text, audio, and image inputs. Ensure `soundfile` is installed for audio output.

```python
import soundfile as sf

from transformers import Qwen3OmniMoeForConditionalGeneration, Qwen3OmniMoeProcessor
from qwen_omni_utils import process_mm_info

MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Instruct"
# MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Thinking"

model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
    MODEL_PATH,
    dtype="auto",
    device_map="auto",
    attn_implementation="flash_attention_2",
)

processor = Qwen3OmniMoeProcessor.from_pretrained(MODEL_PATH)

conversation = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cars.jpg"},
            {"type": "audio", "audio": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cough.wav"},
            {"type": "text", "text": "What can you see and hear? Answer in one short sentence."}
        ],
    },
]

# Set whether to use audio in video
USE_AUDIO_IN_VIDEO = True

# Preparation for inference
text = processor.apply_chat_template(conversation, add_generation_prompt=True, tokenize=False)
audios, images, videos = process_mm_info(conversation, use_audio_in_video=USE_AUDIO_IN_VIDEO)
inputs = processor(text=text, 
                   audio=audios, 
                   images=images, 
                   videos=videos, 
                   return_tensors="pt", 
                   padding=True, 
                   use_audio_in_video=USE_AUDIO_IN_VIDEO)
inputs = inputs.to(model.device).to(model.dtype)

# Inference: Generation of the output text and audio
text_ids, audio = model.generate(**inputs, 
                                 speaker="Ethan", 
                                 thinker_return_dict_in_generate=True,
                                 use_audio_in_video=USE_AUDIO_IN_VIDEO)

text = processor.batch_decode(text_ids.sequences[:, inputs["input_ids"].shape[1] :],
                              skip_special_tokens=True,
                              clean_up_tokenization_spaces=False)
print(text)
if audio is not None:
    sf.write(
        "output.wav",
        audio.reshape(-1).detach().cpu().numpy(),
        samplerate=24000,
    )

```

--------------------------------

### Start Web Demo Inside Docker Container

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Command to run the web demo service within the Qwen3 Omni Docker container. It's crucial to specify `0.0.0.0` for the server name to ensure proper port forwarding from the host.

```bash
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Instruct --server-port 80 --server-name 0.0.0.0
```

--------------------------------

### Example Input Sequence for Multimodal Data

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Illustrates how to structure input messages for multimodal data, including audio, image, and video, followed by a text prompt. Ensure the text prompt comes after multimodal data unless specified otherwise by the benchmark.

```python
messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": "/path/to/audio.wav"},
            {"type": "image", "image": "/path/to/image.png"},
            {"type": "video", "video": "/path/to/video.mp4"},
            {"type": "text", "text": "Describe the audio, image and video."},
        ],
    },
]
```

--------------------------------

### Analyze MP3 Audio with Qwen3-Omni

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/sound_analysis.ipynb

Analyzes an MP3 audio file using the Qwen3-Omni model. This example demonstrates sending an audio file and a question about its content and context. The original audio is displayed before the model's response.

```python
audio_path = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/cookbook/sound2.mp3"

messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": audio_path},
            {"type": "text", "text": "What is this sound? In what kind of situation might it occur?"},
        ]
    }
]

display(Audio(librosa.load(audioread.ffdec.FFmpegAudioFile(audio_path), sr=16000)[0], rate=16000))

response, audio = run_model(model=model, messages=messages, processor=processor, return_audio=RETURN_AUDIO, use_audio_in_video=USE_AUDIO_IN_VIDEO)

print(response)
if audio is not None:
    display(Audio(audio, rate=24000))
```

--------------------------------

### Batch Inference with Qwen3-Omni using vLLM

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

This example demonstrates efficient batch inference with Qwen3-Omni using vLLM. It includes a helper function `build_input` to format multiple conversations for batch processing. This is useful for handling large datasets or benchmarking.

```python
import os
import torch

from vllm import LLM, SamplingParams
from transformers import Qwen3OmniMoeProcessor
from qwen_omni_utils import process_mm_info

def build_input(processor, messages, use_audio_in_video):
    text = processor.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )
    audios, images, videos = process_mm_info(messages, use_audio_in_video=use_audio_in_video)

    inputs = {
        'prompt': text,
        'multi_modal_data': {},
        "mm_processor_kwargs": {
            "use_audio_in_video": use_audio_in_video,
        },
    }

    if images is not None:
        inputs['multi_modal_data']['image'] = images
    if videos is not None:
        inputs['multi_modal_data']['video'] = videos
    if audios is not None:
        inputs['multi_modal_data']['audio'] = audios
    
    return inputs

if __name__ == '__main__':
    MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Instruct"
    # MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Thinking"

    llm = LLM(
            model=MODEL_PATH, trust_remote_code=True, gpu_memory_utilization=0.95,
            tensor_parallel_size=torch.cuda.device_count(),
            limit_mm_per_prompt={'image': 3, 'video': 3, 'audio': 3},
            max_num_seqs=8,
            max_model_len=32768,
            seed=1234,
    )

    sampling_params = SamplingParams(
        temperature=0.6,
        top_p=0.95,
        top_k=20,
        max_tokens=16384,
    )

    processor = Qwen3OmniMoeProcessor.from_pretrained(MODEL_PATH)

    # Conversation with image only
    conversation1 = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cars.jpg"},
                {"type": "text", "text": "What can you see in this image? Answer in one sentence."},
            ]
        }
    ]

    # Conversation with audio only
    conversation2 = [
        {
            "role": "user",
            "content": [

```

--------------------------------

### Run Qwen3 Omni Docker Container

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Launches the Qwen3 Omni Docker container with GPU access, volume mounts for workspace and data, and port forwarding. Ensure NVIDIA Container Toolkit is installed and replace `/path/to/your/workspace` with your actual directory.

```bash
LOCAL_WORKDIR=/path/to/your/workspace
HOST_PORT=8901
CONTAINER_PORT=80
docker run --gpus all --name qwen3-omni \
    -v /var/run/docker.sock:/var/run/docker.sock -p $HOST_PORT:$CONTAINER_PORT \
    --mount type=bind,source=$LOCAL_WORKDIR,target=/data/shared/Qwen3-Omni \
    --shm-size=4gb \
    -it qwenllm/qwen3-omni:3-cu124
```

--------------------------------

### Run Qwen3-Omni-30B-A3B-Instruct Web Demo

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Launch the web demo for Qwen3-Omni-30B-A3B-Instruct. Choose between vLLM or Transformers backend, with options for audio generation and FlashAttention.

```bash
# For Qwen3-Omni-30B-A3B-Instruct with vLLM backend
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Instruct
```

```bash
# For Qwen3-Omni-30B-A3B-Instruct with Transformers backend
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Instruct --use-transformers --generate-audio
```

```bash
# For Qwen3-Omni-30B-A3B-Instruct with Transformers backend and FlashAttention support
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Instruct --use-transformers --generate-audio --flash-attn2
```

--------------------------------

### Translate English Speech to Chinese

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/speech_translation.ipynb

Translates English speech to Chinese text. Similar to the previous example, this involves loading an audio file and using a text prompt to guide the translation.

```python
audio_path = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/cookbook/asr_en.wav"

messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": audio_path},
            {"type": "text", "text": "Listen to the provided English speech and produce a translation in Chinese text."},
        ]
    }
]

display(Audio(librosa.load(audioread.ffdec.FFmpegAudioFile(audio_path), sr=16000)[0], rate=16000))

response, audio = run_model(model=model, messages=messages, processor=processor, return_audio=RETURN_AUDIO, use_audio_in_video=USE_AUDIO_IN_VIDEO)

print(response)
if audio is not None:
    display(Audio(audio, rate=24000))
```

--------------------------------

### Serve Qwen3-Omni-30B-A3B-Instruct on Single GPU

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Use this command to serve the Instruct model on a single GPU. Ensure the model path and port are correctly specified.

```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8901 --host 127.0.0.1 --dtype bfloat16 --max-model-len 32768 --allowed-local-media-path / -tp 1
```

--------------------------------

### Qwen3-Omni Configuration and Initialization

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/image_math.ipynb

Sets up essential configurations and initializes the Qwen3-Omni model and processor. This includes defining the model path and choosing between Transformers and vLLM backends.

```python
import librosa
import audioread

from IPython.display import Video
from IPython.display import Image
from IPython.display import Audio

MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Thinking"

USE_TRANSFORMERS = False
TRANSFORMERS_USE_FLASH_ATTN2 = True

model, processor = _load_model_processor()

USE_AUDIO_IN_VIDEO = True
RETURN_AUDIO = False

```

--------------------------------

### Install Hugging Face Transformers and Accelerate

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Install or upgrade Hugging Face Transformers to version 5.2.0 or later for optimal performance and accuracy. Also install the 'accelerate' library.

```bash
# If you already have transformers installed, please uninstall it first, or create a new Python environment
# pip uninstall transformers
pip install "transformers>=5.2.0"
pip install accelerate
```

--------------------------------

### Model Response Example

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/audio_visual_dialogue.ipynb

This is an example of a conversational response from the model, indicating it recognized an object and is asking a follow-up question.

```text
Oh, that's a guitar you're drawing! It looks pretty cool. What kind of music do you play?
```

--------------------------------

### Launch Qwen3 Omni Web Demo with Transformers backend and audio

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Run the Gradio web demo with the Transformers backend, enabling audio generation and FlashAttention 2 for improved performance.

```bash
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Instruct \
    --use-transformers --generate-audio --flash-attn2
```

--------------------------------

### Install FlashAttention 2

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Install FlashAttention 2 to reduce GPU memory usage when running with Hugging Face Transformers. This is not needed if using vLLM. Ensure your hardware is compatible and the model is loaded in torch.float16 or torch.bfloat16.

```bash
pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Run Qwen3-Omni-30B-A3B-Captioner Web Demo

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Launch the web demo for Qwen3-Omni-30B-A3B-Captioner. Options include vLLM backend, Transformers backend, and FlashAttention support.

```bash
# For Qwen3-Omni-30B-A3B-Captioner with vLLM backend
python web_demo_captioner.py -c Qwen/Qwen3-Omni-30B-A3B-Captioner
```

```bash
# For Qwen3-Omni-30B-A3B-Captioner with Transformers backend
python web_demo_captioner.py -c Qwen/Qwen3-Omni-30B-A3B-Captioner --use-transformers
```

```bash
# For Qwen3-Omni-30B-A3B-Captioner with Transformers backend and FlashAttention support
python web_demo_captioner.py -c Qwen/Qwen3-Omni-30B-A3B-Captioner --use-transformers --flash-attn2
```

--------------------------------

### Serve Qwen3-Omni-30B-A3B-Instruct on Multi-GPU

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Deploy the Instruct model across multiple GPUs (e.g., 4 GPUs) by adjusting the tensor parallelism (-tp) flag. This allows for larger context lengths.

```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8901 --host 127.0.0.1 --dtype bfloat16 --max-model-len 65536 --allowed-local-media-path / -tp 4
```

--------------------------------

### Load Model and Processor

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/video_description.ipynb

Loads the model and processor for video processing. Ensure necessary packages like ipywidgets are installed.

```python
MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Thinking"

USE_TRANSFORMERS = False
TRANSFORMERS_USE_FLASH_ATTN2 = True

model, processor = _load_model_processor()

USE_AUDIO_IN_VIDEO = False
RETURN_AUDIO = False
```

--------------------------------

### Run Qwen3-Omni-30B-A3B-Thinking Web Demo

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Launch the web demo for Qwen3-Omni-30B-A3B-Thinking. Supports vLLM backend, Transformers backend, and FlashAttention.

```bash
# For Qwen3-Omni-30B-A3B-Thinking with vLLM backend
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Thinking
```

```bash
# For Qwen3-Omni-30B-A3B-Thinking with Transformers backend
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Thinking --use-transformers
```

```bash
# For Qwen3-Omni-30B-A3B-Thinking with Transformers backend and FlashAttention support
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Thinking --use-transformers --flash-attn2
```

--------------------------------

### Load Model and Processor

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/audio_visual_question.ipynb

Loads the Qwen3-Omni model and its associated processor. Ensure necessary packages like `ipywidgets` are installed for rich output.

```python
USE_TRANSFORMERS = False
TRANSFORMERS_USE_FLASH_ATTN2 = True

model, processor = _load_model_processor()

USE_AUDIO_IN_VIDEO = True
RETURN_AUDIO = False
```

--------------------------------

### Manage Docker container lifecycle

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Commands to start a stopped Docker container and to enter an active container's shell for interactive use.

```bash
docker start qwen3-omni
docker exec -it qwen3-omni bash
```

--------------------------------

### Launch Qwen3 Omni Web Demo with Thinking model

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Initiate the Gradio web demo using the 'Thinking' model variant with the vLLM backend.

```bash
python web_demo.py -c Qwen/Qwen3-Omni-30B-A3B-Thinking
```

--------------------------------

### Import necessary libraries and set model path

Source: https://github.com/qwenlm/qwen3-omni/blob/main/cookbooks/ocr.ipynb

Imports libraries for audio processing, display, and sets the model path for Qwen3 Omni.

```python
import librosa
import audioread

from IPython.display import Video
from IPython.display import Image
from IPython.display import Audio

MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Instruct"
```

--------------------------------

### French Speech to English Text Translation

Source: https://context7.com/qwenlm/qwen3-omni/llms.txt

Translate spoken French to English text using the Qwen3-Omni model. This example demonstrates handling French audio input.

```python
messages_fr2en = [{"role": "user", "content": [
    {"type": "audio", "audio": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/cookbook/asr_fr.wav"},
    {"type": "text",  "text":  "Listen to the provided French speech and produce a translation in English text."},
]}]
text = processor.apply_chat_template(messages_fr2en, tokenize=False, add_generation_prompt=True)
audios, _, _ = process_mm_info(messages_fr2en, use_audio_in_video=False)
inp = {"prompt": text, "multi_modal_data": {"audio": audios},
       "mm_processor_kwargs": {"use_audio_in_video": False}}
out = llm.generate([inp], sampling_params)
print(out[0].outputs[0].text)
```

--------------------------------

### Download Qwen3-Omni Models with ModelScope

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Use ModelScope to download Qwen3-Omni model weights to a local directory. This is recommended for users in Mainland China. Ensure you have the 'modelscope' library installed.

```bash
pip install -U modelscope
modelscope download --model Qwen/Qwen3-Omni-30B-A3B-Instruct --local_dir ./Qwen3-Omni-30B-A3B-Instruct
modelscope download --model Qwen/Qwen3-Omni-30B-A3B-Thinking --local_dir ./Qwen3-Omni-30B-A3B-Thinking
modelscope download --model Qwen/Qwen3-Omni-30B-A3B-Captioner --local_dir ./Qwen3-Omni-30B-A3B-Captioner
```

--------------------------------

### Chat API Request with Multimodal Inputs

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Example cURL command to interact with the chat API, demonstrating how to send image, audio, and text inputs simultaneously. Ensure the server is running and accessible.

```bash
curl http://localhost:8901/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cars.jpg"}},
        {"type": "audio_url", "audio_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cough.wav"}},
        {"type": "text", "text": "What can you see and hear? Answer in one sentence."}
    ]}
    ] 
    }'
```

--------------------------------

### Batch Inference with Qwen3-Omni

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Shows how to perform batch inference with Qwen3-Omni, supporting mixed media types like text, images, and audio. This example prepares multiple conversations for batched processing.

```python
from transformers import Qwen3OmniMoeForConditionalGeneration, Qwen3OmniMoeProcessor
from qwen_omni_utils import process_mm_info

MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Instruct"
# MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Thinking"

model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
    MODEL_PATH,
    dtype="auto",
    device_map="auto",
    attn_implementation="flash_attention_2",
)
model.disable_talker()

processor = Qwen3OmniMoeProcessor.from_pretrained(MODEL_PATH)

# Conversation with image only
conversation1 = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cars.jpg"},
            {"type": "text", "text": "What can you see in this image? Answer in one sentence."},
        ]
    }
]

# Conversation with audio only
conversation2 = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cough.wav"},
            {"type": "text", "text": "What can you hear in this audio?"},
        ]
    }
]

# Conversation with pure text and system prompt
conversation3 = [
    {
        "role": "system",
        "content": [
            {"type": "text", "text": "You are Qwen-Omni."}
        ],
    },
    {
        "role": "user",
        "content": "Who are you?"
    }
]

# Conversation with mixed media
conversation4 = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cars.jpg"},
            {"type": "audio", "audio": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cough.wav"},
            {"type": "text", "text": "What can you see and hear? Answer in one sentence."}
        ],
    }
]

# Combine messages for batch processing
conversations = [conversation1, conversation2, conversation3, conversation4]

# Set whether to use audio in video
USE_AUDIO_IN_VIDEO = True


```

--------------------------------

### Accessing the Web UI

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

After running the demo commands, a local URL will be provided in the terminal. Paste this URL into your web browser to access the UI. Adjust the address if running on a server or in a Docker container.

```bash
Running on local: http://127.0.0.1:8901/
```

--------------------------------

### Serve Qwen3-Omni-30B-A3B-Thinking on Single GPU

Source: https://github.com/qwenlm/qwen3-omni/blob/main/README.md

Command to serve the Thinking model variant on a single GPU. Adjust parameters as needed for your environment.

```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Thinking --port 8901 --host 127.0.0.1 --dtype bfloat16 --max-model-len 32768 --allowed-local-media-path / -tp 1
```