### Setup Environment and Data Preparation

Source: https://github.com/qwenlm/qwen3-tts/blob/main/finetuning/README.md

Commands to clone the repository and prepare training data by generating audio codes from raw audio files.

```bash
git clone https://github.com/QwenLM/Qwen3-TTS.git
cd Qwen3-TTS/finetuning
python prepare_data.py \
  --device cuda:0 \
  --tokenizer_model_path Qwen/Qwen3-TTS-Tokenizer-12Hz \
  --input_jsonl train_raw.jsonl \
  --output_jsonl train_with_codes.jsonl
```

--------------------------------

### Install Qwen3-TTS Package

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Commands to install the Qwen3-TTS library, including options for development environments and performance optimizations like FlashAttention 2.

```bash
pip install -U qwen-tts
```

```bash
git clone https://github.com/QwenLM/Qwen3-TTS.git
cd Qwen3-TTS
pip install -e .
pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Encode and decode audio with tokenizer

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Provides examples for encoding various audio sources (URLs, numpy arrays) into discrete codes and decoding them back to waveforms. It supports batch processing and custom dictionary-based inputs.

```python
import io
import requests
import soundfile as sf
import numpy as np
from qwen_tts import Qwen3TTSTokenizer

tokenizer = Qwen3TTSTokenizer.from_pretrained("Qwen/Qwen3-TTS-Tokenizer-12Hz", device_map="cuda:0")

# Encode from URL
enc = tokenizer.encode("https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/tokenizer_demo_1.wav")
wavs, out_sr = tokenizer.decode(enc)

# Batch encode
enc_batch = tokenizer.encode(["url1", "url2"])
wavs_batch, out_sr = tokenizer.decode(enc_batch)
```

--------------------------------

### Load Qwen3-TTS Models

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Demonstrates how to initialize different Qwen3-TTS model variants (CustomVoice, VoiceDesign, Base) using the from_pretrained method with GPU acceleration.

```python
import torch
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)
```

--------------------------------

### Initialize Qwen3TTSTokenizer

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Shows how to load the speech tokenizer from a pre-trained model path. It also demonstrates how to retrieve essential model properties like sample rates and downsample/upsample rates.

```python
from qwen_tts import Qwen3TTSTokenizer

tokenizer = Qwen3TTSTokenizer.from_pretrained(
    "Qwen/Qwen3-TTS-Tokenizer-12Hz",
    device_map="cuda:0",
)

print(f"Model type: {tokenizer.get_model_type()}")
print(f"Input sample rate: {tokenizer.get_input_sample_rate()} Hz")
print(f"Output sample rate: {tokenizer.get_output_sample_rate()} Hz")
print(f"Encode downsample rate: {tokenizer.get_encode_downsample_rate()}")
print(f"Decode upsample rate: {tokenizer.get_decode_upsample_rate()}")
```

--------------------------------

### Encode and Decode Audio with Qwen3TTSTokenizer

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Demonstrates how to initialize the Qwen3TTSTokenizer, encode an audio file from a URL, and decode the resulting tokens back into a waveform saved as a file.

```python
import soundfile as sf
from qwen_tts import Qwen3TTSTokenizer

tokenizer = Qwen3TTSTokenizer.from_pretrained(
    "Qwen/Qwen3-TTS-Tokenizer-12Hz",
    device_map="cuda:0",
)

enc = tokenizer.encode("https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/tokenizer_demo_1.wav")
wavs, sr = tokenizer.decode(enc)
sf.write("decode_output.wav", wavs[0], sr)
```

--------------------------------

### Launch Qwen3-TTS Web UI Demo

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Commands to launch the local web interface for various Qwen3-TTS model variants including CustomVoice, VoiceDesign, and Base models.

```bash
qwen-tts-demo --help
qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --ip 0.0.0.0 --port 8000
qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign --ip 0.0.0.0 --port 8000
qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base --ip 0.0.0.0 --port 8000
```

--------------------------------

### Prepare data for fine-tuning

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Step-by-step process for preparing training data for the base model, including the extraction of audio codes from raw JSONL files.

```bash
python finetuning/prepare_data.py \
  --device cuda:0 \
  --tokenizer_model_path Qwen/Qwen3-TTS-Tokenizer-12Hz \
  --input_jsonl train_raw.jsonl \
  --output_jsonl train_with_codes.jsonl
```

--------------------------------

### Launch Web UI Demo

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Commands to launch the Gradio web interface for different model types. Includes configuration options for device, port, and SSL settings for secure remote access.

```bash
qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --ip 0.0.0.0 --port 8000

qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --device cuda:0 \
  --dtype bfloat16 \
  --ip 127.0.0.1 \
  --port 8080
```

--------------------------------

### Configure HTTPS for Web UI Deployment

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Steps to generate a self-signed SSL certificate and run the Qwen3-TTS demo server with HTTPS enabled to resolve browser microphone permission issues.

```bash
openssl req -x509 -newkey rsa:2048 \
  -keyout key.pem -out cert.pem \
  -days 365 -nodes \
  -subj "/CN=localhost"

qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base \
  --ip 0.0.0.0 --port 8000 \
  --ssl-certfile cert.pem \
  --ssl-keyfile key.pem \
  --no-ssl-verify
```

--------------------------------

### Execute Supervised Fine-Tuning

Source: https://github.com/qwenlm/qwen3-tts/blob/main/finetuning/README.md

Run the SFT process using the prepared JSONL file to train the model on a specific speaker.

```bash
python sft_12hz.py \
  --init_model_path Qwen/Qwen3-TTS-12Hz-1.7B-Base \
  --output_model_path output \
  --train_jsonl train_with_codes.jsonl \
  --batch_size 32 \
  --lr 2e-6 \
  --num_epochs 10 \
  --speaker_name speaker_test
```

--------------------------------

### POST /models/load

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Loads a Qwen3 TTS model and its processor into memory using HuggingFace-style pretrained configuration.

```APIDOC
## POST /models/load

### Description
Initializes the Qwen3TTSModel by loading weights and configuration from a specified model repository.

### Method
POST

### Endpoint
Qwen3TTSModel.from_pretrained(model_id, **kwargs)

### Parameters
#### Path Parameters
- **model_id** (string) - Required - The HuggingFace model path (e.g., 'Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice')

#### Request Body
- **device_map** (string) - Optional - Device to load the model on (e.g., 'cuda:0')
- **dtype** (torch.dtype) - Optional - Precision format (e.g., torch.bfloat16)
- **attn_implementation** (string) - Optional - Attention optimization (e.g., 'flash_attention_2')

### Request Example
model = Qwen3TTSModel.from_pretrained("Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice", device_map="cuda:0", dtype=torch.bfloat16)

### Response
#### Success Response (200)
- **model** (object) - The initialized Qwen3TTSModel instance.
```

--------------------------------

### Configure Advanced Generation Parameters

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Demonstrates how to control audio generation quality using sampling parameters like top-k, top-p, temperature, and repetition penalty. These settings are applicable to both the main model and the sub-talker components.

```python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

wavs, sr = model.generate_custom_voice(
    text="This demonstrates all the sampling parameters available.",
    language="English",
    speaker="Ryan",
    instruct="Clear and professional",
    do_sample=True,
    top_k=50,
    top_p=1.0,
    temperature=0.9,
    repetition_penalty=1.05,
    max_new_tokens=2048,
    subtalker_dosample=True,
    subtalker_top_k=50,
    subtalker_top_p=1.0,
    subtalker_temperature=0.9,
)
sf.write("full_params_output.wav", wavs[0], sr)
```

--------------------------------

### Optimizing Voice Cloning with Reusable Prompts

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Demonstrates creating a reusable voice clone prompt to avoid redundant feature extraction when generating multiple utterances with the same cloned voice.

```python
prompt_items = model.create_voice_clone_prompt(
    ref_audio=ref_audio,
    ref_text=ref_text,
    x_vector_only_mode=False,
)

sentences = ["Welcome to the presentation.", "Let me explain the key findings.", "Thank you for your attention."]

for i, sentence in enumerate(sentences):
    wavs, sr = model.generate_voice_clone(
        text=sentence,
        language="English",
        voice_clone_prompt=prompt_items,
    )
    sf.write(f"cloned_sentence_{i}.wav", wavs[0], sr)
```

--------------------------------

### Generate Custom Voice with Qwen3-TTS

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Demonstrates how to load the custom voice model and perform single or batch text-to-speech synthesis. It supports specifying language, speaker, and emotional instructions.

```python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

# single inference
wavs, sr = model.generate_custom_voice(
    text="其实我真的有发现，我是一个特别善于观察别人情绪的人。",
    language="Chinese",
    speaker="Vivian",
    instruct="用特别愤怒的语气说",
)
sf.write("output_custom_voice.wav", wavs[0], sr)

# batch inference
wavs, sr = model.generate_custom_voice(
    text=[
        "其实我真的有发现，我是一个特别善于观察别人情绪的人。", 
        "She said she would be here by noon."
    ],
    language=["Chinese", "English"],
    speaker=["Vivian", "Ryan"],
    instruct=["", "Very happy."]
)
sf.write("output_custom_voice_1.wav", wavs[0], sr)
sf.write("output_custom_voice_2.wav", wavs[1], sr)
```

--------------------------------

### Generate Voice Design with Qwen3-TTS

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Shows how to use the voice design model to generate audio based on natural language instructions. This allows for fine-grained control over the tone and style of the output speech.

```python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

# single inference
wavs, sr = model.generate_voice_design(
    text="哥哥，你回来啦，人家等了你好久好久了，要抱抱！",
    language="Chinese",
    instruct="体现撒娇稚嫩的萝莉女声，音调偏高且起伏明显，营造出黏人、做作又刻意卖萌的听觉效果。",
)
sf.write("output_voice_design.wav", wavs[0], sr)

# batch inference
wavs, sr = model.generate_voice_design(
    text=[
      "哥哥，你回来啦，人家等了你好久好久了，要抱抱！",
      "It's in the top drawer... wait, it's empty? No way, that's impossible! I'm sure I put it there!"
    ],
    language=["Chinese", "English"],
    instruct=[
      "体现撒娇稚嫩的萝莉女声，音调偏高且起伏明显，营造出黏人、做作又刻意卖萌的听觉效果。",
      "Speak in an incredulous tone, but with a hint of panic beginning to creep into your voice."
    ]
)
sf.write("output_voice_design_1.wav", wavs[0], sr)
sf.write("output_voice_design_2.wav", wavs[1], sr)
```

--------------------------------

### Optimizing Voice Cloning with Reusable Prompts

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Creates a reusable voice clone prompt from reference audio and text. This prompt can then be passed to `generate_voice_clone` for multiple generations, avoiding the recomputation of prompt features and improving efficiency. This is useful for generating multiple sentences with the same cloned voice.

```Python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

prompt_items = model.create_voice_clone_prompt(
    ref_audio=ref_audio,
    ref_text=ref_text,
    x_vector_only_mode=False,
)
wavs, sr = model.generate_voice_clone(
    text=["Sentence A.", "Sentence B."],
    language=["English", "English"],
    voice_clone_prompt=prompt_items,
)
sf.write("output_voice_clone_1.wav", wavs[0], sr)
sf.write("output_voice_clone_2.wav", wavs[1], sr)
```

--------------------------------

### Fine-tune Qwen3-TTS Model

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Executes the supervised fine-tuning script for the Qwen3-TTS model. It requires a base model path, training data in JSONL format, and specific hyperparameters like learning rate and batch size.

```bash
python finetuning/sft_12hz.py \
  --init_model_path Qwen/Qwen3-TTS-12Hz-1.7B-Base \
  --output_model_path output \
  --train_jsonl train_with_codes.jsonl \
  --batch_size 32 \
  --lr 2e-6 \
  --num_epochs 10 \
  --speaker_name my_custom_speaker
```

--------------------------------

### Batch Voice Design Generation

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Demonstrates how to generate multiple audio outputs with distinct personas using text and descriptive instructions. This process outputs audio waveforms and sample rates suitable for saving to disk.

```python
wavs, sr = model.generate_voice_design(
    text=[
        "Welcome to our podcast, everyone!",
        "The experiment results were unexpected.",
    ],
    language=["English", "English"],
    instruct=[
        "Enthusiastic male podcaster voice, mid-30s, clear and engaging.",
        "Thoughtful female scientist voice, calm and analytical.",
    ],
    max_new_tokens=2048,
)
sf.write("voice_design_podcaster.wav", wavs[0], sr)
sf.write("voice_design_scientist.wav", wavs[1], sr)
```

--------------------------------

### Perform Inference with Fine-Tuned Model

Source: https://github.com/qwenlm/qwen3-tts/blob/main/finetuning/README.md

Load the fine-tuned checkpoint and generate speech from text using the Qwen3TTSModel class.

```python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

device = "cuda:0"
tts = Qwen3TTSModel.from_pretrained(
    "output/checkpoint-epoch-2",
    device_map=device,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

wavs, sr = tts.generate_custom_voice(
    text="She said she would be here by noon.",
    speaker="speaker_test",
)
sf.write("output.wav", wavs[0], sr)
```

--------------------------------

### Integrated Voice Design and Clone Workflow

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Combines the Voice Design model to create a custom reference audio, followed by the Base model to create a reusable clone prompt for consistent character synthesis.

```python
# Step 1: Create reference audio using VoiceDesign
ref_wavs, sr = design_model.generate_voice_design(
    text=ref_text,
    language="English",
    instruct=ref_instruct,
)

# Step 2: Create reusable clone prompt
voice_clone_prompt = clone_model.create_voice_clone_prompt(
    ref_audio=(ref_wavs[0], sr),
    ref_text=ref_text,
)
```

--------------------------------

### Voice Design then Clone Workflow

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Synthesizes a reference audio clip with a target persona using the VoiceDesign model, then uses this clip to create a reusable voice clone prompt. This workflow is ideal for maintaining a consistent character voice across multiple generated audio segments.

```Python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

# create a reference audio in the target style using the VoiceDesign model
design_model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

ref_text = "H-hey! You dropped your... uh... calculus notebook? I mean, I think it's yours? Maybe?"
ref_instruct = "Male, 17 years old, tenor range, gaining confidence - deeper breath support now, though vowels still tighten when nervous"
ref_wavs, sr = design_model.generate_voice_design(
    text=ref_text,
    language="English",
    instruct=ref_instruct
)
sf.write("voice_design_reference.wav", ref_wavs[0], sr)

# build a reusable clone prompt from the voice design reference
clone_model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

voice_clone_prompt = clone_model.create_voice_clone_prompt(
    ref_audio=(ref_wavs[0], sr),   # or "voice_design_reference.wav"
    ref_text=ref_text,
)

sentences = [
    "No problem! I actually... kinda finished those already? If you want to compare answers or something...",
    "What? No! I mean yes but not like... I just think you're... your titration technique is really precise!",
]

# reuse it for multiple single calls
wavs, sr = clone_model.generate_voice_clone(
    text=sentences[0],
    language="English",
    voice_clone_prompt=voice_clone_prompt,
)
sf.write("clone_single_1.wav", wavs[0], sr)

wavs, sr = clone_model.generate_voice_clone(
    text=sentences[1],
    language="English",
    voice_clone_prompt=voice_clone_prompt,
)
sf.write("clone_single_2.wav", wavs[0], sr)

# or batch generate in one call
wavs, sr = clone_model.generate_voice_clone(
    text=sentences,
    language=["English", "English"],
    voice_clone_prompt=voice_clone_prompt,
)
for i, w in enumerate(wavs):
    sf.write(f"clone_batch_{i}.wav", w, sr)
```

--------------------------------

### Create Voice Clone Prompt API

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Creates reusable voice clone prompt items from reference audio. This is useful for generating multiple utterances with the same cloned voice, avoiding redundant feature extraction.

```APIDOC
## Create Voice Clone Prompt API

### Description
Creates reusable voice clone prompt items from reference audio. This is useful for generating multiple utterances with the same cloned voice, avoiding redundant feature extraction.

### Method
`create_voice_clone_prompt`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **ref_audio** (string or tuple) - Required - URL, path, or (waveform, sample_rate) tuple of the reference audio file.
- **ref_text** (string) - Required - The reference text corresponding to the `ref_audio`.
- **x_vector_only_mode** (boolean) - Optional - If True, extracts only speaker embedding (x-vector). If False (default), extracts features for ICL mode.

### Request Example
```python
ref_audio = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone.wav"
ref_text = "Okay. Yeah. I resent you. I love you. But you know what? You blew it!"

# Create reusable prompt (extracted once, used many times)
prompt_items = model.create_voice_clone_prompt(
    ref_audio=ref_audio,
    ref_text=ref_text,
    x_vector_only_mode=False,
)

# Generate multiple utterances with the same voice
sentences = [
    "Welcome to the presentation.",
    "Let me explain the key findings.",
    "Thank you for your attention.",
]

for i, sentence in enumerate(sentences):
    wavs, sr = model.generate_voice_clone(
        text=sentence,
        language="English",
        voice_clone_prompt=prompt_items,
    )
    sf.write(f"cloned_sentence_{i}.wav", wavs[0], sr)

# Batch generation with reusable prompt
wavs, sr = model.generate_voice_clone(
    text=sentences,
    language=["English", "English", "English"],
    voice_clone_prompt=prompt_items,
)
for i, w in enumerate(wavs):
    sf.write(f"cloned_batch_{i}.wav", w, sr)
```

### Response
#### Success Response (200)
- **prompt_items** (object) - A reusable object containing extracted voice features.

#### Response Example
(A `prompt_items` object is returned, which can be passed to `generate_voice_clone`.)
```

--------------------------------

### Voice Cloning with ICL and x-vector Modes

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Shows how to clone a voice from reference audio. It supports ICL mode for high-quality synthesis using reference text and x-vector mode for speaker embedding only.

```python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

ref_audio = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone.wav"
ref_text = "Okay. Yeah. I resent you. I love you. But you know what? You blew it!"

# Voice clone with ICL mode
wavs, sr = model.generate_voice_clone(
    text="I am solving the equation: x equals negative b plus or minus square root.",
    language="English",
    ref_audio=ref_audio,
    ref_text=ref_text,
    x_vector_only_mode=False,
)
sf.write("cloned_voice_icl.wav", wavs[0], sr)

# Voice clone with x-vector only mode
wavs, sr = model.generate_voice_clone(
    text="This is a test of the voice cloning system.",
    language="English",
    ref_audio=ref_audio,
    x_vector_only_mode=True,
)
sf.write("cloned_voice_xvec.wav", wavs[0], sr)
```

--------------------------------

### Run Qwen3-TTS Inference Tasks

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Executes the end2end.py script to perform text-to-speech tasks. Supports single sample execution, batch processing, and custom mode tagging for specific task types.

```bash
# Run a single sample with VoiceDesign task
python end2end.py --query-type VoiceDesign

# Batch sample (multiple prompts in one run) with VoiceDesign task:
python end2end.py --query-type VoiceDesign --use-batch-sample

# Run a single sample with Base task in icl mode-tag
python end2end.py --query-type Base --mode-tag icl
```

--------------------------------

### Generate Custom Voice Speech

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Shows how to generate speech using predefined speakers with optional emotional instructions. Supports both single and batch inference modes.

```python
wavs, sr = model.generate_custom_voice(
    text="I can't believe you actually did it! This is amazing!",
    language="English",
    speaker="Ryan",
    instruct="Very excited and happy",
)
sf.write("output_excited.wav", wavs[0], sr)
```

--------------------------------

### Automated Fine-Tuning Shell Script

Source: https://github.com/qwenlm/qwen3-tts/blob/main/finetuning/README.md

A complete bash script that automates the data preparation and fine-tuning steps.

```bash
#!/usr/bin/env bash
set -e

DEVICE="cuda:0"
TOKENIZER_MODEL_PATH="Qwen/Qwen3-TTS-Tokenizer-12Hz"
INIT_MODEL_PATH="Qwen/Qwen3-TTS-12Hz-1.7B-Base"

RAW_JSONL="train_raw.jsonl"
TRAIN_JSONL="train_with_codes.jsonl"
OUTPUT_DIR="output"

python prepare_data.py \
  --device ${DEVICE} \
  --tokenizer_model_path ${TOKENIZER_MODEL_PATH} \
  --input_jsonl ${RAW_JSONL} \
  --output_jsonl ${TRAIN_JSONL}

python sft_12hz.py \
  --init_model_path ${INIT_MODEL_PATH} \
  --output_model_path ${OUTPUT_DIR} \
  --train_jsonl ${TRAIN_JSONL} \
  --batch_size 2 \
  --lr 2e-5 \
  --num_epochs 3 \
  --speaker_name "speaker_1"
```

--------------------------------

### Voice Cloning with Qwen3-TTS

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Clones a voice from a reference audio and text, synthesizing new content in the same voice. Supports local files, URLs, base64 strings, or numpy arrays for reference audio. The `x_vector_only_mode` can be used to only extract speaker embeddings, omitting the need for reference text but potentially reducing quality.

```Python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

ref_audio = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone.wav"
ref_text  = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."

wavs, sr = model.generate_voice_clone(
    text="I am solving the equation: x = [-b ± √(b²-4ac)] / 2a? Nobody can — it's a disaster (◍•͈⌔•͈◍), very sad!",
    language="English",
    ref_audio=ref_audio,
    ref_text=ref_text,
)
sf.write("output_voice_clone.wav", wavs[0], sr)
```

--------------------------------

### Generate consistent character voice lines

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Demonstrates how to generate multiple lines of dialogue using a cloned voice model. It iterates through a list of strings and saves the resulting audio to individual WAV files.

```python
dialogue_lines = [
    "No problem! I actually finished those already.",
    "What? No! I mean yes but not like that!",
    "I just think your titration technique is really precise!",
]

for i, line in enumerate(dialogue_lines):
    wavs, sr = clone_model.generate_voice_clone(
        text=line,
        language="English",
        voice_clone_prompt=voice_clone_prompt,
    )
    sf.write(f"character_line_{i}.wav", wavs[0], sr)
```

--------------------------------

### Perform Offline Inference with vLLM-Omni

Source: https://github.com/qwenlm/qwen3-tts/blob/main/README.md

Commands to execute offline inference for Qwen3-TTS using the vLLM-Omni framework, supporting both single and batch processing.

```bash
python end2end.py --query-type CustomVoice
python end2end.py --query-type CustomVoice --use-batch-sample
```

--------------------------------

### Batch Voice Design with Different Personas

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Generates audio clips with specified personas by providing text, language, and detailed instructions for each utterance. Supports batch processing for multiple audio generations.

```APIDOC
## Batch Voice Design with Different Personas

### Description
Generates audio clips with specified personas by providing text, language, and detailed instructions for each utterance. Supports batch processing for multiple audio generations.

### Method
`generate_voice_design`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **text** (list of strings) - Required - The text content to synthesize for each audio clip.
- **language** (list of strings) - Required - The language of the text for each audio clip.
- **instruct** (list of strings) - Required - Instructions defining the persona for each audio clip (e.g., age, gender, tone, style).
- **max_new_tokens** (int) - Optional - Maximum number of new tokens to generate.

### Request Example
```python
wavs, sr = model.generate_voice_design(
    text=[
        "Welcome to our podcast, everyone!",
        "The experiment results were unexpected.",
    ],
    language=["English", "English"],
    instruct=[
        "Enthusiastic male podcaster voice, mid-30s, clear and engaging.",
        "Thoughtful female scientist voice, calm and analytical.",
    ],
    max_new_tokens=2048,
)
sf.write("voice_design_podcaster.wav", wavs[0], sr)
sf.write("voice_design_scientist.wav", wavs[1], sr)
```

### Response
#### Success Response (200)
- **wavs** (list of numpy arrays) - Synthesized audio waveforms.
- **sr** (int) - Sample rate of the audio waveforms.

#### Response Example
(Audio files 'voice_design_podcaster.wav' and 'voice_design_scientist.wav' are generated.)
```

--------------------------------

### Voice Design then Clone Workflow

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Combines voice design with voice cloning to create a consistent character voice that can be reused across many utterances without re-extracting features. First, a reference audio is designed using `generate_voice_design`, then a reusable clone prompt is created from this designed voice.

```APIDOC
## Voice Design then Clone Workflow

### Description
Combines voice design with voice cloning to create a consistent character voice that can be reused across many utterances without re-extracting features. First, a reference audio is designed using `generate_voice_design`, then a reusable clone prompt is created from this designed voice.

### Method
`generate_voice_design` followed by `create_voice_clone_prompt` and `generate_voice_clone`.

### Parameters
See parameters for `generate_voice_design`, `create_voice_clone_prompt`, and `generate_voice_clone`.

### Request Example
```python
# Step 1: Create reference audio using VoiceDesign
design_model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

ref_text = "Hey! You dropped your calculus notebook? I think it's yours?"
ref_instruct = "Male, 17 years old, tenor range, gaining confidence"

ref_wavs, sr = design_model.generate_voice_design(
    text=ref_text,
    language="English",
    instruct=ref_instruct,
)
sf.write("designed_voice_reference.wav", ref_wavs[0], sr)

# Step 2: Create reusable clone prompt from designed voice
clone_model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

voice_clone_prompt = clone_model.create_voice_clone_prompt(
    ref_audio=(ref_wavs[0], sr), # Using the generated audio directly
    ref_text=ref_text,
)

# Now use voice_clone_prompt with generate_voice_clone for multiple utterances
sentences = ["Hello there!", "How are you today?"]
wavs, sr = clone_model.generate_voice_clone(
    text=sentences,
    language=["English", "English"],
    voice_clone_prompt=voice_clone_prompt,
)
for i, w in enumerate(wavs):
    sf.write(f"designed_cloned_{i}.wav", w, sr)
```

### Response
#### Success Response (200)
- **wavs** (list of numpy arrays) - Synthesized audio waveforms.
- **sr** (int) - Sample rate of the audio waveforms.

#### Response Example
(Audio files like 'designed_voice_reference.wav', 'designed_cloned_0.wav', 'designed_cloned_1.wav' are generated.)
```

--------------------------------

### POST /generate/voice_design

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Generates speech with dynamically designed voices based on natural-language style descriptions.

```APIDOC
## POST /generate/voice_design

### Description
Creates unique speech characteristics based on descriptive text instructions without requiring reference audio.

### Method
POST

### Endpoint
model.generate_voice_design(text, language, instruct)

### Parameters
#### Request Body
- **text** (string) - Required - The text to be synthesized.
- **language** (string) - Required - The target language.
- **instruct** (string) - Required - Natural language description of the desired voice style.

### Request Example
wavs, sr = model.generate_voice_design(text="Hello", language="English", instruct="Deep, raspy, and calm")

### Response
#### Success Response (200)
- **wavs** (list) - Generated audio waveform.
- **sr** (int) - Sample rate.
```

--------------------------------

### Generate Voice Design Speech

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Demonstrates generating speech with unique voice characteristics defined by natural language descriptions rather than predefined speakers.

```python
wavs, sr = model.generate_voice_design(
    text="It's in the top drawer... wait, it's empty? No way!",
    language="English",
    instruct="Speak in an incredulous tone with panic creeping into your voice.",
)
sf.write("voice_design_panic.wav", wavs[0], sr)
```

--------------------------------

### POST /generate/custom_voice

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Generates speech using predefined speaker voices with optional emotional instruction control.

```APIDOC
## POST /generate/custom_voice

### Description
Synthesizes audio from text using a specific speaker profile and emotional instruction.

### Method
POST

### Endpoint
model.generate_custom_voice(text, language, speaker, instruct)

### Parameters
#### Request Body
- **text** (string/list) - Required - The text to be spoken.
- **language** (string/list) - Required - Language code (e.g., 'English').
- **speaker** (string/list) - Required - Predefined speaker name.
- **instruct** (string/list) - Optional - Emotional or stylistic instruction.

### Request Example
wavs, sr = model.generate_custom_voice(text="Hello", language="English", speaker="Ryan", instruct="Friendly")

### Response
#### Success Response (200)
- **wavs** (list) - List of generated audio waveforms.
- **sr** (int) - Sample rate of the generated audio.
```

--------------------------------

### Voice Cloning API

Source: https://context7.com/qwenlm/qwen3-tts/llms.txt

Clones a voice from reference audio to synthesize new content. Supports two modes: ICL mode for higher quality using reference text and audio codes, and x-vector only mode using only speaker embedding.

```APIDOC
## Voice Cloning API

### Description
Clones a voice from reference audio to synthesize new content. Supports two modes: ICL mode for higher quality using reference text and audio codes, and x-vector only mode using only speaker embedding.

### Method
`generate_voice_clone`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **text** (string or list of strings) - Required - The text content to synthesize.
- **language** (string or list of strings) - Required - The language of the text.
- **ref_audio** (string) - Required - URL or path to the reference audio file for voice cloning.
- **ref_text** (string) - Optional - The reference text corresponding to the `ref_audio`. Required for ICL mode.
- **x_vector_only_mode** (boolean) - Optional - If True, uses only speaker embedding (x-vector). If False (default), uses ICL mode if `ref_text` is provided.
- **voice_clone_prompt** (object) - Optional - A pre-computed voice clone prompt object created by `create_voice_clone_prompt`.
- **max_new_tokens** (int) - Optional - Maximum number of new tokens to generate.

### Request Example
```python
# Voice clone with ICL mode (recommended for best quality)
ref_audio = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone.wav"
ref_text = "Okay. Yeah. I resent you. I love you. But you know what? You blew it!"

wavs, sr = model.generate_voice_clone(
    text="I am solving the equation: x equals negative b plus or minus square root.",
    language="English",
    ref_audio=ref_audio,
    ref_text=ref_text,
    x_vector_only_mode=False,  # ICL mode
)
sf.write("cloned_voice_icl.wav", wavs[0], sr)

# Voice clone with x-vector only mode (no ref_text required)
wavs, sr = model.generate_voice_clone(
    text="This is a test of the voice cloning system.",
    language="English",
    ref_audio=ref_audio,
    x_vector_only_mode=True,  # Only speaker embedding
)
sf.write("cloned_voice_xvec.wav", wavs[0], sr)

# Batch voice cloning
wavs, sr = model.generate_voice_clone(
    text=[
        "Good morning, how can I help you?",
        "Thank you for calling, goodbye!",
    ],
    language=["English", "English"],
    ref_audio=ref_audio,
    ref_text=ref_text,
    max_new_tokens=2048,
)
sf.write("clone_batch_0.wav", wavs[0], sr)
sf.write("clone_batch_1.wav", wavs[1], sr)
```

### Response
#### Success Response (200)
- **wavs** (list of numpy arrays) - Synthesized audio waveforms.
- **sr** (int) - Sample rate of the audio waveforms.

#### Response Example
(Audio files like 'cloned_voice_icl.wav', 'cloned_voice_xvec.wav', 'clone_batch_0.wav', 'clone_batch_1.wav' are generated.)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.