### Benchmark Client Configuration Example

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Example of how to configure the benchmark client with custom parameters like number of tasks and log directory.

```python
python3 client_grpc.py \
    --server-addr localhost \
    --model-name spark_tts \
    --num-tasks 2 \
    --mode streaming \
    --log-dir ./log_concurrent_tasks_2_streaming_new
```

--------------------------------

### Inference Logging Example

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/inference_cli.md

Example log output during inference, showing model loading, saving paths, device usage, and progress.

```text
2025-03-12 15:42:30 - INFO - Using model from: pretrained_models/Spark-TTS-0.5B
2025-03-12 15:42:30 - INFO - Saving audio to: example/results
2025-03-12 15:42:31 - INFO - Using CUDA device: cuda:0
2025-03-12 15:42:35 - INFO - Starting inference...
2025-03-12 15:42:45 - INFO - Audio saved at: example/results/20250312154245.wav
```

--------------------------------

### Start Web UI

Source: https://github.com/sparkaudio/spark-tts/blob/main/README.md

Run the 'webui.py' script to start the web interface for Voice Cloning and Voice Creation. Specify the device to use.

```sh
python webui.py --device 0
```

--------------------------------

### Run Inference Demo Script

Source: https://github.com/sparkaudio/spark-tts/blob/main/README.md

Navigate to the 'example' directory and execute the 'infer.sh' script to run the demo.

```sh
cd example
bash infer.sh
```

--------------------------------

### Execute `run.sh` Script Stages

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Automate model conversion and Triton server launch by specifying start and stop stages. Stages 0-3 are used here to prepare models and start the server.

```sh
bash run.sh 0 3
```

--------------------------------

### Benchmark Client with Custom Dataset

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Example of how to specify a custom Hugging Face dataset and split for benchmarking.

```python
python3 client_grpc.py --num-tasks 2 --huggingface-dataset yuekai/seed_tts --split-name wenetspeech4tts --mode [streaming|offline]
```

--------------------------------

### Launch Triton Server with Docker Compose

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Use this command to quickly start the Triton Inference Server using Docker Compose.

```sh
docker compose up
```

--------------------------------

### Generating Control Tokens

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/token_parser.md

This example demonstrates how to use the TokenParser class to generate individual control tokens for gender, pitch, and speed, and then combine them.

```APIDOC
## Generating Control Tokens

### Description
This example demonstrates how to use the TokenParser class to generate individual control tokens for gender, pitch, and speed, and then combine them.

### Method
```python
from sparktts.utils.token_parser import TokenParser

# Generate control tokens for female voice at high pitch and moderate speed
gender_token = TokenParser.gender("female")
pitch_token = TokenParser.mel_level("high")
speed_token = TokenParser.speed_level("moderate")

combined = gender_token + pitch_token + speed_token
print(combined)
# <|gender_0|><|pitch_label_3|><|speed_label_2|>
```

### Parameters
#### TokenParser.gender(gender: str)
- **gender** (str) - Required - Specifies the desired voice gender (e.g., "female", "male").

#### TokenParser.mel_level(level: str)
- **level** (str) - Required - Specifies the desired pitch level (e.g., "low", "medium", "high").

#### TokenParser.speed_level(level: str)
- **level** (str) - Required - Specifies the desired speech speed (e.g., "slow", "moderate", "fast").

### Output
The combined string represents the concatenated control tokens.
```

--------------------------------

### GET /v2/health/ready

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/INDEX.md

Checks the readiness of the Triton inference server.

```APIDOC
## GET /v2/health/ready

### Description
Checks the readiness of the Triton inference server.

### Method
GET

### Endpoint
/v2/health/ready

### Parameters
(Not specified in source)

### Request Example
(Not specified in source)

### Response
#### Success Response (200)
(Not specified in source)

#### Response Example
(Not specified in source)
```

--------------------------------

### GET /v2/models/{model_name}/ready

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/INDEX.md

Checks the readiness of a specific model within the Triton inference server.

```APIDOC
## GET /v2/models/{model_name}/ready

### Description
Checks the readiness of a specific model within the Triton inference server.

### Method
GET

### Endpoint
/v2/models/{model_name}/ready

### Parameters
#### Path Parameters
- **model_name** (string) - Required - The name of the model to check.

### Request Example
(Not specified in source)

### Response
#### Success Response (200)
(Not specified in source)

#### Response Example
(Not specified in source)
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/sparkaudio/spark-tts/blob/main/README.md

Create a new conda environment named 'sparktts' with Python 3.12, activate it, and install project dependencies from requirements.txt. An alternative pip mirror for mainland China is provided.

```sh
conda create -n sparktts -y python=3.12
conda activate sparktts
pip install -r requirements.txt
```

```sh
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com
```

--------------------------------

### GET /v2/models/{model_name}

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/INDEX.md

Retrieves information about a specific model.

```APIDOC
## GET /v2/models/{model_name}

### Description
Retrieves information about a specific model.

### Method
GET

### Endpoint
/v2/models/{model_name}

### Parameters
#### Path Parameters
- **model_name** (string) - Required - The name of the model to retrieve information for.

### Request Example
(Not specified in source)

### Response
#### Success Response (200)
(Not specified in source)

#### Response Example
(Not specified in source)
```

--------------------------------

### Run Benchmark Client (Streaming Mode)

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Initiate a benchmark test against the Triton server using a dataset in streaming mode.

```sh
bash run.sh 4 4 streaming
```

--------------------------------

### Custom Configuration Loading

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/README.md

Illustrates loading a configuration file using `load_config`. Modifying the sample rate is not recommended as the model expects 16000 Hz.

```python
from sparktts.utils.file import load_config

config = load_config("config.yaml")
# Modify as needed
config.sample_rate = 22050  # Not recommended—model expects 16000
```

--------------------------------

### GET /v2

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/INDEX.md

Retrieves basic information or status from the Triton API.

```APIDOC
## GET /v2

### Description
Retrieves basic information or status from the Triton API.

### Method
GET

### Endpoint
/v2

### Parameters
(Not specified in source)

### Request Example
(Not specified in source)

### Response
#### Success Response (200)
(Not specified in source)

#### Response Example
(Not specified in source)
```

--------------------------------

### SpeakerEncoder Get Indices

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/speaker_encoder.md

Extracts speaker token indices directly from a mel-spectrogram.

```APIDOC
## SpeakerEncoder

Extracts speaker token indices from mel-spectrogram.

### `get_indices`

```python
def get_indices(self, mels: torch.Tensor) -> torch.Tensor
```

#### Parameters

- **mels** (torch.Tensor) - Required - Mel-spectrogram tensor

#### Returns

- torch.Tensor - Token indices
```

--------------------------------

### SpeakerEncoder Get Codes From Indices

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/speaker_encoder.md

Converts speaker token indices into their corresponding quantized codes.

```APIDOC
## SpeakerEncoder

Converts token indices to quantized codes.

### `get_codes_from_indices`

```python
def get_codes_from_indices(self, indices: torch.Tensor) -> torch.Tensor
```

#### Parameters

- **indices** (torch.Tensor) - Required - Token indices

#### Returns

- torch.Tensor - Quantized codes
```

--------------------------------

### Initialize BiCodecTokenizer

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec_tokenizer.md

Initializes the BiCodec tokenizer. Ensure the model directory contains the necessary checkpoint and config files. Defaults to GPU if available.

```python
from pathlib import Path
import torch
from sparktts.models.audio_tokenizer import BiCodecTokenizer

device = torch.device("cuda:0")
tokenizer = BiCodecTokenizer(
    model_dir=Path("pretrained_models/Spark-TTS-0.5B"),
    device=device
)
```

--------------------------------

### Initialize WaveGenerator

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/encoder_decoder.md

Instantiate the WaveGenerator module with specified dimensions, upsampling rates, and kernel sizes. This is used for setting up the neural vocoder.

```python
from sparktts.modules.encoder_decoder.wave_generator import WaveGenerator

vocoder = WaveGenerator(
    input_channel=256,
    channels=512,
    rates=[4, 4, 2, 2],
    kernel_sizes=[16, 16, 4, 4],
    d_out=1
)
```

--------------------------------

### Load Audio with Options

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/audio_utils.md

Loads an audio file with optional resampling, normalization, and segmentation. Use this to load audio data into a NumPy array for further processing. Ensure the audio file path is correct and specify desired parameters like sampling rate or normalization.

```python
from sparktts.utils.audio import load_audio
from pathlib import Path

# Load full audio at 16kHz with normalization
audio = load_audio(
    Path("audio.wav"),
    sampling_rate=16000,
    volume_normalize=True
)

# Load 3-second random segment
audio_segment = load_audio(
    Path("audio.wav"),
    sampling_rate=16000,
    segment_duration=3
)
```

--------------------------------

### Clone Spark-TTS Repository

Source: https://github.com/sparkaudio/spark-tts/blob/main/README.md

Clone the Spark-TTS repository to your local machine. Navigate into the cloned directory to proceed with installation.

```sh
git clone https://github.com/SparkAudio/Spark-TTS.git
cd Spark-TTS
```

--------------------------------

### Run Benchmark Client (Offline Mode)

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Initiate a benchmark test against the Triton server using a dataset in offline mode.

```sh
bash run.sh 4 4 offline
```

--------------------------------

### Initialize ResidualFSQ

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/quantization.md

Instantiate the ResidualFSQ module with specified quantization levels, number of quantizers, feature dimension, and channel ordering. Ensure the 'levels' parameter is a list of integers representing the number of levels for each quantizer.

```python
from sparktts.modules.fsq.residual_fsq import ResidualFSQ

fsq = ResidualFSQ(
    levels=[4, 4, 4, 4, 4, 4],
    num_quantizers=1,
    dim=128,
    is_channel_first=True
)
```

--------------------------------

### Basic Triton Client Usage

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/triton_client.md

Use this command to interact with a local Triton server for text-to-speech synthesis. Ensure the server is running and the model is loaded.

```bash
python runtime/triton_trtllm/client_http.py \
    --server-url localhost:8000 \
    --reference-audio example/prompt_audio.wav \
    --reference-text "吃燕窝就选燕之屋" \
    --target-text "身临其境，换新体验。" \
    --model-name spark_tts \
    --output-audio output.wav
```

--------------------------------

### Spark TTS Special Tokens

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/types.md

Examples of special string tokens used for controlling tasks, gender, pitch, speed, and detokenization in Spark TTS.

```python
"<|task_tts|>"              # Task token
"<|gender_0|>"              # Gender control
"<|pitch_label_3|>"         # Pitch control
"<|speed_label_2|>"         # Speed control
"<|bicodec_global_5|>"      # Global token from detokenizer
"<|bicodec_semantic_123|>"  # Semantic token from detokenizer
```

--------------------------------

### Get Server Metadata

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/endpoints.md

Retrieves general information about the Triton Inference Server, including its name, version, and supported extensions. This endpoint does not require any parameters.

```JSON
{
    "name": "tritonserver",
    "version": "2.x.x",
    "extensions": [...] 
}
```

--------------------------------

### SparkTTS Initialization

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/sparktts.md

Initializes the SparkTTS model with configurations and specifies the computation device. Ensure the model directory contains all necessary files.

```APIDOC
## SparkTTS

Main class for text-to-speech synthesis and voice cloning.

### `__init__`

Initializes the SparkTTS model with configurations and device.

#### Parameters

- **model_dir** (Path) - Required - Directory containing model files, config.yaml, and pretrained components (LLM, BiCodec, wav2vec2)
- **device** (torch.device) - Optional - Default: `torch.device("cuda:0")` - Computation device (CPU, CUDA, or MPS for Apple Silicon)

#### Raises

- FileNotFoundError: If config.yaml or required model files not found in model_dir

#### Example

```python
import torch
from pathlib import Path
from cli.SparkTTS import SparkTTS

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = SparkTTS(Path("pretrained_models/Spark-TTS-0.5B"), device=device)
```
```

--------------------------------

### Download Spark-TTS Model via Git LFS

Source: https://github.com/sparkaudio/spark-tts/blob/main/README.md

Download the Spark-TTS-0.5B model using git clone. Ensure git-lfs is installed and initialized before cloning.

```sh
mkdir -p pretrained_models

# Make sure you have git-lfs installed (https://git-lfs.com)
git lfs install

git clone https://huggingface.co/SparkAudio/Spark-TTS-0.5B pretrained_models/Spark-TTS-0.5B
```

--------------------------------

### Decoder Initialization

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/encoder_decoder.md

Initializes the Decoder module with specified dimensions and upsampling ratios. Use this to set up the decoder for audio reconstruction.

```python
def __init__(
    self,
    input_channels: int,
    vocos_dim: int,
    vocos_intermediate_dim: int,
    vocos_num_layers: int,
    out_channels: int,
    sample_ratios: List[int] = [1, 1],
    condition_dim: int = None,
)
```

--------------------------------

### Create and Run Triton Server Docker Container

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Launch a Docker container for the Triton server, enabling GPU access and mounting volumes.

```sh
your_mount_dir=/mnt:/mnt
docker run -it --name "spark-tts-server" --gpus all --net host -v $your_mount_dir --shm-size=2g soar97/triton-spark-tts:25.02
```

--------------------------------

### Spark-TTS Token Format Example

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/README.md

Illustrates the XML-like token format used for model input in Spark-TTS, including task, content, and global token sections.

```plaintext
<|task_tts|>
<|start_content|>text<|end_content|>
<|start_global_token|>...tokens...<|end_global_token|>
```

--------------------------------

### Convert Gender Label to Token

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/token_parser.md

Use this method to convert a gender string ('female' or 'male') into its corresponding special token. For example, 'female' maps to '<|gender_0|>'.

```python
from sparktts.utils.token_parser import TokenParser

token = TokenParser.gender("female")
print(token)  # <|gender_0|>
```

--------------------------------

### CPU Inference CLI

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/inference_cli.md

Execute inference on the CPU by setting the device to -1. Provide the text, device, and save directory.

```bash
python -m cli.inference \
    --text "Test text" \
    --device -1 \
    --save_dir example/results
```

--------------------------------

### ResidualFSQ Initialization

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/quantization.md

Initializes the ResidualFSQ module with specified quantization levels and configuration.

```APIDOC
## ResidualFSQ(`__init__`)

### Description
Initializes the Residual Finite Scalar Quantizer with multiple quantization levels.

### Parameters
#### `levels`
- **Type**: List[int]
- **Required**: Yes
- **Description**: Number of levels per FSQ (e.g., [4, 4, 4, 4])

#### `num_quantizers`
- **Type**: int
- **Required**: No
- **Default**: 1
- **Description**: Number of residual quantizers

#### `dim`
- **Type**: int
- **Required**: No
- **Default**: 128
- **Description**: Feature dimension

#### `is_channel_first`
- **Type**: bool
- **Required**: No
- **Default**: False
- **Description**: Whether input is (B, D, T) vs (B, T, D)

#### `quantize_dropout`
- **Type**: float
- **Required**: No
- **Default**: False
- **Description**: Dropout for regularization

### Example
```python
from sparktts.modules.fsq.residual_fsq import ResidualFSQ

fsq = ResidualFSQ(
    levels=[4, 4, 4, 4, 4, 4],
    num_quantizers=1,
    dim=128,
    is_channel_first=True
)
```
```

--------------------------------

### random_select_audio_segment

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/audio_utils.md

Extracts a random segment of a fixed length from an audio waveform. If the input audio is shorter than the target length, it pads with zeros; otherwise, it randomly selects a starting position.

```APIDOC
## random_select_audio_segment

### Description
Extracts a random segment of fixed length from audio.

### Method Signature
```python
def random_select_audio_segment(audio: np.ndarray, length: int) -> np.ndarray
```

### Parameters
#### Path Parameters
- **audio** (np.ndarray) - Required - Input audio waveform
- **length** (int) - Required - Target segment length in samples

### Returns
- **np.ndarray** - Audio segment of shape (length,)

### Behavior
If input is shorter than target, pads with zeros. Otherwise, randomly selects start position.

### Example
```python
from sparktts.utils.audio import random_select_audio_segment

# Extract 3-second segment from 16kHz audio
segment = random_select_audio_segment(audio, length=16000*3)
```
```

--------------------------------

### detect_speech_boundaries

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/audio_utils.md

Detects the start and end indices of speech segments within an audio waveform using RMS energy. It allows customization of the detection window, energy threshold, and boundary extension margin.

```APIDOC
## detect_speech_boundaries

### Description
Detects start and end indices of speech segments using RMS energy.

### Method Signature
```python
def detect_speech_boundaries(
    wav: np.ndarray,
    sample_rate: int,
    window_duration: float = 0.1,
    energy_threshold: float = 0.01,
    margin_factor: int = 2
) -> Tuple[int, int]
```

### Parameters
#### Path Parameters
- **wav** (np.ndarray) - Required - Audio waveform with values in [-1, 1]
- **sample_rate** (int) - Required - Audio sample rate in Hz

#### Query Parameters
- **window_duration** (float) - Optional - Detection window size in seconds
- **energy_threshold** (float) - Optional - RMS energy threshold for speech
- **margin_factor** (int) - Optional - Factor to extend boundaries (× window_size)

### Returns
- **Tuple[int, int]** - (start_index, end_index) of speech segment in samples

### Raises
- **ValueError**: If audio contains only silence

### Example
```python
from sparktts.utils.audio import detect_speech_boundaries

start, end = detect_speech_boundaries(
    audio,
    sample_rate=16000,
    energy_threshold=0.01
)
speech_segment = audio[start:end]
```
```

--------------------------------

### Build Triton Server Docker Image

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Build the Docker image for the Triton Inference Server from the provided Dockerfile.

```sh
docker build . -f Dockerfile.server -t soar97/triton-spark-tts:25.02
```

--------------------------------

### Get Model Metadata

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/endpoints.md

Fetches detailed metadata for a specific model, including its name, platform, input tensor specifications, and output tensor specifications. Replace `{model_name}` with the desired model's name.

```JSON
{
    "name": "spark_tts",
    "platform": "python_backend",
    "inputs": [
        {
            "name": "reference_wav",
            "datatype": "FP32",
            "dims": [1, -1]
        },
        ...
    ],
    "outputs": [
        {
            "name": "output_audio",
            "datatype": "FP32",
            "dims": [-1]
        }
    ]
}
```

--------------------------------

### Initialize Inference

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/sparktts.md

Loads and initializes the tokenizer, LLM, and BiCodec audio tokenizer. This method is called automatically during initialization and moves the model to the configured device.

```python
def _initialize_inference(self)
```

--------------------------------

### Load Spark TTS Configuration

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/configuration.md

Demonstrates how to load a configuration file using `load_config` and access its parameters. Ensure the configuration file path is correct.

```python
from sparktts.utils.file import load_config
from pathlib import Path

config = load_config(Path("pretrained_models/Spark-TTS-0.5B/config.yaml"))

# Access configuration values
print(config.sample_rate)                      # 16000
print(config.audio_tokenizer.speaker_encoder.out_dim)  # 512
print(config.audio_tokenizer.quantizer.codebook_size)  # 2048

# Iterate over configuration
for key, value in config.items():
    print(f"{key}: {value}")
```

--------------------------------

### Detect Speech Boundaries

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/audio_utils.md

Detects the start and end indices of speech segments within an audio waveform using RMS energy. This function is helpful for isolating spoken parts of an audio file. Adjust `energy_threshold` and `margin_factor` for different audio conditions.

```python
from sparktts.utils.audio import detect_speech_boundaries

start, end = detect_speech_boundaries(
    audio,
    sample_rate=16000,
    energy_threshold=0.01
)
speech_segment = audio[start:end]
```

--------------------------------

### Server Ready Check

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/endpoints.md

Checks if the server is ready to accept requests. Returns an empty JSON object on success.

```APIDOC
## GET /v2/health/ready

### Description
Checks if the server is ready to accept requests.

### Method
GET

### Endpoint
/v2/health/ready

### Response
#### Success Response (200 OK)
An empty JSON object `{}` indicates the server is ready.
```

--------------------------------

### Initialize BiCodec Model

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec.md

Instantiate the BiCodec model by providing all necessary component modules and mel-spectrogram parameters. Ensure all required modules like encoder, decoder, quantizer, speaker_encoder, prenet, and postnet are defined before initialization.

```python
from sparktts.models.bicodec import BiCodec
from sparktts.modules.encoder_decoder.feat_encoder import Encoder
from sparktts.modules.encoder_decoder.feat_decoder import Decoder
from sparktts.modules.encoder_decoder.wave_generator import WaveGenerator
from sparktts.modules.speaker.speaker_encoder import SpeakerEncoder
from sparktts.modules.vq.factorized_vector_quantize import FactorizedVectorQuantize

model = BiCodec(
    mel_params={"sample_rate": 16000, "n_fft": 400, ...},
    encoder=encoder_module,
    decoder=decoder_module,
    quantizer=quantizer_module,
    speaker_encoder=speaker_encoder_module,
    prenet=prenet_module,
    postnet=postnet_module
)
```

--------------------------------

### Check if Server is Ready

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/endpoints.md

Use this endpoint to determine if the server is ready to accept requests. It returns an empty JSON object on success.

```HTTP
GET /v2/health/ready
```

--------------------------------

### Initialize FactorizedVectorQuantize

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/quantization.md

Instantiate the FactorizedVectorQuantize module with specified dimensions, codebook size, and quantization parameters. Ensure input_dim, codebook_size, codebook_dim, and commitment are correctly set for your model.

```python
from sparktts.modules.vq.factorized_vector_quantize import FactorizedVectorQuantize

quantizer = FactorizedVectorQuantize(
    input_dim=256,
    codebook_size=1024,
    codebook_dim=256,
    commitment=0.25,
    decay=0.99
)
```

--------------------------------

### Generate and Combine Control Tokens

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/token_parser.md

Demonstrates how to generate gender, pitch, and speed control tokens using TokenParser and combine them into a single string. This is useful for setting specific voice characteristics.

```python
from sparktts.utils.token_parser import TokenParser

# Generate control tokens for female voice at high pitch and moderate speed
gender_token = TokenParser.gender("female")
pitch_token = TokenParser.mel_level("high")
speed_token = TokenParser.speed_level("moderate")

combined = gender_token + pitch_token + speed_token
print(combined)
# <|gender_0|><|pitch_label_3|><|speed_label_2|>
```

--------------------------------

### FiniteScalarQuantization Initialization

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/quantization.md

Initializes the FiniteScalarQuantization module with specified levels, dimensions, and channel position.

```python
def __init__(
    self,
    levels: List[int],
    dim: int,
    is_channel_first: bool = False,
)
```

--------------------------------

### WaveGenerator Initialization

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/encoder_decoder.md

Initializes the WaveGenerator with specified dimensions, upsampling rates, and kernel sizes.

```APIDOC
## `__init__`

### Description
Initializes the WaveGenerator neural vocoder.

### Parameters
#### Path Parameters
- **input_channel** (int) - Required - Input feature dimension
- **channels** (int) - Required - Base channel dimension
- **rates** (List[int]) - Required - Upsampling rates (e.g., [4, 4, 2, 2] for 64x upsampling)
- **kernel_sizes** (List[int]) - Required - Transpose convolution kernel sizes
- **d_out** (int) - Optional - Output channels (1 for mono audio) - Default: 1

### Request Example
```python
from sparktts.modules.encoder_decoder.wave_generator import WaveGenerator

vocoder = WaveGenerator(
    input_channel=256,
    channels=512,
    rates=[4, 4, 2, 2],
    kernel_sizes=[16, 16, 4, 4],
    d_out=1
)
```
```

--------------------------------

### Initialize Encoder Module

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/encoder_decoder.md

Instantiate the Encoder module with specified dimensions and downsampling ratios. This is used for compressing audio features.

```python
from sparktts.modules.encoder_decoder.feat_encoder import Encoder

encoder = Encoder(
    input_channels=1024,
    vocos_dim=384,
    vocos_intermediate_dim=2048,
    vocos_num_layers=12,
    out_channels=256,
    sample_ratios=[2, 2]
)
```

--------------------------------

### get_args

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/triton_client.md

Parses command-line arguments for the Triton client, including server URL, audio/text references, model name, and output path.

```APIDOC
## Function: `get_args`

Parses command-line arguments for Triton client.

### Returns
`argparse.Namespace` with attributes:

- **server_url** (str) - Optional - Triton server URL (host:port). Default: "localhost:8000"
- **reference_audio** (str) - Optional - Path to reference audio. Default: "../../example/prompt_audio.wav"
- **reference_text** (str) - Optional - Transcript of reference audio. Default: (Chinese text)
- **target_text** (str) - Optional - Target text to synthesize. Default: (Chinese text)
- **model_name** (str) - Optional - Model name: "spark_tts" or "f5_tts". Default: "spark_tts"
- **output_audio** (str) - Optional - Output audio file path. Default: "output.wav"
```

--------------------------------

### Handle Configuration Loading Errors

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/errors.md

Loads configuration from a YAML file, providing default values for sample rate and n_mels if keys are missing or the file is not found. Catches FileNotFoundError and KeyError.

```python
from sparktts.utils.file import load_config
from pathlib import Path

try:
    config = load_config(Path("config.yaml"))
    sr = config.get("sample_rate", 16000)  # Use default if missing
    n_mels = config.get("n_mels", 128)
except (FileNotFoundError, KeyError) as e:
    print(f"Config error: {e}")
    # Use hardcoded defaults
    sr = 16000
    n_mels = 128
```

--------------------------------

### Configure Speaker Encoder

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/configuration.md

Set up the speaker encoder with parameters like input and output dimensions, latent space size, and tokenization details. Adjust FSQ levels and quantizer counts as needed.

```yaml
speaker_encoder:
  input_dim: 100                 # Mel-spectrogram dimension
  out_dim: 512                   # Output embedding dimension
  latent_dim: 128                # Latent dimension before quantization
  token_num: 32                  # Number of speaker tokens
  fsq_levels: [4, 4, 4, 4, 4, 4]  # FSQ quantizer levels
  fsq_num_quantizers: 1          # Number of FSQ quantizers
```

--------------------------------

### BiCodecTokenizer Initialization

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec_tokenizer.md

Initializes the BiCodec tokenizer with model components. Requires a directory containing the model checkpoint and configuration.

```APIDOC
## BiCodecTokenizer `__init__`

### Description
Initializes the BiCodec tokenizer with model components.

### Parameters
#### Path Parameters
- **model_dir** (Path) - Required - Directory containing BiCodec checkpoint and config.yaml
- **device** (torch.device) - Optional - Device for computation (defaults to GPU if available)

### Example
```python
from pathlib import Path
import torch
from sparktts.models.audio_tokenizer import BiCodecTokenizer

device = torch.device("cuda:0")
tokenizer = BiCodecTokenizer(
    model_dir=Path("pretrained_models/Spark-TTS-0.5B"),
    device=device
)
```
```

--------------------------------

### Process Prompt for Controllable Generation

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/sparktts.md

Preprocesses input for controllable voice generation by formatting the text with specified style tokens. Ensure gender, pitch, and speed parameters are valid.

```python
prompt = model.process_prompt_control("female", "high", "moderate", "Hello world")
```

--------------------------------

### Handle Device Selection with Fallback to CPU

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/errors.md

Selects a CUDA device if available, otherwise falls back to CPU. Catches RuntimeError during device initialization.

```python
import torch
from cli.SparkTTS import SparkTTS

try:
    device = torch.device("cuda:0")
    model = SparkTTS("model_dir", device=device)
except RuntimeError as e:
    print(f"GPU error: {e}, falling back to CPU")
    device = torch.device("cpu")
    model = SparkTTS("model_dir", device=device)
```

--------------------------------

### ResidualFSQ Initialization Signature

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/quantization.md

Signature for the ResidualFSQ constructor, detailing parameters for quantization levels, number of quantizers, feature dimension, channel ordering, and quantization dropout.

```python
def __init__(
    self,
    levels: List[int],
    num_quantizers: int = 1,
    dim: int = 128,
    is_channel_first: bool = False,
    quantize_dropout: float = False,
)
```

--------------------------------

### Initialize SparkTTS Model

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/sparktts.md

Initializes the SparkTTS model. Ensure the model directory contains all necessary configuration and pretrained components. The device can be set to 'cuda:0', 'cpu', or 'mps' for Apple Silicon.

```python
import torch
from pathlib import Path
from cli.SparkTTS import SparkTTS

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = SparkTTS(Path("pretrained_models/Spark-TTS-0.5B"), device=device)
```

--------------------------------

### File Utilities

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/README.md

Utilities for file input and output operations, including configuration loading.

```APIDOC
## File Utilities

### Description
Provides functions for handling file operations, particularly for loading configurations and data.

### Functions
- `load_config(path)`: Loads configuration from a specified file path.
- File I/O: Operations for JSONL, CSV, and metadata files.
```

--------------------------------

### Deploy Spark TTS with Triton Server

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/endpoints.md

Launches the Triton Inference Server with the Spark TTS model repository mounted. Ensure the model repository path is correctly specified.

```bash
docker run -d \
  --gpus all \
  -p 8000:8000 \
  -p 8001:8001 \
  -p 8002:8002 \
  -v /path/to/model_repo:/models \
  nvcr.io/nvidia/tritonserver:latest \
  tritonserver --model-repository=/models
```

--------------------------------

### Basic Voice Cloning CLI

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/inference_cli.md

Use this command for basic voice cloning. Provide the target text, device, model directory, prompt text, prompt speech path, and save directory.

```bash
python -m cli.inference \
    --text "你好,这是一个测试。" \
    --device 0 \
    --model_dir pretrained_models/Spark-TTS-0.5B \
    --prompt_text "燕之屋质量好" \
    --prompt_speech_path example/prompt_audio.wav \
    --save_dir example/results
```

--------------------------------

### Perform Inference via Command Line

Source: https://github.com/sparkaudio/spark-tts/blob/main/README.md

Execute inference directly from the command line. Provide the text to synthesize, device, save directory, model directory, and optional prompt text/speech.

```sh
python -m cli.inference \
    --text "text to synthesis." \
    --device 0 \
    --save_dir "path/to/save/audio" \
    --model_dir pretrained_models/Spark-TTS-0.5B \
    --prompt_text "transcript of the prompt audio" \
    --prompt_speech_path "path/to/prompt_audio"
```

--------------------------------

### FactorizedVectorQuantize Methods

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/INDEX.md

Methods for initializing and using the FactorizedVectorQuantize.

```APIDOC
## FactorizedVectorQuantize

### `__init__`

#### Description
Initialize quantizer.

#### Parameters
- **input_dim** (int) - Input dimension.
- **codebook_size** (int) - Size of the codebook.
- **codebook_dim** (int) - Dimension of each codebook entry.
- **...** - Additional parameters may be available.

### `forward`

#### Description
Quantize features.

#### Parameters
- **z** (object) - Input features to quantize.

### `decode_latents`

#### Description
Map quantized features to codebook entries.

#### Parameters
- **z_e** (object) - Quantized features.
```

--------------------------------

### Load YAML Configuration

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/file_utils.md

Loads a YAML configuration file. Supports merging with a base configuration if specified.

```python
from sparktts.utils.file import load_config
from pathlib import Path

config = load_config(Path("pretrained_models/Spark-TTS-0.5B/config.yaml"))
print(config.sample_rate)  # Access as attributes
print(config["sample_rate"])  # Or as dict
```

--------------------------------

### Command Line - Voice Cloning

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/README.md

Execute voice cloning from the command line using the `cli.inference` module. Provide the model directory, reference audio path, and its text.

```bash
python -m cli.inference \
    --text "你好，这是 Spark-TTS。" \
    --model_dir pretrained_models/Spark-TTS-0.5B \
    --prompt_text "参考音频文本" \
    --prompt_speech_path reference.wav \
    --save_dir results/
```

--------------------------------

### Load BiCodec Model from Checkpoint

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec.md

Load a pretrained BiCodec model from a specified directory containing model weights and configuration files. This class method handles the initialization and loading process, returning the model in evaluation mode.

```python
from pathlib import Path
model = BiCodec.load_from_checkpoint(
    Path("pretrained_models/Spark-TTS-0.5B/BiCodec")
)
```

--------------------------------

### Accessing DictConfig Values

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/types.md

Demonstrates how to access configuration values from a DictConfig object using both dot notation and dictionary notation.

```python
config = load_config("config.yaml")
value = config.sample_rate  # Dot notation
value = config["sample_rate"]  # Dict notation
```

--------------------------------

### Custom Feature Extraction with STFT

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/README.md

Demonstrates how to perform Short-Time Fourier Transform (STFT) for custom audio feature extraction using PyTorch. Ensure audio is a 1D tensor.

```python
from sparktts.utils.audio import stft
import torch

audio = torch.randn(16000)
spec = stft(audio, 400, 160, 400, "hann", use_complex=False)
```

--------------------------------

### Process Prompt for Controllable Generation

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/sparktts.md

Preprocesses input for controllable voice generation by formatting style parameters (gender, pitch, speed) and the target text into a prompt string. This is used in conjunction with the `inference` method for style-controlled synthesis.

```APIDOC
### `process_prompt_control`

Preprocesses input for controllable voice generation with style parameters.

#### Parameters

- **gender** (str) - Required - Speaker gender: "female" or "male"
- **pitch** (str) - Required - Pitch level: "very_low", "low", "moderate", "high", "very_high"
- **speed** (str) - Required - Speech rate: "very_low", "low", "moderate", "high", "very_high"
- **text** (str) - Required - Target text for synthesis

#### Returns

str - Formatted input prompt with control tokens

#### Raises

- AssertionError: If any parameter is not in the valid set

#### Example

```python
prompt = model.process_prompt_control("female", "high", "moderate", "Hello world")
```
```

--------------------------------

### prepare_request

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/triton_client.md

Prepares the HTTP request payload for Triton inference, structuring input data for audio waveform, reference text, and target text.

```APIDOC
## Function: `prepare_request`

Prepares HTTP request payload for Triton inference.

### Parameters

- **waveform** (np.ndarray) - Required - Audio waveform (1D array)
- **reference_text** (str) - Required - Transcript of reference audio
- **target_text** (str) - Required - Target synthesis text
- **sample_rate** (int) - Optional - Audio sample rate (Hz). Default: 16000
- **padding_duration** (int) - Optional - Pad audio to nearest multiple of this duration (seconds). Default: None
- **audio_save_dir** (str) - Optional - Directory for saving audio (not used in this function). Default: "./"

### Returns
`Dict[str, Any]` - Triton V2 inference protocol request:

```json
{
    "inputs": [
        {
            "name": "reference_wav",
            "shape": [1, audio_length],
            "datatype": "FP32",
            "data": [float, float, ...]
        },
        {
            "name": "reference_wav_len",
            "shape": [1, 1],
            "datatype": "INT32",
            "data": [length]
        },
        {
            "name": "reference_text",
            "shape": [1, 1],
            "datatype": "BYTES",
            "data": ["reference text"]
        },
        {
            "name": "target_text",
            "shape": [1, 1],
            "datatype": "BYTES",
            "data": ["target text"]
        }
    ]
}
```

### Example
```python
from runtime.triton_trtllm.client_http import prepare_request
import soundfile as sf
import numpy as np

waveform, sr = sf.read("reference.wav")
waveform = np.array(waveform, dtype=np.float32)

request = prepare_request(
    waveform=waveform,
    reference_text="Reference transcript",
    target_text="Text to synthesize",
    sample_rate=16000
)
```
```

--------------------------------

### Custom Audio Processing Pipeline

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/README.md

Shows how to load, normalize volume, and remove silence from audio files using utility functions. The audio is expected to be at a 16000 Hz sample rate.

```python
from sparktts.utils.audio import (
    load_audio,
    audio_volume_normalize,
    remove_silence_on_both_ends
)

wav = load_audio("input.wav", sampling_rate=16000)
wav = audio_volume_normalize(wav, coeff=0.2)
wav = remove_silence_on_both_ends(wav, sample_rate=16000)
```

--------------------------------

### Run Single Utterance Client (Streaming Mode)

Source: https://github.com/sparkaudio/spark-tts/blob/main/runtime/triton_trtllm/README.md

Execute the client script to perform a single inference request in streaming mode using gRPC.

```sh
bash run.sh 5 5 streaming
```

--------------------------------

### Select Device Type

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/types.md

Specify the target device for PyTorch operations. Use 'cuda:0' for NVIDIA GPUs, 'cpu' for the CPU, or 'mps:0' for Apple Silicon.

```python
import torch

device = torch.device("cuda:0")   # GPU device
device = torch.device("cpu")      # CPU device
device = torch.device("mps:0")    # Apple Silicon (macOS)
```

--------------------------------

### Tokenize Audio Batch

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec_tokenizer.md

Tokenizes a batch of preprocessed audio samples. Expects a dictionary containing audio data.

```APIDOC
## `tokenize_batch`

### Description
Tokenizes a batch of preprocessed audio samples.

### Parameters
#### Request Body
- **batch** (Dict[str, Any]) - Required - Dictionary with keys: "wav" (np.ndarray batch), "ref_wav" (reference audio tensor)

### Returns
Tuple[torch.Tensor, torch.Tensor]
- global_tokens: Shape (batch_size, global_dim)
- semantic_tokens: Shape (batch_size, seq_len)

### Example
```python
batch = {
    "wav": audio_array,
    "ref_wav": reference_audio_tensor
}
global_tokens, semantic_tokens = tokenizer.tokenize_batch(batch)
```
```

--------------------------------

### Tokenize Audio Batch

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec_tokenizer.md

Tokenizes a batch of preprocessed audio samples. Requires a dictionary containing 'wav' and optionally 'ref_wav'.

```python
batch = {
    "wav": audio_array,
    "ref_wav": reference_audio_tensor
}
global_tokens, semantic_tokens = tokenizer.tokenize_batch(batch)
```

--------------------------------

### Load BiCodec from Checkpoint

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec.md

Loads a pretrained BiCodec model from a specified checkpoint directory. The directory must contain 'model.safetensors' and 'config.yaml'.

```APIDOC
### `load_from_checkpoint`

Class method to load a pretrained BiCodec model from checkpoint.

#### Parameters
- **model_dir** (Path) - Required - Directory containing model.safetensors and config.yaml

#### Returns
BiCodec - Initialized model in eval mode with weight norm removed

#### Raises
- FileNotFoundError: If model.safetensors or config.yaml not found
- KeyError: If required config keys missing

#### Example
```python
from pathlib import Path
model = BiCodec.load_from_checkpoint(
    Path("pretrained_models/Spark-TTS-0.5B/BiCodec")
)
```
```

--------------------------------

### Initialize SparkTTS with Custom Model Configuration

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/configuration.md

Specify a custom model configuration by providing the path to your custom model directory during SparkTTS initialization. Ensure the custom model directory contains a valid config.yaml file.

```python
model = SparkTTS(Path("custom_models/my-model-config/"), device)
```

--------------------------------

### BiCodec Methods

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/INDEX.md

Methods for initializing and using the BiCodec model for audio processing.

```APIDOC
## BiCodec

### `__init__`

#### Description
Initialize BiCodec model.

#### Parameters
- **mel_params** (object) - Mel spectrogram parameters.
- **encoder** (object) - The encoder model.
- **decoder** (object) - The decoder model.
- **quantizer** (object) - The quantizer model.
- **...** - Additional parameters may be available.

### `forward`

#### Description
Forward pass for training.

#### Parameters
- **batch** (object) - Input batch for training.

### `tokenize`

#### Description
Convert audio to tokens.

#### Parameters
- **batch** (object) - Input batch.

### `detokenize`

#### Description
Reconstruct waveform from tokens.

#### Parameters
- **semantic_tokens** (object) - Semantic tokens.
- **global_tokens** (object) - Global tokens.

### `load_from_checkpoint`

#### Description
Load pretrained BiCodec model from a checkpoint.

#### Parameters
- **model_dir** (string) - Path to the model directory containing the checkpoint.
```

--------------------------------

### init_mel_transformer

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec.md

Initializes the mel-spectrogram transform using configuration parameters from torchaudio.

```APIDOC
## init_mel_transformer

### Description
Initializes mel-spectrogram transform from torchaudio.

### Method Signature
```python
def init_mel_transformer(self, config: Dict[str, Any])
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **config** (Dict[str, Any]) - Required - Mel-spectrogram parameters

**Configuration Keys:**
- sample_rate (int): Audio sample rate
- n_fft (int): FFT size
- win_length (int): Window length
- hop_length (int): Hop length
- mel_fmin (float): Minimum frequency
- mel_fmax (float): Maximum frequency
- num_mels (int): Number of mel bins

### Request Example
```json
{
  "config": {
    "sample_rate": 16000,
    "n_fft": 1024,
    "win_length": 1024,
    "hop_length": 256,
    "mel_fmin": 0.0,
    "mel_fmax": 8000.0,
    "num_mels": 80
  }
}
```

### Response
None (This is a method call, not an API endpoint with a response body)

### Response Example
None
```

--------------------------------

### Common Configuration Keys

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/types.md

Illustrates a typical structure and common keys found within a Spark-TTS configuration dictionary.

```json
{
    "sample_rate": 16000,          # int - Audio sample rate
    "n_fft": 400,                  # int - FFT size
    "hop_length": 160,             # int - STFT hop length
    "n_mels": 128,                 # int - Mel bins
    "mel_fmin": 0,                 # int - Min mel frequency
    "mel_fmax": 8000,              # int - Max mel frequency
    "ref_segment_duration": 3.0,   # float - Reference clip duration
    "volume_normalize": True,      # bool - Normalize volume
    "latent_hop_length": 320,      # int - Latent frame hop
    "audio_tokenizer": {...},      # dict - BiCodec config
    "encoder": {...},              # dict - Encoder config
    "decoder": {...},              # dict - Decoder config
}
```

--------------------------------

### Initialize Mel Spectrogram Transformer

Source: https://github.com/sparkaudio/spark-tts/blob/main/_autodocs/api-reference/bicodec.md

Initializes the mel-spectrogram transform using configuration parameters from torchaudio. Ensure all required configuration keys are provided.

```python
def init_mel_transformer(self, config: Dict[str, Any])
```