### Install Kimi-Audio via Pip

Source: https://github.com/moonshotai/kimi-audio/blob/master/README.md

Install the Kimi-Audio library directly using pip, including PyTorch.

```bash
pip install torch
pip install git+https://github.com/MoonshotAI/Kimi-Audio.git
```

--------------------------------

### Install NeuralODE

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/errors.md

Provides commands to install the NeuralODE package using pip. It also shows how to install Kimi-Audio with development dependencies.

```bash
pip install neural-ode
# or use with dependencies
pip install git+https://github.com/MoonshotAI/Kimi-Audio.git[dev]
```

--------------------------------

### Clone Repository and Install Dependencies

Source: https://github.com/moonshotai/kimi-audio/blob/master/README.md

Clone the Kimi-Audio repository and install its dependencies using pip. Ensure to update git submodules.

```bash
git clone https://github.com/MoonshotAI/Kimi-Audio.git
cd Kimi-Audio
git submodule update --init --recursive
pip install -r requirements.txt
```

--------------------------------

### Install Kimi-Audio Dependencies

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/configuration.md

Install the necessary Python packages for Kimi-Audio. Ensure you have torch, torchaudio, and transformers version 4.30 or higher.

```bash
pip install torch torchaudio transformers>=4.30 diffusers soundfile librosa
pip install git+https://github.com/MoonshotAI/Kimi-Audio.git
```

--------------------------------

### ASR Example: Audio to Text Generation

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAudio.md

This example demonstrates how to use the generate method for Automatic Speech Recognition (ASR). Provide an audio file path in the messages and set output_type to 'text'.

```python
import soundfile as sf

# ASR example: Audio to text
messages_asr = [
    {"role": "user", "message_type": "text", "content": "Transcribe the audio:"},
    {"role": "user", "message_type": "audio", "content": "audio.wav"}
]

wav, text = model.generate(
    messages_asr,
    output_type="text",
    text_temperature=0.0,
    max_new_tokens=500
)
print("Transcription:", text)
```

--------------------------------

### Audio-to-Audio Conversation with KimiAudio

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/INDEX.md

This snippet demonstrates an audio-to-audio conversation. Initialize KimiAudio with `load_detokenizer=True` and specify `output_type="both"` to get both audio and text.

```python
model = KimiAudio(model_path="moonshotai/Kimi-Audio-7B-Instruct", load_detokenizer=True)

messages = [
    {"role": "user", "message_type": "audio", "content": "question.wav"}
]

wav, text = model.generate(
    messages,
    output_type="both",
    audio_temperature=0.8,
    audio_top_k=10
)

import soundfile as sf
sf.write("response.wav", wav.cpu().numpy().flatten(), 24000)
```

--------------------------------

### Streaming Detokenization Example

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/PrefixStreamingFlowMatchingDetokenizer.md

Demonstrates how to use `detokenize_streaming` for generating waveform chunks and concatenating them. Ensure the detokenizer is initialized and prefilled before use.

```python
import torch

# Assume detokenizer is initialized and prefilled
semantic_tokens = torch.tensor([[100, 101, 102, ...]])  # [1, 30]

# Generate first chunk
wav1 = detokenizer.detokenize_streaming(
    semantic_tokens[:, :30],
    is_final=False,
    upsample_factor=4
)

# Generate final chunk
wav2 = detokenizer.detokenize_streaming(
    semantic_tokens[:, 30:],
    is_final=True,
    upsample_factor=4
)

# Concatenate
full_wav = torch.cat([wav1, wav2], dim=-1)
```

--------------------------------

### Download Pretrained Kimi-Audio Model

Source: https://github.com/moonshotai/kimi-audio/blob/master/finetune_codes/README.md

Download the pretrained Kimi-Audio model and save it to the specified output directory. This is the first step before finetuning.

```bash
CUDA_VISIBLE_DEVICES=0 python -m finetune_codes.model --model_name "moonshotai/Kimi-Audio-7B" --output_dir "output/pretrained_hf"
```

--------------------------------

### NeuralODEImportError Trigger

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/errors.md

This code explicitly raises an ImportError if the NeuralODE package is not installed. It prompts the user to install the package.

```python
raise ImportError("NeuralODE is not installed, please install it first.")
```

--------------------------------

### Finetune Kimi-Audio Model

Source: https://github.com/moonshotai/kimi-audio/blob/master/finetune_codes/README.md

Execute the finetuning process using a bash script. This command requires the path to the pretrained model and the preprocessed data file.

```bash
bash finetune_codes/finetune_ds.sh     --model_path "output/pretrained_hf"     --data "finetune_codes/demo_data/audio_understanding/data_with_semantic_codes.jsonl"
```

--------------------------------

### Initialize KimiAPromptManager

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAPromptManager.md

Instantiate the KimiAPromptManager with model path and token offsets. Ensure the model path contains necessary tokenizer and encoder configurations.

```python
from kimia_infer.api.prompt_manager import KimiAPromptManager

prompt_mgr = KimiAPromptManager(
    model_path="path/to/model",
    kimia_token_offset=10000,
    kimia_text_audiodelaytokens=4
)
```

--------------------------------

### Initialize and Extend KimiAContent

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAContent.md

Demonstrates initializing a KimiAContent object and extending its audio and text token sequences.

```python
from kimia_infer.utils.data import KimiAContent

content = KimiAContent()
content.audio_extend([100, 101, 102])
content.text_extend([200, 201, 202])
assert content.is_valid()
```

--------------------------------

### Initialize KimiAudio with Detokenizer

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAudio.md

Use this snippet to load the KimiAudio model when both audio and text generation are required. Ensure the model path is correct.

```python
from kimia_infer.api.kimia import KimiAudio

# Load model with detokenizer for audio+text generation
model = KimiAudio(
    model_path="moonshotai/Kimi-Audio-7B-Instruct",
    load_detokenizer=True
)
```

--------------------------------

### to_dtype

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/BigVGANWrapper.md

Converts the vocoder model to a specified PyTorch data type (dtype). This is useful for managing memory usage, for example, by converting to bfloat16.

```APIDOC
## to_dtype(dtype: torch.dtype)

### Description
Converts the vocoder model to a specified PyTorch data type.

### Method
`to_dtype`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **dtype** (torch.dtype) - Required - Target dtype (e.g., torch.float32, torch.bfloat16).

### Request Example
```python
# Convert to bfloat16 for memory efficiency
wrapper.to_dtype(torch.bfloat16)
```

### Response
#### Success Response (None)
Returns None.

#### Response Example
None
```

--------------------------------

### PrefixStreamingFlowMatchingDetokenizer.from_pretrained

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/PrefixStreamingFlowMatchingDetokenizer.md

Loads pretrained vocoder and flow-matching models and creates an instance of the detokenizer. This method simplifies the setup by handling model loading from specified paths.

```APIDOC
## PrefixStreamingFlowMatchingDetokenizer.from_pretrained(vocoder_config: str, vocoder_ckpt: str, fm_config: str, fm_ckpt: str, device: torch.device, look_ahead_tokens: int = 0, max_prompt_chunk: int = 2, max_kv_cache_tokens: int = 900, use_cfg: bool = False, use_cfg_rescale: bool = True, cfg_init: float = 1.5, cfg_scale: float = 7.5, cfg_schedule: str = "linear") -> PrefixStreamingFlowMatchingDetokenizer

### Description
Load pretrained vocoder and flow-matching models and create detokenizer.

### Parameters
#### Path Parameters
- **vocoder_config** (str) - Required - Path to BigVGAN config.json file.
- **vocoder_ckpt** (str) - Required - Path to BigVGAN model checkpoint (.pt file).
- **fm_config** (str) - Required - Path to flow-matching config.yaml file.
- **fm_ckpt** (str) - Required - Path to flow-matching model checkpoint (.pt file).
- **device** (torch.device) - Required - Target device (cuda:0, etc.).
- **look_ahead_tokens** (int) - Optional - Lookahead tokens for streaming. (Default: 0)
- **max_prompt_chunk** (int) - Optional - Maximum chunks for prompt prefilling. (Default: 2)
- **max_kv_cache_tokens** (int) - Optional - Maximum KV cache size for transformer. (Default: 900)
- **use_cfg** (bool) - Optional - Enable classifier-free guidance. (Default: False)
- **use_cfg_rescale** (bool) - Optional - Rescale CFG output for numerical stability. (Default: True)
- **cfg_init** (float) - Optional - Initial CFG scale value. (Default: 1.5)
- **cfg_scale** (float) - Optional - CFG guidance scale strength. (Default: 7.5)
- **cfg_schedule** (str) - Optional - CFG schedule type: "linear" or other. (Default: "linear")

### Returns
Fully initialized PrefixStreamingFlowMatchingDetokenizer
```

--------------------------------

### Initialize KimiASampler for Greedy Sampling

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiASampler.md

Use this configuration for deterministic audio and text generation. Set temperature to 0.0 for both.

```python
sampler = KimiASampler(
    audio_top_k=5,
    audio_temperature=0.0,  # Greedy
    audio_repetition_penalty=1.0,  # No penalty
    audio_repetition_window_size=64,
    text_top_k=5,
    text_temperature=0.0,  # Greedy
    text_repetition_penalty=1.0,
    text_repetition_window_size=16
)
```

--------------------------------

### ShapeValidationError Handling

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/errors.md

Provides examples of ensuring correct tensor shapes for semantic tokens and mel-spectrograms to avoid ShapeValidationError. Correct shapes are crucial for detokenizer functions.

```python
import torch

# Ensure correct shapes
semantic_tokens = torch.tensor([[100, 101, 102]])  # [1, 3]
mel = torch.randn(100, 80)  # [time, mels]

# These will work
detokenizer.detokenize_streaming(semantic_tokens)
detokenizer.prefill(mel, torch.tensor([100, 101, ...]))
```

--------------------------------

### Custom Sampling with KimiASampler

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/INDEX.md

Demonstrates how to use KimiASampler to sample audio and text tokens from provided logits. Ensure KimiASampler and torch are imported.

```python
from kimia_infer.utils.sampler import KimiASampler
import torch

sampler = KimiASampler(
    audio_top_k=5,
    audio_temperature=0.8,
    audio_repetition_penalty=1.1,
    audio_repetition_window_size=64,
    text_top_k=5,
    text_temperature=0.0,
    text_repetition_penalty=1.0,
    text_repetition_window_size=16
)

# Sample from logits
audio_logits = torch.randn(1, 50000)
text_logits = torch.randn(1, 151552)

audio_token = sampler.sample_audio_logits(audio_logits)
text_token = sampler.sample_text_logits(text_logits)
```

--------------------------------

### Get State Dictionary for Checkpointing

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Retrieves the current state of the wrapper as a dictionary, which can be used for saving checkpoints. This includes the current position ID, ODE wrapper state, and cached conditions.

```python
state = fm_wrapper.state_dict()
```

--------------------------------

### KimiAudio Constructor

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAudio.md

Initializes the Kimi-Audio model for inference. It can be configured to load the audio detokenizer for audio generation or be optimized for text-only tasks.

```APIDOC
## KimiAudio(model_path: str, load_detokenizer: bool = True)

### Description
Initializes the Kimi-Audio model for inference. It loads the audio language model (ALM) and optionally the audio detokenizer.

### Parameters
#### Path Parameters
- **model_path** (str) - Required - Local path to model directory or HuggingFace model ID.
- **load_detokenizer** (bool) - Optional - Whether to load the audio detokenizer module. Defaults to True.

### Returns
KimiAudio instance

### Example
```python
from kimia_infer.api.kimia import KimiAudio

# Load model with detokenizer for audio+text generation
model = KimiAudio(
    model_path="moonshotai/Kimi-Audio-7B-Instruct",
    load_detokenizer=True
)

# Load model for text-only generation (faster)
text_model = KimiAudio(
    model_path="moonshotai/Kimi-Audio-7B-Instruct",
    load_detokenizer=False
)
```
```

--------------------------------

### Initialize KimiASampler for High Diversity with Repetition Control

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiASampler.md

Use this for highly diverse audio and text generation, with active control over repetition. Higher temperatures and penalties increase diversity and reduce repetition.

```python
sampler = KimiASampler(
    audio_top_k=50,
    audio_temperature=1.2,
    audio_repetition_penalty=1.2,  # Penalize repetition
    audio_repetition_window_size=128,
    text_top_k=5,
    text_temperature=0.7,
    text_repetition_penalty=1.1,
    text_repetition_window_size=32
)
```

--------------------------------

### Generate Mel-Spectrogram Chunk with StreamingSemanticFMWrapper

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Generate a chunk of mel-spectrograms from semantic tokens using ODE integration. Provide initial noise, semantic tokens, and the starting position ID. Optionally use a cache for lookahead.

```python
import torch

# Assuming wrapper is an initialized StreamingSemanticFMWrapper instance
# chunk_size = 10
# xt_chunk = torch.randn(chunk_size, 80)
# semantic_tokens_chunk = torch.randint(0, 1000, (chunk_size,))
# start_position_id = 0

# mel_chunk = wrapper.infer_chunk(
#     xt_chunk=xt_chunk,
#     semantic_tokens_chunk=semantic_tokens_chunk,
#     start_position_id=start_position_id,
#     ode_steps=15,
#     verbose=True
# )
```

--------------------------------

### Initialize KimiASampler for Balanced Diversity

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiASampler.md

This configuration provides a balance between diversity in audio and deterministic text generation. Adjust audio temperature for more or less randomness.

```python
sampler = KimiASampler(
    audio_top_k=10,
    audio_temperature=0.8,
    audio_repetition_penalty=1.0,
    audio_repetition_window_size=64,
    text_top_k=5,
    text_temperature=0.0,  # Deterministic text
    text_repetition_penalty=1.0,
    text_repetition_window_size=16
)
```

--------------------------------

### KimiAPromptManager Constructor

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAPromptManager.md

Initializes the KimiAPromptManager with model paths and configuration parameters. It loads necessary tokenizers and encoders for audio and text processing.

```APIDOC
## KimiAPromptManager(model_path: str, kimia_token_offset: int, kimia_text_audiodelaytokens: int)

### Description
Initializes the prompt manager with tokenizers and encoders. It loads the GLM4 audio tokenizer, Whisper encoder, and a text tokenizer, placing all models on the current CUDA device.

### Parameters
#### Path Parameters
- **model_path** (str) - Required - Path to model directory containing tokenizer_config.json, whisper-large-v3 subdirectory, and optional text tokenizer.
- **kimia_token_offset** (int) - Required - Offset value added to audio tokens to distinguish from text tokens (from model config).
- **kimia_text_audiodelaytokens** (int) - Required - Number of delay tokens between text and audio generation (from model config).

### Example
```python
from kimia_infer.api.prompt_manager import KimiAPromptManager

prompt_mgr = KimiAPromptManager(
    model_path="path/to/model",
    kimia_token_offset=10000,
    kimia_text_audiodelaytokens=4
)
```
```

--------------------------------

### Initialize BigVGANWrapper from Pretrained Model

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/BigVGANWrapper.md

Load a BigVGAN model and its configuration from specified paths and initialize the wrapper. Ensure the device is correctly specified for model loading.

```python
import torch
from kimia_infer.models.detokenizer.bigvgan_wrapper import BigVGANWrapper

wrapper = BigVGANWrapper.from_pretrained(
    model_config="vocoder/config.json",
    ckpt_path="vocoder/model.pt",
    device=torch.device("cuda:0")
)
```

--------------------------------

### Class Method: from_pretrained

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/BigVGANWrapper.md

Loads a BigVGAN model from a pretrained checkpoint and configuration file, initializing the BigVGANWrapper.

```APIDOC
## Class Method: from_pretrained

### Description
Load BigVGAN from pretrained checkpoint and config.

### Parameters
#### Parameters
- **model_config** (str) - Required - Path to config.json containing BigVGAN hyperparameters.
- **ckpt_path** (str) - Required - Path to model checkpoint (.pt file).
- **device** (torch.device) - Required - Target device for model loading.

### Returns
Initialized BigVGANWrapper ready for inference.

### Behavior
- Loads hyperparameters from JSON config
- Creates BigVGAN model instance
- Loads checkpoint state dict
- Moves model to device and sets to eval mode
```

--------------------------------

### Perform Inference with Finetuned Model

Source: https://github.com/moonshotai/kimi-audio/blob/master/finetune_codes/README.md

Run this command to perform inference using the finetuned Kimi-Audio model. Ensure the model has been correctly converted for inference.

```bash
CUDA_VISIBLE_DEVICES=0 python -m finetune_codes.check_sft_infer
```

--------------------------------

### Initialize KimiAudio for Text-Only Generation

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAudio.md

Initialize KimiAudio for text-only tasks to potentially speed up loading and inference by skipping the detokenizer. Set load_detokenizer to False.

```python
from kimia_infer.api.kimia import KimiAudio

# Load model for text-only generation (faster)
text_model = KimiAudio(
    model_path="moonshotai/Kimi-Audio-7B-Instruct",
    load_detokenizer=False
)
```

--------------------------------

### Initialize KimiASampler

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiASampler.md

Initialize the KimiASampler with independent parameters for audio and text decoding. This sets up the sampling strategy for both modalities.

```python
from kimia_infer.utils.sampler import KimiASampler

sampler = KimiASampler(
    audio_top_k=5,
    audio_temperature=0.8,
    audio_repetition_penalty=1.0,
    audio_repetition_window_size=64,
    text_top_k=5,
    text_temperature=0.0,
    text_repetition_penalty=1.0,
    text_repetition_window_size=16
)
```

--------------------------------

### Initialize and Infer Chunk

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Demonstrates initializing the wrapper with noise and performing inference on a single chunk of data. This is useful for generating a portion of a mel-spectrogram when the full sequence is not available or for testing purposes.

```python
xt = torch.randn(30, 80)
tokens = torch.tensor([100, 101, 102, ...])  # [30]

mel = fm_wrapper.infer_chunk(
    xt_chunk=xt,
    semantic_tokens_chunk=tokens,
    start_position_id=0,
    ode_steps=30
)
print(mel.shape)  # [30, 80]
```

--------------------------------

### Load Pretrained StreamingSemanticFMWrapper

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Load a pretrained flow-matching model from configuration and checkpoint files. Specify the target device and optional parameters for streaming and guidance.

```python
from kimia_infer.models.detokenizer.semantic_fm_prefix_streaming import StreamingSemanticFMWrapper
import torch

wrapper = StreamingSemanticFMWrapper.from_pretrained(
    fm_config="path/to/config.yaml",
    fm_ckpt="path/to/model.pt",
    device=torch.device("cuda:0" if torch.cuda.is_available() else "cpu"),
    max_prompt_chunk=2,
    max_kv_cache_tokens=900,
    use_cfg=False,
    cfg_scale=7.5
)
```

--------------------------------

### Reinitialize KimiAudio with Detokenizer

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/errors.md

Demonstrates how to reinitialize the KimiAudio model with `load_detokenizer=True` to resolve a 'Detokenizer is not initialized' ValueError when using `output_type='both'`.

```python
# Reinitialize with detokenizer
model = KimiAudio(model_path=model_path, load_detokenizer=True)
```

--------------------------------

### Preprocess Data for Finetuning

Source: https://github.com/moonshotai/kimi-audio/blob/master/finetune_codes/README.md

Preprocess the input data file to extract semantic tokens, creating a new file with the added tokens. This is necessary for the finetuning process.

```bash
CUDA_VISIBLE_DEVICES=0 python -m finetune_codes.extract_semantic_codes --input_file "finetune_codes/demo_data/audio_understanding/data.jsonl" --output_file "finetune_codes/demo_data/audio_understanding/data_with_semantic_codes.jsonl"
```

--------------------------------

### Initialize StreamingSemanticFMWrapper

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Instantiate the streaming flow-matching wrapper for mel-generation. Configure parameters like KV cache size, prompt chunking, and classifier-free guidance settings.

```python
from kimia_infer.models.detokenizer.semantic_fm_prefix_streaming import StreamingSemanticFMWrapper
import torch

# Assuming DiTPrefix is defined and speech_model is an initialized instance
# speech_model = DiTPrefix(...)

wrapper = StreamingSemanticFMWrapper(
    speech_model=speech_model,
    max_kv_cache_tokens=900,
    max_prompt_chunk=2,
    use_cfg=True,
    use_cfg_rescale=True,
    cfg_init=1.5,
    cfg_scale=7.5,
    cfg_schedule="linear",
    normalize_mel=False,
    device=torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
)
```

--------------------------------

### Prefill with Reference Audio

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Shows how to prefill the flow-matching cache with a reference mel-spectrogram and corresponding semantic tokens. This is crucial for ensuring consistency in speaker identity and style during subsequent streaming generation.

```python
# Prefill with reference audio
mel_ref = torch.randn(150, 80)
tokens_ref = torch.tensor([100, 101, 102, ...])  # [150]

fm_wrapper.prefill(mel_ref, tokens_ref, chunk_size=150)

# Now ready for streaming generation
```

--------------------------------

### Instantiate ExtraTokens

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/types.md

Demonstrates how to instantiate the ExtraTokens dataclass using a Hugging Face tokenizer. This is typically done once at the beginning of a session.

```python
from transformers import AutoTokenizer
from kimia_infer.utils.special_tokens import instantiate_extra_tokens

tokenizer = AutoTokenizer.from_pretrained("moonshotai/Kimi-Audio-7B-Instruct")
extra_tokens = instantiate_extra_tokens(tokenizer)
```

--------------------------------

### Load KimiAudio Model for Inference

Source: https://github.com/moonshotai/kimi-audio/blob/master/README.md

Load the Kimi-Audio model for inference tasks like ASR and conversational turns. Ensure the model path is correct and the detokenizer is loaded.

```python
import soundfile as sf
from kimia_infer.api.kimia import KimiAudio

# --- 1. Load Model ---
model_path = "moonshotai/Kimi-Audio-7B-Instruct" 
model = KimiAudio(model_path=model_path, load_detokenizer=True)
```

--------------------------------

### audio_prepend

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAContent.md

Adds a single audio token to the beginning of the audio sequence.

```APIDOC
## audio_prepend

### Description
Add a single audio token to the beginning of the sequence.

### Parameters
#### Path Parameters
- **index** (int) - Required - Audio token ID to prepend.
- **is_continuous** (bool) - Optional - Whether this token represents continuous features (1) or discrete token (0). Default: False.
- **audio_token_loss_mask** (bool) - Optional - Whether to include this token in loss calculation. Default: False.
```

--------------------------------

### audio_pretend

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAContent.md

Adds multiple audio tokens to the beginning of the audio sequence.

```APIDOC
## audio_pretend

### Description
Add multiple audio tokens to the beginning of the sequence.

### Parameters
#### Path Parameters
- **ids** (list[int]) - Required - List of audio token IDs to prepend.
- **is_continuous** (bool) - Optional - Applied uniformly to all tokens in ids. Default: False.
- **audio_token_loss_mask** (bool) - Optional - Applied uniformly to all tokens in ids. Default: False.
```

--------------------------------

### StreamingSemanticFMWrapper.from_pretrained

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Loads a pretrained flow-matching model from configuration and checkpoint files. Requires paths to the model configuration and checkpoint, as well as the target device.

```APIDOC
## StreamingSemanticFMWrapper.from_pretrained

### Description
Loads a pretrained flow-matching model from specified configuration and checkpoint files. This class method facilitates the initialization of the `StreamingSemanticFMWrapper` with pre-trained weights.

### Method
`from_pretrained(fm_config: str, fm_ckpt: str, device: torch.device, max_prompt_chunk: int = 2, max_kv_cache_tokens: int = 900, use_cfg: bool = False, cfg_scale: float = 7.5, use_cfg_rescale: bool = True, cfg_init: float = 1.5, cfg_schedule: str = "linear") -> StreamingSemanticFMWrapper`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

#### Parameters
- **fm_config** (str) - Required - Path to config.yaml with model architecture parameters.
- **fm_ckpt** (str) - Required - Path to model checkpoint (.pt file).
- **device** (torch.device) - Required - Target device (cuda:0, cpu, etc.).
- **max_prompt_chunk** (int) - Optional - Maximum prefill chunks. Default: 2.
- **max_kv_cache_tokens** (int) - Optional - KV cache size limit. Default: 900.
- **use_cfg** (bool) - Optional - Classifier-free guidance flag. Default: False.
- **cfg_scale** (float) - Optional - CFG strength. Default: 7.5.
- **use_cfg_rescale** (bool) - Optional - CFG rescaling flag. Default: True.
- **cfg_init** (float) - Optional - CFG init scale. Default: 1.5.
- **cfg_schedule** (str) - Optional - CFG schedule type. Default: "linear".

### Returns
Fully initialized `StreamingSemanticFMWrapper` instance.
```

--------------------------------

### KimiASampler Constructor

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiASampler.md

Initializes the KimiASampler with independent parameters for audio and text decoding, including top-k, temperature, and repetition penalty settings.

```APIDOC
## KimiASampler(audio_top_k: int, audio_temperature: float, audio_repetition_penalty: float, audio_repetition_window_size: int, text_top_k: int, text_temperature: float, text_repetition_penalty: float, text_repetition_window_size: int)

### Description
Initialize sampler with independent parameters for audio and text decoding.

### Parameters
#### Constructor Parameters
- **audio_top_k** (int) - Yes - Number of highest-probability audio tokens to consider for sampling. Set to 0 to disable.
- **audio_temperature** (float) - Yes - Softmax temperature for audio logits. 0.0 = greedy, > 0 = stochastic sampling.
- **audio_repetition_penalty** (float) - Yes - Penalty multiplier for repeated tokens. > 1.0 reduces repetition.
- **audio_repetition_window_size** (int) - Yes - Number of recent tokens to track for repetition penalty.
- **text_top_k** (int) - Yes - Number of highest-probability text tokens to consider for sampling.
- **text_temperature** (float) - Yes - Softmax temperature for text logits.
- **text_repetition_penalty** (float) - Yes - Penalty multiplier for repeated text tokens.
- **text_repetition_window_size** (int) - Yes - Number of recent text tokens to track for repetition penalty.

### Example
```python
from kimia_infer.utils.sampler import KimiASampler

sampler = KimiASampler(
    audio_top_k=5,
    audio_temperature=0.8,
    audio_repetition_penalty=1.0,
    audio_repetition_window_size=64,
    text_top_k=5,
    text_temperature=0.0,
    text_repetition_penalty=1.0,
    text_repetition_window_size=16
)
```
```

--------------------------------

### Extend KimiAContent with Continuous and Loss Masks

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAContent.md

Shows how to extend audio tokens with options for continuous features and loss masking.

```python
content = KimiAContent()
content.audio_extend([100, 101, 102], is_continuous=True, audio_token_loss_mask=True)
# Adds 3 tokens with continuous=True and loss_mask=True for each
```

--------------------------------

### get_prompt

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAPromptManager.md

Builds a complete prompt from a list of messages, handling tokenization, feature extraction, and special token insertion for audio and text.

```APIDOC
## get_prompt(messages: List[Dict], output_type: str = "text", add_assistant_start_msg: bool = True) -> KimiAContent

### Description
Builds a complete prompt from message history. It tokenizes messages, extracts Whisper features for audio, inserts special tokens, and merges everything into a KimiAContent object.

### Parameters
#### Query Parameters
- **messages** (List[Dict]) - Required - List of message dictionaries with keys: role ("user" or "assistant"), message_type ("text", "audio", or "audio-text"), content (str or list).
- **output_type** (str) - Optional - Default: "text". Controls special token insertion and feature extraction. Accepts "text" or "both".
- **add_assistant_start_msg** (bool) - Optional - Default: True. If True, prepends assistant message start marker to the prompt.

### Returns
- **KimiAContent** - An object containing combined audio/text token IDs, continuous features, and loss masks.

### Example
```python
messages = [
    {"role": "user", "message_type": "text", "content": "Transcribe:"},
    {"role": "user", "message_type": "audio", "content": "audio.wav"}
]

prompt = prompt_mgr.get_prompt(messages, output_type="text")
audio_ids, text_ids, is_continuous, _, _ = prompt.to_tensor()
```
```

--------------------------------

### Basic Audio-to-Text Transcription with KimiAudio

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/INDEX.md

Use this snippet for basic audio-to-text transcription. Ensure KimiAudio is initialized with `load_detokenizer=False` for transcription tasks.

```python
from kimia_infer.api.kimia import KimiAudio

model = KimiAudio(model_path="moonshotai/Kimi-Audio-7B-Instruct", load_detokenizer=False)

messages = [
    {"role": "user", "message_type": "text", "content": "Transcribe:"},
    {"role": "user", "message_type": "audio", "content": "speech.wav"}
]

_, text = model.generate(messages, output_type="text", text_temperature=0.0)
print(text)  # Transcribed text
```

--------------------------------

### Initialize PrefixStreamingFlowMatchingDetokenizer

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/PrefixStreamingFlowMatchingDetokenizer.md

Instantiate the detokenizer by providing initialized vocoder and flow-matching components. The look_ahead_tokens parameter can be adjusted to improve streaming latency.

```python
from kimia_infer.models.detokenizer import (
    PrefixStreamingFlowMatchingDetokenizer,
    BigVGANWrapper,
    StreamingSemanticFMWrapper
)

vocoder = BigVGANWrapper.from_pretrained(config, ckpt, device)
fm = StreamingSemanticFMWrapper.from_pretrained(config, ckpt, device)

detokenizer = PrefixStreamingFlowMatchingDetokenizer(
    vocoder=vocoder,
    fm=fm,
    look_ahead_tokens=12
)
```

--------------------------------

### Balanced Quality Preset (Audio + Text)

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/configuration.md

This preset offers a balance between audio and text quality. Ensure load_detokenizer is True for audio output.

```python
model = KimiAudio(
    model_path="moonshotai/Kimi-Audio-7B-Instruct",
    load_detokenizer=True
)

params = {
    "output_type": "both",
    "max_new_tokens": 1500,
    "audio_temperature": 0.8,
    "audio_top_k": 10,
    "text_temperature": 0.0,
    "text_top_k": 5,
}
```

--------------------------------

### Convert Finetuned Model for Inference

Source: https://github.com/moonshotai/kimi-audio/blob/master/finetune_codes/README.md

Convert the finetuned model checkpoints into a format suitable for inference. Specify the input directory containing checkpoints and the output directory for the converted model.

```bash
CUDA_VISIBLE_DEVICES=0 python -m finetune_codes.model --model_name "moonshotai/Kimi-Audio-7B"     --action "export_model"     --input_dir "output/kimiaudio_ckpts"     --output_dir "output/finetuned_hf_for_inference"
```

--------------------------------

### Initialize Glm4Tokenizer

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/Glm4Tokenizer.md

Initializes the Glm4Tokenizer with a specified path to the tokenizer directory or a HuggingFace model ID. The tokenizer should be moved to the appropriate device (e.g., CUDA) after initialization.

```python
from kimia_infer.models.tokenizer.glm4_tokenizer import Glm4Tokenizer

tokenizer = Glm4Tokenizer("THUDM/glm-4-voice-tokenizer")
tokenizer = tokenizer.to(torch.device("cuda:0"))
```

--------------------------------

### Sample Audio Logits

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiASampler.md

Sample an audio token from logits using top-k, temperature, and repetition penalty. Supports greedy decoding (temperature=0) or stochastic sampling. Recent tokens can be provided to apply the repetition penalty.

```python
import torch

# Logits from model [batch=1, vocab_size]
audio_logits = torch.randn(1, 50000)

# Sample with greedy decoding (temp=0)
token = sampler.sample_audio_logits(audio_logits)

# Sample with stochastic decoding
token = sampler.sample_audio_logits(
    audio_logits,
    recent_tokens=torch.tensor([100, 101, 102])
)
```

--------------------------------

### Configure Debug Logging with Loguru

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/errors.md

Enables debug logging for the Kimi-Audio library by adding a handler to `loguru.logger` that directs output to `sys.stderr` at the 'DEBUG' level. This is useful for troubleshooting model operations.

```python
from loguru import logger
import sys

# Add stderr handler with debug level
logger.add(sys.stderr, level="DEBUG")

# Now all model operations will log detailed information
model = KimiAudio(model_path=model_path, load_detokenizer=True)
```

--------------------------------

### Greedy Sampling Configuration

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/types.md

A typical configuration for deterministic text and audio generation using greedy sampling. This sets temperatures to 0.0 and top_k to 5 for both modalities.

```python
{
    "text_temperature": 0.0,
    "text_top_k": 5,
    "audio_temperature": 0.0,
    "audio_top_k": 5,
}
```

--------------------------------

### Prepare Librispeech ASR Task Data

Source: https://github.com/moonshotai/kimi-audio/blob/master/finetune_codes/README.md

Use this script to prepare demo data for the Librispeech ASR task. The output directory will store the generated data.

```bash
python finetune_codes/demo_data/audio_understanding/prepare_librispeech_asrtask.py --output_dir "output/data/librispeech"
```

--------------------------------

### High Quality Preset (Slow, Audio + Text)

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/configuration.md

For the highest audio and text quality, use this preset. It involves slower generation due to increased steps and specific repetition parameters.

```python
model = KimiAudio(
    model_path="moonshotai/Kimi-Audio-7B-Instruct",
    load_detokenizer=True
)

params = {
    "output_type": "both",
    "max_new_tokens": 1500,
    "audio_temperature": 0.6,
    "audio_top_k": 5,
    "text_temperature": 0.0,
    "audio_repetition_penalty": 1.1,
    "audio_repetition_window_size": 128,
}
```

--------------------------------

### Kimi Audio Model Configuration

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/types.md

Configuration parameters specific to the Kimi Audio model, loaded from model config.json. Includes settings for audio-text delay and token offsets.

```python
{
    "kimia_mimo_audiodelaytokens": int,  # Delay between text and audio
    "kimia_token_offset": int,  # Offset for audio token IDs
    # ... other transformer config
}
```

--------------------------------

### Constructor: BigVGANWrapper

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/BigVGANWrapper.md

Initializes the BigVGAN vocoder wrapper with the specified BigVGAN model, device, and hyperparameters.

```APIDOC
## Constructor: BigVGANWrapper

### Description
Initializes the BigVGAN vocoder wrapper.

### Parameters
#### Parameters
- **vocoder** (BigVGAN) - Required - Underlying BigVGAN model instance.
- **device** (torch.device) - Required - Target device (cuda:0, cpu, etc.).
- **h** (AttrDict) - Required - Hyperparameter dict from config.json containing: n_fft, num_mels, sampling_rate, hop_size, win_size, fmin, fmax.
- **dtype** (torch.dtype) - Optional - Optional dtype for conversion (float32, bfloat16, etc.).

### Example
```python
import torch
from kimia_infer.models.detokenizer.bigvgan_wrapper import BigVGANWrapper

wrapper = BigVGANWrapper.from_pretrained(
    model_config="vocoder/config.json",
    ckpt_path="vocoder/model.pt",
    device=torch.device("cuda:0")
)
```
```

--------------------------------

### sample_audio_logits

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiASampler.md

Samples an audio token from the provided logits using top-k, temperature, and repetition penalty configurations.

```APIDOC
## sample_audio_logits(logits: torch.Tensor, recent_tokens: torch.Tensor = None) -> torch.Tensor

### Description
Sample an audio token from logits with top-k, temperature, and repetition penalty.

### Parameters
#### Path Parameters
- **logits** (torch.Tensor) - Yes - Audio logits tensor of shape [batch, vocab_size] or [batch, seq_len, vocab_size]. If 3D, only the last token's logits are used.
- **recent_tokens** (torch.Tensor) - No - Tensor of recent token IDs for repetition penalty, shape [window_size]. Only applied if repetition_penalty > 1.0 and provided.

### Returns
torch.Tensor of shape [batch] containing sampled token IDs.

### Sampling Process
1. If logits is 3D [batch, seq, vocab], extract last position: [batch, vocab]
2. Apply repetition penalty to recent tokens if penalty > 1.0
3. Compute log-softmax of logits
4. If temperature > 0:
   - Scale logits by temperature
   - Apply top-k filtering: select top-k tokens, zero out others
   - Sample from filtered distribution using multinomial sampling
5. If temperature == 0: return argmax (greedy)

### Repetition Penalty Mechanism
- For each token in recent_tokens window:
  - If log-probability < 0: multiply by penalty (more negative)
  - If log-probability >= 0: divide by penalty (less positive)
- This reduces probability of repeating tokens

### Example
```python
import torch

# Logits from model [batch=1, vocab_size]
audio_logits = torch.randn(1, 50000)

# Sample with greedy decoding (temp=0)
token = sampler.sample_audio_logits(audio_logits)

# Sample with stochastic decoding
token = sampler.sample_audio_logits(
    audio_logits,
    recent_tokens=torch.tensor([100, 101, 102])
)
```
```

--------------------------------

### tokenize Method

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/Glm4Tokenizer.md

Converts audio input (either a waveform tensor or an audio file path) into a sequence of discrete semantic tokens. The tokens represent audio semantics extracted at a 12.5Hz rate.

```APIDOC
## tokenize(speech: torch.Tensor = None, audio_path: str = None, sr: int = 16000) -> torch.Tensor

### Description
Converts audio to discrete semantic tokens. This method processes either a raw audio waveform tensor or a path to an audio file, outputting quantized token IDs.

### Parameters
#### Query Parameters
- **speech** (torch.Tensor) - Optional - Audio waveform. Can be 1D [samples] or 2D [1, samples].
- **audio_path** (str) - Optional - Path to .wav file to load.
- **sr** (int) - Optional - Default: 16000 - Sample rate for audio. If audio_path provided, audio is loaded at 16kHz regardless.

*Note: Exactly one of speech or audio_path must be provided.*

### Returns
torch.Tensor of shape [1, num_tokens] containing quantized token IDs.

### Behavior
- Loads audio from file (if audio_path) or converts tensor to required format
- Extracts Whisper mel-spectrogram features
- Passes through WhisperVQEncoder to get quantized tokens
- Returns discrete token IDs at 12.5Hz sampling rate (50 tokens per second)

### Output Token Frequency
- At 16kHz input: 50 tokens per second
- A 1-second audio produces ~50 tokens
- A 10-second audio produces ~500 tokens

### Example
```python
import torch
import librosa

# From file
tokens_from_file = tokenizer.tokenize(audio_path="speech.wav")
print(tokens_from_file.shape)  # [1, num_tokens]

# From tensor (1 second at 16kHz)
audio = torch.randn(16000)
tokens_from_tensor = tokenizer.tokenize(speech=audio, sr=16000)

# From pre-loaded array
audio_arr, sr = librosa.load("speech.wav", sr=16000)
tokens = tokenizer.tokenize(speech=audio_arr, sr=16000)
```
```

--------------------------------

### Build Prompt from Message History

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAPromptManager.md

Construct a complete prompt from a list of messages, including tokenization and feature extraction for audio. Use 'output_type="text"' for text-only output or 'output_type="both"' for combined features.

```python
messages = [
    {"role": "user", "message_type": "text", "content": "Transcribe:"},
    {"role": "user", "message_type": "audio", "content": "audio.wav"}
]

prompt = prompt_mgr.get_prompt(messages, output_type="text")
audio_ids, text_ids, is_continuous, _, _ = prompt.to_tensor()
```

--------------------------------

### Custom Token Handling with KimiAContent

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/INDEX.md

Use KimiAContent for custom token handling and prompt construction. This snippet demonstrates extending content with audio and text tokens and converting them to tensors.

```python
from kimia_infer.utils.data import KimiAContent

content = KimiAContent()
content.audio_extend([100, 101, 102], is_continuous=True)
content.text_extend([200, 201, 202])

audio_ids, text_ids, _, _, _ = content.to_tensor()
```

--------------------------------

### Multiturn Audio Conversation

Source: https://github.com/moonshotai/kimi-audio/blob/master/README.md

Demonstrates a multiturn conversation where previous assistant responses (audio and text) are included as context for the next user query. This allows for more coherent and context-aware interactions.

```python
messages = [
    {"role": "user", "message_type": "audio", "content": "test_audios/multiturn/case2/multiturn_q1.wav"},
    {"role": "assistant", "message_type": "audio-text", "content": ["test_audios/multiturn/case2/multiturn_a1.wav", "当然可以，这很简单。一二三四五六七八九十。"]},
    {"role": "user", "message_type": "audio", "content": "test_audios/multiturn/case2/multiturn_q2.wav"}
]
wav, text = model.generate(messages, **sampling_params, output_type="both")


wav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type="both")

output_audio_path = "output_audio.wav"
sf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000)
print(f">>> Conversational Output Audio saved to: {output_audio_path}")
print(">>> Conversational Output Text: ", text_output)
```

--------------------------------

### Tokenize Audio from File

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/Glm4Tokenizer.md

Converts an audio file into discrete semantic tokens. The audio is loaded at 16kHz, and the resulting tokens are of shape [1, num_tokens].

```python
import torch
import librosa

# From file
tokens_from_file = tokenizer.tokenize(audio_path="speech.wav")
print(tokens_from_file.shape)  # [1, num_tokens]
```

--------------------------------

### text_pretend

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/KimiAContent.md

Adds multiple text tokens to the beginning of the text sequence.

```APIDOC
## text_pretend

### Description
Add multiple text tokens to the beginning.

### Parameters
#### Path Parameters
- **ids** (list[int]) - Required - List of text token IDs to prepend.
- **text_token_loss_mask** (bool) - Optional - Applied uniformly to all tokens. Default: False.
```

--------------------------------

### prefill

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Prefills the flow-matching cache with a reference mel-spectrogram and semantic tokens, preparing the model for subsequent streaming generation.

```APIDOC
## prefill(mel: torch.Tensor, semantic_token: torch.Tensor, chunk_size: int = 150, verbose: bool = False)

### Description
Prefill flow-matching cache with reference mel-spectrogram and tokens.

### Parameters
#### Path Parameters
None
#### Query Parameters
None
#### Request Body
None

### Parameters
- **mel** (torch.Tensor) - Required - Reference mel-spectrogram [time_steps, 80].
- **semantic_token** (torch.Tensor) - Required - Semantic tokens [seq_len].
- **chunk_size** (int) - Optional - Chunk size for prefilling. Default: 150.
- **verbose** (bool) - Optional - Print timing. Default: False.

### Returns
None

### Behavior
- Pads/truncates mel to match token length
- Processes mel and tokens in chunks
- Fills transformer KV cache with reference context
- Saves state dict for recovery
- Prepares model for subsequent streaming generation

### Request Example
```python
# Prefill with reference audio
mel_ref = torch.randn(150, 80)
tokens_ref = torch.tensor([100, 101, 102, ...])  # [150]

fm_wrapper.prefill(mel_ref, tokens_ref, chunk_size=150)

# Now ready for streaming generation
```
```

--------------------------------

### Load State Dictionary from Checkpoint

Source: https://github.com/moonshotai/kimi-audio/blob/master/_autodocs/api-reference/StreamingSemanticFMWrapper.md

Restores the wrapper's state from a previously saved state dictionary. This allows for resuming generation from a specific point or transferring state between different instances.

```python
fm_wrapper.load_state_dict(state)
```