### Run Benchmarks Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Execute the benchmark script to reproduce performance metrics on your machine. Ensure you have the necessary environment set up. ```shell ./bench.sh ``` -------------------------------- ### Build Linux with OpenBLAS Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Builds the project on Linux using OpenBLAS for acceleration. This is the recommended build for Linux systems. ```bash make blas ``` -------------------------------- ### Download Model Weights Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Downloads the necessary model weights, tokenizer, and voice embeddings from HuggingFace. Requires the huggingface_hub CLI or wget. ```bash ./download_model.sh voxtral-tts-model ``` -------------------------------- ### Build macOS with Accelerate Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Builds the project on macOS utilizing the Accelerate framework for optimization. ```bash make apple ``` -------------------------------- ### Build with CUDA and OpenBLAS Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Builds the project with CUDA support for NVIDIA GPUs, leveraging OpenBLAS. Requires NVIDIA GPU and CUDA toolkit. ```bash make cuda ``` -------------------------------- ### Build with CUDA for Blackwell GPU Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Builds the project with CUDA support, specifically targeting Blackwell GPU architecture (e.g., B200). ```bash make cuda CUDA_ARCH=sm_100 ``` -------------------------------- ### Build with CUDA for Hopper GPU Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Builds the project with CUDA support, specifically targeting Hopper GPU architecture (e.g., H100). ```bash make cuda CUDA_ARCH=sm_90 ``` -------------------------------- ### Project Structure Overview Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Overview of the main C source files and their roles within the Voxtral TTS project. This helps in understanding the modular design. ```text voxtral_tts.h Main header (constants, structs, API) voxtral_tts.c Model loading and inference orchestrator voxtral_tts_llm.c 26-layer Mistral decoder with KV cache voxtral_tts_acoustic.c Flow-matching acoustic transformer voxtral_tts_codec.c Audio codec decoder (ALiBi + weight_norm) voxtral_tts_kernels.{c,h} Math kernels (matmul, attention, conv, RoPE, ...) voxtral_tts_tokenizer.{c,h} Tekken BPE tokenizer (encode + decode) voxtral_tts_voice.c Voice embedding loader (.pt) + audio codebook embeddings voxtral_tts_wav.c WAV file writer voxtral_tts_safetensors.{c,h} Safetensors mmap reader main.c CLI entry point ``` -------------------------------- ### Utilities for Model Inspection and Conversion Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md List of utility commands available for inspecting model weights and converting voice embeddings. These are useful for debugging and data preparation. ```text inspect_weights -- dump tensor names/shapes from safetensors (make inspect) convert_voice.py -- convert .pt voice embeddings to raw binary download_model.sh -- download model from HuggingFace ``` -------------------------------- ### Voxtral TTS Command Line Options Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Lists the available command-line options for the voxtral_tts executable, including model directory, voice selection, output file, and verbosity. ```bash Usage: ./voxtral_tts [options] "text to speak" -d Model directory (required) -v Voice name (default: neutral_female) -o Output WAV file (default: output.wav) -s Random seed for reproducibility --verbose Enable verbose output --inspect Print model tensor info and exit ``` -------------------------------- ### Build with CUDA for Ada Lovelace GPU Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Builds the project with CUDA support, specifically targeting Ada Lovelace GPU architecture (e.g., RTX 4090). ```bash make cuda CUDA_ARCH=sm_89 ``` -------------------------------- ### Build without BLAS Acceleration Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Builds the project without BLAS acceleration, resulting in slower performance. This is a portable build option. ```bash make noblas ``` -------------------------------- ### Inference Pipeline Steps Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Detailed steps of the autoregressive inference loop in Voxtral TTS. This explains how the LLM and acoustic transformer interact to generate audio. ```text 1. LLM produces a hidden state 2. Acoustic transformer predicts a semantic code (greedy argmax) and 36 acoustic codes (flow matching with 8 Euler ODE steps and CFG alpha=1.2) 3. The 37 codes are embedded back into LLM input space via multi-vocabulary embeddings (sum across codebooks) 4. Repeat until [END_AUDIO] is generated 5. All collected codes are decoded by the audio codec into a 24kHz waveform ``` -------------------------------- ### Run Voxtral TTS Inference Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md Executes the Voxtral TTS inference engine to generate speech. Specify the model directory, voice, output file, and text. ```bash ./voxtral_tts -d voxtral-tts-model -v neutral_female -o output.wav "Hello world" ``` -------------------------------- ### Voxtral TTS Prompt Format Source: https://github.com/mudler/voxtral-tts.c/blob/main/README.md The expected prompt format for the Voxtral TTS model, including special tokens and voice embedding placeholders. This format is crucial for correct inference. ```text [BOS] [BEGIN_AUDIO] [voice_embedding x N] [/INST] text_tokens [INST] [BEGIN_AUDIO] ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.