### Install Matcha-TTS using Pip

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command installs the Matcha-TTS package directly from PyPI using pip, providing the simplest way to get started with the library.

```Bash
pip install matcha-tts
```

--------------------------------

### Install Matcha-TTS from Source for Training

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

After cloning the repository, this command installs the Matcha-TTS package in editable mode, making it ready for development and training purposes.

```Bash
pip install -e .
```

--------------------------------

### Install Matcha-TTS from Source

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This set of commands allows users to clone the Matcha-TTS repository and install it in editable mode from the source, which is useful for development or contributing to the project.

```Bash
pip install git+https://github.com/shivammehta25/Matcha-TTS.git
cd Matcha-TTS
pip install -e .
```

--------------------------------

### Install Matcha-TTS using Conda

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This snippet provides the commands to create and activate a new Conda environment for Matcha-TTS, ensuring a clean and isolated setup for project dependencies.

```Bash
conda create -n matcha-tts python=3.10 -y
conda activate matcha-tts
```

--------------------------------

### Install ONNX Library

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

Command to install the ONNX library, which is a prerequisite for exporting Matcha-TTS checkpoints to the ONNX format for optimized inference.

```bash
pip install onnx
```

--------------------------------

### Install ONNX Runtime for Inference

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

Commands to install the necessary ONNX Runtime packages for performing inference on exported ONNX models, supporting both CPU and GPU execution.

```bash
pip install onnxruntime
```

```bash
pip install onnxruntime-gpu
```

--------------------------------

### Run Matcha-TTS Model Training

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

Commands to initiate the training process for Matcha-TTS, offering various configurations including default, minimum memory usage, and multi-GPU training setups.

```bash
make train-ljspeech
```

```bash
python matcha/train.py experiment=ljspeech
```

```bash
python matcha/train.py experiment=ljspeech_min_memory
```

```bash
python matcha/train.py experiment=ljspeech trainer.devices=[0,1]
```

--------------------------------

### Launch Matcha-TTS Gradio Application

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command starts the Gradio web application for Matcha-TTS, providing a user-friendly graphical interface for speech synthesis.

```Bash
matcha-tts-app
```

--------------------------------

### Launch Gradio Controller for LLaMA-Omni2

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Starts the controller service for the LLaMA-Omni2 Gradio demo, binding it to all network interfaces on port 10000.

```shell
python -m llama_omni2.serve.controller --host 0.0.0.0 --port 10000
```

--------------------------------

### Launch Gradio Web Server for LLaMA-Omni2

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Starts the Gradio web interface for LLaMA-Omni2, connecting to the specified controller, listening on port 8000, and providing the vocoder directory.

```shell
python -m llama_omni2.serve.gradio_web_server --controller http://localhost:10000 --port 8000 --vocoder-dir models/cosy2_decoder
```

--------------------------------

### Llama-Omni2 Python Project Dependencies List

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/requirements.txt

This snippet provides the complete list of Python packages and their specified versions required for the Llama-Omni2 project. The dependencies are organized into logical sections such as deep learning frameworks, configuration tools, logging libraries, and general utilities, facilitating environment setup typically via `pip install -r`.

```Python
# --------- pytorch --------- #
torch>=2.0.0
torchvision>=0.15.0
lightning>=2.0.0
torchmetrics>=0.11.4

# --------- hydra --------- #
hydra-core==1.3.2
hydra-colorlog==1.2.0
hydra-optuna-sweeper==1.2.0

# --------- loggers --------- #
# wandb
# neptune-client
# mlflow
# comet-ml
# aim>=3.16.2  # no lower than 3.16.2, see https://github.com/aimhubio/aim/issues/2550

# --------- others --------- #
rootutils       # standardizing the project root setup
pre-commit      # hooks for applying linters on commit
rich            # beautiful text formatting in terminal
pytest          # tests
# sh            # for running bash commands in some tests (linux/macos only)
phonemizer      # phonemization of text
tensorboard
librosa
Cython
numpy
einops
inflect
Unidecode
scipy
torchaudio
matplotlib
pandas
conformer==0.3.2
diffusers==0.25.0
notebook
ipywidgets
gradio==3.43.2
gdown
wget
seaborn
piper_phonemize
```

--------------------------------

### Launch LLaMA-Omni2 Model Worker

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Starts a model worker service for LLaMA-Omni2, loading a specific language model from the provided path, connecting to the controller, and exposing itself on port 40000.

```shell
python -m llama_omni2.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path models/$model_name --model-name $model_name
```

--------------------------------

### Install LLaMA-Omni2 Python Packages

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Creates a new Conda environment named 'llama-omni2' with Python 3.10, activates it, and installs the necessary project dependencies in editable mode.

```shell
conda create -n llama-omni2 python=3.10
conda activate llama-omni2
pip install -e .
```

--------------------------------

### Train HiFi-GAN Model

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/matcha/hifigan/README.md

This command initiates the training process for the HiFi-GAN model. Users can specify different model configurations (V1, V2, V3) by changing the `config_vX.json` file. Checkpoints and a copy of the configuration file are saved in the `cp_hifigan` directory by default, with an option to change the path using the `--checkpoint_path` argument.

```python
python train.py --config config_v1.json
```

--------------------------------

### Set Euler ODE Solver Steps via CLI

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command shows how to specify the number of Euler ODE solver steps, impacting the quality and computational cost of the speech synthesis process.

```Bash
matcha-tts --text "<INPUT TEXT>" --steps 10
```

--------------------------------

### Clone Matcha-TTS Repository for Training

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command clones the official Matcha-TTS GitHub repository, which is the first step required to set up the project for training with custom datasets.

```Bash
git clone https://github.com/shivammehta25/Matcha-TTS.git
cd Matcha-TTS
```

--------------------------------

### Configure Dataset File Paths for Training

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This YAML snippet shows how to configure the training and validation file list paths within the `ljspeech.yaml` configuration file, essential for pointing the training process to the correct dataset files.

```YAML
train_filelist_path: data/filelists/ljs_audio_text_train_filelist.txt
valid_filelist_path: data/filelists/ljs_audio_text_val_filelist.txt
```

--------------------------------

### Fine-Tune HiFi-GAN Model

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/matcha/hifigan/README.md

This command allows fine-tuning of the HiFi-GAN model. It requires pre-generated mel-spectrograms in `.npy` format, where the file name of the mel-spectrogram should match the corresponding audio file. These `.npy` files must be placed in an `ft_dataset` folder. The `--fine_tuning True` flag enables this mode, and configuration files can be specified similar to standard training.

```python
python train.py --fine_tuning True --config config_v1.json
```

--------------------------------

### Run ONNX Inference with Matcha-TTS

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

Various commands for running inference on an exported Matcha-TTS ONNX model, including basic text-to-mel, controlling synthesis parameters, GPU acceleration, and integrating an external vocoder for full audio generation.

```bash
python3 -m matcha.onnx.infer model.onnx --text "hey" --output-dir ./outputs
```

```bash
python3 -m matcha.onnx.infer model.onnx --text "hey" --output-dir ./outputs --temperature 0.4 --speaking_rate 0.9 --spk 0
```

```bash
python3 -m matcha.onnx.infer model.onnx --text "hey" --output-dir ./outputs --gpu
```

```bash
python3 -m matcha.onnx.infer model.onnx --text "hey" --output-dir ./outputs --vocoder hifigan.small.onnx
```

--------------------------------

### Synthesize Speech via CLI from Text

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command demonstrates how to use the Matcha-TTS command-line interface to synthesize speech from a given input text string. It automatically downloads required models.

```Bash
matcha-tts --text "<INPUT TEXT>"
```

--------------------------------

### Download CosyVoice 2 Models

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Downloads the flow-matching model and vocoder components of CosyVoice 2 from Hugging Face, resuming download if interrupted, to a local directory.

```shell
huggingface-cli download --resume-download ICTNLP/cosy2_decoder --local-dir models/cosy2_decoder
```

--------------------------------

### Perform Inference with HiFi-GAN

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/matcha/hifigan/README.md

This command performs speech synthesis inference from WAV audio files. Users need to create a `test_files` directory and copy their input WAV files into it. The command requires providing the path to a generator checkpoint file using `--checkpoint_file`. Generated WAV files are saved in the `generated_files` directory by default, which can be changed using the `--output_dir` option.

```python
python inference.py --checkpoint_file [generator checkpoint file path]
```

--------------------------------

### Generate Normalization Statistics for Dataset

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command uses the `matcha-data-stats` utility to generate normalization statistics based on the specified dataset configuration file, a crucial step before training a new model.

```Bash
matcha-data-stats -i ljspeech.yaml
```

--------------------------------

### Download LLaMA-Omni2 Language Models

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Downloads a specified LLaMA-Omni2 series model (e.g., LLaMA-Omni2-7B-Bilingual) from Hugging Face to a local directory, resuming download if needed.

```shell
model_name=LLaMA-Omni2-7B-Bilingual
huggingface-cli download --resume-download ICTNLP/$model_name --local-dir models/$model_name
```

--------------------------------

### Clone LLaMA-Omni2 Repository

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Clones the LLaMA-Omni2 GitHub repository to your local machine and navigates into the project directory.

```shell
git clone https://github.com/ictnlp/LLaMA-Omni2
cd LLaMA-Omni2
```

--------------------------------

### Synthesize Audio from Custom Trained Model

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

Command to synthesize audio from input text using a custom trained Matcha-TTS model. Requires specifying the input text and the path to the trained model checkpoint.

```bash
matcha-tts --text "<INPUT TEXT>" --checkpoint_path <PATH TO CHECKPOINT>
```

--------------------------------

### Adjust Speaking Rate via CLI

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command demonstrates how to control the speaking rate of the synthesized speech using a command-line argument, allowing for customization of the output audio.

```Bash
matcha-tts --text "<INPUT TEXT>" --speaking_rate 1.0
```

--------------------------------

### Synthesize Speech via CLI from File

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command allows synthesizing speech by providing a path to a file containing the input text, suitable for processing longer texts or multiple inputs.

```Bash
matcha-tts --file <PATH TO FILE>
```

--------------------------------

### Batch Synthesize Speech via CLI from File

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command enables batch synthesis from a specified file, optimizing the process for multiple text inputs contained within a single file.

```Bash
matcha-tts --file <PATH TO FILE> --batched
```

--------------------------------

### Set Hugging Face Mirror Endpoint (Optional)

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Sets an environment variable to use a Hugging Face mirror endpoint, which can help improve download stability, especially from within China.

```shell
export HF_ENDPOINT=https://hf-mirror.com
```

--------------------------------

### Run End-to-End Speech Synthesis Inference

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/matcha/hifigan/README.md

Executes the `inference_e2e.py` script to perform end-to-end speech synthesis. This command requires a path to a generator checkpoint file and will save the generated WAV files to `generated_files_from_mel` by default, which can be overridden using the `--output_dir` option.

```Python
python inference_e2e.py --checkpoint_file [generator checkpoint file path]
```

--------------------------------

### Adjust Sampling Temperature via CLI

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

This command illustrates how to modify the sampling temperature during speech synthesis, which can influence the variability and naturalness of the generated audio.

```Bash
matcha-tts --text "<INPUT TEXT>" --temperature 0.667
```

--------------------------------

### Matcha-TTS Citation Information

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

BibTeX entry for citing the Matcha-TTS paper, essential for academic publications and acknowledging the work.

```text
@inproceedings{mehta2024matcha,
  title={Matcha-{TTS}: A fast {TTS} architecture with conditional flow matching},
  author={Mehta, Shivam and Tu, Ruibo and Beskow, Jonas and Sz\'{e}kely, \'{E}va and Henter, Gustav Eje},
  booktitle={Proc. ICASSP},
  year={2024}
}
```

--------------------------------

### Update Data Statistics in YAML Configuration

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

Illustrates how to update the `data_statistics` section in `configs/data/ljspeech.yaml` with computed mel mean and standard deviation values, which are essential for model training.

```bash
data_statistics:  # Computed for ljspeech dataset
  mel_mean: -5.536622
  mel_std: 2.116101
```

--------------------------------

### Run Local Inference for LLaMA-Omni2

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

This snippet demonstrates how to perform local inference with the LLaMA-Omni2 model. It involves running a Python script to generate answers from a question file and then using another script to convert these answers into speech (WAV files).

```shell
output_dir=examples/$model_name
mkdir -p $output_dir

python llama_omni2/inference/run_llama_omni2.py \
    --model_path models/$model_name \
    --question_file examples/questions.json \
    --answer_file $output_dir/answers.jsonl \
    --temperature 0 \
    --s2s

python llama_omni2/inference/run_cosy2_decoder.py \
    --input-path $output_dir/answers.jsonl \
    --output-dir $output_dir/wav \
    --lang en
```

--------------------------------

### Export Matcha-TTS Checkpoint to ONNX

Source: https://github.com/ictnlp/llama-omni2/blob/main/third_party/Matcha-TTS/README.md

Command to convert a trained Matcha-TTS model checkpoint into an ONNX graph. Optionally, a vocoder can be embedded for end-to-end waveform generation.

```bash
python3 -m matcha.onnx.export matcha.ckpt model.onnx --n-timesteps 5
```

--------------------------------

### Download Whisper Large v3 Model

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

Downloads the 'large-v3' version of the Whisper speech recognition model to a specified local directory for use as a speech encoder.

```python
import whisper
model = whisper.load_model("large-v3", download_root="models/speech_encoder/")
```

--------------------------------

### Cite LLaMA-Omni2 Research Papers

Source: https://github.com/ictnlp/llama-omni2/blob/main/README.md

BibTeX entries for citing the LLaMA-Omni 2 and LLaMA-Omni research papers. These citations are provided for academic use when referencing the work in research publications.

```bibtex
@inproceedings{
  fang2025llamaomni2,
  title={{LL}a{MA}-{O}mni 2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis},
  author={Fang, Qingkai and Zhou, Yan and Guo, Shoutao and Zhang, Shaolei and Feng, Yang},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics},
  year={2025}
}

@inproceedings{
  fang2025llamaomni,
  title={{LL}a{MA}-{O}mni: Seamless Speech Interaction with Large Language Models},
  author={Qingkai Fang and Shoutao Guo and Yan Zhou and Zhengrui Ma and Shaolei Zhang and Yang Feng},
  booktitle={The Thirteenth International Conference on Learning Representations},
  year={2025},
  url={https://openreview.net/forum?id=PYmrUQmMEw}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.