### Install Amphion using Setup Installer

Source: https://github.com/open-mmlab/amphion/blob/main/README.md

This snippet details the steps to clone the Amphion repository, set up a Python environment using conda, and install necessary Python packages via the env.sh script.

```bash
git clone https://github.com/open-mmlab/Amphion.git
cd Amphion

# Install Python Environment
conda create --name amphion python=3.9.15
conda activate amphion

# Install Python Packages Dependencies
sh env.sh
```

--------------------------------

### Install Amphion and Dependencies

Source: https://context7.com/open-mmlab/amphion/llms.txt

This section provides instructions for installing the Amphion toolkit and its dependencies. It covers cloning the repository, creating and activating a Conda environment, installing general dependencies via `env.sh`, and installing specific requirements for models like MaskGCT and Vevo. A Docker alternative is also provided for easier setup.

```bash
# Clone repository
git clone https://github.com/open-mmlab/Amphion.git
cd Amphion

# Create conda environment
conda create --name amphion python=3.9.15
conda activate amphion

# Install dependencies
sh env.sh

# For specific models (e.g., MaskGCT)
pip install -r models/tts/maskgct/requirements.txt

# For Vevo models
sudo apt-get install espeak-ng  # Required for phonemizer
pip install -r models/vc/vevo/requirements.txt

# Docker alternative
docker pull realamphion/amphion
docker run --runtime=nvidia --gpus all -it -v .:/app realamphion/amphion
```

--------------------------------

### Install Dependencies and Start Training (Shell)

Source: https://github.com/open-mmlab/amphion/blob/main/models/tta/picoaudio/README.md

This shell script demonstrates how to navigate to the PicoAudio directory, install necessary dependencies using pip, and launch the controllable training process using the 'accelerate' tool.

```shell
cd picoaudio
pip install -r requirements.txt

accelerate launch runner/controllable_train.py
```

--------------------------------

### Install Common Training/Inference Libraries

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/README.md

Installs or upgrades essential Python libraries commonly used in machine learning projects, including Weights & Biases for experiment tracking, Protocol Buffers for serialization, and Transformers for NLP tasks. This is often a prerequisite for environment setup.

```bash
pip install -U wandb protobuf transformers
```

--------------------------------

### Download English Audio Example

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/example.ipynb

Downloads an example English audio file using curl. This is a prerequisite for running the English inference example.

```bash
! curl -o example_wav_en.wav https://fireredteam.github.io/demos/firered_tts/audios/ZeroShotICL_samples/english_to_mix/p276%23p276_354.wav
```

--------------------------------

### Install DualCodec Package

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/README.md

Installs the DualCodec library using pip. This is the primary method for setting up the codec for use.

```bash
pip install dualcodec
```

--------------------------------

### Install Amphion using Docker Image

Source: https://github.com/open-mmlab/amphion/blob/main/README.md

This snippet provides instructions for installing and running Amphion using a Docker image. It includes pulling the image and running a container with NVIDIA GPU support and volume mounting for datasets.

```bash
git clone https://github.com/open-mmlab/Amphion.git
cd Amphion

docker pull realamphion/amphion
docker run --runtime=nvidia --gpus all -it -v .:/app realamphion/amphion
```

--------------------------------

### Download Chinese Audio Example

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/example.ipynb

Downloads an example Chinese audio file using curl. This is a prerequisite for running the Chinese inference example.

```python
! curl -o example_wav_zh.wav https://fireredteam.github.io/demos/firered_tts/audios/SpeakerFinetune_samples/wukong/wukong_000099.wav
```

--------------------------------

### Setup and Run Emilia Dataset Preprocessing Pipeline

Source: https://context7.com/open-mmlab/amphion/llms.txt

This snippet details the setup and execution of the Emilia-Pipe, a pipeline for transforming raw speech data into high-quality training data. It includes environment setup, configuration of the pipeline parameters via a JSON file, and running the main preprocessing script. Dependencies include Python 3.9 and specific model paths.

```bash
conda create -y -n AudioPipeline python=3.9
conda activate AudioPipeline
bash env.sh

cat > config.json << 'EOF'
{
    "language": {
        "multilingual": true,
        "supported": ["zh", "en", "fr", "ja", "ko", "de"]
    },
    "entrypoint": {
        "input_folder_path": "./raw_audio_folder",
        "SAMPLE_RATE": 24000
    },
    "separate": {
        "step1": {
            "model_path": "/path/to/UVR-MDX-NET-Inst_HQ_3.onnx",
            "denoise": true,
            "margin": 44100,
            "chunks": 15
        }
    },
    "mos_model": {
        "primary_model_path": "/path/to/sig_bak_ovr.onnx"
    },
    "huggingface_token": "<YOUR_HF_TOKEN>"
}
EOF

export CUDA_VISIBLE_DEVICES=0
python main.py
```

--------------------------------

### Install espeak-ng for Phonemizer

Source: https://github.com/open-mmlab/amphion/blob/main/models/tts/metis/README.md

Installs the 'espeak-ng' package, a dependency for the 'phonemizer' library used in the project for text-to-phoneme conversion. Installation commands are provided for Debian-like and RedHat-like Linux distributions, with a note for Windows users to download an installer.

```bash
# For Debian-like distribution (e.g. Ubuntu, Mint, etc.)
sudo apt-get install espeak-ng
# For RedHat-like distribution (e.g. CentOS, Fedora, etc.) 
sudo yum install espeak-ng

# For Windows
# Please visit https://github.com/espeak-ng/espeak-ng/releases to download .msi installer
```

--------------------------------

### Set Up Conda Environment and Install Requirements

Source: https://github.com/open-mmlab/amphion/blob/main/models/svc/vevosing/README.md

Creates and activates a Conda environment named 'vevo' with Python 3.10, then installs project dependencies using pip from a requirements file. This ensures a consistent and isolated environment for running Amphion.

```bash
conda create -n vevo python=3.10
conda activate vevo

pip install -r models/vc/vevo/requirements.txt
```

--------------------------------

### Install DualCodec Training Components

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/README.md

Installs the necessary Python packages for training the DualCodec model. This command includes optional dependencies required for the training pipeline.

```bash
pip install "dualcodec[train]"
```

--------------------------------

### Environment Setup Commands

Source: https://github.com/open-mmlab/amphion/blob/main/preprocessors/Emilia/README.md

Commands to set up the Python environment using Conda and activate it. It also includes a script for environment setup.

```bash
conda create -y -n AudioPipeline python=3.9 
conda activate AudioPipeline

bash env.sh
```

--------------------------------

### Setup WeNet Pretrained Model

Source: https://github.com/open-mmlab/amphion/blob/main/pretrained/README.md

Instructions for downloading and setting up the WeNet pretrained model. This involves downloading a tar archive, extracting it, and modifying the 'train.yaml' file to point to the correct 'cmvn_file'.

```shell
cd Amphion/pretrained/wenet

### Unzip the expt dir
tar -xvf wenetspeech_u2pp_conformer_exp.tar.gz

### Specify the updated path in train.yaml
cd 20220506_u2pp_conformer_exp
vim train.yaml
# TODO: Change the value of "cmvn_file" (Line 2) to the absolute path of the `global_cmvn` file. (Eg: [YourPath]/Amphion/pretrained/wenet/20220506_u2pp_conformer_exp/global_cmvn
```

```text
Amphion
 ┣ pretrained
 ┃ ┣ wenet
 ┃ ┃ ┣ 20220506_u2pp_conformer_exp
 ┃ ┃ ┃ ┣ final.pt
 ┃ ┃ ┃ ┣ global_cmvn
 ┃ ┃ ┃ ┣ train.yaml
 ┃ ┃ ┃ ┣ units.txt
```

--------------------------------

### Specify Multiple Metrics for Evaluation

Source: https://github.com/open-mmlab/amphion/blob/main/egs/metrics/README.md

An example of how to specify multiple metrics to be calculated by the `run.sh` script. This example includes Mel Cepstral Distortion (MCD), Perceptual Evaluation of Speech Quality (PESQ), and Frechet Audio Distance (FAD).

```bash
--metrics "mcd pesq fad"
```

--------------------------------

### Install espeak-ng Dependency

Source: https://github.com/open-mmlab/amphion/blob/main/models/svc/vevosing/README.md

Installs the 'espeak-ng' package, a prerequisite for the 'phonemizer' library used in Amphion. Instructions are provided for Debian-like and RedHat-like Linux distributions.

```bash
# For Debian-like distribution (e.g. Ubuntu, Mint, etc.)
sudo apt-get install espeak-ng
# For RedHat-like distribution (e.g. CentOS, Fedora, etc.) 
sudo yum install espeak-ng
```

--------------------------------

### Simulated Dataset Metadata Example (Python)

Source: https://github.com/open-mmlab/amphion/blob/main/models/tta/picoaudio/README.md

This Python dictionary represents the metadata structure for a simulated audio event. It includes the file path, a caption detailing the on/off times for events, and a caption specifying the frequency of events.

```python
{
  "filepath": "data/multi_event_test/syn_1.wav",
  "onoffCaption": "cat meowing at 0.5-2.0, 3.0-4.5 and whistling at 5.0-6.5 and explosion at 7.0-8.0, 8.5-9.5",
  "frequencyCaption": "cat meowing two times and whistling one times and explosion two times"
}
```

--------------------------------

### Install MaskGCT Environment with Conda

Source: https://github.com/open-mmlab/amphion/blob/main/models/tts/maskgct/README.md

Sets up a Python 3.10 environment named 'maskgct' using conda and installs project dependencies from a requirements file. This is the recommended way to configure the project environment.

```bash
conda create -n maskgct python=3.10
conda activate maskgct

pip install -r models/tts/maskgct/requirements.txt
```

--------------------------------

### Install DualCodec TTS Dependencies

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/README.md

Installs the necessary components for training DualCodec-based TTS models. This command includes optional dependencies for text-to-speech functionalities.

```bash
pip install "dualcodec[tts]"
```

--------------------------------

### Run TTS Inference with VALLE Example (Bash)

Source: https://github.com/open-mmlab/amphion/blob/main/egs/tts/VALLE/README.md

This bash script demonstrates how to run text-to-speech inference using the VALLE model. It specifies the stage, GPU, experimental directory, output directory, inference mode, text, text prompt, and audio prompt.

```bash
sh egs/tts/VALLE/run.sh --stage 3 --gpu "0" \
    --infer_expt_dir Amphion/ckpts/tts/[YourExptName] \
    --infer_output_dir Amphion/ckpts/tts/[YourExptName]/result \
    --infer_mode "single" \
    --infer_text "This is a clip of generated speech with the given text from a TTS model." \
    --infer_text_prompt "But even the unsuccessful dramatist has his moments." \
    --infer_audio_prompt egs/tts/VALLE/prompt_examples/7176_92135_000004_000000.wav
```

--------------------------------

### Start Noro Clean Training (Bash)

Source: https://github.com/open-mmlab/amphion/blob/main/egs/vc/Noro/README.md

This command starts the training process for the clean version of the Noro voice conversion model. Ensure the configuration file and dataset paths are correctly set.

```bash
bash path/to/Amphion/egs/vc/noro_train_clean.sh
```

--------------------------------

### Environment Setup for Debatts TTS Model

Source: https://github.com/open-mmlab/amphion/blob/main/models/tts/debatts/README.md

Sets up the necessary environment for the Debatts TTS model. This script handles the installation of all required dependencies. It assumes you have Git and Conda installed.

```bash
git clone https://github.com/open-mmlab/Amphion.git
# create env
bash ./models/tts/debatts/env.sh
```

--------------------------------

### Run FastSpeech2 Training Stage

Source: https://github.com/open-mmlab/amphion/blob/main/egs/tts/FastSpeech2/README.md

This bash command initiates the training stage for FastSpeech2 using the `run.sh` script. Set `--stage 2` and provide a unique `--name` for your experiment. Logs and checkpoints will be saved under `ckpts/tts/[YourExptName]`. You can specify multiple GPUs using the `--gpu` argument.

```bash
sh egs/tts/FastSpeech2/run.sh --stage 2 --name [YourExptName]
```

--------------------------------

### Launch DualCodec-VALLE Gradio Demo (Bash)

Source: https://context7.com/open-mmlab/amphion/llms.txt

Command to launch the Gradio-based graphical user interface for the DualCodec-VALLE text-to-speech model. This allows for interactive speech generation through a web interface.

```bash
python -m dualcodec.infer.valle.gradio_valle_demo
```

--------------------------------

### Run Training Stage with Bash

Source: https://github.com/open-mmlab/amphion/blob/main/egs/svc/MultipleContentsSVC/README.md

This bash command initiates the training stage of the Amphion project. It runs `run.sh` with `--stage 2` and requires an experimental name (`--name`) for saving logs and checkpoints. Tensorboard logs and checkpoints are saved in `Amphion/ckpts/svc/[YourExptName]`.

```bash
sh egs/svc/MultipleContentsSVC/run.sh --stage 2 --name [YourExptName]
```

--------------------------------

### Launch Content-Style Tokenizer Training

Source: https://github.com/open-mmlab/amphion/blob/main/models/svc/vevosing/README.md

Launches the training script for the content-style tokenizer using a shell script. This command should be executed from the Amphion root directory.

```bash
sh egs/codec/coco/contentstyle_fvq16384_12.5hz.sh
```

--------------------------------

### Install DualCodec from Source with TTS

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/README.md

Installs the DualCodec package from the local source code, including optional dependencies for text-to-speech (TTS) training. This is useful when working with the latest development version.

```bash
pip install -e .[tts]
```

--------------------------------

### Resume or Finetune Training (Bash)

Source: https://github.com/open-mmlab/amphion/blob/main/egs/vocoder/diffusion/README.md

Allows resuming training from a checkpoint or finetuning a pretrained model. Requires specifying the stage, experiment name, resume type, and checkpoint path.

```bash
sh egs/vocoder/diffusion/{vocoder_name}/run.sh --stage 2 \
	--name [YourExptName] \
	--resume_type ["resume" for resuming training and "finetune" for loading parameters only] \
	--checkpoint Amphion/ckpts/vocoder/[YourExptName]/checkpoint \

```

--------------------------------

### Start Noro Noisy Training (Bash)

Source: https://github.com/open-mmlab/amphion/blob/main/egs/vc/Noro/README.md

This command starts the training process for the noisy version of the Noro voice conversion model. Ensure the configuration file and noisy dataset paths are correctly set.

```bash
bash path/to/Amphion/egs/vc/noro_train_noisy.sh
```

--------------------------------

### Run Training and Inference Scripts (Shell)

Source: https://github.com/open-mmlab/amphion/blob/main/egs/tta/RECIPE.md

These shell commands initiate the training of the VAE model, the latent diffusion model, and perform inference. Ensure you are in the Amphion root directory before execution.

```shell
sh egs/tta/autoencoderkl/run_train.sh

# Train the latent diffusion model
sh egs/tta/audioldm/run_train.sh

# Inference
sh egs/tta/audioldm/run_inference.sh \
--text "A man is whistling"
```

--------------------------------

### Download MaskGCT Pretrained Checkpoint with Hugging Face Hub

Source: https://github.com/open-mmlab/amphion/blob/main/models/tts/maskgct/README.md

Demonstrates how to download a pretrained checkpoint for the semantic codec component of MaskGCT using the `hf_hub_download` function from the `huggingface_hub` library. Similar methods apply to other model components.

```python
from huggingface_hub import hf_hub_download

# download semantic codec ckpt
semantic_code_ckpt = hf_hub_download("amphion/MaskGCT", filename="semantic_codec/model.safetensors")

# same for other models
```

--------------------------------

### Display Chinese Input and Output Audio

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/example.ipynb

Displays the input and output audio for the Chinese inference example using IPython.display.Audio. This allows for playback of the generated audio.

```python
print("Input audio")
display(Audio(audio.cpu().numpy().squeeze(), rate=24000))
print("Output audio (8vq)")
display(Audio(out_audio.cpu().numpy().squeeze(), rate=24000))
```

--------------------------------

### Display English Input and Output Audio

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/example.ipynb

Displays the input and output audio for the English inference example using IPython.display.Audio. This allows for playback of the generated audio.

```python
from IPython.display import Audio, display
# Play the input audio
print("Input audio")
display(Audio(audio.cpu().numpy().squeeze(), rate=24000))
# Play the output audio
print("Output audio (8vq)")
Audio(out_audio.cpu().numpy().squeeze(), rate=24000)
```

--------------------------------

### Launch Prosody Tokenizer Training

Source: https://github.com/open-mmlab/amphion/blob/main/models/svc/vevosing/README.md

Launches the training script for the prosody tokenizer using a shell script. This command should be executed from the Amphion root directory.

```bash
sh egs/codec/coco/prosody_fvq512_6.25hz.sh
```

--------------------------------

### Run GAN Vocoder Training Stage

Source: https://github.com/open-mmlab/amphion/blob/main/egs/vocoder/gan/README.md

Initiates the training stage for GAN vocoders using `run.sh` with `--stage 2`. Requires an experimental name for saving logs and checkpoints. Supports multi-GPU training and resuming from checkpoints.

```bash
sh egs/vocoder/gan/{vocoder_name}/run.sh --stage 2 --name [YourExptName]
```

--------------------------------

### Start Noro Inference (Bash)

Source: https://github.com/open-mmlab/amphion/blob/main/egs/vc/Noro/README.md

This command initiates the Noro voice conversion inference process. Ensure that the paths in the 'noro_inference.sh' script are correctly configured before execution.

```bash
bash path/to/Amphion/egs/vc/noro_inference.sh
```

--------------------------------

### JSON Output Structure for Processed Audio

Source: https://github.com/open-mmlab/amphion/blob/main/preprocessors/Emilia/README.md

This JSON structure represents the output for a single processed audio file. It includes transcription text, start and end timestamps, language, and a DNSMOS score, indicating the quality of the audio.

```json
[
    {
        "text": "So, don't worry about that. But, like for instance, like yesterday was very hard for me to say, you know what, I should go to bed.", // Transcription
        "start": 67.18, // Start timestamp, in second unit
        "end": 74.41, // End timestamp, in second unit
        "language": "en", // Language
        "dnsmos": 3.44 // DNSMOS P.835 score
    }
]
```

--------------------------------

### Initialize Configuration

Source: https://github.com/open-mmlab/amphion/blob/main/visualization/SingVisio/webpage/index.html

This JavaScript function initializes the application's configuration by loading settings from a specified JSON file. It's a crucial step for setting up the application's behavior and parameters.

```javascript
initConfig('./config/default.json')
```

--------------------------------

### Train VALL-E Non-Autoregressive (NAR) Model

Source: https://github.com/open-mmlab/amphion/blob/main/egs/tts/VALLE/README.md

This bash command starts the training of the VALL-E Non-Autoregressive (NAR) model. It requires the path to the trained AR model checkpoint and specifies an experimental name.

```bash
sh egs/tts/VALLE/run.sh --stage 2 --model_train_stage 2 --ar_model_ckpt_dir [ARModelPath] --name [YourExptName]
```

--------------------------------

### Run GAN Vocoder Data Preparation Stage

Source: https://github.com/open-mmlab/amphion/blob/main/egs/vocoder/gan/README.md

Executes the data preparation stage for GAN vocoder training using the `run.sh` script. The `--stage 1` argument initiates the data preprocessing. Users can specify GPU devices using the `--gpu` argument.

```bash
sh egs/vocoder/gan/{vocoder_name}/run.sh --stage 1
```

--------------------------------

### Run Metis Omni Inference

Source: https://github.com/open-mmlab/amphion/blob/main/models/tts/metis/README.md

Executes the inference script for the Metis Omni model, which supports multiple tasks including TTS, VC, TSE, and SE. This script automatically downloads pretrained models and starts the inference process.

```bash
# Metis Omni
python -m models.tts.metis.metis_infer_omni
```

--------------------------------

### Clone Amphion Repository

Source: https://github.com/open-mmlab/amphion/blob/main/models/svc/vevosing/README.md

Clones the Amphion repository from GitHub and navigates into the project directory. This is the initial step for setting up the project locally.

```bash
git clone https://github.com/open-mmlab/Amphion.git
cd Amphion
```

--------------------------------

### DualCodec-VALLE Text-to-Speech CLI Inference (Bash)

Source: https://context7.com/open-mmlab/amphion/llms.txt

Provides instructions for installing the DualCodec TTS components and performing text-to-speech inference using the command-line interface. It requires reference audio and text to generate speech from the provided text.

```bash
# Install TTS components
pip install "dualcodec[tts]"

# CLI inference
python -m dualcodec.infer.valle.cli_valle_infer \
    --ref_audio ./reference.wav \
    --ref_text "TEXT OF YOUR REF AUDIO" \
    --gen_text "This is the text to synthesize" \
    --output_dir ./output \
    --output_file generated.wav
```

--------------------------------

### Vevo Voice Conversion Initialization in Python

Source: https://context7.com/open-mmlab/amphion/llms.txt

This Python code snippet initializes the Vevo inference pipeline for zero-shot timbre transfer voice conversion. It downloads necessary tokenizer components from HuggingFace and sets up the device for processing.

```python
import os
import torch
from huggingface_hub import snapshot_download
from models.vc.vevo.vevo_utils import VevoInferencePipeline, save_audio

device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

# Download Content-Style Tokenizer
local_dir = snapshot_download(
    repo_id="amphion/Vevo",
    repo_type="model",
    cache_dir="./ckpts/Vevo",
    allow_patterns=["tokenizer/vq8192/*"],
)
tokenizer_ckpt_path = os.path.join(local_dir, "tokenizer/vq8192")

```

--------------------------------

### Initialize DualCodec Inference Engine (Python)

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/example.ipynb

Initializes the DualCodec inference engine with specified model paths and device. This involves loading the DualCodec model and setting up the inference pipeline.

```python
dualcodec_model = dualcodec.get_model("25hz_v1", dualcodec_model_path) # using the 12hz_v1 model
inference = dualcodec.Inference(dualcodec_model=dualcodec_model, dualcodec_path=dualcodec_model_path, w2v_path=w2v_path, device='cpu')
```

--------------------------------

### Finetune DualCodec (12Hz V1)

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/README.md

Starts the finetuning process for a 12Hz V1 DualCodec model. This command uses `accelerate` for distributed training and assumes the necessary components and checkpoints are available. It configures batch size and data segmentation length.

```bash
accelerate launch train.py --config-name=dualcodec_ft_12hzv1 \
trainer.batch_size=3 \
data.segment_speech.segment_length=24000
```

--------------------------------

### MaskGCT TTS Inference with Faster English Output

Source: https://github.com/open-mmlab/amphion/blob/main/models/tts/maskgct/maskgct_demo.ipynb

This example demonstrates generating English speech with a reduced target duration, making the output audio faster. It utilizes the same maskgct_inference function but adjusts the 'target_len' parameter. The output shape of the predicted semantic is also shown.

```python
prompt_wav_path = "./models/tts/maskgct/wav/prompt.wav"
prompt_text = " We do not break. We never give in. We never back down."
target_text = "In this paper, we introduce MaskGCT, a fully non-autoregressive TTS model that eliminates the need for explicit alignment information between text and speech supervision."

# Specify the target duration (in seconds). If target_len = None, we use a simple rule to predict the target duration.
target_len = 12 # Make it faster

recovered_audio = maskgct_inference(
    prompt_wav_path,
    prompt_text,
    target_text,
    language="en",
    target_language="en",
    target_len=target_len
)

Audio(recovered_audio, rate=24000)
```

--------------------------------

### Run Flow-matching Transformer Content-Style Script (Bash)

Source: https://github.com/open-mmlab/amphion/blob/main/models/vc/vevo/README.md

This bash script initiates the voice conversion process using the Flow-matching Transformer model. It relies on the `fm_contentstyle.json` configuration file.

```bash
sh egs/vc/FlowMatchingTransformer/fm_contentstyle.sh
```

--------------------------------

### Resume or Finetune GAN Vocoder Training

Source: https://github.com/open-mmlab/amphion/blob/main/egs/vocoder/gan/README.md

Enables resuming training or finetuning a pre-trained GAN vocoder model. Uses the `run.sh` script with `--stage 2`, specifying the experiment name, resume type, and checkpoint path.

```bash
sh egs/vocoder/gan/{vocoder_name}/run.sh --stage 2 \
	--name [YourExptName] \
	--resume_type ["resume" for resuming training and "finetune" for loading parameters only] \
	--checkpoint Amphion/ckpts/vocoder/[YourExptName]/checkpoint \

```

--------------------------------

### Load and Use DualCodec Neural Audio Codec (Python)

Source: https://context7.com/open-mmlab/amphion/llms.txt

This snippet demonstrates how to load and use the DualCodec neural audio codec. It utilizes the `dualcodec` library to get a pre-trained model (e.g., '12hz_v1') and then initializes an inference object. The inference object can be used for audio encoding and decoding tasks, leveraging the model's ability to separate semantic and acoustic information.

```python
import dualcodec
import torchaudio

# Load model (automatically downloads from HuggingFace)
model_id = "12hz_v1"  # Options: "12hz_v1", "25hz_v1"
dualcodec_model = dualcodec.get_model(model_id)
dualcodec_inference = dualcodec.Inference(dualcodec_model=dualcodec_model, device="cuda")
```

--------------------------------

### Run Inference/Conversion with run.sh Script

Source: https://github.com/open-mmlab/amphion/blob/main/egs/svc/MultipleContentsSVC/README.md

This script executes inference or conversion tasks. It requires specifying experimental and output directories, source audio files or directories, the target speaker, and an optional key shift for pitch adjustment. The script is designed for shell execution.

```bash
sh egs/svc/MultipleContentsSVC/run.sh --stage 3 --gpu "0" \
	--infer_expt_dir ckpts/svc/[YourExptName] \
	--infer_output_dir ckpts/svc/[YourExptName]/result \
	--infer_source_audio_dir [Your Audios Folder] \
	--infer_target_speaker "opencpop_female1" \
	--infer_key_shift "autoshift"
```

--------------------------------

### Run Training Stage (Bash)

Source: https://github.com/open-mmlab/amphion/blob/main/egs/vocoder/diffusion/README.md

Initiates the training process for the diffusion-based vocoder. This command requires specifying the stage (`--stage 2`) and an experimental name (`--name`).

```bash
sh egs/vocoder/diffusion/{vocoder_name}/run.sh --stage 2 --name [YourExptName]
```

--------------------------------

### Load, Encode, and Decode Audio with DualCodec (Python)

Source: https://context7.com/open-mmlab/amphion/llms.txt

Demonstrates loading, resampling, and encoding audio to discrete tokens using torchaudio and DualCodec. It then shows decoding these tokens back to audio, both with and without acoustic codes for varying quality.

```python
import torchaudio
import torchaudio.functional as F

# Load and preprocess audio
audio, sr = torchaudio.load("input.wav")
audio = F.resample(audio, sr, 24000)
audio = audio.reshape(1, 1, -1).to("cuda")

# Encode audio to discrete tokens
# Assuming dualcodec_inference is imported and initialized elsewhere
# semantic_codes, acoustic_codes = dualcodec_inference.encode(audio, n_quantizers=8)
# semantic_codes shape: torch.Size([B, 1, T]) - RVQ-1 semantic tokens
# acoustic_codes shape: torch.Size([B, n_quantizers-1, T]) - RVQ-rest acoustic tokens

# Decode tokens back to audio
# out_audio = dualcodec_inference.decode(semantic_codes, acoustic_codes)
# torchaudio.save("reconstructed.wav", out_audio.cpu().squeeze(0), 24000)

# Decode only semantic codes (lower quality, semantic-only reconstruction)
# out_audio_semantic = dualcodec_inference.decode(semantic_codes, acoustic_codes=None)
# torchaudio.save("semantic_only.wav", out_audio_semantic.cpu().squeeze(0), 24000)
```

--------------------------------

### Resume Training From Specific Checkpoint (Bash)

Source: https://github.com/open-mmlab/amphion/blob/main/egs/tts/VITS/README.md

Resumes training from a specific checkpoint file. This allows for more granular control over the training resumption process, enabling recovery from a particular saved state. It uses the `--resume_from_ckpt_path` argument to specify the exact checkpoint file.

```bash
sh egs/tts/VITS/run.sh --stage 2 --name [YourExptName] \
    --resume true \
    --resume_from_ckpt_path "Amphion/ckpts/tts/[YourExptName]/checkpoint/[SpecificCheckpoint]"
```

--------------------------------

### Download DualCodec Checkpoints

Source: https://github.com/open-mmlab/amphion/blob/main/models/codec/dualcodec/README.md

Downloads necessary model checkpoints and statistics for DualCodec inference using the huggingface-cli. This is required for using local checkpoints.

```bash
# export HF_ENDPOINT=https://hf-mirror.com      # uncomment this to use huggingface mirror if you're in China
huggingface-cli download facebook/w2v-bert-2.0 --local-dir w2v-bert-2.0
huggingface-cli download amphion/dualcodec dualcodec_12hz_16384_4096.safetensors dualcodec_25hz_16384_1024.safetensors w2vbert2_mean_var_stats_emilia.pt --local-dir dualcodec_ckpts
```