### Install Dependencies with Python Virtual Environment

Source: https://github.com/moonintheriver/diffsinger/blob/master/README.md

Sets up a Python virtual environment and installs necessary packages including PyTorch, Cython, and NumPy. It also installs dependencies from a requirements.txt file. Ensure Python 3.8 is installed prior to running these commands.

```shell
## Install Python 3.8 first. 
python -m venv venv
source venv/bin/activate
# install requirements.
pip install -U pip
pip install Cython numpy==1.19.1
pip install torch==1.9.0
pip install -r requirements.txt
```

--------------------------------

### Environment Setup for DiffSinger

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Instructions for setting up the Python environment using Anaconda or virtualenv, including installing dependencies from requirements files tailored for different GPU setups.

```bash
# Option 1: Using Anaconda
conda create -n diffsinger python=3.8
source activate diffsinger
pip install -r requirements_2080.txt   # For GPU 2080Ti, CUDA 10.2
# OR
pip install -r requirements_3090.txt   # For GPU 3090, CUDA 11.4

# Option 2: Using Python virtualenv
python -m venv venv
source venv/bin/activate
pip install -U pip
pip install Cython numpy==1.19.1
pip install torch==1.9.0
pip install -r requirements.txt
```

--------------------------------

### DiffSinger Training Example

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-popcs.md

This command initiates the training process for the DiffSinger model. It requires a specific configuration file and an experiment name. Ensure the 'fs2_ckpt' parameter in the configuration file points to the correct FFT-Singer checkpoint.

```shell
CUDA_VISIBLE_DEVICES=0 python tasks/run.py --config usr/configs/popcs_ds_beta6_offline.yaml --exp_name popcs_ds_beta6_offline_pmf0_1230 --reset
```

--------------------------------

### Install Dependencies with Anaconda

Source: https://github.com/moonintheriver/diffsinger/blob/master/README.md

Sets up a Conda environment with Python 3.8 and installs project dependencies using a requirements file. Specific files are provided for different GPU configurations (e.g., 2080Ti with CUDA 10.2 or 3090 with CUDA 11.4).

```shell
conda create -n your_env_name python=3.8
source activate your_env_name 
pip install -r requirements_2080.txt   (GPU 2080Ti, CUDA 10.2)
or pip install -r requirements_3090.txt   (GPU 3090, CUDA 11.4)
```

--------------------------------

### Train FFT-Singer Model for DiffSinger

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-cascade.md

This command initiates the training of the FFT-Singer model, which serves as a prerequisite for training DiffSinger. It requires setting an environment variable for the experiment name and uses a specific configuration file. The `--reset` flag indicates that the experiment should be started from scratch.

```sh
export MY_FS_EXP_NAME=0302_opencpop_fs_midi
CUDA_VISIBLE_DEVICES=0 python tasks/run.py --config usr/configs/midi/cascade/opencs/aux_rel.yaml --exp_name $MY_FS_EXP_NAME --reset
```

--------------------------------

### DiffSinger Inference Example

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-popcs.md

This command performs inference using a trained DiffSinger model. It utilizes the same configuration file and experiment name as used during training. The `--infer` flag specifies that inference should be performed.

```shell
CUDA_VISIBLE_DEVICES=0 python tasks/run.py --config usr/configs/popcs_ds_beta6_offline.yaml --exp_name popcs_ds_beta6_offline_pmf0_1230 --reset --infer
```

--------------------------------

### Run DiffSinger Inference from Raw Inputs

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-cascade.md

This command executes the DiffSinger inference process using raw input data. It requires specifying the configuration file and experiment name. The inference can be performed with either word-level or phoneme-level inputs, as demonstrated in the provided examples.

```python
python inference/svs/ds_cascade.py --config usr/configs/midi/cascade/opencs/ds60_rel.yaml --exp_name $MY_DS_EXP_NAME
```

--------------------------------

### FFT-Singer Training and Inference

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-popcs.md

These commands demonstrate the process of training and then inferring with a pre-trained FFT-Singer model. It requires a configuration file and an experiment name. The `--reset` flag is used to clear previous experiment data.

```shell
# First, train fft-singer;
CUDA_VISIBLE_DEVICES=0 python tasks/run.py --config usr/configs/popcs_fs2.yaml --exp_name popcs_fs2_pmf0_1230 --reset
# Then, infer fft-singer;
CUDA_VISIBLE_DEVICES=0 python tasks/run.py --config usr/configs/popcs_fs2.yaml --exp_name popcs_fs2_pmf0_1230 --reset --infer
```

--------------------------------

### Prepare Opencpop Dataset for DiffSinger Training

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-cascade.md

This script prepares the Opencpop dataset for DiffSinger training by binarizing the data. It requires the Opencpop dataset to be downloaded and linked, and uses a specified configuration file. The output is a binarized dataset ready for model training.

```sh
export PYTHONPATH=.
CUDA_VISIBLE_DEVICES=0 python data_gen/tts/bin/binarize.py --config usr/configs/midi/cascade/opencs/aux_rel.yaml
```

--------------------------------

### Prepare and Binarize Dataset

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-pndm.md

Commands to link the dataset and run the binarization script to prepare data for training.

```bash
ln -s /xxx/opencpop data/raw/
export PYTHONPATH=.
CUDA_VISIBLE_DEVICES=0 python data_gen/tts/bin/binarize.py --config usr/configs/midi/cascade/opencs/aux_rel.yaml
```

--------------------------------

### Data Preparation for DiffSinger Training

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-popcs.md

This script prepares the dataset for training DiffSinger. It involves creating a symbolic link to the dataset and then binarizing it using a specified configuration file. The output is a binary dataset ready for model training.

```shell
export PYTHONPATH=.
CUDA_VISIBLE_DEVICES=0 python data_gen/tts/bin/binarize.py --config usr/configs/popcs_ds_beta6.yaml
```

--------------------------------

### Gradio Web Interface for Singing Voice Synthesis

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Launches an interactive Gradio web interface for demonstrating singing voice synthesis. It loads configurations from a YAML file and allows users to input text, MIDI notes, and durations to generate audio.

```python
# inference/svs/gradio/infer.py
import yaml
from inference.svs.gradio.infer import GradioInfer

# Load Gradio configuration
gradio_config = yaml.safe_load(open('inference/svs/gradio/gradio_settings.yaml'))

# Create and run interface
g = GradioInfer(**gradio_config)
g.run()

# Interface accepts:
# - input text: Chinese lyrics
# - input note: MIDI notes separated by |
# - input duration: note durations in seconds
```

```bash
# Launch Gradio interface for cascade model
python inference/svs/gradio/infer.py \
    --config usr/configs/midi/cascade/opencs/ds60_rel.yaml \
    --exp_name 0303_opencpop_ds58_midi

# Launch for E2E model
CUDA_VISIBLE_DEVICES=0 python inference/svs/gradio/infer.py \
    --config usr/configs/midi/e2e/opencpop/ds100_adj_rel.yaml \
    --exp_name 0228_opencpop_ds100_rel
```

--------------------------------

### Training DiffSpeech (TTS)

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Instructions for training DiffSpeech, a text-to-speech model that utilizes shallow diffusion initialized from a pre-trained FastSpeech2 checkpoint. Configuration details for hyperparameters are provided.

```bash
# Ensure fs2_ckpt path is set in usr/configs/lj_ds_beta6.yaml:
# fs2_ckpt: checkpoints/fs2_lj_1/model_ckpt_steps_150000.ckpt

# Train DiffSpeech
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config usr/configs/lj_ds_beta6.yaml \
    --exp_name lj_ds_beta6_1213 \
    --reset

# Key hyperparameters in config:
# timesteps: 100          # Total diffusion steps
# K_step: 71              # Steps for shallow diffusion (starts from K_step instead of 0)
# diff_loss_type: l1      # Loss function for diffusion
# diff_decoder_type: wavenet  # Denoiser architecture
# schedule_type: linear   # Beta schedule for diffusion
# max_beta: 0.06          # Maximum beta value
```

--------------------------------

### Initialize GaussianDiffusion Model

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Configures the core shallow diffusion model for mel-spectrogram generation, wrapping a denoiser network and defining diffusion parameters.

```python
from usr.diff.shallow_diffusion_tts import GaussianDiffusion
from usr.diff.net import DiffNet
from utils.text_encoder import TokenTextEncoder

model = GaussianDiffusion(
    phone_encoder=phone_encoder,
    out_dims=80,
    denoise_fn=DiffNet(80),
    timesteps=100,
    K_step=71,
    loss_type='l1',
    spec_min=[-5.0] * 80,
    spec_max=[0.5] * 80
)
```

--------------------------------

### Custom Singing Voice Synthesis Inference Implementation

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Extends the BaseSVSInfer class to create a custom inference pipeline for singing voice synthesis. This involves defining methods for building the acoustic model, processing inputs, and running the vocoder to generate audio.

```python
from inference.svs.base_svs_infer import BaseSVSInfer
from utils.hparams import set_hparams, hparams

class CustomSVSInfer(BaseSVSInfer):
    def build_model(self):
        # Load and return your acoustic model
        model = GaussianDiffusion(...)
        load_ckpt(model, hparams['work_dir'], 'model')
        return model

    def forward_model(self, inp):
        # Process input and generate audio
        sample = self.input_to_batch(inp)
        with torch.no_grad():
            output = self.model(sample['txt_tokens'], ...)
            mel_out = output['mel_out']
            f0_pred = output['f0_denorm']
            wav_out = self.run_vocoder(mel_out, f0=f0_pred)
        return wav_out.cpu().numpy()[0]

# Usage
set_hparams(print_hparams=False)
infer = CustomSVSInfer(hparams)

# Preprocess input
item = infer.preprocess_input({
    'text': '你好世界',
    'notes': 'C4 | D4 | E4 | F4',
    'notes_duration': '0.5 | 0.5 | 0.5 | 0.5',
    'input_type': 'word'
})

# Run inference
wav = infer.infer_once(inp)
```

--------------------------------

### Training FastSpeech2 Baseline Models

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Commands to train the FastSpeech2 acoustic model, which serves as a baseline and is a prerequisite for training DiffSpeech and DiffSinger. Supports training for both TTS and SVS tasks.

```bash
# Train FastSpeech2 on LJSpeech for TTS
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config configs/tts/lj/fs2.yaml \
    --exp_name fs2_lj_1 \
    --reset

# Train FFT-Singer (FastSpeech2 variant) on Opencpop for SVS
export MY_FS_EXP_NAME=0302_opencpop_fs_midi
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config usr/configs/midi/cascade/opencs/aux_rel.yaml \
    --exp_name $MY_FS_EXP_NAME \
    --reset
```

--------------------------------

### Hyperparameter Configuration Management

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Manages project hyperparameters using a hierarchical YAML configuration system that supports inheritance and command-line overrides. This allows flexible customization of model training and inference settings.

```python
from utils.hparams import set_hparams, hparams

# Load config from YAML file
set_hparams(config='usr/configs/lj_ds_beta6.yaml')

# Access hyperparameters
print(hparams['audio_sample_rate'])  # 22050
print(hparams['audio_num_mel_bins'])  # 80
print(hparams['timesteps'])           # 100
print(hparams['K_step'])              # 71

# Override via command line
# python tasks/run.py --config config.yaml --hparams "lr=0.0001,batch_size=32"
```

```yaml
# Example config: usr/configs/lj_ds_beta6.yaml
base_config:
  - configs/tts/lj/fs2.yaml  # Inherit from base config
  - ./base.yaml

task_cls: usr.diffspeech_task.DiffSpeechTask
vocoder: vocoders.hifigan.HifiGAN
vocoder_ckpt: checkpoints/0414_hifi_lj_1

# Diffusion parameters
timesteps: 100
K_step: 71
diff_loss_type: l1
diff_decoder_type: 'wavenet'
schedule_type: 'linear'
max_beta: 0.06

# Training parameters
max_updates: 160000
lr: 0.001

# FastSpeech2 checkpoint for shallow diffusion
fs2_ckpt: checkpoints/fs2_lj_1/model_ckpt_steps_150000.ckpt
```

--------------------------------

### Training DiffSinger (SVS) - End-to-End Version

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Instructions for training the end-to-end version of DiffSinger, which jointly generates mel-spectrograms with implicit pitch modeling, avoiding explicit F0 prediction for more natural pitch contours.

```bash
export MY_DS_EXP_NAME=0228_opencpop_ds100_rel

# Train DiffSinger E2E (no separate F0 prediction)
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config usr/configs/midi/e2e/opencpop/ds100_adj_rel.yaml \
    --exp_name $MY_DS_EXP_NAME \
    --reset
```

--------------------------------

### Run DiffSinger Inference via Command Line

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-e2e.md

Executes the end-to-end inference script using a specified configuration file and experiment name. The output is saved to the ./infer_out directory by default.

```bash
python inference/svs/ds_e2e.py --config usr/configs/midi/e2e/opencpop/ds100_adj_rel.yaml --exp_name $MY_DS_EXP_NAME
```

--------------------------------

### Inference with DiffSinger from Raw Inputs (Phoneme Type)

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-cascade.md

This script performs DiffSinger inference using raw input data specified as phoneme sequences, note sequences, and durations. The input type is set to 'phoneme'. This format is compatible with the Opencpop dataset structure. Results are saved to a default output directory.

```python
inp = {
        'text': '小酒窝长睫毛AP是你最美的记号',
        'ph_seq': 'x iao j iu w o ch ang ang j ie ie m ao AP sh i n i z ui m ei d e j i h ao',
        'note_seq': 'C#4/Db4 C#4/Db4 F#4/Gb4 F#4/Gb4 G#4/Ab4 G#4/Ab4 A#4/Bb4 A#4/Bb4 F#4/Gb4 F#4/Gb4 F#4/Gb4 C#4/Db4 C#4/Db4 C#4/Db4 rest C#4/Db4 C#4/Db4 A#4/Bb4 A#4/Bb4 G#4/Ab4 G#4/Ab4 A#4/Bb4 A#4/Bb4 G#4/Ab4 G#4/Ab4 F4 F4 C#4/Db4 C#4/Db4',
        'note_dur_seq': '0.407140 0.407140 0.376190 0.376190 0.242180 0.242180 0.509550 0.509550 0.183420 0.315400 0.315400 0.235020 0.361660 0.361660 0.223070 0.377270 0.377270 0.340550 0.340550 0.299620 0.299620 0.344510 0.344510 0.283770 0.283770 0.323390 0.323390 0.360340 0.360340',
        'is_slur_seq': '0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0',
        'input_type': 'phoneme'
    }  # input like Opencpop dataset.
```

--------------------------------

### Run Inference from Raw Inputs

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-pndm.md

Python script execution for inference from raw text or phoneme inputs.

```bash
python inference/svs/ds_e2e.py --config usr/configs/midi/e2e/opencpop/ds1000.yaml --exp_name $MY_DS_EXP_NAME
```

```python
inp = {
    'text': '小酒窝长睫毛AP是你最美的记号',
    'notes': 'C#4/Db4 | F#4/Gb4 | G#4/Ab4 | A#4/Bb4 F#4/Gb4 | F#4/Gb4 C#4/Db4 | C#4/Db4 | rest | C#4/Db4 | A#4/Bb4 | G#4/Ab4 | A#4/Bb4 | G#4/Ab4 | F4 | C#4/Db4',
    'notes_duration': '0.407140 | 0.376190 | 0.242180 | 0.509550 0.183420 | 0.315400 0.235020 | 0.361660 | 0.223070 | 0.377270 | 0.340550 | 0.299620 | 0.344510 | 0.283770 | 0.323390 | 0.360340',
    'input_type': 'word'
}
```

--------------------------------

### Inference from Test Set

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Placeholder for commands to run inference on the test dataset, typically used for generating audio samples and evaluating model performance after training.

```bash
# Commands for inference will be added here.
```

--------------------------------

### PNDM Acceleration for Faster Diffusion Inference

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Enables Pseudo Linear Multi-Step (PNDM) acceleration to speed up inference by reducing the number of diffusion steps required. This involves configuring the PNDM speedup factor and adjusting beta parameters in the configuration file.

```yaml
# usr/configs/lj_ds_pndm.yaml
base_config:
  - ./lj_ds_beta6.yaml

# Enable PNDM speedup
pndm_speedup: 10  # Reduce steps by factor of 10
max_beta: 0.02    # Adjusted for PNDM
```

```bash
# Train with PNDM
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config usr/configs/lj_ds_pndm.yaml \
    --exp_name lj_ds_pndm \
    --reset
```

--------------------------------

### Train and Infer with DiffSinger

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-pndm.md

Commands for training the model and running inference from a packed test set.

```bash
export MY_DS_EXP_NAME=0831_opencpop_ds1000
CUDA_VISIBLE_DEVICES=0 python tasks/run.py --config usr/configs/midi/e2e/opencpop/ds1000.yaml --exp_name $MY_DS_EXP_NAME --reset
CUDA_VISIBLE_DEVICES=0 python tasks/run.py --config usr/configs/midi/e2e/opencpop/ds1000.yaml --exp_name $MY_DS_EXP_NAME --reset --infer
```

--------------------------------

### Train DiffSinger Model

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-cascade.md

This command trains the DiffSinger model. It depends on a pre-trained FFT-Singer model and uses a specific configuration file. Ensure the 'fs2_ckpt' parameter in the configuration file points to the correct FFT-Singer checkpoint. The `--reset` flag restarts the training.

```sh
export MY_DS_EXP_NAME=0303_opencpop_ds58_midi
CUDA_VISIBLE_DEVISES=0 python tasks/run.py --config usr/configs/midi/cascade/opencs/ds60_rel.yaml --exp_name $MY_DS_EXP_NAME --reset
```

--------------------------------

### Training DiffSinger (SVS) - Cascade Version

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Steps to train the cascade version of DiffSinger for singing voice synthesis. This approach involves a melody frontend for explicit F0 and duration prediction, followed by a diffusion acoustic model.

```bash
export MY_DS_EXP_NAME=0303_opencpop_ds58_midi

# Ensure fs2_ckpt path is set in usr/configs/midi/cascade/opencs/ds60_rel.yaml

# Train DiffSinger Cascade
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config usr/configs/midi/cascade/opencs/ds60_rel.yaml \
    --exp_name $MY_DS_EXP_NAME \
    --reset
```

--------------------------------

### HiFiGAN Vocoder for Mel-Spectrogram to Audio Conversion

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Utilizes the HiFiGAN vocoder to convert mel-spectrograms into audio waveforms. It supports standard conversion and NSF (Neural Source Filter) for singing voice with optional F0 conditioning. Requires vocoder checkpoints and configuration files.

```python
from vocoders.hifigan import HifiGAN
from utils.hparams import hparams

# Initialize vocoder (loads from checkpoint automatically)
# Requires: checkpoints/{vocoder_ckpt}/config.yaml and model_ckpt_steps_*.ckpt
vocoder = HifiGAN()

# Convert mel-spectrogram to waveform
mel = generated_mel  # numpy array [T, 80]
f0 = predicted_f0    # numpy array [T] (optional, for NSF vocoder)

# Without F0 (standard HiFiGAN)
wav = vocoder.spec2wav(mel)

# With F0 (NSF-HiFiGAN for singing)
wav = vocoder.spec2wav(mel, f0=f0)

# Save audio
from utils.audio import save_wav
save_wav(wav, 'output.wav', hparams['audio_sample_rate'])
```

--------------------------------

### Inference with DiffSinger from Raw Inputs (Word Type)

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-cascade.md

This script performs DiffSinger inference using raw input data specified as text, notes, and note durations. The input type is set to 'word'. The inference results are saved to a default output directory.

```python
inp = {
        'text': '小酒窝长睫毛AP是你最美的记号',
        'notes': 'C#4/Db4 | F#4/Gb4 | G#4/Ab4 | A#4/Bb4 F#4/Gb4 | F#4/Gb4 C#4/Db4 | C#4/Db4 | rest | C#4/Db4 | A#4/Bb4 | G#4/Ab4 | A#4/Bb4 | G#4/Ab4 | F4 | C#4/Db4',
        'notes_duration': '0.407140 | 0.376190 | 0.242180 | 0.509550 0.183420 | 0.315400 0.235020 | 0.361660 | 0.223070 | 0.377270 | 0.340550 | 0.299620 | 0.344510 | 0.283770 | 0.323390 | 0.360340',
        'input_type': 'word'
    }  # user input: Chinese characters
```

--------------------------------

### Run DiffSinger End-to-End Inference

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Performs singing voice synthesis using the end-to-end pipeline, which uses implicit pitch prediction for more natural results.

```python
from inference.svs.ds_e2e import DiffSingerE2EInfer

inp = {
    'text': '小酒窝长睫毛AP是你最美的记号',
    'notes': 'C#4/Db4 | F#4/Gb4 | G#4/Ab4 | A#4/Bb4 F#4/Gb4 | F#4/Gb4 C#4/Db4 | C#4/Db4 | rest | C#4/Db4 | A#4/Bb4 | G#4/Ab4 | A#4/Bb4 | G#4/Ab4 | F4 | C#4/Db4',
    'notes_duration': '0.407140 | 0.376190 | 0.242180 | 0.509550 0.183420 | 0.315400 0.235020 | 0.361660 | 0.223070 | 0.377270 | 0.340550 | 0.299620 | 0.344510 | 0.283770 | 0.323390 | 0.360340',
    'input_type': 'word'
}

DiffSingerE2EInfer.example_run(inp)
```

```bash
python inference/svs/ds_e2e.py \
    --config usr/configs/midi/e2e/opencpop/ds100_adj_rel.yaml \
    --exp_name 0228_opencpop_ds100_rel
```

--------------------------------

### Inference with DiffSinger using Packed Test Set

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-cascade.md

Performs inference using the trained DiffSinger model on a packed test set. The results are saved by default in the 'generated_' subdirectory of the experiment's checkpoint folder. This command requires the experiment name and configuration file to be set.

```sh
CUDA_VISIBLE_DEVICES=0 python tasks/run.py --config usr/configs/midi/cascade/opencs/ds60_rel.yaml --exp_name $MY_DS_EXP_NAME --reset --infer
```

--------------------------------

### Data Binarization for TTS and SVS

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Scripts to binarize datasets for DiffSpeech (TTS) and DiffSinger (SVS). This process converts raw audio and text into a format suitable for model training, extracting features like mel-spectrograms and pitch.

```bash
# Set Python path
export PYTHONPATH=.

# Binarize LJSpeech dataset for TTS (DiffSpeech)
CUDA_VISIBLE_DEVICES=0 python data_gen/tts/bin/binarize.py --config configs/tts/lj/fs2.yaml
# Output: data/binary/ljspeech

# Binarize Opencpop dataset for SVS (DiffSinger)
CUDA_VISIBLE_DEVICES=0 python data_gen/tts/bin/binarize.py --config usr/configs/midi/cascade/opencs/aux_rel.yaml
# Output: data/binary/opencpop-midi-dp
```

--------------------------------

### Run DiffSinger Cascade Inference

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Performs singing voice synthesis using the cascade pipeline, which requires explicit F0 prediction. Supports both word-level and phoneme-level input formats.

```python
from inference.svs.ds_cascade import DiffSingerCascadeInfer

inp_word = {
    'text': '小酒窝长睫毛AP是你最美的记号',
    'notes': 'C#4/Db4 | F#4/Gb4 | G#4/Ab4 | A#4/Bb4 F#4/Gb4 | F#4/Gb4 C#4/Db4 | C#4/Db4 | rest | C#4/Db4 | A#4/Bb4 | G#4/Ab4 | A#4/Bb4 | G#4/Ab4 | F4 | C#4/Db4',
    'notes_duration': '0.407140 | 0.376190 | 0.242180 | 0.509550 0.183420 | 0.315400 0.235020 | 0.361660 | 0.223070 | 0.377270 | 0.340550 | 0.299620 | 0.344510 | 0.283770 | 0.323390 | 0.360340',
    'input_type': 'word'
}

DiffSingerCascadeInfer.example_run(inp_word)
```

```bash
python inference/svs/ds_cascade.py \
    --config usr/configs/midi/cascade/opencs/ds60_rel.yaml \
    --exp_name $MY_DS_EXP_NAME
```

--------------------------------

### Define DiffSinger Input Data Structures

Source: https://github.com/moonintheriver/diffsinger/blob/master/docs/README-SVS-opencpop-e2e.md

Defines the dictionary structures required for inference, supporting either word-level input with MIDI notes or phoneme-level input with explicit sequences.

```python
inp_word = {
    'text': '小酒窝长睫毛AP是你最美的记号',
    'notes': 'C#4/Db4 | F#4/Gb4 | G#4/Ab4 | A#4/Bb4 F#4/Gb4 | F#4/Gb4 C#4/Db4 | C#4/Db4 | rest | C#4/Db4 | A#4/Bb4 | G#4/Ab4 | A#4/Bb4 | G#4/Ab4 | F4 | C#4/Db4',
    'notes_duration': '0.407140 | 0.376190 | 0.242180 | 0.509550 0.183420 | 0.315400 0.235020 | 0.361660 | 0.223070 | 0.377270 | 0.340550 | 0.299620 | 0.344510 | 0.283770 | 0.323390 | 0.360340',
    'input_type': 'word'
}

inp_phoneme = {
    'text': '小酒窝长睫毛AP是你最美的记号',
    'ph_seq': 'x iao j iu w o ch ang ang j ie ie m ao AP sh i n i z ui m ei d e j i h ao',
    'note_seq': 'C#4/Db4 C#4/Db4 F#4/Gb4 F#4/Gb4 G#4/Ab4 G#4/Ab4 A#4/Bb4 A#4/Bb4 F#4/Gb4 F#4/Gb4 F#4/Gb4 C#4/Db4 C#4/Db4 C#4/Db4 rest C#4/Db4 C#4/Db4 A#4/Bb4 A#4/Bb4 G#4/Ab4 G#4/Ab4 A#4/Bb4 A#4/Bb4 G#4/Ab4 G#4/Ab4 F4 F4 C#4/Db4 C#4/Db4',
    'note_dur_seq': '0.407140 0.407140 0.376190 0.376190 0.242180 0.242180 0.509550 0.509550 0.183420 0.315400 0.315400 0.235020 0.361660 0.361660 0.223070 0.377270 0.377270 0.340550 0.340550 0.299620 0.299620 0.344510 0.344510 0.283770 0.283770 0.323390 0.323390 0.360340 0.360340',
    'is_slur_seq': '0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0',
    'input_type': 'phoneme'
}
```

--------------------------------

### DiffSinger Model Forward Pass for Inference

Source: https://context7.com/moonintheriver/diffsinger/llms.txt

Performs a forward pass of the DiffSinger model in inference mode to generate mel-spectrograms and F0 curves. Inputs include tokenized text, mel-to-phoneme alignment, speaker embeddings, and F0 information.

```python
output = model(
    txt_tokens=txt_tokens,
    mel2ph=mel2ph,
    spk_embed=spk_embed,
    ref_mels=None,  # no reference for inference
    f0=f0,
    uv=uv,
    infer=True      # inference mode
)
# output['mel_out']: generated mel-spectrogram [B, T_mel, 80]
# output['f0_denorm']: predicted F0 curve
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.