### Download ProDiff Checkpoints

Source: https://github.com/rongjiehuang/prodiff/blob/main/readme.md

Use this command to download pretrained ProDiff model weights from Hugging Face. Ensure you have the `huggingface_hub` library installed. The downloaded checkpoints should be moved to the `checkpoints/$Model/` directory.

```python
from huggingface_hub import snapshot_download 
downloaded_path = snapshot_download(repo_id="Rongjiehuang/ProDiff")
```

```bash
mv ${downloaded_path}/checkpoints/  checkpoints/
```

--------------------------------

### Multi-GPU Training Setup

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Configures distributed training across multiple GPUs for faster model training. Uses CUDA_VISIBLE_DEVICES to specify GPUs and PyTorch automatically handles DataParallel.

```bash
# Use specific GPUs
CUDA_VISIBLE_DEVICES=0,1,2,3 python tasks/run.py \
    --config modules/ProDiff/config/prodiff_teacher.yaml \
    --exp_name ProDiff_Teacher_MultiGPU \
    --reset

# PyTorch automatically uses DataParallel with torch.cuda.device_count() GPUs

```

--------------------------------

### Define ProDiff Configuration

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Example YAML configuration for defining model architecture, data paths, and diffusion parameters.

```yaml
# modules/ProDiff/config/prodiff.yaml
base_config:
  - ./base.yaml

# Data directories
raw_data_dir: 'data/raw/LJSpeech'
processed_data_dir: 'data/processed/LJSpeech'
binary_data_dir: 'data/binary/LJSpeech'

# Task class for training
task_cls: modules.ProDiff.task.ProDiff_task.ProDiff_Task

# Diffusion parameters
timesteps: 4                    # Student model diffusion steps
teacher_ckpt: checkpoints/ProDiff_Teacher/model_ckpt_steps_188000.ckpt
diff_decoder_type: 'wavenet'
schedule_type: 'vpsde'          # VP-SDE noise schedule

# Vocoder settings (FastDiff)
vocoder: FastDiff
N: 4                            # Vocoder denoising steps
vocoder_ckpt: checkpoints/FastDiff
```

--------------------------------

### Launch Gradio Web Interface

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Launches an interactive web demo for text-to-speech synthesis using Gradio. Opens a shareable link for the web-based TTS demo.

```bash
python inference/gradio/infer.py

# Opens shareable link for web-based TTS demo
```

--------------------------------

### Run Data Preprocessing Pipeline

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Executes the multi-step pipeline to prepare raw audio datasets, including alignment and binarization.

```bash
# Step 1: Preprocess - Unify file structure and extract metadata
python data_gen/tts/bin/pre_align.py --config modules/ProDiff/config/prodiff_teacher.yaml

# Step 2: Forced Alignment - Run Montreal Forced Aligner (MFA)
python data_gen/tts/runs/train_mfa_align.py --config modules/ProDiff/config/prodiff_teacher.yaml

# Step 3: Binarization - Create binary data files for fast I/O
CUDA_VISIBLE_DEVICES=0 python data_gen/tts/bin/binarize.py \
    --config modules/ProDiff/config/prodiff_teacher.yaml
```

--------------------------------

### Initialize and Run ProDiff Model

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Demonstrates how to instantiate the GaussianDiffusion model and perform forward passes for both training and inference.

```python
# Initialize the GaussianDiffusion model
model = GaussianDiffusion(
    phone_encoder=phone_encoder,      # Phoneme vocabulary encoder
    out_dims=80,                       # Mel-spectrogram bins
    denoise_fn=DiffNet(80),           # WaveNet-based denoising network
    timesteps=4,                       # Number of diffusion steps
    time_scale=1,                      # Time scaling factor
    loss_type='l1',                   # Loss function (l1 or l2)
    spec_min=hparams['spec_min'],     # Spectrogram normalization min
    spec_max=hparams['spec_max'],     # Spectrogram normalization max
)

# Forward pass for training
txt_tokens = torch.LongTensor([[1, 2, 3, 4, 5]]).cuda()  # Phoneme tokens
mel2ph = torch.LongTensor([[1, 1, 2, 2, 3, 3, 4, 5, 5]]).cuda()  # Mel-to-phone alignment
ref_mels = torch.randn(1, 9, 80).cuda()  # Target mel-spectrogram

output = model(
    txt_tokens,
    mel2ph=mel2ph,
    ref_mels=ref_mels,
    infer=False  # Training mode
)
loss = output['mel_out']  # Diffusion loss

# Forward pass for inference
output = model(txt_tokens, infer=True)
mel_spectrogram = output['mel_out']  # Generated mel-spectrogram [B, T, 80]
```

--------------------------------

### Preprocess Dataset for Training

Source: https://github.com/rongjiehuang/prodiff/blob/main/readme.md

This command initiates the dataset preprocessing step, which unifies file structures before training. Ensure that `raw_data_dir`, `processed_data_dir`, and `binary_data_dir` are correctly set in your configuration file, and the dataset is downloaded to `raw_data_dir`.

```bash
# Preprocess step: unify the file structure.
python data_gen/tts/bin/pre_align.py --config $path/to/config
```

--------------------------------

### Download ProDiff Pretrained Models

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Use `snapshot_download` from Hugging Face Hub to download all necessary pretrained checkpoints for ProDiff, ProDiff_Teacher, and FastDiff.

```python
from huggingface_hub import snapshot_download

# Download all pretrained models (ProDiff, ProDiff_Teacher, FastDiff)
downloaded_path = snapshot_download(repo_id="Rongjiehuang/ProDiff")

# Move checkpoints to expected location
# checkpoints/ProDiff/model_ckpt_steps_*.ckpt
# checkpoints/ProDiff_Teacher/model_ckpt_steps_*.ckpt
# checkpoints/FastDiff/model_ckpt_steps_*.ckpt
```

--------------------------------

### Train ProDiff Teacher and Student Models

Source: https://github.com/rongjiehuang/prodiff/blob/main/readme.md

Commands to initiate training for the teacher and student ProDiff models.

```bash
CUDA_VISIBLE_DEVICES=$GPU python tasks/run.py --config modules/ProDiff/config/prodiff_teacher.yaml  --exp_name ProDiff_Teacher --reset
```

```bash
CUDA_VISIBLE_DEVICES=$GPU python tasks/run.py --config modules/ProDiff/config/prodiff.yaml  --exp_name ProDiff --reset
```

--------------------------------

### Run ProDiff Teacher for Speech Synthesis (High Quality)

Source: https://github.com/rongjiehuang/prodiff/blob/main/readme.md

Use this command for higher quality speech synthesis with the ProDiff Teacher model, employing 4-iteration ProDiff Teacher and 6-iteration FastDiff vocoder. Specify your GPU ID with `$GPU` and input text with `$txt`. Output is saved in `infer_out`.

```bash
CUDA_VISIBLE_DEVICES=$GPU python inference/ProDiff_teacher.py --config modules/ProDiff/config/prodiff_teacher.yaml --exp_name ProDiff_Teacher --hparams="N=6,text='$txt'" --reset
```

--------------------------------

### Run MFA Alignment and Binarization

Source: https://github.com/rongjiehuang/prodiff/blob/main/readme.md

Execute the alignment and binarization scripts to prepare data for training.

```bash
python data_gen/tts/runs/train_mfa_align.py --config $CONFIG_NAME
CUDA_VISIBLE_DEVICES=$GPU python data_gen/tts/bin/binarize.py --config $path/to/config
```

--------------------------------

### Train ProDiff Student Model

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Command to initiate training for the distilled student model.

```bash
# Train ProDiff student (requires trained teacher checkpoint)
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config modules/ProDiff/config/prodiff.yaml \
    --exp_name ProDiff \
    --reset
```

--------------------------------

### Run ProDiff for Speech Synthesis (Fast)

Source: https://github.com/rongjiehuang/prodiff/blob/main/readme.md

Execute this command for fast speech synthesis using the ProDiff model with a 2-iteration ProDiff and 4-iteration FastDiff vocoder. Replace `$GPU` with your GPU ID and `$txt` with the input text. Generated audio files are saved in the `infer_out` directory.

```bash
CUDA_VISIBLE_DEVICES=$GPU python inference/ProDiff.py --config modules/ProDiff/config/prodiff.yaml --exp_name ProDiff --hparams="N=4,text='$txt'" --reset
```

--------------------------------

### Load FastDiff Neural Vocoder

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Initializes the FastDiff vocoder for converting mel-spectrograms to audio.

```python
from vocoders.fastdiff import FastDiff, load_fastdiff_model
from utils.hparams import hparams
import torch

# Load FastDiff vocoder
model, diffusion_hyperparams, noise_schedule, config, device = load_fastdiff_model(
    config_path='checkpoints/FastDiff/config.yaml',
    checkpoint_path='checkpoints/FastDiff/model_ckpt_steps_400000.ckpt'
)
```

--------------------------------

### Checkpoint Loading Utilities

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Loads model weights from checkpoints with flexible path and state dict handling. Retrieves the latest or all checkpoints and loads them into a model.

```python
from utils.ckpt_utils import load_ckpt, get_last_checkpoint, get_all_ckpts

# Get latest checkpoint from directory
checkpoint, ckpt_path = get_last_checkpoint('checkpoints/ProDiff')
# Returns: (checkpoint_dict, 'checkpoints/ProDiff/model_ckpt_steps_100000.ckpt')

# Get all checkpoints sorted by step (descending)
all_ckpts = get_all_ckpts('checkpoints/ProDiff')
# Returns: ['model_ckpt_steps_100000.ckpt', 'model_ckpt_steps_50000.ckpt', ...]

# Load checkpoint into model
load_ckpt(
    cur_model=model,
    ckpt_base_dir='checkpoints/ProDiff',
    model_name='model',     # Key in state_dict
    force=True,             # Raise error if not found
    strict=True             # Require exact parameter match
)

# Load with flexible matching (ignores shape mismatches)
load_ckpt(model, 'checkpoints/ProDiff', 'model', strict=False)
```

--------------------------------

### Move Downloaded Checkpoints

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

After downloading, move the checkpoints from the snapshot directory to the project's expected 'checkpoints/' directory.

```bash
# Move downloaded checkpoints to project structure
mv ${downloaded_path}/checkpoints/ checkpoints/
```

--------------------------------

### Perform Inference with ProDiff Models

Source: https://github.com/rongjiehuang/prodiff/blob/main/readme.md

Commands to run inference using the trained teacher or student models.

```bash
CUDA_VISIBLE_DEVICES=$GPU python tasks/run.py --config modules/ProDiff/config/prodiff_teacher.yaml  --exp_name ProDiff_Teacher --infer
```

```bash
CUDA_VISIBLE_DEVICES=$GPU python tasks/run.py --config modules/ProDiff/config/prodiff.yaml  --exp_name ProDiff --infer
```

--------------------------------

### Run Batch Inference

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Commands for running batch inference on test sets for both student and teacher models.

```bash
# Inference with ProDiff (student model)
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config modules/ProDiff/config/prodiff.yaml \
    --exp_name ProDiff \
    --infer

# Inference with ProDiff Teacher
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config modules/ProDiff/config/prodiff_teacher.yaml \
    --exp_name ProDiff_Teacher \
    --infer
```

--------------------------------

### Train ProDiff Teacher Model

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Commands to train, resume, or validate the teacher diffusion model.

```bash
# Train ProDiff Teacher from scratch
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config modules/ProDiff/config/prodiff_teacher.yaml \
    --exp_name ProDiff_Teacher \
    --reset

# Resume training from checkpoint
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config modules/ProDiff/config/prodiff_teacher.yaml \
    --exp_name ProDiff_Teacher

# Run validation during training
CUDA_VISIBLE_DEVICES=0 python tasks/run.py \
    --config modules/ProDiff/config/prodiff_teacher.yaml \
    --exp_name ProDiff_Teacher \
    --validate
```

--------------------------------

### High-Quality Inference with ProDiff Teacher

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Generate higher quality speech using the ProDiff Teacher model with more diffusion iterations for both the acoustic model and the vocoder. This provides better audio fidelity at the cost of slightly longer inference time.

```bash
# Use ProDiff Teacher for better quality (4-iter acoustic + 6-iter vocoder)
CUDA_VISIBLE_DEVICES=0 python inference/ProDiff_teacher.py \
    --config modules/ProDiff/config/prodiff_teacher.yaml \
    --exp_name ProDiff_Teacher \
    --hparams="N=6,text='ProDiff enables extremely fast text to speech synthesis.'" \
    --reset

# Output saved to: infer_out/ProDiff enables extremely fast text to speech synthesis..wav
```

--------------------------------

### Audio Utilities: Save WAV

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Saves synthesized waveforms to audio files in WAV format. Supports normalization and specifies sample rate.

```python
from utils.audio import save_wav
import numpy as np

# Save waveform to WAV file
waveform = np.random.randn(22050)  # 1 second at 22050 Hz
save_wav(
    wav=waveform,
    path='output.wav',
    sr=22050,           # Sample rate
    norm=True           # Normalize to [-1, 1] before saving
)
```

--------------------------------

### Quick Inference with ProDiff

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Perform fast speech synthesis using the distilled ProDiff model with the FastDiff vocoder. Specify GPU, configuration, experiment name, and synthesis parameters like text and diffusion iterations.

```bash
# Set GPU device and run inference
# N=4 specifies 4 reverse sampling iterations for FastDiff vocoder
CUDA_VISIBLE_DEVICES=0 python inference/ProDiff.py \
    --config modules/ProDiff/config/prodiff.yaml \
    --exp_name ProDiff \
    --hparams="N=4,text='The quick brown fox jumps over the lazy dog.'" \
    --reset

# Output saved to: infer_out/The quick brown fox jumps over the lazy dog..wav
```

--------------------------------

### Hyperparameter Configuration API

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Programmatically sets and accesses model hyperparameters. Loads configuration from YAML, overrides specific parameters, and accesses them globally.

```python
from utils.hparams import set_hparams, hparams

# Load configuration from YAML file
config = set_hparams(
    config='modules/ProDiff/config/prodiff.yaml',
    exp_name='MyExperiment',
    hparams_str='N=6,timesteps=8',  # Override specific params
    print_hparams=True,
    global_hparams=True
)

# Access hyperparameters globally
print(hparams['audio_sample_rate'])    # 22050
print(hparams['hop_size'])             # 256
print(hparams['audio_num_mel_bins'])   # 80
print(hparams['timesteps'])            # Diffusion steps
print(hparams['work_dir'])             # checkpoints/MyExperiment

# Override at runtime via command line
# --hparams="key1=value1,key2=value2,nested.key=value"
```

--------------------------------

### BaseBinarizer Data Processing

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Processes raw audio and text data into binary format for efficient training data loading. Initializes hyperparameters and processes dataset splits.

```python
from data_gen.tts.base_binarizer import BaseBinarizer
from utils.hparams import set_hparams

# Initialize hyperparameters
set_hparams(config='modules/ProDiff/config/prodiff_teacher.yaml')

# Create binarizer and process dataset
binarizer = BaseBinarizer()

# Access dataset splits
train_items = binarizer.train_item_names  # Training samples
valid_items = binarizer.valid_item_names  # Validation samples
test_items = binarizer.test_item_names    # Test samples

# Process all splits
binarizer.process()
# Creates:
#   data/binary/LJSpeech/train.data
#   data/binary/LJSpeech/valid.data
#   data/binary/LJSpeech/test.data
#   data/binary/LJSpeech/phone_set.json
#   data/binary/LJSpeech/spk_map.json
```

--------------------------------

### ProDiffTeacherInfer Class for High-Quality Synthesis

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Utilize the `ProDiffTeacherInfer` class for higher fidelity speech synthesis. This class is designed for the teacher model and supports additional parameters like speaker name and item name for more controlled generation.

```python
import torch
from inference.ProDiff_Teacher import ProDiffTeacherInfer
from utils.hparams import set_hparams, hparams
from utils.audio import save_wav

# Initialize with teacher config
set_hparams(config='modules/ProDiff/config/prodiff_teacher.yaml',
            exp_name='ProDiff_Teacher')

# Create teacher inference instance
infer = ProDiffTeacherInfer(hparams)

# Synthesize with teacher model (higher quality, slightly slower)
input_data = {
    'text': 'The ProDiff Teacher model produces higher fidelity audio output.',
    'item_name': 'sample_001',  # optional identifier
    'spk_name': 'SPK1'          # speaker name (default: SPK1)
}

wav_output = infer.infer_once(input_data)
save_wav(wav_output, 'teacher_output.wav', hparams['audio_sample_rate'])
```

--------------------------------

### Convert Mel-Spectrogram to Waveform

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Converts a mel-spectrogram to an audio waveform using the ProDiff model. Requires model, diffusion hyperparameters, noise schedule, and mel-spectrogram as input.

```python
mel = torch.randn(1, 100, 80).cuda()  # [batch, time, mel_bins]
mel_transposed = mel.transpose(2, 1)   # [batch, mel_bins, time]

from modules.FastDiff.module.util import sampling_given_noise_schedule

audio_length = mel.shape[1] * hparams['hop_size']  # hop_size=256 typically
waveform = sampling_given_noise_schedule(
    model,
    (1, 1, audio_length),           # Output shape
    diffusion_hyperparams,
    noise_schedule,                  # Noise schedule for N steps
    condition=mel_transposed,
    ddim=False,
    return_sequence=False
)
# waveform shape: [1, 1, audio_length]
```

--------------------------------

### ProDiffInfer Class for Programmatic Inference

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

Use the `ProDiffInfer` class for programmatic speech synthesis. It handles automatic model loading and vocoder integration. Ensure hyperparameters are set correctly using `set_hparams` before instantiation.

```python
import torch
from inference.ProDiff import ProDiffInfer
from utils.hparams import set_hparams, hparams
from utils.audio import save_wav

# Initialize hyperparameters from config
set_hparams(config='modules/ProDiff/config/prodiff.yaml',
            exp_name='ProDiff')

# Create inference instance (loads model and vocoder automatically)
infer = ProDiffInfer(hparams)

# Synthesize speech from text
input_data = {
    'text': 'Hello world, this is a test of ProDiff text to speech synthesis.'
}

# Run inference and get waveform
wav_output = infer.infer_once(input_data)

# Save output audio (22050 Hz sample rate by default)
save_wav(wav_output, 'output.wav', hparams['audio_sample_rate'])
```

--------------------------------

### GaussianDiffusion Model Forward Pass

Source: https://context7.com/rongjiehuang/prodiff/llms.txt

This snippet shows the import statement for the `GaussianDiffusion` model, which is central to the ProDiff architecture for generating mel-spectrograms via iterative denoising. It requires `DiffNet` and `hparams`.

```python
import torch
from modules.ProDiff.model.ProDiff_teacher import GaussianDiffusion
from usr.diff.net import DiffNet
from utils.hparams import hparams
```

--------------------------------

### ProDiff Citation BibTeX

Source: https://github.com/rongjiehuang/prodiff/blob/main/readme.md

BibTeX entries for citing the ProDiff and FastDiff research papers.

```bib
@inproceedings{huang2022prodiff,
  title={ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech},
  author={Huang, Rongjie and Zhao, Zhou and Liu, Huadai and Liu, Jinglin and Cui, Chenye and Ren, Yi},
  booktitle={Proceedings of the 30th ACM International Conference on Multimedia},
  year={2022}
}

@article{huang2022fastdiff,
  title={FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis},
  author={Huang, Rongjie and Lam, Max WY and Wang, Jun and Su, Dan and Yu, Dong and Ren, Yi and Zhao, Zhou},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  year={2022}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.