PaddleSpeech

https://paddlespeech.readthedocs.io/en/latest/
Admin
PaddleSpeech is an open-source speech processing toolkit that provides comprehensive solutions for...
Tokens:1,179,287
Snippets:5,102
Trust Score:9.6
Update:6 months ago
Show doc for...
Tokens:
Raw
### Experiment Setup and Execution Example

Source: https://paddlespeech.readthedocs.io/en/latest/api/paddlespeech.t2s.training.experiment.html

This example demonstrates how to set up and run an experiment using the ExperimentBase class. It includes configuration loading, argument parsing, and conditional execution for multi-GPU training.

```python
>>> def main_sp(config, args):
>>>     exp = Experiment(config, args)
>>>     exp.setup()
>>>     exe.resume_or_load()
>>>     exp.run()
>>>
>>> config = get_cfg_defaults()
>>> parser = default_argument_parser()
>>> args = parser.parse_args()
>>> if args.config:
>>>     config.merge_from_file(args.config)
>>> if args.opts:
>>>     config.merge_from_list(args.opts)
>>> config.freeze()
>>>
>>> if args.ngpu > 1:
>>>     dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
>>> else:
>>>     main_sp(config, args)
```

--------------------------------

### Navigate to Example Directory

Source: https://paddlespeech.readthedocs.io/en/latest/asr/quick_start.html

Change the current working directory to the tiny example folder.

```bash
cd examples/tiny
```

--------------------------------

### Example Usage of ExperimentBase

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/training/experiment.html

Demonstrates how to use the ExperimentBase class, including configuration loading, argument parsing, and distributed training setup.

```python
def main_sp(config, args):
        exp = Experiment(config, args)
        exp.setup()
        exe.resume_or_load()
        exp.run()
        
        config = get_cfg_defaults()
        parser = default_argument_parser()
        args = parser.parse_args()
        if args.config:
            config.merge_from_file(args.config)
        if args.opts:
            config.merge_from_list(args.opts)
        config.freeze()
        
        if args.ngpu > 1:
            dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
        else:
            main_sp(config, args)
```

--------------------------------

### Trainer Class Example Usage

Source: https://paddlespeech.readthedocs.io/en/latest/api/paddlespeech.s2t.training.trainer.html

This example demonstrates the basic usage of the Trainer class, including setup and running the experiment. It also shows how to configure the experiment using a config file and command-line arguments, and how to handle multi-GPU training.

```python
def main_sp(config, args):
    exp = Trainer(config, args)
    exp.setup()
    exp.run()

config = get_cfg_defaults()
parser = default_argument_parser()
args = parser.parse_args()
if args.config:
    config.merge_from_file(args.config)
if args.opts:
    config.merge_from_list(args.opts)
config.freeze()

if args.ngpu > 1:
    dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
else:
    main_sp(config, args)
```

--------------------------------

### Initialize Training in PaddleSpeech

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/s2t/training/trainer.html

Handles the initial setup for training, either resuming from a checkpoint or starting from scratch. If resuming, it increments the epoch and iteration counters. If starting from scratch, it saves the initial model state.

```python
from_scratch = self.resume_or_scratch()
        if from_scratch:
            # scratch: save init model, i.e. 0 epoch
            self.save(tag='init', infos=None)
        else:
            # resume: train next_epoch and next_iteration
            self.epoch += 1
            self.iteration += 1
            logger.info(
                f"Resume train: epoch {self.epoch }, step {self.iteration}!")

        self.maybe_batch_sampler_step()
```

--------------------------------

### Install Kaldi and OpenBLAS

Source: https://paddlespeech.readthedocs.io/en/latest/install.html

Executes the installation scripts for OpenBLAS and Kaldi from the tools directory.

```bash
pushd tools
bash extras/install_openblas.sh
bash extras/install_kaldi.sh
popd
```

--------------------------------

### Load Example by File Path

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/vector/exps/ge2e/speaker_verification_dataset.html

Loads a mel spectrogram from a given file path. This is the core data loading function for individual examples.

```python
def __getitem__(self, fpath):
        return np.load(fpath)

```

--------------------------------

### Trainer: Multi-GPU Training Setup

Source: https://paddlespeech.readthedocs.io/en/latest/api/paddlespeech.s2t.training.trainer.html

Handles the setup for multi-GPU training using `dist.spawn`. If more than one GPU is available, it spawns processes; otherwise, it runs on a single GPU.

```python
if args.ngpu > 1:
    dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
else:
    main_sp(config, args)
```

--------------------------------

### Trainer.run

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/training/trainer.html

Starts the training loop.

```APIDOC
## Trainer.run

### Description
Executes the training loop until the stop trigger is met.
```

--------------------------------

### Example Usage of DistributedBatchSampler

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/datasets/sampler.html

Demonstrates how to initialize and use DistributedBatchSampler with a custom dataset. This example shows iterating through the sampler to get data batches.

```python
import numpy as np
from paddle.io import Dataset, DistributedBatchSampler
# init with dataset
class RandomDataset(Dataset):
    def __init__(self, num_samples):
        self.num_samples = num_samples

    def __getitem__(self, idx):
        image = np.random.random([784]).astype('float32')
        label = np.random.randint(0, 9, (1, )).astype('int64')
        return image, label
    
    def __len__(self):
        return self.num_samples

dataset = RandomDataset(100)
sampler = DistributedBatchSampler(dataset, batch_size=64)
for data in sampler:
    # do something
    break
```

--------------------------------

### Clip LJSpeech Examples

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/waveflow/ljspeech.html

Clips mel-spectrograms and waveforms from LJSpeech examples to a fixed size. Randomly selects a starting point for clipping. Requires `clip_frames` and `hop_length` to be set.

```python
[docs]
class LJSpeechClipCollector(object):
    def __init__(self, clip_frames=65, hop_length=256):
        self.clip_frames = clip_frames
        self.hop_length = hop_length

    def __call__(self, examples):
        mels = []
        wavs = []
        for example in examples:
            mel_clip, wav_clip = self.clip(example)
            mels.append(mel_clip)
            wavs.append(wav_clip)
        mels = np.stack(mels)
        wavs = np.stack(wavs)
        return mels, wavs


[docs]    def clip(self, example):
        mel, wav = example
        frames = mel.shape[-1]
        start = np.random.randint(0, frames - self.clip_frames)
        mel_clip = mel[:, start:start + self.clip_frames]
        wav_clip = wav[start * self.hop_length:(start + self.clip_frames) *
                       self.hop_length]
        return mel_clip, wav_clip

```

--------------------------------

### ExperimentBase Setup Method

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/training/experiment.html

Sets up the experiment environment, including device configuration, parallel environment initialization, output directories, logging, visualization, data loaders, and model.

```python
def setup(self):
        """Setup the experiment.
        """
        if self.args.ngpu == 0:
            paddle.set_device("cpu")
        elif self.args.ngpu > 0:
            paddle.set_device("gpu")
        else:
            print("ngpu should >= 0 !")
        if self.parallel:
            self.init_parallel()

        self.setup_output_dir()
        self.dump_config()
        self.setup_visualizer()
        self.setup_logger()
        self.setup_checkpointer()

        self.setup_dataloader()
        self.setup_model()

        self.iteration = 0
        self.epoch = 0
```

--------------------------------

### Get Span Boundaries

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/ernie_sat/utils.html

Calculates the start and end boundaries for a mel frame span based on MFA start and end times and the span to replace. Handles cases where the span might be out of bounds.

```python
def get_span_bdy(mfa_start: List[float],
                 mfa_end: List[float],
                 span_to_repl: List[List[int]]):
    if span_to_repl[0] >= len(mfa_start):
        span_bdy = [mfa_end[-1], mfa_end[-1]]
    else:
        span_bdy = [mfa_start[span_to_repl[0]], mfa_end[span_to_repl[1] - 1]]
    return span_bdy
```

--------------------------------

### Execute Main Entry Point

Source: https://paddlespeech.readthedocs.io/en/latest/asr/quick_start.html

Run the main shell script to start the demo process.

```bash
bash run.sh
```

--------------------------------

### Main Execution Entry Point

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/ort_predict.html

Initializes the device and triggers the inference process.

```python
def main():
    args = parse_args()

    paddle.set_device(args.device)

    ort_predict(args)


if __name__ == "__main__":
    main()
```

--------------------------------

### Trainer: Setup and Run

Source: https://paddlespeech.readthedocs.io/en/latest/api/paddlespeech.s2t.training.trainer.html

Instantiates and sets up the Trainer, then runs the experiment. This is a core part of the training workflow.

```python
exp = Trainer(config, args)
exp.setup()
exp.run()
```

--------------------------------

### Setup Trainer and Extensions

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/transformer_tts/train.html

Initializes the Trainer, Updater, and Evaluator. Configures extensions like VisualDL for visualization and Snapshot for saving model checkpoints. Copies the configuration file to the output directory.

```python
    output_dir = Path(args.output_dir)
    output_dir.mkdir(parents=True, exist_ok=True)
    if dist.get_rank() == 0:
        config_name = args.config.split("/")[-1]
        # copy conf to output_dir
        shutil.copyfile(args.config, output_dir / config_name)

    updater = TransformerTTSUpdater(
        model=model,
        optimizer=optimizer,
        dataloader=train_dataloader,
        output_dir=output_dir,
        **config["updater"])

    trainer = Trainer(updater, (config.max_epoch, 'epoch'), output_dir)

    evaluator = TransformerTTSEvaluator(
        model, dev_dataloader, output_dir=output_dir, **config["updater"])

    if dist.get_rank() == 0:
        trainer.extend(evaluator, trigger=(1, "epoch"))
        trainer.extend(VisualDL(output_dir), trigger=(1, "iteration"))
    trainer.extend(
        Snapshot(max_size=config.num_snapshots), trigger=(1, 'epoch'))
    trainer.run()

```

--------------------------------

### Main Execution Entry Point

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/transformer_tts/synthesize.html

Parses command-line arguments, configures the device (CPU/GPU), and initializes the evaluation process.

```python
def main():
    # parse args and config and redirect to train_sp
    parser = argparse.ArgumentParser(
        description="Synthesize with transformer tts & waveflow.")
    parser.add_argument(
        "--transformer-tts-config",
        type=str,
        help="transformer tts config file.")
    parser.add_argument(
        "--transformer-tts-checkpoint",
        type=str,
        help="transformer tts checkpoint to load.")
    parser.add_argument(
        "--transformer-tts-stat",
        type=str,
        help="mean and standard deviation used to normalize spectrogram when training transformer tts."
    )
    parser.add_argument(
        "--waveflow-config", type=str, help="waveflow config file.")
    # not normalize when training waveflow
    parser.add_argument(
        "--waveflow-checkpoint", type=str, help="waveflow checkpoint to load.")
    parser.add_argument(
        "--phones-dict", type=str, default=None, help="phone vocabulary file.")

    parser.add_argument("--test-metadata", type=str, help="test metadata.")
    parser.add_argument("--output-dir", type=str, help="output dir.")
    parser.add_argument(
        "--ngpu", type=int, default=1, help="if ngpu == 0, use cpu.")

    args = parser.parse_args()

    if args.ngpu == 0:
        paddle.set_device("cpu")
    elif args.ngpu > 0:
        paddle.set_device("gpu")
    else:
        print("ngpu should >= 0 !")

    with open(args.transformer_tts_config) as f:
        transformer_tts_config = CfgNode(yaml.safe_load(f))
    with open(args.waveflow_config) as f:
        waveflow_config = CfgNode(yaml.safe_load(f))

    print("========Args========")
    print(yaml.safe_dump(vars(args)))
    print("========Config========")
    print(transformer_tts_config)
    print(waveflow_config)

    evaluate(args, transformer_tts_config, waveflow_config)


if __name__ == "__main__":
    main()
```

--------------------------------

### Make Guided Attention Mask

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/modules/losses.html

Generates a guided attention mask for a given input and output length, controlled by a sigma value. This mask encourages attention to follow a diagonal path. Examples show the shape and values of the generated mask for different input/output lengths.

```python
def _make_guided_attention_mask(ilen, olen, sigma):
    """Make guided attention mask.

    Examples
    ----------
    >>> guided_attn_mask =_make_guided_attention(5, 5, 0.4)
    >>> guided_attn_mask.shape
    [5, 5]
    >>> guided_attn_mask
    tensor([[0.0000, 0.1175, 0.3935, 0.6753, 0.8647],
            [0.1175, 0.0000, 0.1175, 0.3935, 0.6753],
            [0.3935, 0.1175, 0.0000, 0.1175, 0.3935],
            [0.6753, 0.3935, 0.1175, 0.0000, 0.1175],
            [0.8647, 0.6753, 0.3935, 0.1175, 0.0000]])
    >>> guided_attn_mask =_make_guided_attention(3, 6, 0.4)
    >>> guided_attn_mask.shape
    [6, 3]
    >>> guided_attn_mask
    tensor([[0.0000, 0.2934, 0.7506],
            [0.0831, 0.0831, 0.5422],
            [0.2934, 0.0000, 0.2934],
            [0.5422, 0.0831, 0.0831],
            [0.7506, 0.2934, 0.0000],
            [0.8858, 0.5422, 0.0831]])

    """
    grid_x, grid_y = paddle.meshgrid(
        paddle.arange(olen), paddle.arange(ilen))
    grid_x = grid_x.cast(dtype=paddle.float32)
    grid_y = grid_y.cast(dtype=paddle.float32)
    return 1.0 - paddle.exp(-( 
        (grid_y / ilen - grid_x / olen)**2) / (2 * (sigma**2)))
```

--------------------------------

### Setup Output Directory

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/s2t/training/trainer.html

Configures and creates necessary directories for output, checkpoints, and logs.

```APIDOC
## Setup Output Directory

### Description
Initializes and creates the output directory structure for the experiment, including directories for checkpoints and logs. The location is determined by `args.output`, `args.checkpoint_path`, or `args.export_path`.

### Method
`setup_output_dir()`

### Endpoint
N/A (Internal method)

### Parameters
None

### Request Body
None

### Response
- `self.output_dir` (Path): The main output directory.
- `self.checkpoint_dir` (Path): The directory for saving checkpoints.
- `self.log_dir` (Path): The directory for saving logs.

### Logic
- Determines `output_dir` based on provided arguments.
- Creates `output_dir`, `checkpoint_dir`, and `log_dir` if they do not exist.
```

--------------------------------

### Train PLDA Model and Example Usage

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/vector/cluster/plda.html

Demonstrates training a PLDA model using i-vectors/x-vectors and then performing scoring. Includes setup for enrollment and test data, and trial definition.

```python
if __name__ == '__main__':
    import random

    dim, N, n_spkrs = 10, 100, 10
    train_xv = numpy.random.rand(N, dim)
    md = ['md' + str(random.randrange(1, n_spkrs, 1)) for i in range(N)]  # spk
    modelset = numpy.array(md, dtype="|O")
    sg = ['sg' + str(i) for i in range(N)]  # utt
    segset = numpy.array(sg, dtype="|O")
    stat0 = numpy.array([[1.0]] * N)
    xvectors_stat = EmbeddingMeta(
        modelset=modelset, segset=segset, stats=train_xv)
    # Training PLDA model: M ~ (mean, F, Sigma)
    plda = PLDA(rank_f=5)
    plda.plda(xvectors_stat)
    print(plda.mean.shape)  #(10,)
    print(plda.F.shape)  #(10, 5)
    print(plda.Sigma.shape)  #(10, 10)
    # Enrollment (20 utts),
    en_N = 20
    en_xv = numpy.random.rand(en_N, dim)
    en_sgs = ['en' + str(i) for i in range(en_N)]
    en_sets = numpy.array(en_sgs, dtype="|O")
    en_stat = EmbeddingMeta(modelset=en_sets, segset=en_sets, stats=en_xv)
    # Test (30 utts)
    te_N = 30
    te_xv = numpy.random.rand(te_N, dim)
    te_sgs = ['te' + str(i) for i in range(te_N)]
    te_sets = numpy.array(te_sgs, dtype="|O")
    te_stat = EmbeddingMeta(modelset=te_sets, segset=te_sets, stats=te_xv)
    ndx = Ndx(models=en_sets, testsegs=te_sets)  # trials
    # PLDA Scoring
```

--------------------------------

### Install pytest-runner for Pip

Source: https://paddlespeech.readthedocs.io/en/latest/install.html

Installs 'pytest-runner' using pip with a specific mirror. This is a prerequisite for installing PaddleSpeech, as it can resolve potential issues with installing the 'kaldiio' package.

```bash
pip install pytest-runner -i https://pypi.tuna.tsinghua.edu.cn/simple
```

--------------------------------

### Define Setup Methods

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/s2t/training/trainer.html

Required methods for subclasses to implement for model and dataloader initialization.

```python
def setup_model(self):
    """Setup model, criterion and optimizer, etc. A subclass should
    implement this method.
    """
    raise NotImplementedError("setup_model should be implemented.")
```

```python
def setup_dataloader(self):
    """Setup training dataloader and validation dataloader. A subclass
    should implement this method.
    """
    raise NotImplementedError("setup_dataloader should be implemented.")
```

--------------------------------

### Install Conda Dependencies

Source: https://paddlespeech.readthedocs.io/en/latest/install.html

Installs necessary dependencies for PaddleSpeech using Conda. Ensure you have Conda installed and configured.

```bash
conda install -y -c conda-forge sox libsndfile bzip2

```

--------------------------------

### Define Main Training Entry Point

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/speedyspeech/train.html

Parses command-line arguments and configuration files, then dispatches the training process.

```python
[docs]def main():
    # parse args and config and redirect to train_sp
    parser = argparse.ArgumentParser(
        description="Train a Speedyspeech model with a single speaker dataset.")
    parser.add_argument("--config", type=str, help="config file.")
    parser.add_argument("--train-metadata", type=str, help="training data.")
    parser.add_argument("--dev-metadata", type=str, help="dev data.")
    parser.add_argument("--output-dir", type=str, help="output dir.")
    parser.add_argument(
        "--nxpu",
        type=int,
        default=0,
        help="if nxpu == 0 and ngpu == 0, use cpu.")
    parser.add_argument(
        "--ngpu", type=int, default=1, help="if ngpu == 0, use cpu or xpu")

    parser.add_argument(
        "--use-relative-path",
        type=str2bool,
        default=False,
        help="whether use relative path in metadata")

    parser.add_argument(
        "--phones-dict", type=str, default=None, help="phone vocabulary file.")

    parser.add_argument(
        "--tones-dict", type=str, default=None, help="tone vocabulary file.")

    parser.add_argument(
        "--speaker-dict",
        type=str,
        default=None,
        help="speaker id map file for multiple speaker model.")

    # 这里可以多传入 max_epoch 等
    args, rest = parser.parse_known_args()

    with open(args.config) as f:
        config = CfgNode(yaml.safe_load(f))

    if rest:
        extra = []
        # to support key=value format
        for item in rest:
            # remove "--"
            item = item[2:]
            extra.extend(item.split("=", maxsplit=1))
        config.merge_from_list(extra)

    print("========Args========")
    print(yaml.safe_dump(vars(args)))
    print("========Config========")
    print(config)
    print(
        f"master see the word size: {dist.get_world_size()}, from pid: {os.getpid()}"
    )

    # dispatch
    if args.ngpu > 1:
        dist.spawn(train_sp, (args, config), nprocs=args.ngpu)
    else:
        train_sp(args, config)



if __name__ == "__main__":
    main()
```

--------------------------------

### Install Miniconda on Linux/Mac

Source: https://paddlespeech.readthedocs.io/en/latest/install.html

Downloads and installs Miniconda, a minimal installer for Conda, on Linux or macOS systems. It then initializes Conda for use in the shell.

```bash
# download the miniconda
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -P tools/
# install the miniconda
bash tools/Miniconda3-latest-Linux-x86_64.sh -b
# conda init
$HOME/miniconda3/bin/conda init

```

--------------------------------

### Main Entry Point for Training

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/gan_vocoder/multi_band_melgan/train.html

Parses command-line arguments and configuration files, then dispatches the training process to single or multiple GPUs.

```python
[docs]def main():
    # parse args and config and redirect to train_sp

    parser = argparse.ArgumentParser(
        description="Train a Multi-Band MelGAN model.")
    parser.add_argument(
        "--config", type=str, help="Multi-Band MelGAN config file.")
    parser.add_argument("--train-metadata", type=str, help="training data.")
    parser.add_argument("--dev-metadata", type=str, help="dev data.")
    parser.add_argument("--output-dir", type=str, help="output dir.")
    parser.add_argument(
        "--ngpu", type=int, default=1, help="if ngpu == 0, use cpu.")

    args = parser.parse_args()

    with open(args.config, 'rt') as f:
        config = CfgNode(yaml.safe_load(f))

    print("========Args========")
    print(yaml.safe_dump(vars(args)))
    print("========Config========")
    print(config)
    print(
        f"master see the word size: {dist.get_world_size()}, from pid: {os.getpid()}"
    )

    # dispatch
    if args.ngpu > 1:
        dist.spawn(train_sp, (args, config), nprocs=args.ngpu)
    else:
        train_sp(args, config)



if __name__ == "__main__":
    main()
```

--------------------------------

### Prepare Output Directory and Copy Configuration

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/gan_vocoder/parallelwave_gan/train.html

Creates the output directory if it doesn't exist and copies the configuration file to the output directory for record-keeping. This is done only on the rank 0 process.

```python
output_dir = Path(args.output_dir)
output_dir.mkdir(parents=True, exist_ok=True)
if dist.get_rank() == 0:
    config_name = args.config.split("/")[-1]
    # copy conf to output_dir
    shutil.copyfile(args.config, output_dir / config_name)
```

--------------------------------

### Download and Install Miniconda

Source: https://paddlespeech.readthedocs.io/en/latest/install.html

Downloads the latest Miniconda installer for Linux 64-bit and then installs it silently in the 'tools/' directory. This provides a Python environment manager.

```bash
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -P tools/
bash tools/Miniconda3-latest-Linux-x86_64.sh -b
```

--------------------------------

### FilterDataset: Filter dataset examples with a function

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/datasets/dataset.html

FilterDataset creates a new dataset by applying a filter function to each example of the base dataset. Only examples for which the filter function returns True are included. The filter function should accept a single example and return a boolean.

```python
class FilterDataset(Dataset):
    def __init__(self, dataset, filter_fn):
        """A filtered dataset.

        Args:
            dataset (Dataset): the base dataset.
            filter_fn (callable): a callable which takes an example of the base dataset and return a boolean.
        """
        self._dataset = dataset
        self._indices = [
            i for i in range(len(dataset)) if filter_fn(dataset[i])
        ]
        self._size = len(self._indices)

    def __len__(self):
        return self._size

    def __getitem__(self, i):
        index = self._indices[i]
        return self._dataset[index]
```

--------------------------------

### Main Training Entry Point

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/wavernn/train.html

Parses command-line arguments, loads configuration, and dispatches training to single or multiple GPUs.

```python
def main():
    # parse args and config and redirect to train_sp

    parser = argparse.ArgumentParser(description="Train a WaveRNN model.")
    parser.add_argument("--config", type=str, help="WaveRNN config file.")
    parser.add_argument("--train-metadata", type=str, help="training data.")
    parser.add_argument("--dev-metadata", type=str, help="dev data.")
    parser.add_argument("--output-dir", type=str, help="output dir.")
    parser.add_argument(
        "--ngpu", type=int, default=1, help="if ngpu == 0, use cpu.")

    args = parser.parse_args()

    with open(args.config, 'rt') as f:
        config = CfgNode(yaml.safe_load(f))

    print("========Args========")
    print(yaml.safe_dump(vars(args)))
    print("========Config========")
    print(config)
    print(
        f"master see the word size: {dist.get_world_size()}, from pid: {os.getpid()}"
    )

    # dispatch
    if args.ngpu > 1:
        dist.spawn(train_sp, (args, config), nprocs=args.ngpu)
    else:
        train_sp(args, config)



if __name__ == "__main__":
    main()
```

--------------------------------

### Install pytest-runner

Source: https://paddlespeech.readthedocs.io/en/latest/install.html

Installs pytest-runner to potentially resolve issues with installing 'kaldiio' from the default download source. It's recommended to use the specified Tsinghua mirror.

```bash
pip install pytest-runner -i https://pypi.tuna.tsinghua.edu.cn/simple

```

--------------------------------

### main / main_sp

Source: https://paddlespeech.readthedocs.io/en/latest/api/paddlespeech.vector.exps.ge2e.train.html

Entry point functions for initiating the GE2E training experiment.

```APIDOC
## main(config, args)

### Description
Main entry point for the GE2E training experiment.

### Parameters
- **config** (object) - Required - Configuration settings for the experiment.
- **args** (object) - Required - Command line arguments.
```

--------------------------------

### install(package_name)

Source: https://paddlespeech.readthedocs.io/en/latest/api/paddlespeech.s2t.utils.dynamic_pip_install.html

Installs a specified Python package dynamically at runtime.

```APIDOC
## Function: paddlespeech.s2t.utils.dynamic_pip_install.install

### Description
Dynamically installs a Python package by name during runtime execution.

### Parameters
#### Arguments
- **package_name** (string) - Required - The name of the package to be installed via pip.
```

--------------------------------

### Output Directory Setup

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/training/experiment.html

Creates the primary output directory for the experiment.

```python
    def setup_output_dir(self):
        """Create a directory used for output.
        """
        # output dir
        output_dir = Path(self.args.output).expanduser()
        output_dir.mkdir(parents=True, exist_ok=True)

        self.output_dir = output_dir
```

--------------------------------

### Dynamic Package Installation

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/s2t/utils/dynamic_pip_install.html

The `install` function facilitates the installation of specified Python packages using pip. It dynamically imports the appropriate `pip.main` function based on the pip version.

```APIDOC
## Function: install

### Description
Installs a specified Python package using pip.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **package_name** (string) - Required - The name of the package to install.

### Request Example
```python
import paddlespeech.s2t.utils.dynamic_pip_install as dp

dp.install("some_package")
```

### Response
#### Success Response (200)
This function does not return a value upon successful installation. Success is indicated by the absence of errors.

#### Response Example
None
```

--------------------------------

### Main Preprocessing Entry Point

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/ernie_sat/preprocess.html

Configures command-line arguments, initializes paths, and handles dataset-specific file splitting for training, development, and testing.

```python
[docs]def main():
    # parse config and args
    parser = argparse.ArgumentParser(
        description="Preprocess audio and then extract features.")

    parser.add_argument(
        "--dataset",
        default="baker",
        type=str,
        help="name of dataset, should in {baker, aishell3, ljspeech, vctk} now")

    parser.add_argument(
        "--rootdir", default=None, type=str, help="directory to dataset.")

    parser.add_argument(
        "--dumpdir",
        type=str,
        required=True,
        help="directory to dump feature files.")
    parser.add_argument(
        "--dur-file", default=None, type=str, help="path to durations.txt.")

    parser.add_argument("--config", type=str, help="fastspeech2 config file.")

    parser.add_argument(
        "--num-cpu", type=int, default=1, help="number of process.")

    parser.add_argument(
        "--cut-sil",
        type=str2bool,
        default=True,
        help="whether cut sil in the edge of audio")

    parser.add_argument(
        "--spk_emb_dir",
        default=None,
        type=str,
        help="directory to speaker embedding files.")
    args = parser.parse_args()

    rootdir = Path(args.rootdir).expanduser()
    dumpdir = Path(args.dumpdir).expanduser()
    # use absolute path
    dumpdir = dumpdir.resolve()
    dumpdir.mkdir(parents=True, exist_ok=True)
    dur_file = Path(args.dur_file).expanduser()

    if args.spk_emb_dir:
        spk_emb_dir = Path(args.spk_emb_dir).expanduser().resolve()
    else:
        spk_emb_dir = None

    assert rootdir.is_dir()
    assert dur_file.is_file()

    with open(args.config, 'rt') as f:
        config = CfgNode(yaml.safe_load(f))

    sentences, speaker_set = get_phn_dur(dur_file)

    merge_silence(sentences)
    phone_id_map_path = dumpdir / "phone_id_map.txt"
    speaker_id_map_path = dumpdir / "speaker_id_map.txt"
    get_input_token(sentences, phone_id_map_path, args.dataset)
    get_spk_id_map(speaker_set, speaker_id_map_path)

    if args.dataset == "baker":
        wav_files = sorted(list((rootdir / "Wave").rglob("*.wav")))
        # split data into 3 sections
        num_train = 9800
        num_dev = 100
        train_wav_files = wav_files[:num_train]
        dev_wav_files = wav_files[num_train:num_train + num_dev]
        test_wav_files = wav_files[num_train + num_dev:]
    elif args.dataset == "aishell3":
        sub_num_dev = 5
        wav_dir = rootdir / "train" / "wav"
        train_wav_files = []
        dev_wav_files = []
        test_wav_files = []
        for speaker in os.listdir(wav_dir):
            wav_files = sorted(list((wav_dir / speaker).rglob("*.wav")))
            if len(wav_files) > 100:
                train_wav_files += wav_files[:-sub_num_dev * 2]
                dev_wav_files += wav_files[-sub_num_dev * 2:-sub_num_dev]
                test_wav_files += wav_files[-sub_num_dev:]
            else:
                train_wav_files += wav_files

    elif args.dataset == "ljspeech":
        wav_files = sorted(list((rootdir / "wavs").rglob("*.wav")))
        # split data into 3 sections
        num_train = 12900
        num_dev = 100
        train_wav_files = wav_files[:num_train]
        dev_wav_files = wav_files[num_train:num_train + num_dev]
        test_wav_files = wav_files[num_train + num_dev:]
    elif args.dataset == "vctk":
        sub_num_dev = 5
        wav_dir = rootdir / "wav48_silence_trimmed"
        train_wav_files = []
        dev_wav_files = []
        test_wav_files = []
        for speaker in os.listdir(wav_dir):
            wav_files = sorted(list((wav_dir / speaker).rglob("*_mic2.flac")))
            if len(wav_files) > 100:
                train_wav_files += wav_files[:-sub_num_dev * 2]
                dev_wav_files += wav_files[-sub_num_dev * 2:-sub_num_dev]
                test_wav_files += wav_files[-sub_num_dev:]
            else:
                train_wav_files += wav_files

    else:
```

--------------------------------

### Install PaddleSpeech (Standard)

Source: https://paddlespeech.readthedocs.io/en/latest/install.html

Installs the PaddleSpeech package from the root directory of the cloned repository using pip. This command installs the package in the current environment, making its modules available for import.

```bash
pip install . -i https://pypi.tuna.tsinghua.org/simple
```

--------------------------------

### Main Entry Point for ErnieLinear Testing

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/text/exps/ernie_linear/test.html

Parses command-line arguments and initializes the device before triggering the test process.

```python
def main():
    # parse args and config and redirect to train_sp
    parser = argparse.ArgumentParser(description="Test a ErnieLinear model.")
    parser.add_argument("--config", type=str, help="ErnieLinear config file.")
    parser.add_argument("--checkpoint", type=str, help="snapshot to load.")
    parser.add_argument("--print_eval", type=str2bool, default=True)
    parser.add_argument(
        "--ngpu", type=int, default=1, help="if ngpu=0, use cpu.")

    args = parser.parse_args()

    if args.ngpu == 0:
        paddle.set_device("cpu")
    elif args.ngpu > 0:
        paddle.set_device("gpu")
    else:
        print("ngpu should >= 0 !")

    test(args)


if __name__ == "__main__":
    main()
```

--------------------------------

### Initialize Frontend

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/syn_utils.html

Creates a language-specific frontend instance based on the provided language code and vocabulary paths.

```python
def get_frontend(lang: str='zh',
                 phones_dict: Optional[os.PathLike]=None,
                 tones_dict: Optional[os.PathLike]=None,
                 pinyin_phone: Optional[os.PathLike]=None,
                 use_rhy=False):
    if lang == 'zh':
        frontend = Frontend(
            phone_vocab_path=phones_dict,
            tone_vocab_path=tones_dict,
            use_rhy=use_rhy)
    elif lang == 'canton':
        frontend = CantonFrontend(phone_vocab_path=phones_dict)
    elif lang == 'en':
        frontend = English(phone_vocab_path=phones_dict)
    elif lang == 'mix':
        frontend = MixFrontend(
            phone_vocab_path=phones_dict, tone_vocab_path=tones_dict)
    elif lang == 'sing':
        frontend = SingFrontend(
            pinyin_phone_path=pinyin_phone, phone_vocab_path=phones_dict)
    else:
        print("wrong lang!")
    return frontend
```

--------------------------------

### Install GCC on Mac

Source: https://paddlespeech.readthedocs.io/en/latest/install.html

Installs the GCC compiler on macOS using Homebrew. This is required for the C++ compilation environment.

```bash
brew install gcc

```

--------------------------------

### Initialize SpeedySpeech Training Environment

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/speedyspeech/train.html

Configures the training environment, including device selection (CPU/GPU/XPU), distributed training setup, and random seeding.

```python
def train_sp(args, config):
    # decides device type and whether to run in parallel
    # setup running environment correctly
    world_size = paddle.distributed.get_world_size()
    if (not paddle.is_compiled_with_cuda()) or args.ngpu == 0:
        if (not paddle.is_compiled_with_xpu()) or args.nxpu == 0:
            paddle.set_device("cpu")
        else:
            paddle.set_device("xpu")
    else:
        paddle.set_device("gpu")
        if world_size > 1:
            paddle.distributed.init_parallel_env()

    # set the random seed, it is a must for multiprocess training
    seed_everything(config.seed)

    print(
        f"rank: {dist.get_rank()}, pid: {os.getpid()}, parent_pid: {os.getppid()}",
    )
```

--------------------------------

### Install a Python Package

Source: https://paddlespeech.readthedocs.io/en/latest/api/paddlespeech.s2t.utils.dynamic_pip_install.html

Use this function to dynamically install a Python package. Ensure the package name is correct.

```python
paddlespeech.s2t.utils.dynamic_pip_install.install(_package_name_)
```

--------------------------------

### Initialize MelGAN Training Environment

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/gan_vocoder/multi_band_melgan/train.html

Sets up the training environment, including device selection, distributed training initialization, and random seed configuration.

```python
def train_sp(args, config):
    # decides device type and whether to run in parallel
    # setup running environment correctly
    world_size = paddle.distributed.get_world_size()
    if (not paddle.is_compiled_with_cuda()) or args.ngpu == 0:
        paddle.set_device("cpu")
    else:
        paddle.set_device("gpu")
        if world_size > 1:
            paddle.distributed.init_parallel_env()

    # set the random seed, it is a must for multiprocess training
    seed_everything(config.seed)

    print(
        f"rank: {dist.get_rank()}, pid: {os.getpid()}, parent_pid: {os.getppid()}",
    )

    # dataloader has been too verbose
    logging.getLogger("DataLoader").disabled = True
```

--------------------------------

### Calculate Guided Attention Loss

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/modules/losses.html

Calculates the guided attention loss by applying guided attention masks to attention weights. It ensures masks are generated if not already present and applies a mean operation on the masked attention weights. Masks are reset if reset_always is true.

```python
def forward(self, att_ws, ilens, olens):
    if self.guided_attn_masks is None:
        self.guided_attn_masks = self._make_guided_attention_masks(ilens,
                                                                   olens)
    if self.masks is None:
        self.masks = self._make_masks(ilens, olens)
    losses = self.guided_attn_masks * att_ws
    loss = paddle.mean(
        losses.masked_select(self.masks.broadcast_to(losses.shape)))
    if self.reset_always:
        self._reset_masks()
    return self.alpha * loss
```

--------------------------------

### Install Python Package Dynamically

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/s2t/utils/dynamic_pip_install.html

Installs a specified Python package using pip. It handles different pip versions for compatibility.

```python
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import pip




def install(package_name):
    if int(pip.__version__.split('.')[0]) > 9:
        from pip._internal import main
    else:
        from pip import main
    main(['install', package_name])

```

--------------------------------

### Main Preprocessing Entry Point

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/tacotron2/preprocess.html

Configures command-line arguments and initializes the dataset processing pipeline.

```python
[docs]def main():
    # parse config and args
    parser = argparse.ArgumentParser(
        description="Preprocess audio and then extract features.")

    parser.add_argument(
        "--dataset",
        default="baker",
        type=str,
        help="name of dataset, should in {baker, aishell3, ljspeech, vctk} now")

    parser.add_argument(
        "--rootdir", default=None, type=str, help="directory to dataset.")

    parser.add_argument(
        "--dumpdir",
        type=str,
        required=True,
        help="directory to dump feature files.")
    parser.add_argument(
        "--dur-file", default=None, type=str, help="path to durations.txt.")

    parser.add_argument("--config", type=str, help="fastspeech2 config file.")

    parser.add_argument(
        "--num-cpu", type=int, default=1, help="number of process.")

    parser.add_argument(
        "--cut-sil",
        type=str2bool,
        default=True,
        help="whether cut sil in the edge of audio")

    parser.add_argument(
        "--spk_emb_dir",
        default=None,
        type=str,
        help="directory to speaker embedding files.")
    args = parser.parse_args()

    rootdir = Path(args.rootdir).expanduser()
    dumpdir = Path(args.dumpdir).expanduser()
    # use absolute path
    dumpdir = dumpdir.resolve()
    dumpdir.mkdir(parents=True, exist_ok=True)
    dur_file = Path(args.dur_file).expanduser()

    if args.spk_emb_dir:
        spk_emb_dir = Path(args.spk_emb_dir).expanduser().resolve()
    else:
        spk_emb_dir = None

    assert rootdir.is_dir()
    assert dur_file.is_file()

    with open(args.config, 'rt') as f:
        config = CfgNode(yaml.safe_load(f))

    sentences, speaker_set = get_phn_dur(dur_file)

    merge_silence(sentences)
    phone_id_map_path = dumpdir / "phone_id_map.txt"
    speaker_id_map_path = dumpdir / "speaker_id_map.txt"
    get_input_token(sentences, phone_id_map_path, args.dataset)
    get_spk_id_map(speaker_set, speaker_id_map_path)

    if args.dataset == "baker":
        wav_files = sorted(list((rootdir / "Wave").rglob("*.wav")))
        # split data into 3 sections
        num_train = 9800
        num_dev = 100
        train_wav_files = wav_files[:num_train]
        dev_wav_files = wav_files[num_train:num_train + num_dev]
        test_wav_files = wav_files[num_train + num_dev:]
    elif args.dataset == "aishell3":
        sub_num_dev = 5
        wav_dir = rootdir / "train" / "wav"
        train_wav_files = []
        dev_wav_files = []
        test_wav_files = []
        for speaker in os.listdir(wav_dir):
            wav_files = sorted(list((wav_dir / speaker).rglob("*.wav")))
            if len(wav_files) > 100:
                train_wav_files += wav_files[:-sub_num_dev * 2]
                dev_wav_files += wav_files[-sub_num_dev * 2:-sub_num_dev]
                test_wav_files += wav_files[-sub_num_dev:]
            else:
                train_wav_files += wav_files

    elif args.dataset == "ljspeech":
        wav_files = sorted(list((rootdir / "wavs").rglob("*.wav")))
        # split data into 3 sections
        num_train = 12900
        num_dev = 100
        train_wav_files = wav_files[:num_train]
        dev_wav_files = wav_files[num_train:num_train + num_dev]
        test_wav_files = wav_files[num_train + num_dev:]
    elif args.dataset == "vctk":
        sub_num_dev = 5
        wav_dir = rootdir / "wav48_silence_trimmed"
        train_wav_files = []
        dev_wav_files = []
        test_wav_files = []
        for speaker in os.listdir(wav_dir):
            wav_files = sorted(list((wav_dir / speaker).rglob("*_mic2.flac")))
            if len(wav_files) > 100:
                train_wav_files += wav_files[:-sub_num_dev * 2]
                dev_wav_files += wav_files[-sub_num_dev * 2:-sub_num_dev]
                test_wav_files += wav_files[-sub_num_dev:]
            else:
                train_wav_files += wav_files

    else:
        print("dataset should in {baker, aishell3, ljspeech, vctk} now!")

    train_dump_dir = dumpdir / "train" / "raw"
    train_dump_dir.mkdir(parents=True, exist_ok=True)
    dev_dump_dir = dumpdir / "dev" / "raw"
    dev_dump_dir.mkdir(parents=True, exist_ok=True)
    test_dump_dir = dumpdir / "test" / "raw"
    test_dump_dir.mkdir(parents=True, exist_ok=True)

    # Extractor
    mel_extractor = LogMelFBank(
        sr=config.fs,
        n_fft=config.n_fft,
        hop_length=config.n_shift,
```

--------------------------------

### FastSpeech2 Trainer Setup

Source: https://paddlespeech.readthedocs.io/en/latest/_modules/paddlespeech/t2s/exps/fastspeech2/train.html

Initializes and extends the trainer with an evaluator and snapshotting for a FastSpeech2 model. This is typically used for distributed training setups.

```python
trainer = Trainer(updater, (config.max_epoch, 'epoch'), output_dir)

evaluator = FastSpeech2Evaluator(
    model,
    dev_dataloader,
    output_dir=output_dir,
    enable_spk_cls=enable_spk_cls,
    **config["updater"],
    )

if dist.get_rank() == 0:
    trainer.extend(evaluator, trigger=(1, "epoch"))
    trainer.extend(VisualDL(output_dir), trigger=(1, "iteration"))
trainer.extend(
    Snapshot(max_size=config.num_snapshots), trigger=(1, 'epoch'))
trainer.run()
```
Tokens:
Raw