### Clone and Setup 🐸TTS Repository (Bash) Source: https://docs.coqui.ai/en/latest/contributing This snippet shows how to clone the 🐸TTS repository and set up the upstream remote for development. It assumes you have Git installed and are authenticated with GitHub. ```bash git clone git@github.com:/TTS.git cd TTS git remote add upstream https://github.com/coqui-ai/TTS.git ``` -------------------------------- ### Install 🐸TTS for Development (Bash) Source: https://docs.coqui.ai/en/latest/contributing Instructions for installing 🐸TTS for development purposes on an Ubuntu system. This includes installing system dependencies and the package itself. ```bash make system-deps # intended to be used on Ubuntu (Debian). Let us know if you have a different OS. make install ``` -------------------------------- ### Install TTS locally for development Source: https://docs.coqui.ai/en/latest/index This command clones the 🐸TTS repository and installs it locally with development and notebook extras, intended for users who plan to code or train models. It requires git to be installed. ```bash git clone https://github.com/coqui-ai/TTS pip install -e .[all,dev,notebooks] # Select the relevant extras ``` -------------------------------- ### Install TTS with Pip Source: https://docs.coqui.ai/en/latest/tutorial_for_nervous_beginners Installs the TTS library using pip. This is a user-friendly method recommended for synthesizing voice. ```bash $ pip install TTS ``` -------------------------------- ### Install TTS from Source Source: https://docs.coqui.ai/en/latest/tutorial_for_nervous_beginners Clones the TTS repository from GitHub and installs it in developer mode using pip. This method is suitable for developers who need to modify or extend the library. ```bash $ git clone https://github.com/coqui-ai/TTS $ cd TTS $ pip install -e . ``` -------------------------------- ### Install TTS from Source Source: https://docs.coqui.ai/en/latest/installation Install Coqui TTS from its source code repository. This method is recommended for developers who require more control over the library or wish to contribute to its development. It involves cloning the repository and using make commands. ```bash git clone https://github.com/coqui-ai/TTS/ cd TTS make system-deps # only on Linux systems. make install ``` -------------------------------- ### Install system dependencies and TTS on Ubuntu Source: https://docs.coqui.ai/en/latest/index These commands are for Ubuntu (Debian) users to install system dependencies and then the 🐸TTS library. The `make system-deps` command is specifically intended for Ubuntu. ```bash $ make system-deps # intended to be used on Ubuntu (Debian). Let us know if you have a different OS. $ make install ``` -------------------------------- ### Install TTS from GitHub using pip Source: https://docs.coqui.ai/en/latest/installation Install the Coqui TTS library directly from its GitHub repository using pip. This allows for installation of the latest development version. ```bash pip install git+https://github.com/coqui-ai/TTS # from Github ``` -------------------------------- ### Start Coqui TTS Demo Server Source: https://docs.coqui.ai/en/latest/inference Starts a local demo server for Coqui TTS, providing a web interface for model inference. The server offers similar functionality to the CLI but is not optimized for performance. ```bash tts-server -h # see the help tts-server --list_models # list the available models. ``` -------------------------------- ### Install Coqui TTS using pip Source: https://docs.coqui.ai/en/latest/inference Installs the Coqui Text-to-Speech (TTS) library using pip. This is the recommended method for installation and makes the 'tts' and 'tts-server' commands available. ```bash pip install TTS ``` -------------------------------- ### Initialize Trainer and Start Training Source: https://docs.coqui.ai/en/latest/training_a_model Initializes the Trainer with necessary arguments, configuration, output path, model, and data samples. The `trainer.fit()` method then starts the model training process. ```python # INITIALIZE THE TRAINER # Trainer provides a generic API to train all the 🐸TTS models with all its perks like mixed-precision training, # distributed training, etc. trainer = Trainer( TrainerArgs(), config, output_path, model=model, train_samples=train_samples, eval_samples=eval_samples ) # AND... 3,2,1... 🚀 trainer.fit() ``` -------------------------------- ### Install TTS from PyPI using pip Source: https://docs.coqui.ai/en/latest/installation Install the Coqui TTS library from the Python Package Index (PyPI) using pip. This method is recommended for users who only need TTS for inference. ```bash pip install TTS # from PyPI ``` -------------------------------- ### Run Coqui TTS Docker Image Source: https://docs.coqui.ai/en/latest/index This snippet demonstrates how to run the Coqui TTS Docker image to use the Text-to-Speech server without local installation. It maps the necessary port and provides commands to list available models and start a server with a specific model. It's useful for quick testing and deployment. ```shell docker run --rm -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/coqui-ai/tts-cpu python3 TTS/server/server.py --list_models #To get the list of available models python3 TTS/server/server.py --model_name tts_models/en/vctk/vits # To start a server ``` -------------------------------- ### Commandline TTS Synthesis Source: https://docs.coqui.ai/en/latest/index Example of using the `tts` command-line interface to synthesize speech. This is a quick way to test models without writing Python code. ```bash tts --text "Hello world" --out_path output.wav ``` -------------------------------- ### Bark Model Inference Example Source: https://docs.coqui.ai/en/latest/index Example usage of the Bark text-to-audio model for inference. Bark is capable of generating speech, music, and sound effects. ```python from TTS.tts.configs.bark_config import BarkConfig from TTS.tts.models.bark import Bark config = BarkConfig() config.load_json("/path/to/bark_config.json") # Load your specific config model = Bark(config=config) model.load_checkpoint( model.get_config_path(), checkpoint_dir="/path/to/bark/checkpoints", eval=True ) sentence = "Hello, this is a test of the Bark model." audio_samples = model.inference(sentence) # Save audio_samples to a file (e.g., using scipy.io.wavfile.write) ``` -------------------------------- ### Start TensorBoard for Training Monitoring Source: https://docs.coqui.ai/en/latest/training_a_model Launches the TensorBoard web server to visualize training progress. The --logdir argument should point to the directory where training logs and checkpoints are stored. This allows for real-time monitoring of metrics and generated samples. ```bash tensorboard --logdir= ``` -------------------------------- ### XTTS Model Initialization Example (Python) Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/models/xtts Demonstrates how to initialize and load a checkpoint for the XTTS model using its configuration. This involves creating an XttsConfig object, instantiating the Xtts model, and then loading the model weights from a specified directory. ```python from TTS.tts.configs.xtts_config import XttsConfig from TTS.tts.models.xtts import Xtts config = XttsConfig() model = Xtts.inif_from_config(config) model.load_checkpoint(config, checkpoint_dir="paths/to/models_dir/", eval=True) ``` -------------------------------- ### Instantiate FastPitchConfig Source: https://docs.coqui.ai/en/latest/models/forward_tts This snippet demonstrates how to import and create an instance of the FastPitchConfig class from the Coqui TTS library. This is the starting point for configuring the FastPitch model. ```python from TTS.tts.configs.fast_pitch_config import FastPitchConfig config = FastPitchConfig() ``` -------------------------------- ### Build Docker Development Image (Docker) Source: https://docs.coqui.ai/en/latest/contributing This command builds a Docker image tagged as 'tts-dev:latest' using the specified Dockerfile. This image contains all dependencies required for development, simplifying environment setup. ```shell docker build --tag=tts-dev:latest -f .\dockerfiles\Dockerfile.dev . ``` -------------------------------- ### Run XTTS v2 Fine-tuning Demo Locally Source: https://docs.coqui.ai/en/latest/models/xtts Installs Gradio demo requirements and runs the XTTS_v2 fine-tuning demo script locally. This allows users to preprocess audio, train the GPT encoder, and perform inference. ```shell python3 -m pip install -r TTS/demos/xtts_ft_demo/requirements.txt python3 TTS/demos/xtts_ft_demo/xtts_demo.py ``` -------------------------------- ### ForwardTTS Model Configuration and Initialization (Python) Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/models/forward_tts This snippet shows the initialization of the ForwardTTS model, including setting up internal components like the encoder, decoder, and duration predictor based on the provided configuration. It also handles multi-speaker configurations and audio processing setup. ```python from TTS.tts.models.fast_pitch import ForwardTTS, ForwardTTSArgs from TTS.tts.configs.shared_configs import Coqpit from TTS.tts.utils.audio import AudioProcessor from TTS.tts.utils.text.tokenizer import TTSTokenizer from TTS.tts.layers.speaker_manager import SpeakerManager import torch.nn as nn # Assuming Coqpit, TTSTokenizer, AudioProcessor, SpeakerManager are defined elsewhere # For demonstration purposes, we'll use simplified placeholders if not provided class Coqpit: def __init__(self, **kwargs): self.__dict__.update(kwargs) class TTSTokenizer: pass class AudioProcessor: pass class SpeakerManager: pass # Placeholder for field decorator if not available def field(default_factory=None): if default_factory: return default_factory() return None class ForwardTTSArgs(Coqpit): def __init__(self, **kwargs): # Default values as seen in the provided text self.speaker_embedding_channels = kwargs.get('speaker_embedding_channels', 256) self.use_d_vector_file = kwargs.get('use_d_vector_file', False) self.d_vector_dim = kwargs.get('d_vector_dim', 0) self.num_chars = kwargs.get('num_chars', None) self.out_channels = kwargs.get('out_channels', 80) self.hidden_channels = kwargs.get('hidden_channels', 384) self.use_aligner = kwargs.get('use_aligner', True) self.use_pitch = kwargs.get('use_pitch', True) self.pitch_predictor_hidden_channels = kwargs.get('pitch_predictor_hidden_channels', 256) self.pitch_predictor_kernel_size = kwargs.get('pitch_predictor_kernel_size', 3) self.pitch_predictor_dropout_p = kwargs.get('pitch_predictor_dropout_p', 0.1) self.pitch_embedding_kernel_size = kwargs.get('pitch_embedding_kernel_size', 3) self.use_energy = kwargs.get('use_energy', False) self.energy_predictor_hidden_channels = kwargs.get('energy_predictor_hidden_channels', 256) self.energy_predictor_kernel_size = kwargs.get('energy_predictor_kernel_size', 3) self.energy_predictor_dropout_p = kwargs.get('energy_predictor_dropout_p', 0.1) self.energy_embedding_kernel_size = kwargs.get('energy_embedding_kernel_size', 3) self.duration_predictor_hidden_channels = kwargs.get('duration_predictor_hidden_channels', 256) self.duration_predictor_kernel_size = kwargs.get('duration_predictor_kernel_size', 3) self.duration_predictor_dropout_p = kwargs.get('duration_predictor_dropout_p', 0.1) self.positional_encoding = kwargs.get('positional_encoding', True) self.poisitonal_encoding_use_scale = kwargs.get('poisitonal_encoding_use_scale', True) self.length_scale = kwargs.get('length_scale', 1) self.encoder_type = kwargs.get('encoder_type', "fftransformer") self.encoder_params = kwargs.get('encoder_params', {"hidden_channels_ffn": 1024, "num_heads": 1, "num_layers": 6, "dropout_p": 0.1}) self.decoder_type = kwargs.get('decoder_type', "fftransformer") self.decoder_params = kwargs.get('decoder_params', {"hidden_channels_ffn": 1024, "num_heads": 1, "num_layers": 6, "dropout_p": 0.1}) self.detach_duration_predictor = kwargs.get('detach_duration_predictor', False) self.max_duration = kwargs.get('max_duration', 75) self.num_speakers = kwargs.get('num_speakers', 1) self.use_speaker_embedding = kwargs.get('use_speaker_embedding', False) self.speakers_file = kwargs.get('speakers_file', None) self.d_vector_file = kwargs.get('d_vector_file', None) super().__init__(**kwargs) class BaseTTS: def __init__(self, config: Coqpit, ap: AudioProcessor = None, tokenizer: TTSTokenizer = None, speaker_manager: SpeakerManager = None): self.config = config self.ap = ap self.tokenizer = tokenizer self.speaker_manager = speaker_manager self.args = config # Assuming args are directly accessible from config self.embedded_speaker_dim = 0 # Placeholder def _set_model_args(self, config): pass # Placeholder def init_multispeaker(self, config): pass # Placeholder class Encoder(nn.Module): def __init__(self, in_channels, out_channels, type, params, speaker_dim): super().__init__() pass # Placeholder class PositionalEncoding(nn.Module): def __init__(self, hidden_channels): super().__init__() pass # Placeholder class Decoder(nn.Module): def __init__(self, out_channels, hidden_channels, type, params): super().__init__() pass # Placeholder class DurationPredictor(nn.Module): def __init__(self, in_channels, hidden_channels, kernel_size, dropout_p): super().__init__() pass # Placeholder # Example Usage: config = ForwardTTSArgs() # model = ForwardTTS(config) # --- Actual class definition from provided text --- class ForwardTTS(BaseTTS): """General forward TTS model implementation that uses an encoder-decoder architecture with an optional alignment network and a pitch predictor. If the alignment network is used, the model learns the text-to-speech alignment from the data instead of using pre-computed durations. If the pitch predictor is used, the model trains a pitch predictor that predicts average pitch value for each input character as in the FastPitch model. `ForwardTTS` can be configured to one of these architectures, - FastPitch - SpeedySpeech - FastSpeech - FastSpeech2 (requires average speech energy predictor) Args: config (Coqpit): Model coqpit class. speaker_manager (SpeakerManager): Speaker manager for multi-speaker training. Only used for multi-speaker models. Defaults to None. Examples: >>> from TTS.tts.models.fast_pitch import ForwardTTS, ForwardTTSArgs >>> config = ForwardTTSArgs() >>> model = ForwardTTS(config) """ # pylint: disable=dangerous-default-value def __init__( self, config: Coqpit, ap: "AudioProcessor" = None, tokenizer: "TTSTokenizer" = None, speaker_manager: SpeakerManager = None, ): super().__init__(config, ap, tokenizer, speaker_manager) self._set_model_args(config) self.init_multispeaker(config) self.max_duration = self.args.max_duration self.use_aligner = self.args.use_aligner self.use_pitch = self.args.use_pitch self.use_energy = self.args.use_energy self.binary_loss_weight = 0.0 self.length_scale = \ float(self.args.length_scale) if isinstance(self.args.length_scale, int) else self.args.length_scale self.emb = nn.Embedding(self.args.num_chars, self.args.hidden_channels) self.encoder = Encoder( self.args.hidden_channels, self.args.hidden_channels, self.args.encoder_type, self.args.encoder_params, self.embedded_speaker_dim, ) if self.args.positional_encoding: self.pos_encoder = PositionalEncoding(self.args.hidden_channels) self.decoder = Decoder( self.args.out_channels, self.args.hidden_channels, self.args.decoder_type, self.args.decoder_params, ) self.duration_predictor = DurationPredictor( self.args.hidden_channels, self.args.duration_predictor_hidden_channels, self.args.duration_predictor_kernel_size, self.args.duration_predictor_dropout_p, ) if self.args.use_pitch: self.pitch_predictor = DurationPredictor( self.args.hidden_channels, self.args.pitch_predictor_hidden_channels, self.args.pitch_predictor_kernel_size, self.args.pitch_predictor_dropout_p, ) self.pitch_emb = nn.Conv1d( 1, self.args.hidden_channels, ``` -------------------------------- ### Start Coqui TTS Server with Docker Source: https://docs.coqui.ai/en/latest/docker_images Starts a Coqui TTS server within a Docker container, exposing the server on port 5002. This allows you to interact with TTS models remotely. You can list available models or specify a model to load. Ensure NVIDIA drivers are installed for GPU usage. ```shell docker run --rm -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/coqui-ai/tts-cpu python3 TTS/server/server.py --list_models #To get the list of available models python3 TTS/server/server.py --model_name tts_models/en/vctk/vits ``` ```shell docker run --rm -it -p 5002:5002 --gpus all --entrypoint /bin/bash ghcr.io/coqui-ai/tts python3 TTS/server/server.py --list_models #To get the list of available models python3 TTS/server/server.py --model_name tts_models/en/vctk/vits --use_cuda true ``` -------------------------------- ### Synthesize Speech using Tortoise TTS Command Line Source: https://docs.coqui.ai/en/latest/models/tortoise Provides command-line examples for using Tortoise TTS. It covers cloning a specific voice by specifying the model name, text, output path, voice directory, and speaker index. It also demonstrates generating speech with a random voice. Requires the TTS command-line tool to be installed. ```bash # cloning the `lj` voice tts --model_name tts_models/en/multi-dataset/tortoise-v2 \ --text "This is an example." \ --out_path "output.wav" \ --voice_dir path/to/tortoise/voices/dir/ \ --speaker_idx "lj" \ --progress_bar True # Random voice generation tts --model_name tts_models/en/multi-dataset/tortoise-v2 \ --text "This is an example." \ --out_path "output.wav" \ --progress_bar True ``` -------------------------------- ### Synthesize Speech and Manage Models via CLI Source: https://docs.coqui.ai/en/latest/tutorial_for_nervous_beginners These commands allow you to interact with the Coqui TTS system directly from your terminal. You can view help information, list available models, and start a local demo server for speech synthesis. These commands are useful for quick testing and deployment. ```bash $ tts -h # see the help $ tts --list_models # list the available models. ``` ```bash $ tts-server -h # see the help $ tts-server --list_models # list the available models. ``` -------------------------------- ### Run TTS Training Script Source: https://docs.coqui.ai/en/latest/tutorial_for_nervous_beginners Executes the TTS training script. It can be run on a specific GPU or multiple GPUs. Options to continue a previous run or restore from a checkpoint are also provided. ```bash CUDA_VISIBLE_DEVICES=0 python train.py ``` ```bash CUDA_VISIBLE_DEVICES=0 python train.py --continue_path path/to/previous/run/folder/ ``` ```bash CUDA_VISIBLE_DEVICES=0 python train.py --restore_path path/to/model/checkpoint.pth ``` ```bash CUDA_VISIBLE_DEVICES=0,1,2 python -m trainer.distribute --script train.py ``` -------------------------------- ### Run 🐸TTS Tests (Bash) Source: https://docs.coqui.ai/en/latest/contributing These commands are used to run the test suite for 🐸TTS. `make test` stops at the first error, while `make test_all` runs all tests and reports all errors. ```bash make test # stop at the first error make test_all # run all the tests, report all the errors ``` -------------------------------- ### Format and Lint 🐸TTS Code (Bash) Source: https://docs.coqui.ai/en/latest/contributing Commands to format the code using `black` and `isort`, and to lint the code using `pylint` to ensure adherence to coding standards. ```bash make style make lint ``` -------------------------------- ### Configure and Train TTS Model via CLI Source: https://docs.coqui.ai/en/latest/tutorial_for_nervous_beginners This snippet shows how to define a JSON configuration file for training a TTS model and then initiate the training process using the command line. It requires a `config.json` file and the `train_tts.py` script. ```json { "run_name": "my_run", "model": "glow_tts", "batch_size": 32, "eval_batch_size": 16, "num_loader_workers": 4, "num_eval_loader_workers": 4, "run_eval": true, "test_delay_epochs": -1, "epochs": 1000, "text_cleaner": "english_cleaners", "use_phonemes": false, "phoneme_language": "en-us", "phoneme_cache_path": "phoneme_cache", "print_step": 25, "print_eval": true, "mixed_precision": false, "output_path": "recipes/ljspeech/glow_tts/", "datasets":[{"formatter": "ljspeech", "meta_file_train":"metadata.csv", "path": "recipes/ljspeech/LJSpeech-1.1/"}] } ``` ```bash $ CUDA_VISIBLE_DEVICES="0" python TTS/bin/train_tts.py --config_path config.json ``` -------------------------------- ### Train GlowTTS Model (Pure Python) Source: https://docs.coqui.ai/en/latest/tutorial_for_nervous_beginners Defines and initiates the training process for a GlowTTS model using a pure Python script. It configures the dataset, model parameters, audio processor, tokenizer, and trainer. Requires the LJSpeech dataset to be downloaded. ```python import os from trainer import Trainer, TrainerArgs from TTS.tts.configs.glow_tts_config import GlowTTSConfig from TTS.tts.configs.shared_configs import BaseDatasetConfig from TTS.tts.datasets import load_tts_samples from TTS.tts.models.glow_tts import GlowTTS from TTS.tts.utils.text.tokenizer import TTSTokenizer from TTS.utils.audio import AudioProcessor output_path = os.path.dirname(os.path.abspath(__file__)) dataset_config = BaseDatasetConfig( formatter="ljspeech", meta_file_train="metadata.csv", path=os.path.join(output_path, "../LJSpeech-1.1/") ) config = GlowTTSConfig( batch_size=32, eval_batch_size=16, num_loader_workers=4, num_eval_loader_workers=4, run_eval=True, test_delay_epochs=-1, epochs=1000, text_cleaner="phoneme_cleaners", use_phonemes=True, phoneme_language="en-us", phoneme_cache_path=os.path.join(output_path, "phoneme_cache"), print_step=25, print_eval=False, mixed_precision=True, output_path=output_path, datasets=[dataset_config], ) ap = AudioProcessor.init_from_config(config) tokenizer, config = TTSTokenizer.init_from_config(config) train_samples, eval_samples = load_tts_samples( dataset_config, eval_split=True, eval_split_max_size=config.eval_split_max_size, eval_split_size=config.eval_split_size, ) model = GlowTTS(config, ap, tokenizer, speaker_manager=None) trainer = Trainer( TrainerArgs(), config, output_path, model=model, train_samples=train_samples, eval_samples=eval_samples ) trainer.fit() ``` -------------------------------- ### Bash: Install DeepSpeed for TTS Model Acceleration Source: https://docs.coqui.ai/en/latest/models/xtts Installs the DeepSpeed library, version 0.10.3, which is required for potential speedups when loading TTS models with `use_deepspeed=True`. This command is run in a terminal or package manager. ```bash pip install deepspeed==0.10.3 ``` -------------------------------- ### Control Data-Dependent Initialization Start (Python) Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/models/glow_tts Determines whether to enable or disable data-dependent initialization at the start of each training step. It checks if the current total steps done is less than the specified steps for data-dependent initialization. ```python def on_train_step_start(self, trainer): """Decide on every training step wheter enable/disable data depended initialization.""" self.run_data_dep_init = trainer.total_steps_done < self.data_dep_init_steps ``` -------------------------------- ### Train TTS Model using Multi-GPUs Source: https://docs.coqui.ai/en/latest/faq Command to launch distributed training of a TTS model across multiple GPUs. This utilizes the `trainer.distribute` module and requires specifying the GPU IDs and the training script. ```bash python3 -m trainer.distribute --gpus "0,1" --script TTS/bin/train_tts.py --config_path config.json ``` -------------------------------- ### Handle Initialization Start for Normalization Statistics Source: https://docs.coqui.ai/en/latest/models/overflow Manages the start of the initialization process for the Overflow TTS model. If the dataset lacks normalisation statistics or initialisation transition probabilities, it computes them; otherwise, it loads existing ones. ```python Overflow.on_init_start(_trainer_) ``` -------------------------------- ### Download LJSpeech Dataset Source: https://docs.coqui.ai/en/latest/tutorial_for_nervous_beginners Downloads the LJSpeech dataset using a Python script. This dataset is commonly used for training TTS models. The download path can be specified. ```python $ python -c 'from TTS.utils.downloaders import download_ljspeech; download_ljspeech("../recipes/ljspeech/");' ``` -------------------------------- ### Commit Changes in 🐸TTS (Bash) Source: https://docs.coqui.ai/en/latest/contributing Steps to add new or modified files to the staging area and commit them with a descriptive message. ```bash git add my_file1.py my_file2.py ... git commit ``` -------------------------------- ### Define Test Sentences Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/configs/fast_speech_config Provides a default list of sentences to be used for testing. These sentences cover various linguistic structures and examples. ```python # testing test_sentences: List[str] = field( default_factory=lambda: [ "It took me quite a long time to develop a voice, and now that I have it I'm not going to be silent.", "Be a voice, not an echo.", "I'm sorry Dave. I'm afraid I can't do that.", "This cake is great. It's so delicious and moist.", "Prior to November 22, 1963.", ] ) ``` -------------------------------- ### Enable Discriminator Training Source: https://docs.coqui.ai/en/latest/_modules/TTS/vocoder/models/gan Enables discriminator training based on the total steps completed by the trainer. This method is called at the start of each training step. ```python def on_train_step_start(self, trainer) -> None: """Enable the discriminator training based on `steps_to_start_discriminator` Args: trainer (Trainer): Trainer object. """ self.train_disc = trainer.total_steps_done >= self.config.steps_to_start_discriminator ``` -------------------------------- ### BaseTacotron: Get Model Criterion Source: https://docs.coqui.ai/en/latest/models/tacotron1-2 Retrieves the loss criterion used during model training. This method is part of the BaseTacotron class. ```python get_criterion() ``` -------------------------------- ### Get Padding ID in F0Dataset Python Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/datasets/dataset Returns the padding ID used in the F0Dataset. This is typically a special value indicating padding in the F0 sequences. ```python def get_pad_id(self): return self.pad_id ``` -------------------------------- ### Train Vocoder Model using Python API Source: https://docs.coqui.ai/en/latest/tutorial_for_nervous_beginners This Python script demonstrates how to train a vocoder model (specifically Hifigan) using the Coqui TTS library. It involves configuring `HifiganConfig`, initializing an `AudioProcessor`, loading audio data, instantiating the `GAN` model, and running the `Trainer`. Ensure you have the necessary audio data path set. ```python import os from trainer import Trainer, TrainerArgs from TTS.utils.audio import AudioProcessor from TTS.vocoder.configs import HifiganConfig from TTS.vocoder.datasets.preprocess import load_wav_data from TTS.vocoder.models.gan import GAN output_path = os.path.dirname(os.path.abspath(__file__)) config = HifiganConfig( batch_size=32, eval_batch_size=16, num_loader_workers=4, num_eval_loader_workers=4, run_eval=True, test_delay_epochs=5, epochs=1000, seq_len=8192, pad_short=2000, use_noise_augment=True, eval_split_size=10, print_step=25, print_eval=False, mixed_precision=False, lr_gen=1e-4, lr_disc=1e-4, data_path=os.path.join(output_path, "../LJSpeech-1.1/wavs/"), output_path=output_path, ) # init audio processor ap = AudioProcessor(**config.audio.to_dict()) # load training samples eval_samples, train_samples = load_wav_data(config.data_path, config.eval_split_size) # init model model = GAN(config, ap) # init the trainer and 🚀 trainer = Trainer( TrainerArgs(), config, output_path, model=model, train_samples=train_samples, eval_samples=eval_samples ) trainer.fit() ``` -------------------------------- ### Get TTS Loss Criterion Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/models/forward_tts Retrieves the loss criterion used for training the TTS model. This function imports and returns an instance of `ForwardTTSLoss`. ```python def get_criterion(self): from TTS.tts.layers.losses import ForwardTTSLoss # pylint: disable=import-outside-toplevel return ForwardTTSLoss ``` -------------------------------- ### Create and Checkout a New Branch (Bash) Source: https://docs.coqui.ai/en/latest/contributing This command demonstrates how to create a new Git branch with a descriptive name for your planned changes. ```bash git checkout -b an_informative_name_for_my_branch ``` -------------------------------- ### Sync Local 🐸TTS Repo with Upstream (Bash) Source: https://docs.coqui.ai/en/latest/contributing Instructions for fetching the latest changes from the upstream repository and rebasing your local branch to incorporate them. ```bash git fetch upstream git rebase upstream/master # or for the development version git rebase upstream/dev ``` -------------------------------- ### Get GAN Learning Rates Source: https://docs.coqui.ai/en/latest/_modules/TTS/vocoder/models/gan Returns a list containing the initial learning rates for the discriminator and generator optimizers, as defined in the model's configuration. ```python def get_lr(self) -> List: """Set the initial learning rates for each optimizer. Returns: List: learning rates for each optimizer. """ return [self.config.lr_disc, self.config.lr_gen] ``` -------------------------------- ### DataLoader Initialization and Configuration Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/datasets/dataset Initializes the DataLoader with various parameters for processing audio and text samples. It sets up datasets for phonemes, F0, and energy based on configuration flags, and handles sample preprocessing. ```python def __init__(self, samples, tokenizer, ap, compute_f0=True, compute_energy=True, f0_cache_path=None, energy_cache_path=None, min_audio_len=0, max_audio_len=float('inf'), min_text_len=0, max_text_len=500, ap_phoneme=None, phoneme_cache_path=None, speaker_id_mapping=None, d_vector_mapping=None, language_id_mapping=None, use_noise_augment=False, start_by_longest=False, verbose=False, precompute_num_workers=0): self.compute_f0 = compute_f0 self.compute_energy = compute_energy self.f0_cache_path = f0_cache_path self.energy_cache_path = energy_cache_path self.min_audio_len = min_audio_len self.max_audio_len = max_audio_len self.min_text_len = min_text_len self.max_text_len = max_text_len self.ap = ap self.phoneme_cache_path = phoneme_cache_path self.speaker_id_mapping = speaker_id_mapping self.d_vector_mapping = d_vector_mapping self.language_id_mapping = language_id_mapping self.use_noise_augment = use_noise_augment self.start_by_longest = start_by_longest self.verbose = verbose self.rescue_item_idx = 1 self.pitch_computed = False self.tokenizer = tokenizer if self.tokenizer.use_phonemes: self.phoneme_dataset = PhonemeDataset( self.samples, self.tokenizer, phoneme_cache_path, precompute_num_workers=precompute_num_workers ) if compute_f0: self.f0_dataset = F0Dataset( self.samples, self.ap, cache_path=f0_cache_path, precompute_num_workers=precompute_num_workers ) if compute_energy: self.energy_dataset = EnergyDataset( self.samples, self.ap, cache_path=energy_cache_path, precompute_num_workers=precompute_num_workers ) if self.verbose: self.print_logs() ``` -------------------------------- ### Push Branch and Create Pull Request (Bash) Source: https://docs.coqui.ai/en/latest/contributing This snippet shows how to push your local branch to your fork and initiate a pull request on GitHub. ```bash git push -u origin an_informative_name_for_my_branch ``` -------------------------------- ### Get GAN Loss Criterions Source: https://docs.coqui.ai/en/latest/_modules/TTS/vocoder/models/gan Returns a list of loss functions for the GAN: one for the discriminator and one for the generator. These are initialized using the model's configuration. ```python def get_criterion(self): """Return criterions for the optimizers""" return [DiscriminatorLoss(self.config), GeneratorLoss(self.config)] ``` -------------------------------- ### BaseTacotron: On Epoch Start Callback Source: https://docs.coqui.ai/en/latest/models/tacotron1-2 A callback function executed at the beginning of each training epoch. Used for adjusting model parameters according to a training schedule. ```python on_epoch_start(_trainer_) ``` -------------------------------- ### Get Duration of WAV File using Librosa Source: https://docs.coqui.ai/en/latest/main_classes/audio_processor Retrieves the duration of a WAV audio file. Accepts the file path as a string and returns the duration, typically in seconds. ```python # Example of getting audio duration # duration = ap.get_duration('path/to/your/audio.wav') ``` -------------------------------- ### Bark TTS Model Initialization and Synthesis (Python) Source: https://docs.coqui.ai/en/latest/models/bark Demonstrates how to initialize and use the Bark TTS model directly. This involves loading the configuration, initializing the model, and synthesizing speech from text, with options for random speakers or cloning existing ones. Requires the TTS library and a pre-trained model checkpoint. ```python text = "Hello, my name is Manmay , how are you?" from TTS.tts.configs.bark_config import BarkConfig from TTS.tts.models.bark import Bark config = BarkConfig() model = Bark.init_from_config(config) model.load_checkpoint(config, checkpoint_dir="path/to/model/dir/", eval=True) # with random speaker output_dict = model.synthesize(text, config, speaker_id="random", voice_dirs=None) # cloning a speaker. # It assumes that you have a speaker file in `bark_voices/speaker_n/speaker.wav` or `bark_voices/speaker_n/speaker.npz` output_dict = model.synthesize(text, config, speaker_id="ljspeech", voice_dirs="bark_voices/") ``` -------------------------------- ### Get TTS Loss Criterion (Python) Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/models/glow_tts Returns the appropriate loss function for the TTS model. Currently, it provides the GlowTTSLoss. This method is static and does not require an instance of the model. ```python @staticmethod def get_criterion(): from TTS.tts.layers.losses import GlowTTSLoss # pylint: disable=import-outside-toplevel return GlowTTSLoss() ``` -------------------------------- ### Train TTS Model using Single GPU Source: https://docs.coqui.ai/en/latest/faq Command to initiate the training of a TTS model on a single GPU. It requires specifying the configuration file path. Ensure CUDA is properly set up and the `train_tts.py` script is accessible. ```bash CUDA_VISIBLE_DEVICES="0" python train_tts.py --config_path config.json ``` -------------------------------- ### Get GAN Learning Rate Schedulers Source: https://docs.coqui.ai/en/latest/_modules/TTS/vocoder/models/gan Sets up and returns learning rate schedulers for both the generator and discriminator optimizers. It uses configuration parameters to determine the scheduler type and its parameters. ```python def get_scheduler(self, optimizer) -> List: """Set the schedulers for each optimizer. Args: optimizer (List[`torch.optim.Optimizer`]): List of optimizers. Returns: List: Schedulers, one for each optimizer. """ scheduler1 = get_scheduler(self.config.lr_scheduler_gen, self.config.lr_scheduler_gen_params, optimizer[0]) scheduler2 = get_scheduler(self.config.lr_scheduler_disc, self.config.lr_scheduler_disc_params, optimizer[1]) return [scheduler2, scheduler1] ``` -------------------------------- ### Get Tacotron Model Training Criterion Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/models/base_tacotron Returns the loss function (criterion) used during the training of the Tacotron model. It utilizes the `TacotronLoss` class, configured with the model's settings. ```python def get_criterion(self) -> nn.Module: """Get the model criterion used in training.""" return TacotronLoss(self.config) ``` -------------------------------- ### XTTS Model Initialization and Configuration (Python) Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/models/xtts Shows the core initialization logic for the Xtts class, inheriting from BaseTTS. It sets up essential components like the tokenizer, GPT model, and registers necessary buffers, while also handling checkpoint loading paths and batch size configurations. ```python class Xtts(BaseTTS): def __init__(self, config: Coqpit): super().__init__(config, ap=None, tokenizer=None) self.mel_stats_path = None self.config = config self.gpt_checkpoint = self.args.gpt_checkpoint self.decoder_checkpoint = self.args.decoder_checkpoint self.models_dir = config.model_dir self.gpt_batch_size = self.args.gpt_batch_size self.tokenizer = VoiceBpeTokenizer() self.gpt = None self.init_models() self.register_buffer("mel_stats", torch.ones(80)) ``` -------------------------------- ### Get TTS Model Information by Index Source: https://docs.coqui.ai/en/latest/index Fetches information for a TTS or vocoder model using its index number, as provided by the `--list_models` command. This offers an alternative way to query model details. ```bash tts --model_info_by_idx "/" ``` ```bash tts --model_info_by_idx tts_models/3 ``` -------------------------------- ### Verbose Logging for DataLoader Initialization Source: https://docs.coqui.ai/en/latest/_modules/TTS/tts/datasets/dataset Prints detailed initialization logs for the DataLoader, including tokenizer information and the number of samples. This is controlled by the 'verbose' flag during initialization. ```python def print_logs(self, level: int = 0) -> None: indent = "\t" * level print("\n") print(f"{indent}> DataLoader initialization") print(f"{indent}| > Tokenizer:") self.tokenizer.print_logs(level + 1) print(f"{indent}| > Number of instances : {len(self.samples)}") ``` -------------------------------- ### Get Locales from Mary-TTS API (cURL) Source: https://docs.coqui.ai/en/latest/marytts This cURL command retrieves a list of supported locales from a running Mary-TTS compatible server. It's useful for checking available language settings. ```shell curl http://localhost:59125/locales ``` -------------------------------- ### Setup Spectrogram Scalers (Python) Source: https://docs.coqui.ai/en/latest/_modules/TTS/utils/audio/processor Initializes StandardScaler objects for both mel and linear spectrograms using provided mean and standard deviation values. This method is essential for applying mean-std normalization. ```python def setup_scaler( self, mel_mean: np.ndarray, mel_std: np.ndarray, linear_mean: np.ndarray, linear_std: np.ndarray ) -> None: """Initialize scaler objects used in mean-std normalization. Args: mel_mean (np.ndarray): Mean for melspectrograms. mel_std (np.ndarray): STD for melspectrograms. linear_mean (np.ndarray): Mean for full scale spectrograms. linear_std (np.ndarray): STD for full scale spectrograms. """ self.mel_scaler = StandardScaler() self.mel_scaler.set_stats(mel_mean, mel_std) self.linear_scaler = StandardScaler() self.linear_scaler.set_stats(linear_mean, linear_std) ``` -------------------------------- ### Train a TTS Model Source: https://docs.coqui.ai/en/latest/index Illustrates the basic command for training a Text-to-Speech (TTS) model. This typically requires a configuration file specifying dataset paths and model parameters. ```bash tts --train_configs /path/to/your/config.json --restore_path /path/to/your/pretrained/model --output_path /path/to/output/directory ```