### Training Log Output Example Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/training/training_a_model.md An example of the initial output lines from a training log, showing experiment details and setup information. ```console > Experiment folder: /your/output_path/-Juni-23-2021_02+52-78899209 > Using CUDA: True > Number of GPUs: 1 > Setting up Audio Processor... | > sample_rate:22050 | > resample:False | > num_mels:80 | > min_level_db:-100 ``` -------------------------------- ### Start HiFiGAN Model Training Source: https://github.com/idiap/coqui-ai-tts/blob/dev/recipes/bel-alex73/README.md Initiates the training process for the HiFiGAN model. Ensure you have the necessary directories and dependencies installed. Adjust `` to your GPU IDs. ```bash OMP_NUM_THREADS=2 CUDA_VISIBLE_DEVICES= python3 -m trainer.distribute --script recipes/bel-alex73/train_hifigan.py ``` -------------------------------- ### Start GlowTTS Model Training Source: https://github.com/idiap/coqui-ai-tts/blob/dev/recipes/bel-alex73/README.md Initiates the training process for the GlowTTS model. Ensure you have the necessary directories and dependencies installed. Adjust `` to your GPU IDs. ```bash OMP_NUM_THREADS=2 CUDA_VISIBLE_DEVICES= python3 -m trainer.distribute --script recipes/bel-alex73/train_glowtts.py ``` -------------------------------- ### Start Coqui TTS Demo Server (CLI) Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/tutorial_for_nervous_beginners.md Run `tts-server` to launch a local web server for interactive speech synthesis. Ensure `coqui-tts[server]` is installed. ```bash $ tts-server --list_models # list the available models. ``` -------------------------------- ### Install and Run Tensorboard Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/Tutorial_2_train_your_first_TTS_model.ipynb Install the Tensorboard package and launch it to monitor the training progress, view figures, and sample outputs. The log directory should be set to your training output path. ```bash !pip install tensorboard !tensorboard --logdir=tts_train_dir ``` -------------------------------- ### Install Coqui TTS Locally from Source Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/installation.md Clones the Coqui TTS repository and installs it locally using uv, recommended for development or training models. ```bash git clone https://github.com/idiap/coqui-ai-TTS cd coqui-ai-TTS uv pip install -e . ``` -------------------------------- ### Install Development Dependencies Source: https://github.com/idiap/coqui-ai-tts/blob/dev/CONTRIBUTING.md Installs system dependencies (intended for Ubuntu/Debian) and then installs the project's development dependencies using make targets. ```bash make system-deps make install_dev ``` -------------------------------- ### Install System Dependencies and Coqui TTS on Ubuntu Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/installation.md Installs system dependencies and Coqui TTS on Ubuntu using make commands. ```bash make system-deps make install ``` -------------------------------- ### Start Fine-tuning with TTS/bin/train_tts.py Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/training/finetuning.md Starts fine-tuning using the generic `train_tts.py` script. Requires specifying both the configuration path and the pre-trained model path. ```python CUDA_VISIBLE_DEVICES="0" python TTS/bin/train_tts.py \ --config_path /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts/config.json \ --restore_path /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts/model_file.pth ``` -------------------------------- ### Install PyTorch and Dependencies with uv Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/installation.md Installs PyTorch, torchaudio, and torchcodec using uv, automatically selecting the appropriate version for your system. ```bash uv pip install torch torchaudio torchcodec --torch-backend=auto ``` -------------------------------- ### Dataset Folder Structure Example Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/datasets/formatting_your_dataset.md Organize your audio clips in a dedicated folder named 'wavs'. ```text /wavs | - audio1.wav | - audio2.wav | - audio3.wav ... ``` -------------------------------- ### Install Dependencies for XTTS Finetuning Source: https://github.com/idiap/coqui-ai-tts/blob/dev/TTS/demos/xtts_ft_demo/XTTS_finetune_colab.ipynb Installs the coqui-ai-TTS repository and necessary libraries like Gradio and faster-whisper. It also includes a command to remove the existing repository to ensure a clean reinstallation. ```bash !rm -rf coqui-ai-TTS/ # delete repo to be able to reinstall if needed !git clone -q https://github.com/idiap/coqui-ai-TTS.git !pip install -q -e coqui-ai-TTS !pip install -q gradio faster_whisper ``` -------------------------------- ### Start TTS Training from CLI Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/tutorial_for_nervous_beginners.md Execute this command to start a TTS training process using a specified configuration file. Ensure CUDA_VISIBLE_DEVICES is set if using a GPU. ```bash $ CUDA_VISIBLE_DEVICES="0" python TTS/bin/train_tts.py --config_path config.json ``` -------------------------------- ### Start Docker Compose Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/docker_images.md Starts the Docker Compose services defined in the compose.yaml file. ```bash docker-compose up ``` -------------------------------- ### Install Coqui TTS Fork Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/faq.md Uninstall previous versions of TTS and install the recommended coqui-tts fork for compatibility and new features. It is advised to start with a new virtual environment. ```bash pip uninstall TTS trainer coqpit pip cache purge pip install coqui-tts ``` -------------------------------- ### Example: Run TTS with Glow-TTS and UnivNet Vocoder Source: https://github.com/idiap/coqui-ai-tts/blob/dev/README.md An example specifying the Glow-TTS model and the UnivNet vocoder for English LJSpeech. ```sh tts --text "Text for TTS" \ --model_name "tts_models/en/ljspeech/glow-tts" \ --vocoder_name "vocoder_models/en/ljspeech/univnet" \ --out_path output/path/speech.wav ``` -------------------------------- ### Example: Run TTS with Glow-TTS Model Source: https://github.com/idiap/coqui-ai-tts/blob/dev/README.md An example demonstrating how to run TTS with a specific model, the Glow-TTS model for English LJSpeech. ```sh tts --text "Text for TTS" \ --model_name "tts_models/en/ljspeech/glow-tts" \ --out_path output/path/speech.wav ``` -------------------------------- ### Install uv for Environment Management Source: https://github.com/idiap/coqui-ai-tts/blob/dev/CONTRIBUTING.md Installs uv, a fast Python package installer and virtual environment manager, using a standalone script. This is a prerequisite for managing the project's development environment. ```bash curl -LsSf https://astral.sh/uv/install.sh | sh ``` -------------------------------- ### Start TTS Model Training Source: https://context7.com/idiap/coqui-ai-tts/llms.txt Initiate TTS model training using a configuration file. This command launches the training process with specified parameters. ```bash python -m TTS.bin.train_tts --config_path recipes/ljspeech/glow_tts/config.json ``` -------------------------------- ### Launch XTTS Fine-tuning Demo Source: https://context7.com/idiap/coqui-ai-tts/llms.txt Start the Gradio web application for XTTS fine-tuning. This command launches the UI on a specified port and configures training parameters. ```bash python TTS/demos/xtts_ft_demo/xtts_demo.py \ --port 5003 \ --out_path /tmp/xtts_ft/ \ --num_epochs 10 \ --batch_size 4 \ --grad_acumm 1 \ --max_audio_length 11 ``` -------------------------------- ### Start TTS Flask HTTP Server Source: https://context7.com/idiap/coqui-ai-tts/llms.txt Launch a web server wrapping the TTS API with REST, MaryTTS-compatible, and OpenAI-compatible endpoints. Requires `pip install coqui-tts[server]`. Supports GPU and Docker. ```sh # Start server with a pretrained model on port 5002 tts-server --model_name tts_models/en/ljspeech/tacotron2-DDC --port 5002 # Start server with a multi-speaker VITS model on GPU tts-server --model_name tts_models/en/vctk/vits --device cuda --port 5002 # Docker (CPU image) docker run --rm -it -p 5002:5002 ghcr.io/idiap/coqui-tts-cpu \ python3 TTS/server/server.py --model_name tts_models/en/ljspeech/tacotron2-DDC ``` ```sh # Native /api/tts endpoint — returns audio/wav curl -G "http://localhost:5002/api/tts" \ --data-urlencode "text=Hello from the TTS server" \ --output tts_response.wav ``` -------------------------------- ### Install Coqui TTS Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/Tutorial_1_use-pretrained-TTS.ipynb Installs the Coqui TTS library and its dependencies. Ensure pip is up-to-date before installation. ```python ! pip install -U pip ! pip install coqui-tts ``` -------------------------------- ### Install Coqui TTS Locally with Optional Dependencies Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/installation.md Installs Coqui TTS from a local clone with specific optional dependencies, such as server and Japanese G2P support, using uv. ```bash uv pip install -e .[server,ja] ``` -------------------------------- ### Install DeepSpeed Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/models/xtts.md Install the deepspeed library to enable faster inference with `load_checkpoint` and `use_deepspeed=True`. ```console pip install deepspeed ``` -------------------------------- ### Install Coqui TTS from PyPI Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/installation.md Installs the Coqui TTS package from PyPI using uv, suitable for users only needing speech synthesis with pretrained models. ```bash uv pip install coqui-tts ``` -------------------------------- ### Install Coqui TTS with Optional Dependencies Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/installation.md Installs Coqui TTS with specific optional dependencies, such as server and Japanese G2P support, using uv. ```bash uv pip install coqui-tts[server,ja] ``` -------------------------------- ### Voice Conversion via CLI Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/vc.md This command-line interface example shows how to perform voice conversion. It specifies the model, source audio, target audio(s), and output path. ```bash tts --model_name "voice_conversion_models/multilingual/multi-dataset/knnvc" \ --source_wav "source.wav" \ --target_wav "target1.wav" "target2.wav" \ --out_path "output.wav" ``` -------------------------------- ### Start Model Training Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/Tutorial_2_train_your_first_TTS_model.ipynb Initiate the training process for the TTS model by calling the `fit` method on the initialized Trainer object. ```python trainer.fit() ``` -------------------------------- ### Start Fine-tuning with recipes/ljspeech/glow_tts/train_glowtts.py Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/training/finetuning.md Initiates fine-tuning using a specific training script. The `--restore_path` flag is crucial for specifying the pre-trained model to fine-tune. ```python CUDA_VISIBLE_DEVICES="0" python recipes/ljspeech/glow_tts/train_glowtts.py \ --restore_path /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts/model_file.pth ``` -------------------------------- ### Metadata File Format Example (LJSpeech-like) Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/datasets/formatting_your_dataset.md Create a metadata file (e.g., metadata.txt) where each line maps an audio file to its transcription and normalized transcription, separated by '|'. ```text # metadata.txt audio1|This is my sentence.|This is my sentence. audio2|1469 and 1470|fourteen sixty-nine and fourteen seventy audio3|It'll be $16 sir.|It'll be sixteen dollars sir. ... ``` -------------------------------- ### Define Filename and Directory Setup Function Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/ExtractTTSpectrogram.ipynb A utility function to create necessary output directories and derive filenames for processed audio and spectrograms. It ensures that 'quant' and 'mel' subdirectories exist. ```python # Function to create directories and file names def set_filename(wav_path, out_path): wav_file = os.path.basename(wav_path) file_name = wav_file.split(".")[0] os.makedirs(os.path.join(out_path, "quant"), exist_ok=True) os.makedirs(os.path.join(out_path, "mel"), exist_ok=True) wavq_path = os.path.join(out_path, "quant", file_name) mel_path = os.path.join(out_path, "mel", file_name) return file_name, wavq_path, mel_path ``` -------------------------------- ### Run Coqui TTS Training Script (CLI) Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/tutorial_for_nervous_beginners.md Execute the Python training script using `CUDA_VISIBLE_DEVICES` to specify the GPU. This command starts the training process. ```bash CUDA_VISIBLE_DEVICES=0 python train.py ``` -------------------------------- ### Implement Voice Cloning - XTTS Example Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/cloning.md Implement the _clone_voice method to return speaker embeddings and metadata. This method is specific to the model being developed. ```python def _clone_voice( self, speaker_wav: str | os.PathLike[Any] | list[str | os.PathLike[Any]], **generate_kwargs: Any ) -> tuple[dict[str, Any], dict[str, Any]]: gpt_conditioning_latents, speaker_embedding = self.get_conditioning_latents( audio_path=speaker_wav, **generate_kwargs, ) voice = {"gpt_conditioning_latents": gpt_conditioning_latents, "speaker_embedding": speaker_embedding} metadata = {"name": self.config["model"]} return voice, metadata ``` -------------------------------- ### Start TTS Server with CPU Docker Image Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/docker_images.md Launches a TTS server in a CPU Docker container, exposing port 5002. Lists available models. ```bash docker run --rm -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/idiap/coqui-tts-cpu tts-server --list_models #To get the list of available models tts-server --model_name tts_models/en/vctk/vits ``` -------------------------------- ### Run TTS and Vocoder Models Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/server.md Starts the TTS server using both a specified TTS model and a vocoder model. Ensure the vocoder is compatible with the TTS model. ```bash tts-server --model_name "///" \ --vocoder_name "///" ``` -------------------------------- ### Serve WAV Files Locally Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/plot_embeddings_umap.ipynb Changes the current directory to the dataset path and starts a local Python HTTP server. This is required to serve audio files when clicking on data points in the plot. ```python %cd $dataset_path %pwd # !python -m http.server ``` -------------------------------- ### Start TTS Server with GPU Docker Image Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/docker_images.md Launches a TTS server in a GPU Docker container, exposing port 5002 and enabling CUDA. Lists available models. ```bash docker run --rm -it -p 5002:5002 --gpus all --entrypoint /bin/bash ghcr.io/idiap/coqui-tts tts-server --list_models #To get the list of available models tts-server --model_name tts_models/en/vctk/vits --use_cuda ``` -------------------------------- ### Initialize Output Directory and Dataset Configuration Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/Tutorial_2_train_your_first_TTS_model.ipynb Sets up the output directory for training artifacts and defines the dataset configuration, specifying the LJSpeech formatter and its path. This prepares the environment for data loading. ```python import os # BaseDatasetConfig: defines name, formatter and path of the dataset. from TTS.tts.configs.shared_configs import BaseDatasetConfig output_path = "tts_train_dir" if not os.path.exists(output_path): os.makedirs(output_path) ``` ```python dataset_config = BaseDatasetConfig( formatter="ljspeech", meta_file_train="metadata.csv", path=os.path.join(output_path, "LJSpeech-1.1/") ) ``` -------------------------------- ### Run Docker Container with GPU Support Source: https://github.com/idiap/coqui-ai-tts/blob/dev/CONTRIBUTING.md Starts a Docker container from the 'tts-dev:latest' image, enabling GPU support and providing an interactive bash shell within the container. ```bash docker run -it --gpus all tts-dev:latest /bin/bash ``` -------------------------------- ### Run Coqui TTS Docker Image Source: https://github.com/idiap/coqui-ai-tts/blob/dev/README.md Run the Coqui TTS Docker image to use the TTS server without local installation. This command starts a container and exposes the TTS server on port 5002. ```bash docker run --rm -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/idiap/coqui-tts-cpu python3 TTS/server/server.py --list_models #To get the list of available models python3 TTS/server/server.py --model_name tts_models/en/vctk/vits # To start a server ``` -------------------------------- ### Run Tensorboard for Monitoring Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/training/training_a_model.md Use this command to launch Tensorboard, which helps visualize training progress and metrics. Replace `` with the actual path where your training logs are stored. ```bash tensorboard --logdir= ``` -------------------------------- ### TTS Command-Line Interface Examples Source: https://context7.com/idiap/coqui-ai-tts/llms.txt The `tts` CLI allows model listing, info lookup, single/multi-speaker synthesis, voice cloning, and voice conversion from the terminal. Supports GPU usage and piping audio. ```sh # List all available models tts --list_models # Get model info tts --model_info_by_name tts_models/tr/common-voice/glow-tts tts --model_info_by_name vocoder_models/en/ljspeech/hifigan_v2 # Print the installed version tts --version # Single-speaker synthesis with the default model (tacotron2-DDC) tts --text "Hello from the command line." --out_path output.wav # Specific TTS model with matching vocoder tts --text "Hello world" \ --model_name "tts_models/en/ljspeech/glow-tts" \ --vocoder_name "vocoder_models/en/ljspeech/univnet" \ --out_path glow_univnet.wav # Multi-speaker model: list speakers then pick one tts --model_name "tts_models/en/vctk/vits" --list_speaker_idxs tts --text "Multi-speaker output." \ --model_name "tts_models/en/vctk/vits" \ --speaker_idx "p225" \ --out_path vctk_output.wav # XTTS with voice cloning from a reference file tts --text "Cloned voice output." \ --model_name "tts_models/multilingual/multi-dataset/xtts_v2" \ --speaker_wav reference.wav \ --language_idx en \ --out_path cloned.wav # Voice conversion tts --model_name "voice_conversion_models/multilingual/vctk/freevc24" \ --source_wav source.wav \ --target_wav target.wav \ --out_path converted.wav # Run on GPU tts --text "GPU synthesis." --device cuda --out_path gpu_output.wav # Pipe audio to audio player tts --text "Piped audio." --pipe_out --out_path /dev/null | aplay # Custom local model tts --text "Custom checkpoint." \ --model_path /path/to/model.pth \ --config_path /path/to/config.json \ --out_path custom.wav ``` -------------------------------- ### Monitor Training with TensorBoard Source: https://github.com/idiap/coqui-ai-tts/blob/dev/recipes/bel-alex73/README.md Launches TensorBoard to visualize training metrics such as alignment and average loss, and to check audio evaluations. Requires event files in the specified log directory. ```bash tensorboard --logdir=/storage/output-/ ``` -------------------------------- ### Initialize Audio Processor, Tokenizer, and Load Dataset Samples Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/dataset_analysis/CheckPitch.ipynb Initializes the AudioProcessor and TTSTokenizer from the configuration. It then loads training and evaluation samples from the dataset. ```python ap = AudioProcessor(CONFIG.audio) tokenizer, CONFIG = TTSTokenizer.init_from_config(CONFIG) train_samples, eval_samples = load_tts_samples(CONFIG, eval_split=False) samples = train_samples + eval_samples dataset = TTSDataset( samples=samples, ap=ap, tokenizer=tokenizer, compute_f0=True, f0_cache_path=f0_cache_path, ) ``` -------------------------------- ### Run XTTS Finetuning Gradio UI Source: https://github.com/idiap/coqui-ai-tts/blob/dev/TTS/demos/xtts_ft_demo/XTTS_finetune_colab.ipynb Launches the Gradio-based user interface for XTTS finetuning. Adjust batch size and number of epochs as needed for your training. ```bash !python -m TTS.demos.xtts_ft_demo.xtts_demo --batch_size 2 --num_epochs 6 ``` -------------------------------- ### Download Finetuning Dataset Source: https://github.com/idiap/coqui-ai-tts/blob/dev/TTS/demos/xtts_ft_demo/XTTS_finetune_colab.ipynb Zips the default dataset located at /tmp/xtts_ft/dataset and initiates a download of the created zip file. ```python from google.colab import files !zip -q -r dataset.zip /tmp/xtts_ft/dataset files.download("dataset.zip") ``` -------------------------------- ### Initialize Trainer Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/Tutorial_2_train_your_first_TTS_model.ipynb Set up the Trainer with necessary arguments, configuration, output path, model, and training/evaluation samples. The Trainer API handles training processes like mixed-precision and distributed training. ```python from trainer import Trainer, TrainerArgs trainer = Trainer( TrainerArgs(), config, output_path, model=model, train_samples=train_samples, eval_samples=eval_samples ) ``` -------------------------------- ### Synthesizer Initialization and Basic Usage Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/main_classes/synthesizer.md Demonstrates how to load a model using ModelManager and perform basic text-to-speech synthesis with the Synthesizer class. ```APIDOC ## Synthesizer Initialization and Basic Usage ### Description This snippet shows how to initialize the `Synthesizer` class by downloading a model and then using it to synthesize speech from text. It also demonstrates saving the synthesized audio to a file. ### Method ```python from TTS.utils.manage import ModelManager from TTS.utils.synthesizer import Synthesizer # Download a model model_path, config_path, _ = ModelManager().download_model("tts_models/en/ljspeech/vits") # Initialize Synthesizer synth = Synthesizer(tts_checkpoint=model_path, tts_config_path=config_path) # Synthesize speech wav = synth.tts("Hello World") # Save the synthesized audio synth.save_wav(wav, "test_audio.wav") ``` ### Parameters - `tts_checkpoint` (str): Path to the TTS model checkpoint. - `tts_config_path` (str): Path to the TTS model configuration file. ### Returns - `wav` (list): A list of audio samples representing the synthesized speech. ``` -------------------------------- ### Voice Metadata Structure Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/cloning.md Example structure of metadata stored within a cached voice file. ```json { 'model': {'name': 'wavlm', 'layer': 6}, 'speaker_id': 'LJ', 'source_files': [ 'tests/data/ljspeech/wavs/LJ001-0001.wav', 'tests/data/ljspeech/wavs/LJ001-0002.wav', ..., ], 'created_at': '2025-06-25T12:17+00:00', 'coqui_version': '0.27.0', } ``` -------------------------------- ### Compute SNR for a Single Example File Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/dataset_analysis/CheckDatasetSNR.ipynb Demonstrates the usage of the `compute_file_snr` function with a specific WAV file. ```python wav_file = "/home/erogol/Data/LJSpeech-1.1/wavs/LJ001-0001.wav" output = compute_file_snr(wav_file) ``` -------------------------------- ### Getting Coqui Speaker Embeddings Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/models/xtts.md Retrieves the conditioning latent and speaker embedding for a named Coqui speaker. ```APIDOC ## model.speaker_manager.speakers ### Description Accesses pre-defined Coqui speaker embeddings for use in inference. ### Method `model.speaker_manager.speakers` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python # Assuming 'model' is an initialized Xtts instance gpt_cond_latent, speaker_embedding = model.speaker_manager.speakers["Ana Florence"].values() ``` ### Response #### Success Response (200) - **gpt_cond_latent** (tensor) - The conditioning latent vector for the speaker. - **speaker_embedding** (tensor) - The speaker embedding vector. #### Response Example None ``` -------------------------------- ### Get Coqui Speaker Embeddings Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/models/xtts.md Retrieve the conditioning latent and speaker embedding for a named Coqui speaker, which can then be used for inference. ```python gpt_cond_latent, speaker_embedding = model.speaker_manager.speakers["Ana Florence"].values() ``` -------------------------------- ### Single-GPU Training Command Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/training/training_a_model.md Command to start training a model on a single GPU. The `CUDA_VISIBLE_DEVICES` environment variable specifies which GPU to use. ```bash CUDA_VISIBLE_DEVICES="0" python train_glowtts.py ``` -------------------------------- ### Show TTS Server Help Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/server.md Use this command to view all available command-line options for the TTS server. ```bash tts-server -h ``` -------------------------------- ### Get Top 10 Speakers Source: https://github.com/idiap/coqui-ai-tts/blob/dev/recipes/bel-alex73/choose_speaker.ipynb Selects the top 10 speakers based on the number of records, using the previously sorted DataFrame. ```python # get top 10 speakers top_10_speakers = df_sorted.head(10) top_10_speakers ``` -------------------------------- ### Access First Sample Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/dataset_analysis/AnalyzeDataset.ipynb Displays the structure and content of the first item in the loaded dataset samples. ```python items[0] ``` -------------------------------- ### CLI: Create Voice with Metadata Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/cloning.md Create a voice using a voice conversion model and specify source and target audio files. The voice will be cached with metadata. ```bash tts --model_name "voice_conversion_models/multilingual/multi-dataset/knnvc" \ --source_wav source.wav \ --target_wav tests/data/ljspeech/wavs/*.wav \ --speaker_idx LJ \ --voice_dir wavlm-voices ``` -------------------------------- ### Pull GPU Docker Image Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/docker_images.md Pulls the pre-built Docker image for Coqui TTS with GPU support. Ensure NVIDIA drivers are installed. ```bash docker pull ghcr.io/idiap/coqui-tts ``` -------------------------------- ### Import necessary libraries Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/dataset_analysis/PhonemeCoverage.ipynb Imports libraries for data manipulation, multiprocessing, plotting, and TTS functionalities. Ensures the multiprocessing start method is set. ```python import collections import operator from multiprocessing import Pool, cpu_count, set_start_method from matplotlib import pylab as plt from tqdm.notebook import tqdm from TTS.tts.datasets import load_tts_samples from TTS.tts.utils.text.tokenizer import TTSTokenizer %matplotlib inline set_start_method("fork", force=True) ``` -------------------------------- ### train_tts - Model training entry point Source: https://context7.com/idiap/coqui-ai-tts/llms.txt Launches TTS model training given a config.json. Supports continuing from a previous experiment. ```APIDOC ## python -m TTS.bin.train_tts ### Description Launches TTS model training given a `config.json`. Uses the `coqui-tts-trainer` backend and logs to TensorBoard. Supports continuing from a previous experiment with `--continue_path`. ### Usage ```sh # Start training from a config file python -m TTS.bin.train_tts --config_path recipes/ljspeech/glow_tts/config.json # Continue training from a checkpoint directory python -m TTS.bin.train_tts --continue_path /output/run-2024-01-01/ # Override config values from command line python -m TTS.bin.train_tts \ --config_path config.json \ --coqpit-overrides batch_size=32 lr=0.0002 ``` ### Programmatic Usage ```python from TTS.bin.train_tts import main as train_main train_main([ "--config_path", "recipes/ljspeech/glow_tts/config.json", ]) ``` ``` -------------------------------- ### Get Mel Spectrogram Shape Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/ExtractTTSpectrogram.ipynb Calculates and returns the shape of the mel spectrogram for a given audio file using the audio processing utility. ```python idx = 1 ap.melspectrogram(ap.load_wav(data["item_idxs"][idx])).shape ``` -------------------------------- ### Load audio files and select a sample Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/dataset_analysis/CheckSpectrograms.ipynb Finds all audio files with the specified extension in the data path and selects one sample file by its index. The `sample_file_index` can be adjusted to choose a different file. ```python file_paths = list(data_path.rglob(f"*{file_ext}")) # Change this to the index of the desired file listed below sample_file_index = 10 SAMPLE_FILE_PATH = file_paths[sample_file_index] print("File list, by index:") dict(enumerate(file_paths)) ``` -------------------------------- ### Load Model and Initialize Audio Processor Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/ExtractTTSpectrogram.ipynb Downloads a pre-trained TTS model, sets up paths, and initializes the AudioProcessor. It's crucial to disable silence trimming for accurate alignment. ```python manager = ModelManager() MODEL_FILE, CONFIG_PATH, _ = manager.download_model("tts_models/en/ljspeech/tacotron2-DDC") # Paths and configurations DATA_PATH = Path("../tests/data/ljspeech") OUT_PATH = Path(os.environ.get("NB_OUTPUT_DIR", DATA_PATH)) / "specs" OUT_PATH.mkdir(parents=True, exist_ok=True) FORMATTER = "ljspeech" METADATA_FILE = "metadata.csv" BATCH_SIZE = 32 QUANTIZE_BITS = 0 # if non-zero, quantize wav files with the given number of bits DRY_RUN = False # if True, does not generate output files, only computes loss and visuals. # Check CUDA availability use_cuda = torch.cuda.is_available() print(" > CUDA enabled: ", use_cuda) # Load the configuration dataset_config = BaseDatasetConfig(formatter=FORMATTER, meta_file_train=METADATA_FILE, path=DATA_PATH) C = load_config(CONFIG_PATH) C.audio["do_trim_silence"] = False # IMPORTANT!!!!!!!!!!!!!!! disable to align mel specs with the wav files ap = AudioProcessor(C.audio) ``` -------------------------------- ### Import Libraries and Configure CUDA Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/ExtractTTSpectrogram.ipynb Imports necessary libraries for TTS model processing and configures CUDA visibility. Ensure all required packages are installed. ```python import os import pickle from pathlib import Path import numpy as np import soundfile as sf import torch from matplotlib import pylab as plt from torch.utils.data import DataLoader from tqdm.notebook import tqdm from TTS.config import load_config from TTS.tts.configs.shared_configs import BaseDatasetConfig from TTS.tts.datasets import load_tts_samples from TTS.tts.datasets.dataset import TTSDataset from TTS.tts.layers.losses import L1LossMasked from TTS.tts.models import setup_model from TTS.tts.utils.helpers import sequence_mask from TTS.tts.utils.text.tokenizer import TTSTokenizer from TTS.tts.utils.visual import plot_spectrogram from TTS.utils.audio import AudioProcessor from TTS.utils.audio.numpy_transforms import quantize from TTS.utils.manage import ModelManager %matplotlib inline # Configure CUDA visibility os.environ["CUDA_VISIBLE_DEVICES"] = "2" ``` -------------------------------- ### Basic Inference with CPU Docker Image Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/docker_images.md Generates an audio file using the CPU Docker image. Mounts a local directory for output. ```bash docker run --rm -v ~/tts-output:/root/tts-output ghcr.io/idiap/coqui-tts-cpu --text "Hello." --out_path /root/tts-output/hello.wav ``` -------------------------------- ### Download Model with ModelManager Source: https://context7.com/idiap/coqui-ai-tts/llms.txt Use ModelManager to download TTS models and get paths to checkpoint and config files. Supports custom output directories. ```python from TTS.utils.manage import ModelManager # Download a model explicitly (returns paths to checkpoint, config, and metadata) model_path, config_path, model_item = manager.download_model( "tts_models/en/ljspeech/glow-tts" ) print(model_path) # Path to .pth checkpoint file print(config_path) # Path to config.json # Custom output directory manager_custom = ModelManager(output_prefix="/opt/tts_models", progress_bar=True) manager_custom.download_model("tts_models/multilingual/multi-dataset/xtts_v2") ``` -------------------------------- ### Get Mary-TTS Locales Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/marytts.md Use this CURL request to retrieve a list of supported locales from the Mary-TTS server. The output is a newline-separated list of locale identifiers. ```bash curl http://localhost:59125/locales ``` -------------------------------- ### Import Libraries for Dataset Analysis Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/dataset_analysis/AnalyzeDataset.ipynb Imports necessary libraries for data manipulation, audio processing, plotting, and multiprocessing. Includes setup for matplotlib and multiprocessing. ```python import os from collections import Counter from multiprocessing import Pool, set_start_method import librosa import numpy as np import pandas as pd from matplotlib import pyplot as plt from scipy.stats import norm from tqdm.notebook import tqdm from TTS.config.shared_configs import BaseDatasetConfig from TTS.tts.configs.shared_configs import BaseTTSConfig from TTS.tts.datasets import load_tts_samples %matplotlib inline set_start_method("fork", force=True) ``` -------------------------------- ### Run Tests with uv Source: https://github.com/idiap/coqui-ai-tts/blob/dev/CONTRIBUTING.md Executes the project's test suite using uv. 'make test' stops at the first error, while 'make test_all' runs all tests and reports all errors. ```bash uv run make test uv run make test_all ``` -------------------------------- ### Train LJSpeech Model with Python Source: https://github.com/idiap/coqui-ai-tts/blob/dev/recipes/ljspeech/README.md Use this command to start training a TTS model with the LJSpeech dataset. Ensure CUDA_VISIBLE_DEVICES is set to the desired GPU ID. ```bash CUDA_VISIBLE_DEVICES="0" python train_modelX.py ``` -------------------------------- ### Load TTS Samples and Create Utterance Mapping Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/plot_embeddings_umap.ipynb Initializes a dataset configuration and loads training samples. It then creates a dictionary mapping unique audio names to their relative file paths within the dataset. ```python dataset_config = BaseDatasetConfig() dataset_config.formatter = formatter_name dataset_config.dataset_name = dataset_name dataset_config.path = dataset_path dataset_config.meta_file_train = meta_file_train config = BaseTTSConfig(datasets=[dataset_config]) meta_data_train, meta_data_eval = load_tts_samples(config, eval_split=False) utt_to_wav = { item["audio_unique_name"]: str(Path(item["audio_file"]).relative_to(dataset_path)) for item in meta_data_train } ``` -------------------------------- ### Recommended Dataset Folder Structure Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/datasets/formatting_your_dataset.md Finalize your dataset by placing the metadata file and the 'wavs' folder within your main dataset directory. ```text /MyTTSDataset | | -> metadata.txt | -> /wavs | -> audio1.wav | -> audio2.wav | ... ``` -------------------------------- ### Initialize Model Training Configuration Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/Tutorial_2_train_your_first_TTS_model.ipynb Imports necessary configuration classes for setting up TTS model training. This step is crucial before defining specific model parameters. ```python # GlowTTSConfig: all model related values for training, validating and testing. from TTS.config.shared_configs import BaseAudioConfig from TTS.tts.configs.glow_tts_config import GlowTTSConfig ``` -------------------------------- ### Run Multilingual TTS Model with Default Language Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/server.md Starts the TTS server with a multilingual model and sets a default language index. The default is English ('en'). ```bash tts-server --model_name tts_models/multilingual/multi-dataset/xtts_v2 --language_idx es ``` -------------------------------- ### Load and play audio file Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/dataset_analysis/CheckSpectrograms.ipynb Loads the selected audio file using the configured AudioProcessor and plays it back. This step verifies that the audio file can be loaded correctly. ```python wav = AP.load_wav(SAMPLE_FILE_PATH) ipd.Audio(data=wav, rate=AP.sample_rate) ``` -------------------------------- ### Run TTS Model with Default Speaker Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/server.md Starts the TTS server with a specified model and sets a default speaker index. This speaker can be overridden in subsequent requests. ```bash tts-server --model_name tts_models/en/vctk/vits --speaker_idx p376 ``` -------------------------------- ### Import necessary libraries Source: https://github.com/idiap/coqui-ai-tts/blob/dev/notebooks/dataset_analysis/CheckSpectrograms.ipynb Imports libraries for data path manipulation, audio display, configuration, and audio processing utilities. ```python %matplotlib inline from pathlib import Path import IPython.display as ipd from TTS.config.shared_configs import BaseAudioConfig from TTS.tts.utils.visual import plot_spectrogram from TTS.utils.audio import AudioProcessor ``` -------------------------------- ### Fine-tune with Custom Parameters Source: https://github.com/idiap/coqui-ai-tts/blob/dev/docs/source/training/finetuning.md Fine-tune a model while overriding configuration parameters via command-line arguments. This example sets a custom run name and learning rate. ```python CUDA_VISIBLE_DEVICES="0" python recipes/ljspeech/glow_tts/train_glowtts.py \ --restore_path /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts/model_file.pth \ --coqpit.run_name "glow-tts-finetune" \ --coqpit.lr 0.00001 ```