### Install CTC Forced Aligner (CPU via ONNXRuntime) Source: https://github.com/deskpai/ctc_forced_aligner/blob/master/src/README.md Installs the CTC Forced Aligner package for CPU inference using ONNXRuntime. This is the base installation with minimal dependencies. ```bash pip install ctc_forced_aligner ``` -------------------------------- ### Install CTC Forced Aligner (PyTorch) Source: https://github.com/deskpai/ctc_forced_aligner/blob/master/README.md Install the ctc_forced_aligner package with PyTorch support for CPU/GPU inference. This installation includes necessary PyTorch dependencies. ```bash pip install ctc_forced_aligner[torch] ``` -------------------------------- ### Install CTC Forced Aligner Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt Installs the CTC Forced Aligner library with different backend options. Use `pip install ctc_forced_aligner` for CPU ONNX, `ctc_forced_aligner[gpu]` for GPU ONNX, `ctc_forced_aligner[torch]` for PyTorch, or `ctc_forced_aligner[all]` for all dependencies. ```bash # CPU inference via ONNXRuntime (minimal dependencies) pip install ctc_forced_aligner # GPU inference via ONNXRuntime pip install ctc_forced_aligner[gpu] # CPU/GPU inference via PyTorch pip install ctc_forced_aligner[torch] # Install all dependencies pip install ctc_forced_aligner[all] ``` -------------------------------- ### Install CTC Forced Aligner (CPU/GPU ONNXRuntime) Source: https://github.com/deskpai/ctc_forced_aligner/blob/master/README.md Install the ctc_forced_aligner package for CPU inference using ONNXRuntime. For GPU inference, install with the 'gpu' extra. ```bash pip install ctc_forced_aligner ``` ```bash pip install ctc_forced_aligner[gpu] ``` -------------------------------- ### Install All CTC Forced Aligner Dependencies Source: https://github.com/deskpai/ctc_forced_aligner/blob/master/README.md Install the ctc_forced_aligner package along with all its optional dependencies, ensuring all features and backends are available. ```bash pip install ctc_forced_aligner[all] ``` -------------------------------- ### Install CTC Forced Aligner (GPU via ONNXRuntime) Source: https://github.com/deskpai/ctc_forced_aligner/blob/master/src/README.md Installs the CTC Forced Aligner package with GPU support for ONNXRuntime inference. This requires compatible GPU drivers and CUDA. ```bash pip install ctc_forced_aligner[gpu] ``` -------------------------------- ### ONNX Alignment Singleton - Generate Subtitles Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt Uses the `AlignmentSingleton` class for ONNX-based audio-text alignment. It automatically downloads and caches models, optimizing for production environments. This example shows generating SRT and WebVTT files from audio and text inputs. ```python from ctc_forced_aligner import AlignmentSingleton import os # Initialize singleton (model downloads automatically on first use) alignment_service = AlignmentSingleton() # Define input/output paths input_audio_path = "audio.mp3" input_text_path = "lyrics.txt" output_srt_path = "output.srt" output_vtt_path = "output.vtt" # Generate SRT subtitle file success = alignment_service.generate_srt( input_audio_path, input_text_path, output_srt_path, language="eng", # ISO 639-3 language code batch_size=4 # Batch size for inference ) if success: print(f"SRT generated at {output_srt_path}") # Generate WebVTT subtitle file success = alignment_service.generate_webvtt( input_audio_path, input_text_path, output_vtt_path, language="eng", batch_size=4 ) if success: print(f"WebVTT generated at {output_vtt_path}") # Check if model is already loaded (useful for conditional initialization) if AlignmentSingleton.is_loaded(): print("Model is ready for inference") # Custom model path (optional) custom_service = AlignmentSingleton(model_path="/path/to/custom/model.onnx") ``` -------------------------------- ### PyTorch Alignment - Generate Subtitles with Multiple Models Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt Utilizes the `AlignmentTorch` class for PyTorch/TorchAudio-based alignment, supporting 18 ASR models including Wav2Vec2, VoxPopuli, and HuBERT. This example demonstrates generating SRT and WebVTT files using different model types for varying accuracy and language needs. ```python from ctc_forced_aligner import AlignmentTorch # Initialize PyTorch alignment service alignment = AlignmentTorch() input_audio = "speech.wav" input_text = "transcript.txt" # Generate SRT with default MMS_FA model success = alignment.generate_srt( input_audio, input_text, "output_mms.srt", model_type='MMS_FA' ) # Generate with Wav2Vec2 large model (best English accuracy) success = alignment.generate_srt( input_audio, input_text, "output_wav2vec.srt", model_type='WAV2VEC2_ASR_LARGE_960H' ) # Generate with HuBERT for high accuracy success = alignment.generate_srt( input_audio, input_text, "output_hubert.srt", model_type='HUBERT_ASR_XLARGE' ) # Generate German subtitles with VoxPopuli model success = alignment.generate_webvtt( "german_audio.wav", "german_transcript.txt", "german_output.vtt", model_type='VOXPOPULI_ASR_BASE_10K_DE' ) # Available models: # Wav2Vec2: WAV2VEC2_ASR_BASE_960H, WAV2VEC2_ASR_BASE_100H, WAV2VEC2_ASR_BASE_10M, # WAV2VEC2_ASR_LARGE_10M, WAV2VEC2_ASR_LARGE_100H, WAV2VEC2_ASR_LARGE_960H, # WAV2VEC2_ASR_LARGE_LV60K_10M, WAV2VEC2_ASR_LARGE_LV60K_100H, WAV2VEC2_ASR_LARGE_LV60K_960H # VoxPopuli: VOXPOPULI_ASR_BASE_10K_DE, VOXPOPULI_ASR_BASE_10K_EN, VOXPOPULI_ASR_BASE_10K_ES, # VOXPOPULI_ASR_BASE_10K_FR, VOXPOPULI_ASR_BASE_10K_IT # HuBERT: HUBERT_ASR_LARGE, HUBERT_ASR_XLARGE ``` -------------------------------- ### Generate SRT/WebVTT with PyTorch Source: https://github.com/deskpai/ctc_forced_aligner/blob/master/README.md Perform speech alignment and generate SRT/WebVTT files using the AlignmentTorch class. This is suitable for environments with PyTorch installed. ```python from ctc_forced_aligner import AlignmentTorch at = AlignmentTorch() input_audio_path = "audio.mp3" input_text_path = "input.txt" output_srt_path = "output.srt" ret = at.generate_srt(input_audio_path, input_text_path, output_srt_path) if ret: print(f"aligned srt is generated to {output_srt_path}") output_vtt_path = "output.vtt" ret = at.generate_webvtt(input_audio_path, input_text_path, output_vtt_path) if ret: print(f"aligned VTT is generated to {output_vtt_path}") ``` -------------------------------- ### PyTorch Alignment Singleton - Reuse Model Instance Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt Provides a singleton pattern for PyTorch-based alignment using `AlignmentTorchSingleton`. This ensures that the PyTorch model is loaded only once and reused across multiple alignment operations, which is beneficial for long-running applications and efficient resource management. ```python from ctc_forced_aligner import AlignmentTorchSingleton # Get singleton instance (creates on first call) alignment = AlignmentTorchSingleton() ``` -------------------------------- ### Non-Singleton ONNX Alignment with Alignment Class Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt This Python snippet demonstrates using the `Alignment` class for non-singleton ONNX-based alignment. It allows for multiple independent model instances by initializing the class with a custom model path. It also shows how to access the model and tokenizer directly and generate subtitles in SRT and WebVTT formats. ```python from ctc_forced_aligner import Alignment import os # Initialize with custom model path model_path = os.path.expanduser("~/ctc_forced_aligner/model.onnx") alignment = Alignment(model_path) # Access model and tokenizer directly model = alignment.alignment_model tokenizer = alignment.alignment_tokenizer # Generate subtitles success = alignment.generate_srt( "audio.mp3", "transcript.txt", "output.srt", language="eng", batch_size=4 ) success = alignment.generate_webvtt( "audio.mp3", "transcript.txt", "output.vtt", language="eng", batch_size=4 ) ``` -------------------------------- ### Generate SRT/WebVTT with ONNXRuntime Source: https://github.com/deskpai/ctc_forced_aligner/blob/master/README.md Perform speech alignment and generate SRT/WebVTT files using the AlignmentSingleton class with ONNXRuntime. Requires input audio and text file paths. ```python from ctc_forced_aligner import AlignmentSingleton alignment_service = AlignmentSingleton() input_audio_path = "audio.mp3" input_text_path = "input.txt" output_srt_path = "output.srt" ret = alignment_service.generate_srt(input_audio_path, input_text_path, output_srt_path) if ret: print(f"Aligned SRT is generated at {output_srt_path}") output_vtt_path = "output.vtt" ret = alignment_service.generate_webvtt(input_audio_path, input_text_path, output_vtt_path) if ret: print(f"aligned VTT is generated to {output_vtt_path}") ``` -------------------------------- ### Process Multiple Files with CTC Forced Aligner Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt This Python snippet demonstrates how to efficiently process multiple audio and text files using a single model instance. It iterates through pairs of audio and text files, generates SRT subtitles, and prints success messages. It also includes a check for the loaded PyTorch model status. ```python from ctc_forced_aligner import Alignment, AlignmentTorchSingleton audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"] text_files = ["text1.txt", "text2.txt", "text3.txt"] # Assuming 'alignment' is an instance of Alignment or a similar class alignment = Alignment() for audio, text in zip(audio_files, text_files): output_srt = audio.replace(".wav", ".srt") success = alignment.generate_srt(audio, text, output_srt, model_type='WAV2VEC2_ASR_LARGE_960H') if success: print(f"Generated: {output_srt}") # Check if singleton is initialized if AlignmentTorchSingleton.is_loaded(): print("PyTorch model is loaded and ready") ``` -------------------------------- ### Generate SRT/WebVTT with Specific Model (PyTorch) Source: https://github.com/deskpai/ctc_forced_aligner/blob/master/README.md Generate SRT/WebVTT files using the AlignmentTorch class, specifying a particular model type like 'WAV2VEC2_ASR_BASE_960H'. ```python from ctc_forced_aligner import AlignmentTorch at = AlignmentTorch() input_audio_path = "audio.mp3" input_text_path = "input.txt" output_srt_path = "output.srt" ret = at.generate_srt(input_audio_path, input_text_path, output_srt_path, model_type='WAV2VEC2_ASR_BASE_960H') if ret: print(f"aligned srt is generated to {output_srt_path}") output_vtt_path = "output.vtt" ret = at.generate_webvtt(input_audio_path, input_text_path, output_vtt_path, model_type='WAV2VEC2_ASR_BASE_960H') if ret: print(f"aligned VTT is generated to {output_vtt_path}") ``` -------------------------------- ### Perform CTC Forced Alignment with NumPy Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt This snippet demonstrates how to perform CTC forced alignment using NumPy arrays for log probabilities and target sequences. It involves preparing log probabilities, defining target tokens, and calling the `align_sequences` function. The output includes the alignment path and scores. ```python import numpy as np # Placeholder for the actual alignment function def align_sequences(log_probs, targets, blank_id): # This is a mock implementation. Replace with actual CTC alignment logic. batch_size, time_steps, _ = log_probs.shape aligned_tokens = np.random.randint(1, 32, size=(batch_size, time_steps)) scores = np.random.randn(time_steps).astype(np.float32) return aligned_tokens, scores # These would typically come from an ASR model's output batch_size = 1 time_steps = 1000 num_classes = 32 # Vocabulary size log_probs = np.random.randn(batch_size, time_steps, num_classes).astype(np.float32) # Normalize to get probabilities, then convert to log probabilities probabilities = np.exp(log_probs) / np.sum(np.exp(log_probs), axis=-1, keepdims=True) log_probs = np.log(probabilities) # Target sequence (token indices, excluding blank) targets = np.array([[4, 5, 6, 7, 8]], dtype=np.int64) # Example: "a i e n o" # Perform alignment blank_id = 0 paths, scores = align_sequences(log_probs, targets, blank_id) # paths: (batch_size, time_steps) - aligned token indices # scores: (time_steps,) - alignment scores per frame print(f"Alignment path shape: {paths.shape}") print(f"Scores shape: {scores.shape}") print(f"First 20 aligned tokens: {paths[0, :20]}") ``` -------------------------------- ### Low-Level CTC Alignment with align_sequences Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt This Python snippet provides direct access to the C++ CTC forced alignment algorithm via the `align_sequences` function. It is intended for custom implementations requiring maximum performance and demonstrates the necessary preparation of log probabilities as input. ```python from ctc_forced_aligner.ctc_aligner import align_sequences import numpy as np # Prepare log probabilities (batch_size, time_steps, num_classes) # Example placeholder for log probabilities log_probs = np.random.rand(1, 100, 50).astype(np.float32) # Example placeholder for target sequence (character IDs) target_ids = np.array([1, 2, 3, 4, 5], dtype=np.int32) # Perform alignment (example usage - actual parameters may vary) # alignments = align_sequences(log_probs, target_ids, num_classes=50) # print("Alignment results:", alignments) ``` -------------------------------- ### PyTorch Word-Level Timestamps with get_word_stamps Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt This Python snippet utilizes the `get_word_stamps` function for direct word-level timestamp extraction using PyTorch models. It shows how to obtain detailed timing information, reuse loaded models for efficiency, and specify different model types like 'MMS_FA' and 'HUBERT_ASR_LARGE'. ```python from ctc_forced_aligner import get_word_stamps # Get word timestamps with default MMS_FA model word_timestamps, model, lyrics_lines = get_word_stamps( "audio.wav", "transcript.txt", model=None, # Will load model automatically model_type='MMS_FA' ) # Process timestamps for word in word_timestamps: print(f"Word: {word['text']}") print(f" Start: {word['start']:.3f}s") print(f" End: {word['end']:.3f}s") print(f" Score: {word['score']:.4f}") # Reuse model for subsequent calls (more efficient) word_timestamps_2, model, _ = get_word_stamps( "audio2.wav", "transcript2.txt", model=model, # Reuse loaded model model_type='MMS_FA' ) # Use different model types word_timestamps_hubert, _, _ = get_word_stamps( "audio.wav", "transcript.txt", model_type='HUBERT_ASR_LARGE' ) ``` -------------------------------- ### Low-Level CTC Alignment Functions Source: https://context7.com/deskpai/ctc_forced_aligner/llms.txt This Python snippet showcases the low-level functions available for custom CTC alignment pipelines. It covers audio loading, text normalization and preprocessing, emission generation using ONNX models, performing alignment, extracting word spans, and postprocessing results into timestamped words. ```python from ctc_forced_aligner import ( load_audio, preprocess_text, generate_emissions, get_alignments, get_spans, postprocess_results, text_normalize, AlignmentSingleton ) import numpy as np # Load and preprocess audio (returns numpy array at 16kHz) audio_waveform = load_audio("audio.mp3", ret_type='np') # For ONNX # audio_waveform = load_audio("audio.mp3", ret_type='torch') # For PyTorch # Normalize and preprocess text raw_text = "Hello world, this is a test." language = "eng" # ISO 639-3 code # Normalize text (removes punctuation, handles unicode) normalized = text_normalize(raw_text, language, lower_case=True, remove_numbers=True) print(f"Normalized: {normalized}") # Preprocess for alignment (tokenize and add star tokens) tokens_starred, text_starred = preprocess_text( raw_text, romanize=True, # Convert to romanized form language=language, split_size="word", # Options: "word", "sentence", "char" star_frequency="segment" # Options: "segment", "edges" ) # Get ONNX model session service = AlignmentSingleton() model = service.alignment_model tokenizer = service.alignment_tokenizer # Generate emissions from audio emissions, stride = generate_emissions( model, audio_waveform, window_length=30, # Window length in seconds context_length=2, # Context overlap in seconds batch_size=4 ) # Perform alignment segments, scores, blank_token = get_alignments(emissions, tokens_starred, tokenizer) # Get word spans spans = get_spans(tokens_starred, segments, blank_token) # Extract timestamps word_timestamps = postprocess_results( text_starred, spans, stride, scores, merge_threshold=0.0 ) # Output: list of dicts with start, end, text, score for word in word_timestamps: print(f"{word['text']}: {word['start']:.3f}s - {word['end']:.3f}s (score: {word['score']:.4f})") ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.