### Training Arguments Configuration Example Source: https://context7.com/resemble-ai/dramabox/llms.txt An example YAML configuration file for training arguments. This file specifies data directories, speaker index, output directory, checkpoints, model base, LoRA parameters, training steps, learning rate, and scheduling. ```yaml data_dir: - /path/to/preprocessed_dataset_a/ speaker_index: - /path/to/preprocessed_dataset_a/index.txt output_dir: tts_iclora_v1 checkpoint: dramabox-dit-v1.safetensors full_checkpoint: dramabox-audio-components.safetensors base_model: dev # 'dev' uses ShiftedLogitNormal; 'distilled' uses DistilledTimestepSampler lora_rank: 128 lora_alpha: 128 lora_dropout: 0.1 steps: 10000 lr: 1.0e-04 lr_scheduler: cosine warmup_steps: 500 batch_size: 1 grad_accum: 4 save_every: 500 val_config: configs/val_config.example.yaml # optional per-checkpoint validation ``` -------------------------------- ### Validation Configuration Example Source: https://context7.com/resemble-ai/dramabox/llms.txt An example YAML configuration for validation. It lists speakers with their prompts and optional reference audio files. This configuration is used by `src/validate.py` and can be automatically spawned by `src/train.py`. ```yaml speakers: - name: villain_growl prompt: 'A shadowy villain speaks with cold menace, "You have entered my domain, mortal." He chuckles darkly, "Such arrogance will be your undoing."' reference: /path/to/voice_refs/male_villain.wav - name: tender_whisper prompt: 'A woman speaks tenderly, "It has been a long day, my love." She whispers, "Close your eyes. I am right here."' reference: /path/to/voice_refs/female_warm.wav - name: catgirl_giggle prompt: 'A playful girl already mid-giggle, "Hehehe, oh my gosh you should see your face!" She gasps, "Oh my, hehe, I cannot stop!"' # No reference: pure prompt-driven generation ``` -------------------------------- ### CLI Inference - Advanced Options Source: https://context7.com/resemble-ai/dramabox/llms.txt This example shows how to use advanced options with the CLI inference script, including guidance tuning, explicit duration, and custom LoRA weights. ```APIDOC ## CLI Inference - Advanced Options ### Description Demonstrates advanced usage of the CLI inference script with parameters for guidance, duration, and custom LoRA models. ### Command ```bash python src/inference.py \ --voice-sample \ --prompt '' \ --output \ --cfg-scale \ --stg-scale \ --gen-duration \ --seed \ --lora \ --lora-rank ``` ### Arguments * `--voice-sample` (str): Path to the voice sample file for cloning. * `--prompt` (str): The text prompt for speech generation. * `--output` (str): The path to save the generated audio file. * `--cfg-scale` (float): Classifier-free guidance scale. * `--stg-scale` (float): Style/tone guidance scale. * `--gen-duration` (float): Explicitly set the generation duration in seconds. * `--seed` (int): Random seed for reproducible generation. * `--lora` (str): Path to a trained LoRA model file. * `--lora-rank` (int): Rank of the LoRA model. ``` -------------------------------- ### Launch Gradio App Source: https://github.com/resemble-ai/dramabox/blob/master/README.md Start the Gradio web interface for DramaBox by setting the CUDA_VISIBLE_DEVICES environment variable and running the app script. ```bash CUDA_VISIBLE_DEVICES=4 python app.py ``` -------------------------------- ### CLI Inference - Basic Generation Source: https://context7.com/resemble-ai/dramabox/llms.txt The `src/inference.py` script allows for one-shot TTS generation from the command line. This example demonstrates basic generation with voice cloning. ```APIDOC ## CLI Inference - Basic Generation ### Description Performs TTS generation using the command-line interface, including voice cloning. ### Command ```bash python src/inference.py \ --voice-sample \ --prompt '' \ --output ``` ### Arguments * `--voice-sample` (str): Path to the voice sample file for cloning. * `--prompt` (str): The text prompt for speech generation. * `--output` (str): The path to save the generated audio file. ``` -------------------------------- ### CLI Inference: `src/inference.py` Source: https://context7.com/resemble-ai/dramabox/llms.txt Use the CLI script for single-shot TTS generation. This involves a cold start for model loading (~30s). Supports various checkpoints, LoRA weights, and guidance parameters. ```bash # Basic generation with voice cloning python src/inference.py \ --voice-sample assets/voices/female_american.wav \ --prompt 'A playful girl already mid-giggle, "Hehehe, oh my gosh you should see your face!" She gasps, "Oh my, I cannot stop!"' \ --output output/catgirl.wav # With explicit guidance tuning and long duration python src/inference.py \ --voice-sample assets/voices/male_conan.mp3 \ --prompt 'A talk show host gasps, "No! You did NOT just say that!" He bursts into uncontrollable laughter, "Hahahaha!"' \ --output output/conan_laugh.wav \ --cfg-scale 2.5 \ --stg-scale 1.5 \ --gen-duration 30.0 \ --seed 42 # With a trained LoRA for custom speaker/style python src/inference.py \ --voice-sample reference.wav \ --prompt 'A woman speaks warmly, "Welcome back to the show."' \ --output output/custom_speaker.wav \ --lora tts_iclora_v1/lora_step_10000.safetensors \ --lora-rank 128 ``` -------------------------------- ### Detect and Apply Perth Watermark in Audio Source: https://context7.com/resemble-ai/dramabox/llms.txt Use the perth library to detect watermarks in audio files or apply watermarks to existing waveforms. Ensure librosa and torchaudio are installed for audio loading and saving. ```python import perth, librosa # Detect watermark in generated audio wav, sr = librosa.load("output.wav", sr=None, mono=True) detector = perth.PerthImplicitWatermarker() confidence = detector.get_watermark(wav, sample_rate=sr) print(confidence) # confidence ≈ 1.0 for watermarked audio, < 0.5 for non-watermarked # Apply watermark manually to an existing waveform import torch, numpy as np, torchaudio wav_tensor, sr = torchaudio.load("generated.wav") wm = perth.PerthImplicitWatermarker() mono = wav_tensor.mean(dim=0).numpy() mono_wm = wm.apply_watermark(mono, sample_rate=sr) mono_wm_t = torch.from_numpy(np.asarray(mono_wm, dtype=np.float32)).unsqueeze(0) torchaudio.save("watermarked.wav", mono_wm_t, sr) ``` -------------------------------- ### LoRA Fine-tuning with Accelerate Source: https://context7.com/resemble-ai/dramabox/llms.txt Launches LoRA fine-tuning using HuggingFace Accelerate. Configuration is typically done via a YAML file, but specific arguments can be overridden on the command line. Supports multi-GPU training and resuming from checkpoints. ```bash accelerate launch src/train.py \ --config configs/training_args.example.yaml ``` ```bash accelerate launch src/train.py \ --config configs/training_args.example.yaml \ --lr 5e-5 \ --steps 5000 \ --lora-dropout 0.1 ``` ```bash CUDA_VISIBLE_DEVICES=0,1,2,3 accelerate launch --num_processes=4 src/train.py \ --config configs/training_args.example.yaml ``` ```bash accelerate launch src/train.py \ --config configs/training_args.example.yaml \ --resume-lora tts_iclora_v1/lora_step_05000.safetensors ``` -------------------------------- ### Manifest Data Format (JSONL) Source: https://github.com/resemble-ai/dramabox/blob/master/README.md Recommended format for new datasets. Includes audio filepath, text, and optional duration. ```jsonl {"audio_filepath": "wavs/spk01_001.wav", "text": "A woman speaks warmly, \"Hello, how are you today?\""} {"audio_filepath": "wavs/spk01_002.wav", "text": "Hello, how are you today?"} {"audio_filepath": "wavs/spk02_001.flac", "text": "An exhausted father sighs, \"Sweetie, daddy is asking very nicely.\"", "duration": 4.7} ``` -------------------------------- ### Training Script Source: https://github.com/resemble-ai/dramabox/blob/master/README.md Command to launch the training process using accelerate. Configuration is typically done via a YAML file, with CLI flags overriding YAML settings. ```bash accelerate launch src/train.py \ --config configs/training_args.example.yaml ``` -------------------------------- ### Launch Gradio Web App for Dramabox Source: https://context7.com/resemble-ai/dramabox/llms.txt Run the Gradio web application for Dramabox. Customize the port, specify GPU usage, or configure for HuggingFace Spaces deployment using environment variables. ```bash # Local launch (port 7860) python app.py # Custom port GRADIO_SERVER_PORT=8080 python app.py # With GPU selection CUDA_VISIBLE_DEVICES=4 python app.py # HuggingFace Spaces (ZeroGPU — torch.compile disabled, bf16 dtype) LTX_DTYPE=bf16 python app.py ``` -------------------------------- ### Preprocess Data with JSONL Manifest Source: https://context7.com/resemble-ai/dramabox/llms.txt Use this command to preprocess data from a JSONL manifest file. Specify input and output directories, checkpoint, and model root. Duration constraints can also be set. ```bash python src/preprocess.py \ --dataset-type manifest \ --index your_data.jsonl \ --audio-dir /path/to/wavs \ --output-dir /path/to/preprocessed/ \ --checkpoint dramabox-audio-components.safetensors \ --gemma-root ~/.cache/dramabox/gemma-3-12b-it-bnb-4bit/ \ --max-duration 20.0 \ --min-duration 2.0 ``` -------------------------------- ### Run Inference with Heun Sampler Source: https://context7.com/resemble-ai/dramabox/llms.txt Use the Heun sampler for cleaner audio generation with approximately 2x model calls. Requires a voice sample, prompt, and output path. ```bash python src/inference.py \ --voice-sample reference.wav \ --prompt 'An exhausted father sighs, "Sweetie, daddy is asking very nicely." He laughs helplessly, "Hahaha."' \ --output output/heun.wav \ --sampler heun \ --steps 20 ``` -------------------------------- ### Preprocessing Script Source: https://github.com/resemble-ai/dramabox/blob/master/README.md Command to run the preprocessing script. Requires specifying dataset type, index file, audio directory, output directory, checkpoint, Gemma root, and duration constraints. ```bash python src/preprocess.py \ --dataset-type manifest \ --index your_data.jsonl \ --audio-dir /path/to/wavs \ --output-dir /path/to/preprocessed/ \ --checkpoint /path/to/dramabox-audio-components.safetensors \ --gemma-root /path/to/gemma-3-12b-it-bnb-4bit/ \ --max-duration 20.0 --min-duration 2.0 ``` -------------------------------- ### TTSServer - Generate to File Source: https://context7.com/resemble-ai/dramabox/llms.txt The TTSServer class provides a warm inference server for low-latency TTS generation. The `generate_to_file` method saves the generated audio directly to a WAV file. It supports basic generation, voice cloning with a reference audio clip, and explicit duration control. ```APIDOC ## TTSServer.generate_to_file ### Description Generates audio from a text prompt and saves it to a specified file. Supports voice cloning and various generation parameters. ### Method Signature ```python server.generate_to_file(prompt: str, output: str, voice_ref: Optional[str] = None, cfg_scale: float = 2.0, stg_scale: float = 1.0, duration_multiplier: float = 1.0, seed: Optional[int] = None, watermark: bool = True, gen_duration: Optional[float] = None, ref_duration: Optional[float] = None) ``` ### Parameters * **prompt** (str) - The text prompt to generate speech from, including style and emotion cues. * **output** (str) - The path to save the generated WAV audio file. * **voice_ref** (Optional[str]) - Path to an optional 10+ second audio clip for voice cloning. * **cfg_scale** (float) - Classifier-free guidance scale. Defaults to 2.0. * **stg_scale** (float) - Style/tone guidance scale. Defaults to 1.0. * **duration_multiplier** (float) - Multiplier to adjust the auto-estimated duration. Defaults to 1.0. * **seed** (Optional[int]) - Random seed for reproducible generation. * **watermark** (bool) - Whether to apply Perth imperceptible watermarking. Defaults to True. * **gen_duration** (Optional[float]) - Explicitly set the generation duration in seconds. Required for clips > ~20s. * **ref_duration** (Optional[float]) - Duration in seconds of the reference clip to condition on. Used with `voice_ref`. ``` -------------------------------- ### Generate Audio with TTSServer (Python) Source: https://github.com/resemble-ai/dramabox/blob/master/README.md Use the TTSServer class for generating speech to a file. Provide a prompt and an optional voice reference for cloning. Ensure the device is set to 'cuda' for GPU acceleration. ```python from src.inference_server import TTSServer server = TTSServer(device="cuda") server.generate_to_file( prompt='A woman speaks warmly, "Hello, how are you today?" She laughs, "Hahaha, it is so good to see you!"', output="output.wav", voice_ref="reference.wav", # optional, 10+ seconds ) ``` -------------------------------- ### Parallel Sharded Preprocessing Source: https://context7.com/resemble-ai/dramabox/llms.txt This script demonstrates parallel preprocessing across multiple GPUs using a loop and `CUDA_VISIBLE_DEVICES`. It preprocesses data from a JSONL manifest, specifying shard information and enabling skipping existing files. ```bash for SHARD in 0 1 2 3; do CUDA_VISIBLE_DEVICES=$SHARD python src/preprocess.py \ --dataset-type manifest \ --index large_dataset.jsonl \ --audio-dir /data/wavs \ --output-dir /data/preprocessed/ \ --checkpoint dramabox-audio-components.safetensors \ --gemma-root ~/.cache/dramabox/gemma-3-12b-it-bnb-4bit/ \ --shard $SHARD --num-shards 4 \ --skip-existing & done wait ``` -------------------------------- ### TTSServer: Warm Inference Server Source: https://context7.com/resemble-ai/dramabox/llms.txt Use TTSServer for low-latency programmatic TTS generation. Load models once for cached GPU inference. Supports voice cloning, guidance scales, and explicit duration control. ```python from src.inference_server import TTSServer # Load once; all models cached on GPU (~24 GB VRAM peak on H100) server = TTSServer( device="cuda", dtype="bf16", # or "fp16" compile_model=True, # torch.compile for faster denoising (disable under ZeroGPU) bnb_4bit=True, # required for default unsloth/gemma-3-12b-it-bnb-4bit weights ) # Basic generation — auto-estimates duration from prompt server.generate_to_file( prompt='A woman speaks warmly, "Hello, how are you today?" She laughs, "Hahaha, it is so good to see you!"', output="output.wav", ) # With voice cloning — supply a 10+ second reference clip server.generate_to_file( prompt='A shadowy villain speaks with cold menace, "You have entered my domain, mortal." He chuckles darkly, "Hehe."', output="villain.wav", voice_ref="assets/voices/male_harvey_keitel.mp3", cfg_scale=2.5, stg_scale=1.5, duration_multiplier=1.1, # add 10% breathing room to auto-estimate seed=42, watermark=True, # Perth imperceptible watermark (default on) ) # Long-form generation (30 seconds, explicit duration) server.generate_to_file( prompt='A radio host settles into a warm tone, "Good evening, dear listeners. Tonight we are going to talk about love."', output="radio.wav", voice_ref="assets/voices/male_old_movie.wav", gen_duration=30.0, # override auto-estimate; required for clips > ~20 s ref_duration=10.0, # seconds of the reference clip to condition on ) # Return waveform tensor directly (for downstream processing) waveform, sample_rate = server.generate( prompt='A confident announcer, "And now, the moment you have all been waiting for."', cfg_scale=2.5, stg_scale=1.5, seed=0, ) # waveform: torch.Tensor [channels, samples], sample_rate: int import torchaudio torchaudio.save("announcer.wav", waveform.cpu().float(), sample_rate) ``` -------------------------------- ### Download DramaBox and Gemma Models Source: https://context7.com/resemble-ai/dramabox/llms.txt Download all required DramaBox and Gemma text encoder models. Models are cached locally. Supports HF_TOKEN for gated repositories and a custom cache directory. ```python from src.model_downloader import get_model_path, get_gemma_path, get_all_paths import os # Download all required models at once (returns dict of local paths) paths = get_all_paths(cache_dir=os.path.expanduser("~/.cache/dramabox")) # paths = { # 'transformer': '/home/user/.cache/dramabox/.../dramabox-dit-v1.safetensors', # 6.6 GB # 'audio_components': '/home/user/.cache/dramabox/.../dramabox-audio-components.safetensors', # 1.9 GB # 'silence_latent': '/home/user/.cache/dramabox/.../silence_latent_frame.pt', # 'gemma_root': '/home/user/.cache/dramabox/.../gemma-3-12b-it-bnb-4bit/', # ~8 GB # } # Download a single model file transformer_path = get_model_path("transformer") audio_path = get_model_path("audio_components") # Download Gemma text encoder snapshot (full directory) gemma_dir = get_gemma_path() # Use with TTSServer from src.inference_server import TTSServer server = TTSServer( checkpoint=paths["transformer"], full_checkpoint=paths["audio_components"], gemma_root=paths["gemma_root"], device="cuda", ) ``` -------------------------------- ### Detect Watermark in Audio (Python) Source: https://github.com/resemble-ai/dramabox/blob/master/README.md Verify the presence of the Resemble Perth neural watermark in generated audio files. Load the audio using librosa and use the PerthImplicitWatermarker to detect the watermark. ```python import perth, librosa wav, sr = librosa.load("output.wav", sr=None, mono=True) detector = perth.PerthImplicitWatermarker() print(detector.get_watermark(wav, sample_rate=sr)) # confidence ≈ 1.0 ``` -------------------------------- ### Generate Audio via CLI Source: https://github.com/resemble-ai/dramabox/blob/master/README.md Run inference directly from the command line using the inference script. Specify voice samples, prompts, output files, and various configuration scales. ```bash python src/inference.py \ --voice-sample reference.wav \ --prompt 'A woman speaks warmly, "Hello, how are you today?"' \ --output output.wav \ --cfg-scale 2.5 --stg-scale 1.5 ``` -------------------------------- ### Apply Voice Conditioning with AudioConditionByReferenceLatent Source: https://context7.com/resemble-ai/dramabox/llms.txt Implement IC-LoRA conditioning by patchifying a VAE-encoded reference audio latent. This appends reference tokens to the latent state for voice cloning. `strength=1.0` freezes reference tokens. ```python import torch from ltx_core.components.patchifiers import AudioPatchifier from ltx_core.tools import AudioLatentTools from ltx_core.types import AudioLatentShape, VideoPixelShape from ltx_pipelines.utils.blocks import AudioConditioner from ltx_core.model.audio_vae import encode_audio as vae_encode_audio from ltx_pipelines.utils.media_io import decode_audio_from_file from src.audio_conditioning import AudioConditionByReferenceLatent device = torch.device("cuda") dtype = torch.bfloat16 # Encode a 10-second reference clip through the audio VAE audio = decode_audio_from_file("reference.wav", device, 0.0, 10.0) w = audio.waveform if w.dim() == 2: # [C, T] → [1, C, T] w = w.unsqueeze(0) if w.shape[1] == 1: # mono → stereo w = w.repeat(1, 2, 1) from ltx_core.types import Audio voice = Audio(waveform=w, sampling_rate=audio.sampling_rate) ac = AudioConditioner(checkpoint_path="dramabox-audio-components.safetensors", dtype=dtype, device=device) ref_latent = ac(lambda enc: vae_encode_audio(voice, enc, None)) # ref_latent: [1, 8, T_ref, 16] # Create a target latent state for a 5-second generation fps = 25.0 n_frames = int(round(5.0 * fps)) + 1 n_frames = ((n_frames - 1 + 4) // 8) * 8 + 1 pixel_shape = VideoPixelShape(batch=1, frames=n_frames, height=64, width=64, fps=fps) tgt_shape = AudioLatentShape.from_video_pixel_shape(pixel_shape) patchifier = AudioPatchifier(patch_size=1) audio_tools = AudioLatentTools(patchifier=patchifier, target_shape=tgt_shape) state = audio_tools.create_initial_state(device=device, dtype=dtype) # Apply voice conditioning — appends ref tokens to state cond = AudioConditionByReferenceLatent(latent=ref_latent.to(device, dtype), strength=1.0) state = cond.apply_to(latent_state=state, latent_tools=audio_tools) # state.latent: [1, tgt_T + ref_T, 128] (target + reference concatenated) # state.denoise_mask:[1, tgt_T + ref_T, 1] (1.0 for target, 0.0 for frozen ref) # state.attention_mask: asymmetric [1, total, total] ``` -------------------------------- ### TTSServer - Generate Source: https://context7.com/resemble-ai/dramabox/llms.txt The `generate` method of the TTSServer class returns the generated audio as a waveform tensor and its sample rate, suitable for further processing or direct use. ```APIDOC ## TTSServer.generate ### Description Generates audio from a text prompt and returns it as a waveform tensor and sample rate. ### Method Signature ```python server.generate(prompt: str, cfg_scale: float = 2.0, stg_scale: float = 1.0, seed: Optional[int] = None) ``` ### Parameters * **prompt** (str) - The text prompt to generate speech from, including style and emotion cues. * **cfg_scale** (float) - Classifier-free guidance scale. Defaults to 2.0. * **stg_scale** (float) - Style/tone guidance scale. Defaults to 1.0. * **seed** (Optional[int]) - Random seed for reproducible generation. ### Returns * **waveform** (torch.Tensor) - The generated audio waveform tensor [channels, samples]. * **sample_rate** (int) - The sample rate of the generated audio. ``` -------------------------------- ### Standalone Validation Runner Source: https://context7.com/resemble-ai/dramabox/llms.txt Runs validation independently using a specified configuration file. This command generates audio samples at a given checkpoint using provided LoRA weights and various scaling factors. ```bash python src/validate.py \ --val-config configs/val_config.example.yaml \ --output-dir output/validation/step_05000 \ --lora tts_iclora_v1/lora_step_05000.safetensors \ --lora-rank 128 \ --cfg-scale 2.5 \ --stg-scale 1.5 \ --duration-multiplier 1.1 ``` -------------------------------- ### Inference Script Source: https://github.com/resemble-ai/dramabox/blob/master/README.md Command to perform inference using a trained LoRA. Requires paths to the LoRA, a voice sample, a prompt, and an output file. ```bash python src/inference.py \ --lora /path/to/your/lora_step_5000.safetensors \ --voice-sample reference.wav \ --prompt 'A woman speaks warmly, "..."' \ --output output.wav ``` -------------------------------- ### Preprocess Data with TSV Format Source: https://context7.com/resemble-ai/dramabox/llms.txt This command preprocesses data from a TSV file, where each line contains audio path and transcript. Ensure the output directory, checkpoint, and model root are specified. ```bash python src/preprocess.py \ --dataset-type tsv \ --index data.tsv \ --output-dir /path/to/preprocessed/ \ --checkpoint dramabox-audio-components.safetensors \ --gemma-root ~/.cache/dramabox/gemma-3-12b-it-bnb-4bit/ ``` -------------------------------- ### LibriHeavy Data Format Source: https://github.com/resemble-ai/dramabox/blob/master/README.md A '~'-separated format for unprompted text-only data. ```text id~speaker~lang~samples~dur_ms~phonemes~text spk01_001~spk01~en~93000~3875~_~Hello, how are you today? ``` -------------------------------- ### Run Inference with Debugging Options Source: https://context7.com/resemble-ai/dramabox/llms.txt Disable the Perth watermark for debugging purposes during inference. This command also skips reference processing. ```bash python src/inference.py \ --prompt 'A man speaks clearly, "Testing one two three."' \ --output output/test.wav \ --no-watermark \ --no-ref ``` -------------------------------- ### Gemini Synthetic Data Format Source: https://github.com/resemble-ai/dramabox/blob/master/README.md A '~'-separated format used for prompted synthetic data. ```text id~speaker~lang~sr~samples~dur~phonemes~text spk01_001~spk01~en~24000~93000~3.875~_~A woman speaks warmly, "Hello, how are you today?" ``` -------------------------------- ### TSV Data Format Source: https://github.com/resemble-ai/dramabox/blob/master/README.md A simple tab-separated values format for dataset entries. ```tsv wavs/spk01_001.wav A woman speaks warmly, "Hello, how are you today?" wavs/spk01_002.wav Hello, how are you today? ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.