### build_optimizer usage example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.optim.md Example of building a Muon optimizer with cosine scheduler. Requires model and TrainConfig. ```python from irodori_tts.config import TrainConfig from irodori_tts.optim import build_optimizer, build_scheduler cfg = TrainConfig(optimizer="muon", lr_scheduler="cosine", max_steps=10000, warmup_steps=500) optimizer = build_optimizer(model, cfg) scheduler = build_scheduler(optimizer, cfg) ``` -------------------------------- ### Launch Gradio Web UI for reference-audio cloning Source: https://github.com/aratako/irodori-tts/blob/main/README.md Starts the Gradio app for reference-audio cloning and Speaker Inversion on port 7860. Access the UI at `http://localhost:7860`. ```bash uv run --no-sync python gradio_app.py --server-name 0.0.0.0 --server-port 7860 ``` -------------------------------- ### Full fine-tune from inference weights Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.train.md Runs a full fine-tune using torchrun with 4 processes, starting from inference weights. Requires a config file, manifest, output directory, and initial checkpoint. ```bash uv run --no-sync torchrun --nproc_per_node 4 train.py \ --config configs/train_v4_small.yaml \ --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_tts \ --init-checkpoint path/to/Irodori-TTS-v4.1-Small/model.safetensors ``` -------------------------------- ### Launch Gradio Web UI for VoiceDesign Source: https://github.com/aratako/irodori-tts/blob/main/README.md Starts the Gradio app for VoiceDesign with caption and reference-audio conditioning on port 7861. Access the UI at `http://localhost:7861`. ```bash uv run --no-sync python gradio_app_voicedesign.py --server-name 0.0.0.0 --server-port 7861 ``` -------------------------------- ### TrainProgress usage example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.progress.md Example of creating a TrainProgress instance, logging metrics, and closing. Requires `from irodori_tts.progress import TrainProgress`. The example uses rank=0, world_size=1, enabled=True, show_all_ranks=False, description="train", and logs loss and lr metrics with global_batch_size=32. ```python from irodori_tts.progress import TrainProgress pbar = TrainProgress(max_steps=1000, start_step=0, rank=0, world_size=1, enabled=True, show_all_ranks=False, description="train") pbar.log(step=10, epoch=0, metrics={"loss": 0.512, "lr": 1e-4}, global_batch_size=32) pbar.close() ``` -------------------------------- ### LoRA Fine-Tuning with train_v4_small_lora.yaml Source: https://github.com/aratako/irodori-tts/blob/main/README.md Starts a new LoRA training run from released inference weights. Initializes only model weights; optimizer/scheduler state starts fresh. The duration predictor is saved as part of the adapter by default. Targets diffusion attention by default; use pretrained_backbone_attn or pretrained_backbone_attn_mlp to adapt the ModernBERT backbone. ```bash uv run --no-sync python train.py \ --config configs/train_v4_small_lora.yaml \ --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_tts_lora \ --init-checkpoint path/to/Irodori-TTS-v4.1-Small/model.safetensors ``` -------------------------------- ### Example JSONL Manifest Entry Source: https://github.com/aratako/irodori-tts/blob/main/README.md Shows the structure of a manifest entry produced by prepare_manifest.py. Includes text, caption, latent_path, speaker_id, and num_frames fields. Speaker_id is omitted for Speaker Inversion manifests. ```json {"text": "こんにちは", "caption": "落ち着いた、近い距離感の女性話者", "latent_path": "data/latents/00001.pt", "speaker_id": "myorg/my_dataset:speaker_001", "num_frames": 750} ``` -------------------------------- ### Train MeanFlow distillation with configs/train_v4_small_meanflow.yaml Source: https://github.com/aratako/irodori-tts/blob/main/README.md Distill a MeanFlow model using the provided training config. Set train.teacher_checkpoint in the config before starting. See the MeanFlow guide for details. ```bash uv run --no-sync python train.py \ --config configs/train_v4_small_meanflow.yaml \ --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_v4_1_meanflow ``` -------------------------------- ### Clone and install Irodori-TTS with CUDA 12.8 Source: https://github.com/aratako/irodori-tts/blob/main/README.md Clones the repository and syncs dependencies with the NVIDIA CUDA 12.8 backend extra. Use this on Linux/Windows with a compatible GPU. ```bash git clone https://github.com/Aratako/Irodori-TTS.git cd Irodori-TTS uv sync --extra cu128 # NVIDIA CUDA 12.8 (Linux/Windows) ``` -------------------------------- ### SpeakerInversionEmbedding Usage Example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.speaker_inversion.md Example of training a new embedding with SpeakerInversionEmbedding. Creates an instance and calls forward, asserting the output shape. ```python from irodori_tts.speaker_inversion import SpeakerInversionEmbedding embed = SpeakerInversionEmbedding(num_tokens=16, speaker_dim=1280, init_std=0.02) state, mask = embed(batch_size=4, device="cuda", dtype=torch.float32) assert state.shape == (4, 16, 1280) ``` -------------------------------- ### YAML config file structure Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/configuration.md Example YAML configuration for training, with model and train sections. Keys map to ModelConfig and TrainConfig fields. CLI arguments override YAML values; unknown keys raise ValueError. ```yaml model: flow_parameterization: rf_velocity latent_dim: 128 use_caption_condition: true use_speaker_condition: true use_duration_predictor: true train: train_mode: rf batch_size: 8 max_steps: 200000 optimizer: muon lr_scheduler: cosine warmup_steps: 1000 teacher_checkpoint: null # set for train_mode: meanflow_distill ``` -------------------------------- ### Batch synthesis example with error handling Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.inference_runtime.md Shows how to call synthesize_batch with two requests and handle a ValueError. The except block prints the rejection reason and falls back to per-request synthesis. Note the example references 'requests' in the except block, which is not defined in the visible code. ```python try: results = runtime.synthesize_batch([ SamplingRequest(text="一つ目の文。", ref_wav="a.wav", seed=1), SamplingRequest(text="二つ目の文。", ref_wav="b.wav", seed=2), ]) except ValueError as exc: print("Batch rejected:", exc) # e.g. differing cfg settings for req in requests: results = [runtime.synthesize(req) for req in requests] ``` -------------------------------- ### Basic usage example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.tokenizer.md Loads a tokenizer and encodes Japanese text with BOS prepended, returning padded tensors. ```python from irodori_tts import PretrainedTextTokenizer tok = PretrainedTextTokenizer.from_pretrained( "sbintuitions/modernbert-ja-310m", add_bos=True ) ids, mask = tok.batch_encode(["こんにちは", "テスト"], max_length=64) # ids: (2, 64) long, mask: (2, 64) bool, BOS at column 0 ``` -------------------------------- ### Full-Model Fine-Tuning on Single GPU Source: https://github.com/aratako/irodori-tts/blob/main/README.md Fine-tunes v4-Small from its inference checkpoint on a single GPU. Updates the RF body, duration predictor, and shared text/caption backbone jointly. --init-checkpoint loads model weights and starts a new optimizer and schedule. ```bash uv run --no-sync python train.py \ --config configs/train_v4_small.yaml \ --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_tts \ --init-checkpoint path/to/Irodori-TTS-v4.1-Small/model.safetensors ``` -------------------------------- ### build_duration_features usage example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.duration.md Example usage of build_duration_features with a Japanese string, token count, max_text_len, and has_speaker flag. ```python from irodori_tts.duration import build_duration_features features = build_duration_features( ["こんにちは、😆 世界。"], token_counts=[6], max_text_len=256, has_speaker=[True], ) assert features.shape == (1, 14) ``` -------------------------------- ### Example usage of LatentTextDataset Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.dataset.md Shows how to instantiate the dataset with a manifest, latent_dim, speaker conditioning, and reference frame bounds, then access a sample. ```python from irodori_tts.dataset import LatentTextDataset train_ds = LatentTextDataset("data/train_manifest.jsonl", latent_dim=32, enable_speaker_condition=True, ref_min_frames=200, ref_max_frames=2400) item = train_ds[0] print(item["text"], item["latent"].shape, item["ref_latent"].shape, item["has_speaker"]) ``` -------------------------------- ### Basic synthesis example using InferenceRuntime.from_key Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.inference_runtime.md Shows how to create a runtime from a checkpoint and synthesize speech. Uses RuntimeKey with model_device and codec_device set to cuda, then calls synthesize with text, caption, reference audio, and num_candidates. Prints sample rate, audio shape, and used seed, then unloads the runtime. ```python from irodori_tts.inference_runtime import ( InferenceRuntime, RuntimeKey, SamplingRequest, ) runtime = InferenceRuntime.from_key( RuntimeKey(checkpoint="outputs/model.safetensors", model_device="cuda", codec_device="cuda") ) result = runtime.synthesize( SamplingRequest( text="こんにちは、私はAIです。", caption="落ち着いた、近い距離感の女性話者", ref_wav="ref.wav", num_candidates=1, ), log_fn=lambda msg: print(msg), ) print(result.sample_rate, result.audio.shape, result.used_seed) runtime.unload() ``` -------------------------------- ### Usage example for context attention Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.attention.md Shows how to build a context attention plan and run attention with it. Requires self_mask and text_mask tensors, and q, k, v in (B, S, H, D) layout. ```python import torch from irodori_tts.attention import ( build_context_attention_plan, context_attention, ) plan = build_context_attention_plan( [self_mask, text_mask], query_len=750, attention_dtype=torch.bfloat16, ) out = context_attention(q, k, v, plan) # each (B, S, H, D) ``` -------------------------------- ### JSONL output format example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.prepare_manifest.md Shows the expected JSONL row structure for the manifest. `caption` may be a list; training selects one non-empty value randomly per row load. A Speaker Inversion manifest does not require `speaker_id`. ```json {"text": "こんにちは", "caption": "落ち着いた、近い距離感の女性話者", "latent_path": "data/latents/00001.pt", "speaker_id": "myorg/my_dataset:speaker_001", "num_frames": 750} ``` -------------------------------- ### Usage example for get_cached_runtime Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.inference_runtime.md Shows how to use get_cached_runtime and check the created flag. ```python runtime, created = get_cached_runtime(key) if created: print("reloaded runtime for", key.checkpoint) ``` -------------------------------- ### from_pretrained classmethod Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.tokenizer.md Loads a tokenizer from Hugging Face Hub with specified options. Raises RuntimeError if transformers is not installed. ```python @classmethod def from_pretrained( cls, repo_id: str, add_bos: bool = True, local_files_only: bool = False, revision: str | None = None, ) -> "PretrainedTextTokenizer": ``` -------------------------------- ### Launch two student processes with torchrun Source: https://github.com/aratako/irodori-tts/blob/main/docs/meanflow.md Launches two student processes for the separate-teacher-GPU setup. Students use cuda:0,1; their teachers use cuda:2,3. With eight visible GPUs, use four student ranks and an offset of four to map teachers to cuda:4..7. ```bash uv run --no-sync torchrun --nproc_per_node=2 train.py \ --config configs/train_v4_small_meanflow.yaml \ --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_v4_1_meanflow ``` -------------------------------- ### Example: set_gradient_checkpointing and forward call with caption conditioning Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.model.md Shows how to instantiate TextToLatentRFDiT with caption conditioning, enable gradient checkpointing with store_every=4, and perform a train-time forward call that returns (v_pred, duration_pred) on the duration path. ```python from irodori_tts import ModelConfig, TextToLatentRFDiT model = TextToLatentRFDiT(ModelConfig(use_caption_condition=True)) model.set_gradient_checkpointing(True, store_every=4) # Train-time call: returns (v_pred, duration_pred) on the duration path v_pred, dur_pred = model( x_t=noise, t=timesteps, text_input_ids=ids, text_mask=mask, ref_latent=ref, ref_mask=ref_mask, caption_input_ids=caption_ids, caption_mask=caption_mask, duration_features=features, duration_has_speaker=has_sp, ) ``` -------------------------------- ### Basic usage example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.watermark.md Creates a watermarker on CUDA with default model_type "44.1k". Checks readiness before encoding; falls back to passthrough if not ready. ```python from irodori_tts.watermark import SilentCipherWatermarker wm = SilentCipherWatermarker(device="cuda") # model_type="44.1k" assert wm.ready or wm.model is None # ready() is the safe check if wm.ready: marked = wm.encode_one(audio, sample_rate=48000) else: marked = audio # silently passthrough ``` -------------------------------- ### Basic usage of sample_euler_rf_cfg Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.rf.md Minimal example calling sample_euler_rf_cfg with a loaded TextToLatentRFDiT model, text and reference latents, sequence length 750, 24 Euler steps, and CFG scales. Asserts the output shape matches (1, 750, patched_latent_dim). ```python import torch from irodori_tts.rf import sample_euler_rf_cfg z = sample_euler_rf_cfg( model=model, # loaded TextToLatentRFDiT text_input_ids=text_ids, text_mask=text_mask, ref_latent=ref_latent, ref_mask=ref_mask, sequence_length=750, num_steps=24, cfg_scale_text=3.0, cfg_scale_speaker=5.0, seed=42, ) assert z.shape == (1, 750, model.cfg.patched_latent_dim) ``` -------------------------------- ### Single-GPU prepare_manifest.py command Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.prepare_manifest.md Runs manifest preparation on a single GPU. Requires `--dataset`, `--audio-column`, `--text-column`, `--output-manifest`, and `--latent-dir`; optional columns and device are shown. ```bash uv run --no-sync python prepare_manifest.py \ --dataset myorg/my_dataset --split train \ --audio-column audio --text-column text \ --caption-column caption --speaker-column speaker \ --output-manifest data/train_manifest.jsonl \ --latent-dir data/latents --device cuda ``` -------------------------------- ### Prepare Training Manifest with prepare_manifest.py Source: https://github.com/aratako/irodori-tts/blob/main/README.md Encodes audio from a Hugging Face dataset into DACVAE latents and produces a JSONL manifest for training. Requires a dataset with audio, text, caption, and speaker columns. The caption may be a list of strings; training randomly selects one non-empty caption per row. ```bash uv run --no-sync python prepare_manifest.py \ --dataset myorg/my_dataset \ --split train \ --audio-column audio \ --text-column text \ --caption-column caption \ --speaker-column speaker \ --output-manifest data/train_manifest.jsonl \ --latent-dir data/latents \ --device cuda ``` -------------------------------- ### Multi-GPU prepare_manifest.py command with merge Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.prepare_manifest.md Runs manifest preparation across 4 GPUs and merges outputs. Uses `--num-gpus 4` and `--merge-output`. ```bash # Multi-GPU + merge uv run --no-sync python prepare_manifest.py \ --dataset myorg/my_dataset --audio-column audio --text-column text \ --output-manifest data/train_manifest.jsonl --latent-dir data/latents \ --num-gpus 4 --merge-output ``` -------------------------------- ### Multi-clip reference and Sway sampling with infer.py Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.infer.md Uses multiple reference audio clips (--ref-wavs) and Sway sampling with a custom step count and sway coefficient. Requires a local checkpoint file (--checkpoint) instead of a Hugging Face one. ```bash # Multi-clip reference + Sway sampling python infer.py --checkpoint model.safetensors --text "複数参照のテスト。" \ --ref-wavs a.wav b.wav --num-steps 6 --t-schedule-mode sway --sway-coeff -1.0 ``` -------------------------------- ### VoiceDesign with Reference and Caption Source: https://github.com/aratako/irodori-tts/blob/main/README.md Run VoiceDesign using both reference speech and caption conditioning by passing --ref-wav along with --caption. ```bash uv run --no-sync python infer.py \ --hf-checkpoint Aratako/Irodori-TTS-v4.1-Small \ --text "あははっ🤭、それ本当に言ってるの?…😮‍💨まぁ、君らしいけどね。" \ --caption "余裕のある大人の男性。親しい相手に対して、くだけた雰囲気で呆れながらも楽しそうに話している。" \ --ref-wav path/to/reference.wav \ --output-wav outputs/sample_voice_design_ref_caption.wav ``` -------------------------------- ### Resume full-model training with --resume Source: https://github.com/aratako/irodori-tts/blob/main/README.md Resumes an existing full-model training run from a .pt checkpoint, restoring optimizer, scheduler, and step state. Use with the same config and manifest as the original run. ```bash uv run --no-sync python train.py \ --config configs/train_v4_small.yaml \ --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_tts \ --resume outputs/irodori_tts/checkpoint_0010000.pt ``` -------------------------------- ### Error handling example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.tokenizer.md Catches ValueError when batch_encode is called with an empty list. ```python try: ids, mask = tok.batch_encode([], max_length=64) except ValueError as exc: print(exc) # "texts must contain at least one item." ``` -------------------------------- ### Resume training Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.train.md Resumes training from a checkpoint file. Requires a config, manifest, output directory, and the --resume flag pointing to a .pt checkpoint. ```bash uv run --no-sync python train.py \ --config configs/train_v4_small.yaml --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_tts \ --resume outputs/irodori_tts/checkpoint_0010000.pt ``` -------------------------------- ### is_torchao_quantized_state_dict signature Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.quantization.md Returns True if any tensor type's module starts with "torchao.". ```python def is_torchao_quantized_state_dict(state_dict: Mapping[str, torch.Tensor]) -> bool: ``` -------------------------------- ### PretrainedTextTokenizer.from_pretrained Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.tokenizer.md Loads a PretrainedTextTokenizer from a Hugging Face repository using AutoTokenizer. Raises RuntimeError if transformers is not installed. ```APIDOC ## PretrainedTextTokenizer.from_pretrained ### Description Loads a PretrainedTextTokenizer from a Hugging Face repository using AutoTokenizer. Raises RuntimeError if transformers is not installed. ### Method Classmethod ### Signature ```python @classmethod def from_pretrained( cls, repo_id: str, add_bos: bool = True, local_files_only: bool = False, revision: str | None = None, ) -> "PretrainedTextTokenizer": ``` ### Parameters - **repo_id** (str) - Required - Hugging Face repository ID - **add_bos** (bool) - Optional, default `True` - Prepend BOS token on each encode - **local_files_only** (bool) - Optional, default `False` - Only use local files - **revision** (str | None) - Optional - Model revision to use ### Raises - `RuntimeError` — transformers not installed. ### Example ```python from irodori_tts import PretrainedTextTokenizer tok = PretrainedTextTokenizer.from_pretrained( "sbintuitions/modernbert-ja-310m", add_bos=True ) ids, mask = tok.batch_encode(["こんにちは", "テスト"], max_length=64) # ids: (2, 64) long, mask: (2, 64) bool, BOS at column 0 ``` ``` -------------------------------- ### Basic voice cloning with infer.py Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.infer.md Runs voice cloning using a Hugging Face checkpoint, reference audio, and input text. Requires --ref-wav for speaker-conditioned checkpoints unless --no-ref is used. Outputs a WAV file specified by --output-wav. ```bash # Basic voice cloning python infer.py --hf-checkpoint Aratako/Irodori-TTS-v4.1-Small \ --text "こんにちは、私はAIです。" --ref-wav ref.wav --output-wav out.wav ``` -------------------------------- ### Run simple inference with uv Source: https://github.com/aratako/irodori-tts/blob/main/README.md Executes the inference script with a Hugging Face checkpoint, input text, reference audio, and output path. Uses `--no-sync` to avoid syncing dependencies. ```bash uv run --no-sync python infer.py \ --hf-checkpoint Aratako/Irodori-TTS-v4.1-Small \ --text "こんにちは、私はAIです。これは音声合成のテストです。" \ --ref-wav path/to/reference.wav \ --output-wav outputs/sample.wav ``` -------------------------------- ### current_pretrained_text_encoder_lr Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.optim.md Returns the learning rate of the first parameter group whose name starts with 'pretrained_text_encoder_', or None if not found. ```APIDOC ## current_pretrained_text_encoder_lr ### Description Returns the learning rate of the first parameter group whose name starts with 'pretrained_text_encoder_', or None if not found. ### Method current_pretrained_text_encoder_lr ### Signature ```python def current_pretrained_text_encoder_lr(optimizer) -> float | None: ``` ### Parameters - **optimizer** - Required - The optimizer. ### Returns - **float | None** - The learning rate of the first matching group, or None. ``` -------------------------------- ### Sway Sampling with fewer steps Source: https://github.com/aratako/irodori-tts/blob/main/docs/parameters.md Use for lower-latency experiments. Sets --num-steps 6, --t-schedule-mode sway, and --sway-coeff -1.0. Requires a reference WAV file and an output path. ```bash uv run python infer.py \ --hf-checkpoint Aratako/Irodori-TTS-v4.1-Small \ --text "こんにちは、私はAIです。これは音声合成のテストです。" \ --ref-wav path/to/reference.wav \ --num-steps 6 \ --t-schedule-mode sway \ --sway-coeff -1.0 \ --output-wav outputs/sample_sway.wav ``` -------------------------------- ### Resume LoRA training with --resume Source: https://github.com/aratako/irodori-tts/blob/main/README.md Resumes a LoRA training run from a checkpoint directory. If the original base-checkpoint path is invalid in a new environment, pass --init-checkpoint path/to/base_model.safetensors together with --resume to override the saved base-model path. ```bash uv run --no-sync python train.py \ --config configs/train_v4_small_lora.yaml \ --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_tts_lora \ --resume outputs/irodori_tts_lora/checkpoint_0010000 ``` -------------------------------- ### current_lr functions Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.optim.md Functions to retrieve current learning rates. The second returns LR of first group with group_name starting with 'pretrained_text_encoder_'. ```python def current_lr(optimizer) -> float: # line 444 — group 0 "lr" def current_pretrained_text_encoder_lr(optimizer) -> float | None: # line 448 ``` -------------------------------- ### Example: enabling rare token split Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.token_split.md Shows how to load the tokenizer and enable rare token splitting, then batch-encode a test string with max_length=128. ```python from irodori_tts.tokenizer import PretrainedTextTokenizer from irodori_tts.token_split import SPLIT_TOKENIZER_REPO tok = PretrainedTextTokenizer.from_pretrained(SPLIT_TOKENIZER_REPO, add_bos=True) tok.enable_rare_token_split() ids, mask = tok.batch_encode(["とても長い珍しい語彙のテスト"], max_length=128) ``` -------------------------------- ### Launch Gradio demo with queue and launch options Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.gradio_apps.md Common launch pattern for both Gradio apps. Builds the UI, sets default_concurrency_limit=1, and launches with server name, port, share, debug, and CSS options. ```python demo = build_ui() demo.queue(default_concurrency_limit=1) demo.launch(server_name=args.server_name, server_port=args.server_port, share=bool(args.share), debug=bool(args.debug), css=EMOJI_PALETTE_CSS) ``` -------------------------------- ### merge_dataclass_overrides usage example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.config.md Shows how to merge overrides into a default ModelConfig with section label 'checkpoint model_config'. The resulting cfg has latent_dim=128 and num_layers=24. ```python from irodori_tts.config import ModelConfig, merge_dataclass_overrides cfg = merge_dataclass_overrides( ModelConfig(), {"latent_dim": 128, "num_layers": 24}, section="checkpoint model_config", ) assert cfg.latent_dim == 128 and cfg.num_layers == 24 ``` -------------------------------- ### Multi-GPU DDP Training with torchrun Source: https://github.com/aratako/irodori-tts/blob/main/README.md Runs full-model fine-tuning with Distributed Data Parallel across 4 GPUs. Uses the same config and manifest as single-GPU training, with --device cuda. ```bash uv run --no-sync torchrun --nproc_per_node 4 train.py \ --config configs/train_v4_small.yaml \ --manifest data/train_manifest.jsonl \ --output-dir outputs/irodori_tts \ --init-checkpoint path/to/Irodori-TTS-v4.1-Small/model.safetensors \ --device cuda ``` -------------------------------- ### Inference with Local Checkpoint Source: https://github.com/aratako/irodori-tts/blob/main/README.md Run inference using a local checkpoint file (.pt or .safetensors). Specify the checkpoint path, input text, reference audio, and output file. ```bash uv run --no-sync python infer.py \ --checkpoint outputs/checkpoint_final.safetensors \ --text "こんにちは、私はAIです。これは音声合成のテストです。" \ --ref-wav path/to/reference.wav \ --output-wav outputs/sample.wav ``` -------------------------------- ### is_torchao_quantized_state_dict(state_dict: Mapping[str, torch.Tensor]) -> bool Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.quantization.md Checks if any tensor type's module starts with 'torchao.' to determine if the state dict is torchao-quantized. ```APIDOC ## is_torchao_quantized_state_dict(state_dict: Mapping[str, torch.Tensor]) -> bool ### Description Checks if any tensor type's module starts with `torchao.` to determine if the state dict is torchao-quantized. ### Method Function ### Signature `is_torchao_quantized_state_dict(state_dict: Mapping[str, torch.Tensor]) -> bool` ### Parameters - **state_dict** (Mapping[str, torch.Tensor]) - Required - The state dict to check. ### Returns - **bool** - `True` if any tensor type's module starts with `torchao.`, otherwise `False`. ``` -------------------------------- ### VoiceDesign with Caption Only Source: https://github.com/aratako/irodori-tts/blob/main/README.md Run VoiceDesign using caption conditioning only by passing --no-ref. Requires v4-Small checkpoint. ```bash uv run --no-sync python infer.py \ --hf-checkpoint Aratako/Irodori-TTS-v4.1-Small \ --text "こんにちは、私はAIです。これは音声合成のテストです。" \ --caption "落ち着いた、近い距離感の女性話者" \ --no-ref \ --output-wav outputs/sample_voice_design.wav ``` -------------------------------- ### sample_euler_meanflow error handling example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.meanflow.md Shows catching ValueError when passing a non-MeanFlow model. The error message is "sample_euler_meanflow requires a MeanFlow model." ```python try: sample_euler_meanflow(model=rf_model, text_input_ids=ids, text_mask=mask, ref_latent=None, ref_mask=None, sequence_length=750) except ValueError as exc: print(exc) # "sample_euler_meanflow requires a MeanFlow model." ``` -------------------------------- ### Quantize a checkpoint with quantize_checkpoint.py Source: https://github.com/aratako/irodori-tts/blob/main/README.md Quantize a compatible inference checkpoint locally. INT8 weight-only is the default; the default core profile quantizes attention and MLP weights in text, speaker, and diffusion Transformer blocks. --profile all-linear is available for more aggressive experimentation. ```bash uv run --no-sync python quantize_checkpoint.py path/to/model.safetensors \ --quantization int8-weight-only \ --output path/to/quantized/model.safetensors ``` -------------------------------- ### encode_waveform basic usage example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.codec.md Shows loading the codec and encoding a waveform to latent, then decoding back to audio. Requires a CUDA device and a 48 kHz waveform. ```python from irodori_tts.codec import DACVAECodec codec = DACVAECodec.load(device="cuda") latent = codec.encode_waveform(wav, sample_rate=48000) # (1, T, D) audio = codec.decode_latent(latent) # (1, 1, samples) ``` -------------------------------- ### Run quantized inference with int8-weight-only checkpoint Source: https://github.com/aratako/irodori-tts/blob/main/README.md Infer speech using a pre-quantized v4-Small checkpoint. Select a variant by appending its subdirectory name to the Hugging Face repository ID. --model-precision bf16 controls unquantized layers and floating-point activations; quantized weights retain their stored format. ```bash uv run --no-sync python infer.py \ --hf-checkpoint Aratako/Irodori-TTS-v4.1-Small-Quantized/int8-weight-only \ --model-precision bf16 \ --text "こんにちは、私はAIです。" \ --no-ref \ --output-wav outputs/sample_int8.wav ``` -------------------------------- ### Usage examples for normalize_text and strip_outer_brackets Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.text_normalization.md Shows expected outputs for bracket stripping and normalization. Note that non-fully-enclosing brackets remain unchanged, and wave dashes become the long-vowel mark. ```python from irodori_tts.text_normalization import normalize_text, strip_outer_brackets strip_outer_brackets("(こんにちは)") # "こんにちは" strip_outer_brackets("((入れ子))は残る)") # unchanged (not fully enclosing) normalize_text("(こんにちは!!!)") # "こんにちは!!!" normalize_text("こんにちは。。。") # "こんにちは…" normalize_text("さようなら〜") # "さようならー" (wave dash → long vowel) ``` -------------------------------- ### Select PyTorch backend extras Source: https://github.com/aratako/irodori-tts/blob/main/README.md Choose one mutually exclusive backend extra for PyTorch. The cu128, rocm, xpu, and cpu extras target different hardware; the cpu extra falls back to PyPI wheels on macOS. ```bash # NVIDIA CUDA 12.8 on Linux/Windows uv sync --extra cu128 ``` ```bash # AMD ROCm on Linux/WSL uv sync --extra rocm ``` ```bash # Intel XPU on Linux/Windows uv sync --extra xpu ``` ```bash # CPU-only, or macOS CPU/MPS via PyPI uv sync --extra cpu ``` -------------------------------- ### checkpoint_state_uses_lora Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.lora.md Checks if a model state dictionary contains LoRA-related keys. Returns `True` if any key starts with `"base_model.model."` or contains `".lora_"`. ```APIDOC ## checkpoint_state_uses_lora ### Description Checks if a model state dictionary contains LoRA-related keys. Returns `True` if any key starts with `"base_model.model."` or contains `".lora_"`. ### Function Signature ```python def checkpoint_state_uses_lora(model_state: Mapping[str, torch.Tensor]) -> bool: ``` ### Parameters - **model_state** (Mapping[str, torch.Tensor]) - Required - The model state dictionary to inspect. ### Returns - **bool** - `True` if LoRA keys are present, `False` otherwise. ``` -------------------------------- ### sample_euler_meanflow basic usage example Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.meanflow.md Basic usage of sample_euler_meanflow with a MeanFlow model, text inputs, reference latent, and sequence length 750. Uses 4 steps and seed 7. ```python from irodori_tts.meanflow import sample_euler_meanflow z = sample_euler_meanflow( model=mf_model, text_input_ids=ids, text_mask=mask, ref_latent=ref_latent, ref_mask=ref_mask, sequence_length=750, num_steps=4, seed=7, ) ``` -------------------------------- ### load_config_yaml function signature Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.config.md Loads a training config YAML and returns the parsed top-level mapping, or {} for an empty document. Raises RuntimeError if PyYAML is not installed, and ValueError if the config root is not a mapping. ```python def load_config_yaml(path: str | Path) -> dict[str, Any]: ``` -------------------------------- ### Convert checkpoint to safetensors Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.checkpoint_tools.md Run the conversion script on a .pt checkpoint or a LoRA adapter directory. The first command converts a training checkpoint; the second converts a LoRA adapter directory (requires --base-checkpoint if adapter-only). ```bash uv run --no-sync python convert_checkpoint_to_safetensors.py outputs/checkpoint_final.pt ``` ```bash uv run --no-sync python convert_checkpoint_to_safetensors.py outputs/irodori_tts_lora/checkpoint_final ``` -------------------------------- ### Speaker Inversion (embedding-only output) Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.train.md Runs speaker inversion to produce an embedding-only output. Requires a speaker inversion config, manifest, initial checkpoint, and output directory. ```bash uv run --no-sync python train.py \ --config configs/train_v4_small_speaker_inversion.yaml \ --manifest data/tts_manifest.jsonl \ --init-checkpoint path/to/model.safetensors \ --output-dir outputs/speaker_inversion/name ``` -------------------------------- ### Example: save and reuse speaker inversion checkpoint Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.speaker_inversion.md Shows saving a speaker inversion checkpoint and loading it back for inference. The loaded payload contains the speaker embedding tensor with shape (tokens, speaker_dim). ```python from irodori_tts.speaker_inversion import ( save_speaker_inversion_checkpoint, load_speaker_inversion_payload, ) save_speaker_inversion_checkpoint("out/voice.speaker.safetensors", model=model) payload = load_speaker_inversion_payload("out/voice.speaker.safetensors") print(payload["speaker_embedding"].shape) # (tokens, speaker_dim) ``` -------------------------------- ### Quantize checkpoint with torchao Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.checkpoint_tools.md Quantize an inference .safetensors checkpoint using the specified quantization scheme and output path. The input must exist and have a .safetensors suffix; output collisions require --force. ```bash uv run --no-sync python quantize_checkpoint.py path/to/model.safetensors \ --quantization int8-weight-only --output quantized/model.safetensors ``` -------------------------------- ### MuonWithAuxAdamW methods Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.optim.md Methods for zeroing gradients, stepping, and state dict handling. load_state_dict raises ValueError if 'muon' key is missing. ```python def zero_grad(self, set_to_none: bool = True) -> None: # line 24 def step(self) -> None: # line 29 def state_dict(self) -> dict: # line 34 → {"muon": ..., "aux": ... | None} def load_state_dict(self, state_dict: dict) -> None: # line 40 ``` -------------------------------- ### LoRA Adapter Inference Source: https://github.com/aratako/irodori-tts/blob/main/README.md Load a LoRA adapter directory dynamically at inference time without exporting a merged checkpoint. Use --lora-adapter to specify the adapter directory. ```bash uv run --no-sync python infer.py \ --checkpoint path/to/base_model.safetensors \ --lora-adapter outputs/irodori_tts_lora/checkpoint_final \ --text "こんにちは、私はAIです。これはLoRA推論のテストです。" \ --ref-wav path/to/reference.wav \ --output-wav outputs/sample_lora.wav ``` -------------------------------- ### DACVAECodec.load classmethod Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.codec.md Loads the DACVAE model from an installed package or local clone, resolves the model location, disables watermark, and infers latent dimension. Default repo_id is 'Aratako/Semantic-DACVAE-Japanese-32dim', device 'cuda', and normalize_db -16.0. ```python @classmethod def load( cls, repo_id: str = "Aratako/Semantic-DACVAE-Japanese-32dim", device: str = "cuda", dtype: torch.dtype | None = None, deterministic_encode: bool = True, deterministic_decode: bool = True, normalize_db: float | None = -16.0, ) -> DACVAECodec: ``` -------------------------------- ### Sway Sampling Inference Source: https://github.com/aratako/irodori-tts/blob/main/README.md Combine Sway Sampling with fewer Euler steps for faster experimental inference. Use --num-steps 6, --t-schedule-mode sway, and --sway-coeff -1.0. ```bash uv run --no-sync python infer.py \ --hf-checkpoint Aratako/Irodori-TTS-v4.1-Small \ --text "こんにちは、私はAIです。これは音声合成のテストです。" \ --ref-wav path/to/reference.wav \ --num-steps 6 \ --t-schedule-mode sway \ --sway-coeff -1.0 \ --output-wav outputs/sample_sway.wav ``` -------------------------------- ### MuonWithAuxAdamW constructor Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.optim.md Initializes the wrapper with a Muon optimizer and an optional AdamW optimizer. param_groups is the union of both optimizers' groups. ```python def __init__(self, muon_opt: torch.optim.Optimizer, aux_opt: torch.optim.Optimizer | None): ``` -------------------------------- ### Catching patterns for built-in exceptions Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/errors.md Shows how to catch the built-in exceptions raised by the library. Validation failures raise ValueError, environment/model-state failures raise RuntimeError, and missing files raise FileNotFoundError. The example catches each type and passes the exception arguments to a handler. ```python try: runtime.synthesize(SamplingRequest(text=text, ref_wav="a.wav")) except ValueError as exc: # validation failures (most common) handle(*exc.args) except RuntimeError as exc: # environment / model-state failures handle(*exc.args) except FileNotFoundError as exc: handle(*exc.args) ``` -------------------------------- ### MuonWithAuxAdamW.__init__ Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.optim.md Constructor for the MuonWithAuxAdamW wrapper. It combines a Muon optimizer for Muon-compatible parameters and an AdamW optimizer for auxiliary parameters. ```APIDOC ## MuonWithAuxAdamW.__init__ ### Description Constructor for the MuonWithAuxAdamW wrapper. It combines a Muon optimizer for Muon-compatible parameters and an AdamW optimizer for auxiliary parameters. ### Method Constructor ### Signature ```python def __init__(self, muon_opt: torch.optim.Optimizer, aux_opt: torch.optim.Optimizer | None): ``` ### Parameters - **muon_opt** (torch.optim.Optimizer) - Required - Muon optimizer for Muon-compatible parameters. - **aux_opt** (torch.optim.Optimizer | None) - Required - AdamW optimizer for auxiliary parameters; can be None. ### Notes `param_groups` is the union of both optimizers' groups (None aux allowed). ``` -------------------------------- ### list_available_runtime_devices and default_runtime_device signatures Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.inference_runtime.md Returns available devices in order ["cuda", "mps", "xpu", "cpu"] and the first available device. ```python def list_available_runtime_devices() -> list[str]: def default_runtime_device() -> str: ``` -------------------------------- ### VoiceDesign with caption only using infer.py Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/cli.infer.md Uses VoiceDesign by providing a caption instead of a reference audio. Requires --no-ref to bypass the reference option guard for speaker-conditioned checkpoints. The caption describes the desired voice characteristics. ```bash # VoiceDesign, caption only python infer.py --hf-checkpoint Aratako/Irodori-TTS-v4.1-Small \ --text "あははっ🤭、本当?" \ --caption "余裕のある大人の男性。親しい相手に話している。" \ --no-ref --output-wav out.wav ``` -------------------------------- ### quantize_model signature Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.quantization.md Quantizes a model in place with torchao.quantization.quantize_. profile="core" quantizes attention/MLP nn.Linear layers inside pretrained_text_backbone and the four transformers; "all-linear" quantizes every nn.Linear. Returns list of quantized module FQNs. Raises ValueError for unknown profile, no modules selected, no modules quantized, or unsupported INT4 group size; RuntimeError if torchao not installed. ```python def quantize_model( model: torch.nn.Module, *, quantization_type: str = QUANTIZATION_TYPE_INT8_WEIGHT_ONLY, profile: str = "core", int4_group_size: int = DEFAULT_INT4_GROUP_SIZE, ) -> list[str]: ``` -------------------------------- ### PretrainedTextTokenizer constructor Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.tokenizer.md Initializes the wrapper with a backing tokenizer and optional BOS prepend. Raises ValueError if pad or BOS tokens are missing. ```python def __init__(self, tokenizer, add_bos: bool = True) -> None: ``` -------------------------------- ### Convert LoRA adapter checkpoint to safetensors Source: https://github.com/aratako/irodori-tts/blob/main/README.md Converts a LoRA adapter checkpoint directly. The adapter is merged into the base model automatically during conversion, so the exported .safetensors file is directly usable for inference. To avoid merging, pass the adapter directory directly to infer.py --lora-adapter or the matching Gradio field. ```bash uv run --no-sync python convert_checkpoint_to_safetensors.py outputs/irodori_tts_lora/checkpoint_final ``` -------------------------------- ### Convert full-model checkpoint to safetensors Source: https://github.com/aratako/irodori-tts/blob/main/README.md Converts a training checkpoint to inference-only safetensors format. For checkpoints with a pretrained text encoder, conversion also writes a tokenizer/ directory beside the safetensors file and embeds the encoder architecture config in the file; keep both together when publishing or moving the model. ```bash uv run --no-sync python convert_checkpoint_to_safetensors.py outputs/checkpoint_final.pt ``` -------------------------------- ### InferenceRuntime.__init__ Source: https://github.com/aratako/irodori-tts/blob/main/_autodocs/api-reference/irodori_tts.inference_runtime.md Constructs an InferenceRuntime from already-loaded components. Most callers should use InferenceRuntime.from_key instead. ```APIDOC ## InferenceRuntime.__init__ ### Description Constructs a runtime from already-loaded components. Most callers use `InferenceRuntime.from_key` instead. ### Method Constructor ### Signature ```python def __init__( self, *, key: RuntimeKey, model_cfg: ModelConfig, train_cfg: dict | None, model: TextToLatentRFDiT, tokenizer: PretrainedTextTokenizer, caption_tokenizer: PretrainedTextTokenizer | None, codec: DACVAECodec, default_text_max_len: int, default_caption_max_len: int, default_max_ref_seconds: float = _LEGACY_MAX_REF_SECONDS, ) -> None: ``` ### Parameters - **key** (RuntimeKey) - Required - Runtime identity (devices, precision, compile flags) - **model_cfg** (ModelConfig) - Required - Model architecture config - **train_cfg** (dict | None) - Required - Inference-relevant train config subset (e.g. `max_text_len`, `ref_max_seconds`) - **model** (TextToLatentRFDiT) - Required - Loaded, eval-mode model - **tokenizer** (PretrainedTextTokenizer) - Required - Text tokenizer - **caption_tokenizer** (PretrainedTextTokenizer | None) - Required - Caption tokenizer (caption checkpoints only) - **codec** (DACVAECodec) - Required - DACVAE codec - **default_text_max_len** (int) - Required - Fallback `max_text_len` - **default_caption_max_len** (int) - Required - Fallback `max_caption_len` - **default_max_ref_seconds** (float) - Optional - Default: `30.0` - Checkpoint-recommended reference cap ### Source `irodori_tts/inference_runtime.py` (line 680). ```