### SageAttention Installation (Windows)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

Commands and steps for installing SageAttention on Windows. This involves cloning the repository, setting up the environment, and running the installation script. Dependencies include triton and Visual Studio.

```shell
git clone https://github.com/thu-ml/SageAttention.git
```

```shell
set DISTUTILS_USE_SDK=1
python setup.py install
```

--------------------------------

### Install SageAttention using setup.py

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

This command installs SageAttention after cloning the repository. It requires a Python environment with necessary build tools. If DISTUTILS is not configured, set the environment variable DISTUTILS_USE_SDK=1.

```shell
python setup.py install
```

--------------------------------

### Start Training LoRA Network

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

Initiates the training process for a LoRA network using the accelerate library. This command requires specifying paths to the base model checkpoints and dataset configuration. It supports various optimization techniques like mixed precision, FP8, gradient checkpointing, and different attention mechanisms (SDPA, FlashAttention, xformers). Memory optimization flags like `--fp8_base` and `--blocks_to_swap` are detailed, along with options for visualizing training progress.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 hv_train_network.py \
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt \
    --dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base \
    --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing \
    --max_data_loader_n_workers 2 --persistent_data_loader_workers \
    --network_module networks.lora --network_dim 32 \
    --timestep_sampling shift --discrete_flow_shift 7.0 \
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \
    --output_dir path/to/output_dir --output_name name-of-lora
```

```bash
uv run accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 hv_train_network.py \
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt \
    --dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base \
    --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing \
    --max_data_loader_n_workers 2 --persistent_data_loader_workers \
    --network_module networks.lora --network_dim 32 \
    --timestep_sampling shift --discrete_flow_shift 7.0 \
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \
    --output_dir path/to/output_dir --output_name name-of-lora
```

--------------------------------

### Install uv on Windows

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Installs the 'uv' package installer on Windows systems using PowerShell. 'uv' is a fast Python package installer and resolver. The installation script may prompt you to restart your system to update the PATH environment variable.

```powershell
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
```

--------------------------------

### Install Optional Dependencies (ascii-magic, matplotlib, tensorboard)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Installs optional Python packages that enhance Musubi Tuner's functionality. 'ascii-magic' is used for dataset visualization, 'matplotlib' for visualizing timesteps, and 'tensorboard' for logging training progress. Install these if you need these specific features.

```bash
pip install ascii-magic matplotlib tensorboard
```

--------------------------------

### Install uv on Linux/MacOS

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Installs the 'uv' package installer on Linux or macOS systems. 'uv' is a fast Python package installer and resolver. Follow the on-screen instructions to add 'uv' to your system's PATH for it to be accessible.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Install Musubi Tuner Dependencies

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Installs all necessary Python packages required by Musubi Tuner. This command reads the dependencies from the 'requirements.txt' file in the project's root directory. It's crucial to run this after cloning the repository.

```bash
pip install -r requirements.txt
```

--------------------------------

### Help Command for hv_generate_video.py

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

This command displays all available options and their descriptions for the hv_generate_video.py script, allowing users to explore advanced configurations and understand parameter functionalities.

```bash
python hv_generate_video.py --help
```

--------------------------------

### Get Training Help

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

Retrieves detailed help information for the training script, listing all available command-line options. Many of these options may not be fully verified.

```python
python wan_train_network.py --help
```

--------------------------------

### Frame Extraction Examples: head, chunk, slide, uniform

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

Illustrates different frame extraction methods (head, chunk, slide, uniform) with example parameters and their resulting frame selections from an original video of 40 frames.

```text
Original Video, 40 frames: x = frame, o = no frame
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

head, target_frames = [1, 13, 25] -> extract head frames:
xooooooooooooooooooooooooooooooooooooooo
xxxxxxxxxxxxxooooooooooooooooooooooooooo
xxxxxxxxxxxxxxxxxxxxxxxxxooooooooooooooo

chunk, target_frames = [13, 25] -> extract frames by splitting into chunks, into 13 and 25 frames:
xxxxxxxxxxxxxooooooooooooooooooooooooooo
xxxxxxxxxxxxxooooooooooooooooooooooooooo
xxxxxxxxxxxxxxxxxxxxxxxxxooooooooooooooo

NOTE: Please do not include 1 in target_frames if you are using the frame_extraction "chunk". It will make the all frames to be extracted.
注: frame_extraction "chunk" を使用する場合、target_frames に 1 を含めないでください。全てのフレームが抽出されてしまいます。

slide, target_frames = [1, 13, 25], frame_stride = 10 -> extract N frames with a stride of 10:
xooooooooooooooooooooooooooooooooooooooo
xxxxxxxxxxxxxooooooooooooooooooooooooooo
xxxxxxxxxxxxxxxxxxxxxxxxxooooooooooooooo
xxxxxxxxxxxxxxxxxxxxxxxxxooooooooooooooo

uniform, target_frames =[1, 13, 25], frame_sample = 4 -> extract `frame_sample` samples uniformly, N frames each:
xooooooooooooooooooooooooooooooooooooooo
xxxxxxxxxxxxxooooooooooooooooooooooooooo
xxxxxxxxxxxxxxxxxxxxxxxxxooooooooooooooo
xxxxxxxxxxxxxxxxxxxxxxxxxooooooooooooooo
```

--------------------------------

### Clone SageAttention Repository using Git

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

This command clones the SageAttention repository from GitHub. Ensure you have Git installed and configured on your system.

```shell
git clone https://github.com/thu-ml/SageAttention.git
```

--------------------------------

### Start LoRA Training with Accelerate

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

This command initiates the LoRA training process using `accelerate launch`. It specifies paths to the DiT model, dataset configuration, and output directory. Key parameters include mixed precision, optimizer type, learning rate, gradient checkpointing, network module and dimension, timestep sampling, and seed. The `--fp8_base` flag can reduce VRAM usage at the potential cost of quality. VRAM constraints can be further addressed using `--blocks_to_swap`.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 hv_train_network.py 
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt 
    --dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base 
    --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing 
    --max_data_loader_n_workers 2 --persistent_data_loader_workers 
    --network_module networks.lora --network_dim 32 
    --timestep_sampling shift --discrete_flow_shift 7.0 
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42
    --output_dir path/to/output_dir --output_name name-of-lora
```

--------------------------------

### Training Command Line Options for Sampling

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/sampling_during_training.md

This snippet shows example command-line arguments for training with sample image generation. It includes options for VAE, text encoders, prompt files, and sampling frequency. Refer to the documentation for the specific meanings of each argument.

```bash
--vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt 
--vae_chunk_size 32 --vae_spatial_tile_sample_min_size 128
--text_encoder1 path/to/ckpts/text_encoder 
--text_encoder2 path/to/ckpts/text_encoder_2 
--sample_prompts /path/to/prompt_file.txt 
--sample_every_n_epochs 1 --sample_every_n_steps 1000 --sample_at_first
```

--------------------------------

### Install PyTorch and Torchvision (CUDA 12.4)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Installs PyTorch and Torchvision with CUDA 12.4 support. Ensure you have a compatible CUDA version installed on your system. This is a prerequisite for running Musubi Tuner and other deep learning tasks.

```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
```

--------------------------------

### Example Directory Structure for HunyuanVideo Models

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Illustrates the expected directory structure for downloaded HunyuanVideo models. Models should be placed in the 'ckpts' directory, with subdirectories for 'hunyuan-video-t2v-720p', 'text_encoder', and 'text_encoder_2'. This structure is necessary for the training scripts to locate the model files correctly.

```file structure
ckpts
  ├──hunyuan-video-t2v-720p
  │  ├──transformers
  │  ├──vae
  ├──text_encoder
  ├──text_encoder_2
  ├──...

```

--------------------------------

### Video Generation with Compile Option

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

This demonstrates enabling PyTorch's compile feature for potentially faster inference. It requires triton and specific build tools on Windows. The `--compile_args` can be used to pass arguments to the compiler. Note that the first run with `--compile` is slow, but subsequent runs are faster.

```bash
python hv_generate_video.py --compile --compile_args "--disable-cuda" --fp8 --video_size 544 960 --video_length 5 --infer_steps 30 
    --prompt "A cat walks on the grass, realistic style."  --save_path path/to/save/dir --output_type both 
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt --attn_mode sdpa --split_attn
    --vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt 
    --vae_chunk_size 32 --vae_spatial_tile_sample_min_size 128 
    --text_encoder1 path/to/ckpts/text_encoder 
    --text_encoder2 path/to/ckpts/text_encoder_2 
    --seed 1234 --lora_multiplier 1.0 --lora_weight path/to/lora.safetensors
```

--------------------------------

### Start Training LoRA Network

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

Initiates the training process for a LoRA network using the accelerate library. Requires specifying task, DiT weights, dataset configuration, and various training parameters like learning rate and gradient checkpointing. The optimal values for timestep_sampling and discrete_flow_shift may require experimentation.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 wan_train_network.py \
    --task t2v-1.3B \
    --dit path/to/wan2.1_xxx_bf16.safetensors \
    --dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base \
    --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing \
    --max_data_loader_n_workers 2 --persistent_data_loader_workers \
    --network_module networks.lora_wan --network_dim 32 \
    --timestep_sampling shift --discrete_flow_shift 3.0 \
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \
    --output_dir path/to/output_dir --output_name name-of-lora
```

--------------------------------

### Pre-cache Latents using Python Script

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

This script pre-caches latents for the dataset. It requires a dataset configuration file and paths to the VAE model. Users can adjust chunk size, tiling, and batch size for VRAM optimization. Debugging options are available to visualize dataset images and captions.

```bash
python cache_latents.py --dataset_config path/to/toml --vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt --vae_chunk_size 32 --vae_tiling
```

```bash
uv run cache_latents.py --dataset_config path/to/toml --vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt --vae_chunk_size 32 --vae_tiling
```

--------------------------------

### Pre-cache Text Encoder Outputs using Python Script

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

This script pre-caches text encoder outputs, essential for training. It requires a dataset configuration and paths to the text encoder models. Batch size can be adjusted based on VRAM availability. An option for fp8 LLM mode is provided for systems with limited VRAM.

```bash
python cache_text_encoder_outputs.py --dataset_config path/to/toml  --text_encoder1 path/to/ckpts/text_encoder --text_encoder2 path/to/ckpts/text_encoder_2 --batch_size 16
```

```bash
uv run cache_text_encoder_outputs.py --dataset_config path/to/toml  --text_encoder1 path/to/ckpts/text_encoder --text_encoder2 path/to/ckpts/text_encoder_2 --batch_size 16
```

--------------------------------

### Prompt File Format for Sample Image Generation

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/sampling_during_training.md

This snippet illustrates the format of a prompt file used for generating sample images during training. Each line can contain a prompt with optional parameters for width, height, frames, seed, steps, and guidance scale. Lines starting with '#' are treated as comments.

```text
# prompt 1: for generating a cat video
A cat walks on the grass, realistic style. --w 640 --h 480 --f 25 --d 1 --s 20

# prompt 2: for generating a dog image
A dog runs on the beach, realistic style. --w 960 --h 544 --f 1 --d 2 --s 20
```

--------------------------------

### Configure Image Dataset with Metadata JSONL (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This TOML configuration sets up an image dataset using a JSONL file for metadata, which includes image paths and captions. The cache directory is required for this setup. General parameters like resolution and batch size are also defined.

```toml
# resolution, batch_size, num_repeats, enable_bucket, bucket_no_upscale should be set in either general or datasets
# caption_extension is not required for metadata jsonl file
# cache_directory is required for each dataset with metadata jsonl file

# general configurations
[general]
resolution = [960, 544]
batch_size = 1
enable_bucket = true
bucket_no_upscale = false

[[datasets]]
image_jsonl_file = "/path/to/metadata.jsonl"
cache_directory = "/path/to/cache_directory" # required for metadata jsonl file
num_repeats = 1 # optional, default is 1. Same as above.

# other datasets can be added here. each dataset can have different configurations
```

--------------------------------

### Generate Videos with hv_generate_video.py

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

This command generates videos using the hv_generate_video.py script. It includes options for fp8 inference, video size and length, prompt, save path, model paths (DiT, VAE, text encoders), attention modes, VAE chunking, and LoRA weights. The --fp8 flag enables fp8 mode for reduced memory consumption, potentially impacting quality. The --attn_mode parameter allows selection from 'flash', 'torch', 'sageattn', or 'xformers'.

```bash
python hv_generate_video.py --fp8 --video_size 544 960 --video_length 5 --infer_steps 30 
    --prompt "A cat walks on the grass, realistic style."  --save_path path/to/save/dir --output_type both 
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt --attn_mode sdpa --split_attn
    --vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt 
    --vae_chunk_size 32 --vae_spatial_tile_sample_min_size 128 
    --text_encoder1 path/to/ckpts/text_encoder 
    --text_encoder2 path/to/ckpts/text_encoder_2 
    --seed 1234 --lora_multiplier 1.0 --lora_weight path/to/lora.safetensors
```

```bash
uv run hv_generate_video.py --fp8 --video_size 544 960 --video_length 5 --infer_steps 30 
    --prompt "A cat walks on the grass, realistic style."  --save_path path/to/save/dir --output_type both 
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt --attn_mode sdpa --split_attn
    --vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt 
    --vae_chunk_size 32 --vae_spatial_tile_sample_min_size 128 
    --text_encoder1 path/to/ckpts/text_encoder 
    --text_encoder2 path/to/ckpts/text_encoder_2 
    --seed 1234 --lora_multiplier 1.0 --lora_weight path/to/lora.safetensors
```

--------------------------------

### SkyReels V1 I2V Inference Image Path

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

Specifies the input image path for image-to-video inference with the SkyReels V1 I2V model. The image will be resized according to the --video_size parameter.

```bash
--image_path path/to/image.jpg
```

--------------------------------

### Convert LoRA Model Format

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

Converts a LoRA model to a format compatible with ComfyUI (Diffusion-pipe) or other specified targets. Requires specifying input, output paths, and the target format.

```python
python convert_lora.py --input path/to/musubi_lora.safetensors --output path/to/another_format.safetensors --target other
```

```python
uv run convert_lora.py --input path/to/musubi_lora.safetensors --output path/to/another_format.safetensors --target other
```

--------------------------------

### Save Merged DiT Model after LoRA

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

This command saves the DiT model after merging with LoRA weights. The `--save_merged_model` option specifies the output path for the merged model. When this option is used, inference is not performed.

```bash
python hv_generate_video.py --save_merged_model path/to/merged_model.safetensors --fp8 --video_size 544 960 --video_length 5 --infer_steps 30 
    --prompt "A cat walks on the grass, realistic style."  --save_path path/to/save/dir --output_type both 
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt --attn_mode sdpa --split_attn
    --vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt 
    --vae_chunk_size 32 --vae_spatial_tile_sample_min_size 128 
    --text_encoder1 path/to/ckpts/text_encoder 
    --text_encoder2 path/to/ckpts/text_encoder_2 
    --seed 1234 --lora_multiplier 1.0 --lora_weight path/to/lora.safetensors
```

--------------------------------

### Merge LoRA Weights into Base Model

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.md

Merges pre-trained LoRA weights into a specified base model checkpoint. Users need to provide paths to the base model, the LoRA weights, and the desired output location for the merged model. The `--device` parameter controls the computation device (CPU or CUDA), and `--lora_multiplier` allows scaling the effect of the LoRA weights during merging. Note that Wan2.1 is not supported for this operation.

```bash
python merge_lora.py \
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt \
    --lora_weight path/to/lora.safetensors \
    --save_merged_model path/to/merged_model.safetensors \
    --device cpu \
    --lora_multiplier 1.0
```

```bash
uv run merge_lora.py \
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt \
    --lora_weight path/to/lora.safetensors \
    --save_merged_model path/to/merged_model.safetensors \
    --device cpu \
    --lora_multiplier 1.0
```

--------------------------------

### Training Options for Sampling

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

Specifies command-line arguments used during training to generate sample images. Includes paths to VAE and T5 models, prompt file location, and frequency of sampling during training. These options are consistent with inference or HunyuanVideo.

```bash
--vae path/to/wan_2.1_vae.safetensors 
--t5 path/to/models_t5_umt5-xxl-enc-bf16.pth 
--sample_prompts /path/to/prompt_file.txt 
--sample_every_n_epochs 1 --sample_every_n_steps 1000 -- sample_at_first
```

--------------------------------

### General Configuration Settings (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

Defines general configuration parameters that apply to all datasets unless overridden. Includes default resolution, caption extension, batch size, and bucketing settings.

```toml
[general]
resolution = [960, 544] # optional, [W, H], default is None. This is the default resolution for all datasets
caption_extension = ".txt" # optional, default is None. This is the default caption extension for all datasets
batch_size = 1 # optional, default is 1. This is the default batch size for all datasets
enable_bucket = true # optional, default is false. Enable bucketing for datasets
bucket_no_upscale = false # optional, default is false. Disable upscaling for bucketing. Ignored if enable_bucket is false
```

--------------------------------

### TOML Configuration for General and Dataset Settings

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This TOML configuration defines general parameters like resolution, batch size, and bucketing options, along with dataset-specific settings for video metadata, frame extraction, and caching.

```toml
# general configurations
[general]
resolution = [960, 544]
batch_size = 1
enable_bucket = true
bucket_no_upscale = false

[[datasets]]
video_jsonl_file = "/path/to/metadata.jsonl"
target_frames = [1, 25, 45]
frame_extraction = "head"
cache_directory = "/path/to/cache_directory_head"

# same metadata jsonl file can be used for multiple datasets
[[datasets]]
video_jsonl_file = "/path/to/metadata.jsonl"
target_frames = [1]
frame_stride = 10
cache_directory = "/path/to/cache_directory_stride"

# other datasets can be added here. each dataset can have different configurations

# general configurations
[general]
resolution = [960, 544] # optional, [W, H], default is None. This is the default resolution for all datasets
caption_extension = ".txt" # optional, default is None. This is the default caption extension for all datasets
batch_size = 1 # optional, default is 1. This is the default batch size for all datasets
num_repeats = 1 # optional, default is 1. Number of times to repeat the dataset. Useful to balance the multiple datasets with different sizes.
enable_bucket = true # optional, default is false. Enable bucketing for datasets
bucket_no_upscale = false # optional, default is false. Disable upscaling for bucketing. Ignored if enable_bucket is false
```

--------------------------------

### I2V Model Training with CLIP

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

An additional command-line argument to specify the CLIP model when training I2V (Image-to-Video) models. This is necessary for proper model training.

```bash
--clip path/to/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
```

--------------------------------

### SkyReels V1 T2V Inference Options with Bash

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Additional command-line options for performing Text-to-Video (T2V) inference with SkyReels V1 models. These include guidance scales, negative prompts, and options to split attention and unconditional parts of the model for VRAM management.

```bash
--guidance_scale 6.0 --embedded_cfg_scale 1.0 --negative_prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion" --split_uncond
```

--------------------------------

### Pre-cache Text Encoder Outputs with WAN Script

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

Creates a cache for text encoder outputs, similar to HunyuanVideo. This command requires the dataset configuration, T5 model path, and batch size. For systems with limited VRAM, the T5 model can be run in fp8 mode using the `--fp8_t5` flag.

```bash
python wan_cache_text_encoder_outputs.py --dataset_config path/to/toml  --t5 path/to/models_t5_umt5-xxl-enc-bf16.pth --batch_size 16 --fp8_t5
```

--------------------------------

### Pre-cache Text Encoder Outputs using Python

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

This script pre-caches the outputs from the Text Encoder. It requires the dataset configuration TOML file and paths to both text encoder models. The batch size can be adjusted based on VRAM availability. For systems with limited VRAM (around 16GB or less), the `--fp8_llm` flag can be used to run the LLM in fp8.

```bash
python cache_text_encoder_outputs.py --dataset_config path/to/toml  --text_encoder1 path/to/ckpts/text_encoder --text_encoder2 path/to/ckpts/text_encoder_2 --batch_size 16
```

--------------------------------

### Specify network_args in Command Line

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/advanced_config.md

Demonstrates how to specify detailed arguments for LoRA using the --network_args option directly on the command line. Arguments are passed as key=value pairs. This method allows for dynamic configuration during script execution.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 hv_train_network.py --dit \
    --network_module networks.lora --network_dim 32 \
    --network_args "key1=value1" "key2=value2" ...
```

```bash
--network_args "verbose=True" "key1=value1" "key2=value2" ...
```

--------------------------------

### Save and View Logs in TensorBoard Format

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/advanced_config.md

Allows saving logs in TensorBoard format using the `--logging_dir` option. A specified directory will be created, and logs will be saved within date-stamped subfolders. The `--log_prefix` option can prepend a string to these subfolders for easier identification. To view the logs, the TensorBoard command-line tool is used.

```shell
tensorboard --logdir=logs
```

--------------------------------

### Classifier Free Guidance Options in Prompt File

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/sampling_during_training.md

This snippet demonstrates options for classifier-free guidance within a prompt file. It includes specifying a negative prompt and the guidance scale, which are important for certain model types like SkyReels V1 and Wan2.1.

```text
--n negative prompt...
--l 6.0
```

--------------------------------

### Pre-cache Latents using Python

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

This command-line script pre-caches latents for the dataset. It requires the dataset configuration TOML file and the path to the VAE model. Options include batch size, VAE chunk size, tiling, and debugging modes. Insufficient VRAM can be managed by reducing VAE spatial tile sample size and batch size.

```bash
python cache_latents.py --dataset_config path/to/toml --vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt --vae_chunk_size 32 --vae_tiling
```

--------------------------------

### Include Specific Modules for LoRA Training

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/advanced_config.md

Illustrates how to use 'include_patterns' in conjunction with 'exclude_patterns' via --network_args to precisely define which modules should be targeted by LoRA. This allows for highly specific LoRA application, targeting modules based on complex regular expressions.

```bash
--network_args "exclude_patterns=[r'.*']" "include_patterns=[r'.*single_blocks\.\d{2}\.linear.*']"
```

--------------------------------

### I2V Inference using Python Script

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

Executes Image-to-Video (I2V) inference. This script is similar to T2V inference but requires additional arguments like `--clip` for specifying the CLIP model and `--image_path` for the initial frame image. It also supports various optimization and model selection parameters.

```bash
python wan_generate_video.py --fp8 --task i2v-14B --video_size 832 480 --video_length 81 --infer_steps 20 
--prompt "prompt for the video" --save_path path/to/save.mp4 --output_type both 
--dit path/to/wan2.1_i2v_480p_14B_bf16_etc.safetensors --vae path/to/wan_2.1_vae.safetensors 
--t5 path/to/models_t5_umt5-xxl-enc-bf16.pth --clip path/to/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth 
--attn_mode torch --image_path path/to/image.jpg
```

--------------------------------

### Configure Image Datasets with JSONL Metadata (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

Configures an image dataset using a JSONL file containing image file paths and their corresponding captions. Specifies the JSONL file path, resolution, and cache directory. Other parameters like batch size and bucketing can also be set.

```toml
[[datasets]]
image_jsonl_file = "/path/to/metadata.jsonl" # includes pairs of image files and captions
resolution = [960, 544] # required if general resolution is not set
cache_directory = "/path/to/cache_directory" # required for metadata jsonl file
# caption_extension is not required for metadata jsonl file
# batch_size, num_repeats, enable_bucket, bucket_no_upscale are also available for metadata jsonl file
```

--------------------------------

### Configure Image Dataset with Captions (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This TOML configuration is for an image dataset using separate text files for captions. It specifies general settings like resolution and batch size, and dataset-specific paths and repetition counts. The cache directory is recommended to be unique per dataset.

```toml
# resolution, caption_extension, batch_size, num_repeats, enable_bucket, bucket_no_upscale should be set in either general or datasets
# otherwise, the default values will be used for each item

# general configurations
[general]
resolution = [960, 544]
caption_extension = ".txt"
batch_size = 1
enable_bucket = true
bucket_no_upscale = false

[[datasets]]
image_directory = "/path/to/image_dir"
cache_directory = "/path/to/cache_directory"
num_repeats = 1 # optional, default is 1. Number of times to repeat the dataset. Useful to balance the multiple datasets with different sizes.

# other datasets can be added here. each dataset can have different configurations
```

--------------------------------

### Configure Image Datasets with Text Captions (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

Configures an image dataset using a directory of images and associated text files for captions. Specifies image directory, caption file extension, resolution, batch size, repeat count, bucketing settings, and cache directory.

```toml
[[datasets]]
image_directory = "/path/to/image_dir"
caption_extension = ".txt" # required for caption text files, if general caption extension is not set
resolution = [960, 544] # required if general resolution is not set
batch_size = 4 # optional, overwrite the default batch size
num_repeats = 1 # optional, overwrite the default num_repeats
enable_bucket = false # optional, overwrite the default bucketing setting
bucket_no_upscale = true # optional, overwrite the default bucketing setting
cache_directory = "/path/to/cache_directory" # optional, default is None to use the same directory as the image directory. NOTE: caching is always enabled
```

--------------------------------

### Configure Image Dataset with Lance (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This snippet illustrates the configuration for an image dataset using the Lance format. It specifies the path to the Lance dataset and the desired resolution for the images. This is useful for loading large image datasets efficiently. Outputs are processed image data from the Lance file.

```TOML
[["datasets"]]
image_lance_dataset = "/path/to/lance_dataset"
resolution = [960, 544]
```

--------------------------------

### Configure Video Dataset with Metadata JSONL (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This configuration defines a video dataset where metadata, including video file paths and captions, is stored in a JSONL file. It requires the path to the JSONL file and a cache directory for storing processed data. Target frames for extraction are also specified. Outputs are frames and captions derived from the metadata.

```TOML
[["datasets"]]
video_jsonl_file = "/path/to/metadata.jsonl"
target_frames = [1, 79]
cache_directory = "/path/to/cache_directory"
```

--------------------------------

### Configure Video Dataset with Captions (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This TOML configuration defines a video dataset using text files for captions. It includes parameters specific to video processing like target frames and frame extraction methods, alongside general settings. A unique cache directory is recommended.

```toml
# resolution, caption_extension, target_frames, frame_extraction, frame_stride, frame_sample, 
# batch_size, num_repeats, enable_bucket, bucket_no_upscale should be set in either general or datasets
# num_repeats is also available for video dataset, example is not shown here

# general configurations
[general]
resolution = [960, 544]
caption_extension = ".txt"
batch_size = 1
enable_bucket = true
bucket_no_upscale = false

[[datasets]]
video_directory = "/path/to/video_dir"
cache_directory = "/path/to/cache_directory" # recommended to set cache directory
target_frames = [1, 25, 45]
frame_extraction = "head"

# other datasets can be added here. each dataset can have different configurations
```

--------------------------------

### JSONL Metadata File Format

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This JSONL format is used for metadata files, where each line represents a video entry with its path and a corresponding caption.

```json
{"video_path": "/path/to/video1.mp4", "caption": "A caption for video1"}
{"video_path": "/path/to/video2.mp4", "caption": "A caption for video2"}
```

--------------------------------

### Configure Video Datasets with JSONL Metadata (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

Configures a video dataset using a JSONL file containing video file paths and their corresponding captions. Specifies the JSONL file path, target frames for extraction, and cache directory. Frame extraction parameters can also be configured.

```toml
[[datasets]]
video_jsonl_file = "/path/to/metadata.jsonl" # includes pairs of video files and captions

target_frames = [1, 79]

cache_directory = "/path/to/cache_directory" # required for metadata jsonl file
# frame_extraction, frame_stride, frame_sample are also available for metadata jsonl file
```

--------------------------------

### Configure Video Datasets with Text Captions (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

Configures a video dataset using a directory of video files and associated text files for captions. Specifies video directory, caption extension, resolution, target frames for extraction, frame extraction method, frame stride/sample, and cache directory.

```toml
[[datasets]]
video_directory = "/path/to/video_dir"
caption_extension = ".txt" # required for caption text files, if general caption extension is not set
resolution = [960, 544] # required if general resolution is not set

target_frames = [1, 25, 79] # required for video dataset. list of video lengths to extract frames. each element must be N*4+1 (N=0,1,2,...)

# NOTE: Please do not include 1 in target_frames if you are using the frame_extraction "chunk". It will make the all frames to be extracted.

frame_extraction = "head" # optional, "head" or "chunk", "slide", "uniform". Default is "head"
frame_stride = 1 # optional, default is 1, available for "slide" frame extraction
frame_sample = 4 # optional, default is 1 (same as "head"), available for "uniform" frame extraction
# batch_size, num_repeats, enable_bucket, bucket_no_upscale, cache_directory are also available for video dataset
```

--------------------------------

### Configure Video Dataset with Caption Text Files (TOML)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This snippet shows how to configure a video dataset that uses individual text files for captions. It specifies the video directory, caption file extension, target frame extraction, and frame extraction method. Dependencies include the video files and corresponding .txt caption files. Outputs are frames extracted from videos and associated captions.

```TOML
[["datasets"]]
video_directory = "/path/to/video_dir"
caption_extension = ".txt"
resolution = [960, 544]
target_frames = [1, 25, 79]
frame_extraction = "head"
frame_stride = 1
frame_sample = 4
```

--------------------------------

### Video Generation Inference with Bash

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Generates videos using a specified model. Supports various options for controlling video size, length, prompt, inference steps, and attention modes. FP8 and fast FP8 options can reduce memory usage at the cost of potential quality degradation. VAE chunk size and spatial tile sample min size can be adjusted for memory management. Supports LoRA weights and multipliers.

```bash
python hv_generate_video.py --fp8 --video_size 544 960 --video_length 5 --infer_steps 30 
    --prompt "A cat walks on the grass, realistic style."  --save_path path/to/save/dir --output_type both 
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt --attn_mode sdpa --split_attn
    --vae path/to/ckpts/hunyuan-video-t2v-720p/vae/pytorch_model.pt 
    --vae_chunk_size 32 --vae_spatial_tile_sample_min_size 128 
    --text_encoder1 path/to/ckpts/text_encoder 
    --text_encoder2 path/to/ckpts/text_encoder_2 
    --seed 1234 --lora_multiplier 1.0 --lora_weight path/to/lora.safetensors
```

--------------------------------

### Pre-cache Latents with WAN Script

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

Generates a cache for latents, essential for training. It requires a dataset configuration and the path to the VAE model. For I2V models, the CLIP model path must also be specified. VRAM can be reduced by using CPU for the VAE internal cache via the `--vae_cache_cpu` flag.

```bash
python wan_cache_latents.py --dataset_config path/to/toml --vae path/to/wan_2.1_vae.safetensors --clip path/to/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth --vae_cache_cpu
```

--------------------------------

### T2V Inference using Python Script

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

Executes Text-to-Video (T2V) inference. This script requires several arguments to specify the task, model paths, video parameters, and optimization settings. It supports options like fp8 for memory reduction and various attention modes.

```bash
python wan_generate_video.py --fp8 --task t2v-1.3B --video_size  832 480 --video_length 81 --infer_steps 20 
--prompt "prompt for the video" --save_path path/to/save.mp4 --output_type both 
--dit path/to/wan2.1_t2v_1.3B_bf16_etc.safetensors --vae path/to/wan_2.1_vae.safetensors 
--t5 path/to/models_t5_umt5-xxl-enc-bf16.pth 
--attn_mode torch
```

--------------------------------

### Image2Video Inference Option in Prompt File

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/sampling_during_training.md

This shows an additional option that can be used within a prompt file when training Image-to-Video (I2V) models. It specifies the path to an input image for the image2video inference process.

```text
--i path/to/image.png
```

--------------------------------

### Convert LoRA Format with Bash

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Converts LoRA weights between different formats, specifically for use with ComfyUI. Requires specifying input and output file paths. The `--target` argument determines the conversion direction; 'other' is used to convert to a format compatible with this repository, while 'default' converts from other formats to this repository's format.

```bash
python convert_lora.py --input path/to/musubi_lora.safetensors --output path/to/another_format.safetensors --target other
```

--------------------------------

### Specify network_args in Configuration File

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/advanced_config.md

Illustrates how to configure detailed LoRA arguments within a TOML configuration file. The 'network_args' parameter accepts a list of strings, where each string is a key=value pair. This is useful for persistent configurations.

```toml
network_args = ["key1=value1", "key2=value2", ...]
```

--------------------------------

### JSONL Metadata File Format (JSON)

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/dataset/dataset_config.md

This is the expected JSON format for metadata files used with video or image datasets. Each line in the JSONL file should be a JSON object containing at least 'image_path' (or 'video_path') and 'caption' keys.

```json
{"image_path": "/path/to/image1.jpg", "caption": "A caption for image1"}
{"image_path": "/path/to/image2.jpg", "caption": "A caption for image2"}
```

--------------------------------

### Merge LoRA Weights with Bash

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/README.ja.md

Merges LoRA weights into a base model. Requires specifying paths to the base model (DiT), LoRA weights, and the desired output path for the merged model. The device can be set to 'cpu' or 'cuda' for accelerated computation. LoRA multipliers can be specified for multiple LoRA weights.

```bash
python merge_lora.py \
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt \
    --lora_weight path/to/lora.safetensors \
    --save_merged_model path/to/merged_model.safetensors \
    --device cpu \
    --lora_multiplier 1.0
```

--------------------------------

### Save and View Logs in Wandb Format

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/advanced_config.md

Enables saving logs in wandb format via the `--log_with wandb` option. This feature supports integration with Weights & Biases for experiment tracking. The `--log_tracker_name` option is used to specify the project name within wandb.

--------------------------------

### Convert LoRA Weights

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/wan.md

A script used for converting LoRA weights after the training process is complete. This is similar to the process used in HunyuanVideo.

```python
convert_lora.py
```

--------------------------------

### Configure LoRA+ Learning Rate Ratio

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/advanced_config.md

Shows how to enable and configure LoRA+ using the --network_args option. LoRA+ speeds up training by increasing the learning rate for the UP side (LoRA-B). The 'loraplus_lr_ratio' parameter specifies the multiplier for this increased learning rate.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 hv_train_network.py --dit \
    --network_module networks.lora --network_dim 32 --network_args "loraplus_lr_ratio=4" ...
```

--------------------------------

### Exclude Modules from LoRA Training

Source: https://github.com/clcarwin/musubi-tuner-wan-lora-trainer/blob/main/docs/advanced_config.md

Demonstrates using 'exclude_patterns' within --network_args to prevent specific modules from being targeted by LoRA. This is useful for fine-tuning specific parts of a model. Patterns are regular expressions for module names and are applied as complete matches.

```bash
--network_args "exclude_patterns=[r'.*single_blocks.*']"
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.