### Install SageAttention

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/README.md

Installs the SageAttention package after cloning the repository and activating the virtual environment. Ensure the correct command prompt and environment variables are set.

```shell
python setup.py install
```

--------------------------------

### Kisekaeichi Training Prompt Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/framepack_1f.md

This prompt format is for Kisekaeichi-style training. It requires specifying both a reference image with alpha and the start image for control.

```bash
The girl wears a school uniform. --i path/to/start_with_alpha.png --ci path/to/ref_with_alpha.png --ci path/to/start_with_alpha.png --of no_post,no_2x,no_4x,target_index=5,control_index=0;10 --d 1111 --f 1 --s 10 --fs 7 --d 1234 --w 384 --h 576
```

--------------------------------

### Normal Single Frame Training Prompt Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/framepack_1f.md

Use this prompt format for standard single-frame training. Ensure `--i` points to the start image and `--ci` to the control image.

```bash
The girl wears a school uniform. --i path/to/start.png --ci path/to/start.png --of no_2x,no_4x,target_index=1,control_index=0 --d 1111 --f 1 --s 10 --fs 7 --d 1234 --w 384 --h 576
```

--------------------------------

### Normal One-Frame Training Prompt Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/wan_1f.md

Use this command for normal one-frame training. It specifies the start image, control image, and inference options.

```bash
The girl wears a school uniform. --i path/to/start.png --ci path/to/start.png --of target_index=1,control_index=0 --d 1111 --f 1 --s 10 --fs 7 --d 1234 --w 384 --h 576
```

--------------------------------

### Start Hunyuan Video Training

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/hunyuan_video.md

Use this command to start the training process for Hunyuan Video models. Ensure all paths and configurations are correctly set.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 src/musubi_tuner/hv_train_network.py 
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt 
    --dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base 
    --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing 
    --max_data_loader_n_workers 2 --persistent_data_loader_workers 
    --network_module networks.lora --network_dim 32 
    --timestep_sampling shift --discrete_flow_shift 7.0 
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42
    --output_dir path/to/output_dir --output_name name-of-lora
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/README.md

Install the required dependencies for the Musubi-Tuner project using pip. This command performs an editable installation.

```bash
pip install -e .
```

--------------------------------

### Get Training Help

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/wan.md

Run this command to view all available training options and their descriptions. Note that many options may be experimental or unverified.

```bash
python src/musubi_tuner/wan_train_network.py --help
```

--------------------------------

### Start Video Training with Accelerate

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/wan.md

Use this command to start training for tasks like text-to-video (t2v) or image-to-video (i2v). Adjust parameters like `--task`, `--dit`, and dataset configuration based on your specific model and data. Experiment with `timestep_sampling` and `discrete_flow_shift` for optimal results.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 src/musubi_tuner/wan_train_network.py \
    --task t2v-1.3B \
    --dit path/to/wan2.1_xxx_bf16.safetensors \
    --dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base \
    --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing \
    --max_data_loader_n_workers 2 --persistent_data_loader_workers \
    --network_module networks.lora_wan --network_dim 32 \
    --timestep_sampling shift --discrete_flow_shift 3.0 \
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \
    --output_dir path/to/output_dir --output_name name-of-lora
```

--------------------------------

### Prompt File Format Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/wan.md

Example of a prompt file format for batch generation. Each line is a prompt, and inline parameters can be specified using command-line style flags.

```text
A beautiful sunset over mountains --w 832 --h 480 --f 81 --d 42 --s 20
A busy city street at night --w 480 --h 832 --g 7.5 --n low quality, blurry
```

--------------------------------

### Example TOML Configuration File

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

A sample TOML file demonstrating how to specify training options. This file can contain any command-line arguments, organized for readability.

```toml
[optimizer]
learning_rate = 1e-5

[network]
type = "unet"

[dataset]
resolution = 512
```

--------------------------------

### Launch Training with Configuration File

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

Execute the training script using a TOML configuration file instead of command-line arguments. This simplifies managing complex training setups.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 src/musubi_tuner/hv_train_network.py --config_file config.toml
```

--------------------------------

### Install Musubi Tuner in Development Mode

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/CONTRIBUTING.md

Install the Musubi Tuner package in editable mode, allowing changes in the source code to be reflected immediately.

```shell
pip install -e .
```

--------------------------------

### Example Project Directory Structure

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/src/musubi_tuner/gui/gui.md

Illustrates the expected file and folder structure for a Musubi-Tuner project, including the 'training' subfolder for images and their corresponding caption files.

```text
my_lora_project/
  training/
    image001.jpg
    image001.txt
    image002.png
    image002.txt
```

--------------------------------

### Intermediate Frame One-Frame Training Prompt Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/wan_1f.md

Use this command for intermediate frame one-frame training. It includes start and end images, control images, and specific inference parameters.

```bash
The girl wears a school uniform. --i path/to/start.png --ei path/to/end.png --ci path/to/start.png --ci path/to/end.png --of target_index=1,control_index=0;2 --d 1111 --f 1 --s 10 --fs 7 --d 1234 --w 384 --h 576
```

--------------------------------

### Specify kisekaeichi Method Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/framepack_1f.md

Example command for specifying the kisekaeichi method with control images and masks. Ensure control_index is set appropriately relative to latent_window_size and is larger than target_index.

```bash
--video_sections 1 --output_type latent_images --image_path start_image.png --control_image_path start_image.png clean_latent_post_image.png \
--one_frame_inference target_index=1,control_index=0;10,no_post,no_2x,no_4x --control_image_mask_path ctrl_mask1.png ctrl_mask2.png
```

--------------------------------

### Install uv Package Manager (Windows)

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/README.md

Install the uv package manager on Windows using a PowerShell command. Follow the on-screen instructions to add the uv path, or reboot your system.

```powershell
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
```

--------------------------------

### Video Dataset Configuration Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/dataset_config.md

This TOML configuration demonstrates how to set up video datasets, including common and video-specific parameters. Ensure `cache_directory` is set for metadata JSONL files.

```toml
# Common parameters (resolution, caption_extension, batch_size, num_repeats, enable_bucket, bucket_no_upscale) 
# can be set in either general or datasets sections
# Video-specific parameters (target_frames, frame_extraction, frame_stride, frame_sample, max_frames, source_fps)
# must be set in each datasets section

# caption_extension is not required for metadata jsonl file
# cache_directory is required for each dataset with metadata jsonl file

# general configurations
[general]
resolution = [960, 544]
batch_size = 1
enable_bucket = true
bucket_no_upscale = false

[[datasets]]
video_jsonl_file = "/path/to/metadata.jsonl"
target_frames = [1, 25, 45]
frame_extraction = "head"
cache_directory = "/path/to/cache_directory_head"
source_fps = 30.0 # optional, source fps for videos in the jsonl file
# same metadata jsonl file can be used for multiple datasets
[[datasets]]
video_jsonl_file = "/path/to/metadata.jsonl"
target_frames = [1]
frame_stride = 10
cache_directory = "/path/to/cache_directory_stride"

# other datasets can be added here. each dataset can have different configurations

```

--------------------------------

### Install Development Dependencies

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/CONTRIBUTING.md

Install additional dependencies required for development, including linters and formatters.

```shell
pip install --group dev
```

--------------------------------

### Install uv Package Manager (Linux/MacOS)

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/README.md

Install the uv package manager on Linux or MacOS using a curl command. Follow the on-screen instructions to add the uv path.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Start Hunyuan Video Training with UV

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/hunyuan_video.md

This command initiates Hunyuan Video model training using uv, which may offer performance benefits. Ensure all paths and configurations are correctly set.

```bash
uv run --extra cu124 accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 src/musubi_tuner/hv_train_network.py 
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt 
    --dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base 
    --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing 
    --max_data_loader_n_workers 2 --persistent_data_loader_workers 
    --network_module networks.lora --network_dim 32 
    --timestep_sampling shift --discrete_flow_shift 7.0 
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42
    --output_dir path/to/output_dir --output_name name-of-lora
```

--------------------------------

### Training with Sampling Command Line Options

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/wan.md

Example command line arguments for training with sampling enabled. These options are similar to those used for image generation and HunyuanVideo. Refer to the sampling during training documentation for detailed explanations.

```bash
--vae path/to/wan_vae.safetensors \
--t5 path/to/models_t5_umt5-xxl-enc-bf16.pth \
--sample_prompts /path/to/prompt_file.txt \
--sample_every_n_epochs 1 --sample_every_n_steps 1000 --sample_at_first
```

--------------------------------

### Command-line Override Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

This command launches the training script using a configuration file but overrides the learning rate specified in the file.

```bash
accelerate launch --mixed_precision bf16 src/musubi_tuner/hv_train_network.py --config_file config --learning_rate 2e-4
```

--------------------------------

### TOML Configuration File Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

This TOML file defines various training parameters for Musubi Tuner. All options can be specified at the top level or within sections, and the section structure is ignored during parsing.

```toml
dit = "/path/to/dit"
dataset_config = "/path/to/dataset.toml"
network_module = "networks.lora"
network_dim = 32
network_alpha = 16

[optimizer]
optimizer_type = "AdamW"
learning_rate = 1e-4

[training]
max_train_epochs = 10
save_every_n_epochs = 2
mixed_precision = "bf16"

[output]
output_dir = "/path/to/output"
output_name = "my_lora"
logging_dir = "./logs"
```

--------------------------------

### Install Optional Dependencies

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/README.md

Install optional dependencies for additional features like dataset verification, timestep visualization, logging, and interactive prompt editing. Prompt-toolkit is especially useful for interactive prompt editing in Linux environments.

```bash
pip install ascii-magic matplotlib tensorboard prompt-toolkit
```

--------------------------------

### Check Git Version

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/src/musubi_tuner/gui/gui.ja.md

Verify Git installation by checking its version in the command prompt. Essential for cloning repositories.

```bash
git --version
```

--------------------------------

### Specify 1f-mc (Multi-Control) Method Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/framepack_1f.md

Example command for the 1f-mc (Multi-Control) method, specifying multiple control images and their corresponding clean latent indices. The target_index determines the generated image's characteristics.

```bash
--video_sections 1 --output_type latent_images --image_path start_image.png --control_image_path start_image.png 2nd_image.png \
--one_frame_inference target_index=9,control_index=0;1,no_2x,no_4x
```

--------------------------------

### Launch Musubi Tuner GUI (CUDA 12.4)

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/src/musubi_tuner/gui/gui.ja.md

Starts the Musubi Tuner GUI using uv, specifying CUDA 12.4. This is the recommended stable version for launching the GUI.

```bash
uv run --extra cu124 --extra gui python src/musubi_tuner/gui/gui.py
```

--------------------------------

### Launch Musubi Tuner GUI (CUDA 12.8)

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/src/musubi_tuner/gui/gui.ja.md

Starts the Musubi Tuner GUI using uv, specifying CUDA 12.8. This version is intended for newer GPUs.

```bash
uv run --extra cu128 --extra gui python src/musubi_tuner/gui/gui.py
```

--------------------------------

### Configuring LoRA+ Learning Rate Ratio

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

This command-line example shows how to configure LoRA+ by specifying the learning rate multiplier for the UP side (LoRA-B) using the 'loraplus_lr_ratio' argument within --network_args.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 src/musubi_tuner/hv_train_network.py --dit ... 
    --network_module networks.lora --network_dim 32 --network_args "loraplus_lr_ratio=4" ...
```

--------------------------------

### Basic PyTorch Dynamo Training Optimization

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

Example of enabling PyTorch Dynamo with the INDUCTOR backend and default mode for training optimization.

```bash
python src/musubi_tuner/hv_train_network.py --dynamo_backend INDUCTOR --dynamo_mode default
```

--------------------------------

### Start Hunyuan Video Training

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/hunyuan_video.md

This command initiates the training process for the Hunyuan Video model. Adjust `discrete_flow_shift` to around 3.0 if details become blurry. Refer to the help command for other options, though many are unverified.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 src/musubi_tuner/hv_train_network.py \
    --dit path/to/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt \
    --dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base \
    --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing \
    --max_data_loader_n_workers 2 --persistent_data_loader_workers \
    --network_module networks.lora --network_dim 32 \
    --timestep_sampling shift --discrete_flow_shift 7.0 \
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \
    --output_dir path/to/output_dir --output_name name-of-lora
```

--------------------------------

### Specify Schedule Free Optimizer

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

Installs the schedulefree library and specifies a schedule-free optimizer type. Use '--optimizer_args' to pass specific arguments to the optimizer.

```bash
pip install schedulefree
--optimizer_type schedulefree.AdamWScheduleFree --optimizer_args "weight_decay=0.01" "betas=(0.9,0.95)"
```

--------------------------------

### Section-Specific Prompts Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/framepack.md

Use `--prompt` with a specific format to provide different prompts for different video sections. Prompts are separated by `;;;`, and each section prompt is prefixed with its section index.

```bash
python src/musubi_tuner/fpack_generate_video.py \
 --prompt "0:A dog runs towards a thrown ball, catches it, and runs back;;;1:The dog catches the ball and then runs back towards the viewer;;;2:The dog runs back towards the viewer holding the ball" \
 --image_path "0:./img_start_running.png;;;1:./img_catching.png;;;2:./img_running_back.png" \
 --end_image_path ./img_returned.png \
 --save_path ./output \
 # ... other arguments
```

--------------------------------

### Install PyTorch with CUDA Support

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/CONTRIBUTING.md

Install PyTorch and torchvision, specifying the CUDA version. Adjust 'cu128' to match your installed CUDA version.

```shell
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
```

--------------------------------

### Install Optional Dependency for DINOv3 Loss

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

Install the 'hidream_o1' extra to enable the DINOv3 auxiliary perceptual loss. This is a prerequisite for using the DINOv3 loss features.

```bash
uv pip install ".[hidream_o1]"
```

--------------------------------

### Install Dependencies with uv

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/CONTRIBUTING.md

Install project dependencies using the 'uv' package manager, specifying the CUDA version. Replace 'cu128' with your CUDA version if different.

```shell
uv sync --extra cu128
```

--------------------------------

### Load LoHa/LoKr Weights with LyCORIS

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/loha_lokr.md

For HunyuanVideo and FLUX.1 Kontext, the `--lycoris` option is required. Ensure the LyCORIS library is installed via `pip install lycoris-lora`.

```bash
pip install lycoris-lora
```

```bash
python src/musubi_tuner/hv_generate_video.py ... --lora_weight path/to/loha_or_lokr.safetensors --lycoris
```

--------------------------------

### Recommended Training Settings

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/torch_compile.md

Use these settings for training to balance performance and compile times. Ensure CUDA and cuDNN are enabled for optimal results.

```bash
--compile \
--compile_mode default \
--compile_cache_size_limit 32 \
--cuda_allow_tf32 \
--cuda_cudnn_benchmark
```

--------------------------------

### Install PyTorch and Torchvision

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/README.md

Install PyTorch and torchvision matching your CUDA version. PyTorch 2.5.1 or later is required. This command uses a specific index URL for CUDA 12.4.

```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
```

--------------------------------

### Full Generation Command with kisekaeichi

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/framepack_1f.md

A comprehensive command-line example for generating video frames using the kisekaeichi method, including model paths and inference parameters. Adjust --fp8_scaled and --blocks_to_swap based on VRAM capacity.

```bash
python fpack_generate_video.py --video_size 832 480 --video_sections 1 --infer_steps 25 \
    --prompt "The girl in a school blazer in a classroom." --save_path path/to/output --output_type latent_images \
    --dit path/to/dit --vae path/to/vae --text_encoder1 path/to/text_encoder1 --text_encoder2 path/to/text_encoder2 \
    --image_encoder path/to/image_encoder --attn_mode sdpa --vae_spatial_tile_sample_min_size 128 --vae_chunk_size 32 \
    --image_path path/to/kisekaeichi_start.png --control_image_path path/to/kisekaeichi_start.png path/to/kisekaeichi_ref.png \
    --one_frame_inference target_index=1,control_index=0;10,no_2x,no_4x,no_post \
    --control_image_mask_path path/to/kisekaeichi_start_mask.png path/to/kisekaeichi_ref_mask.png --seed 1234
```

--------------------------------

### Run Full Finetuning with zimage_train.py

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/zimage.md

Use this script for full model finetuning. It requires specifying paths to various model components and dataset configuration. Memory-saving options like --sdpa, --mixed_precision bf16, and --gradient_checkpointing are recommended due to high VRAM requirements.

```bash
accelerate launch --num_cpu_threads_per_process 1 src/musubi_tuner/zimage_train.py \
    --dit path/to/dit_model \
    --vae path/to/vae_model \
    --text_encoder path/to/text_encoder \
    --dataset_config path/to/toml \
    --sdpa --mixed_precision bf16 --gradient_checkpointing \
    --optimizer_type adafactor --learning_rate 1e-6 --fused_backward_pass \
    --optimizer_args "relative_step=False" "scale_parameter=False" "warmup_init=False" \
    --max_grad_norm 0 --lr_scheduler constant_with_warmup --lr_warmup_steps 10 \
    --max_data_loader_n_workers 2 --persistent_data_loader_workers \
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \
    --output_dir path/to/output_dir --output_name name-of-model
```

--------------------------------

### Advanced Image Captioning with fp8, Custom Prompt, and Max Size

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/tools.md

This example demonstrates advanced configuration options, including using fp8 precision for reduced memory usage, setting a custom prompt with a newline character, and specifying a maximum image size.

```bash
python src/musubi_tuner/caption_images_by_qwen_vl.py \
  --image_dir /path/to/images \
  --model_path /path/to/qwen_model.safetensors \
  --output_file /path/to/captions.jsonl \
  --fp8_vl \
  --max_size 1024 \
  --prompt "A detailed and descriptive caption for this image is:\n"
```

--------------------------------

### FLUX.2 Dataset Configuration Example

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/dataset_config.md

Example TOML configuration for FLUX.2 dataset, including options to disable control image resizing and specify control image resolution. Follow official inference settings by specifying [2024, 2024] for single control images or [1024, 1024] for multiple.

```toml
[[datasets]]
no_resize_control = false # オプション、デフォルトはfalse。制御画像のリサイズを無効にします
control_resolution = [1024, 1024] # オプション、デフォルトはNone。制御画像の解像度を指定します
```

--------------------------------

### Clone SageAttention Repository

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/README.md

Clones the SageAttention repository from GitHub. This is a prerequisite for installing SageAttention on Windows.

```shell
git clone https://github.com/thu-ml/SageAttention.git
```

--------------------------------

### Start Network Training

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/kandinsky5.md

Initiate the training process for Kandinsky 5 network. This command includes numerous options for task configuration, model paths, precision, optimization, and memory management. Refer to `--help` for a full list of options.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 \
    kandinsky5_train_network.py \
    --mixed_precision bf16 \
    --dataset_config path/to/dataset.toml \
    --task k5-pro-t2v-5s-sd \
    --dit path/to/kandinsky5pro_t2v_pretrain_5s.safetensors \
    --text_encoder_qwen Qwen/Qwen2.5-VL-7B-Instruct \
    --text_encoder_clip openai/clip-vit-large-patch14 \
    --vae path/to/vae/diffusion_pytorch_model.safetensors \
    --fp8_base \
    --sdpa \
    --gradient_checkpointing \
    --max_data_loader_n_workers 1 \
    --persistent_data_loader_workers \
    --learning_rate 1e-4 \
    --optimizer_type AdamW8Bit \
    --optimizer_args "weight_decay=0.001" "betas=(0.9,0.95)" \
    --max_grad_norm 1.0 \
    --lr_scheduler constant_with_warmup \
    --lr_warmup_steps 100 \
    --network_module networks.lora_kandinsky \
    --network_dim 32 \
    --network_alpha 32 \
    --timestep_sampling shift \
    --discrete_flow_shift 5.0 \
    --output_dir path/to/output \
    --output_name k5_lora \
    --save_every_n_epochs 1 \
    --max_train_epochs 50 \
    --scheduler_scale 10.0
```

--------------------------------

### Check uv Version

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/src/musubi_tuner/gui/gui.ja.md

Verify the installation of the uv package manager by checking its version in a new command prompt.

```bash
uv --version
```

--------------------------------

### Full Finetuning with Qwen Image Train Script

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/qwen_image.md

Use this script for full model finetuning. It requires specifying paths to the DIT, VAE, and text encoder models, along with dataset configuration. Memory-saving options like `--sdpa`, `--mixed_precision bf16`, and `--gradient_checkpointing` are recommended, especially for limited VRAM. The Adafactor optimizer with `--fused_backward_pass` is suggested for finetuning.

```bash
accelerate launch --num_cpu_threads_per_process 1 src/musubi_tuner/qwen_image_train.py \
    --dit path/to/dit_model \
    --vae path/to/vae_model \
    --text_encoder path/to/text_encoder \
    --model_version original \
    --dataset_config path/to/toml \
    --sdpa --mixed_precision bf16 --gradient_checkpointing \
    --optimizer_type adafactor --learning_rate 1e-6 --fused_backward_pass \
    --optimizer_args "relative_step=False" "scale_parameter=False" "warmup_init=False" \
    --max_grad_norm 0 --lr_scheduler constant_with_warmup --lr_warmup_steps 10 \
    --max_data_loader_n_workers 2 --persistent_data_loader_workers \
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \
    --output_dir path/to/output_dir --output_name name-of-model
```

--------------------------------

### Check Python Version

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/src/musubi_tuner/gui/gui.ja.md

Verify Python installation by checking its version in the command prompt. Requires Python to be added to PATH.

```bash
python --version
```

--------------------------------

### Clone Musubi Tuner Repository

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/src/musubi_tuner/gui/gui.ja.md

Downloads the Musubi Tuner source code from GitHub. Navigate to your desired installation directory before running this command.

```bash
git clone https://github.com/kohya-ss/musubi-tuner.git
```

--------------------------------

### Create Musubi Tuner GUI Launch Batch File

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/src/musubi_tuner/gui/gui.ja.md

Creates a batch file to easily launch the Musubi Tuner GUI with CUDA 12.4. Place this file in the musubi-tuner directory.

```batch
@echo off
cd /d "%~dp0"
uv run --extra cu124 --extra gui python src/musubi_tuner/gui/gui.py
pause
```

--------------------------------

### Aggressive PyTorch Dynamo Training Optimization

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

Example of using PyTorch Dynamo with aggressive optimization settings, including max-autotune mode and fullgraph enabled.

```bash
python src/musubi_tuner/hv_train_network.py --dynamo_backend INDUCTOR --dynamo_mode max-autotune --dynamo_fullgraph
```

--------------------------------

### Enable wandb logging

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/advanced_config.md

Use the --log_with wandb option to save logs in wandb format. Specify the project name with --log_tracker_name.

```bash
--log_with wandb --log_tracker_name=<your_project_name>
```

--------------------------------

### Image-to-Video (I2V) Training Configuration

Source: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/hunyuan_video_1_5.md

Use this configuration for Image-to-Video training. Requires specifying `--task i2v` and `--image_encoder`. Ensure `--vae`, `--text_encoder`, and `--byt5` are also provided.

```bash
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 src/musubi_tuner/hv_1_5_train_network.py \
    --dit path/to/dit_model \
    --vae path/to/vae_model \
    --text_encoder path/to/text_encoder \
    --byt5 path/to/byt5 \
    --image_encoder path/to/image_encoder \
    --dataset_config path/to/toml \
    --task i2v \
    --sdpa --mixed_precision bf16 \
    --timestep_sampling shift --weighting_scheme none --discrete_flow_shift 2.0 \
    --optimizer_type adamw8bit --learning_rate 1e-4 --gradient_checkpointing \
    --max_data_loader_n_workers 2 --persistent_data_loader_workers \
    --network_module networks.lora_hv_1_5 --network_dim 32 \
    --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \
    --output_dir path/to/output_dir --output_name name-of-lora
```