### Launch Local Gradio Interface

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Shell commands to start the Gradio web interface for Wan2.1, supporting different model configurations for 480P and 720P resolutions.

```bash
cd gradio
DASH_API_KEY=your_key python i2v_14B_singleGPU.py --prompt_extend_method 'dashscope' --ckpt_dir_480p ./Wan2.1-I2V-14B-480P --ckpt_dir_720p ./Wan2.1-I2V-14B-720P
```

--------------------------------

### Run Wan2.1 Text-to-Video with Diffusers

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Inference for Wan2.1-T2V using the Diffusers library. This example demonstrates loading the model, setting up the scheduler, and generating a video from a text prompt. It requires PyTorch and Diffusers. The output is saved as an MP4 file.

```python
import torch
from diffusers.utils import export_to_video
from diffusers import AutoencoderKLWan, WanPipeline
from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler

# Available models: Wan-AI/Wan2.1-T2V-14B-Diffusers, Wan-AI/Wan2.1-T2V-1.3B-Diffusers
model_id = "Wan-AI/Wan2.1-T2V-14B-Diffusers"
vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
flow_shift = 5.0 # 5.0 for 720P, 3.0 for 480P
scheduler = UniPCMultistepScheduler(prediction_type='flow_prediction', use_flow_sigmas=True, num_train_timesteps=1000, flow_shift=flow_shift)
pipe = WanPipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16)
pipe.scheduler = scheduler
pipe.to("cuda")

prompt = "A cat and a dog baking a cake together in a kitchen. The cat is carefully measuring flour, while the dog is stirring the batter with a wooden spoon. The kitchen is cozy, with sunlight streaming through the window."
negative_prompt = "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"

output = pipe(
     prompt=prompt,
     negative_prompt=negative_prompt,
     height=720,
     width=1280,
     num_frames=81,
     guidance_scale=5.0,
    ).frames[0]
export_to_video(output, "output.mp4", fps=16)
```

--------------------------------

### Run Image-to-Video Generation (Multi-GPU)

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Performs Image-to-Video generation using multiple GPUs with FSDP and xDiT USP. This command requires installing the 'xfuser' package and utilizes distributed training arguments like '--nproc_per_node', '--dit_fsdp', '--t5_fsdp', and '--ulysses_size'.

```shell
pip install "xfuser>=0.4.1"
torchrun --nproc_per_node=8 generate.py --task i2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-I2V-14B-720P --image examples/i2v_input.JPG --dit_fsdp --t5_fsdp --ulysses_size 8 --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside."
```

--------------------------------

### Image-to-Video Generation with WanI2V

Source: https://context7.com/wan-video/wan2.1/llms.txt

Creates videos from a single input image guided by a text prompt using the WanI2V class. Utilizes CLIP for visual encoding and maintains input image aspect ratio. Suitable for dynamic content creation from static images.

```python
import torch
from PIL import Image
import wan
from wan.configs import WAN_CONFIGS, MAX_AREA_CONFIGS
from wan.utils.utils import cache_video

# Initialize the Image-to-Video model
cfg = WAN_CONFIGS['i2v-14B']
wan_i2v = wan.WanI2V(
    config=cfg,
    checkpoint_dir='./Wan2.1-I2V-14B-720P',
    device_id=0,
    rank=0,
    t5_fsdp=False,
    dit_fsdp=False,
    use_usp=False,
    t5_cpu=False,
)

# Load input image
img = Image.open('input_image.jpg').convert("RGB")

# Generate video from image
prompt = "A white cat wearing sunglasses sits on a surfboard, relaxed expression, sea breeze blowing."
video = wan_i2v.generate(
    input_prompt=prompt,
    img=img,
    max_area=720 * 1280,        # Maximum pixel area (maintains aspect ratio)
    frame_num=81,               # Number of frames
    shift=5.0,                  # Use 3.0 for 480P resolution
    sample_solver='unipc',
    sampling_steps=40,          # I2V uses 40 steps by default
    guide_scale=5.0,
    n_prompt="",
    seed=42,
    offload_model=True
)

# Save the generated video
cache_video(
    tensor=video[None],
    save_file="i2v_output.mp4",
    fps=16,
    nrow=1,
    normalize=True,
    value_range=(-1, 1)
)
```

--------------------------------

### First-Last-Frame-to-Video Generation with WanFLF2V

Source: https://context7.com/wan-video/wan2.1/llms.txt

Generates videos by interpolating between a first and last frame image, guided by a text prompt, using the WanFLF2V class. Useful for creating smooth transitions between keyframes. Handles frame resizing and cropping if necessary.

```python
import torch
from PIL import Image
import wan
from wan.configs import WAN_CONFIGS, MAX_AREA_CONFIGS
from wan.utils.utils import cache_video

# Initialize the First-Last-Frame-to-Video model
cfg = WAN_CONFIGS['flf2v-14B']
wan_flf2v = wan.WanFLF2V(
    config=cfg,
    checkpoint_dir='./Wan2.1-FLF2V-14B-720P',
    device_id=0,
    rank=0,
    t5_fsdp=False,
    dit_fsdp=False,
    use_usp=False,
    t5_cpu=False,
)
```

--------------------------------

### Expand Prompts using DashScope API

Source: https://context7.com/wan-video/wan2.1/llms.txt

This snippet shows how to use the `DashScopePromptExpander` to enhance simple text prompts into detailed descriptions for better video generation. It supports both text-only and vision-language models and requires setting up the DashScope API key. The example includes expanding a text prompt and a prompt with an image context.

```python
import os
from wan.utils.prompt_extend import DashScopePromptExpander
from PIL import Image

# Set API key (or use environment variable DASH_API_KEY)
os.environ['DASH_API_KEY'] = 'your_dashscope_api_key'

# Initialize the prompt expander for text-only
prompt_expander = DashScopePromptExpander(
    model_name='qwen-plus',     # or 'qwen-vl-max' for VL tasks
    is_vl=False,                # True for image-to-video tasks
    max_image_size=512 * 512,
    retry_times=4
)

# Expand a simple prompt
simple_prompt = "A cat sitting on a beach"
result = prompt_expander(
    prompt=simple_prompt,
    tar_lang='en',              # 'zh' for Chinese, 'en' for English
    seed=42
)

if result.status:
    expanded_prompt = result.prompt
    print(f"Expanded: {expanded_prompt}")
    # Output: "Summer beach vacation style, a fluffy white cat sits relaxed on
    # golden sand. The feline gazes at the camera with content expression..."
else:
    print(f"Expansion failed: {result.message}")

# Expand prompt with image context (for I2V tasks)

prompt_expander_vl = DashScopePromptExpander(
    model_name='qwen-vl-max',
    is_vl=True
)

image = Image.open('input.jpg')
result = prompt_expander_vl(
    prompt="A cat enjoying the beach",
    tar_lang='zh',
    image=image,
    seed=42
)
```

--------------------------------

### WanFLF2V - First-Last-Frame-to-Video Generation

Source: https://context7.com/wan-video/wan2.1/llms.txt

Generates video by interpolating between a first and last frame, guided by a text prompt, using the WanFLF2V class. Useful for smooth transitions.

```APIDOC
## WanFLF2V - First-Last-Frame-to-Video Generation

### Description
Generates video by interpolating between a first and last frame, guided by a text prompt, using the WanFLF2V class. Useful for smooth transitions.

### Method
N/A (Python Class)

### Endpoint
N/A (Python Class)

### Parameters
#### Initialization Parameters
- **config** (dict) - Required - Configuration for the model (e.g., WAN_CONFIGS['flf2v-14B']).
- **checkpoint_dir** (str) - Required - Path to the model checkpoints.
- **device_id** (int) - Optional - GPU device ID to use (default: 0).
- **rank** (int) - Optional - Rank for distributed training (default: 0).
- **t5_fsdp** (bool) - Optional - Whether to use FSDP for T5 model.
- **dit_fsdp** (bool) - Optional - Whether to use FSDP for DiT model.
- **use_usp** (bool) - Optional - Whether to use USP.
- **t5_cpu** (bool) - Optional - Whether to place T5 model on CPU to save VRAM.

#### Generate Method Parameters
- **input_prompt** (str) - Required - The text prompt guiding the video generation.
- **first_frame** (PIL.Image.Image) - Required - The first frame image.
- **last_frame** (PIL.Image.Image) - Required - The last frame image.
- **frame_num** (int) - Optional - Number of frames to generate (default: 81).
- **shift** (float) - Optional - Noise schedule shift (use 3.0 for 480P resolution) (default: 5.0).
- **sample_solver** (str) - Optional - Sampling solver ('unipc' or 'dpm++') (default: 'unipc').
- **sampling_steps** (int) - Optional - Number of diffusion steps (default: 40).
- **guide_scale** (float) - Optional - Classifier-free guidance scale (default: 5.0).
- **n_prompt** (str) - Optional - Negative prompt.
- **seed** (int) - Optional - Random seed (-1 for random) (default: 42).
- **offload_model** (bool) - Optional - Offload model to CPU to save VRAM (default: True).

### Request Example
```python
import torch
from PIL import Image
import wan
from wan.configs import WAN_CONFIGS, MAX_AREA_CONFIGS
from wan.utils.utils import cache_video

# Initialize the First-Last-Frame-to-Video model
cfg = WAN_CONFIGS['flf2v-14B']
wan_flf2v = wan.WanFLF2V(
    config=cfg,
    checkpoint_dir='./Wan2.1-FLF2V-14B-720P',
    device_id=0,
    rank=0,
    t5_fsdp=False,
    dit_fsdp=False,
    use_usp=False,
    t5_cpu=False,
)

# Load first and last frames
first_frame = Image.open('first_frame.jpg').convert("RGB")
last_frame = Image.open('last_frame.jpg').convert("RGB")

# Generate video from first and last frames
prompt = "Smooth transition between two scenes."
video = wan_flf2v.generate(
    input_prompt=prompt,
    first_frame=first_frame,
    last_frame=last_frame,
    frame_num=81,
    shift=5.0,
    sample_solver='unipc',
    sampling_steps=40,
    guide_scale=5.0,
    n_prompt="",
    seed=42,
    offload_model=True
)

# Save the generated video
cache_video(
    tensor=video[None],
    save_file="flf2v_output.mp4",
    fps=16,
    nrow=1,
    normalize=True,
    value_range=(-1, 1)
)
```

### Response
#### Success Response (200)
- **video** (torch.Tensor) - The generated video tensor.

#### Response Example
```python
# The 'video' variable will contain the generated video tensor.
# Example of saving the video is shown in the Request Example.
```
```

--------------------------------

### Run Text-to-Image Generation

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Generates images using the Wan2.1 unified model. Supports standard generation and prompt extension modes across single and multi-GPU setups.

```bash
python generate.py --task t2i-14B --size 1024*1024 --ckpt_dir ./Wan2.1-T2V-14B --prompt '一个朴素端庄的美人'
```

```bash
torchrun --nproc_per_node=8 generate.py --dit_fsdp --t5_fsdp --ulysses_size 8 --base_seed 0 --frame_num 1 --task t2i-14B --size 1024*1024 --prompt '一个朴素端庄的美人' --ckpt_dir ./Wan2.1-T2V-14B
```

```bash
python generate.py --task t2i-14B --size 1024*1024 --ckpt_dir ./Wan2.1-T2V-14B --prompt '一个朴素端庄的美人' --use_prompt_extend
```

--------------------------------

### WanI2V - Image-to-Video Generation

Source: https://context7.com/wan-video/wan2.1/llms.txt

Generates video content from a single input image guided by a text prompt using the WanI2V class. Maintains aspect ratio and supports various resolutions.

```APIDOC
## WanI2V - Image-to-Video Generation

### Description
Generates video content from a single input image guided by a text prompt using the WanI2V class. Maintains aspect ratio and supports various resolutions.

### Method
N/A (Python Class)

### Endpoint
N/A (Python Class)

### Parameters
#### Initialization Parameters
- **config** (dict) - Required - Configuration for the model (e.g., WAN_CONFIGS['i2v-14B']).
- **checkpoint_dir** (str) - Required - Path to the model checkpoints.
- **device_id** (int) - Optional - GPU device ID to use (default: 0).
- **rank** (int) - Optional - Rank for distributed training (default: 0).
- **t5_fsdp** (bool) - Optional - Whether to use FSDP for T5 model.
- **dit_fsdp** (bool) - Optional - Whether to use FSDP for DiT model.
- **use_usp** (bool) - Optional - Whether to use USP.
- **t5_cpu** (bool) - Optional - Whether to place T5 model on CPU to save VRAM.

#### Generate Method Parameters
- **input_prompt** (str) - Required - The text prompt guiding the video generation.
- **img** (PIL.Image.Image) - Required - The input image.
- **max_area** (int) - Optional - Maximum pixel area for the video (maintains aspect ratio) (default: 720 * 1280).
- **frame_num** (int) - Optional - Number of frames to generate (default: 81).
- **shift** (float) - Optional - Noise schedule shift (use 3.0 for 480P resolution) (default: 5.0).
- **sample_solver** (str) - Optional - Sampling solver ('unipc' or 'dpm++') (default: 'unipc').
- **sampling_steps** (int) - Optional - Number of diffusion steps (default: 40).
- **guide_scale** (float) - Optional - Classifier-free guidance scale (default: 5.0).
- **n_prompt** (str) - Optional - Negative prompt.
- **seed** (int) - Optional - Random seed (-1 for random) (default: 42).
- **offload_model** (bool) - Optional - Offload model to CPU to save VRAM (default: True).

### Request Example
```python
import torch
from PIL import Image
import wan
from wan.configs import WAN_CONFIGS, MAX_AREA_CONFIGS
from wan.utils.utils import cache_video

# Initialize the Image-to-Video model
cfg = WAN_CONFIGS['i2v-14B']
wan_i2v = wan.WanI2V(
    config=cfg,
    checkpoint_dir='./Wan2.1-I2V-14B-720P',
    device_id=0,
    rank=0,
    t5_fsdp=False,
    dit_fsdp=False,
    use_usp=False,
    t5_cpu=False,
)

# Load input image
img = Image.open('input_image.jpg').convert("RGB")

# Generate video from image
prompt = "A white cat wearing sunglasses sits on a surfboard, relaxed expression, sea breeze blowing."
video = wan_i2v.generate(
    input_prompt=prompt,
    img=img,
    max_area=720 * 1280,        # Maximum pixel area (maintains aspect ratio)
    frame_num=81,               # Number of frames
    shift=5.0,                  # Use 3.0 for 480P resolution
    sample_solver='unipc',
    sampling_steps=40,          # I2V uses 40 steps by default
    guide_scale=5.0,
    n_prompt="",
    seed=42,
    offload_model=True
)

# Save the generated video
cache_video(
    tensor=video[None],
    save_file="i2v_output.mp4",
    fps=16,
    nrow=1,
    normalize=True,
    value_range=(-1, 1)
)
```

### Response
#### Success Response (200)
- **video** (torch.Tensor) - The generated video tensor.

#### Response Example
```python
# The 'video' variable will contain the generated video tensor.
# Example of saving the video is shown in the Request Example.
```
```

--------------------------------

### Launching Gradio Demos for Video Generation Tasks

Source: https://context7.com/wan-video/wan2.1/llms.txt

This section provides command-line instructions to launch various Gradio web interfaces for different video generation tasks. It includes options for text-to-video, image-to-video, first-last-frame-to-video, and VACE, with support for prompt extension methods like DashScope API and local Qwen models.

```bash
# Launch Text-to-Video Gradio demo
cd gradio

# With DashScope API for prompt extension
DASH_API_KEY=your_key python t2v_14B_singleGPU.py \
    --prompt_extend_method dashscope \
    --ckpt_dir ./Wan2.1-T2V-14B

# With local Qwen model for prompt extension
python t2v_14B_singleGPU.py \
    --prompt_extend_method local_qwen \
    --ckpt_dir ./Wan2.1-T2V-14B

# Launch Image-to-Video Gradio demo
DASH_API_KEY=your_key python i2v_14B_singleGPU.py \
    --prompt_extend_method dashscope \
    --ckpt_dir_720p ./Wan2.1-I2V-14B-720P \
    --ckpt_dir_480p ./Wan2.1-I2V-14B-480P

# Launch First-Last-Frame-to-Video Gradio demo
python flf2v_14B_singleGPU.py \
    --prompt_extend_method dashscope \
    --ckpt_dir_720p ./Wan2.1-FLF2V-14B-720P

# Launch VACE Gradio demo (single GPU)
python vace.py --ckpt_dir ./Wan2.1-VACE-1.3B

# Launch VACE Gradio demo (multi-GPU)
python vace.py --mp --ulysses_size 8 --ckpt_dir ./Wan2.1-VACE-14B
```

--------------------------------

### Run VACE Gradio Interface

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Launches a local Gradio web interface for interactive VACE video generation. Includes options for single and multi-GPU execution.

```bash
python gradio/vace.py --ckpt_dir ./Wan2.1-VACE-1.3B
```

```bash
python gradio/vace.py --mp --ulysses_size 8 --ckpt_dir ./Wan2.1-VACE-14B/
```

--------------------------------

### Run Local Gradio Interface for Wan2.1

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Launches a local Gradio interface for Wan2.1 text-to-video generation. Supports prompt extension using either Dashscope's API or a local Qwen model. Requires changing directory to 'gradio' and setting the API key if using Dashscope.

```shell
cd gradio
# if one uses dashscope’s API for prompt extension
DASH_API_KEY=your_key python t2v_14B_singleGPU.py --prompt_extend_method 'dashscope' --ckpt_dir ./Wan2.1-T2V-14B

# if one uses a local model for prompt extension
python t2v_14B_singleGPU.py --prompt_extend_method 'local_qwen' --ckpt_dir ./Wan2.1-T2V-14B
```

--------------------------------

### Expand Prompts using Local Qwen Model

Source: https://context7.com/wan-video/wan2.1/llms.txt

This snippet demonstrates initializing the `QwenPromptExpander` for offline prompt enhancement using locally deployed Qwen models. It is suitable for scenarios where API access is limited or for processing sensitive data. The expander supports various Qwen model sizes for text-to-video generation.

```python
from wan.utils.prompt_extend import QwenPromptExpander

# Initialize with local Qwen model
# For text-to-video: 'Qwen2.5_3B', 'Qwen2.5_7B', 'Qwen2.5_14B'
# prompt_expander = QwenPromptExpander(model_name='Qwen2.5_3B')
# expanded_prompt = prompt_expander("A dog running in a park")

```

--------------------------------

### Run Image-to-Video Generation (Single-GPU)

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Generates video from an image using the Wan2.1 model on a single GPU. This command specifies the task, desired video size, checkpoint directory, input image, and a text prompt describing the desired video content. The 'size' parameter dictates the output resolution.

```shell
python generate.py --task i2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-I2V-14B-720P --image examples/i2v_input.JPG --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside."
```

--------------------------------

### Prompt Extension using Local Qwen Model

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Extends prompts for video generation using a locally hosted Qwen model from HuggingFace. Supports various Qwen models suitable for text-to-video and image-to-video tasks. Users can specify local paths or Hugging Face model names for extension.

```sh
python generate.py  --task t2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-T2V-14B --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage" --use_prompt_extend --prompt_extend_method 'local_qwen' --prompt_extend_target_lang 'zh'
```

--------------------------------

### Initialize and Use WanVace for Video Creation and Editing

Source: https://context7.com/wan-video/wan2.1/llms.txt

This code initializes the WanVace model for various video generation and editing tasks, including reference-to-video (R2V), video-to-video editing (V2V), and masked video editing (MV2V). It shows the preparation of source materials like reference images and masks, followed by video generation and saving.

```python
import torch
import wan
from wan.configs import WAN_CONFIGS, SIZE_CONFIGS
from wan.utils.utils import cache_video

# Initialize the VACE model
cfg = WAN_CONFIGS['vace-14B']  # or 'vace-1.3B'
wan_vace = wan.WanVace(
    config=cfg,
    checkpoint_dir='./Wan2.1-VACE-14B',
    device_id=0,
    rank=0,
    t5_fsdp=False,
    dit_fsdp=False,
    use_usp=False,
    t5_cpu=False,
)

# Prepare source materials (reference images)
src_video = [None]  # None for text-to-video, or path to source video
src_mask = [None]   # None or path to mask video
src_ref_images = [['reference1.png', 'reference2.png']]  # Reference images

# Preprocess the source materials
src_video, src_mask, src_ref_images = wan_vace.prepare_source(
    src_video=src_video,
    src_mask=src_mask,
    src_ref_images=src_ref_images,
    num_frames=81,
    image_size=SIZE_CONFIGS['1280*720'],
    device=torch.device('cuda:0')
)

# Generate video with reference images
prompt = "A girl in a red dress playing with a cartoon snake in a festive scene."
video = wan_vace.generate(
    input_prompt=prompt,
    input_frames=src_video,
    input_masks=src_mask,
    input_ref_images=src_ref_images,
    size=(1280, 720),
    frame_num=81,
    context_scale=1.0,
    shift=16,                   # VACE uses shift=16 by default
    sample_solver='unipc',
    sampling_steps=50,
    guide_scale=5.0,
    n_prompt="",
    seed=42,
    offload_model=True
)

# Save the generated video
cache_video(
    tensor=video[None],
    save_file="vace_output.mp4",
    fps=16,
    nrow=1,
    normalize=True,
    value_range=(-1, 1)
)
```

--------------------------------

### Generate Video: First-Last-Frame (Local Prompt Extension)

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Generates video using First-Last-Frame-to-Video with local prompt extension. This method leverages a specified local model (e.g., Qwen2.5-VL-7B-Instruct) to enhance the prompt before video generation. It requires the `--use_prompt_extend` and `--prompt_extend_model` flags.

```shell
python generate.py --task flf2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-FLF2V-14B-720P --first_frame examples/flf2v_input_first_frame.png --last_frame examples/flf2v_input_last_frame.png --use_prompt_extend --prompt_extend_model Qwen/Qwen2.5-VL-7B-Instruct --prompt "CG animation style, a small blue bird takes off from the ground, flapping its wings. The bird’s feathers are delicate, with a unique pattern on its chest. The background shows a blue sky with white clouds under bright sunshine. The camera follows the bird upward, capturing its flight and the vastness of the sky from a close-up, low-angle perspective."
```

--------------------------------

### Expand Prompts with Qwen Models

Source: https://context7.com/wan-video/wan2.1/llms.txt

Utilize QwenPromptExpander to enhance text prompts for video generation. Supports standard text prompts and vision-language (VL) models for image-conditioned expansion.

```python
from wan_video.prompt_expander import QwenPromptExpander
from PIL import Image

# Standard prompt expansion
prompt_expander = QwenPromptExpander(model_name='Qwen/Qwen2.5-14B-Instruct', is_vl=False, device=0)
result = prompt_expander(prompt="Two cats fighting", tar_lang='en', seed=42)
print(f"Expanded: {result.prompt}")

# VL-based expansion
vl_expander = QwenPromptExpander(model_name='Qwen/Qwen2.5-VL-7B-Instruct', is_vl=True, device=0)
image = Image.open('cat.jpg')
result = vl_expander(prompt="Animate this cat", tar_lang='zh', image=image, seed=42)
```

--------------------------------

### Run Wan2.1 Inference with Prompt Extension

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Commands to execute image-to-video generation using local or remote prompt extension models. The process requires specifying the task, checkpoint directory, input image, and the prompt extension method.

```bash
python generate.py --task i2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-I2V-14B-720P --image examples/i2v_input.JPG --use_prompt_extend --prompt_extend_model Qwen/Qwen2.5-VL-7B-Instruct --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard..."
```

```bash
DASH_API_KEY=your_key python generate.py --task i2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-I2V-14B-720P --image examples/i2v_input.JPG --use_prompt_extend --prompt_extend_method 'dashscope' --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard..."
```

--------------------------------

### Run Wan2.1 with Diffusers Pipeline

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Python script demonstrating how to load the Wan2.1 model using the Hugging Face Diffusers library. It handles image preprocessing, model configuration, and video export.

```python
import torch
import numpy as np
from diffusers import AutoencoderKLWan, WanImageToVideoPipeline
from diffusers.utils import export_to_video, load_image
from transformers import CLIPVisionModel

model_id = "Wan-AI/Wan2.1-I2V-14B-720P-Diffusers"
image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", torch_dtype=torch.float32)
vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
pipe = WanImageToVideoPipeline.from_pretrained(model_id, vae=vae, image_encoder=image_encoder, torch_dtype=torch.bfloat16)
pipe.to("cuda")

image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg")
max_area = 720 * 1280
aspect_ratio = image.height / image.width
mod_value = pipe.vae_scale_factor_spatial * pipe.transformer.config.patch_size[1]
height = round(np.sqrt(max_area * aspect_ratio)) // mod_value * mod_value
width = round(np.sqrt(max_area / aspect_ratio)) // mod_value * mod_value
image = image.resize((width, height))
prompt = "An astronaut hatching from an egg..."
negative_prompt = "Bright tones, overexposed, static..."

output = pipe(
    image=image,
    prompt=prompt,
    negative_prompt=negative_prompt,
    height=height, width=width,
    num_frames=81,
    guidance_scale=5.0
).frames[0]
export_to_video(output, "output.mp4", fps=16)
```

--------------------------------

### Run Local Gradio Interface for First-Last-Frame Video

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Launches a local Gradio web interface for First-Last-Frame-to-Video generation. This script is designed to run within the `gradio` directory and can be configured to use the 720P model and Dashscope for prompt extension by setting the `DASH_API_KEY`.

```shell
cd gradio
# use 720P model in gradio
DASH_API_KEY=your_key python flf2v_14B_singleGPU.py --prompt_extend_method 'dashscope' --ckpt_dir_720p ./Wan2.1-FLF2V-14B-720P
```

--------------------------------

### Generate Video: First-Last-Frame (No Prompt Extension, Multi-GPU FSDP)

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Performs First-Last-Frame-to-Video generation using multiple GPUs with FSDP and xDiT USP for enhanced performance. This command requires the `xfuser` library and utilizes distributed training capabilities. It takes input frames, a checkpoint, and a detailed prompt to generate the video.

```shell
pip install "xfuser>=0.4.1"
torchrun --nproc_per_node=8 generate.py --task flf2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-FLF2V-14B-720P --first_frame examples/flf2v_input_first_frame.png --last_frame examples/flf2v_input_last_frame.png --dit_fsdp --t5_fsdp --ulysses_size 8 --prompt "CG animation style, a small blue bird takes off from the ground, flapping its wings. The bird’s feathers are delicate, with a unique pattern on its chest. The background shows a blue sky with white clouds under bright sunshine. The camera follows the bird upward, capturing its flight and the vastness of the sky from a close-up, low-angle perspective."
```

--------------------------------

### Prompt Extension using Dashscope API

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Enhances video generation quality by extending input prompts using the Dashscope API. Requires API key configuration and specifies models like 'qwen-plus' for text-to-video. Allows customization of the extension model and target language.

```sh
DASH_API_KEY=your_key python generate.py  --task t2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-T2V-14B --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage" --use_prompt_extend --prompt_extend_method 'dashscope' --prompt_extend_target_lang 'zh'
```

--------------------------------

### Image-to-Video Generation with Diffusers Pipeline

Source: https://context7.com/wan-video/wan2.1/llms.txt

This snippet demonstrates how to use the WanImageToVideoPipeline from the Diffusers library to generate a video from a given image and text prompt. It involves loading the model components, preparing the input image by resizing it to maintain aspect ratio, and then running the pipeline with specified parameters. The output video is then exported.

```python
from transformers import CLIPVisionModel
from diffusers import AutoencoderKLWan, WanImageToVideoPipeline
from PIL import Image
import torch
import numpy as np

def load_image(url):
    # Placeholder for actual image loading function
    # In a real scenario, this would use PIL.Image.open or similar
    print(f"Loading image from {url}")
    # Returning a dummy image for demonstration
    return Image.new('RGB', (640, 480), color = 'red')

def export_to_video(frames, filename, fps):
    # Placeholder for actual video export function
    print(f"Exporting {len(frames)} frames to {filename} at {fps} fps")


i2v_model_id = "Wan-AI/Wan2.1-I2V-14B-720P-Diffusers"
image_encoder = CLIPVisionModel.from_pretrained(
    i2v_model_id, subfolder="image_encoder", torch_dtype=torch.float32
)
vae = AutoencoderKLWan.from_pretrained(
    i2v_model_id, subfolder="vae", torch_dtype=torch.float32
)
pipe = WanImageToVideoPipeline.from_pretrained(
    i2v_model_id, vae=vae, image_encoder=image_encoder, torch_dtype=torch.bfloat16
)
pipe.to("cuda")

# Load and resize image maintaining aspect ratio
image = load_image("https://example.com/astronaut.jpg")
max_area = 720 * 1280
aspect_ratio = image.height / image.width
mod_value = pipe.vae_scale_factor_spatial * pipe.transformer.config.patch_size[1]
height = round(np.sqrt(max_area * aspect_ratio)) // mod_value * mod_value
width = round(np.sqrt(max_area / aspect_ratio)) // mod_value * mod_value
image = image.resize((width, height))

output = pipe(
    image=image,
    prompt="An astronaut hatching from an egg on the moon",
    negative_prompt="static, blurred, low quality",
    height=height,
    width=width,
    num_frames=81,
    guidance_scale=5.0
).frames[0]

export_to_video(output, "i2v_diffusers_output.mp4", fps=16)
```

--------------------------------

### Integrate with Hugging Face Diffusers

Source: https://context7.com/wan-video/wan2.1/llms.txt

Use the Wan2.1 Diffusers pipeline for simplified text-to-video generation workflows. Requires loading VAE, scheduler, and the pipeline model.

```python
import torch
from diffusers import AutoencoderKLWan, WanPipeline
from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler
from diffusers.utils import export_to_video

model_id = "Wan-AI/Wan2.1-T2V-14B-Diffusers"
vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
scheduler = UniPCMultistepScheduler(prediction_type='flow_prediction', flow_shift=5.0)

pipe = WanPipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16).to("cuda")
pipe.scheduler = scheduler

output = pipe(prompt="A cat and a dog baking a cake", height=720, width=1280).frames[0]
export_to_video(output, "diffusers_output.mp4", fps=16)
```

--------------------------------

### Generate Video: First-Last-Frame (No Prompt Extension, Single-GPU)

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Generates a video from the first and last frames without prompt extension using a single GPU. Requires the `generate.py` script and specified checkpoint directory. Outputs a video based on the provided prompt and input frames.

```shell
python generate.py --task flf2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-FLF2V-14B-720P --first_frame examples/flf2v_input_first_frame.png --last_frame examples/flf2v_input_last_frame.png --prompt "CG animation style, a small blue bird takes off from the ground, flapping its wings. The bird’s feathers are delicate, with a unique pattern on its chest. The background shows a blue sky with white clouds under bright sunshine. The camera follows the bird upward, capturing its flight and the vastness of the sky from a close-up, low-angle perspective."
```

--------------------------------

### Run VACE CLI Inference

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Executes VACE video generation tasks via command line. Supports single-GPU and multi-GPU configurations using FSDP and xDiT USP for scaling.

```bash
python generate.py --task vace-1.3B --size 832*480 --ckpt_dir ./Wan2.1-VACE-1.3B --src_ref_images examples/girl.png,examples/snake.png --prompt "在一个欢乐而充满节日气氛的场景中，穿着鲜艳红色春服的小女孩正与她的可爱卡通蛇嬉戏。"
```

```bash
torchrun --nproc_per_node=8 generate.py --task vace-14B --size 1280*720 --ckpt_dir ./Wan2.1-VACE-14B --dit_fsdp --t5_fsdp --ulysses_size 8 --src_ref_images examples/girl.png,examples/snake.png --prompt "在一个欢乐而充满节日气氛的场景中，穿着鲜艳红色春服的小女孩正与她的可爱卡通蛇嬉戏。"
```

--------------------------------

### WanVAE Video Autoencoder Initialization and Usage

Source: https://context7.com/wan-video/wan2.1/llms.txt

This Python snippet shows how to initialize and use the WanVAE class for video encoding and decoding. It demonstrates encoding a video tensor into a latent space and then decoding the latents back into video frames, highlighting the VAE's capability to preserve temporal information and handle long videos efficiently.

```python
import torch
from wan.modules.vae import WanVAE

# Initialize the VAE
vae = WanVAE(
    z_dim=16,
    vae_pth='./Wan2.1-T2V-14B/Wan2.1_VAE.pth',
    dtype=torch.float,
    device='cuda'
)

# Encode video frames to latent space
# Input: list of videos with shape [C, T, H, W] (channels, frames, height, width)
video_tensor = torch.randn(3, 81, 720, 1280).to('cuda')  # Example video
latents = vae.encode([video_tensor])
# Output: list of latent tensors with shape [z_dim, T', H', W']
print(f"Latent shape: {latents[0].shape}")  # e.g., [16, 21, 90, 160]

# Decode latents back to video frames
reconstructed = vae.decode(latents)
# Output: list of video tensors with shape [C, T, H, W], values in [-1, 1]
print(f"Reconstructed shape: {reconstructed[0].shape}")  # [3, 81, 720, 1280]

# The VAE uses causal convolutions for temporal consistency
# and chunk-based processing for memory efficiency with long videos
```

--------------------------------

### Run Inference via CLI

Source: https://context7.com/wan-video/wan2.1/llms.txt

Execute various video generation tasks using the generate.py script. Supports single-GPU, multi-GPU, and low-VRAM configurations.

```bash
# Text-to-Video generation
python generate.py --task t2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-T2V-14B --prompt "Two cats boxing" --sample_steps 50

# Image-to-Video generation
python generate.py --task i2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-I2V-14B-720P --image input.jpg --prompt "The cat moves"

# Multi-GPU inference with FSDP
torchrun --nproc_per_node=8 generate.py --task t2v-14B --dit_fsdp --t5_fsdp --prompt "Epic cinematic scene"
```

--------------------------------

### Generate Video: First-Last-Frame (Remote Prompt Extension via Dashscope)

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Generates video using First-Last-Frame-to-Video with prompt extension facilitated by the Dashscope API. This requires setting the `DASH_API_KEY` environment variable and specifying `'dashscope'` for the prompt extension method. It allows for remote processing of prompt extensions.

```shell
DASH_API_KEY=your_key python generate.py --task flf2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-FLF2V-14B-720P --first_frame examples/flf2v_input_first_frame.png --last_frame examples/flf2v_input_last_frame.png --use_prompt_extend --prompt_extend_method 'dashscope' --prompt "CG animation style, a small blue bird takes off from the ground, flapping its wings. The bird’s feathers are delicate, with a unique pattern on its chest. The background shows a blue sky with white clouds under bright sunshine. The camera follows the bird upward, capturing its flight and the vastness of the sky from a close-up, low-angle perspective."
```

--------------------------------

### Single-GPU Inference for Wan2.1

Source: https://github.com/wan-video/wan2.1/blob/main/README.md

Executes text-to-video generation on a single GPU. Supports model checkpoint loading and prompt specification. Includes options like `--offload_model` and `--t5_cpu` to manage GPU memory for larger models or lower-spec hardware.

```sh
python generate.py  --task t2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-T2V-14B --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."
```

```sh
python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --offload_model True --t5_cpu --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."
```