### Install huggingface-cli

Source: https://github.com/tencent/hunyuancustom/blob/main/models/README.md

Install the huggingface-cli tool, which is required for downloading models. Detailed instructions are available in the huggingface-hub documentation.

```shell
python -m pip install "huggingface_hub[cli]"
```

--------------------------------

### Install Pip Dependencies

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Install the remaining project dependencies listed in the 'requirements.txt' file using pip.

```shell
# 4. Install pip dependencies
python -m pip install -r requirements.txt
```

--------------------------------

### Launch Gradio Web UI Demos

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Scripts to launch the Gradio web UI for different generation modes. These commands start the interactive demo server.

```bash
# Default: single-subject customization
bash ./scripts/run_gradio.sh
```

```bash
# Video-editing mode
bash ./scripts/run_gradio.sh --video
```

```bash
# Audio-driven mode
bash ./scripts/run_gradio.sh --audio
```

--------------------------------

### Install Flash Attention v2

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Install Flash Attention v2 for acceleration, which requires CUDA 11.8 or above. This step involves installing ninja first.

```shell
# 5. Install flash attention v2 for acceleration (requires CUDA 11.8 or above)
python -m pip install ninja
python -m pip install git+https://github.com/Dao-AILab/flash-attention.git@v2.6.3
```

--------------------------------

### Install PyTorch with CUDA 11.8

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Uninstall existing packages, then install a specific version of PyTorch compiled for CUDA 11.8, followed by other requirements and ninja. Flash-attention is installed from a git repository.

```bash
pip uninstall -r requirements.txt
pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt
pip install ninja
pip install git+https://github.com/Dao-AILab/flash-attention.git@v2.6.3
```

--------------------------------

### Single-GPU Low-VRAM Inference Setup

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Sets up environment variables for single-GPU inference, potentially with FP8 quantization or CPU offloading for lower VRAM GPUs. Suitable for GPUs with 24 GB+ VRAM or any GPU with --cpu-offload.

```bash
export MODEL_BASE="./models"
export PYTHONPATH=./
```

--------------------------------

### Run HunyuanVideo Docker Image (CUDA 11.8)

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Pulls the HunyuanVideo Docker image for CUDA 11.8 and runs it in detached mode with all GPUs enabled and host networking. Installs specific versions of gradio, diffusers, and transformers.

```bash
# For CUDA 11.8
docker pull hunyuanvideo/hunyuanvideo:cuda_11
docker run -itd --gpus all --init --net=host --uts=host --ipc=host --name hunyuanvideo --security-opt=seccomp=unconfined --ulimit=stack=67108864 --ulimit=memlock=-1 --privileged hunyuanvideo/hunyuanvideo:cuda_11
pip install gradio==3.39.0 diffusers==0.33.0 transformers==4.41.2
```

--------------------------------

### Run HunyuanVideo Docker Image (CUDA 12.4)

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Pulls the HunyuanVideo Docker image for CUDA 12.4 and runs it in detached mode with all GPUs enabled and host networking. Installs specific versions of gradio, diffusers, and transformers.

```bash
# For CUDA 12.4 (updated to avoid float point exception)
docker pull hunyuanvideo/hunyuanvideo:cuda_12
docker run -itd --gpus all --init --net=host --uts=host --ipc=host --name hunyuanvideo --security-opt=seccomp=unconfined --ulimit=stack=67108864 --ulimit=memlock=-1 --privileged hunyuanvideo/hunyuanvideo:cuda_12
pip install gradio==3.39.0 diffusers==0.33.0 transformers==4.41.2
```

--------------------------------

### Install PyTorch with CUDA Support

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Install PyTorch, torchvision, and torchaudio using conda. Choose the appropriate command based on your CUDA version (11.8 or 12.4).

```shell
# 3. Install PyTorch and other dependencies using conda
# For CUDA 11.8
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=11.8 -c pytorch -c nvidia
# For CUDA 12.4
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.4 -c pytorch -c nvidia
```

--------------------------------

### LLaVA Image Preprocessing and Inference

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Preprocesses images for LLaVA and performs inference using a sampler. Requires specific imports and device setup. The output is saved as a video grid.

```python
import torch as T
from PIL import Image
import numpy as np
from hymm_sp.inference_engine import HunyuanVideoSampler

# Assuming device is already set, e.g., device = "cuda" if torch.cuda.is_available() else "cpu"
device = "cuda"

llava_transform = T.Compose([
    T.Resize((336, 336), interpolation=T.InterpolationMode.BILINEAR),
    T.ToTensor(),
    T.Normalize((0.48145466, 0.4578275, 0.4082107),
                (0.26862954, 0.26130258, 0.27577711)),
])
img_pil = Image.open("./assets/images/seg_woman_01.png").convert("RGB")
pixel_value_llava = llava_transform(img_pil).unsqueeze(0).to(device)
uncond_llava = llava_transform(Image.fromarray(np.ones((336, 336, 3), dtype=np.uint8) * 255)).unsqueeze(0)

# Assuming sampler and load_ref_image are defined elsewhere
# sampler = HunyuanVideoSampler.from_pretrained("./models/hunyuancustom_720P")
# ref_latents, uncond_ref_latents = load_ref_image(
#     "./assets/images/seg_woman_01.png",
#     video_size=(720, 1280),
#     vae=sampler.vae,
#     device=device,
# )

# outputs = sampler.predict(
#     prompt="Realistic, High-quality. A woman is drinking coffee at a café.",
#     negative_prompt="Aerial view, overexposed, low quality, deformation, bad hands, blurring.",
#     size=(720, 1280),
#     video_length=129,           # frames; use 4n+1 for 3D VAE (e.g. 29, 65, 129)
#     seed=1024,
#     infer_steps=30,
#     guidance_scale=7.5,
#     flow_shift=13.0,
#     pixel_value_llava=pixel_value_llava,
#     uncond_pixel_value_llava=uncond_llava,
#     ref_latents=ref_latents,
#     uncond_ref_latents=uncond_ref_latents,
#     use_deepcache=1,            # 1=enabled (faster), 0=disabled
# )
# outputs = {"samples": T[B,C,T,H,W], "prompts": [...], "seeds": [...], "size": (H,W)}
# save_videos_grid(outputs["samples"][0].unsqueeze(0), "./results/woman_coffee.mp4", fps=25)
```

--------------------------------

### Clone HunyuanCustom Repository

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Clone the official HunyuanCustom repository to your local machine. Navigate into the cloned directory to proceed with the installation.

```shell
git clone https://github.com/Tencent/HunyuanCustom.git
cd HunyuanCustom
```

--------------------------------

### Run Gradio Server for HunyuanCustom

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Execute these bash commands to launch the Gradio server for different customization modes. Use --video for video-driven, and --audio for audio-driven customization.

```bash
cd HunyuanCustom

# Single-Subject Video Customization
bash ./scripts/run_gradio.sh 

# Video-Driven Video Customization
bash ./scripts/run_gradio.sh --video

# Audio-Driven Video Customization
bash ./scripts/run_gradio.sh --audio
```

--------------------------------

### Initialize DataLoader with DistributedSampler

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Sets up a DataLoader for distributed training using DistributedSampler. Ensure dataset, num_replicas, and rank are correctly configured.

```python
from torch.utils.data import DataLoader, DistributedSampler

# Assuming 'dataset' is already defined
sampler = DistributedSampler(dataset, num_replicas=1, rank=0, shuffle=False, drop_last=False)
loader = DataLoader(dataset, batch_size=1, shuffle=False, sampler=sampler)
for batch in loader:
    print(batch["pixel_value_ref"].shape)   # torch.Size([1, 3, 720, 1280])
    print(batch["prompt"])
```

--------------------------------

### HunyuanVideoSampler.from_pretrained

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Loads the HunyuanVideo inference pipeline from a specified checkpoint. This is the primary entry point for instantiating the model, configuring it with provided arguments, and setting up the necessary components for video generation.

```APIDOC
## HunyuanVideoSampler.from_pretrained — Load model from checkpoint

### Description
The primary entry point for instantiating the full inference pipeline. Loads the DiT transformer, 3D VAE (`884-16c-hy0801`), LLaVA-LLaMA-3-8B text encoder, and CLIP-L tokenizer from a checkpoint directory. Accepts an `args` namespace produced by `parse_args()` to control precision, CPU offloading, FP8 quantization, and conditioning mode flags.

### Method
```python
from pathlib import Path
from hymm_sp.config import parse_args
from hymm_sp.sample_inference import HunyuanVideoSampler
import torch

# Build args with required flags
import sys
sys.argv = [
    "inference",
    "--ckpt", "./models/hunyuancustom_720P/mp_rank_00_model_states.pt",
    "--video-size", "720", "1280",
    "--sample-n-frames", "129",
    "--infer-steps", "30",
    "--flow-shift-eval-video", "13.0",
    "--save-path", "./results/output",
    "--seed", "1024",
]
args = parse_args()

device = torch.device("cuda")
sampler = HunyuanVideoSampler.from_pretrained(
    pretrained_model_path=args.ckpt,
    args=args,
    device=device
)
# sampler.vae        — 3D causal VAE
# sampler.model      — DiT transformer
# sampler.pipeline   — HunyuanVideo diffusion pipeline
# sampler.args       — updated args after model load
print("Model loaded successfully")
```

### Parameters
* `pretrained_model_path` (str) - Path to the model checkpoint directory.
* `args` (namespace) - Arguments namespace obtained from `parse_args()` controlling inference settings.
* `device` (torch.device) - The device to load the model onto (e.g., 'cuda').

### Response
* `HunyuanVideoSampler` - An instance of the HunyuanVideoSampler class, ready for inference.
```

--------------------------------

### Load HunyuanVideo Sampler from Pretrained Checkpoint

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Instantiates the full inference pipeline by loading the DiT transformer, 3D VAE, LLaVA-LLaMA-3-8B text encoder, and CLIP-L tokenizer from a checkpoint. Use `args` to control precision, CPU offloading, and FP8 quantization.

```python
from pathlib import Path
from hymm_sp.config import parse_args
from hymm_sp.sample_inference import HunyuanVideoSampler
import torch

# Build args with required flags
import sys
sys.argv = [
    "inference",
    "--ckpt", "./models/hunyuancustom_720P/mp_rank_00_model_states.pt",
    "--video-size", "720", "1280",
    "--sample-n-frames", "129",
    "--infer-steps", "30",
    "--flow-shift-eval-video", "13.0",
    "--save-path", "./results/output",
    "--seed", "1024",
]
args = parse_args()

device = torch.device("cuda")
sampler = HunyuanVideoSampler.from_pretrained(
    pretrained_model_path=args.ckpt,
    args=args,
    device=device
)
# sampler.vae        — 3D causal VAE
# sampler.model      — DiT transformer
# sampler.pipeline   — HunyuanVideo diffusion pipeline
# sampler.args       — updated args after model load
print("Model loaded successfully")
```

--------------------------------

### Download DWPose Model

Source: https://github.com/tencent/hunyuancustom/blob/main/models/README.md

Download the DWPose pretrained model using wget. This involves creating a directory for the model and then downloading the necessary ONNX files.

```shell
cd HunyuanCustom
mkdir -p models/DWPose
wget https://huggingface.co/yzd-v/DWPose/resolve/main/yolox_l.onnx?download=true -O models/DWPose/yolox_l.onnx
wget https://huggingface.co/yzd-v/DWPose/resolve/main/dw-ll_ucoco_384.onnx?download=true -O models/DWPose/dw-ll_ucoco_384.onnx
```

--------------------------------

### Troubleshoot Float Point Exception with CUDA 12

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

If you encounter a float point exception (core dump) on specific GPU types, try this option. Ensure CUDA 12.4, CUBLAS >= 12.4.5.8, and CUDNN >= 9.00 are installed, or use the provided CUDA 12 docker image.

```shell
# Option 1: Making sure you have installed CUDA 12.4, CUBLAS>=12.4.5.8, and CUDNN>=9.00 (or simply using our CUDA 12 docker image).
pip install nvidia-cublas-cu12==12.4.5.8
export LD_LIBRARY_PATH=/opt/conda/lib/python3.8/site-packages/nvidia/cublas/lib/
```

--------------------------------

### On-the-fly Data Preprocessing with DataPreprocess

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Preprocesses input data for image, audio, or video formats on the fly for server applications. Handles image padding, normalization, audio encoding, and video frame extraction.

```python
from hymm_sp.data_kits.video_dataset import DataPreprocess

data_preprocess = DataPreprocess(args, device)

# Image-only batch
batch = data_preprocess.get_batch(
    meta_data={"image_path": "./assets/images/seg_woman_01.png"},
    size=(1280, 720),   # (width, height)
    data_type="image"
)
# batch keys: pixel_value_llava, uncond_pixel_value_llava, pixel_value_ref

# Audio batch — meta_data must include "audio_path"
batch_audio = data_preprocess.get_batch(
    meta_data={
        "image_path": "./assets/images/seg_man_01.png",
        "audio_path": "./assets/audios/milk_man.mp3",
    },
    size=(1280, 720),
    video_length=129,
    data_type="audio"
)
# Extra key: batch_audio["audio_prompts"]  shape: [1, 129, 10, 12, 768]

# Video-editing batch — meta_data must include masked_input_video_path and input_mask_video_path
batch_video = data_preprocess.get_batch(
    meta_data={
        "image_path": "./assets/images/sed_red_panda.png",
        "masked_input_video_path": "./assets/input_videos/001_bg.mp4",
        "input_mask_video_path":  "./assets/input_videos/001_mask.mp4",
    },
    size=(1280, 720),
    data_type="video"
)
# Extra keys: batch_video["pixel_value_bg"], batch_video["pixel_value_mask"]
```

--------------------------------

### Parallel Inference: Audio-Driven Video Customization

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Runs a script for audio-driven video customization using 8 GPUs. Requires reference image, input audio, prompts, checkpoint, and output path. Parameters like `--audio-strength`, `--audio-condition`, and `--use-deepcache` are specific to this mode.

```bash
cd HunyuanCustom

export MODEL_BASE="./models"
export PYTHONPATH=./
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/seg_man_01.png' \
    --input-audio './assets/audios/milk_man.mp3' \
    --audio-strength 0.8 \
    --audio-condition \
    --pos-prompt "Realistic, High-quality. In the study, a man sits at a table featuring a bottle of milk while delivering a product presentation." \
    --neg-prompt "Two people, two persons, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_audio_720P/mp_rank_00_model_states.pt" \
    --seed 1026 \
    --video-size 720 1280 \
    --sample-n-frames 129 \
    --cfg-scale 7.5 \
    --infer-steps 30 \
    --use-deepcache 1 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/sp_audio_720p'
```

--------------------------------

### Single-GPU Inference with HunyuanCustom

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Use this command for generating videos with a single GPU. Ensure the MODEL_BASE and PYTHONPATH environment variables are set correctly. The --use-fp8 flag enables FP8 precision for inference.

```bash
cd HunyuanCustom

export MODEL_BASE="./models"
export DISABLE_SP=1
export PYTHONPATH=./
python hymm_sp/sample_gpu_poor.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_720P/mp_rank_00_model_states_fp8.pt" \
    --video-size 512 896 \
    --seed 1024 \
    --sample-n-frames 129 \
    --infer-steps 30 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/1gpu_540p' \
    --use-fp8
```

--------------------------------

### Low VRAM Inference with HunyuanCustom

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

This command is optimized for running inference with very low VRAM by enabling CPU offloading. Set the CPU_OFFLOAD environment variable and use the --cpu-offload flag. Adjust video size as needed.

```bash
cd HunyuanCustom

export MODEL_BASE="./models"
export CPU_OFFLOAD=1
export PYTHONPATH=./
python hymm_sp/sample_gpu_poor.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_720P/mp_rank_00_model_states_fp8.pt" \
    --video-size 720 1280 \
    --seed 1024 \
    --sample-n-frames 129 \
    --infer-steps 30 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/cpu_720p' \
    --use-fp8 \
    --cpu-offload 
```

--------------------------------

### Python Equivalent for Gradio Launch

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Programmatically launch the Gradio frontend and FastAPI backend. Ensure environment variables for model paths are set correctly.

```python
# Equivalent Python launch (single-subject demo)
import subprocess, os

os.environ["MODEL_BASE"] = "./models"
os.environ["MODEL_OUTPUT_PATH"] = "./models/hunyuancustom_720P/"
os.environ["PYTHONPATH"] = "./"

# Start multi-GPU Flask backend
subprocess.Popen(
    "torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 "
    "hymm_gradio/flask_hycustom.py",
    shell=True
)

# Start Gradio frontend (blocks until closed)
from hymm_gradio.gradio_ref2v import create_demo
demo = create_demo()
demo.launch(server_name="0.0.0.0", server_port=80, share=True, allowed_paths=["/"])
# UI available at http://localhost:80
# Tabs: "Single-Subject Video Customization"
# Controls: prompt, negative prompt, reference image, width/height sliders,
#           num_frames, num_steps, flow_shift, guidance, seed, object name
```

--------------------------------

### DataPreprocess.get_batch

Source: https://context7.com/tencent/hunyuancustom/llms.txt

On-the-fly preprocessing for single requests, handling image padding, LLaVA normalization, Whisper audio encoding, and video frame extraction.

```APIDOC
## DataPreprocess.get_batch — On-the-fly preprocessing for the Flask/Gradio server

Used by the FastAPI server (`flask_hycustom.py`) to preprocess a single request's input data without a Dataset. Handles image padding, LLaVA normalization, Whisper audio encoding, and video frame extraction in one call.

### Method Signature
```python
data_preprocess.get_batch(meta_data: dict, size: tuple, data_type: str, video_length: int = None)
```

### Parameters

*   **meta_data** (dict) - Required - Dictionary containing paths to input data (e.g., `image_path`, `audio_path`, `masked_input_video_path`, `input_mask_video_path`).
*   **size** (tuple) - Required - Target size for preprocessing, specified as `(width, height)`.
*   **data_type** (str) - Required - Type of data to process, can be `"image"`, `"audio"`, or `"video"`.
*   **video_length** (int) - Optional - Specifies the desired length for video or audio processing.

### Usage Examples

**Image-only batch:**
```python
batch = data_preprocess.get_batch(
    meta_data={"image_path": "./assets/images/seg_woman_01.png"},
    size=(1280, 720),   # (width, height)
    data_type="image"
)
# batch keys: pixel_value_llava, uncond_pixel_value_llava, pixel_value_ref
```

**Audio batch:**
```python
batch_audio = data_preprocess.get_batch(
    meta_data={
        "image_path": "./assets/images/seg_man_01.png",
        "audio_path": "./assets/audios/milk_man.mp3",
    },
    size=(1280, 720),
    video_length=129,
    data_type="audio"
)
# Extra key: batch_audio["audio_prompts"]  shape: [1, 129, 10, 12, 768]
```

**Video-editing batch:**
```python
batch_video = data_preprocess.get_batch(
    meta_data={
        "image_path": "./assets/images/sed_red_panda.png",
        "masked_input_video_path": "./assets/input_videos/001_bg.mp4",
        "input_mask_video_path":  "./assets/input_videos/001_mask.mp4",
    },
    size=(1280, 720),
    data_type="video"
)
# Extra keys: batch_video["pixel_value_bg"], batch_video["pixel_value_mask"]
```
```

--------------------------------

### Parse CLI Arguments for Inference Configuration

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Parses command-line arguments to configure inference settings for HunyuanVideoSampler. Supports single-subject, audio-driven, and video editing modes, as well as optimizations like FP8 and CPU offload.

```python
import sys
from hymm_sp.config import parse_args

# Single-subject (default)
sys.argv = [
    "run",
    "--ckpt", "./models/hunyuancustom_720P/mp_rank_00_model_states.pt",
    "--video-size", "720", "1280",
    "--sample-n-frames", "129",
    "--infer-steps", "30",
    "--cfg-scale", "7.5",
    "--flow-shift-eval-video", "13.0",
    "--seed", "1024",
    "--save-path", "./results/output",
    "--use-deepcache", "1",
    "--precision", "bf16",
    "--ref-image", "./assets/images/seg_woman_01.png",
    "--pos-prompt", "Realistic, High-quality. A woman is drinking coffee.",
    "--neg-prompt", "overexposed, low quality, blurring.",
]
args = parse_args()

# Audio-driven mode — add --audio-condition flag
sys.argv += ["--audio-condition", "--input-audio", "./assets/audios/milk_man.mp3",
             "--audio-strength", "0.8",
             "--ckpt", "./models/hunyuancustom_audio_720P/mp_rank_00_model_states.pt"]

# Video editing mode — add --video-condition flag
sys.argv += ["--video-condition", "--input-video", "./assets/input_videos/001_bg.mp4",
             "--mask-video", "./assets/input_videos/001_mask.mp4",
             "--expand-scale", "5",
             "--ckpt", "./models/hunyuancustom_editing_720P/mp_rank_00_model_states.pt"]

# Low-VRAM single GPU — add FP8 and CPU offload flags
sys.argv += ["--use-fp8", "--cpu-offload"]
args = parse_args()
print(args.video_size, args.precision, args.use_fp8)  # [720, 1280]  bf16  True
```

--------------------------------

### CPU Offload Generation (8 GB VRAM)

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Perform generation with CPU offloading, suitable for systems with limited VRAM. This command enables CPU offload for memory-constrained environments.

```bash
CPU_OFFLOAD=1 python hymm_sp/sample_gpu_poor.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, overexposed, low quality, blurring." \
    --ckpt ${MODEL_BASE}/hunyuancustom_720P/mp_rank_00_model_states_fp8.pt \
    --video-size 720 1280 \
    --seed 1024 \
    --sample-n-frames 129 \
    --infer-steps 30 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/cpu_720p' \
    --use-fp8 \
    --cpu-offload
```

--------------------------------

### Parallel Inference: Single-Subject Video Customization

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Navigates to the HunyuanCustom directory, sets environment variables, and runs a script for single-subject video customization using 8 GPUs. Requires specifying reference image, prompts, checkpoint, and output path.

```bash
cd HunyuanCustom

export MODEL_BASE="./models"
export PYTHONPATH=./
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_720P/mp_rank_00_model_states.pt" \
    --video-size 720 1280 \
    --seed 1024 \
    --sample-n-frames 129 \
    --infer-steps 30 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/sp_720p'
```

--------------------------------

### Download HunyuanCustom Model

Source: https://github.com/tencent/hunyuancustom/blob/main/models/README.md

Download the HunyuanCustom model using the huggingface-cli tool. Ensure you are in the 'HunyuanCustom' directory before executing the command. The download may take a significant amount of time.

```shell
# Switch to the directory named 'HunyuanCustom'
cd HunyuanCustom
# Use the huggingface-cli tool to download HunyuanCustom model in HunyuanCustom/models dir.
# The download time may vary from 10 minutes to 1 hour depending on network conditions.
huggingface-cli download tencent/HunyuanCustom --local-dir ./
```

--------------------------------

### Multi-GPU Batch Inference CLI (Audio-Driven)

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Performs multi-GPU batch inference for audio-driven talking-head generation. Includes parameters for audio input, strength, and conditioning.

```bash
export MODEL_BASE="./models"
export PYTHONPATH=./

# ── Audio-driven (talking-head, 8 GPUs) ────────────────────────────────────
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/seg_man_01.png' \
    --input-audio './assets/audios/milk_man.mp3' \
    --audio-strength 0.8 \
    --audio-condition \
    --pos-prompt "Realistic, High-quality. A man delivers a product presentation." \
    --neg-prompt "Two people, overexposed, low quality, blurring." \
    --ckpt ${MODEL_BASE}/hunyuancustom_audio_720P/mp_rank_00_model_states.pt \
    --seed 1026 \
    --video-size 720 1280 \
    --sample-n-frames 129 \
    --cfg-scale 7.5 \
    --infer-steps 30 \
    --use-deepcache 1 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/sp_audio_720p'
```

--------------------------------

### Download Whisper Audio Encoder Model

Source: https://github.com/tencent/hunyuancustom/blob/main/models/README.md

Download the whisper-tiny pretrained audio encoder model using the huggingface-cli tool. The model will be saved in the specified local directory.

```shell
cd HunyuanCustom
huggingface-cli download openai/whisper-tiny --local-dir ./models/whisper-tiny
```

--------------------------------

### Multi-GPU Batch Inference CLI (Video Editing)

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Enables multi-GPU batch inference for video editing tasks, such as replacing a subject in a video. Requires input video and mask video paths.

```bash
export MODEL_BASE="./models"
export PYTHONPATH=./

# ── Video-driven editing (replace subject in video, 8 GPUs) ────────────────
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/sed_red_panda.png' \
    --input-video './assets/input_videos/001_bg.mp4' \
    --mask-video './assets/input_videos/001_mask.mp4' \
    --expand-scale 5 \
    --video-condition \
    --pos-prompt "Realistic, High-quality. A red panda is walking on a stone road." \
    --neg-prompt "Aerial view, overexposed, low quality, blurring." \
    --ckpt ${MODEL_BASE}/hunyuancustom_editing_720P/mp_rank_00_model_states.pt \
    --seed 1024 \
    --infer-steps 50 \
    --flow-shift-eval-video 5.0 \
    --save-path './results/sp_editing_720p'
```

--------------------------------

### Multi-GPU Batch Inference CLI (Single-Subject)

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Runs multi-GPU batch inference for single-subject video customization. Requires setting model base path and Python path. Uses torchrun for distributed execution.

```bash
export MODEL_BASE="./models"
export PYTHONPATH=./

# ── Single-subject video customization (8 GPUs, 720p) ──────────────────────
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, overexposed, low quality, blurring, black border." \
    --ckpt ${MODEL_BASE}/hunyuancustom_720P/mp_rank_00_model_states.pt \
    --video-size 720 1280 \
    --sample-n-frames 129 \
    --seed 1024 \
    --cfg-scale 7.5 \
    --infer-steps 30 \
    --use-deepcache 1 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/sp_720p'
```

--------------------------------

### Parallel Inference: Video-Driven Video Customization

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Executes a script for video editing using 8 GPUs. Requires reference image, input video, mask video, prompts, checkpoint, and output path. The `--video-condition` flag enables video-driven customization.

```bash
cd HunyuanCustom

export MODEL_BASE="./models"
export PYTHONPATH=./
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/sed_red_panda.png' \
    --input-video './assets/input_videos/001_bg.mp4' \
    --mask-video './assets/input_videos/001_mask.mp4' \
    --expand-scale 5 \
    --video-condition \
    --pos-prompt "Realistic, High-quality. A red panda is walking on a stone road." \
    --neg-prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_editing_720P/mp_rank_00_model_states.pt" \
    --seed 1024 \
    --infer-steps 50 \
    --flow-shift-eval-video 5.0 \
    --save-path './results/sp_editing_720p'
    # --pose-enhance # Enable for human videos to improve pose generation quality.
```

--------------------------------

### Multi-GPU Batch Inference CLI

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Command-line interface for running inference at scale using `torchrun` for multi-GPU parallelism. Supports various customization options including single-subject, audio-driven, and video editing.

```APIDOC
## Multi-GPU Batch Inference CLI (`sample_batch.py`)

### Description
The recommended way to run inference at scale. Uses `torchrun` for sequence parallelism across multiple GPUs. Loads metadata from `--ref-image` / `--input-audio` / `--input-video` or from a JSON/list file via `--input-json`.

### Usage
```bash
export MODEL_BASE="./models"
export PYTHONPATH=./
```

#### Single-subject video customization (8 GPUs, 720p)
```bash
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, overexposed, low quality, blurring, black border." \
    --ckpt ${MODEL_BASE}/hunyuancustom_720P/mp_rank_00_model_states.pt \
    --video-size 720 1280 \
    --sample-n-frames 129 \
    --seed 1024 \
    --cfg-scale 7.5 \
    --infer-steps 30 \
    --use-deepcache 1 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/sp_720p'
```

#### Audio-driven (talking-head, 8 GPUs)
```bash
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/seg_man_01.png' \
    --input-audio './assets/audios/milk_man.mp3' \
    --audio-strength 0.8 \
    --audio-condition \
    --pos-prompt "Realistic, High-quality. A man delivers a product presentation." \
    --neg-prompt "Two people, overexposed, low quality, blurring." \
    --ckpt ${MODEL_BASE}/hunyuancustom_audio_720P/mp_rank_00_model_states.pt \
    --seed 1026 \
    --video-size 720 1280 \
    --sample-n-frames 129 \
    --cfg-scale 7.5 \
    --infer-steps 30 \
    --use-deepcache 1 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/sp_audio_720p'
```

#### Video-driven editing (replace subject in video, 8 GPUs)
```bash
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/sed_red_panda.png' \
    --input-video './assets/input_videos/001_bg.mp4' \
    --mask-video './assets/input_videos/001_mask.mp4' \
    --expand-scale 5 \
    --video-condition \
    --pos-prompt "Realistic, High-quality. A red panda is walking on a stone road." \
    --neg-prompt "Aerial view, overexposed, low quality, blurring." \
    --ckpt ${MODEL_BASE}/hunyuancustom_editing_720P/mp_rank_00_model_states.pt \
    --seed 1024 \
    --infer-steps 50 \
    --flow-shift-eval-video 5.0 \
    --save-path './results/sp_editing_720p'
```
```

--------------------------------

### Pad Image to LLaVA Input Size

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Resizes an image to a specific size (336x336) with a white background.

```python
llava_img = pad_image(img.copy(), size=(336, 336), color=(255, 255, 255))
# llava_img.shape == (336, 336, 3)
```

--------------------------------

### PyTorch Datasets for Video Generation Inputs

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Provides `ImageDataset`, `AudioDataset`, and `VideoDataset` for loading reference images, audio, and video inputs respectively. These datasets return standardized dictionaries for the `HunyuanVideoSampler.predict` method.

```python
from torch.utils.data import DataLoader
from torch.utils.data.distributed import DistributedSampler
from hymm_sp.data_kits.video_dataset import ImageDataset, AudioDataset, VideoDataset

# Assuming args and device are defined from previous steps
# device = "cuda"
# ... parse_args() ...

# --- Image dataset (single-subject mode) ---
dataset = ImageDataset(args, device=device)
# Each item: {"pixel_value_llava": T[3,336,336], "uncond_pixel_value_llava": T[3,336,336],
#             "pixel_value_ref": T[3,H,W], "prompt": str, "negative_prompt": str,
#             "seed": int, "name": str, "data_name": str}

# --- Audio dataset (talking-head mode) ---
# Requires: args.audio_condition=True, args.input_audio set
audio_dataset = AudioDataset(args, device=device)
# Extra key in each item: "audio_prompts": T[1,F,10,12,768], "audio_path": str

# --- Video dataset (video editing mode) ---
# Requires: args.video_condition=True, args.input_video and args.mask_video set
video_dataset = VideoDataset(args, device=device)
# Extra keys: "pixel_value_bg": T[3,T,H,W], "pixel_value_mask": T[3,T,H,W]
```

--------------------------------

### FP8 Single GPU Generation

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Run inference on a single GPU using FP8 precision. Ensure the model checkpoint and save path are correctly specified.

```bash
DISABLE_SP=1 python hymm_sp/sample_gpu_poor.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, overexposed, low quality, blurring." \
    --ckpt ${MODEL_BASE}/hunyuancustom_720P/mp_rank_00_model_states_fp8.pt \
    --video-size 512 896 \
    --seed 1024 \
    --sample-n-frames 129 \
    --infer-steps 30 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/1gpu_540p' \
    --use-fp8
```

--------------------------------

### Run Video Generation Inference with HunyuanVideoSampler

Source: https://context7.com/tencent/hunyuancustom/llms.txt

The main generation method that accepts text prompts, reference image latents, and optional conditioning tensors. Returns generated video frames as a tensor under the 'samples' key.

```python
import torch
from einops import rearrange
from hymm_sp.data_kits.data_tools import save_videos_grid
import cv2, numpy as np
from PIL import Image
import torchvision.transforms as T

# Preprocess reference image -> latents
def load_ref_image(image_path, video_size, vae, device, vae_dtype=torch.float16):
    img = cv2.cvtColor(cv2.imread(image_path), cv2.COLOR_BGR2RGB)
    h, w = video_size
    transform = T.Compose([
        T.Resize((h, w)),
        T.ToTensor(),
    ])
    ref = transform(Image.fromarray(img)).unsqueeze(0).to(device)  # [1,3,H,W]
    ref = ref * 2 - 1.0
    ref_for_vae = rearrange(ref, "b c h w -> b c 1 h w")
    with torch.autocast("cuda", dtype=vae_dtype):
        vae.enable_tiling()
        ref_latents = vae.encode(ref_for_vae).latent_dist.sample()
        uncond_ref_latents = vae.encode(torch.ones_like(ref_for_vae)).latent_dist.sample()
        vae.disable_tiling()
    ref_latents.mul_(vae.config.scaling_factor)
    uncond_ref_latents.mul_(vae.config.scaling_factor)
    return ref_latents, uncond_ref_latents

```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/tencent/hunyuancustom/blob/main/README.md

Create a new conda environment named 'HunyuanCustom' with Python 3.10.9 and activate it. This isolates project dependencies.

```shell
# 1. Create conda environment
conda create -n HunyuanCustom python==3.10.9

# 2. Activate the environment
conda activate HunyuanCustom
```

--------------------------------

### Single-GPU Low-VRAM Inference

Source: https://context7.com/tencent/hunyuancustom/llms.txt

Instructions for running inference on a single GPU with options for FP8 quantization and CPU offloading, suitable for lower VRAM configurations.

```APIDOC
## Single-GPU low-VRAM inference (`sample_gpu_poor.py`)

### Description
Runs inference on a single GPU with optional FP8 quantization and CPU offloading. Suitable for GPUs with 24 GB+ VRAM (FP8, 512×896) or any GPU via `--cpu-offload` (slow but functional at 8 GB VRAM with WanGP).

### Usage
```bash
export MODEL_BASE="./models"
export PYTHONPATH=./
```

*(Further details on specific command-line arguments and examples for `sample_gpu_poor.py` would typically follow here, but are not provided in the source text.)*
```