### Run Gradio Demo Locally

Source: https://github.com/wangzhiyaoo/svfr/blob/main/README.md

Starts the local Gradio web demo for SVFR. Ensure Gradio is installed via pip before running this command.

```bash
pip install gradio
python3 demo.py
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/wangzhiyaoo/svfr/blob/main/README.md

Installs all required Python packages listed in the 'requirements.txt' file. This ensures all necessary libraries are available for the project to run.

```bash
pip install -r requirements.txt
```

--------------------------------

### Install PyTorch with CUDA Support

Source: https://github.com/wangzhiyaoo/svfr/blob/main/README.md

Installs specific versions of PyTorch, Torchvision, and Torchaudio. It's crucial to select the CUDA version matching your hardware for GPU acceleration.

```bash
pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2
```

--------------------------------

### Setup Conda Environment for SVFR

Source: https://github.com/wangzhiyaoo/svfr/blob/main/README.md

Creates and activates a new conda environment named 'svfr' with Python 3.9. This is the initial step to set up the project's isolated environment.

```bash
conda create -n svfr python=3.9 -y
conda activate svfr
```

--------------------------------

### Launch Gradio Web Demo

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

Instructions on how to launch an interactive web interface for video face restoration using Gradio. The demo allows users to upload videos, select restoration tasks, and compare input with restored output.

```bash
# Run the demo server
# python3 demo.py

# The demo will start at http://localhost:1203

# Web interface features:
# - Video upload for low-quality input
# - Task selection checkboxes: BFR, Colorization, Inpainting
# - Optional inpainting mask image upload
# - Seed input for reproducibility
# - Random seed generation button
# - Side-by-side comparison of input face region and restored output
```

--------------------------------

### YAML Configuration for Inference Parameters

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

Provides a sample YAML configuration file for controlling inference parameters in the SVFR project. It includes settings for video clip processing, model checkpoint paths, and various inference parameters like data type, number of steps, batch sizes, and noise augmentation strengths.

```yaml
# config/infer.yaml

# Video clip configuration
data:
  n_sample_frames: 16  # Frames per processing batch
  width: 512
  height: 512

# Model checkpoint paths
pretrained_model_name_or_path: "models/stable-video-diffusion-img2vid-xt"
unet_checkpoint_path: "models/face_restoration/unet.pth"
id_linear_checkpoint_path: "models/face_restoration/id_linear.pth"
net_arcface_checkpoint_path: "models/face_restoration/insightface_glint360k.pth"

# Inference parameters
weight_dtype: 'fp16'           # fp16, fp32, or bf16
num_inference_steps: 30        # Denoising steps (more = higher quality)
decode_chunk_size: 16          # VAE decode batch size
overlap: 3                     # Frame overlap between batches
noise_aug_strength: 0.00       # Noise added to reference (0 = no motion)
min_appearance_guidance_scale: 2.0
max_appearance_guidance_scale: 2.0
i2i_noise_strength: 1.0        # Image-to-image noise strength
```

--------------------------------

### Download Stable Video Diffusion Checkpoints

Source: https://github.com/wangzhiyaoo/svfr/blob/main/README.md

Clones the Stable Video Diffusion model from HuggingFace using Git LFS. This command downloads the necessary pre-trained weights for image-to-video tasks.

```bash
conda install git-lfs
git lfs install
git clone https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt models/stable-video-diffusion-img2vid-xt
```

--------------------------------

### Command Line Inference for Video Face Restoration

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

The `infer.py` script enables command-line video face restoration. It supports basic BFR, combined BFR with colorization, and full restoration including inpainting. Input is a low-quality video, and output is a restored high-quality video, with optional frame extraction. Requires a configuration file and specifies task IDs.

```bash
python3 infer.py \
  --config config/infer.yaml \
  --task_ids 0 \
  --input_path ./assert/lq/lq1.mp4 \
  --output_dir ./results/ \
  --crop_face_region \
  --seed 77

# Combined BFR + Colorization (for grayscale videos)
python3 infer.py \
  --config config/infer.yaml \
  --task_ids 0,1 \
  --input_path ./assert/lq/lq2.mp4 \
  --output_dir ./results/ \
  --crop_face_region

# Full restoration: BFR + Colorization + Inpainting with mask
python3 infer.py \
  --config config/infer.yaml \
  --task_ids 0,1,2 \
  --input_path ./assert/lq/lq3.mp4 \
  --output_dir ./results/ \
  --mask_path ./assert/mask/lq3.png \
  --crop_face_region \
  --restore_frames

# Output files:
# ./results/{video_name}_{seed}_gen.mp4  - Restored video
# ./results/{video_name}_{seed}_ori.mp4  - Original input (cropped)
# ./results/result_frames/{video_name}_{seed}/  - Individual frames (if --restore_frames)
```

--------------------------------

### Python API Usage for Video Face Restoration

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

Demonstrates how to use the requests library to send a POST request to the SVFR API for video face restoration. It specifies the input video path, selected tasks (e.g., BFR, Colorization), an optional mask path, and a seed for reproducibility. The API is expected to return paths to the processed video.

```python
import requests

response = requests.post(
    "http://localhost:1203/api/predict",
    json={
        "data": [
            "path/to/video.mp4",        # Input video path
            ["BFR", "Colorization"],     # Selected tasks
            "path/to/mask.png",          # Optional mask
            77                           # Seed
        ]
    }
)
# Returns: [face_region_video_path, restored_video_path]
```

--------------------------------

### Face Cropping and Alignment Utilities

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

Provides utilities for processing face regions, including calculating the union of bounding boxes, expanding and squaring bounding boxes, and performing affine transformations for face alignment. It uses libraries like OpenCV and Pillow for image manipulation.

```python
from src.dataset.dataset import (
    get_affine_transform,
    mean_face_lm5p_256,
    get_union_bbox,
    process_bbox,
    crop_resize_img
)
import numpy as np
import cv2
from PIL import Image

# Get union bounding box across multiple frames
bbox_list = [
    [100, 50, 200, 180],   # Frame 1: [x1, y1, x2, y2]
    [95, 55, 195, 175],    # Frame 2
    [105, 48, 205, 182],   # Frame 3
]
union_bbox = get_union_bbox(bbox_list)  # Returns: [95, 48, 205, 182]

# Process bbox with expansion and square conversion
frame_height, frame_width = 720, 1280
processed_bbox = process_bbox(
    union_bbox,
    expand_radio=0.4,     # Expand by 40%
    height=frame_height,
    width=frame_width
)

# Crop and resize image to face region
img = Image.open("input.jpg")
cropped = crop_resize_img(img, processed_bbox)
cropped = cropped.resize((512, 512))

# Compute affine transform for face alignment
pts5 = np.array([
    [180, 200], [260, 198],  # Eyes
    [220, 260],               # Nose
    [185, 310], [255, 308]   # Mouth
], dtype=np.float32)

warp_mat = get_affine_transform(pts5, mean_face_lm5p_256)
aligned_face = cv2.warpAffine(
    np.array(img),
    warp_mat,
    (512, 512),
    flags=cv2.INTER_CUBIC
)
```

--------------------------------

### Video Saving Utilities

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

Provides utility functions to save generated video frames into MP4 or GIF formats. It allows configuration of the output framerate (FPS) and the arrangement of videos in a grid. The `seed_everything` function ensures reproducibility.

```python
from src.utils.util import save_videos_grid, seed_everything
import torch

# Set random seed for reproducibility
seed_everything(42)

# Save video tensor to file
# video: torch.Tensor of shape (batch, channels, frames, height, width)
# Values should be in range [0, 1]
video = torch.rand(1, 3, 30, 512, 512)  # Example: 30 frames at 512x512

save_videos_grid(
    video,
    path="output/restored_video.mp4",
    rescale=False,  # Set True if input is [-1, 1] range
    n_rows=1,       # Number of videos per row in grid
    fps=25          # Output framerate
)

# Save as GIF instead
save_videos_grid(
    video,
    path="output/restored_video.gif",
    rescale=False,
    n_rows=1,
    fps=8  # Lower FPS for GIF
)
```

--------------------------------

### Detect Faces and Extract Landmarks

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

Detects faces in a frame and returns their landmarks, confidence scores, and bounding boxes. The output includes a list of 5 landmarks (eyes, nose, mouth corners), bounding box coordinates, and a confidence score for each detected face.

```python
five_pts_list, scores_list, bboxes_list = align_instance(frame_bgr, maxface=True)

# five_pts_list[0]: numpy array of shape (5, 2) containing:
#   - Left eye, Right eye, Nose tip, Left mouth corner, Right mouth corner
# bboxes_list[0]: [x, y, width, height] of detected face
# scores_list[0]: confidence score

if five_pts_list:
    pts5 = five_pts_list[0]  # Shape: (5, 2)
    bbox = bboxes_list[0]    # [x, y, w, h]
    score = scores_list[0]   # Float confidence
    print(f"Face detected at {bbox} with confidence {score:.2f}")
    print(f"Landmarks: {pts5}")
```

--------------------------------

### LQ2VideoLongSVDPipeline for Video Face Restoration

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

This Python class implements the core diffusion pipeline for low-quality video face restoration. It inherits from `DiffusionPipeline` and integrates VAE, CLIP image encoder, and a 3D UNet for temporal consistency. It requires specific model weights and configuration for initialization and inference.

```python
from src.pipelines.pipeline import LQ2VideoLongSVDPipeline
from diffusers import AutoencoderKLTemporalDecoder, EulerDiscreteScheduler
from transformers import CLIPVisionModelWithProjection
from src.models.svfr_adapter.unet_3d_svd_condition_ip import UNet3DConditionSVDModel
import torch

# Initialize pipeline components
vae = AutoencoderKLTemporalDecoder.from_pretrained(
    "models/stable-video-diffusion-img2vid-xt",
    subfolder="vae",
    variant="fp16"
)
scheduler = EulerDiscreteScheduler.from_pretrained(
    "models/stable-video-diffusion-img2vid-xt",
    subfolder="scheduler"
)
image_encoder = CLIPVisionModelWithProjection.from_pretrained(
    "models/stable-video-diffusion-img2vid-xt",
    subfolder="image_encoder",
    variant="fp16"
)
unet = UNet3DConditionSVDModel.from_pretrained(
    "models/stable-video-diffusion-img2vid-xt",
    subfolder="unet",
    variant="fp16"
)

# Create pipeline
pipe = LQ2VideoLongSVDPipeline(
    unet=unet,
    image_encoder=image_encoder,
    vae=vae,
    scheduler=scheduler,
    feature_extractor=None
)
pipe = pipe.to("cuda", dtype=torch.float16)

# Run inference
# lq_frames: torch.Tensor of shape (num_frames, 3, 512, 512)
# id_embedding: torch.Tensor of shape (1, num_frames, 49, 1024)
# task_id_input: torch.Tensor [1, 0, 0] for BFR, [1, 1, 0] for BFR+Color
output = pipe(
    lq_frames.to("cuda").to(torch.float16),           # Low-quality input frames
    ref_img_tensor.to("cuda").to(torch.float16),      # Reference image for identity
    id_embedding.to("cuda").to(torch.float16),        # Identity embeddings
    task_id_input.to("cuda").to(torch.float16),       # Task selection tensor
    height=512,
    width=512,
    num_frames=total_frames,
    decode_chunk_size=16,
    noise_aug_strength=0.00,
    min_guidance_scale=2.0,
    max_guidance_scale=2.0,
    overlap=3,
    frames_per_batch=16,
    num_inference_steps=30,
    i2i_noise_strength=1.0,
)

video = output.frames  # Shape: (1, 3, num_frames, 512, 512)
```

--------------------------------

### AlignImage Face Detection and Landmark Extraction

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

The `AlignImage` class performs face detection and extracts 5-point facial landmarks using YOLOFace. It takes an image in BGR format as input and is initialized with a specified device and model path. This is crucial for aligning faces before further processing.

```python
from src.dataset.face_align.align import AlignImage
import numpy as np
from PIL import Image
import cv2

# Initialize face detector
align_instance = AlignImage(
    device="cuda",
    det_path="models/face_align/yoloface_v5m.pt"
)

# Load and detect face in image (BGR format expected)
frame = np.array(Image.open("input.jpg"))
frame_bgr = frame[:, :, [2, 1, 0]]  # RGB to BGR
```

--------------------------------

### Perform Single or Multi-Task Inference

Source: https://github.com/wangzhiyaoo/svfr/blob/main/README.md

Executes the inference script for various tasks (BFR, colorization, inpainting). It requires a configuration file, input path, output directory, and optionally allows face region cropping.

```bash
python3 infer.py \
 --config config/infer.yaml \
 --task_ids 0 \
 --input_path ./assert/lq/lq1.mp4 \
 --output_dir ./results/ \
 --crop_face_region
```

--------------------------------

### Perform Inference with Inpainting Mask

Source: https://github.com/wangzhiyaoo/svfr/blob/main/README.md

Runs inference with an additional inpainting mask. This allows for more precise control over the inpainting process by specifying a mask file.

```bash
python3 infer.py \
 --config config/infer.yaml \
 --task_ids 0,1,2 \
 --input_path ./assert/lq/lq3.mp4 \
 --output_dir ./results/ \
 --mask_path ./assert/mask/lq3.png \
 --crop_face_region
```

--------------------------------

### IDProjConvModel for Identity Embedding

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

Implements an IDProjConvModel to project facial identity features from InsightFace into a diffusion model's embedding space. This is crucial for identity-preserving restoration tasks. It involves initializing ArcFace, loading pretrained weights, and extracting identity features from aligned face images.

```python
from src.models.id_proj import IDProjConvModel
from src.models import model_insightface_360k
import torch
import torch.nn.functional as F

# Initialize identity encoder and projection
net_arcface = model_insightface_360k.getarcface(
    'models/face_restoration/insightface_glint360k.pth'
).eval().to("cuda")

id_linear = IDProjConvModel(
    in_channels=512,
    out_channels=1024
).to("cuda")

# Load pretrained weights
id_linear.load_state_dict(
    torch.load('models/face_restoration/id_linear.pth', map_location="cpu"),
    strict=True
)

# Extract identity features from aligned face image
# Input: face image tensor (1, 3, H, W), normalized to [-1, 1]
ref_img = torch.randn(1, 3, 512, 512).to("cuda")  # Example tensor

# Crop face region for ArcFace (112x112)
ref_img_crop = F.interpolate(
    ref_img[:, :, 0:224, 16:240],
    size=[112, 112],
    mode='bilinear'
)

# Get identity features
with torch.no_grad():
    _, id_feature_conv = net_arcface(ref_img_crop)  # Shape: (1, 512, 7, 7)
    id_embedding = id_linear(id_feature_conv)        # Shape: (1, 49, 1024)

# Expand for all video frames
num_frames = 30
id_embedding_expanded = id_embedding.unsqueeze(1).repeat(1, num_frames, 1, 1)
# Output shape: (1, num_frames, 49, 1024)
```

--------------------------------

### POST /api/predict

Source: https://context7.com/wangzhiyaoo/svfr/llms.txt

Submits a video for processing with specified tasks and optional parameters.

```APIDOC
## POST /api/predict

### Description
Submits a video for processing with specified tasks and optional parameters. The API handles video face restoration, colorization, and inpainting.

### Method
POST

### Endpoint
/api/predict

### Parameters
#### Request Body
- **data** (array) - Required - An array containing the input video path, selected tasks, optional mask path, and seed.
  - **data[0]** (string) - Required - Path to the input video file.
  - **data[1]** (array) - Required - A list of tasks to perform. Supported tasks include "BFR" (Blind Face Restoration), "Colorization", and "Inpainting".
  - **data[2]** (string) - Optional - Path to an optional mask image for inpainting.
  - **data[3]** (integer) - Optional - A seed for reproducibility.

### Request Example
```json
{
  "data": [
    "path/to/video.mp4",
    ["BFR", "Colorization"],
    "path/to/mask.png",
    77
  ]
}
```

### Response
#### Success Response (200)
- **[0]** (string) - Path to the processed face region video.
- **[1]** (string) - Path to the final restored video.

#### Response Example
```json
[
  "path/to/face_region_video.mp4",
  "path/to/restored_video.mp4"
]
```
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.