### Setup Environment and Dependencies

Source: https://github.com/bytedance-seed/seedvr/blob/main/readme.md

Commands to clone the repository, create a Conda environment, and install necessary Python dependencies including Flash Attention and Apex.

```bash
git clone https://github.com/bytedance-seed/SeedVR.git
cd SeedVR
conda create -n seedvr python=3.10 -y
conda activate seedvr
pip install -r requirements.txt
pip install flash_attn==2.5.9.post1 --no-build-isolation
pip install apex-0.1-cp310-cp310-linux_x86_64.whl
```

--------------------------------

### Initialize Distributed Training with Sequence Parallelism in PyTorch

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Initializes PyTorch distributed backend and sets up sequence parallelism for large-scale inference. It provides utilities to get device information, rank details, and synchronize processes across distributed environments. This setup is crucial for efficiently handling large models that require splitting across multiple GPUs.

```python
from common.distributed import (
    get_device,
    get_global_rank,
    get_local_rank,
    get_world_size,
    init_torch,
    barrier_if_distributed,
)
from common.distributed.advanced import (
    init_sequence_parallel,
    get_sequence_parallel_rank,
    get_sequence_parallel_world_size,
    get_data_parallel_rank,
    get_data_parallel_world_size,
)
import datetime

# Initialize PyTorch distributed
init_torch(
    cudnn_benchmark=False,  # Disable for deterministic results
    timeout=datetime.timedelta(seconds=3600)  # 1 hour timeout
)

# Print rank information
print(f"Global rank: {get_global_rank()} / {get_world_size()}")
print(f"Local rank: {get_local_rank()}")
print(f"Device: {get_device()}")

# Initialize sequence parallelism for multi-GPU inference
sp_size = 4  # Split sequence across 4 GPUs
init_sequence_parallel(sp_size)

print(f"Sequence parallel rank: {get_sequence_parallel_rank()} / {get_sequence_parallel_world_size()}")
print(f"Data parallel rank: {get_data_parallel_rank()} / {get_data_parallel_world_size()}")

# Synchronize all processes
barrier_if_distributed()

```

--------------------------------

### Download Pretrained Checkpoints

Source: https://github.com/bytedance-seed/seedvr/blob/main/readme.md

Python script using the huggingface_hub library to download model weights and configuration files for SeedVR2.

```python
from huggingface_hub import snapshot_download

save_dir = "ckpts/"
repo_id = "ByteDance-Seed/SeedVR2-3B"
cache_dir = save_dir + "/cache"

snapshot_download(cache_dir=cache_dir,
  local_dir=save_dir,
  repo_id=repo_id,
  local_dir_use_symlinks=False,
  resume_download=True,
  allow_patterns=["*.json", "*.safetensors", "*.pth", "*.bin", "*.py", "*.md", "*.txt"],
)
```

--------------------------------

### Download Pretrained Checkpoints

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Python script using huggingface_hub to download specific SeedVR or SeedVR2 model weights into a local directory.

```python
from huggingface_hub import snapshot_download

save_dir = "ckpts/"
repo_id = "ByteDance-Seed/SeedVR2-3B"
cache_dir = save_dir + "/cache"

snapshot_download(
    cache_dir=cache_dir,
    local_dir=save_dir,
    repo_id=repo_id,
    local_dir_use_symlinks=False,
    resume_download=True,
    allow_patterns=["*.json", "*.safetensors", "*.pth", "*.bin", "*.py", "*.md", "*.txt"],
)
```

--------------------------------

### Run Model Inference

Source: https://github.com/bytedance-seed/seedvr/blob/main/readme.md

Command to execute video restoration inference using torchrun to support multi-GPU sequence parallelism.

```bash
torchrun --nproc-per-node=NUM_GPUS projects/inference_seedvr2_3b.py --video_path INPUT_FOLDER --output_dir OUTPUT_FOLDER --seed SEED_NUM --res_h OUTPUT_HEIGHT --res_w OUTPUT_WIDTH --sp_size NUM_SP
```

--------------------------------

### Initialize NaDiT Model with SeedVR2 Configuration

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Initializes the NaDiT model with specific configurations for SeedVR2, including input/output channels, dimensions, attention heads, and layer types. Enables gradient checkpointing for memory efficiency and demonstrates a forward pass with sample inputs.

```python
model = NaDiT(
    vid_in_channels=33,       # Input channels (16 latent + 16 condition + 1 mask)
    vid_out_channels=16,      # Output latent channels
    vid_dim=2560,             # Hidden dimension
    txt_in_dim=5120,          # Text embedding input dimension
    txt_dim=2560,             # Text embedding projected dimension
    emb_dim=15360,            # Timestep embedding dimension (6 * vid_dim)
    heads=20,                 # Number of attention heads
    head_dim=128,             # Dimension per head
    expand_ratio=4,           # MLP expansion ratio
    norm="fusedrms",          # Normalization type
    norm_eps=1e-5,
    ada="single",             # Adaptive modulation type
    qk_bias=False,
    qk_norm="fusedrms",
    patch_size=(1, 2, 2),     # Temporal and spatial patch sizes
    num_layers=32,            # Number of transformer blocks
    mm_layers=10,             # Number of multimodal layers
    mlp_type="swiglu",        # MLP activation type
    block_type=["mmdit_sr"] * 32,  # Block type for each layer
    window=[(4, 3, 3)] * 32,  # Window sizes for attention
    rope_type="mmrope3d",     # 3D rotary position embedding
    rope_dim=128,
)

# Enable gradient checkpointing for memory efficiency
model.set_gradient_checkpointing(enable=True)

# Forward pass
# vid: flattened video latents (L, C) where L = sum of all token counts
# txt: text embeddings (L_txt, C_txt)
# vid_shape: (B, 3) tensor with [T, H, W] for each sample
# txt_shape: (B, 1) tensor with text lengths
# timestep: (B,) tensor with diffusion timesteps

output = model(
    vid=video_latents,      # (total_tokens, 33)
    txt=text_embeddings,    # (total_txt_tokens, 5120)
    vid_shape=vid_shapes,   # (batch_size, 3)
    txt_shape=txt_shapes,   # (batch_size, 1)
    timestep=timesteps,     # (batch_size,)
)

# Output contains predicted video sample
predicted_latent = output.vid_sample  # (total_tokens, 16)
```

--------------------------------

### Configure and Use Euler Sampler for Diffusion

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Sets up the Euler Sampler for iterative denoising using a predefined diffusion schedule and sampling timesteps. It demonstrates how to define a model function for the sampler and run the sampling process from initial noise.

```python
from common.diffusion import (
    EulerSampler,
    create_schedule_from_config,
    create_sampling_timesteps_from_config,
)
from common.distributed import get_device

# Create diffusion schedule
schedule = create_schedule_from_config(
    config={
        "type": "lerp",
        "T": 1000.0,  # Total timesteps
    },
    device=get_device(),
)

# Create sampling timesteps
sampling_timesteps = create_sampling_timesteps_from_config(
    config={
        "type": "uniform_trailing",
        "steps": 50,  # Number of sampling steps (1 for SeedVR2)
    },
    schedule=schedule,
    device=get_device(),
)

# Create Euler sampler
sampler = EulerSampler(
    schedule=schedule,
    timesteps=sampling_timesteps,
    prediction_type="v_lerp",  # Velocity prediction
)

# Sampling loop
# f: model forward function that takes SamplerModelArgs and returns prediction
def model_fn(args):
    # args.x_t: current noisy sample
    # args.t: current timestep
    # args.i: step index
    return model(args.x_t, timestep=args.t)

# Run sampling from noise
initial_noise = torch.randn(batch_size, channels, T, H, W, device=get_device())
restored_sample = sampler.sample(x=initial_noise, f=model_fn)
```

--------------------------------

### Run Distributed Video Inference

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Execution commands for running SeedVR2 and SeedVR inference using torchrun, supporting single and multi-GPU configurations with adjustable resolution and sequence parallelism.

```bash
# SeedVR2 3B Single GPU
torchrun --nproc-per-node=1 projects/inference_seedvr2_3b.py --video_path ./test_videos --output_dir ./results --seed 666 --res_h 720 --res_w 1280 --sp_size 1

# SeedVR 7B Multi-GPU
torchrun --nproc-per-node=4 projects/inference_seedvr_7b.py --video_path ./test_videos --output_dir ./results --seed 666 --res_h 1440 --res_w 2560 --sp_size 4
```

--------------------------------

### Initialize VideoDiffusionInfer Class

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Python code to programmatically initialize the inference runner, configure distributed settings, and set up sequence parallelism for custom inference pipelines.

```python
from projects.video_diffusion_sr.infer import VideoDiffusionInfer
from common.config import load_config
from common.distributed import get_device, init_torch
from common.distributed.advanced import init_sequence_parallel
import datetime

config = load_config('./configs_3b/main.yaml')
runner = VideoDiffusionInfer(config)
init_torch(cudnn_benchmark=False, timeout=datetime.timedelta(seconds=3600))
init_sequence_parallel(4)
```

--------------------------------

### Load and Customize YAML Configurations with OmegaConf

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Loads and modifies YAML configuration files for model architecture and diffusion parameters using OmegaConf. It supports inheritance resolution and allows runtime modifications to parameters like classifier-free guidance scale and sampling steps. The configuration can then be used to instantiate model objects.

```python
from common.config import load_config, create_object
from omegaconf import OmegaConf

# Load base configuration with inheritance resolution
config = load_config('./configs_3b/main.yaml')

# Make config mutable for runtime modifications
OmegaConf.set_readonly(config, False)

# Modify diffusion parameters
config.diffusion.cfg.scale = 7.5  # Classifier-free guidance scale
config.diffusion.timesteps.sampling.steps = 50  # Sampling steps

# Access model configuration
print(f"DiT layers: {config.dit.model.num_layers}")  # 32
print(f"DiT dimension: {config.dit.model.vid_dim}")  # 2560
print(f"DiT heads: {config.dit.model.heads}")  # 20

# Create model from config
# The config uses __object__ pattern for factory-style instantiation
dit_model = create_object(config.dit.model)

# Example config structure:
# __object__:
#   path: models.dit_v2.nadit
#   name: NaDiT
#   args: as_params
# vid_in_channels: 33
# vid_out_channels: 16
# vid_dim: 2560
# ...

```

--------------------------------

### Configure DiT, VAE, and Diffusion Models

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Configures the DiT and VAE models, sets memory limits for VAE operations, and adjusts diffusion parameters for SeedVR. This involves setting CUDA device, checkpoint paths, and specific diffusion configurations like CFG scale and sampling steps.

```python
runner.configure_dit_model(
    device="cuda",
    checkpoint='./ckpts/seedvr2_ema_3b.pth'
)

runner.configure_vae_model()

if hasattr(runner.vae, "set_memory_limit"):
    runner.vae.set_memory_limit(conv_max_mem=0.5, norm_max_mem=0.5)

runner.config.diffusion.cfg.scale = 1.0  # CFG scale (1.0 for SeedVR2)
runner.config.diffusion.cfg.rescale = 0.0
runner.config.diffusion.timesteps.sampling.steps = 1  # 1 step for SeedVR2, 50 for SeedVR
runner.configure_diffusion()
```

--------------------------------

### Citation BibTeX

Source: https://github.com/bytedance-seed/seedvr/blob/main/readme.md

BibTeX entries for citing the SeedVR and SeedVR2 research papers.

```bibtex
@inproceedings{wang2025seedvr2,
      title={SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training},
      author={Wang, Jianyi and Lin, Shanchuan and Lin, Zhijie and Ren, Yuxi and Wei, Meng and Yue, Zongsheng and Zhou, Shangchen and Chen, Hao and Zhao, Yang and Yang, Ceyuan and Xiao, Xuefeng and Loy, Chen Change and Jiang, Lu},
      booktitle={ICLR},
      year={2026}
   }

@inproceedings{wang2025seedvr,
      title={SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration},
      author={Wang, Jianyi and Lin, Zhijie and Wei, Meng and Zhao, Yang and Yang, Ceyuan and Loy, Chen Change and Jiang, Lu},
      booktitle={CVPR},
      year={2025}
   }
```

--------------------------------

### Run Full Inference Pipeline with PyTorch

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Executes the complete video inference loop, including text embeddings, noise generation, and video output restoration. It utilizes PyTorch for tensor operations and distributed synchronization, handling device placement and mixed-precision inference. The function returns a list of restored video tensors.

```python
import torch
import mediapy
from einops import rearrange
from common.distributed import get_device
from common.distributed.ops import sync_data
from common.seed import set_seed

def run_inference(runner, input_latents, seed=666):
    """
    Run full inference pipeline.

    Args:
        runner: VideoDiffusionInfer instance
        input_latents: List of encoded latent tensors
        seed: Random seed for reproducibility

    Returns:
        List of restored video tensors
    """
    # Set random seed for reproducibility
    set_seed(seed, same_across_ranks=True)

    # Generate noise
    noises = [torch.randn_like(latent) for latent in input_latents]
    aug_noises = [torch.randn_like(latent) for latent in input_latents]

    # Sync noise across distributed ranks
    noises, aug_noises, input_latents = sync_data(
        (noises, aug_noises, input_latents), 0
    )

    # Move to GPU
    noises = [n.to(get_device()) for n in noises]
    input_latents = [l.to(get_device()) for l in input_latents]

    # Load pre-computed text embeddings
    text_pos_embeds = torch.load('pos_emb.pt').to(get_device())
    text_neg_embeds = torch.load('neg_emb.pt').to(get_device())

    # Create conditions for super-resolution
    conditions = [
        runner.get_condition(noise, latent_blur=latent, task="sr")
        for noise, latent in zip(noises, input_latents)
    ]

    # Run inference with autocast for efficiency
    with torch.no_grad(), torch.autocast("cuda", torch.bfloat16, enabled=True):
        video_tensors = runner.inference(
            noises=noises,
            conditions=conditions,
            texts_pos=[text_pos_embeds],
            texts_neg=[text_neg_embeds],
            dit_offload=True,  # Offload DiT to CPU after inference
        )

    # Rearrange output tensors
    samples = [
        rearrange(video, "c t h w -> t c h w")
        for video in video_tensors
    ]

    return samples

# Example: Save restored video
samples = run_inference(runner, latents, seed=42)
sample = samples[0]

# Convert to uint8 and save
sample = sample.clip(-1, 1).mul_(0.5).add_(0.5).mul_(255)
sample = rearrange(sample, "t c h w -> t h w c")
sample = sample.to(torch.uint8).cpu().numpy()

mediapy.write_video("restored_video.mp4", sample, fps=30)

```

--------------------------------

### Video Transform Pipeline using PyTorch

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Applies a series of preprocessing transformations to video frames using PyTorch's torchvision and custom transforms. It resizes, clamps, crops, normalizes, and rearranges video data for compatibility with downstream models. Input is a video file, output is a transformed tensor.

```python
from torchvision.transforms import Compose, Lambda, Normalize
from torchvision.io.video import read_video
from data.image.transforms.divisible_crop import DivisibleCrop
from data.image.transforms.na_resize import NaResize
from data.video.transforms.rearrange import Rearrange
import torch

# Define target resolution
res_h, res_w = 720, 1280

# Create video transform pipeline
video_transform = Compose([
    # Resize to target resolution while maintaining aspect ratio
    NaResize(
        resolution=(res_h * res_w) ** 0.5,  # Geometric mean of target dimensions
        mode="area",  # Area-based resizing for quality
        downsample_only=False,  # Allow upsampling
    ),
    # Clamp values to valid range
    Lambda(lambda x: torch.clamp(x, 0.0, 1.0)),
    # Crop to dimensions divisible by 16 (required for VAE)
    DivisibleCrop((16, 16)),
    # Normalize to [-1, 1] range
    Normalize(0.5, 0.5),
    # Rearrange from (T, C, H, W) to (C, T, H, W)
    Rearrange("t c h w -> c t h w"),
])

# Read and transform video
video, audio, info = read_video("input_video.mp4", output_format="TCHW")
video = video / 255.0  # Convert to [0, 1]
fps = info["video_fps"]

# Apply transforms
transformed_video = video_transform(video)
print(f"Transformed video shape: {transformed_video.shape}")  # (C, T, H, W)
```

--------------------------------

### VAE Encoding and Decoding with PyTorch

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Encodes video frames into a latent space using the VAE and then decodes these latents back into video frames. This process utilizes PyTorch and the einops library for tensor manipulation, and assumes a 'runner' object with VAE capabilities and a 'get_device' function for device management. Input is a list of tensors, output is a list of restored video tensors.

```python
import torch
from einops import rearrange
from common.distributed import get_device

# Encode video to latent space
# Input: list of tensors with shape (C, T, H, W)
input_videos = [transformed_video.to(get_device())]

# Move VAE to GPU and encode
runner.vae.to(get_device())
latents = runner.vae_encode(input_videos)
# Output: list of tensors with shape (T', H', W', C') in latent space

print(f"Latent shape: {latents[0].shape}")  # Spatial downsampled by 8x, temporal by 4x

# Decode latents back to video
# Input: list of latent tensors
restored_videos = runner.vae_decode(latents)
# Output: list of tensors with shape (C, T, H, W)

# Post-process for saving
sample = restored_videos[0]
sample = rearrange(sample, "c t h w -> t c h w")
sample = sample.clip(-1, 1).mul_(0.5).add_(0.5).mul_(255).round()
sample = sample.to(torch.uint8)
```

--------------------------------

### Define NaDiT Model Architecture with PyTorch

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Defines the Native Resolution Diffusion Transformer (NaDiT) model architecture using PyTorch. This model is designed to support arbitrary video resolutions, enabling high-fidelity video generation and manipulation.

```python
import torch
from models.dit_v2.nadit import NaDiT

```

--------------------------------

### Condition Generation for Video Restoration Tasks

Source: https://context7.com/bytedance-seed/seedvr/llms.txt

Generates conditioning tensors required by diffusion models for various video restoration tasks like super-resolution (sr), image-to-video (i2v), video-to-video (v2v), and text-to-video (t2v). It takes latent representations and a task type to create a conditioning tensor with an added mask channel. Dependencies include PyTorch and a 'get_device' utility.

```python
import torch
from common.distributed import get_device

def get_condition(runner, latent, latent_blur, task="sr"):
    """
    Generate conditioning tensor for diffusion model.

    Args:
        latent: Target latent tensor (T, H, W, C)
        latent_blur: Degraded/blurred latent tensor for super-resolution
        task: One of "t2v", "i2v", "v2v", "sr"

    Returns:
        Conditioning tensor with mask channel (T, H, W, C+1)
    """
    t, h, w, c = latent.shape
    cond = torch.zeros([t, h, w, c + 1], device=latent.device, dtype=latent.dtype)

    if task == "sr":
        # Super-resolution: condition on all blurred frames
        cond[:, ..., :-1] = latent_blur[:]
        cond[:, ..., -1:] = 1.0  # Mask indicating valid conditioning
    elif task == "i2v":
        # Image-to-video: condition only on first frame
        cond[:1, ..., :-1] = latent[:1]
        cond[:1, ..., -1:] = 1.0
    elif task == "v2v":
        # Video-to-video: condition on first two frames
        cond[:2, ..., :-1] = latent[:2]
        cond[:2, ..., -1:] = 1.0
    elif task == "t2v":
        # Text-to-video: no frame conditioning
        pass

    return cond

# Example usage for super-resolution
noise = torch.randn_like(latents[0])
condition = get_condition(runner, noise, latents[0], task="sr")
print(f"Condition shape: {condition.shape}")  # (T, H, W, C+1)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.