### Install Conda Environment for E²FGVI

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Clone the repository and create a conda environment using the provided `environment.yml` file. Ensure Python >= 3.7, PyTorch >= 1.5, and CUDA >= 9.2 are installed. Pretrained weights should be placed in the `release_model/` directory.

```bash
git clone https://github.com/MCG-NKU/E2FGVI.git
cd E2FGVI

conda env create -f environment.yml
conda activate e2fgvi

# Requirements: Python >= 3.7, PyTorch >= 1.5, CUDA >= 9.2, mmcv-full
# Place pretrained weights into release_model/
# release_model/
#   |- E2FGVI-CVPR22.pth       (fixed 432x240 resolution)
#   |- E2FGVI-HQ-CVPR22.pth    (arbitrary resolution)
#   |- i3d_rgb_imagenet.pt      (for VFID evaluation)
```

--------------------------------

### Create Conda Environment and Install Dependencies

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Create a new Conda environment using the provided 'environment.yml' file and activate it. Ensure you have Python >= 3.7, PyTorch >= 1.5, and CUDA >= 9.2 installed.

```bash
conda env create -f environment.yml
conda activate e2fgvi
```

--------------------------------

### Train E2FGVI Model

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Use this command to start training the E2FGVI model. The configuration file path is required.

```shell
python train.py -c configs/train_e2fgvi.json
```

--------------------------------

### Run Quick Test with E2FGVI

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Execute this command to test the E2FGVI model with split video frames or mp4 format video. Ensure you have the model checkpoints and example data prepared.

```shell
# The first example (using split video frames)
python test.py --model e2fgvi (or e2fgvi_hq) --video examples/tennis --mask examples/tennis_mask  --ckpt release_model/E2FGVI-CVPR22.pth (or release_model/E2FGVI-HQ-CVPR22.pth)
# The second example (using mp4 format video)
python test.py --model e2fgvi (or e2fgvi_hq) --video examples/schoolgirls.mp4 --mask examples/schoolgirls_mask  --ckpt release_model/E2FGVI-CVPR22.pth (or release_model/E2FGVI-HQ-CVPR22.pth)
```

--------------------------------

### Train E2FGVI-HQ Model

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Use this command to start training the E2FGVI-HQ model. The configuration file path is required.

```shell
python train.py -c configs/train_e2fgvi_hq.json
```

--------------------------------

### Training E2FGVI-HQ Model (Single-GPU)

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Starts single-GPU training for the E2FGVI-HQ model, supporting arbitrary resolutions. Training can be resumed automatically by re-running the command.

```bash
# Single-GPU training for E2FGVI-HQ (arbitrary resolution)
python train.py -c configs/train_e2fgvi_hq.json
```

--------------------------------

### Monitor Training Loss with TensorBoard

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Starts TensorBoard to visualize training progress, including loss curves. Point TensorBoard to the directory where model checkpoints and logs are saved.

```bash
# Monitor training loss in TensorBoard
tensorboard --logdir release_model
```

--------------------------------

### Sample Global Reference Frame Indices

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Use this function to get reference frame indices sampled globally from a video. It's useful for tasks requiring consistent frame sampling across a video.

```python
from test import get_ref_index
import argparse, sys

# Patch globals for standalone use
import test as t_module
t_module.num_ref = -1     # use all global references
t_module.ref_length = 10  # sample every 10th frame

ref_ids = get_ref_index(f=0, neighbor_ids=[0,1,2,3,4,5], length=100)
# Returns: [10, 20, 30, 40, 50, 60, 70, 80, 90]  (frames not in neighbor_ids)
print(ref_ids)
```

--------------------------------

### Training E2FGVI Model (Single-GPU)

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Initiates single-GPU training for the E2FGVI model using a specified configuration file. The trainer automatically loads the latest checkpoint for resuming training.

```bash
# Single-GPU training for E2FGVI (fixed resolution, 432x240)
python train.py -c configs/train_e2fgvi.json
```

--------------------------------

### Multi-GPU Distributed Training with torchrun

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Launches multi-GPU distributed training for the E2FGVI model using torchrun. Specify the number of processes per node (GPUs) and the configuration file.

```bash
# Multi-GPU distributed training (e.g., 4 GPUs via torchrun)
torchrun --nproc_per_node=4 train.py -c configs/train_e2fgvi.json
```

--------------------------------

### Monitor Training Loss with TensorBoard

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Run this command to monitor training losses using TensorBoard. The log directory should be specified.

```shell
tensorboard --logdir release_model
```

--------------------------------

### evaluate.py

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Runs the model on the DAVIS or YouTube-VOS test split, computes per-video and aggregate metrics, and saves results to a text file.

```APIDOC
## evaluate.py

### Description
Full dataset evaluation with PSNR / SSIM / VFID reporting. Runs the model on the DAVIS or YouTube-VOS test split, computes per-video and aggregate metrics, and saves results to a text file.

### Usage
```bash
# Evaluate the standard E2FGVI model on DAVIS
python evaluate.py \
  --model    e2fgvi \
  --dataset  davis \
  --data_root datasets/ \
  --ckpt     release_model/E2FGVI-CVPR22.pth

# Evaluate E2FGVI-HQ on YouTube-VOS and save per-frame output images
python evaluate.py \
  --model    e2fgvi_hq \
  --dataset  youtube-vos \
  --data_root datasets/ \
  --ckpt     release_model/E2FGVI-HQ-CVPR22.pth \
  --save_results \
  --num_workers 8
```

### Results
Results are written to:
  `results/e2fgvi_davis/e2fgvi_davis_metrics.txt`

Example output:
```
[  1/ 50] Name: blackswan              | PSNR/SSIM: 34.12/0.9751
...
Finish evaluation... Average Frame PSNR/SSIM/VFID: 33.01/0.9721/0.116
```
```

--------------------------------

### Inference with E²FGVI-HQ Model (Custom Resolution)

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Use the E²FGVI-HQ model to inpaint a video and output it at a specific resolution, such as 720p. Use the `--set_size`, `--width`, and `--height` flags.

```bash
python test.py \
  --model e2fgvi_hq \
  --video examples/schoolgirls.mp4 \
  --mask  examples/schoolgirls_mask \
  --ckpt  release_model/E2FGVI-HQ-CVPR22.pth \
  --set_size --width 1280 --height 720
```

--------------------------------

### Initialize TrainDataset for YouTube-VOS/DAVIS

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Loads zipped JPEG frame archives, samples local and reference frame indices, generates random moving masks, and applies horizontal flip augmentation. Use for training.

```python
from core.dataset import TrainDataset

config = {
    "name": "youtube-vos",
    "data_root": "datasets",
    "w": 432,
    "h": 240,
    "num_local_frames": 5,
    "num_ref_frames": 3
}
dataset = TrainDataset(config)
frame_tensors, mask_tensors, video_name = dataset[0]
# frame_tensors: (T, C, H, W) = (8, 3, 240, 432) in [-1, 1]  (5 local + 3 ref)
# mask_tensors:  (T, 1, H, W) in [0, 1]
# video_name:    str, e.g. "0a1b2c3d"
print(frame_tensors.shape, mask_tensors.shape)
# torch.Size([8, 3, 240, 432])  torch.Size([8, 1, 240, 432])
```

--------------------------------

### Inference with E²FGVI Standard Model

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Run video inpainting on an mp4 file using the standard E²FGVI model, which outputs a fixed 432x240 resolution. Provide the video path, mask directory, and checkpoint path.

```bash
python test.py \
  --model e2fgvi \
  --video examples/schoolgirls.mp4 \
  --mask  examples/schoolgirls_mask \
  --ckpt  release_model/E2FGVI-CVPR22.pth
```

--------------------------------

### Initialize TestDataset for Evaluation

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Loads frames and ground-truth test masks (dilated) from zipped archives. Use for evaluation.

```python
import argparse
from core.dataset import TestDataset

args = argparse.Namespace(
    data_root="datasets/",
    dataset="davis",
    size=(432, 240)
)
dataset = TestDataset(args)
frames, masks, video_name, frames_PIL = dataset[0]
# frames:     (T, 3, 240, 432) in [-1, 1]
# masks:      (T, 1, 240, 432) in [0, 1]
# video_name: str, e.g. "blackswan"
# frames_PIL: list[ndarray] uint8, shape (240, 432, 3)
print(frames.shape)  # torch.Size([T, 3, 240, 432])
```

--------------------------------

### Evaluate E2FGVI Model

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Run this command to evaluate the E2FGVI model. Ensure you specify the dataset name, data root, and the path to the checkpoint.

```shell
python evaluate.py --model e2fgvi --dataset <dataset_name> --data_root datasets/ --ckpt release_model/E2FGVI-CVPR22.pth
```

--------------------------------

### train.py

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Single-GPU and Multi-GPU distributed training script.

```APIDOC
## train.py

### Description
Single-GPU and Multi-GPU distributed training. Reads a JSON config, sets up distributed training via PyTorch `mp.spawn` / OpenMPI, and runs the `Trainer` loop. Resume training automatically by re-running the same command; the trainer loads the latest checkpoint from `release_model/`.

### Usage
```bash
# Single-GPU training for E2FGVI (fixed resolution, 432x240)
python train.py -c configs/train_e2fgvi.json

# Single-GPU training for E2FGVI-HQ (arbitrary resolution)
python train.py -c configs/train_e2fgvi_hq.json

# Multi-GPU distributed training (e.g., 4 GPUs via torchrun)
torchrun --nproc_per_node=4 train.py -c configs/train_e2fgvi.json

# Monitor training loss in TensorBoard
tensorboard --logdir release_model
```

### Configuration
Key config fields (e.g., `configs/train_e2fgvi.json`):
```json
{
  "seed": 2021,
  "save_dir": "release_model/",
  "train_data_loader": { "name": "youtube-vos", "data_root": "datasets",
                         "w": 432, "h": 240,
                         "num_local_frames": 5, "num_ref_frames": 3 },
  "losses": { "hole_weight": 1, "valid_weight": 1,
              "flow_weight": 1, "adversarial_weight": 0.01,
              "GAN_LOSS": "hinge" },
  "model":   { "net": "e2fgvi", "no_dis": 0 },
  "trainer": { "lr": 1e-4, "batch_size": 8, "iterations": 500000,
               "save_freq": 5000, "log_freq": 100,
               "scheduler": { "type": "MultiStepLR",
                              "milestones": [400000], "gamma": 0.1 } }
}
```
```

--------------------------------

### Evaluate E2FGVI Model on DAVIS Dataset

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Runs the E2FGVI model on the DAVIS test split, computes per-video and aggregate metrics (PSNR, SSIM, VFID), and saves results to a text file. Specify model, dataset, data root, and checkpoint path.

```bash
# Evaluate the standard E2FGVI model on DAVIS
python evaluate.py \
  --model    e2fgvi \
  --dataset  davis \
  --data_root datasets/ \
  --ckpt     release_model/E2FGVI-CVPR22.pth
```

--------------------------------

### Clone E2FGVI Repository

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Use this command to clone the official repository for the E2FGVI project.

```bash
git clone https://github.com/MCG-NKU/E2FGVI.git
```

--------------------------------

### Evaluate E2FGVI-HQ Model

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Run this command to evaluate the E2FGVI-HQ model. Specify the dataset name, data root, and the checkpoint path.

```shell
python evaluate.py --model e2fgvi_hq --dataset <dataset_name> --data_root datasets/ --ckpt release_model/E2FGVI-HQ-CVPR22.pth
```

--------------------------------

### Evaluate E2FGVI-HQ on YouTube-VOS with Per-Frame Output

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Evaluates the E2FGVI-HQ model on the YouTube-VOS dataset and saves per-frame output images. Allows specifying the number of workers for parallel processing.

```bash
# Evaluate E2FGVI-HQ on YouTube-VOS and save per-frame output images
python evaluate.py \
  --model    e2fgvi_hq \
  --dataset  youtube-vos \
  --data_root datasets/ \
  --ckpt     release_model/E2FGVI-HQ-CVPR22.pth \
  --save_results \
  --num_workers 8
```

--------------------------------

### Custom Output Resolution for E2FGVI-HQ

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Use this command to specify custom output resolution for E2FGVI-HQ. This is useful when you need a specific video quality, such as 720p.

```shell
# Using this command to output a 720p video
python test.py --model e2fgvi_hq --video <video_path> --mask <mask_path>  --ckpt release_model/E2FGVI-HQ-CVPR22.pth --set_size --width 1280 --height 720
```

--------------------------------

### Compress Video Directories

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

This shell script is used to compress video directories within the datasets folder. Ensure the folder path is correctly edited before execution.

```shell
sh datasets/zip_dir.sh
```

--------------------------------

### Inference with E²FGVI-HQ Model (Preserves Resolution)

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Perform video inpainting using the E²FGVI-HQ model, which preserves the original video resolution. Input can be an mp4 video or a folder of frames.

```bash
python test.py \
  --model e2fgvi_hq \
  --video examples/tennis \
  --mask  examples/tennis_mask \
  --ckpt  release_model/E2FGVI-HQ-CVPR22.pth
```

--------------------------------

### Load Per-Frame Binary Masks

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

The `read_mask` function loads PNG mask images from a directory, resizes them, binarizes them, and applies morphological dilation. Masks are expected to be in the `masks/` directory.

```python
from test import read_mask
from PIL import Image
import numpy as np

# masks/ directory contains 00000.png, 00001.png, ... (white = region to fill)
masks = read_mask('examples/schoolgirls_mask', size=(432, 240))
# Returns: list of PIL.Image objects (mode 'L'), one per frame
# Each image: 0 = known region, 255 = hole to be filled (after dilation)
print(len(masks))          # e.g. 100
print(np.array(masks[0]).max())  # 255
```

--------------------------------

### Load Frames from Video or Image Folder

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

The `read_frame_from_videos` function detects whether the input is an `.mp4` file or a directory of images and returns a list of RGB PIL Images. Ensure the `args` object has the correct `video` path and `use_mp4` flag.

```python
import argparse
from test import read_frame_from_videos

args = argparse.Namespace(video='examples/schoolgirls.mp4', use_mp4=True)
frames = read_frame_from_videos(args)
# Returns: list[PIL.Image] in RGB, one per video frame
print(len(frames), frames[0].size)  # e.g. (100, (1280, 720))

# For a frame directory:
args2 = argparse.Namespace(video='examples/tennis', use_mp4=False)
frames2 = read_frame_from_videos(args2)
```

--------------------------------

### Generate Random Masks for Training

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Generates a sequence of binary PIL mask images with random shapes and motion. Pixel values are 255 for masked (hole) and 0 for known.

```python
from core.utils import create_random_shape_with_random_motion
import numpy as np

masks = create_random_shape_with_random_motion(
    video_length=8, imageHeight=240, imageWidth=432
)
# Returns: list[PIL.Image (mode 'L')], length = video_length
# Pixel value 255 = masked (hole), 0 = known
print(len(masks), np.array(masks[0]).shape)  # 8, (240, 432)
print(np.unique(np.array(masks[0])))          # [0, 255]
```

--------------------------------

### InpaintGenerator.forward_bidirect_flow

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Computes forward and backward optical flows between consecutive local frames using the learned SPyNet. Operates at 1/4 spatial resolution internally.

```APIDOC
## InpaintGenerator.forward_bidirect_flow(masked_local_frames)

### Description
Computes forward and backward optical flows between consecutive local frames using the learned SPyNet. Operates at 1/4 spatial resolution internally.

### Method
POST

### Endpoint
`/inpaint/flow` (Assumed, not explicitly stated)

### Parameters
#### Request Body
- **masked_local_frames** (tensor) - Required - Batch of masked local frames, shape `(B, L_t, 3, H, W)`, normalized to `[0, 1]`.

### Request Example
```python
import torch
import importlib

net = importlib.import_module('model.e2fgvi')
model = net.InpaintGenerator().to('cpu')
model.eval()

# masked_local_frames: (B, L_t, 3, H, W) in [0, 1]
local_frames = torch.rand(1, 5, 3, 240, 432)
with torch.no_grad():
    flows_fwd, flows_bwd = model.forward_bidirect_flow(local_frames)
```

### Response
#### Success Response (200)
- **flows_fwd** (tensor) - Forward optical flows, shape `(B, L_t-1, 2, H//4, W//4)`.
- **flows_bwd** (tensor) - Backward optical flows, shape `(B, L_t-1, 2, H//4, W//4)`.

### Response Example
```python
print(flows_fwd.shape)  # torch.Size([1, 4, 2, 60, 108])
print(flows_bwd.shape)  # torch.Size([1, 4, 2, 60, 108])
```
```

--------------------------------

### get_ref_index

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Samples global reference frame indices from a video at specified intervals. It takes the current frame index, local neighbor indices, and a length parameter to determine the sampling range.

```APIDOC
## get_ref_index(f, neighbor_ids, length)

### Description
Given the current frame index `f` and its local neighbor indices, returns a list of reference frame indices sampled globally from the video at intervals of `ref_length` (default: `step=10`).

### Parameters
#### Path Parameters
- **f** (int) - Required - The current frame index.
- **neighbor_ids** (list[int]) - Required - A list of local neighbor frame indices.
- **length** (int) - Required - The total length or number of frames to consider for sampling.

### Request Example
```python
from test import get_ref_index

# Patch globals for standalone use
import test as t_module
t_module.num_ref = -1     # use all global references
t_module.ref_length = 10  # sample every 10th frame

ref_ids = get_ref_index(f=0, neighbor_ids=[0,1,2,3,4,5], length=100)
print(ref_ids)
```

### Response
#### Success Response (200)
- **ref_ids** (list[int]) - A list of sampled global reference frame indices.

### Response Example
```
[10, 20, 30, 40, 50, 60, 70, 80, 90]
```
```

--------------------------------

### Calculate Video FID Score using I3D Activations

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Computes the Fréchet Inception Distance in video feature space using I3D activations. Requires a CUDA GPU and pre-trained I3D model. Activations are expected as 1D numpy arrays per video.

```python
from core.metrics import init_i3d_model, calculate_i3d_activations, calculate_vfid
from PIL import Image
import numpy as np

# Requires: release_model/i3d_rgb_imagenet.pt and a CUDA GPU
i3d_model = init_i3d_model()   # loads to cuda:0

# Per-video: collect activations (list of PIL.Image per video)
real_frames = [Image.fromarray(np.random.randint(0, 255, (240, 432, 3), dtype=np.uint8)) for _ in range(16)]
comp_frames = [Image.fromarray(np.random.randint(0, 255, (240, 432, 3), dtype=np.uint8)) for _ in range(16)]

real_act, comp_act = calculate_i3d_activations(real_frames, comp_frames, i3d_model, device='cuda')
# real_act, comp_act: 1D numpy arrays of I3D logit features

# Aggregate over all videos, then compute VFID
real_all = [real_act]   # one entry per test video
comp_all = [comp_act]
vfid = calculate_vfid(real_all, comp_all)
print(f"VFID: {vfid:.3f}")   # lower is better; paper reports ~0.116 on DAVIS
```

--------------------------------

### InpaintGenerator

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

The core inpainting model (forward pass). It accepts a batch of masked video frames and the number of local frames, returning reconstructed frames and predicted optical flows. Input frames must be normalized to [-1, 1] and padded.

```APIDOC
## InpaintGenerator

### Description
The generator accepts a batch of masked video frames (local + reference concatenated along the time dimension) and the number of local frames, and returns reconstructed frames plus predicted optical flows. Input frames must be normalized to `[-1, 1]` and padded to multiples of `(mod_size_h=60, mod_size_w=108)` before inference.

### Method
POST

### Endpoint
`/inpaint` (Assumed, not explicitly stated)

### Parameters
#### Request Body
- **masked_imgs** (tensor) - Required - Batch of masked input frames, shape `(B, T, C, H, W)`, normalized to `[-1, 1]`. `T` is the total number of frames (local + reference).
- **T_local** (int) - Required - The number of local frames in the input batch.

### Request Example
```python
import torch
import importlib

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the standard model
net = importlib.import_module('model.e2fgvi')
model = net.InpaintGenerator().to(device)
model.load_state_dict(torch.load('release_model/E2FGVI-CVPR22.pth', map_location=device))
model.eval()

# Prepare input tensors
# imgs:  (B, T, C, H, W) in [-1, 1], where T = num_local + num_ref frames
# masks: (B, T, 1, H, W) in [0, 1], 1 = hole
B, T_local, T_ref = 1, 5, 3
T = T_local + T_ref
imgs  = torch.randn(B, T, 3, 240, 432).to(device)   # normalized frames
masks = torch.randint(0, 2, (B, T, 1, 240, 432)).float().to(device) # 1 = hole

masked_imgs = imgs * (1 - masks)

# Padding to multiples of (60, 108)
h, w = 240, 432
h_pad = (60 - h % 60) % 60   # 0
w_pad = (108 - w % 108) % 108 # 0

with torch.no_grad():
    pred_imgs, pred_flows = model(masked_imgs, T_local)
```

### Response
#### Success Response (200)
- **pred_imgs** (tensor) - Reconstructed frames, shape `(B*T, 3, H, W)`, normalized to `[-1, 1]`. Only local frames are inpainted.
- **pred_flows** (tuple) - A tuple containing forward and backward optical flows. Each element has shape `(B, T_local-1, 2, H/4, W/4)`.

### Response Example
```python
# Output after rescaling to [0, 1]
output = (pred_imgs + 1) / 2
print(output.shape)            # torch.Size([8, 3, 240, 432])  (B*T frames)
print(pred_flows[0].shape)     # torch.Size([1, 4, 2, 60, 108])
```
```

--------------------------------

### Compose Stack + ToTorchFormatTensor Transform

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Creates a torchvision.transforms.Compose object that stacks a list of PIL Images into a tensor of shape (T, C, H, W) in [0, 1]. Normalize to [-1, 1] separately if needed for model input.

```python
from core.utils import to_tensors
from PIL import Image
import numpy as np

frames = [Image.fromarray(np.random.randint(0, 255, (240, 432, 3), dtype=np.uint8)) for _ in range(5)]
transform = to_tensors()
tensor = transform(frames)   # shape: (5, 3, 240, 432)  dtype: float32  range: [0, 1]
# Normalize to [-1, 1] for model input:
tensor = tensor * 2.0 - 1.0
print(tensor.shape, tensor.min().item(), tensor.max().item())
# torch.Size([5, 3, 240, 432])  ~-1.0  ~1.0
```

--------------------------------

### Discriminator 3D Temporal Patch GAN

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Initializes and uses the 3D Temporal Patch GAN discriminator for training. This discriminator operates on video clips and is used for computing adversarial loss.

```python
import torch
from model.e2fgvi import Discriminator

disc = Discriminator(in_channels=3, use_sigmoid=False).cuda()
disc.train()

# real_clip / fake_clip: (B, T, C, H, W)
real_clip = torch.randn(2, 8, 3, 240, 432).cuda()
fake_clip  = torch.randn(2, 8, 3, 240, 432).cuda()

real_out = disc(real_clip)  # (B, T', C', H', W') — patch scores
fake_out = disc(fake_clip)
print(real_out.shape)       # e.g. torch.Size([2, 8, 128, 1, 1])
```

--------------------------------

### calc_psnr_and_ssim

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Computes Peak Signal-to-Noise Ratio and Structural Similarity Index between two uint8 RGB numpy arrays of identical shape.

```APIDOC
## calc_psnr_and_ssim(img1, img2)

### Description
Computes Peak Signal-to-Noise Ratio and Structural Similarity Index between two uint8 RGB numpy arrays of identical shape.

### Parameters
- **img1** (numpy.ndarray) - The first input image.
- **img2** (numpy.ndarray) - The second input image.

### Returns
- **psnr** (float) - The calculated PSNR value.
- **ssim** (float) - The calculated SSIM value.

### Example
```python
import numpy as np
from core.metrics import calc_psnr_and_ssim

# Simulate a ground-truth frame and a slightly noisy prediction
gt   = np.random.randint(0, 256, (240, 432, 3), dtype=np.uint8)
pred = np.clip(gt.astype(np.int32) + np.random.randint(-5, 5, gt.shape), 0, 255).astype(np.uint8)

psnr, ssim = calc_psnr_and_ssim(gt, pred)
print(f"PSNR: {psnr:.2f} dB")   # e.g. PSNR: 34.15 dB
print(f"SSIM: {ssim:.4f}")      # e.g. SSIM: 0.9834
```
```

--------------------------------

### calculate_vfid

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Computes the Fréchet Inception Distance in video feature space using I3D activations collected across all test videos.

```APIDOC
## calculate_vfid(real_activations, fake_activations)

### Description
Computes the Fréchet Inception Distance in video feature space using I3D activations collected across all test videos.

### Parameters
- **real_activations** (list) - A list of 1D numpy arrays representing I3D activations for real videos.
- **fake_activations** (list) - A list of 1D numpy arrays representing I3D activations for fake (generated) videos.

### Returns
- **vfid** (float) - The calculated Video FID score.

### Example
```python
from core.metrics import init_i3d_model, calculate_i3d_activations, calculate_vfid
from PIL import Image
import numpy as np

# Requires: release_model/i3d_rgb_imagenet.pt and a CUDA GPU
i3d_model = init_i3d_model()   # loads to cuda:0

# Per-video: collect activations (list of PIL.Image per video)
real_frames = [Image.fromarray(np.random.randint(0, 255, (240, 432, 3), dtype=np.uint8)) for _ in range(16)]
comp_frames = [Image.fromarray(np.random.randint(0, 255, (240, 432, 3), dtype=np.uint8)) for _ in range(16)]

real_act, comp_act = calculate_i3d_activations(real_frames, comp_frames, i3d_model, device='cuda')
# real_act, comp_act: 1D numpy arrays of I3D logit features

# Aggregate over all videos, then compute VFID
real_all = [real_act]   # one entry per test video
comp_all = [comp_act]
vfid = calculate_vfid(real_all, comp_all)
print(f"VFID: {vfid:.3f}")   # lower is better; paper reports ~0.116 on DAVIS
```
```

--------------------------------

### AdversarialLoss

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Wraps BCE, MSE, or hinge criterion and handles the real/fake label bookkeeping for both the discriminator and generator update steps.

```APIDOC
## AdversarialLoss

### Description
GAN loss (nsgan / lsgan / hinge). Wraps BCE, MSE, or hinge criterion and handles the real/fake label bookkeeping for both the discriminator and generator update steps.

### Method
POST

### Endpoint
`/loss/adversarial` (Assumed, not explicitly stated)

### Parameters
#### Request Body
- **scores** (tensor) - Required - Predicted scores from the discriminator or generator.
- **is_real** (bool) - Required - True if the scores are for real data, False for fake data.
- **is_disc** (bool) - Required - True if calculating loss for the discriminator, False for the generator.
- **type** (string) - Optional - Type of adversarial loss ('hinge', 'nsgan', 'lsgan'). Defaults to 'hinge'.

### Request Example
```python
from core.loss import AdversarialLoss
import torch

# Hinge loss (used in default config)
adv_loss = AdversarialLoss(type='hinge')

fake_scores = torch.randn(2, 8, 128, 1, 1)
real_scores = torch.randn(2, 8, 128, 1, 1)

# Discriminator update
dis_real_loss = adv_loss(real_scores, is_real=True,  is_disc=True)
dis_fake_loss = adv_loss(fake_scores, is_real=False, is_disc=True)
dis_loss = (dis_real_loss + dis_fake_loss) / 2

# Generator update
gen_scores = torch.randn(2, 8, 128, 1, 1)
gan_loss = adv_loss(gen_scores, is_real=True, is_disc=False)
```

### Response
#### Success Response (200)
- **loss** (float) - The calculated adversarial loss value.

### Response Example
```python
print(dis_loss.item(), gan_loss.item())  # e.g. 0.512  0.034
```
```

--------------------------------

### AdversarialLoss for GAN Training

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Implements GAN loss (nsgan, lsgan, hinge) for training. This class handles label bookkeeping for both discriminator and generator updates.

```python
from core.loss import AdversarialLoss
import torch

# Hinge loss (used in default config)
adv_loss = AdversarialLoss(type='hinge')

fake_scores = torch.randn(2, 8, 128, 1, 1)
real_scores = torch.randn(2, 8, 128, 1, 1)

# Discriminator update
dis_real_loss = adv_loss(real_scores, is_real=True,  is_disc=True)
dis_fake_loss = adv_loss(fake_scores, is_real=False, is_disc=True)
dis_loss = (dis_real_loss + dis_fake_loss) / 2

# Generator update
gen_scores = torch.randn(2, 8, 128, 1, 1)
gan_loss = adv_loss(gen_scores, is_real=True, is_disc=False)

print(dis_loss.item(), gan_loss.item())  # e.g. 0.512  0.034
```

--------------------------------

### E2FGVI InpaintGenerator Bidirectional Optical Flow

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Computes forward and backward optical flows between consecutive local frames using SPyNet. This operation is performed internally at 1/4 spatial resolution.

```python
import torch, importlib

net = importlib.import_module('model.e2fgvi')
model = net.InpaintGenerator().to('cpu')
model.eval()

# masked_local_frames: (B, L_t, 3, H, W) in [0, 1]
local_frames = torch.rand(1, 5, 3, 240, 432)
with torch.no_grad():
    flows_fwd, flows_bwd = model.forward_bidirect_flow(local_frames)
# flows_fwd: (B, L_t-1, 2, H//4, W//4)
# flows_bwd: (B, L_t-1, 2, H//4, W//4)
print(flows_fwd.shape)  # torch.Size([1, 4, 2, 60, 108])
```

--------------------------------

### Calculate PSNR and SSIM for Image Pairs

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Computes Peak Signal-to-Noise Ratio and Structural Similarity Index between two uint8 RGB numpy arrays of identical shape. Ensure input arrays are of the same dimensions.

```python
import numpy as np
from core.metrics import calc_psnr_and_ssim

# Simulate a ground-truth frame and a slightly noisy prediction
gt   = np.random.randint(0, 256, (240, 432, 3), dtype=np.uint8)
pred = np.clip(gt.astype(np.int32) + np.random.randint(-5, 5, gt.shape), 0, 255).astype(np.uint8)

psnr, ssim = calc_psnr_and_ssim(gt, pred)
print(f"PSNR: {psnr:.2f} dB")   # e.g. PSNR: 34.15 dB
print(f"SSIM: {ssim:.4f}")      # e.g. SSIM: 0.9834
```

--------------------------------

### E2FGVI InpaintGenerator Forward Pass

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Performs the forward pass of the InpaintGenerator model for video inpainting. Input frames must be normalized to [-1, 1] and padded to multiples of (60, 108).

```python
import torch
import importlib

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the standard model
net = importlib.import_module('model.e2fgvi')
model = net.InpaintGenerator().to(device)
model.load_state_dict(torch.load('release_model/E2FGVI-CVPR22.pth', map_location=device))
model.eval()

# Prepare input tensors
# imgs:  (B, T, C, H, W) in [-1, 1], where T = num_local + num_ref frames
# masks: (B, T, 1, H, W) in [0, 1], 1 = hole
B, T_local, T_ref = 1, 5, 3
T = T_local + T_ref
imgs  = torch.randn(B, T, 3, 240, 432).to(device)   # normalized frames
masks = torch.randint(0, 2, (B, T, 1, 240, 432)).float().to(device)

masked_imgs = imgs * (1 - masks)

# Padding to multiples of (60, 108)
h, w = 240, 432
h_pad = (60 - h % 60) % 60   # 0
w_pad = (108 - w % 108) % 108 # 0

with torch.no_grad():
    pred_imgs, pred_flows = model(masked_imgs, T_local)
# pred_imgs:  (B*T, 3, H, W) in [-1, 1]  — only local frames are inpainted
# pred_flows: tuple of (forward_flows, backward_flows), each (B, T_local-1, 2, H/4, W/4)

output = (pred_imgs + 1) / 2   # rescale to [0, 1]
print(output.shape)            # torch.Size([8, 3, 240, 432])  (B*T frames)
print(pred_flows[0].shape)     # torch.Size([1, 4, 2, 60, 108])
```

--------------------------------

### Discriminator

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

A 3D Temporal Patch GAN discriminator used only during training to compute adversarial (hinge) loss. It operates on video clips.

```APIDOC
## Discriminator

### Description
A 3D convolutional discriminator with spectral normalization that operates on video clips `(B, T, C, H, W)`. Used only during training to compute adversarial (hinge) loss.

### Method
POST

### Endpoint
`/discriminator/predict` (Assumed, not explicitly stated)

### Parameters
#### Request Body
- **clip** (tensor) - Required - Video clip tensor, shape `(B, T, C, H, W)`.

### Request Example
```python
import torch
from model.e2fgvi import Discriminator

disc = Discriminator(in_channels=3, use_sigmoid=False).cuda()
disc.train()

# real_clip / fake_clip: (B, T, C, H, W)
real_clip = torch.randn(2, 8, 3, 240, 432).cuda()
fake_clip  = torch.randn(2, 8, 3, 240, 432).cuda()

real_out = disc(real_clip)  # (B, T', C', H', W') — patch scores
fake_out = disc(fake_clip)
```

### Response
#### Success Response (200)
- **scores** (tensor) - Patch scores for the input video clip, shape `(B, T', C', H', W')`.

### Response Example
```python
print(real_out.shape)       # e.g. torch.Size([2, 8, 128, 1, 1])
print(fake_out.shape)       # e.g. torch.Size([2, 8, 128, 1, 1])
```
```

--------------------------------

### Resize a List of Frames

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

The `resize_frames` function resizes all frames in a list to a specified (width, height). If `size` is `None`, the original frame sizes are returned.

```python
from test import resize_frames

frames_resized, size = resize_frames(frames, size=(432, 240))
```

--------------------------------

### BibTeX Citation for E2FGVI Paper

Source: https://github.com/mcg-nku/e2fgvi/blob/master/README.md

Use this BibTeX entry for citing the E2FGVI paper in your research.

```bibtex
@inproceedings{liCvpr22vInpainting,
title={Towards an End-to-End Framework for Flow-Guided Video Inpainting},
author={Li, Zhen and Lu, Cheng-Ze and Qin, Jianhua and Guo, Chun-Le and Cheng, Ming-Ming},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2022}
}
```

--------------------------------

### calculate_epe

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Computes the mean Euclidean distance between two flow fields (used internally during training to monitor flow completion quality).

```APIDOC
## calculate_epe(flow1, flow2)

### Description
Computes the mean Euclidean distance between two flow fields (used internally during training to monitor flow completion quality).

### Parameters
- **flow1** (torch.Tensor) - The first optical flow field.
- **flow2** (torch.Tensor) - The second optical flow field.

### Returns
- **epe** (float) - The mean End-Point Error in pixels.

### Example
```python
import torch
from core.metrics import calculate_epe

flow_gt   = torch.randn(4, 2, 60, 108)   # (N, 2, H, W)
flow_pred = torch.randn(4, 2, 60, 108)

epe = calculate_epe(flow_gt, flow_pred)
print(f"EPE: {epe:.4f}")   # mean endpoint error in pixels
```
```

--------------------------------

### Calculate End-Point Error for Optical Flow

Source: https://context7.com/mcg-nku/e2fgvi/llms.txt

Computes the mean Euclidean distance between two flow fields. Used internally during training to monitor flow completion quality. Input tensors should be of shape (N, 2, H, W).

```python
import torch
from core.metrics import calculate_epe

flow_gt   = torch.randn(4, 2, 60, 108)   # (N, 2, H, W)
flow_pred = torch.randn(4, 2, 60, 108)

epe = calculate_epe(flow_gt, flow_pred)
print(f"EPE: {epe:.4f}")   # mean endpoint error in pixels
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.