### Install STORM Environment

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Set up the necessary environment by cloning the repository, creating a conda environment, and installing Python dependencies including gsplat for rendering.

```bash
# Clone repository
git clone https://github.com/NVlabs/GaussianSTORM.git
cd GaussianSTORM

# Create conda environment
conda create -n storm python=3.10 -y
conda activate storm

# Install Python dependencies
pip install -r requirements.txt

# Install gsplat for batch-wise Gaussian rendering
pip install git+https://github.com/nerfstudio-project/gsplat.git@2b0de894232d21e8963179a7bbbd315f27c52c9c
```

--------------------------------

### Multi-GPU Training Example

Source: https://github.com/nvlabs/gaussianstorm/blob/main/README.md

Launches a multi-GPU training process to reproduce the paper's STORM-B/8 model. Adjust data_root and batch_size as needed. This example uses a global batch size of 32 (8 GPUs * 4 batch size per GPU).

```bash
# with a global batch size= num_gpus * batch_size = 8 * 4 = 32 (We used 64 global batch size for main experiments)
torchrun --nproc_per_node=8 main_storm.py \
    --project 0504_storm \
    --exp_name 0504_pixel_storm \
    --data_root ../storm2.3/data/STORM2 \
    --batch_size 4 --num_iterations 100000 --lr_sched constant \
    --model STORM-B/8 --num_motion_tokens 16 \
    --use_sky_token --use_affine_token \
    --load_depth --load_flow --load_ground \
    --enable_depth_loss --enable_flow_reg_loss --flow_reg_coeff 0.005 --enable_sky_opacity_loss \
    --enable_perceptual_loss --perceptual_loss_start_iter 5000 \
    --enable_wandb \
    --auto_resume
```

--------------------------------

### Set up Data Processing Environment

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Create and activate a Conda environment for data preprocessing, install necessary dependencies, and download raw data using provided scripts. This is a prerequisite for preprocessing the Waymo dataset.

```bash
# 1. Create data processing environment (separate from main env)
conda create -n storm_data python=3.10 -y
conda activate storm_data
pip install -r requirements_data_preprocess.txt

# 2. Download raw data (requires Waymo account and gcloud SDK)
# Download specific scenes
python preproc/waymo_download.py \
    --target_dir ./data/waymo/raw/training \
    --split_file data/dataset_scene_list/waymo_train_list.txt \
    --scene_ids 700 754 23

# Or download all training scenes
python preproc/waymo_download.py \
    --target_dir ./data/waymo/raw/training \
    --split_file data/dataset_scene_list/waymo_train_list.txt
```

--------------------------------

### Setup Optimizer and Loss Scaler for Training

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Configures the optimizer (AdamW with weight decay) and a native gradient scaler for mixed-precision training. A perceptual loss (LPIPS) is also set up.

```python
# Setup optimizer
param_groups = optim_factory.param_groups_weight_decay(model.module, weight_decay=0.05)
optimizer = torch.optim.AdamW(param_groups, lr=4e-4, betas=(0.9, 0.95))
loss_scaler = NativeScaler()

# Setup perceptual loss (LPIPS)
rgb_lpips_loss = RGBLpipsLoss(
    perceptual_weight=0.05,
    enable_perceptual_loss=True,
).to(device)
```

--------------------------------

### Multi-GPU Evaluation Example

Source: https://github.com/nvlabs/gaussianstorm/blob/main/README.md

Runs the evaluation mode for the trained STORM model on multiple GPUs. This command is similar to the training command but includes the `--evaluate` flag.

```bash
torchrun --nproc_per_node=8 main_storm.py \
    --project 0504_storm \
    --exp_name 0504_pixel_storm \
    --data_root ../storm2.3/data/STORM2 \
    --batch_size 4 --num_iterations 100000 --lr_sched constant \
    --model STORM-B/8 --num_motion_tokens 16 \
    --use_sky_token --use_affine_token \
    --load_depth --load_flow --load_ground \
    --enable_depth_loss --enable_flow_reg_loss --flow_reg_coeff 0.005 --enable_sky_opacity_loss \
    --enable_perceptual_loss --perceptual_loss_start_iter 5000 \
    --auto_resume \
    --evaluate
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/nvlabs/gaussianstorm/blob/main/README.md

Installs the GaussianSTORM project and its Python dependencies, including gsplat for batch-wise rendering. Ensure CUDA and PyTorch versions are compatible with your environment.

```bash
git clone https://github.com/NVlabs/GaussianSTORM.git
cd GaussianSTORM

conda create -n storm python=3.10 -y
conda activate storm

pip install -r requirements.txt

pip install git+https://github.com/nerfstudio-project/gsplat.git@2b0de894232d21e8963179a7bbbd315f27c52c9c
```

```bash
#   └─ if the above fails, drop the commit hash:
#       pip install git+https://github.com/nerfstudio-project/gsplat.git
```

--------------------------------

### Setup Data Processing Environment

Source: https://github.com/nvlabs/gaussianstorm/blob/main/docs/WAYMO.md

Create a dedicated Conda environment to avoid dependency conflicts with the main project environment.

```bash
conda create -n storm_data python=3.10
conda activate storm_data
pip install -r requirements_data_preprocess.txt
```

--------------------------------

### Configure STORM Dataset and DataLoader

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Initializes the STORM dataset with specific temporal and spatial parameters and wraps it in an infinite sampler for continuous training.

```python
dataset = STORMDataset(
    data_root="./data/STORM2",
    annotation_txt_file_list="./data/STORM2/scene_list/waymo_train.txt",
    target_size=(160, 240),
    num_context_timesteps=4,
    num_target_timesteps=4,
    num_max_cams=3,
    timespan=2.0,
    load_depth=True,
    load_flow=True,
)
sampler = InfiniteSampler(sample_count=len(dataset), shuffle=True, seed=42)
dataloader = DataLoader(
    dataset,
    sampler=sampler,
    batch_size=4,
    num_workers=16,
    pin_memory=True,
    drop_last=True,
)
```

--------------------------------

### Initialize Perceptual Loss Module

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Initializes the RGBLpipsLoss module for perceptual similarity. Configure weights and enable/disable based on training arguments.

```python
from storm.utils.lpips_loss import RGBLpipsLoss
import torch

# Initialize perceptual loss module
lpips_loss = RGBLpipsLoss(
    perceptual_weight=0.05,      # Weight for LPIPS loss
    enable_perceptual_loss=True,
).to(device)
```

--------------------------------

### Initialize Distributed Training for STORM

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Sets up distributed training using PyTorch's distributed package with NCCL backend. It initializes the process group, determines the local CUDA device, and creates the STORM model.

```python
import torch
import torch.distributed
from torch.utils.data import DataLoader
import storm.models as models
from storm.dataset.storm_dataset import STORMDataset
from storm.dataset.samplers import InfiniteSampler
from storm.dataset.data_utils import prepare_inputs_and_targets
from storm.utils.losses import compute_loss
from storm.utils.lpips_loss import RGBLpipsLoss
from storm.utils.misc import NativeScalerWithGradNormCount as NativeScaler
import timm.optim.optim_factory as optim_factory

# Initialize distributed training
torch.distributed.init_process_group(backend="nccl")
local_rank = torch.distributed.get_rank()
device = torch.device(f"cuda:{local_rank}")

# Create model
model = models.STORM_models["STORM-B/8"](
    img_size=(160, 240),
    gs_dim=3,
    use_sky_token=True,
    use_affine_token=True,
    num_motion_tokens=16,
).to(device)
model = torch.nn.parallel.DistributedDataParallel(model)
```

--------------------------------

### Create Data Directory

Source: https://github.com/nvlabs/gaussianstorm/blob/main/docs/WAYMO.md

Initialize the directory structure for storing raw Waymo dataset files.

```bash
mkdir -p ./data/waymo/raw
```

--------------------------------

### Initialize STORM Model

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Configure and initialize a STORM model instance, specifying architecture variants and training parameters. Available configurations include STORM-B/8, STORM-L/8, STORM-XL/8, STORM-H/8, STORM-B/16, STORM-L/16, and STORM-H/16.

```python
import torch
import storm.models as models

# Available model configurations
print(models.STORM_models.keys())
# dict_keys(['STORM-B/8', 'STORM-L/8', 'STORM-XL/8', 'STORM-H/8', 'STORM-B/16', 'STORM-L/16', 'STORM-H/16'])

# Initialize STORM-B/8 model (Base model with patch size 8)
model = models.STORM_models["STORM-B/8"](
    img_size=(160, 240),           # Input image resolution (height, width)
    gs_dim=3,                       # Gaussian color dimensions (RGB)
    decoder_type="dummy",           # Use "dummy" for STORM, "conv" for Latent-STORM
    grad_checkpointing=True,        # Enable gradient checkpointing for memory efficiency
    use_sky_token=True,             # Learn dedicated sky appearance token
    use_affine_token=True,          # Learn per-camera affine color correction
    num_motion_tokens=16,           # Number of learnable motion basis vectors
    use_latest_gsplat=False,        # Use newer gsplat API if available
)

# Move to GPU and count parameters
device = torch.device("cuda")
model.to(device)
n_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"Model parameters: {n_params / 1e6:.2f}M")
# Output: Model parameters: 90.12M (approximately for STORM-B/8)
```

--------------------------------

### Define Training Arguments

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Sets up hyperparameters and loss configuration flags for the training process.

```python
class Args:
    enable_depth_loss = True
    enable_flow_reg_loss = True
    flow_reg_coeff = 0.005
    enable_sky_opacity_loss = True
    sky_opacity_loss_coeff = 0.1
    enable_perceptual_loss = True
    perceptual_loss_start_iter = 5000
    grad_clip = 3.0
args = Args()
```

--------------------------------

### Launch Multi-GPU Training

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Command to initiate distributed training across 8 GPUs using torchrun.

```bash
torchrun --nproc_per_node=8 main_storm.py \
    --project storm_training \
    --exp_name storm_b8_waymo \
    --data_root ./data/STORM2 \
    --batch_size 4 \
    --num_iterations 100000 \
    --lr_sched constant \
    --model STORM-B/8 \
    --num_motion_tokens 16 \
    --use_sky_token \
    --use_affine_token \
    --load_depth \
    --load_flow \
    --load_ground \
    --enable_depth_loss \
    --enable_flow_reg_loss \
    --flow_reg_coeff 0.005 \
    --enable_sky_opacity_loss \
    --enable_perceptual_loss \
    --perceptual_loss_start_iter 5000 \
    --enable_wandb \
    --auto_resume
```

--------------------------------

### Load STORM Dataset

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Initialize and prepare the STORMDataset for training or evaluation, specifying data paths, temporal settings, and data augmentation options. This includes loading multi-view images, depth, optical flow, and camera parameters.

```python
from storm.dataset.storm_dataset import STORMDataset, STORMDatasetEval
from storm.dataset.data_utils import prepare_inputs_and_targets, to_batch_tensor
import torch

# Initialize training dataset
dataset_train = STORMDataset(
    data_root="./data/STORM2",                              # Path to preprocessed data
    annotation_txt_file_list="./data/STORM2/scene_list/waymo_train.txt",  # Scene list file
    target_size=(160, 240),                                 # Output image size (H, W)
    num_context_timesteps=4,                                # Number of input frames
    num_target_timesteps=4,                                 # Number of target frames to predict
    num_max_cams=3,                                         # Number of cameras (1, 3, 5, 6, or 7)
    timespan=2.0,                                           # Temporal window in seconds
    load_depth=True,                                        # Load LiDAR depth supervision
    load_flow=True,                                         # Load scene flow labels
    skip_sky_mask=False,                                    # Load sky segmentation masks
)

# Get a sample
data_dict = dataset_train[0]  # Returns dict with 'context' and 'target' keys

# Prepare inputs for the model
data_dict = to_batch_tensor(data_dict)  # Add batch dimension
device = torch.device("cuda")
input_dict, target_dict = prepare_inputs_and_targets(data_dict, device)

# Input dict contains:
# - context_image: (B, T, V, C, H, W) - multi-view input images
# - context_camtoworlds: (B, T, V, 4, 4) - camera-to-world transforms
# - context_intrinsics: (B, T, V, 3, 3) - camera intrinsic matrices
# - context_time: (B, T, V) - normalized timestamps

```

--------------------------------

### Download Dataset Subset

Source: https://github.com/nvlabs/gaussianstorm/blob/main/README.md

Downloads a small subset of the Waymo Open Dataset for quick experimentation with the STORM model.

```bash
# download dataset subset (≈ 600 MB)
gdown 14fapsAGoMCQ5Ky82cg2X6bk-mLQ7fdCF
tar -xf STORM_subset.tar.gz
```

--------------------------------

### Run Single-GPU Inference Demo

Source: https://github.com/nvlabs/gaussianstorm/blob/main/README.md

Executes the inference demo on a single GPU using the downloaded dataset subset. Requires a checkpoint path to be set in CKPT_PTH.

```bash
python inference.py \
    --project storm_playground --exp_name visualization \
    --data_root data/STORM_subset \
    --model STORM-B/8 --num_motion_tokens 16 \
    --use_sky_token --use_affine_token \
    --load_depth --load_flow --load_ground \
    --load_from $CKPT_PTH
```

--------------------------------

### Run Command-line Evaluation

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Execute model evaluation using `main_storm.py` with distributed training. This command initiates evaluation on the specified dataset and saves results to a designated directory.

```bash
torchrun --nproc_per_node=8 main_storm.py \
    --project storm_eval \
    --exp_name eval_waymo \
    --data_root ./data/STORM2 \
    --batch_size 4 \
    --model STORM-B/8 \
    --num_motion_tokens 16 \
    --use_sky_token \
    --use_affine_token \
    --load_depth \
    --load_flow \
    --load_ground \
    --auto_resume \
    --evaluate
```

--------------------------------

### Perform Inference and Visualization

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Loads a trained model and dataset to generate visualization videos for specific scenes.

```python
from storm.visualization.video_maker import make_video
from storm.dataset.storm_dataset import SingleSequenceDataset
from storm.dataset.data_utils import to_batch_tensor
import torch
import storm.models as models

# Load model
device = torch.device("cuda")
model = models.STORM_models["STORM-B/8"](
    img_size=(160, 240),
    gs_dim=3,
    use_sky_token=True,
    use_affine_token=True,
    num_motion_tokens=16,
).to(device)

# Load checkpoint
checkpoint = torch.load("path/to/checkpoint.pth", map_location=device)
model.load_state_dict(checkpoint["model"])
model.eval()

# Load dataset for inference
dataset = SingleSequenceDataset(
    data_root="./data/STORM_subset",
    annotation_txt_file_list="./data/STORM_subset/scene_list/waymo_train.txt",
    target_size=(160, 240),
    num_context_timesteps=4,
    num_target_timesteps=4,
    num_max_cams=3,
    timespan=2.0,
    load_depth=True,
    load_flow=True,
)

# Generate video for a specific scene
scene_id = 0
output_path = f"output_scene_{scene_id}.mp4"

make_video(
    dataset=dataset,
    model=model,
    device=device,
    output_filename=output_path,
    scene_id=scene_id,
    skip_plot_gt_depth_and_flow=False,
)
print(f"Video saved to: {output_path}")
```

--------------------------------

### Initialize STORM Dataset for Evaluation

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Initializes the STORMDatasetEval for evaluation purposes. Ensure data paths and desired features like depth, flow, and masks are correctly specified.

```python
dataset_eval = STORMDatasetEval(
    data_root="./data/STORM2",
    annotation_txt_file_list="./data/STORM2/scene_list/waymo_val.txt",
    target_size=(160, 240),
    num_context_timesteps=4,
    num_target_timesteps=4,
    num_max_cams=3,
    timespan=2.0,
    load_depth=True,
    load_flow=True,
    load_dynamic_mask=True,         # Load dynamic object masks for evaluation
    load_ground_label=True,         # Load ground plane labels for flow evaluation
)
```

--------------------------------

### Project Directory Structure

Source: https://github.com/nvlabs/gaussianstorm/blob/main/docs/WAYMO.md

The required file organization for the project data after completing all preprocessing steps.

```bash
ProjectPath/data/
  └── waymo/
    ├── raw/
    │    ├── segment-454855130179746819_4580_000_4600_000_with_camera_labels.tfrecord
    │    └── ...
    └── processed/
         └──training/
              ├── 000/
              │  ├──cam_to_ego/         # camera to ego-vehicle transformations: {cam_id}.txt
              │  ├──cam_to_world/       # camera to world transformations: {timestep:03d}_{cam_id}.txt
              │  ├──depth_flows_4/      # downsampled (1/4) depth flow maps: {timestep:03d}_{cam_id}.npy
              │  ├──dynamic_masks/      # bounding-box-generated dynamic masks: {timestep:03d}_{cam_id}.png
              │  ├──ego_to_world/       # ego-vehicle to world transformations: {timestep:03d}.txt
              │  ├──ground_label_4/     # downsampled (1/4) ground labels extracted from point cloud, used for flow evaluation only: {timestep:03d}.txt
              │  ├──images/             # original camera images: {timestep:03d}_{cam_id}.jpg
              │  ├──images_4/           # downsampled (1/4) camera images: {timestep:03d}_{cam_id}.jpg
              │  ├──intrinsics/         # camera intrinsics: {cam_id}.txt
              │  ├──lidar/              # lidar data: {timestep:03d}.bin
              │  ├──sky_masks/          # sky masks: {timestep:03d}_{cam_id}.png
              ├── 001/
              ├── ...
```

--------------------------------

### Download Waymo Dataset Sequences

Source: https://github.com/nvlabs/gaussianstorm/blob/main/docs/WAYMO.md

Download specific or all sequences from the Waymo Open Dataset using the provided preprocessing script.

```bash
python preproc/waymo_download.py \
    --target_dir ./data/waymo/raw/training \
    --split_file data/dataset_scene_list/waymo_train_list.txt \
    --scene_ids 700 754 23
```

```bash
# training set
python preproc/waymo_download.py \
    --target_dir ./data/waymo/raw/training \
    --split_file data/dataset_scene_list/waymo_train_list.txt

# validation set
python preproc/waymo_download.py \
    --target_dir ./data/waymo/raw/validation \
    --split_file data/dataset_scene_list/waymo_val_list.txt
```

--------------------------------

### Preprocess Waymo Data

Source: https://github.com/nvlabs/gaussianstorm/blob/main/docs/WAYMO.md

Extract and organize dataset components into the processed directory structure.

```bash
python preprocess.py \
    --data_root data/waymo/raw/ \
    --target_dir data/waymo/processed \
    --dataset waymo \
    --split training \
    --scene_list_file data/dataset_scene_list/waymo_train_list.txt \
    --scene_ids 700 754 23 \
    --num_workers 8 \
    --process_keys images lidar calib pose dynamic_masks ground \
    --json_folder_to_save data/STORM_data/annotations/waymo
```

```bash
# training set
python preprocess.py \
    --data_root data/waymo/raw/ \
    --target_dir data/waymo/processed \
    --dataset waymo \
    --split training \
    --scene_list_file data/dataset_scene_list/waymo_train_list.txt \
    --num_workers 8 \
    --process_keys images lidar calib pose dynamic_masks ground \
    --json_folder_to_save data/STORM_data/annotations/waymo 

# validation set
python preprocess.py \
    --data_root data/waymo/raw/ \
    --target_dir data/waymo/processed \
    --dataset waymo \
    --split validation \
    --scene_list_file data/dataset_scene_list/waymo_val_list.txt \
    --num_workers 8 \
    --process_keys images lidar calib pose dynamic_masks ground \
    --json_folder_to_save data/STORM_data/annotations/waymo
```

--------------------------------

### Run Command-line Inference

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Execute inference using the `inference.py` script with specified project, experiment, data, and model configurations. This command generates output files like test_0.mp4 for each segment.

```bash
python inference.py \
    --project storm_playground \
    --exp_name visualization \
    --data_root ./data/STORM_subset \
    --model STORM-B/8 \
    --num_motion_tokens 16 \
    --use_sky_token \
    --use_affine_token \
    --load_depth \
    --load_flow \
    --load_ground \
    --load_from /path/to/checkpoint.pth
```

--------------------------------

### Preprocess Waymo Raw Data

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Preprocess the downloaded Waymo raw data into a usable format for training and evaluation. This involves specifying data roots, target directories, dataset splits, and the types of data to process.

```bash
# 3. Preprocess raw data
python preprocess.py \
    --data_root data/waymo/raw/ \
    --target_dir data/waymo/processed \
    --dataset waymo \
    --split training \
    --scene_list_file data/dataset_scene_list/waymo_train_list.txt \
    --scene_ids 700 754 23 \
    --num_workers 8 \
    --process_keys images lidar calib pose dynamic_masks ground \
    --json_folder_to_save data/STORM_data/annotations/waymo
```

--------------------------------

### Separate Gaussian Extraction and Custom View Rendering

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Separates the process into two steps: first extracting Gaussian parameters from context frames, then rendering from custom camera poses. This allows for flexible novel view synthesis.

```python
with torch.no_grad():
    # Step 1: Extract Gaussian parameters from context frames
    gs_params = model.get_gs_params(input_dict)

    # Step 2: Render from custom camera poses
    custom_target_dict = {
        "target_camtoworlds": custom_cam_poses,  # (B, N_views, V, 4, 4)
        "target_intrinsics": custom_intrinsics,   # (B, N_views, V, 3, 3)
        "target_time": custom_times,              # (B, N_views, V)
        "height": 160,
        "width": 240,
    }
    render_output = model.from_gs_params_to_output(
        gs_params,
        custom_target_dict,
        num_cams=3
    )
```

--------------------------------

### Execute Training Loop

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Iterates through the dataloader, performs forward passes with mixed precision, computes losses, and updates model parameters.

```python
for iteration, data_dict in enumerate(dataloader):
    if iteration > 100000:
        break

    model.train()

    # Enable perceptual loss after warmup
    if iteration >= args.perceptual_loss_start_iter:
        rgb_lpips_loss.set_perceptual_loss(True)

    with torch.autocast("cuda", dtype=torch.bfloat16):
        input_dict, target_dict = prepare_inputs_and_targets(data_dict, device)
        pred_dict = model(input_dict)
        loss_dict = compute_loss(pred_dict, target_dict, args, rgb_lpips_loss)

    # Compute total loss
    total_loss = sum(v for k, v in loss_dict.items() if "loss" in k)

    # Backward with gradient scaling
    grad_norm = loss_scaler(
        total_loss,
        optimizer,
        parameters=model.parameters(),
        clip_grad=args.grad_clip,
    )
    optimizer.zero_grad()

    # Logging
    if iteration % 50 == 0:
        psnr = -10 * torch.log10(loss_dict["rgb_loss"]).item()
        print(f"Iter {iteration}: PSNR={psnr:.2f}, Loss={total_loss.item():.4f}")
```

--------------------------------

### Extract Sky Masks Commands

Source: https://github.com/nvlabs/gaussianstorm/blob/main/docs/WAYMO.md

Commands to generate an image file list, download the DepthAnything-v2 checkpoint, and execute the sky mask extraction script.

```bash
find data/waymo/processed/training/*/images -name "*.jpg" > file_list.txt
```

```bash
mkdir ckpts && wget https://huggingface.co/depth-anything/Depth-Anything-V2-Large/resolve/main/depth_anything_v2_vitl.pth -O ckpts/depth_anything_v2_vitl.pth
```

```bash
python extract_sky.py --file_list ./file_list.txt
```

--------------------------------

### Evaluate Model Performance

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Evaluate the model's performance on the validation set using comprehensive metrics for novel view synthesis and scene flow estimation. Requires setting up the evaluation dataset and dataloader.

```python
from engine_storm import evaluate, evaluate_flow
from storm.dataset.storm_dataset import STORMDatasetEval
from storm.dataset.samplers import NoPaddingDistributedSampler
from torch.utils.data import DataLoader
import torch

# Setup evaluation dataset and dataloader
dataset_eval = STORMDatasetEval(
    data_root="./data/STORM2",
    annotation_txt_file_list="./data/STORM2/scene_list/waymo_val.txt",
    target_size=(160, 240),
    num_context_timesteps=4,
    num_target_timesteps=4,
    num_max_cams=3,
    timespan=2.0,
    load_depth=True,
    load_flow=True,
    load_dynamic_mask=True,     # For dynamic region evaluation
    load_ground_label=True,     # For flow evaluation (exclude ground points)
)

sampler = NoPaddingDistributedSampler(dataset_eval, shuffle=False)
dataloader_eval = DataLoader(
    dataset_eval,
    batch_size=1,
    num_workers=16,
    sampler=sampler,
    drop_last=False,
)

# Simplified args for evaluation
class EvalArgs:
    log_dir = "./work_dirs/eval"
    decoder_type = "dummy"
    load_flow = True
    load_ground = True
args = EvalArgs()

# Run image quality evaluation
# Returns: PSNR, SSIM, Depth RMSE, Dynamic PSNR, Dynamic SSIM
eval_results = evaluate(dataloader_eval, model, args)
print(f"Results: PSNR={eval_results['psnr']:.2f}, "
      f"SSIM={eval_results['ssim']:.4f}, "
      f"Dynamic PSNR={eval_results['dynamic_psnr']:.2f}")

# Run scene flow evaluation
# Returns: EPE3D, Acc3D (strict/relax), Angle Error, Flow RMSE
flow_results = evaluate_flow(dataloader_eval, model, args)
print(f"Flow: EPE={flow_results['flow_epe']:.4f}, "
      f"Acc@5cm={flow_results['flow_acc_strict']:.1f}%, "
      f"Acc@10cm={flow_results['flow_acc_relax']:.1f}%")

# Expected output format:
# Results: PSNR=26.45, SSIM=0.8234, Dynamic PSNR=22.18
# Flow: EPE=0.1523, Acc@5cm=72.4%, Acc@10cm=85.6%
```

--------------------------------

### Compute All Training Losses

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Computes a dictionary of various training losses using predicted and target data, along with loss arguments and the perceptual loss module. The total loss is the sum of enabled losses.

```python
from storm.utils.losses import compute_loss

# Compute all losses
# pred_dict from model forward pass
# target_dict from prepare_inputs_and_targets
loss_dict = compute_loss(pred_dict, target_dict, args, lpips_loss)

# loss_dict contains:
# - rgb_loss: MSE between rendered and GT images
# - perceptual_loss: LPIPS perceptual similarity (if enabled)
# - depth_loss: L1 depth error on valid pixels (if enabled)
# - flow_reg_loss: L2 norm of predicted flow (if enabled)
# - sky_opacity_loss: L1 error pushing sky opacity to 0 (if enabled)

total_loss = sum(v for k, v in loss_dict.items() if "loss" in k)
```

--------------------------------

### Full Forward Pass for Gaussian Reconstruction and Rendering

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Performs a full forward pass of the STORM model to extract 3D Gaussian parameters and render novel views. This requires the model and dataset to be initialized and data to be prepared for the CUDA device.

```python
import torch
from storm.dataset.data_utils import prepare_inputs_and_targets, to_batch_tensor

# Assume model and dataset are initialized
model.eval()
device = torch.device("cuda")

# Get sample data
data_dict = dataset[0]
data_dict = to_batch_tensor(data_dict)
input_dict, target_dict = prepare_inputs_and_targets(data_dict, device)

# Full forward pass: extract Gaussians and render
with torch.no_grad():
    with torch.autocast("cuda", dtype=torch.bfloat16):
        output_dict = model(input_dict)
```

```python
from storm.dataset.constants import MEAN, STD
mean = torch.tensor(MEAN, device=device)
std = torch.tensor(STD, device=device)
pred_rgb = render_results["rendered_image"] * std + mean  # Range [0, 1]
print(f"Rendered images shape: {pred_rgb.shape}")
```

--------------------------------

### Define Training Loss Arguments

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Defines a class to hold arguments controlling which losses are active during training. Enable or disable specific losses and set their coefficients.

```python
class LossArgs:
    enable_depth_loss = True           # L1 depth loss with LiDAR GT
    enable_flow_reg_loss = True        # Regularize flow towards zero
    flow_reg_coeff = 0.005             # Flow regularization weight
    enable_sky_opacity_loss = True     # Push sky Gaussians to be transparent
    sky_opacity_loss_coeff = 0.1       # Sky opacity loss weight
    enable_sky_depth_loss = False      # Alternative: push sky depth to far plane
    sky_depth = 300.0                  # Far plane depth for sky
    enable_perceptual_loss = True      # LPIPS perceptual loss
args = LossArgs()
```

--------------------------------

### Compute Scene Flow Metrics

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Calculates scene flow evaluation metrics such as EPE3D, accuracy, and outliers. Requires predicted and ground truth 3D flow vectors.

```python
from storm.utils.losses import compute_scene_flow_metrics
import torch

# Scene flow evaluation metrics
pred_flow = torch.randn(1000, 3)  # Predicted 3D flow vectors
gt_flow = torch.randn(1000, 3)    # Ground truth flow vectors

metrics = compute_scene_flow_metrics(pred_flow, gt_flow)
# Returns dict with:
# - EPE3D: End-point error in meters
# - acc3d_strict: % points with error < 5cm or < 5% relative
# - acc3d_relax: % points with error < 10cm or < 10% relative
# - outlier: % points with error > 30cm and > 10% relative
# - angle_error: Mean angular error in radians
```

--------------------------------

### STORM Citation (BibTeX)

Source: https://github.com/nvlabs/gaussianstorm/blob/main/README.md

BibTeX entry for citing the STORM paper.

```bibtex
@inproceedings{yang2025storm,
  title   = {STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes},
  author  = {Jiawei Yang and Jiahui Huang and Yuxiao Chen and Yan Wang and Boyi Li and Yurong You and Maximilian Igl and Apoorva Sharma and Peter Karkus and Danfei Xu and Boris Ivanovic and Yue Wang and Marco Pavone},
  booktitle = {ICLR},
  year    = {2025}
}
```

--------------------------------

### Extract Sky Masks

Source: https://context7.com/nvlabs/gaussianstorm/llms.txt

Extract sky masks from images using the Depth Anything V2 model. This process involves generating a file list of images, downloading the model checkpoint, and running the extraction script.

```bash
# 4. Extract sky masks using Depth Anything V2
# Generate file list
find data/waymo/processed/training/*/images -name "*.jpg" > file_list.txt

# Download Depth Anything V2 checkpoint
mkdir ckpts
wget https://huggingface.co/depth-anything/Depth-Anything-V2-Large/resolve/main/depth_anything_v2_vitl.pth \
    -O ckpts/depth_anything_v2_vitl.pth

# Extract sky masks
python extract_sky.py --file_list ./file_list.txt
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.