### Install Project Dependencies (Bash)

Source: https://github.com/dcharatan/flowmap/blob/main/README.md

Installs the project dependencies on Linux by creating a Python virtual environment, activating it, and installing required packages from a requirements file. It also includes steps for initializing Git submodules for pre-training.

```bash
python3.11 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

git submodule update --init --recursive
```

--------------------------------

### Run Ablation Study with Hydra (Bash)

Source: https://github.com/dcharatan/flowmap/blob/main/README.md

Executes an ablation study for FlowMap using Hydra configurations. This example demonstrates disabling point tracking by specifying the corresponding experiment configuration. Multiple ablations can be stacked.

```bash
python3 -m flowmap.overfit dataset=images dataset.images.root=path/to/folder/with/images +experiment=ablation_no_tracks

# Example stacking multiple ablations
python3 -m flowmap.overfit dataset=images dataset.images.root=path/to/folder/with/images +experiment=[ablation_no_tracks,ablation_random_initialization]
```

--------------------------------

### Run Ablation Studies

Source: https://context7.com/dcharatan/flowmap/llms.txt

Executes ablation experiments to evaluate the impact of different FlowMap components. These commands are run from the command line using `python3 -m flowmap.overfit` and specify dataset configurations, experiment names, and output directories. Examples include running without point tracking supervision and without correspondence weights from the backbone.

```bash
# Run without point tracking supervision
python3 -m flowmap.overfit \
    dataset=images \
    dataset.images.root=data/my_scene \
    +experiment=ablation_no_tracks \
    local_save_root=output/ablation_no_tracks

# Run without correspondence weights from backbone
python3 -m flowmap.overfit \
    dataset=llff \
    dataset.llff.root=data/nerf_llff_data/fern \
    +experiment=ablation_no_correspondence_weights \
    local_save_root=output/ablation_no_weights

```

--------------------------------

### Install GMFlow with Conda

Source: https://github.com/dcharatan/flowmap/blob/main/flowmap/third_party/gmflow/README.md

Installs the GMFlow project dependencies using Conda. It requires PyTorch 1.9.0, CUDA 10.2, and Python 3.8, though higher versions are generally compatible. The `environment.yml` file lists all necessary packages for setting up the `gmflow` environment.

```bash
conda env create -f environment.yml
conda activate gmflow
```

--------------------------------

### Compute Tracks for Temporal Segments

Source: https://context7.com/dcharatan/flowmap/llms.txt

Computes a list of tracks for different temporal segments of video data. This function requires batch data, a device, and configuration for tracking and precomputation. It outputs track information including shapes, visibility, query frame, and start frame.

```python
track_list = compute_tracks(batch, device, tracking_cfg, precomputation_cfg)

print(f"Number of track segments: {len(track_list)}")
for i, tracks in enumerate(track_list):
    print(f"Segment {i}:")
    print(f"  Tracks shape: {tracks.tracks.shape}")  # [1, num_tracks, segment_frames, 2] 
    print(f"  Visibility shape: {tracks.visibility.shape}")  # [1, num_tracks, segment_frames]
    print(f"  Query frame: {tracks.query_frame}")
    print(f"  Start frame: {tracks.start_frame}")

    # Access individual track coordinates
    track_0_positions = tracks.tracks[0, 0]  # [segment_frames, 2] xy coordinates
    track_0_visible = tracks.visibility[0, 0] > 0.5  # Boolean visibility mask
```

--------------------------------

### Configure and Initialize Flowmap Model

Source: https://context7.com/dcharatan/flowmap/llms.txt

This snippet demonstrates how to configure model components like backbone, intrinsics, and extrinsics, then initialize the Flowmap model with specified parameters. It also shows how to prepare a video batch and perform a forward pass to obtain outputs like depths, surfaces, and transformations.

```python
from flowmap.model import Model, ModelCfg, BackboneMidasCfg, IntrinsicsRegressedCfg, ExtrinsicsProcrustestCfg, ModelOutput
import torch
from flowmap.dataset.types import Batch
from flowmap.flow.flow_predictor import FlowCfg

# Configure model components
model_cfg = ModelCfg(
    backbone=BackboneMidasCfg(
        name="midas",
        pretrained_path="checkpoints/initialization.ckpt",
        correspondence_weight_mode="softmax"
    ),
    intrinsics=IntrinsicsRegressedCfg(
        name="regressed",
        scale_min=0.5,
        scale_max=2.0
    ),
    extrinsics=ExtrinsicsProcrustestCfg(
        name="procrustes",
        num_iterations=10
    ),
    use_correspondence_weights=True
)

# Initialize model
num_frames = 50
image_shape = (180, 240)
model = Model(model_cfg, num_frames=num_frames, image_shape=image_shape)
model = model.cuda()

# Prepare batch (videos should be normalized to [0, 1])
videos = torch.rand(1, num_frames, 3, 180, 240).cuda()  # [batch, frames, 3, H, W]
batch = Batch(
    videos=videos,
    datasets=["custom"],
    scenes=["scene_01"],
    indices=[torch.arange(num_frames)]
)

# Compute optical flows
device = torch.device("cuda:0")
from flowmap.config.pretrain import FlowCfg
flow_cfg = FlowCfg(name="raft")
flows = compute_flows(batch, image_shape, device, flow_cfg)

# Forward pass
global_step = 1000
output: ModelOutput = model.forward(batch, flows, global_step)

print(f"Depths shape: {output.depths.shape}")        # [1, 50, 180, 240]
print(f"Surfaces shape: {output.surfaces.shape}")    # [1, 50, 180, 240, 3]
print(f"Intrinsics shape: {output.intrinsics.shape}") # [1, 50, 3, 3]
print(f"Extrinsics shape: {output.extrinsics.shape}") # [1, 50, 4, 4]
print(f"Correspondence weights: {output.backward_correspondence_weights.shape}") # [1, 49, 180, 240]

# Export for downstream use
exports = model.export(batch, flows, global_step)
```

--------------------------------

### Pretrain FlowMap Model on Large Datasets (Bash)

Source: https://context7.com/dcharatan/flowmap/llms.txt

Trains an initialization model for FlowMap on large-scale video datasets (e.g., Real Estate 10k, CO3Dv2) to improve generalization. Requires downloading submodules like GMFlow. Allows specifying dataset paths, Weights & Biases project details, training steps, and checkpointing frequency. Can resume training from a saved checkpoint.

```bash
# Download submodules (GMFlow for optical flow)
git submodule update --init --recursive

# Pretrain on Real Estate 10k and CO3Dv2 datasets
python3 -m flowmap.pretrain \
    dataset=[re10k,co3d] \
    dataset.0.re10k.root=path/to/re10k/dataset \
    dataset.1.co3d.root=path/to/co3d/dataset \
    wandb.mode=online \
    wandb.project=flowmap_pretrain \
    trainer.max_steps=100000 \
    checkpoint.every_n_train_steps=5000

# Resume from checkpoint
python3 -m flowmap.pretrain \
    checkpoint.load=checkpoints/step_50000.ckpt \
    trainer.max_steps=150000
```

--------------------------------

### Pre-train FlowMap Model (Bash)

Source: https://github.com/dcharatan/flowmap/blob/main/README.md

Initiates the pre-training process for the FlowMap model. This script requires specific datasets like Real Estate 10k and CO3Dv2 to be downloaded and accessible.

```bash
python3 -m flowmap.pretrain
```

--------------------------------

### Overfit on Image Sequence with FlowMap (Bash)

Source: https://context7.com/dcharatan/flowmap/llms.txt

Runs FlowMap to estimate camera poses, intrinsics, and depth maps from a folder of images using gradient descent. Supports various dataset formats like images, LLFF, and COLMAP. Allows customization via arguments for training parameters and experimental configurations.

```bash
# Activate virtual environment
source venv/bin/activate

# Run FlowMap on a folder of images
python3 -m flowmap.overfit \
    dataset=images \
    dataset.images.root=path/to/folder/with/images \
    local_save_root=output/scene_name

# Run with LLFF dataset format
python3 -m flowmap.overfit \
    dataset=llff \
    dataset.llff.root=path/to/llff/scene \
    local_save_root=output/llff_scene

# Run with COLMAP dataset format
python3 -m flowmap.overfit \
    dataset=colmap \
    dataset.colmap.root=path/to/colmap/scene \
    local_save_root=output/colmap_scene

# Run with custom configuration and ablations
python3 -m flowmap.overfit \
    dataset=images \
    dataset.images.root=data/my_video_frames \
    trainer.max_steps=3000 \
    model_wrapper.lr=5e-5 \
    +experiment=ablation_no_tracks \
    local_save_root=output/custom_run
```

--------------------------------

### Compute Point Tracks using CoTracker (Python)

Source: https://context7.com/dcharatan/flowmap/llms.txt

This snippet illustrates how to extract and track 2D point trajectories across video frames using the CoTracker model. It sets up a video batch, configures the CoTracker predictor with parameters like `grid_size` and `checkpoint`, and specifies precomputation settings for caching tracks.

```python
import torch
from flowmap.tracking import compute_tracks
from flowmap.tracking.track_predictor import Tracks
from flowmap.dataset.types import Batch
from pathlib import Path

# Create video batch
videos = torch.rand(1, 60, 3, 256, 384).cuda()  # [batch, frames, 3, H, W]
batch = Batch(
    videos=videos,
    datasets=["my_dataset"],
    scenes=["living_room"],
    indices=[torch.arange(60)]
)

# Configure tracking
from flowmap.config.overfit import TrackPredictorCoTrackerCfg, TrackPrecomputationCfg
tracking_cfg = TrackPredictorCoTrackerCfg(
    name="cotracker",
    grid_size=50,  # Number of tracks to extract
    checkpoint="checkpoints/cotracker_stride_4_wind_8.pth"
)

precomputation_cfg = TrackPrecomputationCfg(
    cache_path=Path("cache/tracks"),  # Cache tracks to disk
    interval=5,   # Query tracks every 5 frames
    radius=20     # Track for 20 frames around each query
)

device = torch.device("cuda:0")

# Note: The actual call to compute_tracks is missing in the provided text.
```

--------------------------------

### Run GMFlow Inference on a Sequence of Images

Source: https://github.com/dcharatan/flowmap/blob/main/flowmap/third_party/gmflow/README.md

Executes the GMFlow model for inference on a given directory of images. It specifies the input directory, output path for results, and the path to a pretrained model checkpoint. Options for predicting bidirectional flow and enabling forward-backward consistency check are also available.

```bash
CUDA_VISIBLE_DEVICES=0 python main.py \
--inference_dir demo/sintel_market_1 \
--output_path output/gmflow-norefine-sintel_market_1 \
--resume pretrained/gmflow_sintel-0c07dcb3.pth
```

--------------------------------

### Symlink Dataset Root Directory

Source: https://github.com/dcharatan/flowmap/blob/main/flowmap/third_party/gmflow/README.md

Creates a symbolic link to the root directory of your datasets. This is the default location expected by the GMFlow dataloader. If your datasets are stored elsewhere, you should adjust the paths in `data/datasets.py` or use this command to point to your data.

```shell
ln -s $YOUR_DATASET_ROOT datasets
```

--------------------------------

### Run FlowMap Overfitting Script (Bash)

Source: https://github.com/dcharatan/flowmap/blob/main/README.md

Executes the main FlowMap overfitting script. This command requires the path to the dataset folder containing images and assumes the virtual environment has been activated.

```bash
python3 -m flowmap.overfit dataset=images dataset.images.root=path/to/folder/with/images
```

--------------------------------

### Load Image Datasets for FlowMap

Source: https://context7.com/dcharatan/flowmap/llms.txt

Loads image datasets from various formats including folders, LLFF, COLMAP, CO3D, and RE10k. It uses the `get_dataset` function with specific configuration objects like `DatasetImagesCfg` or `DatasetLLFFCfg` and a `FrameSamplerOverfitCfg` to control frame sampling. The loaded data is then converted into a `Batch` object for further processing.

```python
from pathlib import Path
import torch
from torch.utils.data import DataLoader
from flowmap.dataset import get_dataset
from flowmap.dataset.types import Batch

# Configure dataset for images folder
from flowmap.config.overfit import DatasetImagesCfg, FrameSamplerOverfitCfg

dataset_cfg = DatasetImagesCfg(
    name="images",
    image_shape=(180, 240),  # Target image size
    scene=None,  # Use all images in folder
    root=Path("data/my_scene_images"),
    downscale_factor=1,
    images_per_scene=None  # Load all images
)

frame_sampler_cfg = FrameSamplerOverfitCfg(
    name="overfit",
    max_frames=50,  # Maximum frames to load
    stride=1        # Take every Nth frame
)

# Create dataset
split = "train"
dataset = get_dataset(dataset_cfg, split, frame_sampler_cfg)

# Load batch
batch_dict = next(iter(dataset))
frame_paths = batch_dict.pop("frame_paths", None)

# Convert to Batch object
from torch.utils.data import default_collate
batch = Batch(**default_collate([batch_dict]))

print(f"Videos shape: {batch.videos.shape}")  # [1, frames, 3, 180, 240]
print(f"Dataset: {batch.datasets[0]}")
print(f"Scene: {batch.scenes[0]}")
print(f"Indices: {batch.indices[0]}")
print(f"Frame paths: {[str(p) for p in frame_paths]}")

# Example with LLFF dataset
llff_cfg = DatasetLLFFCfg(
    name="llff",
    image_shape=(270, 360),
    scene="fern",  # Specific scene or None for all
    root=Path("data/nerf_llff_data/fern")
)

llff_dataset = get_dataset(llff_cfg, "train", frame_sampler_cfg)
llff_batch = Batch(**default_collate([next(iter(llff_dataset))]))
print(f"LLFF videos shape: {llff_batch.videos.shape}")
```

--------------------------------

### BibTeX Entry for FlowMap Paper

Source: https://github.com/dcharatan/flowmap/blob/main/README.md

Provides the BibTeX entry for citing the FlowMap paper. This entry includes the title, authors, year, and venue of the publication.

```bibtex
@inproceedings{smith25flowmap,
      title={FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent},
      author={Cameron Smith and David Charatan and Ayush Tewari and Vincent Sitzmann},
      year={2025},
      booktitle={3DV},
}
```

--------------------------------

### Compute Bidirectional Optical Flow with Consistency Masks (Python)

Source: https://context7.com/dcharatan/flowmap/llms.txt

This snippet shows how to compute bidirectional optical flows and consistency masks between consecutive frames in a video batch using the `compute_flows` function. It utilizes configuration for the flow predictor, such as 'raft' or 'gmflow', and specifies the target shape for the output flows.

```python
import torch
from flowmap.flow import compute_flows
from flowmap.flow.flow_predictor import Flows
from flowmap.dataset.types import Batch

# Create a video batch (normalized to [0, 1])
videos = torch.rand(2, 10, 3, 256, 384).cuda()  # [batch=2, frames=10, 3, H, W]
batch = Batch(
    videos=videos,
    datasets=["dataset1", "dataset2"],
    scenes=["scene_a", "scene_b"],
    indices=[torch.arange(10), torch.arange(10)]
)

# Configure flow predictor
from flowmap.config.pretrain import FlowCfg
flow_cfg = FlowCfg(
    name="raft",  # or "gmflow"
)

device = torch.device("cuda:0")
target_shape = (256, 384)

# Compute bidirectional flows with consistency masks
flows: Flows = compute_flows(batch, target_shape, device, flow_cfg)

print(f"Forward flow shape: {flows.forward.shape}")   # [2, 9, 256, 384, 2]
print(f"Backward flow shape: {flows.backward.shape}") # [2, 9, 256, 384, 2]
print(f"Forward mask shape: {flows.forward_mask.shape}")   # [2, 9, 256, 384]
print(f"Backward mask shape: {flows.backward_mask.shape}") # [2, 9, 256, 384]

# Access flow vectors (last dimension is [dx, dy])
frame_0_to_1_flow = flows.forward[0, 0]  # Flow from frame 0 to frame 1
confidence_mask = flows.forward_mask[0, 0]  # Consistency-based confidence

# Use flows for visualization or loss computation
valid_flow = flows.forward[flows.forward_mask > 0.5]
print(f"Number of valid flow vectors: {valid_flow.shape[0]}")
```

--------------------------------

### Compute Photometric and Geometric Losses

Source: https://context7.com/dcharatan/flowmap/llms.txt

Calculates photometric and geometric losses for optimization using computed flows and tracks. It involves configuring various loss types (e.g., flow, tracking) with specific parameters and enabling conditions. The function `get_losses` prepares a list of loss functions, which are then applied to batch data, flows, tracks, and model outputs to compute a total loss.

```python
import torch
from flowmap.loss import get_losses
from flowmap.loss.loss import Loss
from flowmap.model.model import ModelOutput
from flowmap.dataset.types import Batch
from flowmap.flow import Flows
from flowmap.tracking import Tracks

# Configure losses
from flowmap.config.overfit import LossFlowCfg, LossTrackingCfg
loss_configs = [
    LossFlowCfg(
        name="flow",
        enable_after=0,
        weight=1.0,
        mapping_mode="huber",
        mapping_weight=1.0,
        smoothness_weight=0.05
    ),
    LossTrackingCfg(
        name="tracking",
        enable_after=500,  # Enable after 500 steps
        weight=0.1,
        smoothness_weight=0.01
    )
]

losses: list[Loss] = get_losses(loss_configs)

# Prepare inputs (from model forward pass and flow/tracking computation)
batch = Batch(videos=videos, datasets=["test"], scenes=["scene"], indices=[torch.arange(30)])
model_output = ModelOutput(
    depths=depths,          # [1, 30, 180, 240]
    surfaces=surfaces,      # [1, 30, 180, 240, 3]
    intrinsics=intrinsics,  # [1, 30, 3, 3]
    extrinsics=extrinsics,  # [1, 30, 4, 4]
    backward_correspondence_weights=weights  # [1, 29, 180, 240]
)
flows = Flows(forward=fwd, backward=bwd, forward_mask=fwd_mask, backward_mask=bwd_mask)
tracks = compute_tracks(batch, device, tracking_cfg, precomp_cfg)

# Compute total loss
global_step = 1000
total_loss = torch.tensor(0.0, device=videos.device)

for loss_fn in losses:
    loss_value = loss_fn(batch, flows, tracks, model_output, global_step)
    total_loss += loss_value
    print(f"{loss_fn.cfg.name} loss: {loss_value.item():.4f} (weight: {loss_fn.cfg.weight})")

print(f"Total loss: {total_loss.item():.4f}")

# Use for backpropagation
total_loss.backward()
```

--------------------------------

### Export FlowMap Results to COLMAP Format (Python)

Source: https://context7.com/dcharatan/flowmap/llms.txt

Exports FlowMap's estimated camera poses, intrinsics, and depth maps into the COLMAP file format. This enables compatibility with downstream 3D reconstruction pipelines like 3D Gaussian Splatting. The function handles mapping tensor outputs to COLMAP's sparse model structure and provides utilities for reading the exported model back.

```python
from pathlib import Path
import torch
from flowmap.export.colmap import export_to_colmap, read_colmap_model
from flowmap.model.model import ModelExports

# Assuming you have model exports from FlowMap
exports = ModelExports(
    extrinsics=extrinsics_tensor,  # [batch, frames, 4, 4]
    intrinsics=intrinsics_tensor,  # [batch, frames, 3, 3]
    colors=color_tensor,           # [batch, frames, 3, H, W]
    depths=depth_tensor            # [batch, frames, H, W]
)

frame_paths = [Path(f"images/frame_{i:04d}.jpg") for i in range(num_frames)]
uncropped_shape = (1080, 1920)
output_path = Path("output/colmap_export")

# Export to COLMAP format (creates sparse/0 directory with cameras.bin, images.bin, points3D.ply)
export_to_colmap(
    exports=exports,
    frame_paths=frame_paths,
    uncropped_exports_shape=uncropped_shape,
    uncropped_videos=videos_tensor,
    path=output_path
)

# Read back COLMAP model
device = torch.device("cuda:0")
extrinsics, intrinsics, image_names = read_colmap_model(
    path=output_path / "sparse/0",
    device=device,
    reorder=True
)

print(f"Loaded {len(image_names)} camera poses")
print(f"Extrinsics shape: {extrinsics.shape}")  # [frames, 4, 4]
print(f"Intrinsics shape: {intrinsics.shape}")  # [frames, 3, 3]
```

--------------------------------

### FlowMap Model Forward Pass (Python)

Source: https://context7.com/dcharatan/flowmap/llms.txt

Performs a forward pass through the FlowMap neural network model to predict depth, intrinsic, and extrinsic camera parameters from video input. This function utilizes configurations for the model backbone (MiDaS), intrinsics regression, and extrinsics estimation. It also involves optical flow computation.

```python
import torch
from flowmap.model.model import Model, ModelCfg, ModelOutput
from flowmap.model.backbone import BackboneMidasCfg
from flowmap.model.intrinsics import IntrinsicsRegressedCfg
from flowmap.model.extrinsics import ExtrinsicsProcrustestCfg
from flowmap.dataset.types import Batch
from flowmap.flow import compute_flows
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.