### Run Bundle Adjustment Example

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Executes the bundle adjustment example script. Additional installation instructions for its dependencies can be found at the provided GitHub link.

```python
depth_cov/bundle_adjustment.py
```

--------------------------------

### Install Core Packages

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Installs essential packages including PyTorch with CUDA support, Matplotlib, and h5py. These are fundamental for the project's operations.

```bash
conda install pytorch torchvision pytorch-cuda=11.7 -c pytorch -c nvidia
conda install matplotlib h5py
```

--------------------------------

### Install DepthCov and Dependencies

Source: https://context7.com/edexheim/depthcov/llms.txt

Sets up the conda environment, installs core PyTorch, LieTorch, and the DepthCov package with CUDA extensions.

```bash
conda create -n depth_cov python=3.8.15
conda activate depth_cov

# Core PyTorch stack
conda install pytorch torchvision pytorch-cuda=11.7 -c pytorch -c nvidia
conda install matplotlib h5py
pip install pytorch-lightning==1.8.2 opencv-python-headless open3d pyrealsense2

# LieTorch (SE3 exponential map, required for odometry)
git clone --recursive https://github.com/princeton-vl/lietorch.git
cd lietorch && python setup.py install && cd ..

# Install DepthCov package with CUDA extensions
pip install -e .
```

--------------------------------

### Install Python Packages

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Installs additional Python packages such as pytorch-lightning, opencv-python-headless, open3d, and pyrealsense2. These are required for specific functionalities like visualization and real-time processing.

```bash
pip install pytorch-lightning==1.8.2 opencv-python-headless open3d pyrealsense2
```

--------------------------------

### DepthCov Core Module: NonstationaryGpModule Example

Source: https://context7.com/edexheim/depthcov/llms.txt

Demonstrates loading a pretrained DepthCov model and performing probabilistic depth estimation using sparse observations. Requires model checkpoint and example dataset.

```python
import torch
from depth_cov.core.NonstationaryGpModule import NonstationaryGpModule
from depth_cov.utils.utils import normalize_coordinates, sample_coords
from depth_cov.data.depth_transforms import BaseTransform
from PIL import Image
import torchvision.transforms.functional as TF
import cv2, numpy as np

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

# Load pretrained model
model = NonstationaryGpModule.load_from_checkpoint(
    "models/scannet.ckpt",
    train_size=torch.Size([192, 256])
)
model.eval().to(device)

# Prepare input: RGB + sparse depth observations
rgb_pil = Image.open("dataset/examples/frame-000338.color.jpg")
rgb = TF.to_tensor(rgb_pil)
depth_np = cv2.imread("dataset/examples/frame-000338.depth.pgm", cv2.IMREAD_ANYDEPTH).astype(np.float32) / 1000.0
depth = torch.from_numpy(depth_np).unsqueeze(0)
depth[depth <= 0.0] = float('nan')
log_depth = torch.log(depth)

transform = BaseTransform([192, 256])
rgb, log_depth = transform(rgb, log_depth)
rgb = rgb.unsqueeze(0).to(device)
log_depth = log_depth.unsqueeze(0).to(device)

# Sample 64 sparse depth observations
coords_train, depth_train, _ = sample_coords(log_depth, None, 64, mode="uniform")
coords_train = coords_train.to(device)
depth_train = depth_train.to(device)
mean_depth = torch.nanmean(depth_train)

test_size = torch.Size([192, 256])

with torch.no_grad():
    # Step 1: Run UNet to extract per-pixel Gaussian covariance parameters
    gaussian_covs = model(rgb)  # list of tensors, one per pyramid level

    # Step 2: GP posterior — condition on sparse observations
    coords_norm = normalize_coordinates(coords_train, rgb.shape[-2:])
    pred_depths, pred_vars = model.condition(
        gaussian_covs, coords_norm, depth_train, mean_depth, test_size
    )

# pred_depths[-1]: shape (1, 1, 192, 256) — posterior mean log-depth at finest level
# pred_vars[-1]:   shape (1, 1, 192, 256) — posterior variance
print("Posterior mean depth shape:", pred_depths[-1].shape)   # (1, 1, 192, 256)
print("Posterior variance shape:  ", pred_vars[-1].shape)     # (1, 1, 192, 256)
print("Max std dev (metres):      ", torch.sqrt(pred_vars[-1]).max().item())
```

--------------------------------

### Install DepthCov Package

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Installs the DepthCov package in editable mode. This makes the executables in 'depth_cov/' available for use.

```bash
cd ..
pip install -e .
```

--------------------------------

### Clone and Install Lietorch

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Clones the Lietorch repository, which provides batched SE3 exponential maps, and installs it. This is a required dependency for the odometry backend.

```bash
git clone --recursive https://github.com/princeton-vl/lietorch.git
cd lietorch
python setup.py install
```

--------------------------------

### Run Real-Time Odometry Demo

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Launches the real-time odometry demo using an Intel RealSense camera. This script is configured for real-time performance.

```python
depth_cov/odom_demo.py
```

--------------------------------

### Initialize and Run Real-time Visual Odometry

Source: https://context7.com/edexheim/depthcov/llms.txt

Initializes the Open3D application and runs the OdomWindow for real-time visual odometry. `is_live=True` disables frame-rate throttling.

```python
import torch, yaml
import open3d.visualization.gui as gui

from depth_cov.odom.OdomWindow import OdomWindow
from depth_cov.odom.odom_datasets import RealsenseDataset

torch.manual_seed(0)
img_size = [192, 256]

with open('./config/realsense.yml') as f:
    rs_cfg = yaml.safe_load(f)
with open('./config/open3d_viz.yml') as f:
    viz_cfg = yaml.safe_load(f)
with open('./config/visual_odom.yml') as f:
    slam_cfg = yaml.safe_load(f)   # multi-threaded, 30 FPS config

dataset = RealsenseDataset(img_size, rs_cfg)

app = gui.Application.instance
app.initialize()
OdomWindow(is_live=True, viz_cfg=viz_cfg, slam_cfg=slam_cfg, dataset=dataset)
app.run()
```

--------------------------------

### Interactive Depth Completion Demo

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Runs an interactive demo to visualize the impact of new observations on depth estimation and the covariance matrix. Users can click on the image to improve the estimate.

```python
depth_cov/active_depth_viz.py
```

--------------------------------

### Real-Time Odometry with Intel RealSense

Source: https://context7.com/edexheim/depthcov/llms.txt

Runs DepthCov-VO in real time using a live Intel RealSense camera stream. Configuration can be adjusted via `config/realsense.yml`.

```python
# depth_cov/odom_demo.py — connect RealSense camera then run:

```

--------------------------------

### Visual Odometry on Datasets (TUM/ScanNet)

Source: https://context7.com/edexheim/depthcov/llms.txt

Runs the DepthCov-VO pipeline on pre-recorded TUM or ScanNet sequences, visualized with Open3D. Requires dataset paths and configuration files.

```python
# depth_cov/odom_dataset.py — edit dataset_dir then run:
# python depth_cov/odom_dataset.py

import torch, yaml
import open3d.visualization.gui as gui
import open3d as o3d

from depth_cov.odom.OdomWindow import OdomWindow
from depth_cov.odom.odom_datasets import TumOdometryDataset, ScanNetOdometryDataset

torch.manual_seed(0)
img_size = [192, 256]

# TUM RGB-D dataset
dataset = TumOdometryDataset(
    dataset_dir="/path/to/tum/rgbd_dataset_freiburg3_long_office_household/",
    img_size=img_size
)

# ScanNet dataset (10-pixel crop to remove black borders)
# dataset = ScanNetOdometryDataset(
#     dataset_dir="/path/to/scannet/scene0155_00/",
#     img_size=img_size, crop_size=10
# )

with open('./config/open3d_viz.yml') as f:
    viz_cfg = yaml.safe_load(f)
with open('./config/tum.yml') as f:               # or visual_odom.yml for real-time config
    slam_cfg = yaml.safe_load(f)

app = gui.Application.instance
app.initialize()
OdomWindow(is_live=False, viz_cfg=viz_cfg, slam_cfg=slam_cfg, dataset=dataset)
app.run()
```

--------------------------------

### Depth Completion Visualization

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Launches a Python script for visualizing depth completion results. This script demonstrates the output conditional mean and variance based on sampled observations.

```python
depth_cov/depth_completion_viz.py
```

--------------------------------

### Simulated Inputs for GP Training and Testing

Source: https://context7.com/edexheim/depthcov/llms.txt

Sets up simulated training and testing data for Gaussian Process models, including covariance matrices, observed data, and prior means/noises. Used for training forward passes and prediction.

```python
K_nn  = torch.eye(N_train).unsqueeze(0) * 0.5        # (1, N_train, N_train) training cov
y     = torch.randn(B, N_train, D_out)                 # (1, N_train, 1) observed log-depths
mean  = torch.zeros(B, 1, D_out)                       # (1, 1, 1) prior mean
noise_var = torch.ones(B, N_train) * 1e-4              # (1, N_train) observation noise

# Training forward pass — also returns negative log marginal likelihood for backprop
L, alpha, neg_lml, info = train_mod(K_nn, y, mean, noise_var)
print("Neg log marginal likelihood:", neg_lml.item())  # e.g. 12.34

# Test prediction
K_nt      = torch.randn(B, N_train, 100)               # (1, N_train, N_test_subset)
K_tt_diag = torch.ones(B, 100)                         # (1, N_test_subset) prior variances

pred_mean, pred_var = test_mod(L, alpha, K_nt, K_tt_diag, mean)
print("Pred mean shape:", pred_mean.shape)  # (1, 100, 1)
print("Pred var  shape:", pred_var.shape)   # (1, 100, 1)
```

--------------------------------

### Batch Depth Completion Visualization

Source: https://context7.com/edexheim/depthcov/llms.txt

Runs batch depth completion with sparse observations and visualizes results across pyramid levels. Requires model and data paths.

```python
# Run from repository root
import torch
from depth_cov.depth_completion_viz import main

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

main(
    model_path="models/scannet.ckpt",
    rgb_path="dataset/examples/frame-000338.color.jpg",
    depth_path="dataset/examples/frame-000338.depth.pgm",
    num_samples=128,   # number of random sparse depth observations
    device=device
)
# Opens a matplotlib window showing:
#   Row 0: GT depth | pred mean lvl0 | pred mean lvl1 | ...
#   Row 1: RGB+pts  | pred std  lvl0 | pred std  lvl1 | ...
```

--------------------------------

### Configure and Use DepthCovDataModule

Source: https://context7.com/edexheim/depthcov/llms.txt

Demonstrates direct dataloader usage with custom transforms and instantiation via PyTorch Lightning's DepthCovDataModule. Ensure dataset files are correctly formatted.

```python
from depth_cov.data.data_modules import DepthCovDataModule
from depth_cov.data.data_loaders import ScanNetDataLoader
from depth_cov.data.depth_transforms import TrainTransform, BaseTransform
import torch

# Direct dataloader usage (bypassing LightningDataModule)
img_size = [192, 256]
train_transform = TrainTransform(img_size, max_angle=15, crop_scale=True)
val_transform   = BaseTransform(img_size)

train_ds = ScanNetDataLoader(filename="./dataset/scannet_train.txt", transform=train_transform)
val_ds   = ScanNetDataLoader(filename="./dataset/scannet_val.txt",   transform=val_transform)

rgb, log_depth = train_ds[0]
print("RGB shape:       ", rgb.shape)        # (3, 192, 256)  float32 in [0,1]
print("Log-depth shape: ", log_depth.shape)  # (1, 192, 256)  NaN for invalid pixels

# Via LightningDataModule (used by train.py)
dm = DepthCovDataModule(batch_size=4)
dm.setup("fit")
for batch in dm.train_dataloader():
    rgb_b, log_depth_b = batch
    print("Batch RGB:   ", rgb_b.shape)       # (4, 3, 192, 256)
    print("Batch depth: ", log_depth_b.shape) # (4, 1, 192, 256)
    break
```

--------------------------------

### Greedy Entropy Sampling: Model Loading and Data Preparation

Source: https://context7.com/edexheim/depthcov/llms.txt

Loads a pre-trained NonstationaryGpModule and prepares RGB image data for processing. This is a prerequisite for using greedy entropy sampling to select informative locations.

```python
import torch
from depth_cov.core.samplers import greedy_conditional_entropy, sample_sparse_coords_norm
from depth_cov.core.NonstationaryGpModule import NonstationaryGpModule

device = torch.device("cpu")
model = NonstationaryGpModule.load_from_checkpoint("models/scannet.ckpt").eval().to(device)

from PIL import Image
import torchvision.transforms.functional as TF
from depth_cov.data.depth_transforms import BaseTransform

rgb = TF.to_tensor(Image.open("dataset/examples/frame-000338.color.jpg")).unsqueeze(0)
rgb, _ = BaseTransform([192, 256])(rgb.squeeze(0), rgb.squeeze(0)[:1])
rgb = rgb.unsqueeze(0)

with torch.no_grad():
    gaussian_covs = model(rgb)
```

--------------------------------

### Interactive Depth Visualizer

Source: https://context7.com/edexheim/depthcov/llms.txt

Launches an interactive visualization where clicking adds depth observations, updating the GP posterior in real-time. Visualizes covariance and correlation maps.

```python
# Run from repository root
import torch
from depth_cov.active_depth_viz import main

device = torch.device("cpu")   # interactive; CPU is sufficient

main(
    model_path="models/scannet.ckpt",
    rgb_path="dataset/examples/frame-000338.color.jpg",
    depth_path="dataset/examples/frame-000338.depth.pgm",
    num_samples_init=5,        # initial number of random observations
    test_size=torch.Size([192, 256]),
    device=device
)
# Opens a 2x3 matplotlib grid:
#   [RGB + obs pts] [Posterior mean]         [Posterior std dev]
#   [Cov matrix K ] [Correlation map pt 0]   [Correlation map pt 1]
# Click any panel to add observations or inspect kernel structure.
```

--------------------------------

### Gaussian Process Inference Modules

Source: https://context7.com/edexheim/depthcov/llms.txt

Provides PyTorch modules for Gaussian Process inference, including training with negative log marginal likelihood and testing with posterior predictive mean and variance.

```python
import torch
from depth_cov.core.gp import GpTrainModule, GpTestModule

train_mod = GpTrainModule()
test_mod  = GpTestModule()

B, N_train, N_test = 1, 16, 192*256
D_out = 1  # scalar depth output
```

--------------------------------

### SE(3)/SO(3) Exponential and Logarithm Maps

Source: https://context7.com/edexheim/depthcov/llms.txt

Implements SE(3)/SO(3) exponential and logarithm maps crucial for pose composition, inversion, and gradient-based pose optimization. The twist vector xi represents a small rotation and translation. Ensure correct input dimensions for twist vectors and matrices.

```python
import torch
from depth_cov.utils.lie_algebra import (
    se3_exp, batch_se3, invertSE3, SE3_logmap,
    SO3_expmap, SO3_logmap, skew_symmetric
)

# Twist vector xi = [omega (3), v (3)] — small rotation + translation
xi = torch.tensor([[0.01, 0.02, -0.01, 0.1, -0.05, 0.2]])  # (1, 6)
T = se3_exp(xi)          # (1, 4, 4) SE(3) matrix
print("SE3 matrix:\n", T)

# Batch exponential map (used inside optimiser)
T_batch = batch_se3(xi)  # (1, 4, 4)

# Invert
T_inv = invertSE3(T)
print("T @ T_inv ≈ I:", torch.allclose(T @ T_inv, torch.eye(4).unsqueeze(0), atol=1e-6))

# Log map back to twist
xi_recovered = SE3_logmap(T)
print("Recovered xi ≈ original:", torch.allclose(xi_recovered, xi, atol=1e-5))

# SO(3) expmap / logmap
omega = torch.tensor([[0.1, -0.05, 0.2]])   # (1, 3)
R = SO3_expmap(omega)                        # (1, 3, 3)
omega_back = SO3_logmap(R)                   # (1, 3)
print("SO3 roundtrip error:", (omega - omega_back).abs().max().item())  # < 1e-6
```

--------------------------------

### Image Pyramid and Gradient Computation

Source: https://context7.com/edexheim/depthcov/llms.txt

Provides differentiable image pyramid generation, Scharr-kernel gradient computation, Gaussian blur, and intrinsics rescaling. These are essential for photometric tracking within the odometry pipeline. Ensure the correct device and data type are specified.

```python
import torch
from depth_cov.utils.image_processing import (
    ImagePyramidModule, ImageGradientModule, GaussianBlurModule,
    IntrinsicsPyramidModule
)

device = torch.device("cpu")
img = torch.rand(1, 1, 192, 256)   # grayscale frame

# Build 4-level Gaussian image pyramid (levels 0–3)
pyr_mod = ImagePyramidModule(channels=1, start_level=0, end_level=3,
                              device=device, dtype=torch.float32)
pyr = pyr_mod(img)
for lvl, p in enumerate(pyr):
    print(f"Pyramid level {lvl}: {p.shape}")
# Pyramid level 0: torch.Size([1, 1, 192, 256])
# Pyramid level 1: torch.Size([1, 1, 96, 128])
# ...

# Scharr gradients
grad_mod = ImageGradientModule(channels=1, device=device, dtype=torch.float32)
gx, gy = grad_mod(img)   # each (1, 1, 192, 256)

# Rescale camera intrinsics for each pyramid level
K = torch.tensor([[525., 0., 319.5],
                   [0., 525., 239.5],
                   [0.,   0.,   1. ]]).unsqueeze(0)   # (1, 3, 3)
intr_pyr = IntrinsicsPyramidModule(start_level=0, end_level=3)
K_pyr = intr_pyr(K)
for lvl, Ki in enumerate(K_pyr):
    print(f"Intrinsics level {lvl}: fx={Ki[0,0,0]:.1f}")
# Intrinsics level 0: fx=525.0
# Intrinsics level 1: fx=262.5  ...
```

--------------------------------

### Visual Odometry Configuration

Source: https://context7.com/edexheim/depthcov/llms.txt

YAML configuration for the visual odometry system, controlling tracking and mapping hyper-parameters. This configuration is designed for real-time performance.

```yaml
# config/visual_odom.yml  — real-time multi-threaded config (~30 FPS on RTX 3080)
tracking:
  device: cpu
  dtype: float
  pyr:
    start_level: 0
    end_level: 3
  term_criteria:
    max_iter: 50          # reduce vs. paper's 100 for real-time speed
    delta_norm: 1.0e-8
  sigmas:
    photo: 1.0e-1
  keyframing:
    kf_depth_motion_ratio: 0.12   # new KF when camera moves >12% of median depth
    kf_num_pixels_frac: 0.75

mapping:
  device: cuda:0
  dtype: double
  model_path: models/scannet.ckpt
  graph:
    num_keyframes: 12       # larger sliding window than paper config (8)
    num_one_way_frames: 6
    radius: 0.1             # edge connection radius in 3D space
  term_criteria:
    max_iter: 20
  sigmas:
    photo: 1.0e-1
    mean_depth_prior: 1.0e+0
    scale_prior: 1.0e-4
    pose_prior: 1.0e-4
  sampling:
    mode: greedy_conditional_entropy
    max_samples: 64
    max_stdev_thresh: 0.04
```

--------------------------------

### Run Odometry on Dataset

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Utilizes the 'odom_dataset.py' script to run the odometry pipeline on a given dataset. This script is part of the DepthCov-VO system.

```python
depth_cov/odom_dataset.py
```

--------------------------------

### Train Depth Estimation Model

Source: https://context7.com/edexheim/depthcov/llms.txt

Trains a non-stationary Gaussian Process model for depth estimation using PyTorch Lightning. Resumes training from a checkpoint if provided.

```python
import pytorch_lightning as pl
from depth_cov.core.NonstationaryGpModule import NonstationaryGpModule
from depth_cov.data.data_modules import DepthCovDataModule
from pytorch_lightning.callbacks import ModelCheckpoint

def train(batch_size, model_path=None):
    data_module = DepthCovDataModule(batch_size)

    model = (NonstationaryGpModule() if model_path is None
             else NonstationaryGpModule.load_from_checkpoint(model_path))

    checkpoint_callback = ModelCheckpoint(
        monitor="loss_val",
        dirpath="./models/",
        filename="gp-{epoch:02d}-{loss_val:.4f}"
    )

    trainer = pl.Trainer(
        callbacks=[checkpoint_callback],
        gpus=1,
        max_epochs=1000,
        limit_train_batches=0.01,
        limit_val_batches=0.01,
    )
    trainer.fit(model, data_module)

if __name__ == "__main__":
    pl.seed_everything(42, workers=True)
    train(batch_size=4, model_path=None)          # pass "models/gp-....ckpt" to resume

# Dataset file format required:
# ScanNet: ./dataset/scannet_train.txt — each line: "rgb_path,depth_path"
# NYUv2:   ./dataset/nyudepthv2_train.txt — each line: path to .h5 file
```

--------------------------------

### Run Network Training

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Executes the training script for jointly training GP hyperparameters and UNet parameters. This script is located at 'depth_cov/train.py'.

```bash
depth_cov/train.py
```

--------------------------------

### DepthCov Training Pipeline

Source: https://context7.com/edexheim/depthcov/llms.txt

Initiates the joint training of the UNet GP hyperparameter network using PyTorch Lightning. Supports resuming training from a checkpoint and saves best models based on validation loss.

```python
# Jointly trains the UNet GP hyperparameter network using PyTorch Lightning.
# Accepts an optional checkpoint path to resume training.
# Saves best checkpoints monitored on validation loss.

```

--------------------------------

### Create Conda Environment

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Creates a new conda environment named 'depth_cov' with Python 3.8.15. This is the first step in setting up the project's dependencies.

```bash
conda create -n depth_cov python=3.8.15
```

--------------------------------

### GP Sparse/VFE Module: Constant Noise Variant

Source: https://context7.com/edexheim/depthcov/llms.txt

Implements the Variational Free Energy (inducing-point) approximation with constant noise. Used for scalable GP inference.

```python
import torch
from depth_cov.core.gp_vfe import GpVfeModuleConstantNoise, GpVfeModuleVaryingNoise

B, M, N = 1, 64, 192*256   # batch, inducing points, test pixels

K_mm = torch.eye(M).unsqueeze(0)                      # (1, M, M) inducing-point cov
K_mn = torch.randn(B, M, N) * 0.1                     # (1, M, N) cross-covariance
K_nn_diag = torch.ones(B, N)                          # (1, N) prior diagonal variances
y_m  = torch.randn(B, M, 1)                           # (1, M, 1) pseudo-observations
mean = torch.zeros(B, 1, 1)

# Constant noise variant
noise = torch.tensor(1e-3)
vfe_const = GpVfeModuleConstantNoise()
pred_mean, pred_var, elbo = vfe_const(K_mm, K_mn, K_nn_diag, y_m, noise, mean, K_nn_diag)
print("VFE pred mean:", pred_mean.shape)  # (1, N, 1)
print("ELBO:         ", elbo.item())
```

--------------------------------

### GP Sparse/VFE Module: Varying Noise Variant

Source: https://context7.com/edexheim/depthcov/llms.txt

Implements the Variational Free Energy (inducing-point) approximation with varying (heteroscedastic) noise. Used for scalable GP inference.

```python
import torch
from depth_cov.core.gp_vfe import GpVfeModuleConstantNoise, GpVfeModuleVaryingNoise

B, M, N = 1, 64, 192*256   # batch, inducing points, test pixels

K_mm = torch.eye(M).unsqueeze(0)                      # (1, M, M) inducing-point cov
K_mn = torch.randn(B, M, N) * 0.1                     # (1, M, N) cross-covariance
K_nn_diag = torch.ones(B, N)                          # (1, N) prior diagonal variances
y_m  = torch.randn(B, M, 1)                           # (1, M, 1) pseudo-observations
mean = torch.zeros(B, 1, 1)

# Varying (heteroscedastic) noise variant
noise_vec = torch.rand(B, M) * 0.01 + 1e-4           # (1, M) per-point noise
vfe_vary  = GpVfeModuleVaryingNoise()
pred_mean2, pred_var2, elbo2 = vfe_vary(K_mm, K_mn, K_nn_diag, y_m, noise_vec, mean, K_nn_diag)
print("VFE pred var (varying):", pred_var2.shape)     # (1, N, 1)
```

--------------------------------

### Sample Sparse Observation Coordinates

Source: https://context7.com/edexheim/depthcov/llms.txt

Greedily selects up to 64 maximally-informative observation coordinates based on conditional entropy or random uniform sampling, with an optional standard deviation threshold.

```python
sparse_coords_norm = sample_sparse_coords_norm(
    gaussian_covs,
    max_samples=64,
    mode="greedy_conditional_entropy",    # or "random_uniform"
    max_stdev_thresh=0.04,
    model=model,
    model_level=-1                        # finest pyramid level
)
print("Selected coords shape:", sparse_coords_norm.shape)  # (1, 64, 2), normalised [-1,1]
```

--------------------------------

### 2x2 Matrix Operations with PyTorch

Source: https://context7.com/edexheim/depthcov/llms.txt

Performs vectorised 2x2 matrix operations like determinant, inverse, and Cholesky decomposition on batches of matrices. Useful for GP covariance computations.

```python
import torch
from depth_cov.utils.lin_alg import det2x2, inv2x2, cholesky2x2, trace2x2, chol_log_det, trace

# (B, H, W, 2, 2) batch of 2x2 SPD matrices
B, H, W = 2, 192, 256
mats = torch.eye(2).expand(B, H, W, 2, 2).clone()
mats[..., 0, 1] = 0.1; mats[..., 1, 0] = 0.1   # slight off-diagonal

dets = det2x2(mats)                    # (2, 192, 256)
invs, dets2 = inv2x2(mats)            # (2, 192, 256, 2, 2), (2, 192, 256)
L_chol = cholesky2x2(mats, upper=True) # (2, 192, 256, 2, 2) upper triangular

print("det shape:", dets.shape)        # (2, 192, 256)
print("inv shape:", invs.shape)        # (2, 192, 256, 2, 2)
print("chol shape:", L_chol.shape)     # (2, 192, 256, 2, 2)

# For larger (B, N, N) matrices
L_big = torch.linalg.cholesky(torch.eye(8).unsqueeze(0))  # (1, 8, 8)
print("Log det via chol:", chol_log_det(L_big).item())     # 0.0  (identity)
```

--------------------------------

### Activate Conda Environment

Source: https://github.com/edexheim/depthcov/blob/master/README.md

Activates the previously created 'depth_cov' conda environment. All subsequent commands should be run within this activated environment.

```bash
conda activate depth_cov
```

--------------------------------

### Convert UNet Output to Covariance Matrices

Source: https://context7.com/edexheim/depthcov/llms.txt

Converts raw UNet output scalars (log_lambda1, log_lambda2, angle) into 2x2 positive-definite covariance matrices. Also provides functionality to interpolate kernel parameters between different resolution levels.

```python
import torch
import depth_cov.core.gaussian_kernel as gk

B, H, W = 1, 192, 256

# Simulate UNet output: (B, 3, H, W) — log_lambda1, log_lambda2, angle
raw = torch.randn(B, 3, H, W)

# Convert to (B, H*W, 2, 2) covariance matrices
E = gk.kernel_params_to_covariance(raw)   # (B, H*W, 2, 2)
print("E shape:", E.shape)                 # (1, 49152, 2, 2)
print("Symmetric:", torch.allclose(E, E.transpose(-1, -2)))  # True

# Interpolate kernel params from a coarser level to a target resolution
coarse_params = torch.randn(B, 3, 48, 64)
fine_params = gk.interpolate_kernel_params(coarse_params, target_size=[192, 256])
print("Interpolated:", fine_params.shape)  # (1, 3, 192, 256)
```

--------------------------------

### Build and Use UNet for Covariance Prediction

Source: https://context7.com/edexheim/depthcov/llms.txt

Constructs a multi-level UNet with residual blocks and GroupNorm for predicting Gaussian covariance parameters. Outputs a feature pyramid at each decoder level. Ensure the model is in evaluation mode and gradients are disabled when not training.

```python
import torch
from depth_cov.nn.UNet import UNet
from depth_cov.nn.layers import ResidualConv, DownConv, UpConv

# Build a 4-level UNet for predicting 3-parameter Gaussian covariance fields
unet = UNet(
    num_levels=4,
    in_channels=3,
    base_feature_channels=32,
    feature_channels=3,          # outputs [log_l1, log_l2, angle] per pixel
    kernel_size=3,
    padding=1,
    stride=1,
    feature_act=torch.nn.Softplus()
)
unet.eval()

rgb = torch.rand(2, 3, 192, 256)   # (batch=2, C, H, W)
with torch.no_grad():
    features = unet(rgb)           # list of 3 tensors (num_levels-1 outputs)

for i, f in enumerate(features):
    print(f"Level {i} features: {f.shape}")
# Level 0 features: torch.Size([2, 3, 192, 256])
# Level 1 features: torch.Size([2, 3, 96, 128])
# Level 2 features: torch.Size([2, 3, 48, 64])
```

--------------------------------

### Compute Non-Stationary Covariance Kernels

Source: https://context7.com/edexheim/depthcov/llms.txt

Calculates non-stationary kernel matrices using various covariance functions and probability products. Wraps low-level functions into PyTorch Modules for use in GP layers.

```python
import torch
from depth_cov.core.kernels import (
    nonstationary, squared_exponential, matern,
    prob_product_quad, prob_product_constant,
    diagonal_prob_product,
)
from depth_cov.core.covariance import CovarianceModule, DiagonalCovarianceModule

# Suppose E are (B, N, 2, 2) covariance matrices predicted by the UNet
B, N = 1, 4
coords = torch.rand(B, N, 2)          # (batch, points, 2) normalised coords in [-1,1]
E = torch.eye(2).unsqueeze(0).unsqueeze(0).expand(B, N, 2, 2) * 0.1  # isotropic

# Quadratic form distances between all pairs
Q = prob_product_quad(coords, E, coords, E)   # (B, N, N)
C = prob_product_constant(E, E)               # (B, N, N)

K_se = squared_exponential(Q) * C            # Squared-exponential non-stationary kernel
K_m  = matern(Q) * C                         # Matérn non-stationary kernel
print("K_se:", K_se.shape)  # (1, 4, 4)

# Diagonal (prior variance only — no cross-covariance)
Q_diag, C_diag = diagonal_prob_product(coords, E)  # each (B, N)
K_diag = squared_exponential(Q_diag) * C_diag
print("K_diag:", K_diag.shape)  # (1, 4)

# Module wrappers used inside NonstationaryGpModule
scale_param = torch.nn.Parameter(torch.zeros(1))
cov_mod = CovarianceModule(iso_cov_fn=squared_exponential,
                            scale_param=scale_param, scale_prior=1.0)
K_full = cov_mod(coords, E)   # (1, 4, 4)
```

--------------------------------

### Incremental Cholesky Update: In-place Variant

Source: https://context7.com/edexheim/depthcov/llms.txt

Performs an efficient in-place rank-1 Cholesky update. Used by the odometry mapper to incrementally incorporate new depth observations without recomputing the full factorization.

```python
import torch
from depth_cov.core.inc_chol import update_chol_inplace, update_obs_info_inplace, get_new_chol_obs_info

B, M = 1, 8   # batch size, current number of inducing points

# Pre-allocated lower-triangular storage (M+1) x (M+1)
L = torch.zeros(B, M+1, M+1)
L[:, :M, :M] = torch.linalg.cholesky(torch.eye(M).unsqueeze(0) + 0.1*torch.randn(B, M, M).abs())

# New observation: cross-covariance column (M x 1) and self-variance (1 x 1)
k_ni = torch.randn(B, M, 1)
k_ii = torch.ones(B, 1, 1) * 2.0

# Update L in-place at position `ind`
ind = M
update_chol_inplace(L, k_ni, k_ii, ind)
print("Updated Cholesky block L[ind,ind]:", L[:, ind, ind])  # new diagonal element
```

--------------------------------

### Incremental Cholesky Update: Non-in-place Variant

Source: https://context7.com/edexheim/depthcov/llms.txt

Calculates new Cholesky information and updated observation information without modifying the original Cholesky factor. Useful for tracking state changes.

```python
import torch
from depth_cov.core.inc_chol import update_chol_inplace, update_obs_info_inplace, get_new_chol_obs_info

B, M = 1, 8   # batch size, current number of inducing points

# Pre-allocated lower-triangular storage (M+1) x (M+1)
L = torch.zeros(B, M+1, M+1)
L[:, :M, :M] = torch.linalg.cholesky(torch.eye(M).unsqueeze(0) + 0.1*torch.randn(B, M, M).abs())

# New observation: cross-covariance column (M x 1) and self-variance (1 x 1)
k_ni = torch.randn(B, M, 1)
k_ii = torch.ones(B, 1, 1) * 2.0

# Non-in-place variant returning new column + obs_info update
obs_info = torch.randn(B, M, 50)    # current L_inv @ K_mn
k_id     = torch.randn(B, 1, 50)   # new row of K_mn
l_ni, l_ii, obs_info_new = get_new_chol_obs_info(L[:, :M, :M], obs_info, k_ni, k_id, k_ii)
print("New obs_info_new shape:", obs_info_new.shape)  # (1, 1, 50)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.