### Install UV on Linux

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Install the UV package manager using a curl script. This is a prerequisite for the UV-based installation scripts.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Install UV on Windows

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Installs the UV package manager on Windows using a PowerShell script. This is a prerequisite for the UV-based installation script on Windows.

```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```

--------------------------------

### Install Dependencies

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut_playground/headless.ipynb

Installs necessary libraries like matplotlib and ipywidgets for visualization and interactive elements. Use this command in your environment.

```bash
!pip install matplotlib ipywidgets --quiet
```

--------------------------------

### Visualize Training Progress Interactively (Viser GUI)

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Launch training with the Viser GUI for interactive visualization. Install 'viser' and forward port 8080 if running remotely.

```bash
python train.py --config-name apps/nerf_synthetic_3dgut.yaml path=data/nerf_synthetic/lego with_viser_gui=True
```

--------------------------------

### Clone Repository

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Clone the 3DGRUT repository and navigate into the directory. This is the initial step for all installation methods.

```bash
git clone --recursive https://github.com/nv-tlabs/3dgrut.git
cd 3dgrut
```

--------------------------------

### Load and Initialize Datasets from Configuration

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

This example demonstrates loading a configuration file, setting the dataset path and type, and then creating train and validation datasets using the `make` function. It also prints the number of frames in each dataset.

```python
from omegaconf import OmegaConf
import threedgrut.datasets as datasets

conf = OmegaConf.load("config.yaml")
conf.path = "data/mipnerf360/bonsai"
conf.dataset.type = "colmap"

train_ds, val_ds = datasets.make(
    conf.dataset.type,
    conf,
    ray_jitter=None
)

print(f"Train frames: {len(train_ds)}")
print(f"Val frames: {len(val_ds)}")
```

--------------------------------

### Create Local venv CUDA Environment (Linux)

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Downloads and installs a specified CUDA toolkit version locally within the virtual environment. Set FORCE_LOCAL_CUDA=1 to ensure local installation. Requires wget.

```bash
# Supported CUDA_VERSION values: 11.8 (or 11), 12.4, 12.6, 12.8 (or 12), 13.0 (or 13)
FORCE_LOCAL_CUDA=1 CUDA_VERSION=12 ./scripts/create_venv_cuda.sh   # ~4 GB download on first run
source .venv/bin/activate
./install_env_uv.sh
```

--------------------------------

### Initialize 3DGRUT Trainer from PLY File

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/trainer.md

Create a trainer instance initialized with data from a PLY file, which is useful for starting training with pre-existing geometry.

```python
conf = OmegaConf.load("configs/apps/colmap_3dgut.yaml")
trainer = Trainer3DGRUT.create_from_ply(
    "gaussians.ply",
    conf
)

```

--------------------------------

### Install 3DGRUT Environment with System CUDA (Linux)

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Installs the 3DGRT Python environment using UV, assuming a system CUDA installation. The virtual environment name defaults to '3dgrut'. Activate the environment after installation.

```bash
./install_env_uv.sh          # venv name defaults to "3dgrut"
source .venv/bin/activate
```

--------------------------------

### Install 3DGRUT Environment on Windows

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Installs the 3DGRT Python environment using UV on Windows. The virtual environment name defaults to '3dgrut'. CUDA and build tools are auto-detected.

```powershell
.\install_env_uv.ps1                     # auto-detects CUDA, venv name defaults to "3dgrut"
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut_playground/headless.ipynb

Imports core libraries for 3DGRUT, Kaolin, PyTorch, and visualization tools. Ensure these are installed before running.

```python
import os
import copy
import numpy as np
import torch
import torchvision.transforms.functional as F
import kaolin
from matplotlib import pyplot as plt

from threedgrut.utils.logger import logger
from threedgrut.gui.ps_extension import initialize_cugl_interop
from threedgrut_playground.utils.video_out import VideoRecorder
from threedgrut_playground.engine import Engine3DGRUT, OptixPrimitiveTypes
```

--------------------------------

### Command-Line Configuration Overrides

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/configuration.md

Example of overriding configuration parameters at runtime using command-line arguments for training. This allows for quick experimentation without modifying config files.

```bash
python train.py \
  --config-name apps/colmap_3dgut.yaml \
  path=data/mipnerf360/bonsai \
  out_dir=runs \
  experiment_name=bonsai_3dgut \
  dataset.downsample_factor=2 \
  optimizer.lr=0.002 \
  n_iterations=50000
```

--------------------------------

### Configuration Hierarchy Example

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/INDEX.md

This YAML snippet illustrates the hierarchical structure of configuration parameters for various aspects of the 3DGRUT project, including model, rendering, dataset, optimizer, strategy, post-processing, and training settings.

```yaml
model:
  background:
    name, color, learnable_color
  density_activation, scale_activation
  progressive_training:
    feature_type, init_n_features, max_n_features

render:
  method, particle_radiance_sph_degree
  particle_kernel_density_clamping, min_transmittance

dataset:
  type, path, downsample_factor
  # Type-specific: load_exif, camera_ids, etc.

optimizer:
  type, lr, lr_scheduler
  betas, eps, max_grad_norm
  # Per-parameter: lr_positions, lr_features, etc.

strategy:
  method
  # Type-specific: gs.*, mcmc.*

post_processing:
  method
  # Type-specific: use_controller, etc.

Training:
  n_iterations, val_frequency, num_workers
  save_checkpoint_frequency, enable_tensorboard

```

--------------------------------

### Create Conda Environment with CUDA (Linux)

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Creates a conda environment and installs a specified CUDA toolkit version. Replace '12' with your desired CUDA version. This is part of the conda-managed CUDA setup.

```bash
# Step 1: create a conda environment with the CUDA toolkit
CUDA_VERSION=12 ./scripts/create_conda.sh 3dgrut
conda activate 3dgrut
# Step 2: install Python dependencies
./install_env_uv.sh  # or conda run -n 3dgrut ./install_env_uv.sh
```

--------------------------------

### setup_gui

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/utilities.md

Initializes an interactive visualization GUI for the 3DGRUT model. It requires the model, training and validation datasets, and the scene bounding box.

```APIDOC
## setup_gui

### Description
Initialize interactive visualization GUI.

### Method
`setup_gui(
    model: MixtureOfGaussians,
    train_dataset: BoundedMultiViewDataset,
    val_dataset: BoundedMultiViewDataset,
    scene_bbox: tuple[torch.Tensor, torch.Tensor]
) -> Any`

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- **model** (MixtureOfGaussians) - Required - Model to visualize
- **train_dataset** (Dataset) - Required - Training dataset for poses
- **val_dataset** (Dataset) - Required - Validation dataset
- **scene_bbox** (tuple) - Required - Scene bounds (tuple of two torch.Tensors)

### Request Example
```python
# Example usage (assuming model, train_dataset, val_dataset, and scene_bbox are defined)
gui_interface = setup_gui(model, train_dataset, val_dataset, scene_bbox)
```

### Response
#### Success Response (200)
- **GUI Interface** (Any) - GUI interface object

#### Response Example
```json
{
  "example": "<GUI Interface Object>"
}
```

### Requirements
`with_gui=true` in config
```

--------------------------------

### Initialize Renderer from Checkpoint

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/renderer.md

Load a trained model from a checkpoint file and set up the output directory for evaluation. This is the standard way to begin rendering.

```python
from threedgrut.render import Renderer

renderer = Renderer.from_checkpoint(
    checkpoint_path="runs/lego/ckpt_last.pt",
    out_dir="outputs/eval"
)

metrics = renderer.render_all()
print(f"Average PSNR: {metrics['psnr']:.2f}")
```

--------------------------------

### Install 3DGRUT with Legacy Script and GCC11 (CUDA 12.8)

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Installs the 3DGRT environment using the legacy install script, specifying CUDA version 12.8.1 and WITH_GCC11 for compatibility with Blackwell/RTX 50 series.

```bash
CUDA_VERSION=12.8.1 ./install_env.sh 3dgrut_cuda12 WITH_GCC11
```

--------------------------------

### Visualize Training Progress Interactively (GUI)

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Launch training with an interactive GUI for visualizing progress. Ensure the DISPLAY environment variable is set if running remotely.

```bash
python train.py --config-name apps/nerf_synthetic_3dgut.yaml path=data/nerf_synthetic/lego with_gui=True
```

--------------------------------

### Initialize and Display 3DGRUT Visualizer

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut_playground/headless.ipynb

Sets up the camera, initializes the `IpyTurntableVisualizer` with the defined rendering functions, and displays the interactive canvas within a Jupyter environment. Includes optional GUI elements.

```python
# Create initial camera
camera = kaolin.render.easy_render.default_camera(512)
camera.change_coordinate_system(
    torch.tensor([[1, 0, 0],
                  [0, 0, 1],
                  [0, -1, 0]]
))
camera = camera.cuda()
# Initialize renderer
visualizer = kaolin.visualize.IpyTurntableVisualizer(
    height=camera.height,
    width=camera.width,
    camera=copy.deepcopy(camera),
    render=render,
    fast_render=fast_render,
    max_fps=8,
    world_up_axis=1
)

# Show the canvas and callback listener
vbox = widgets.VBox([denoiser_checkbox, aa_checkbox, aa_mode_combo, spp_slider])
hbox = widgets.HBox([visualizer.canvas, vbox])
display(hbox, visualizer.out)

# OR without a GUI, its as simple as:
# visualizer.show()
```

--------------------------------

### Set Up Playground Parameters

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut_playground/headless.ipynb

Define paths for the 3DGRUT checkpoint, mesh assets, and default configuration. These parameters are essential for initializing the engine.

```python
gs_object = "3dgrut/runs/<YOUR_MODEL_HERE>/ckpt_last.pt"
mesh_assets_folder = "./assets"
default_config = "apps/colmap_3dgrt.yaml"
```

--------------------------------

### Install Playground Requirements

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut_playground/README.md

Installs necessary dependencies for the 3DGRT Playground using conda and pip. Includes a note about potential OpenGL header requirements.

```bash
conda install -c conda-forge mesa-libgl-devel-cos7-x86_64 # may be necessary for OpenGL headers
pip install -r threedgrut_playground/requirements.txt
```

--------------------------------

### Basic 3DGRUT Training Loop

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/trainer.md

This snippet demonstrates how to load configuration, initialize the Trainer3DGRUT, and run the main training loop, including validation and checkpoint saving.

```python
from omegaconf import OmegaConf
from threedgrut.trainer import Trainer3DGRUT

# Load configuration
conf = OmegaConf.load("configs/apps/colmap_3dgut.yaml")
conf.path = "data/mipnerf360/bonsai"
conf.out_dir = "runs"
conf.experiment_name = "bonsai_3dgut"

# Create trainer
trainer = Trainer3DGRUT(conf)

# Main training loop
while trainer.global_step < conf.n_iterations:
    for batch in trainer.train_dataloader:
        batch = trainer.train_dataset.get_gpu_batch_with_intrinsics(batch)
        loss = trainer.train_iteration(batch, trainer.global_step)
        trainer.global_step += 1
        
        if trainer.global_step % trainer.val_frequency == 0:
            trainer.validation_epoch()
            trainer.save_checkpoint()

```

--------------------------------

### Override CUDA_HOME on Windows

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Sets the CUDA_HOME environment variable to a specific CUDA installation path before running the UV installation script on Windows. This is useful if the auto-detection fails.

```powershell
$env:CUDA_HOME = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8"
.\install_env_uv.ps1
```

--------------------------------

### Basic Training Loop

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/README.md

This snippet demonstrates how to load a configuration, set dataset parameters, create a trainer, and run the training loop for a 3DGRUT model. Ensure the configuration file and data path are correctly specified.

```python
from omegaconf import OmegaConf
from threedgrut.trainer import Trainer3DGRUT

# Load configuration
conf = OmegaConf.load("configs/apps/colmap_3dgut.yaml")
conf.path = "data/mipnerf360/bonsai"
conf.dataset.downsample_factor = 2

# Create trainer
trainer = Trainer3DGRUT(conf)

# Training loop
while trainer.global_step < conf.n_iterations:
    for batch in trainer.train_dataloader:
        batch = trainer.train_dataset.get_gpu_batch_with_intrinsics(batch)
        loss = trainer.train_iteration(batch, trainer.global_step)
        trainer.global_step += 1
```

--------------------------------

### 3DGRUT USD Exporter Help

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut/export/usd/post_processing/README.md

View the canonical CLI surface and available options for the 3DGRUT USD exporter by running its help command.

```bash
python -m threedgrut.export.scripts.export_usd --help
```

--------------------------------

### num_gaussians

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/model.md

Get the current number of Gaussians in the model.

```APIDOC
## num_gaussians

### Description
Get the current number of Gaussians in the model.

### Method
`@property`

### Returns
`int`
```

--------------------------------

### Initialize Trainer3DGRUT

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/trainer.md

Instantiate the Trainer3DGRUT with a configuration object and an optional device. Ensure OmegaConf is used for loading configurations.

```python
from omegaconf import OmegaConf
from threedgrut.trainer import Trainer3DGRUT

conf = OmegaConf.load("config.yaml")
trainer = Trainer3DGRUT(conf, device="cuda:0")
```

--------------------------------

### Get Number of Gaussians

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Retrieve the total count of Gaussians in the model.

```python
accessor.get_num_gaussians()
```

--------------------------------

### Python Configuration Loading and Merging

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/configuration.md

Demonstrates loading a base configuration file using OmegaConf, merging it with a dictionary for overrides, and accessing nested configuration values.

```python
from omegaconf import OmegaConf

# Load base config
conf = OmegaConf.load("configs/apps/colmap_3dgut.yaml")

# Override with command-line args
conf = OmegaConf.merge(conf, {"path": "data/mipnerf360/bonsai"})

# Access nested values
print(conf.model.background.color)
print(conf.dataset.downsample_factor)
```

--------------------------------

### Initialize Densification and Pruning Strategy

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/trainer.md

Sets up the strategy for densification, pruning, and parameter updates. Supports 'GSStrategy' and 'MCMCStrategy'. Requires the model to be initialized beforehand.

```python
def init_densification_and_pruning_strategy(self, conf: DictConfig) -> None:
    Set up the strategy for densification, pruning, and parameter updates.

    **Supported Strategies:**
    - `GSStrategy`: Standard 3D Gaussian Splatting densification/pruning
    - `MCMCStrategy`: MCMC-based densification per arXiv:2404.09591

    | Parameter | Type | Description |
    |-----------|------|-------------|
    | conf | `DictConfig` | Configuration with `strategy.method` setting |

    **Precondition:** `self.model` must be initialized
```

--------------------------------

### Create and Render with MixtureOfGaussians

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/model.md

Demonstrates how to initialize a MixtureOfGaussians model, add Gaussians with specified properties, and render a batch of rays. Requires OmegaConf for configuration.

```python
from omegaconf import OmegaConf
from threedgrut.model.model import MixtureOfGaussians
import torch

conf = OmegaConf.load("config.yaml")
model = MixtureOfGaussians(conf, scene_extent=1.0)
model.build_acc()

# Add some Gaussians
with torch.no_grad():
    model.positions.resize_(100, 3)
    model.rotation.resize_(100, 4)
    model.scale.resize_(100, 3)
    model.density.resize_(100, 1)
    model.features_albedo.resize_(100, 3)
    model.features_specular.resize_(100, 9)
    
    # Initialize random values
    model.positions.uniform_(-1, 1)
    model.rotation.normal_()
    model.rotation /= model.rotation.norm(dim=1, keepdim=True)
    model.scale.uniform_(-2, 2)
    model.density.uniform_(0, 1)
    model.features_albedo.uniform_(-1, 1)
    model.features_specular.uniform_(-1, 1)

# Render a batch
batch = ...  # Batch with rays and poses
output = model.forward(batch)
print(f"Rendered RGB: {output['rgb'].shape}")
print(f"Loss: {output['loss'].item():.4f}")
```

--------------------------------

### check_step_condition

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/utilities.md

Checks if an action should be executed at the current training step based on start, end, and frequency parameters.

```APIDOC
## check_step_condition

### Description
Check if action should execute at given step.

### Method
```python
def check_step_condition(
    current_step: int,
    start_step: int,
    end_step: int,
    frequency: int
) -> bool
```

### Parameters
#### Path Parameters
- **current_step** (int) - Current training step
- **start_step** (int) - Starting step
- **end_step** (int) - Ending step
- **frequency** (int) - Execution frequency

### Returns
bool - True if should execute

### Example
```python
if check_step_condition(step, 500, 15000, 100):
    # Densify every 100 steps between steps 500-15000
    densify()
```
```

--------------------------------

### Constructor

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/optimizers-and-strategies.md

Initializes the Strategy Base Class with configuration and a model.

```APIDOC
## Constructor

### Description
Initializes the Strategy Base Class with configuration and a model.

### Parameters
#### Path Parameters
- **config** (`DictConfig`) - Required - Strategy configuration
- **model** (`MixtureOfGaussians`) - Required - Model to optimize
```

--------------------------------

### Get Number of Gaussians

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Retrieves the total count of Gaussians stored within the GaussianAttributes object. This is a read-only property.

```python
@property
def num_gaussians(self) -> int:
    # Get number of Gaussians.
    pass
```

--------------------------------

### Suspend Strategy for Fine-Tuning

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/optimizers-and-strategies.md

Example of suspending the strategy and freezing Gaussians for fine-tuning, focusing training on a post-processing module.

```python
# Freeze Gaussians and disable densification for PPISP training
trainer.model.freeze_gaussians()
trainer.strategy.suspend()

# Train only post-processing module
for global_step in range(num_distillation_steps):
    batch = next(train_loader)
    batch_gpu = train_dataset.get_gpu_batch_with_intrinsics(batch)
    
    output = trainer.model(batch_gpu)
    loss = output['loss']
    loss.backward()
    
    trainer.post_processing_optimizer.step()
    trainer.optimizer.zero_grad()
```

--------------------------------

### Resume Training from Checkpoint

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/README.md

Load a previously saved training checkpoint and configuration to resume training. Ensure the checkpoint path and configuration file are correct.

```python
conf = OmegaConf.load("config.yaml")
trainer = Trainer3DGRUT.create_from_checkpoint(
    "runs/scene/ckpt_last.pt",
    conf
)
```

--------------------------------

### Resume 3DGRUT Training from Checkpoint

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/trainer.md

Initialize the trainer from a previously saved checkpoint to continue training. Requires the checkpoint path and the configuration object.

```python
conf = OmegaConf.load("configs/apps/colmap_3dgut.yaml")
trainer = Trainer3DGRUT.create_from_checkpoint(
    "runs/bonsai/ckpt_last.pt",
    conf
)
# Continue training

```

--------------------------------

### Get Maximum Supported Spherical Harmonics Degree

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Retrieve the maximum Spherical Harmonics (SH) degree that the model supports.

```python
accessor.get_max_sh_degree()
```

--------------------------------

### Get Observer Points

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Fetches the 3D coordinates of camera centers. These points are used for visibility estimations within the scene.

```python
def get_observer_points(self) -> np.ndarray
```

--------------------------------

### Get Activation Function

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/utilities.md

Retrieves a specified activation function by its name. Supports inverse functions and a variety of common activation types.

```APIDOC
## get_activation_function

### Description
Get activation function by name.

### Method
`get_activation_function(name: str, inverse: bool = False) -> Callable[[torch.Tensor], torch.Tensor]`

### Parameters
#### Path Parameters
- **name** (str) - Required - Activation name (sigmoid, exp, softplus, normalize).
- **inverse** (bool) - Optional - Default: False - Return inverse function.

### Supported Functions
| Name | Function | Inverse |
|------|----------|---------|
| sigmoid | σ(x) = 1/(1+e^(-x)) | log(x/(1-x)) |
| exp | e^x | log(x) |
| softplus | log(1 + e^x) | log(e^x - 1) |
| normalize | x / ‖x‖ | Identity |

### Example
```python
from threedgrut.utils.misc import get_activation_function

sigmoid = get_activation_function("sigmoid")
densities = sigmoid(density_pre_activation)

sigmoid_inv = get_activation_function("sigmoid", inverse=True)
density_pre = sigmoid_inv(densities)
```
```

--------------------------------

### PLYExporter Usage

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Demonstrates how to export Gaussian splats to the PLY file format. Requires an instance of PLYExporter and a GaussianExportAccessor to get model attributes.

```python
from threedgrut.export.formats.ply import PLYExporter

exporter = PLYExporter()
accessor = GaussianExportAccessor(model, conf)
attrs = accessor.get_attributes()

exporter.export("output.ply", attrs)
```

--------------------------------

### Configure Multi-Camera Datasets

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Illustrates how to select specific cameras for a dataset and retrieve frame counts per camera and the camera index for a given frame.

```python
conf = OmegaConf.load("config.yaml")

# Use only specific cameras
conf.dataset.camera_ids = [0, 1]  # Only use cameras 0 and 1

train_ds, val_ds = datasets.make(conf.dataset.type, conf, None)

frames_per_cam = train_ds.get_frames_per_camera()
# [1000, 1000]  # 1000 frames per camera

# Get camera for frame
cam_idx = train_ds.get_camera_idx(500)  # Camera index for frame 500
```

--------------------------------

### Get Active Spherical Harmonics Degree

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Retrieve the currently active Spherical Harmonics (SH) degree, which may be less than the maximum during training.

```python
accessor.get_sh_degree()
```

--------------------------------

### Get Gaussian Model Capabilities

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Retrieve model capabilities such as SH degree and number of Gaussians. Useful for understanding model properties before export.

```python
caps = accessor.get_capabilities()
print(f"SH degree: {caps.sh_degree}")
print(f"Num Gaussians: {caps.num_gaussians}")
```

--------------------------------

### Initialize Interactive GUI

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/utilities.md

Initializes an interactive visualization GUI for a model, training, and validation datasets, and scene bounds. Requires `with_gui=true` in the configuration.

```python
def setup_gui(
    model: MixtureOfGaussians,
    train_dataset: BoundedMultiViewDataset,
    val_dataset: BoundedMultiViewDataset,
    scene_bbox: tuple[torch.Tensor, torch.Tensor]
) -> Any:
    """Initialize interactive visualization GUI."""
    pass
```

--------------------------------

### Get Single Frame (CPU)

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Retrieves a single frame's data, including rays, poses, and images, from the dataset. This operation is performed on the CPU.

```python
def __getitem__(self, index: int) -> dict
```

--------------------------------

### Get Frames Per Camera

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Returns a list containing the number of frames associated with each camera. This helps in understanding the distribution of data across cameras.

```python
def get_frames_per_camera(self) -> list[int]
```

```python
frames_per_cam = dataset.get_frames_per_camera()
num_cameras = len(frames_per_cam)
num_frames = sum(frames_per_cam)
```

--------------------------------

### Create Trainer3DGRUT from Checkpoint

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/trainer.md

Resume training by creating a Trainer3DGRUT instance from a saved checkpoint file. The configuration object's resume path will be automatically updated.

```python
trainer = Trainer3DGRUT.create_from_checkpoint(
    "runs/lego/ckpt_last.pt",
    conf
)
```

--------------------------------

### CUDA Synchronized Timer

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/utilities.md

Use CudaTimer as a context manager for accurate profiling of CUDA operations. It synchronizes with the CUDA device before starting and stopping the timer.

```python
class CudaTimer:
    def __enter__(self) -> 'CudaTimer'
    def __exit__(self, *args) -> None
    
    @property
    def elapsed_ms(self) -> float
```

```python
from threedgrut.utils.timer import CudaTimer

with CudaTimer() as timer:
    # Render a batch
    output = model.forward(batch)

print(f"Render time: {timer.elapsed_ms:.2f} ms")
```

--------------------------------

### Trainer.init_densification_and_pruning_strategy

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/trainer.md

Sets up the strategy for densification, pruning, and parameter updates during training. Supports standard GSStrategy and MCMCStrategy.

```APIDOC
## init_densification_and_pruning_strategy

### Description
Set up the strategy for densification, pruning, and parameter updates.

### Parameters
#### Path Parameters
- **conf** (`DictConfig`) - Required - Configuration with `strategy.method` setting

### Precondition
`self.model` must be initialized

### Supported Strategies
- `GSStrategy`: Standard 3D Gaussian Splatting densification/pruning
- `MCMCStrategy`: MCMC-based densification per arXiv:2404.09591

### Signature
```python
def init_densification_and_pruning_strategy(self, conf: DictConfig) -> None
```
```

--------------------------------

### Get Number of SH Features from Degree

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/utilities.md

Calculates the total number of spherical harmonics coefficients required for a given degree. Supports degrees from 0 to 3.

```python
def sh_degree_to_num_features(degree: int) -> int:
    """Get total number of SH coefficients for given degree."""
    # Implementation details omitted for brevity
```

--------------------------------

### Run the 3DGRT Playground

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut_playground/README.md

Launches the interactive 3DGRT Playground application. Requires a trained scene checkpoint.

```bash
python playground.py --gs_object runs/bonsai/ckpt_last.pt
```

--------------------------------

### Get Number of Frames in Split

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Returns the total number of frames available in the current data split. Use this to determine the size of the dataset or iterate through all frames.

```python
def __len__(self) -> int
```

--------------------------------

### BaseStrategy Get Strategy Parameters Output

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/types.md

The return value from BaseStrategy.get_strategy_parameters() includes gradient norms, accumulated gradients, normalization denominators, and densification control parameters.

```python
{
    'densify_grad_norm': torch.Tensor,      # Gradient norm buffer [N]
    'xyz_gradient_accum': torch.Tensor,     # Accumulated gradients [N, 3]
    'denom': torch.Tensor,                  # Denominator for normalization [N]
    'opacity_reset_interval': int,
    'densify_start_iter': int,
    'densify_stop_iter': int,
    # ... other strategy-specific state
}
```

--------------------------------

### Progressive Training Configuration

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/README.md

Configure progressive training by gradually increasing the Spherical Harmonics (SH) degrees. This helps avoid overfitting by starting with simpler representations.

```yaml
model:
  progressive_training:
    init_n_features: 0    # Start with constant color
    max_n_features: 3     # End with full SH degree 3
    increase_frequency: 500  # Increase every 500 steps
```

--------------------------------

### Load and Prepare COLMAP Dataset

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Demonstrates loading a COLMAP dataset, setting configuration options like path and downsampling, and creating training and validation datasets and dataloaders.

```python
import torch
from omegaconf import OmegaConf
import threedgrut.datasets as datasets

conf = OmegaConf.load("configs/apps/colmap_3dgut.yaml")
conf.path = "data/mipnerf360/bonsai"
conf.dataset.downsample_factor = 2

# Create datasets
train_ds, val_ds = datasets.make(
    conf.dataset.type,
    conf,
    ray_jitter=0.0  # No jitter for validation
)

print(f"Train frames: {len(train_ds)}")
print(f"Val frames: {len(val_ds)}")

# Create dataloaders
train_loader = torch.utils.data.DataLoader(
    train_ds,
    batch_size=1,
    shuffle=True,
    num_workers=4
)

# Training loop
for epoch in range(num_epochs):
    for batch_idx, batch in enumerate(train_loader):
        batch_gpu = train_ds.get_gpu_batch_with_intrinsics(batch)
        
        # batch_gpu.rays_ori: [1, H, W, 3]
        # batch_gpu.rays_dir: [1, H, W, 3]
        # batch_gpu.rgb_gt: [1, H, W, 3]
        # batch_gpu.T_to_world: [1, 4, 4]
        # batch_gpu.intrinsics: [fx, fy, cx, cy]
```

--------------------------------

### Run Interactive Playground

Source: https://github.com/nv-tlabs/3dgrut/blob/main/README.md

Use this command to launch the interactive playground UI for visualizing pretrained scenes with ray-tracing effects. Ensure you provide the path to your checkpoint object.

```bash
python playground.py --gs_object <ckpt_path>
```

--------------------------------

### Get Strategy Parameters

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/optimizers-and-strategies.md

Retrieves strategy-specific parameters necessary for saving the model's state during checkpointing. Returns a dictionary containing the strategy's current state.

```python
def get_strategy_parameters(self) -> dict:
    pass
```

--------------------------------

### Initialize Renderer

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/renderer.md

Instantiate the Renderer class with a trained model, configuration, training step, and output directory. Use this when you have a pre-initialized model object.

```python
renderer = Renderer(
    model=model,
    conf=conf,
    global_step=30000,
    out_dir="outputs/eval",
    compute_extra_metrics=True
)
```

--------------------------------

### Get Camera Index for Frame

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Maps a 0-based frame index within the training split to its corresponding camera index. Useful for associating frames with specific cameras.

```python
def get_camera_idx(self, frame_idx: int) -> int
```

--------------------------------

### MCMCStrategy Constructor

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/optimizers-and-strategies.md

Initializes MCMC samplers for Gaussian operations. This is the entry point for setting up the MCMC strategy with a given configuration and model.

```APIDOC
## __init__ MCMCStrategy

### Description
Initializes MCMC samplers for Gaussian operations.

### Parameters
* **config** (DictConfig) - Required - The configuration object for the MCMC strategy.
* **model** (MixtureOfGaussians) - Required - The 3D Gaussian Mixture model to operate on.
```

--------------------------------

### Get Scene Extent

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Calculates the scene extent, typically the maximum dimension of the bounding box. This provides a single scalar value representing the scene's size.

```python
def get_scene_extent(self) -> float
```

--------------------------------

### Create Trainer3DGRUT from PLY File

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/trainer.md

Initialize a Trainer3DGRUT instance using Gaussian data from a PLY file. The configuration will be updated to enable PLY import.

```python
trainer = Trainer3DGRUT.create_from_ply(
    "path/to/your/gaussian.ply",
    conf
)
```

--------------------------------

### Get Scene Bounding Box

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/datasets.md

Retrieves the minimum and maximum corner tensors defining the scene's bounding box. Use this to understand the spatial extent of the dataset.

```python
def get_scene_bbox(self) -> tuple[torch.Tensor, torch.Tensor]
```

```python
min_corner, max_corner = dataset.get_scene_bbox()
```

--------------------------------

### Set Up Interactive Rendering Controls

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut_playground/headless.ipynb

Creates ipywidgets (checkboxes, dropdown, slider) to control engine rendering parameters like antialiasing, denoiser, and samples per pixel (SPP). These widgets allow real-time adjustments.

```python
import ipywidgets as widgets
from IPython.display import display

aa_checkbox = widgets.Checkbox(
    value=engine.use_spp,
    description='Toggle Antialiasing'
)

aa_mode_combo = widgets.Dropdown(
    options=engine.ANTIALIASING_MODES,
    value='4x MSAA',
    description='AA Mode'
)

denoiser_checkbox = widgets.Checkbox(
    value=engine.use_optix_denoiser,
    description='Toggle Optix Denoiser'
)

spp_slider = widgets.IntSlider(
    value=engine.spp.spp,
    min=1,
    max=64,
    step=1,
    orientation='horizontal',
    description='AA SPP',
    disabled=(engine.spp.mode == 'msaa')
)

def on_change(change):
    engine.use_spp = aa_checkbox.value
    
    engine.antialiasing_mode = aa_mode_combo.value
    if engine.antialiasing_mode == '4x MSAA':
        engine.spp.mode = 'msaa'
        engine.spp.spp = 4
    elif engine.antialiasing_mode == '8x MSAA':
        engine.spp.mode = 'msaa'
        engine.spp.spp = 8
    elif engine.antialiasing_mode == '16x MSAA':
        engine.spp.mode = 'msaa'
        engine.spp.spp = 16
    elif engine.antialiasing_mode == 'Quasi-Random (Sobol)':
        engine.spp.mode = 'low_discrepancy_seq'
        engine.spp.spp = spp_slider.value
    else:
        raise ValueError('unknown antialiasing mode')
        
    engine.spp.reset_accumulation()
    engine.use_optix_denoiser = denoiser_checkbox.value

    spp_slider.value = engine.spp.spp
    spp_slider.disabled = (engine.spp.mode == 'msaa')
    visualizer.render_update() # Assuming 'visualizer' is defined elsewhere and has render_update method

aa_checkbox.observe(on_change, names='value')
aa_mode_combo.observe(on_change, names='value')
denoiser_checkbox.observe(on_change, names='value')
spp_slider.observe(on_change, names='value')
```

--------------------------------

### Initialize GaussianExportAccessor

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Instantiate GaussianExportAccessor with a model and optional configuration. This is the first step to accessing Gaussian data for export.

```python
from threedgrut.export.accessor import GaussianExportAccessor
from threedgrut.model.model import MixtureOfGaussians

model = MixtureOfGaussians(conf)
accessor = GaussianExportAccessor(model, conf)
```

--------------------------------

### Get Specular Dimension from SH Degree

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/utilities.md

Determines the feature dimension for specular components based on the spherical harmonics degree. The specular dimension is calculated as (degree+1)² - 1.

```python
def sh_degree_to_specular_dim(degree: int) -> int:
    """Get specular (higher-order) feature dimension."""
    # Implementation details omitted for brevity
```

--------------------------------

### Initialize MCMCStrategy

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/optimizers-and-strategies.md

Initializes the MCMC samplers for Gaussian operations. Requires a configuration object and a MixtureOfGaussians model.

```python
def __init__(self, config: DictConfig, model: MixtureOfGaussians) -> None:
```

--------------------------------

### Get Model Parameters for Checkpoint

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/model.md

Extracts all necessary parameters from the model for saving to a checkpoint. This includes geometric data, features, background state, training flags, and optimizer state.

```python
def get_model_parameters(self) -> dict:
    pass
```

```python
checkpoint = {
    'global_step': trainer.global_step,
    'model': model.get_model_parameters(),
    'config': conf
}
torch.save(checkpoint, 'ckpt_last.pt')
```

--------------------------------

### Export Model to PLY

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Loads a trained model, extracts its attributes, filters low-opacity Gaussians, and exports the result to a PLY file. Ensure you have PyTorch and the necessary 3DGRUT components installed.

```python
from threedgrut.model.model import MixtureOfGaussians
from threedgrut.export.accessor import GaussianExportAccessor
from threedgrut.export.formats.ply import PLYExporter

# Load trained model
checkpoint = torch.load("ckpt_last.pt", weights_only=False)
conf = checkpoint['config']
model = MixtureOfGaussians(conf)
model.init_from_checkpoint(checkpoint['model'], setup_optimizer=False)

# Extract attributes
accessor = GaussianExportAccessor(model, conf)
attrs = accessor.get_attributes(preactivation=False)

# Filter low-opacity Gaussians
from threedgrut.export.accessor import filter_gaussians, ExportFilterSettings
settings = ExportFilterSettings(
    filter_low_opacity=True,
    opacity_threshold=1e-6
)
filtered_attrs, stats = filter_gaussians(attrs, settings)

# Export
exporter = PLYExporter()
exporter.export("model.ply", filtered_attrs)
print(f"Exported {filtered_attrs.num_gaussians} Gaussians")
```

--------------------------------

### Load Renderer from Checkpoint

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/renderer.md

Create and initialize a Renderer instance directly from a saved training checkpoint. This method handles loading the model, configuration, and building necessary acceleration structures.

```python
renderer = Renderer.from_checkpoint(
    "runs/lego/ckpt_last.pt",
    out_dir="outputs/eval"
)

# Evaluate all test frames
metrics = renderer.render_all()
```

--------------------------------

### Get Visible Gaussian Mask

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Generates a boolean mask for Gaussians that meet a specified visibility threshold. Requires an array of visibility values and a threshold. Returns a boolean NumPy array.

```python
def get_visibility_mask(
    self,
    visibility: np.ndarray,
    threshold: float = 0.0
) -> np.ndarray:
    # Get mask of visible Gaussians.
    pass
```

--------------------------------

### USD Export Pipeline Overview

Source: https://github.com/nv-tlabs/3dgrut/blob/main/threedgrut/export/usd/post_processing/README.md

Illustrates the data flow from Gaussian SH coefficients through radiance scaling and post-ISP processing to the final LDR output.

```text
Gaussian SH coefficients
        |  multiplied by --scene-radiance-scale = k (asset-level, baked)
        v
   per-view render -> HDR buffer at k * (training radiance)
        |
        |
        v
   PPISP shader (spg-runtime only): exposure -> vignette -> ZCA color -> CRF
        v
   LDR
```

--------------------------------

### Initialize Renderer with Post-Processing

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/renderer.md

Load a trained model from a checkpoint, enabling automatic loading of post-processing modules like PPISP if they are present in the checkpoint. This is useful for applying enhancements to the rendered output.

```python
from threedgrut.render import Renderer
from ppisp import PPISP, PPISPConfig

# Load model with PPISP post-processing
renderer = Renderer.from_checkpoint(
    "runs/lego/ckpt_last.pt",
    out_dir="outputs/eval"
)

# PPISP is automatically loaded from checkpoint if present
metrics = renderer.render_all()
```

--------------------------------

### Get Low Opacity Gaussian Mask

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/export.md

Creates a boolean mask to identify Gaussians with opacity below a specified threshold. Useful for removing nearly transparent Gaussians. Defaults to a threshold of 1e-6.

```python
def get_low_opacity_mask(self, threshold: float = 1e-6) -> np.ndarray:
    # Get mask of low-opacity Gaussians.
    pass
```

--------------------------------

### Initialize Renderer from Checkpoint

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/api-reference/renderer.md

Loads a trained model from a checkpoint file and initializes the Renderer. This is the primary way to set up the renderer for evaluation or rendering.

```APIDOC
## Renderer.from_checkpoint

### Description
Initializes the Renderer from a given checkpoint path. It loads the trained model, configuration, and other necessary components.

### Method Signature
```python
Renderer.from_checkpoint(checkpoint_path: str, out_dir: str, path: Optional[str] = None, ...)
```

### Parameters
- **checkpoint_path** (str) - Required - Path to the checkpoint file.
- **out_dir** (str) - Required - Directory to save output files.
- **path** (Optional[str]) - Optional - Override the dataset path specified in the checkpoint configuration.

### Usage Example
```python
from threedgrut.render import Renderer

renderer = Renderer.from_checkpoint(
    checkpoint_path="runs/lego/ckpt_last.pt",
    out_dir="outputs/eval"
)
```
```

--------------------------------

### Get Activation Function by Name

Source: https://github.com/nv-tlabs/3dgrut/blob/main/_autodocs/utilities.md

Retrieves an activation function or its inverse by its string name. Supported functions include sigmoid, exp, softplus, and normalize. The inverse parameter defaults to False.

```python
from threedgrut.utils.misc import get_activation_function

sigmoid = get_activation_function("sigmoid")
densities = sigmoid(density_pre_activation)

sigmoid_inv = get_activation_function("sigmoid", inverse=True)
density_pre = sigmoid_inv(densities)
```