### Install Docker (Ubuntu 22.04)

Source: https://nerfbaselines.github.io/docs/installation.html

Install Docker and enable the Docker service on Ubuntu 22.04. This is a prerequisite for the 'docker' backend.

```bash
# Install docker
sudo apt-get update
sudo apt-get install -y docker.io
sudo systemctl enable --now docker
sudo usermod -aG docker $USER

```

--------------------------------

### install

Source: https://nerfbaselines.github.io/docs/cli.html

Installs a pre-defined or custom method from a spec file. It can also pre-install the backend implementation for a method.

```APIDOC
## install

Install either a pre-defined method or a custom method from a spec file. If installing from spec (–spec argument) - a python file with a register() call - the registered objects will be added to local registry. If installing a method (–method argument) the backend implementation for the method will be pre-installed. This also applies when specifying both –spec and –backend arguments.

### Usage
```
nerfbaselines install [OPTIONS]
```

### Options

- **--method** <method>
  Method to install.
  Options: nerfacto | mipnerf360 | wild-gaussians | 2d-gaussian-splatting | dropgaussian | taming-3dgs | 3dgrt | instant-ngp | kplanes | nerfw-reimpl | mip-splatting | seathru-nerf | colmap | scaffold-gs | pgsr | gaussian-splatting-wild | 3dgut | nerf | octree-gs | tensorf | student-splatting-scooping | gsplat | gaussian-opacity-fields | gaussian-splatting | 3dgs-mcmc | camp | tetra-nerf | sparsegs | nerfonthego | hierarchical-3dgs | zipnerf

- **--spec** <spec>
  Path to a spec file (a Python file with register() calls) to install.

- **--force**
  Overwrite existing specs. Only applies when –spec is provided.

- **--backend** <backend_name>
  The backend to use. If not specified, a supported installed backend is selected automatically. Note, the backend can be specified via the NERFBASELINES_BACKEND environment variable.
  Options: conda | docker | apptainer | python

### Environment variables

- **NERFBASELINES_BACKEND**
  Provide a default for `--backend`
```

--------------------------------

### Install NerfBaselines with Web Dependencies

Source: https://nerfbaselines.github.io/docs/custom-web.html

Install the necessary dependencies for local web development with NerfBaselines.

```bash
pip install 'nerfbaselines[web]'
```

--------------------------------

### LaTeX Table Output Example

Source: https://nerfbaselines.github.io/docs/exporting-results-tables.html

This is an example of a LaTeX table generated by the tool, showcasing results for various methods.

```latex
\providecommand{\pf}[1]{\textbf{#1}}
\providecommand{\ps}[1]{\underline{#1}}
\providecommand{\pt}[1]{#1}
\begin{tabular}{lrrrrr}\
\hline
Method                  &        PSNR &       SSIM & LPIPS (VGG) &         Time &    GPU mem. \\
Zip-NeRF                & \pf{28.553} & \pf{0.829} &  \pf{0.218} &   5h 30m 20s &     26.8 GB \\
Scaffold-GS             & \ps{27.714} &      0.813 &       0.262 &      23m 28s &      8.7 GB \\
Mip-NeRF 360            & \pt{27.681} &      0.792 &       0.272 &  30h 14m 36s &     33.6 GB \\
3DGS-MCMC               &      27.571 &      0.798 &       0.281 &       35m 8s &     21.6 GB \\
Mip-Splatting           &      27.492 & \pt{0.815} &       0.258 &      25m 37s &     11.0 GB \\
Gaussian Splatting      &      27.434 &      0.814 &       0.257 & \pt{23m 25s} &     11.1 GB \\
Gaussian Opacity Fields &      27.421 & \ps{0.826} &  \ps{0.234} &    1h 3m 54s &     28.4 GB \\
gsplet                  &      27.412 & \pt{0.815} &  \pt{0.256} &      29m 19s &      8.3 GB \\
2D Gaussian Splatting   &      26.815 &      0.796 &       0.297 &      31m 10s &     13.2 GB \\
NerfStudio              &      26.388 &      0.731 &       0.343 & \ps{19m 30s} & \ps{5.9 GB} \\
Instant NGP             &      25.507 &      0.684 &       0.398 &  \pf{3m 54s} & \pt{7.8 GB} \\
COLMAP                  &      16.670 &      0.445 &       0.590 &   2h 52m 55s &   \pf{0 MB} \\
\end{tabular}

```

--------------------------------

### Install Apptainer (Unprivileged, Ubuntu 22.04)

Source: https://nerfbaselines.github.io/docs/installation.html

Install Apptainer in an unprivileged mode on Ubuntu 22.04 using a provided script. Add the Apptainer binary directory to your PATH environment variable.

```bash
curl -s https://raw.githubusercontent.com/apptainer/apptainer/main/tools/install-unprivileged.sh | \
    bash -s - ~/.local

# Add the following line to your .bashrc, .bash_profile, or .zshrc
export PATH=$HOME/.local/bin:$PATH

```

--------------------------------

### Install NVIDIA Container Toolkit (Ubuntu 22.04)

Source: https://nerfbaselines.github.io/docs/installation.html

Install the NVIDIA container toolkit on Ubuntu 22.04 to enable GPU support within Docker containers. Configure Docker to use the NVIDIA runtime and restart the Docker service.

```bash
# Install NVIDIA container toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
      sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
          sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
# Enable NVIDIA container toolkit in docker
sudo nvidia-ctk runtime configure --runtime=docker
# Restart docker
sudo systemctl restart docker

```

--------------------------------

### Install NerfBaselines Package

Source: https://nerfbaselines.github.io/docs/installation.html

Install the main nerfbaselines package using pip. It is recommended to do this within a virtual environment.

```bash
pip install nerfbaselines

```

--------------------------------

### Serve the Web Demo Locally

Source: https://nerfbaselines.github.io/docs/exporting-demos.html

Command to start a local HTTP server to view the exported web demo. Navigate to http://localhost:8000 in your browser.

```bash
python -m http.server --directory web-demo 8000

```

--------------------------------

### Start NerfBaselines Viewer

Source: https://nerfbaselines.github.io/docs/viewer.html

Launches the viewer with specified checkpoint and data directories. The port can be customized using the --port argument.

```bash
nerfbaselines viewer --checkpoint <checkpoint> --data <data>
```

--------------------------------

### Install Miniforge Conda Distribution (Linux)

Source: https://nerfbaselines.github.io/docs/installation.html

Download and install the Miniforge Conda distribution on Linux systems. This is used for the 'conda' backend.

```bash
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh

```

--------------------------------

### nerfbaselines.web.start_dev_server

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.web.html

Starts a development server to preview the NeRF baselines website locally. This is useful for development and testing.

```APIDOC
## nerfbaselines.web.start_dev_server

### Description
Starts a development server to preview the NeRF baselines website locally.

### Parameters
- `_data` (str | None) - Optional - Path to the data directory.
- `_datasets` (Tuple[str, ...] | None) - Optional - A tuple of dataset names to include.
- `_include_docs` (Literal['all', 'docs', None]) - Optional - Controls inclusion of documentation ('all', 'docs', or None).
- `_include_demos` (bool) - Optional - Whether to include demos (defaults to False).
- `_port` (int) - Optional - The port to run the server on (defaults to 5500).
```

--------------------------------

### Run Local NerfBaselines Web Development Server

Source: https://nerfbaselines.github.io/docs/custom-web.html

Start a local development server to preview your custom web benchmark. Specify the data directory and optionally control documentation generation.

```bash
nerfbaselines web dev \
    --data /path/to/data \
    --docs none
```

--------------------------------

### Develop Web Benchmark Server CLI

Source: https://nerfbaselines.github.io/docs/cli.html

Start a development server for the web benchmark. This command automatically reloads the site upon source code changes. Configure data paths, datasets, port, and documentation inclusion.

```bash
nerfbaselines web dev [OPTIONS]

```

--------------------------------

### Prepare Rendering Method

Source: https://nerfbaselines.github.io/docs/viewer.html

A placeholder for preparing the rendering method, noting that loading an existing checkpoint requires installation and may take time.

```python
# First, we prepare the method to render. In this example, we load an existing
# checkpoint from the official nerfbaselines repository. However, this requires
# the method to be installed, so we skip it to save time. If you want to see the actual rendering, 
# Please run this cell (it can take long time to install the backend on the first run).

```

--------------------------------

### Start the NeRF Viewer

Source: https://nerfbaselines.github.io/docs/cli.html

Launch the viewer to visualize NeRF models. Provide a checkpoint path to load a trained model and a data path for dataset visualization. The viewer can be configured with specific ports and hosts.

```bash
nerfbaselines viewer [OPTIONS]
```

--------------------------------

### Train a Gaussian-Opacity-Fields Model

Source: https://nerfbaselines.github.io/docs/exporting-meshes.html

Use this command to train a `gaussian-opacity-fields` model on the specified dataset. Ensure the data path is correct for your setup.

```bash
nerfbaselines train \
    --method gaussian-opacity-fields \
    --data external://mipnerf360/bicycle

```

--------------------------------

### Install Nerfbaselines with Conda

Source: https://nerfbaselines.github.io/docs/viewer.html

Installs the nerfbaselines library using conda, specifically for the gsplat method. This is useful for environments like Google Colab.

```python
from nerfbaselines import load_checkpoint

if 'google.colab' in str(get_ipython()):
    # Install conda to google colab
    !pip install -q condacolab
    import condacolab
    condacolab.install()
    !nerfbaselines install --method gsplat --backend conda

# Load the checkpoint
checkpoint = "https://huggingface.co/nerfbaselines/nerfbaselines/resolve/main/gsplat/blender/lego.zip/checkpoint"
model, nb_info = stack.enter_context(load_checkpoint(checkpoint, backend="conda"))
```

--------------------------------

### Train Method and Close Stack

Source: https://nerfbaselines.github.io/docs/python-tutorial.html

This snippet demonstrates initiating the training process for the configured method and closing the context stack. It's the final step in the training pipeline setup.

```python
# Finally, we train the method
trainer.train()

# Close the stack. In real code, you should use the context manager
stack.close()
```

--------------------------------

### Get Supported Datasets, Loggers, and Methods

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

Functions to retrieve lists of all supported dataset loaders, datasets, evaluation protocols, loggers, and methods.

```APIDOC
## get_supported_dataset_loaders

### Description
Get all supported dataset loaders. The loaders are sorted by priority.

### Returns
FrozenSet[str] - List of dataset loader IDs (sorted by priority)
```

```APIDOC
## get_supported_datasets

### Description
Get all supported datasets.

### Returns
FrozenSet[str] - Set of dataset IDs
```

```APIDOC
## get_supported_evaluation_protocols

### Description
Get all supported evaluation protocols.

### Returns
FrozenSet[str] - Set of evaluation protocol IDs
```

```APIDOC
## get_supported_loggers

### Description
Get all supported loggers.

### Returns
FrozenSet[str] - Set of logger IDs
```

```APIDOC
## get_supported_methods

### Description
Get all supported methods. Optionally, filter the methods that support a specific backend.

### Parameters
* **backend_name** (Literal['conda', 'docker', 'apptainer', 'python'] | None) - Backend name

### Returns
FrozenSet[str] - Set of method IDs
```

--------------------------------

### Viewer Command

Source: https://nerfbaselines.github.io/docs/cli.html

Starts the viewer to visualize NeRF models. Can load models from checkpoints and datasets. Provides options to specify port and host.

```APIDOC
## viewer

Start the viewer. If a checkpoint is provided, the viewer will load the model. If a data path is provided, the viewer will load the dataset. Note that even then the checkpoint is provided, providing the –data argument is still recommended as it will improve viewing experience.

```
nerfbaselines viewer [OPTIONS]
```

### Options

--checkpoint <checkpoint>
    
Path to the checkpoint directory. It can also be a remote path (starting with http(s)://) or be a path inside a zip file. 

--data <data>
    
A path to the dataset to load in the viewer. The dataset can be either an external dataset (e.g., a path starting with external://{dataset}/{scene}) or a local path to a dataset. If the dataset is an external dataset, the dataset will be downloaded and cached locally. If the dataset is a local path, the dataset will be loaded directly from the specified path. 

--port <port>
    
Port to run the viewer on. If port=0, a random free port will be assigned. Defaults to 6006. 

--host <host>
    
Host to run the viewer on. Defaults to localhost. 

--backend <backend_name>
    
The backend to use. If not specified, a supported installed backend is selected automatically. Note, the backend can be specified via the NERFBASELINES_BACKEND environment variable. 

Options:
    
conda | docker | apptainer | python

Environment variables 

NERFBASELINES_BACKEND
    
> Provide a default for `--backend`
```

--------------------------------

### Export a Web Demo for a Trained Model

Source: https://nerfbaselines.github.io/docs/exporting-demos.html

Command to export a web demo from a trained model checkpoint. The `--data` argument is recommended for proper demo setup. The output will be stored in the specified directory.

```bash
nerfbaselines export-demo --method gaussian-splatting --data external://mipnerf360/bicycle --checkpoint checkpoint-30000 --output web-demo

```

--------------------------------

### Backend Base Class and Methods

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.backends.html

The `Backend` class provides a base interface for different backend implementations. It defines methods for installation, instance management, and shell access.

```APIDOC
## Class Backend

Bases: `object`

### Methods

- **install()**
  Installs the backend.

- **instance_call(_instance: int, _method: str, *_args, **kwargs)**
  Calls a method on a specific backend instance.

- **instance_del(_instance: int)**
  Deletes a specific backend instance.

- **shell(_args: Tuple[str, ...] | None = None)**
  Opens a shell within the backend environment.

- **static_call(_function: str, *_args, **kwargs)**
  Calls a static function within the backend environment.

### Class Attributes

- **name** (`str`): The name of the backend (default: 'unknown').
```

--------------------------------

### Docker Backend

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.backends.html

The `DockerBackend` class facilitates the use of Docker containers as backends. It inherits from `RemoteProcessRPCBackend` and includes methods for installation and shell access.

```APIDOC
## Class DockerBackend

Bases: `RemoteProcessRPCBackend`

### Methods

- **install()**
  Installs the Docker backend.

- **shell(_args=None)**
  Opens a shell within the Docker environment.

### Class Attributes

- **name** (`str`): The name of the backend, 'docker'.
```

--------------------------------

### Apptainer Backend

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.backends.html

The `ApptainerBackend` class provides an interface for using Apptainer containers as a backend. It inherits from `RemoteProcessRPCBackend` and includes specific installation and shell methods.

```APIDOC
## Class ApptainerBackend

Bases: `RemoteProcessRPCBackend`

### Methods

- **install()**
  Installs the Apptainer backend.

- **shell(_args=None)**
  Opens a shell within the Apptainer environment.

### Class Attributes

- **name** (`str`): The name of the backend, 'apptainer'.
```

--------------------------------

### Train a Gaussian Splatting model

Source: https://nerfbaselines.github.io/docs/quickstart.html

Use this command to start training a Gaussian Splatting model on the Mip-NeRF 360 dataset. The training script automatically evaluates on the test set upon completion.

```bash
nerfbaselines train --method gaussian-splatting --data external://mipnerf360/garden
```

--------------------------------

### Dataset Metadata Example

Source: https://nerfbaselines.github.io/docs/adding-new-datasets.html

This JSON object represents the metadata for the Mip-NeRF 360 dataset. It includes fields like id, name, description, paper details, link, metrics, default metric, and scenes. Ensure all required fields are present for the dataset to be listed.

```json
{
    "id": "mipnerf360",
    "name": "Mip-NeRF 360",
    "description": "Mip-NeRF 360 is a collection of four indoor and five outdoor object-centric scenes. The camera trajectory is an orbit around the object with fixed elevation and radius. The test set takes each n-th frame of the trajectory as test views.",
    "paper_title": "Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields",
    "paper_authors": ["Jonathan T. Barron", "Ben Mildenhall", "Dor Verbin", "Pratul P. Srinivasan", "Peter Hedman"],
    "paper_link": "https://arxiv.org/pdf/2111.12077.pdf",
    "link": "https://jonbarron.info/mipnerf360/",
    "metrics": ["psnr", "ssim", "lpips_vgg"],
    "default_metric": "psnr",
    "scenes": [
        { "id": "garden", "name": "garden" },
        { "id": "bicycle", "name": "bicycle" }
    ]
}
```

--------------------------------

### Build Web Benchmark CLI

Source: https://nerfbaselines.github.io/docs/cli.html

Use this command to build the static web benchmark website. Specify the output directory and optionally configure data paths, datasets, base paths, and documentation inclusion.

```bash
nerfbaselines web build [OPTIONS]

```

--------------------------------

### Build Static NerfBaselines Web Benchmark

Source: https://nerfbaselines.github.io/docs/custom-web.html

Build the static files for your web benchmark. Specify input data, output directory, and optionally a base URL for deployment.

```bash
nerfbaselines web build \
    --data /path/to/data \
    --output /path/to/output \
    --docs latest
```

--------------------------------

### Initialize Viewer in Notebook

Source: https://nerfbaselines.github.io/docs/viewer.html

Demonstrates how to initialize the Nerfbaselines viewer within a Jupyter Notebook or Google Colab environment using ExitStack for context management.

```python
from contextlib import ExitStack
from nerfbaselines.datasets import load_dataset

# We will use ExitStack to keep context open across multiple cells
stack = ExitStack()

# Model can be None (in that case we will only visualize dataset)
model = None

# Create the train and test datasets. Note, that we do not need 
# to load the features, as the images will be automatically loaded
# and cached when they are required by the viewer in the background
# process. This speeds up the viewer load time.
train_dataset = load_dataset("external://blender/lego", split="train", load_features=False)
test_dataset = load_dataset("external://blender/lego", split="train", load_features=False)
```

--------------------------------

### warp_image_between_cameras

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.cameras.html

Warps an image from one camera setup to another by mapping pixels.

```APIDOC
## warp_image_between_cameras

### Description
Warp an image from cameras1 to cameras2 by mapping pixels from cameras2 to cameras1.

### Parameters
* **cameras1** (GenericCameras[ndarray]_) - Cameras
* **cameras2** (GenericCameras[ndarray]_) - Cameras
* **images** (ndarray_) - Images tensor

### Returns
Warped images
```

--------------------------------

### Create Camera Object and Render Image

Source: https://nerfbaselines.github.io/docs/python-tutorial.html

Demonstrates how to create a camera object with specified poses, intrinsics, and image sizes, and then use it to render an image with the model. The rendered image is then displayed.

```python
camera = new_cameras(
    poses=pose,
    intrinsics=np.array([fx, fy, cx, cy], dtype=np.float32),
    image_sizes=np.array([w, h], dtype=np.int32),
    camera_models=np.array(camera_model_to_int("pinhole"), dtype=np.int32),
)

# Render the image
outputs = model.render(camera=camera, options={"output_type_dtypes": {"color": "uint8"}})

# Display the rendered image
display(Image.fromarray(outputs["color"]))

```

--------------------------------

### Get Method Specification

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

Retrieves the specification for a method using its registered ID.

```APIDOC
## get_method_spec

### Description
Get a method by method ID.

### Parameters
* **id** (str) - Method ID

### Returns
MethodSpec - Method spec
```

--------------------------------

### Get Logger Specification

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

Retrieves the specification for a logger using its registered ID.

```APIDOC
## get_logger_spec

### Description
Get a logger specification by registered logger ID.

### Parameters
* **id** (str) - Logger ID

### Returns
LoggerSpec - Logger specification
```

--------------------------------

### Get Dataset Specification

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

Retrieves the specification for a dataset using its registered ID.

```APIDOC
## get_dataset_spec

### Description
Get a dataset specification by registered dataset ID.

### Parameters
* **id** (str) - Dataset ID

### Returns
DatasetSpec - Dataset specification
```

--------------------------------

### Get Evaluation Protocol Specification

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

Retrieves the specification for an evaluation protocol using its registered ID.

```APIDOC
## get_evaluation_protocol_spec

### Description
Get an evaluation protocol specification by registered evaluation protocol ID.

### Parameters
* **id** (str) - Evaluation protocol ID

### Returns
EvaluationProtocolSpec - Evaluation protocol specification
```

--------------------------------

### Get Dataset Loader Specification

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

Retrieves the specification for a dataset loader using its registered ID.

```APIDOC
## get_dataset_loader_spec

### Description
Get a dataset loader specification by registered dataset loader ID.

### Parameters
* **id** (str) - Dataset loader ID

### Returns
DatasetLoaderSpec - Dataset loader specification
```

--------------------------------

### Build Method Class and Initialize Trainer

Source: https://nerfbaselines.github.io/docs/python-tutorial.html

This snippet shows how to build a method class and initialize a trainer with specified datasets, output paths, and saving/evaluation intervals. It configures logging and output artifacts, and overrides the total number of iterations for faster training.

```python
method_cls = stack.enter_context(build_method_class(method_spec, backend))

model = method_cls(
    train_dataset=train_dataset,
    config_overrides=config_overrides,
)

trainer = Trainer(
    train_dataset=train_dataset,
    test_dataset=test_dataset,
    method=model,
    output=output_path,
    save_iters=Indices.every_iters(10_000, zero=True),  # Save the model every 10k iterations
    eval_few_iters=Indices.every_iters(2_000),  # Eval on few images every 2k iterations
    eval_all_iters=Indices([-1]),  # Eval on all images at the end
    logger=build_logger(("tensorboard",)),  # We will log to tensorboard
    generate_output_artifact=True,
    config_overrides=config_overrides,
    applied_presets=frozenset(presets),
)

# In this example we override the total number of iterations
# to make the training faster
trainer.num_iterations = 100
```

--------------------------------

### Dataset Loading Functions

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

Functions to get specifications and load datasets, including supported dataset loaders and specifications.

```APIDOC
## Dataset Loading API

### Description
Provides functions to retrieve information about supported datasets and dataset loading specifications, and to load datasets.

### Functions

- `get_dataset_loader_spec()`: Retrieves the specification for a dataset loader.
- `get_dataset_spec()`: Retrieves the specification for a dataset.
- `get_supported_dataset_loaders()`: Returns a list of supported dataset loaders.
- `get_supported_datasets()`: Returns a list of supported datasets.
- `new_dataset(spec)`: Creates a new dataset instance based on the provided specification.

```

--------------------------------

### Method and Logger Specification Functions

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

Functions to get specifications for NeRF methods and loggers, and to retrieve supported methods and loggers.

```APIDOC
## Method and Logger API

### Description
Provides functions to get specifications for NeRF methods and loggers, and to list supported methods and loggers.

### Functions

- `get_method_spec()`: Retrieves the specification for a NeRF method.
- `get_logger_spec()`: Retrieves the specification for a logger.
- `get_supported_loggers()`: Returns a list of supported loggers.
- `get_supported_methods()`: Returns a list of supported NeRF methods.
- `build_method_class(spec)`: Builds a method class based on the provided specification.

```

--------------------------------

### Implement Training Step for a New Method

Source: https://nerfbaselines.github.io/docs/adding-new-methods.html

This method performs a single iteration of the training process, including sampling data, computing loss, and optimizing parameters. Ensure all necessary datasets and optimizers are initialized.

```python
def step(self, step):
        # Perform a single iteration of the training
        self.step = step

        # Sample a random image
        rand_idx = torch.randint(len(self.train_dataset["images"]), (1,)).cpu().item()
        image = torch.from_numpy(convert_image_dtype(self.train_dataset["images"][rand_idx][:, :, :3], 'float32'))

        # Compute the loss
        w, h = self.train_dataset["cameras"][rand_idx].image_sizes
        pred = self.color[None, None, :].expand(h, w, 3)
        loss = torch.nn.functional.mse_loss(pred, image)

        # Optimize the loss
        self.optimizer.zero_grad()
        loss.backward()
        self.optimizer.step()

        # Return the stats
        return {
            "loss": loss.item(),
        }
```

--------------------------------

### Define Method Capabilities and Information

Source: https://nerfbaselines.github.io/docs/adding-new-methods.html

Implement get_method_info to declare supported camera models and required features, and get_info to provide runtime details like hyperparameters and iteration count.

```python
    def get_method_info(cls):
        return {
            # Method ID is provided by the registry
            "method_id": "",  

            # Supported camera models (e.g., pinhole, opencv, ...)
            "supported_camera_models": frozenset(("pinhole",)),

            # Features required for training (e.g., color, points3D_xyz, ...)
            "required_features": frozenset(("color",)),

            # Declare supported outputs
            "supported_outputs": ("color",),
        }

    def get_info(self):
        return {
            **self.get_method_info(),
            "hparams": self.hparams,
            "loaded_checkpoint": self.checkpoint,
            "loaded_step": self._loaded_step,

            # This number specifies how many iterations 
            # the method should be trained for.
            "num_iterations": 100,
        }

```

--------------------------------

### Get Specific Color Palette

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.utils.html

Retrieve a color palette by its name. The returned palette is a numpy array suitable for image processing.

```python
get_palette(palette='viridis')
```

--------------------------------

### Initialize and Run Nerfbaselines Viewer

Source: https://nerfbaselines.github.io/docs/viewer.html

Initializes the Nerfbaselines viewer with training and testing datasets, and a loaded model. It then displays the viewer in a notebook and runs it to handle rendering requests.

```python
from nerfbaselines.viewer import Viewer

# Initialize the viewer
viewer = stack.enter_context(Viewer(
    train_dataset=train_dataset, 
    test_dataset=test_dataset, 
    model=model))

# The `show_in_notebook` function opens the iframe inside jupyter/colab.
# After running the function, the viewer will already be visible,
# But it will not be able to render the images yet. In order,
# to handle the rendering requests, we need to call `viewer.run()`
viewer.show_in_notebook()

# Finally, we run `viewer.run()` which waits in loop for new render requests 
# and handles them. We recommend to call the function in a separate cell
# as it can be stopped without stopping the viewer itself. This is useful
viewer.run()
```

--------------------------------

### Display Help for generate-dataset-results

Source: https://nerfbaselines.github.io/docs/exporting-results-tables.html

Use this command to view all available options for generating dataset results tables.

```bash
!nerfbaselines generate-dataset-results --help
```

--------------------------------

### Extract Features with COLMAP

Source: https://nerfbaselines.github.io/docs/using-custom-data.html

Use COLMAP's feature_extractor to process images and prepare them for further steps. Ensure COLMAP is installed and accessible in your environment.

```bash
colmap feature_extractor \
  --database_path database.db \
  --image_path images
```

--------------------------------

### SimpleBackend Methods

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.backends.html

Methods for interacting with a SimpleBackend instance.

```APIDOC
## SimpleBackend Methods

### Description
Provides methods for calling and managing SimpleBackend instances.

### Methods
- `instance_call()`: Calls a method on a SimpleBackend instance.
- `instance_del()`: Deletes a SimpleBackend instance.
- `static_call()`: Calls a static method on the SimpleBackend class.
```

--------------------------------

### Conda Backend

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.backends.html

The `CondaBackend` class is used for managing Conda environments as backends. It inherits from `RemoteProcessRPCBackend` and provides methods for installation and shell access.

```APIDOC
## Class CondaBackend

Bases: `RemoteProcessRPCBackend`

### Methods

- **install()**
  Installs the Conda backend.

- **shell(_args=None)**
  Opens a shell within the Conda environment.

### Class Attributes

- **name** (`str`): The name of the backend, 'conda'.
```

--------------------------------

### Initialize Method with PyTorch Parameters and Checkpoint Loading

Source: https://nerfbaselines.github.io/docs/adding-new-methods.html

Implement the __init__ method for training or inference, handling checkpoint loading, hyperparameters, and initializing PyTorch model components.

```python
import json, os
from nerfbaselines import Method
from nerfbaselines.utils import convert_image_dtype
import torch.nn
import torch.optim
import torch.nn.functional

class MyMethod(Method):
    def __init__(self, *, 
                 checkpoint=None,
                 train_dataset=None,
                 config_overrides=None):
        super().__init__()

        # If train_dataset is not None,
        # initialize the method for training
        self.train_dataset = train_dataset
        self.hparams = {
            "initial_color": [1.0, 0.0, 0.0],
        }
        self._loaded_step = None
        self.step = 0
        self.checkpoint = checkpoint
        if config_overrides is not None:
            self.hparams.update(config_overrides)

        # In this example, we just optimize single color to be rendered
        self.color = torch.nn.Parameter(torch.tensor(self.hparams["initial_color"], dtype=torch.float32))
        self.optimizer = torch.optim.Adam([self.color], lr=1e-3)

        if checkpoint is not None:
            # Load the model from the checkpoint
            with open(os.path.join(checkpoint, "params.json"), "r") as f:
                ckpt_meta = json.load(f)
            self.hparams.update(ckpt_meta["hparams"])
            self._loaded_step = self.step = ckpt_meta["step"]

            # We load the ckpt here
            _state, optim_state = torch.load(os.path.join(checkpoint, "model.pth"))
            self.color.data.copy_(_state)
            self.optimizer.load_state_dict(optim_state)
        else:
            assert train_dataset is not None, "train_dataset must be provided for training"

```

--------------------------------

### Training and Evaluation Output

Source: https://nerfbaselines.github.io/docs/python-tutorial.html

This output shows the logs from the training process, including connection information, dataset loading progress, initialization details, training iteration progress with PSNR, and rendering progress for evaluation. It also includes a detailed JSON object with training hyperparameters, dataset metadata, and resource utilization.

```text
info: Connection accepted, protocol: shm-pickle5-buffers
Loading Training Cameras [22/09 22:15:11]
Loading Test Cameras [22/09 22:15:18]
Number of points at initialisation :  54275 [22/09 22:15:18]
training: 100%|██████████| 100/100 [00:11<00:00, 11.91it/s, train/psnr=14.4304]Rendering all images at step=100: 100%|██████████| 25/25 [00:19<00:00,  1.30it/s]
training: 100%|██████████| 100/100 [00:31<00:00, 11.91it/s, train/psnr=14.4304]{'applied_presets': [],
 'checkpoint_sha256': 'acf027e845ff29e9db9558bd0a1166c87d5a77ca07ddb46f609c68661cb228cc',
 'config_overrides': {},
 'dataset_metadata': {'color_space': 'srgb',
                      'downscale_factor': 4,
                      'evaluation_protocol': 'nerf',
                      'expected_scene_scale': 4.299345,
                      'id': 'mipnerf360',
                      'scene': 'bicycle',
                      'type': 'object-centric',
                      'viewer_initial_pose': array([[-0.948501,  0.055669,  0.311843, -0.329421],
       [-0.316162, -0.227448, -0.921037,  0.961329],
       [ 0.019655, -0.972198,  0.233335, -0.144017]], dtype=float32),
                      'viewer_transform': array([[-0.075102, -0.051249,  0.210046, -0.007215],
       [-0.216097,  0.024916, -0.071186,  0.102799],
       [-0.006926, -0.221673, -0.056562,  0.084124]], dtype=float32)},
 'datetime': '2024-09-22T20:15:59',
 'evaluation_protocol': 'nerf',
 'hparams': {'compute_cov3D_python': False,
             'convert_SHs_python': False,
             'debug': False,
             'densification_interval': 100,
             'densify_from_iter': 500,
             'densify_grad_threshold': 0.0002,
             'densify_until_iter': 15000,
             'feature_lr': 0.0025,
             'iterations': 30000,
             'lambda_dssim': 0.2,
             'opacity_lr': 0.05,
             'opacity_reset_interval': 3000,
             'percent_dense': 0.01,
             'position_lr_delay_mult': 0.01,
             'position_lr_final': 1.6e-06,
             'position_lr_init': 0.00016,
             'position_lr_max_steps': 30000,
             'random_background': False,
             'rotation_lr': 0.001,
             'scale_coords': None,
             'scaling_lr': 0.005,
             'sh_degree': 3,
             'white_background': False},
 'method': 'gaussian-splatting',
 'nb_version': '1.2.3.dev6+geacf155.d20240917',
 'num_iterations': 30000,
 'render_dataset_metadata': {'color_space': 'srgb',
                             'downscale_factor': 4,
                             'evaluation_protocol': 'nerf',
                             'id': 'mipnerf360',
                             'scene': 'bicycle',
                             'type': 'object-centric'},
 'render_datetime': '2024-09-22T20:15:59',
 'render_version': '1.2.3.dev6+geacf155.d20240917',
 'resources_utilization': {'gpu_memory': 3324,
                           'gpu_name': 'NVIDIA A100-SXM4-40GB',
                           'memory': 3774},
 'total_train_time': 8.83263}
evaluating all images at step=100: 100%|██████████| 25/25 [00:19<00:00,  1.26it/s, psnr=18.5846]
hashing predictions: 100%|██████████| 25/25 [00:00<00:00, 236.86it/s]
training: 100%|██████████| 100/100 [00:53<00:00,  1.87it/s, train/psnr=14.4304]
info: Backend worker finished

```

--------------------------------

### Get Supported Palette Names

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.utils.html

Retrieve a list of all supported colormap names available in the utility functions. This can be used to select a palette for `apply_colormap`.

```python
get_supported_palette_names()
```

--------------------------------

### Launch the interactive viewer

Source: https://nerfbaselines.github.io/docs/quickstart.html

Launch the interactive viewer to inspect your trained Gaussian Splatting model. The viewer allows for 3D scene inspection, camera trajectory construction, and video rendering.

```bash
nerfbaselines viewer --checkpoint checkpoint-30000 --data external://mipnerf360/garden
```

--------------------------------

### generate-dataset-results Command Options

Source: https://nerfbaselines.github.io/docs/exporting-results-tables.html

This displays the available options for the `generate-dataset-results` command, including input and output configurations.

```text
Usage: nerfbaselines generate-dataset-results [OPTIONS]

Options:
  --results PATH
  --dataset TEXT
  --output-type [markdown|json|latex]
  --method-links [paper|website|results|none]
  --output FILE
  --scenes TEXT                   Comma-separated list of scenes to include in
                                  the results.
  -v, --verbose                   Enable verbose logging.
  --help                          Show this message and exit.
```

--------------------------------

### Implement Rendering Method for a New Method

Source: https://nerfbaselines.github.io/docs/adding-new-methods.html

This method renders an image using the provided camera parameters. It currently renders a single color image and yields the result. Ensure the camera object has `image_sizes` attribute.

```python
@torch.no_grad()
def render(self, camera, *, options=None):
        # Render the images
        w, h = camera.image_sizes

        # Here we simply render a single color image
        yield {
            "color": self.color[None, None, :].expand(h, w, 3).detach().cpu().numpy(),
        }
```

--------------------------------

### Method Class

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.html

The Method class defines the interface for NeRF methods, including abstract methods for training, rendering, and saving, as well as concrete methods for getting information and embeddings.

```APIDOC
## Class Method

### Description
Represents the interface for NeRF methods, providing abstract methods for core functionalities and concrete methods for information retrieval.

### Methods
- **`get_info()`**: Abstract method to get method defaults for the trainer.
- **`_get_method_info()`**: Abstract class method to get method info needed to initialize datasets.
- **`get_train_embedding(index: int)`**: Get the embedding for the given image index.
- **`optimize_embedding(dataset: Dataset, *, embedding: ndarray | None = None)`**: Optimize embedding for a single image.
- **`_render(camera: Cameras, *, options: RenderOptions | None = None)`**: Abstract method to render a single image.
- **`_save(path: str)`**: Abstract method to save the model.
- **`_train_iteration(step: int)`**: Abstract method to train one iteration.
```

--------------------------------

### Initialize Trainer Class for NeRF Training

Source: https://nerfbaselines.github.io/docs/python-tutorial.html

Sets up the necessary components for training a NeRF model using the `Trainer` class. This includes preparing the output directory and managing contexts.

```python
from contextlib import ExitStack
from nerfbaselines import backends
from nerfbaselines import (
    build_method_class,
)
from nerfbaselines.training import (
    Trainer, 
    Indices, 
    build_logger,
)

# We use the exit stack to simplify the context management
stack = ExitStack().__enter__()

# Prepare the output directory, and mount it if necessary (e.g., for docker backend)
stack.enter_context(backends.mount(output_path, output_path))

```

--------------------------------

### Load Checkpoint and Model Info - Python

Source: https://nerfbaselines.github.io/docs/python-tutorial.html

Loads a NerfBaselines checkpoint using a utility function and prints its information. Supports local paths or URLs, and can automatically handle zip files.

```python
from contextlib import ExitStack

stack = ExitStack().__enter__()

import pprint
from nerfbaselines import load_checkpoint

checkpoint_path = "https://huggingface.co/nerfbaselines/nerfbaselines/resolve/main/gaussian-splatting/mipnerf360/garden.zip/checkpoint"

model, _ = stack.enter_context(load_checkpoint(checkpoint_path, backend="conda"))

pprint.pprint(model.get_info())

```

--------------------------------

### Load Custom Dataset

Source: https://nerfbaselines.github.io/docs/custom-dataset-loader.html

Constructs a NerfBaselines dataset from image paths and camera parameters, then applies a train/test split. Requires the `_load_cameras` helper function.

```python
import os
def load_my_dataset(path, split, **kwargs):
    image_names, cameras = _load_cameras(os.path.join(path, "cameras.csv"))
    image_paths = [os.path.join(path, "images", name) for name in image_names]
    dataset = new_dataset(
        image_paths=image_paths,
        cameras=cameras,
        metadata={"id": "my-dataset", 
                  "type": "object-centric",
                  "scene": "my-scene"})

    # Now, we need to do train/test split
    test_indices = np.linspace(0, len(image_paths), 10, endpoint=False, dtype=np.int32)
    indices = test_indices if split == "test" else np.setdiff1d(np.arange(len(image_paths)), test_indices)
    return dataset_index_select(dataset, indices)

```

--------------------------------

### build_static_viewer

Source: https://nerfbaselines.github.io/docs/api/nerfbaselines.viewer.html

Builds a static version of the viewer in the specified output path.

```APIDOC
## build_static_viewer(output, params=None)

### Description
Builds the viewer in the given path.

### Parameters
- **output**: The path to write the viewer to.
- **params**: Optional parameters for building the viewer.
```

--------------------------------

### Generate Markdown Table for All Methods

Source: https://nerfbaselines.github.io/docs/exporting-results-tables.html

Use this command to generate a markdown table of results for all methods on a specified dataset. If you want to render your own data, use `--data {path to your data argument}`.

```bash
!nerfbaselines generate-dataset-results --output-type markdown --dataset mipnerf360
```

--------------------------------

### Train NeRF Method with Custom Training Loop

Source: https://nerfbaselines.github.io/docs/python-tutorial.html

Shows how to train a NeRF method using a custom training loop. It involves setting up the method, dataset, and then iterating through training steps with progress tracking. Ensure the output directory is prepared and mounted if necessary.

```python
from tqdm import tqdm
from contextlib import ExitStack
from nerfbaselines import backends
from nerfbaselines.datasets import load_dataset
from nerfbaselines import (
    build_method_class,
    get_method_spec,
)
from nerfbaselines.training import (
    get_presets_and_config_overrides,
)

# We will use the "gaussian-splatting" method and "conda" backend
method_name = "gaussian-splatting"
backend = "conda"

# We will write the output to directory "output"
output_path = "output"

# We will use the data from the "mipnerf360/bicycle" scene
data = "external://mipnerf360/bicycle"

# We use the exit stack to simplify the context management
stack = ExitStack().__enter__()

# Prepare the output directory, and mount it if necessary (e.g., for docker backend)
stack.enter_context(backends.mount(output_path, output_path))

# Get the method specification for a registered method
method_spec = get_method_spec(method_name)

# Build the method class and start the backend
method_cls = stack.enter_context(build_method_class(method_spec, backend))

# Load train dataset
# We use the method info to load the required features and supported camera models
method_info = method_cls.get_method_info()
train_dataset = load_dataset(data, 
                             split="train", 
                             features=method_info.get("required_features"),
                             supported_camera_models=method_info.get("supported_camera_models"),
                             load_features=True)
# Load eval dataset
test_dataset = load_dataset(data, 
                            split="test", 
                            features=method_info.get("required_features"),
                            supported_camera_models=method_info.get("supported_camera_models"),
                            load_features=True)

# Each method can specify custom presets and config overrides
# Apply config overrides for the train dataset
presets, config_overrides = get_presets_and_config_overrides(
    method_spec, train_dataset["metadata"])

# Build the method
model = method_cls(
    train_dataset=train_dataset,
    config_overrides=config_overrides,
)

# Training loop
model_info = model.get_info()

# In this example we override the total number of iterations
# to make the training faster
model_info["num_iterations"] = 100

with tqdm(total=model_info["num_iterations"]) as pbar:
    for step in range(model_info["num_iterations"]):
        metrics = model.train_iteration(step)
        pbar.set_postfix({"psnr": f"{metrics['psnr']:.2f}"})
        pbar.update()

# Save the model
model.save("checkpoint")
# Create a minimal nb-info.json file such that the model can be loaded
with open("nb-info.json", "w") as f:
    f.write(f'{{"method": "{method_name}"}}')

# Close the stack. In real code, you should use the context manager
stack.close()

```