### Install NerfAcc from Source

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

Install NerfAcc directly from its GitHub repository. CUDA code is built during installation.

```bash
pip install git+https://github.com/nerfstudio-project/nerfacc.git
```

--------------------------------

### Install NerfAcc via pip

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/docs/source/index.md

Standard installation methods for the NerfAcc library.

```console
$ pip install nerfacc
```

```console
$ pip install git+https://github.com/nerfstudio-project/nerfacc.git
```

```console
// e.g. torch 1.13.0 + CUDA 11.7
$ pip install nerfacc -f https://nerfacc-bucket.s3.us-west-2.amazonaws.com/whl/torch-1.13.0_cu117.html
```

--------------------------------

### Implement training loop with OccGridEstimator

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

A complete training loop example integrating a radiance field model with NerfAcc's occupancy grid estimator and volumetric rendering.

```python
import torch
import torch.nn.functional as F
import nerfacc

# Setup device and model
device = "cuda"
aabb = torch.tensor([-1.5, -1.5, -1.5, 1.5, 1.5, 1.5], device=device)

# Create occupancy grid estimator
estimator = nerfacc.OccGridEstimator(
    roi_aabb=aabb,
    resolution=128,
    levels=1,
).to(device)

# Simple MLP radiance field
class RadianceField(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.density_net = torch.nn.Sequential(
            torch.nn.Linear(3, 128), torch.nn.ReLU(),
            torch.nn.Linear(128, 128), torch.nn.ReLU(),
            torch.nn.Linear(128, 1),
        )
        self.color_net = torch.nn.Sequential(
            torch.nn.Linear(3 + 3, 128), torch.nn.ReLU(),
            torch.nn.Linear(128, 3), torch.nn.Sigmoid(),
        )

    def query_density(self, x):
        return F.relu(self.density_net(x).squeeze(-1))

    def forward(self, x, d):
        density = self.query_density(x)
        color = self.color_net(torch.cat([x, d], dim=-1))
        return color, density

radiance_field = RadianceField().to(device)
optimizer = torch.optim.Adam(radiance_field.parameters(), lr=1e-3)

# Training loop
render_step_size = 5e-3
for step in range(10000):
    # Generate random rays (replace with real camera rays)
    n_rays = 1024
    rays_o = torch.randn(n_rays, 3, device=device) * 0.5
    rays_d = torch.randn(n_rays, 3, device=device)
    rays_d = rays_d / rays_d.norm(dim=-1, keepdim=True)
    pixels_gt = torch.rand(n_rays, 3, device=device)  # Ground truth colors

    # Update occupancy grid
    def occ_eval_fn(x):
        return radiance_field.query_density(x) * render_step_size

    estimator.update_every_n_steps(
        step=step, occ_eval_fn=occ_eval_fn, occ_thre=1e-2, n=16
    )

    # Define callback functions
    def sigma_fn(t_starts, t_ends, ray_indices):
        t_origins = rays_o[ray_indices]
        t_dirs = rays_d[ray_indices]
        positions = t_origins + t_dirs * (t_starts + t_ends)[:, None] / 2.0
        return radiance_field.query_density(positions)

    def rgb_sigma_fn(t_starts, t_ends, ray_indices):
        t_origins = rays_o[ray_indices]
        t_dirs = rays_d[ray_indices]
        positions = t_origins + t_dirs * (t_starts + t_ends)[:, None] / 2.0
        rgbs, sigmas = radiance_field(positions, t_dirs)
        return rgbs, sigmas

    # Ray marching with spatial skipping
    ray_indices, t_starts, t_ends = estimator.sampling(
        rays_o, rays_d,
        sigma_fn=sigma_fn,
        near_plane=0.0, far_plane=10.0,
        render_step_size=render_step_size,
        early_stop_eps=1e-4, alpha_thre=1e-2,
    )

    # Skip if no samples
    if ray_indices.shape[0] == 0:
        continue

    # Volumetric rendering
    colors, opacities, depths, extras = nerfacc.rendering(
        t_starts, t_ends, ray_indices,
        n_rays=n_rays,
        rgb_sigma_fn=rgb_sigma_fn,
        render_bkgd=torch.ones(3, device=device),
    )

    # Compute loss and update
    loss = F.mse_loss(colors, pixels_gt)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if step % 1000 == 0:
        psnr = -10 * torch.log10(loss)
        print(f"Step {step}: Loss={loss.item():.6f}, PSNR={psnr.item():.2f}")
```

--------------------------------

### Install NerfAcc with Pre-built Wheels

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

Install NerfAcc using pre-built wheels for specific PyTorch and CUDA versions. Example for torch 1.13.0 + cu117.

```bash
# e.g., torch 1.13.0 + cu117
pip install nerfacc -f https://nerfacc-bucket.s3.us-west-2.amazonaws.com/whl/torch-1.13.0_cu117.html
```

--------------------------------

### Install NerfAcc from PyPI

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

Install NerfAcc using pip. CUDA code is built on the first run via JIT.

```bash
pip install nerfacc
```

--------------------------------

### Compute Transmittance and Alpha from Density

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Computes transmittance and alpha values from density and sample intervals, following the standard NeRF formulation. Requires sample start and end points.

```python
import torch
import nerfacc

t_starts = torch.tensor([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0], device="cuda")
t_ends = torch.tensor([1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0], device="cuda")
sigmas = torch.tensor([0.4, 0.8, 0.1, 0.8, 0.1, 0.0, 0.9], device="cuda")
ray_indices = torch.tensor([0, 0, 0, 1, 1, 2, 2], device="cuda")

# Compute transmittance and alpha
transmittance, alphas = nerfacc.render_transmittance_from_density(
    t_starts=t_starts,
    t_ends=t_ends,
    sigmas=sigmas,
    ray_indices=ray_indices,
)
```

--------------------------------

### Pack Ray Indices to Packed Info

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Converts a flat tensor of ray indices into a packed format, which specifies the start index and count of samples for each ray. This is crucial for efficient batch processing of ray data where rays have varying numbers of samples.

```python
import torch
import nerfacc

# Ray indices for samples (sorted)
ray_indices = torch.tensor([0, 0, 1, 1, 1, 2, 2, 2, 2], device="cuda")

# Convert to packed info
packed_info = nerfacc.pack_info(ray_indices, n_rays=3)

# packed_info: [[0, 2], [2, 3], [5, 4]]
# Each row: [start_index, count]
# Ray 0: starts at index 0, has 2 samples
# Ray 1: starts at index 2, has 3 samples
# Ray 2: starts at index 5, has 4 samples
```

--------------------------------

### Clone NerfAcc Repository

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command clones the nerfacc repository, including its submodules. Ensure you have git installed.

```bash
git clone --recursive git://github.com/nerfstudio-project/nerfacc/
```

--------------------------------

### RaySamples Data Class

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Data class for representing ray samples with support for both batched and flattened tensor formats. This example shows initialization with batched sample values.

```python
import torch
import nerfacc

# Batched format: (n_rays, n_samples)
batched_samples = nerfacc.RaySamples(
    vals=torch.rand(100, 64, device="cuda")  # 100 rays, 64 samples each
)
```

--------------------------------

### Initialize and Update OccGridEstimator

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Configures an occupancy grid estimator and demonstrates the update loop using a radiance field callback.

```python
import torch
import nerfacc

# Define the axis-aligned bounding box (xmin, ymin, zmin, xmax, ymax, zmax)
aabb = torch.tensor([-1.5, -1.5, -1.5, 1.5, 1.5, 1.5], device="cuda")

# Create an occupancy grid estimator with 128^3 resolution and 1 level
estimator = nerfacc.OccGridEstimator(
    roi_aabb=aabb,
    resolution=128,  # Can also be [128, 128, 128] for non-cubic grids
    levels=1,        # Number of cascade levels (use 4 for unbounded scenes)
).to("cuda")

# Define a radiance field (your NeRF model)
class SimpleRadianceField(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.mlp = torch.nn.Sequential(
            torch.nn.Linear(3, 64),
            torch.nn.ReLU(),
            torch.nn.Linear(64, 4),  # RGB + density
        )

    def forward(self, positions, directions):
        out = self.mlp(positions)
        rgbs = torch.sigmoid(out[..., :3])
        sigmas = torch.nn.functional.relu(out[..., 3])
        return rgbs, sigmas

    def query_density(self, positions):
        out = self.mlp(positions)
        return torch.nn.functional.relu(out[..., 3])

radiance_field = SimpleRadianceField().to("cuda")

# Training loop
for step in range(20000):
    # Update occupancy grid every 16 steps
    def occ_eval_fn(positions):
        """Evaluate occupancy at given positions."""
        density = radiance_field.query_density(positions)
        return density * 1e-3  # density * step_size

    estimator.update_every_n_steps(
        step=step,
        occ_eval_fn=occ_eval_fn,
        occ_thre=1e-2,       # Threshold for binary occupancy
        ema_decay=0.95,      # EMA decay for occupancy updates
        warmup_steps=256,    # Sample all cells during warmup
        n=16,                # Update every n steps
    )
```

--------------------------------

### Define RaySamples with different formats

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Demonstrates initializing RaySamples using either packed_info or ray_indices for flattened sample data.

```python
flattened_samples = nerfacc.RaySamples(
    vals=torch.rand(1000, device="cuda"),  # All samples flattened
    packed_info=torch.tensor([[0, 100], [100, 200], [300, 700]], device="cuda"),
)
```

```python
indexed_samples = nerfacc.RaySamples(
    vals=torch.rand(1000, device="cuda"),
    ray_indices=torch.randint(0, 100, (1000,), device="cuda").sort()[0],
)
```

--------------------------------

### Run K-Planes Benchmark Script

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command executes a benchmark script for K-Planes, integrating with nerfacc. Navigate to the benchmarks/kplanes directory and ensure the environment is set up.

```bash
cd benchmarks/kplanes/
bash script.sh dnerf-nerfacc-occgrid 0
```

--------------------------------

### Implement PropNetEstimator

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Sets up lightweight proposal networks and the PropNetEstimator to perform importance sampling based on density distributions.

```python
import torch
import nerfacc

# Create lightweight proposal networks
class ProposalNetwork(torch.nn.Module):
    def __init__(self, hidden_dim=64):
        super().__init__()
        self.mlp = torch.nn.Sequential(
            torch.nn.Linear(3, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, 1),
        )

    def forward(self, t_starts, t_ends):
        """Predict density for sample intervals."""
        # t_starts, t_ends: (n_rays, n_samples)
        t_mids = (t_starts + t_ends) / 2.0
        # In practice, you'd convert these to 3D positions
        sigmas = torch.nn.functional.softplus(
            self.mlp(t_mids.unsqueeze(-1).expand(-1, -1, 3))
        ).squeeze(-1)
        return sigmas  # (n_rays, n_samples)

# Create proposal networks and their optimizer
prop_net_1 = ProposalNetwork(hidden_dim=64).to("cuda")
prop_net_2 = ProposalNetwork(hidden_dim=128).to("cuda")
prop_optimizer = torch.optim.Adam(
    list(prop_net_1.parameters()) + list(prop_net_2.parameters()),
    lr=1e-2,
)
prop_scheduler = torch.optim.lr_scheduler.ExponentialLR(prop_optimizer, gamma=0.9999)

# Create estimator with optimizer
estimator = nerfacc.PropNetEstimator(
    optimizer=prop_optimizer,
    scheduler=prop_scheduler,
)

# Define proposal sigma functions
def prop_sigma_fn_1(t_starts, t_ends):
    return prop_net_1(t_starts, t_ends)

def prop_sigma_fn_2(t_starts, t_ends):
    return prop_net_2(t_starts, t_ends)

# Get schedule for when to compute gradients for proposal networks
proposal_requires_grad_fn = nerfacc.estimators.prop_net.get_proposal_requires_grad_fn(
    target=5.0,      # Average steps between gradient updates
    num_steps=1000,  # Steps to reach full update frequency
)
```

--------------------------------

### Run TensoRF Benchmark Script

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command executes a benchmark script for TensoRF, integrating with nerfacc. Navigate to the benchmarks/tensorf directory and ensure the environment is set up.

```bash
cd benchmarks/tensorf/
bash script.sh nerfsyn-nerfacc-occgrid 0
```

```bash
cd benchmarks/tensorf/
bash script.sh tt-nerfacc-occgrid 0
```

--------------------------------

### Perform Ray Marching with OccGridEstimator

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Executes efficient ray marching using the occupancy grid to skip empty space, utilizing a sigma function for visibility-based pruning.

```python
import torch
import nerfacc

# Setup estimator and rays
aabb = torch.tensor([-1.5, -1.5, -1.5, 1.5, 1.5, 1.5], device="cuda")
estimator = nerfacc.OccGridEstimator(roi_aabb=aabb, resolution=128).to("cuda")

# Generate rays (typically from camera)
n_rays = 1024
rays_o = torch.randn(n_rays, 3, device="cuda")  # Ray origins
rays_d = torch.randn(n_rays, 3, device="cuda")  # Ray directions
rays_d = rays_d / rays_d.norm(dim=-1, keepdim=True)  # Normalize

# Define sigma function for visibility-based skipping
def sigma_fn(t_starts, t_ends, ray_indices):
    """Compute density at sample positions."""
    t_origins = rays_o[ray_indices]
    t_dirs = rays_d[ray_indices]
    positions = t_origins + t_dirs * (t_starts + t_ends)[:, None] / 2.0
    sigmas = torch.rand(t_starts.shape[0], device="cuda")  # Replace with actual model
    return sigmas  # Shape: (n_samples,)

# Perform ray marching with spatial skipping
ray_indices, t_starts, t_ends = estimator.sampling(
    rays_o=rays_o,
    rays_d=rays_d,
    sigma_fn=sigma_fn,           # Optional: for visibility-based skipping
    near_plane=0.0,              # Near clipping plane
    far_plane=1e10,              # Far clipping plane
    render_step_size=1e-3,       # Step size for marching
    early_stop_eps=1e-4,         # Stop when transmittance drops below this
    alpha_thre=1e-2,             # Skip samples with alpha below this
    stratified=False,            # Randomize sample positions
    cone_angle=0.0,              # For mip-mapping (0 = constant step size)
)
```

--------------------------------

### Run TiNeuVox Benchmark Scripts

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

These commands execute benchmark scripts for TiNeuVox on D-NeRF and HyperNeRF datasets, integrating with nerfacc. Navigate to the benchmarks/tineuvox directory and ensure the environment is set up.

```bash
cd benchmarks/tineuvox/
bash script.sh dnerf-nerfacc-occgrid 0
```

```bash
cd benchmarks/tineuvox/
bash script.sh hypernerf-nerfacc-occgrid 0
```

```bash
cd benchmarks/tineuvox/
bash script.sh hypernerf-nerfacc-propnet 0
```

--------------------------------

### Train Instant-NGP NeRF on Mip-NeRF 360 Dataset (Proposal Net)

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command trains an Instant-NGP NeRF model with a proposal net estimator on the Mip-NeRF 360 dataset. Adjust the scene and data root as needed.

```bash
python examples/train_ngp_nerf_prop.py --scene garden --data_root data/360_v2
```

--------------------------------

### Train Instant-NGP NeRF with Occupancy Grid Estimator

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command trains an Instant-NGP NeRF model using an occupancy grid estimator on the NeRF-Synthetic dataset. Specify the scene and data root path.

```bash
python examples/train_ngp_nerf_occ.py --scene lego --data_root data/nerf_synthetic
```

--------------------------------

### Train Instant-NGP NeRF with Proposal Net Estimator

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command trains an Instant-NGP NeRF model using a proposal net estimator on the NeRF-Synthetic dataset. Specify the scene and data root path.

```bash
python examples/train_ngp_nerf_prop.py --scene lego --data_root data/nerf_synthetic
```

--------------------------------

### Train Instant-NGP NeRF on Mip-NeRF 360 Dataset (Occupancy Grid)

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command trains an Instant-NGP NeRF model with an occupancy grid estimator on the Mip-NeRF 360 dataset. Adjust the scene and data root as needed.

```bash
python examples/train_ngp_nerf_occ.py --scene garden --data_root data/360_v2
```

--------------------------------

### render_visibility_from_alpha

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Computes sample visibility for early ray termination and empty space skipping.

```APIDOC
## render_visibility_from_alpha

### Description
Computes sample visibility for early ray termination and empty space skipping. Returns a boolean mask indicating which samples should be kept.

### Method
None (Function call)

### Endpoint
None (Function call)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **alphas** (torch.Tensor) - Alpha (opacity) values.
- **ray_indices** (torch.Tensor) - Indices of the rays.
- **early_stop_eps** (float, optional) - Threshold for transmittance to stop early. Defaults to a small value.
- **alpha_thre** (float, optional) - Threshold for alpha to consider a sample significant. Defaults to a small value.

### Request Example
```python
visibility = nerfacc.render_visibility_from_alpha(
    alphas=alphas,
    ray_indices=ray_indices,
    early_stop_eps=0.3,
    alpha_thre=0.2,
)
```

### Response
#### Success Response (200)
- **visibility** (torch.Tensor) - Boolean mask indicating visible samples.

#### Response Example
```json
{
  "visibility": "[n_samples] boolean tensor"
}
```
```

--------------------------------

### Run BARF Benchmark Script

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command executes a benchmark script for BARF on the NeRF-Synthetic dataset, integrating with nerfacc. Navigate to the benchmarks/barf directory and ensure the environment is set up.

```bash
cd benchmarks/barf/
bash script.sh nerfsyn-nerfacc-occgrid 0
```

--------------------------------

### Optimization Step in NerfAcc

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This snippet illustrates a typical optimization step using an optimizer. It includes zeroing gradients, calculating loss (e.g., MSE loss), backpropagation, and updating weights.

```python
optimizer.zero_grad()
loss = F.mse_loss(color, color_gt)
loss.backward()
optimizer.step()
```

--------------------------------

### Monitor GPU Utilization via Batch Size

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/docs/source/methodology/coding.md

Demonstrates how increasing the number of threads/elements processed on the GPU affects utilization percentage.

```python
# Only 1000 threads are running at the same time to create this tensor.
# So we see 28% GPU utilization.
while True: torch.zeros((1000), device="cuda")
# Now we see 100% GPU utilization.
while True: torch.zeros((10000000), device="cuda")
```

--------------------------------

### Efficient Raymarching with NerfAcc

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This snippet demonstrates efficient raymarching using the estimator.sampling function. It requires rays_o, rays_d, and a sigma_fn. Parameters like near_plane, far_plane, early_stop_eps, and alpha_thre can be adjusted for performance.

```python
ray_indices, t_starts, t_ends = estimator.sampling(
    rays_o, rays_d, sigma_fn=sigma_fn, near_plane=0.2, far_plane=1.0, early_stop_eps=1e-4, alpha_thre=1e-2
)
```

--------------------------------

### Compute Rendering Weights from Alpha

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Computes rendering weights and transmittance directly from pre-computed alpha (opacity) values. Useful when alpha is already known.

```python
import torch
import nerfacc

# Pre-computed alpha values
alphas = torch.tensor([0.4, 0.8, 0.1, 0.8, 0.1, 0.0, 0.9], device="cuda")
ray_indices = torch.tensor([0, 0, 0, 1, 1, 2, 2], device="cuda")
n_rays = 3

# Compute weights and transmittance
weights, transmittance = nerfacc.render_weight_from_alpha(
    alphas=alphas,
    ray_indices=ray_indices,
    n_rays=n_rays,
)
```

--------------------------------

### Avoid Unnecessary Synchronization

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/docs/source/methodology/coding.md

Shows how creating tensors directly on the device avoids the synchronization overhead incurred by moving data from CPU to GPU.

```python
# no sync. 67.3 µs ± 5.01 ns per loop
while True: torch.zeros((10000000), device="cuda")
# sync. 13.7 ms ± 320 µs per loop
while True: torch.zeros((10000000)).to("cuda")
```

--------------------------------

### Train Vanilla MLP NeRF with Occupancy Grid Estimator

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command trains a vanilla MLP NeRF model using an occupancy grid estimator on the NeRF-Synthetic dataset. Specify the scene and data root path.

```bash
python examples/train_mlp_nerf.py --scene lego --data_root data/nerf_synthetic
```

--------------------------------

### Traverse Hierarchical Occupancy Grids

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Performs ray traversal through a hierarchy of binary occupancy grids. Generates sample intervals within occupied voxels, considering ray direction, AABBs, and termination planes. Supports constant or cone-based step sizes.

```python
import torch
import nerfacc

# Ray data
n_rays = 1024
rays_o = torch.randn(n_rays, 3, device="cuda")
rays_d = torch.randn(n_rays, 3, device="cuda")
rays_d = rays_d / rays_d.norm(dim=-1, keepdim=True)

# Binary occupancy grids (n_levels, resx, resy, resz)
resolution = 128
n_levels = 2
binaries = torch.rand(n_levels, resolution, resolution, resolution, device="cuda") > 0.5

# AABBs for each level (typically 2x scaled for cascades)
aabbs = torch.tensor([
    [-1.0, -1.0, -1.0, 1.0, 1.0, 1.0],
    [-2.0, -2.0, -2.0, 2.0, 2.0, 2.0],
], device="cuda")

# Near and far planes per ray
near_planes = torch.zeros(n_rays, device="cuda")
far_planes = torch.full((n_rays,), 10.0, device="cuda")

# Traverse grids
intervals, samples, termination_planes = nerfacc.traverse_grids(
    rays_o=rays_o,
    rays_d=rays_d,
    binaries=binaries,
    aabbs=aabbs,
    near_planes=near_planes,
    far_planes=far_planes,
    step_size=1e-3,
    cone_angle=0.0,  # 0 for constant step size
)

# intervals: RayIntervals object with sample boundaries
# samples: RaySamples object with sample data
# termination_planes: (n_rays,) final t values
```

--------------------------------

### Train T-NeRF with Occupancy Grid Estimator

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This command trains a T-NeRF model using an occupancy grid estimator on the D-NeRF dataset. Specify the scene and data root path.

```bash
python examples/train_mlp_tnerf.py --scene lego --data_root data/dnerf
```

--------------------------------

### render_visibility_from_density

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Computes sample visibility from density values, combining transmittance and alpha thresholds for efficient pruning.

```APIDOC
## render_visibility_from_density

### Description
Computes sample visibility from density values. Combines transmittance and alpha thresholds for efficient pruning.

### Parameters
- **t_starts** (torch.Tensor) - Start distances of samples.
- **t_ends** (torch.Tensor) - End distances of samples.
- **sigmas** (torch.Tensor) - Density values at samples.
- **ray_indices** (torch.Tensor) - Indices mapping samples to rays.
- **early_stop_eps** (float) - Epsilon for early stopping.
- **alpha_thre** (float) - Alpha threshold for visibility pruning.
```

--------------------------------

### pack_info

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Converts ray indices to packed info format for efficient per-ray operations.

```APIDOC
## pack_info

### Description
Converts ray indices to packed info format for efficient per-ray operations on flattened tensors.

### Parameters
- **ray_indices** (torch.Tensor) - Sorted ray indices for samples.
- **n_rays** (int) - Total number of rays.
```

--------------------------------

### Compute Sample Visibility from Alpha

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Determines sample visibility using alpha values and optional thresholds for early ray termination and empty space skipping. Returns a boolean mask.

```python
import torch
import nerfacc

alphas = torch.tensor([0.4, 0.8, 0.1, 0.8, 0.1, 0.0, 0.9], device="cuda")
ray_indices = torch.tensor([0, 0, 0, 1, 1, 2, 2], device="cuda")

# Compute visibility mask
visibility = nerfacc.render_visibility_from_alpha(
    alphas=alphas,
    ray_indices=ray_indices,
    early_stop_eps=0.3,   # Skip samples where transmittance < 0.3
    alpha_thre=0.2,       # Skip samples where alpha < 0.2
)
```

--------------------------------

### Inclusive and Exclusive Product on Rays

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Computes inclusive or exclusive cumulative products along rays, essential for computing transmittance in volumetric rendering. `exclusive_prod` is particularly useful for transmittance calculations.

```python
import torch
import nerfacc

inputs = torch.tensor([1., 2., 3., 4., 5., 6., 7., 8., 9.], device="cuda")
packed_info = torch.tensor([[0, 2], [2, 3], [5, 4]], device="cuda")

# Inclusive product
inclusive = nerfacc.inclusive_prod(inputs, packed_info=packed_info)
# inclusive: [1., 2., 3., 12., 60., 6., 42., 336., 3024.]

# Exclusive product (for transmittance calculation)
exclusive = nerfacc.exclusive_prod(inputs, packed_info=packed_info)
# exclusive: [1., 1., 1., 3., 12., 1., 6., 42., 336.]
# Each ray starts with 1

# Used for transmittance: T_i = exclusive_prod(1 - alpha)
```

--------------------------------

### render_transmittance_from_density

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Computes transmittance and alpha values from density and sample intervals. This is the standard NeRF formulation.

```APIDOC
## render_transmittance_from_density

### Description
Computes transmittance and alpha values from density and sample intervals. This is the standard NeRF formulation.

### Method
None (Function call)

### Endpoint
None (Function call)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **t_starts** (torch.Tensor) - Start of the ray intervals.
- **t_ends** (torch.Tensor) - End of the ray intervals.
- **sigmas** (torch.Tensor) - Volume densities at sample points.
- **ray_indices** (torch.Tensor) - Indices of the rays.

### Request Example
```python
transmittance, alphas = nerfacc.render_transmittance_from_density(
    t_starts=t_starts,
    t_ends=t_ends,
    sigmas=sigmas,
    ray_indices=ray_indices,
)
```

### Response
#### Success Response (200)
- **transmittance** (torch.Tensor) - Transmittance values for each sample.
- **alphas** (torch.Tensor) - Alpha (opacity) values for each sample.

#### Response Example
```json
{
  "transmittance": "[n_samples] tensor",
  "alphas": "[n_samples] tensor"
}
```
```

--------------------------------

### Accumulate Volumetric Values Along Rays

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Accumulates volumetric data (like colors, opacities, or depths) along rays based on provided weights. When 'values' is None, it sums the weights to compute total opacity. Requires pre-computed weights and ray indices.

```python
import torch
import nerfacc

# Sample data
n_rays = 3
weights = torch.tensor([0.4, 0.48, 0.012, 0.8, 0.02, 0.0, 0.9], device="cuda")
ray_indices = torch.tensor([0, 0, 0, 1, 1, 2, 2], device="cuda")

# RGB values at each sample
rgbs = torch.rand(7, 3, device="cuda")

# Accumulate colors along rays
colors = nerfacc.accumulate_along_rays(
    weights=weights,
    values=rgbs,
    ray_indices=ray_indices,
    n_rays=n_rays,
)
# colors: (n_rays, 3) - weighted sum of RGB values per ray

# Accumulate opacities (no values = sum of weights)
opacities = nerfacc.accumulate_along_rays(
    weights=weights,
    values=None,  # Sum weights when values=None
    ray_indices=ray_indices,
    n_rays=n_rays,
)
# opacities: (n_rays, 1) - total accumulated opacity per ray

# Accumulate depths
t_starts = torch.tensor([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0], device="cuda")
t_ends = torch.tensor([1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0], device="cuda")
depths = nerfacc.accumulate_along_rays(
    weights=weights,
    values=(t_starts + t_ends)[:, None] / 2.0,  # Midpoint distances
    ray_indices=ray_indices,
    n_rays=n_rays,
)
# depths: (n_rays, 1) - weighted average depth per ray
```

--------------------------------

### Implement NerfAcc Rendering Pipeline

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/docs/source/index.md

Define density and color functions to integrate NerfAcc with a custom radiance field for efficient raymarching and differentiable rendering.

```python
import torch
from torch import Tensor
import nerfacc

radiance_field = ...  # network: a NeRF model
rays_o: Tensor = ...  # ray origins. (n_rays, 3)
rays_d: Tensor = ...  # ray normalized directions. (n_rays, 3)
optimizer = ...  # optimizer

estimator = nerfacc.OccGridEstimator(...)

def sigma_fn(
   t_starts: Tensor, t_ends:Tensor, ray_indices: Tensor
) -> Tensor:
   """ Define how to query density for the estimator."""
   t_origins = rays_o[ray_indices]  # (n_samples, 3)
   t_dirs = rays_d[ray_indices]  # (n_samples, 3)
   positions = t_origins + t_dirs * (t_starts + t_ends)[:, None] / 2.0
   sigmas = radiance_field.query_density(positions)
   return sigmas  # (n_samples,)

def rgb_sigma_fn(
   t_starts: Tensor, t_ends: Tensor, ray_indices: Tensor
) -> Tuple[Tensor, Tensor]:
   """ Query rgb and density values from a user-defined radiance field. """
   t_origins = rays_o[ray_indices]  # (n_samples, 3)
   t_dirs = rays_d[ray_indices]  # (n_samples, 3)
   positions = t_origins + t_dirs * (t_starts + t_ends)[:, None] / 2.0
   rgbs, sigmas = radiance_field(positions, condition=t_dirs)
   return rgbs, sigmas  # (n_samples, 3), (n_samples,)

# Efficient Raymarching:
# ray_indices: (n_samples,). t_starts: (n_samples,). t_ends: (n_samples,).
ray_indices, t_starts, t_ends = estimator.sampling(
   rays_o, rays_d, sigma_fn=sigma_fn, near_plane=0.2, far_plane=1.0,
   early_stop_eps=1e-4, alpha_thre=1e-2,
)

# Differentiable Volumetric Rendering.
# colors: (n_rays, 3). opaicity: (n_rays, 1). depth: (n_rays, 1).
color, opacity, depth, extras = nerfacc.rendering(
   t_starts, t_ends, ray_indices, n_rays=rays_o.shape[0], rgb_sigma_fn=rgb_sigma_fn
)

# Optimize: Both the network and rays will receive gradients
optimizer.zero_grad()
loss = F.mse_loss(color, color_gt)
loss.backward()
optimizer.step()
```

--------------------------------

### Compute Visibility from Density

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Calculates ray visibility using density values, incorporating transmittance and alpha thresholds for efficient pruning. Useful for determining which ray samples are likely to contribute to the final pixel color.

```python
import torch
import nerfacc

t_starts = torch.tensor([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0], device="cuda")
t_ends = torch.tensor([1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0], device="cuda")
sigmas = torch.tensor([0.4, 0.8, 0.1, 0.8, 0.1, 0.0, 0.9], device="cuda")
ray_indices = torch.tensor([0, 0, 0, 1, 1, 2, 2], device="cuda")

# Compute visibility mask
visibility = nerfacc.render_visibility_from_density(
    t_starts=t_starts,
    t_ends=t_ends,
    sigmas=sigmas,
    ray_indices=ray_indices,
    early_stop_eps=0.3,
    alpha_thre=0.2,
)

# visibility: [True, True, False, True, False, False, True]
```

--------------------------------

### traverse_grids

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Performs ray traversal through multiple hierarchical occupancy grids to generate sample intervals.

```APIDOC
## traverse_grids

### Description
Performs ray traversal through multiple hierarchical occupancy grids, generating sample intervals within occupied voxels.

### Parameters
- **rays_o** (torch.Tensor) - Ray origins.
- **rays_d** (torch.Tensor) - Ray directions.
- **binaries** (torch.Tensor) - Binary occupancy grids.
- **aabbs** (torch.Tensor) - AABBs for each grid level.
- **near_planes** (torch.Tensor) - Near planes per ray.
- **far_planes** (torch.Tensor) - Far planes per ray.
- **step_size** (float) - Step size for traversal.
- **cone_angle** (float) - Cone angle for sampling.
```

--------------------------------

### Define RGB and Sigma Function for Rendering

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

Define the rgb_sigma_fn to query color and density for differentiable volumetric rendering. This function receives sample intervals and ray indices.

```python
import torch
from torch import Tensor
import nerfacc 
from typing import Tuple

radiance_field = ...  # network: a NeRF model
rays_o: Tensor = ...  # ray origins. (n_rays, 3)
rays_d: Tensor = ...  # ray normalized directions. (n_rays, 3)
optimizer = ...       # optimizer

estimator = nerfacc.OccGridEstimator(...)

def rgb_sigma_fn(
    t_starts: Tensor, t_ends: Tensor, ray_indices: Tensor
) -> Tuple[Tensor, Tensor]:
    """ Query rgb and density values from a user-defined radiance field. """
    t_origins = rays_o[ray_indices]  # (n_samples, 3)
    t_dirs = rays_d[ray_indices]  # (n_samples, 3)
    positions = t_origins + t_dirs * (t_starts + t_ends)[:, None] / 2.0
    rgbs, sigmas = radiance_field(positions, condition=t_dirs)  
    return rgbs, sigmas  # (n_samples, 3), (n_samples,)
```

--------------------------------

### Compute Transmittance from Alpha

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Calculates transmittance values from alpha (opacity) values. The first sample of each ray always has a transmittance of 1.0.

```python
import torch
import nerfacc

alphas = torch.tensor([0.4, 0.8, 0.1, 0.8, 0.1, 0.0, 0.9], device="cuda")
ray_indices = torch.tensor([0, 0, 0, 1, 1, 2, 2], device="cuda")

# Compute transmittance
transmittance = nerfacc.render_transmittance_from_alpha(
    alphas=alphas,
    ray_indices=ray_indices,
)
```

--------------------------------

### Compute 3D Sample Positions

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Calculates 3D coordinates for ray samples based on interval midpoints and ray directions.

```python
t_mids = (t_starts + t_ends) / 2.0
sample_positions = rays_o[ray_indices] + t_mids[:, None] * rays_d[ray_indices]
```

--------------------------------

### render_weight_from_alpha

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Computes rendering weights directly from pre-computed alpha (opacity) values.

```APIDOC
## render_weight_from_alpha

### Description
Computes rendering weights directly from pre-computed alpha (opacity) values. Useful when alpha is pre-computed or when working with alpha-based radiance fields.

### Method
None (Function call)

### Endpoint
None (Function call)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **alphas** (torch.Tensor) - Pre-computed alpha (opacity) values.
- **ray_indices** (torch.Tensor) - Indices of the rays.
- **n_rays** (int) - Number of rays.

### Request Example
```python
weights, transmittance = nerfacc.render_weight_from_alpha(
    alphas=alphas,
    ray_indices=ray_indices,
    n_rays=n_rays,
)
```

### Response
#### Success Response (200)
- **weights** (torch.Tensor) - Computed rendering weights.
- **transmittance** (torch.Tensor) - Transmittance values for each sample.

#### Response Example
```json
{
  "weights": "[n_samples] tensor",
  "transmittance": "[n_samples] tensor"
}
```
```

--------------------------------

### Compute Rendering Weights from Density

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Calculates rendering weights, transmittance, and alpha values from volume density and sample intervals. This is a core function for volumetric rendering.

```python
import torch
import nerfacc

# Sample data
t_starts = torch.tensor([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0], device="cuda")
t_ends = torch.tensor([1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0], device="cuda")
sigmas = torch.tensor([0.4, 0.8, 0.1, 0.8, 0.1, 0.0, 0.9], device="cuda")
ray_indices = torch.tensor([0, 0, 0, 1, 1, 2, 2], device="cuda")
n_rays = 3

# Compute weights, transmittance, and alpha values
weights, transmittance, alphas = nerfacc.render_weight_from_density(
    t_starts=t_starts,
    t_ends=t_ends,
    sigmas=sigmas,
    ray_indices=ray_indices,
    n_rays=n_rays,
)
```

--------------------------------

### render_weight_from_density

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Computes rendering weights from density (sigma) values and sample intervals. This is a core function for volumetric rendering.

```APIDOC
## render_weight_from_density

### Description
Computes rendering weights from density (sigma) values and sample intervals. This is a core function for volumetric rendering.

### Method
None (Function call)

### Endpoint
None (Function call)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **t_starts** (torch.Tensor) - Start of the ray intervals.
- **t_ends** (torch.Tensor) - End of the ray intervals.
- **sigmas** (torch.Tensor) - Volume densities at sample points.
- **ray_indices** (torch.Tensor) - Indices of the rays.
- **n_rays** (int) - Number of rays.

### Request Example
```python
weights, transmittance, alphas = nerfacc.render_weight_from_density(
    t_starts=t_starts,
    t_ends=t_ends,
    sigmas=sigmas,
    ray_indices=ray_indices,
    n_rays=n_rays,
)
```

### Response
#### Success Response (200)
- **weights** (torch.Tensor) - Computed rendering weights.
- **transmittance** (torch.Tensor) - Transmittance values for each sample.
- **alphas** (torch.Tensor) - Opacity values for each sample.

#### Response Example
```json
{
  "weights": "[n_samples] tensor",
  "transmittance": "[n_samples] tensor",
  "alphas": "[n_samples] tensor"
}
```
```

--------------------------------

### Benchmark Synchronization Latency

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/docs/source/methodology/coding.md

Uses %timeit to measure the performance difference between index-based and mask-based tensor access.

```python
# 177 µs ± 15.4 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
%timeit data[ids]
# 355 µs ± 466 ns per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
%timeit data[mask]
```

--------------------------------

### Perform Differentiable Volumetric Rendering

Source: https://context7.com/nerfstudio-project/nerfacc/llms.txt

Renders colors, opacities, and depths using a provided RGB-sigma function and background color. Gradients can be backpropagated through the rendering process.

```python
colors, opacities, depths, extras = nerfacc.rendering(
    t_starts=t_starts,
    t_ends=t_ends,
    ray_indices=ray_indices,
    n_rays=n_rays,
    rgb_sigma_fn=rgb_sigma_fn,  # Or use rgb_alpha_fn for pre-computed alpha
    render_bkgd=torch.ones(3, device=device),  # White background
    expected_depths=True,  # Normalize depths by accumulated weight
)
```

--------------------------------

### Differentiable Volumetric Rendering with NerfAcc

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/README.md

This snippet shows how to perform differentiable volumetric rendering using nerfacc.rendering. It takes t_starts, t_ends, ray_indices, and rgb_sigma_fn as input to compute color, opacity, and depth.

```python
color, opacity, depth, extras = nerfacc.rendering(
    t_starts, t_ends, ray_indices, n_rays=rays_o.shape[0], rgb_sigma_fn=rgb_sigma_fn
)
```

--------------------------------

### Identify CPU-GPU Synchronization Impact

Source: https://github.com/nerfstudio-project/nerfacc/blob/master/docs/source/methodology/coding.md

Compares GPU utilization when using index-based access versus mask-based access, where the latter triggers synchronization.

```python
data = torch.rand((10000000), device="cuda")
mask = data > 0.5
ids = torch.where(mask)[0]
assert torch.all(data[mask] == data[ids])

# 100% GPU utilization.
while True: data[ids]
# 95% GPU utilization.
while True: data[mask]
```