### Install APEX

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/INSTALL.md

Installs the APEX library from source. A specific commit is recommended for compatibility. Use --global-option flags for C++ and CUDA extensions.

```bash
git clone https://github.com/NVIDIA/apex
cd apex
git checkout 5633f6  # recent commit doesn't build in our system 
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
```

--------------------------------

### Install spconv

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/INSTALL.md

Installs the spconv library, which requires Boost. This process involves cloning the repository, checking out a specific commit, building a wheel, and then installing it.

```bash
sudo apt-get install libboost-all-dev
git clone https://github.com/traveller59/spconv.git --recursive
cd spconv && git checkout 7342772
python setup.py bdist_wheel
cd ./dist && pip install *
```

--------------------------------

### Install CenterPoint and Dependencies

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Follow these steps to set up the CenterPoint environment, including PyTorch, CUDA extensions, and spconv. Ensure correct versions are installed for compatibility.

```bash
conda create --name centerpoint python=3.6
conda activate centerpoint
conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=10.0 -c pytorch
```

```bash
git clone https://github.com/tianweiy/CenterPoint.git
cd CenterPoint
pip install -r requirements.txt
```

```bash
export PYTHONPATH="${PYTHONPATH}:/path/to/CenterPoint"
```

```bash
cd det3d/ops/iou3d_nms
python setup.py build_ext --inplace
```

```bash
sudo apt-get install libboost-all-dev
git clone https://github.com/traveller59/spconv.git --recursive
cd spconv && git checkout 7342772
python setup.py bdist_wheel
cd ./dist && pip install *
```

```bash
pip install nuscenes-devkit
```

```bash
pip install waymo-open-dataset-tf-1-15-0==1.2.0
```

--------------------------------

### Activate Environment and Install Libraries

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/WAYMO.md

Activate the conda environment and install the Waymo Open Dataset library for TensorFlow 1.15.0.

```bash
conda activate centerpoint 
pip install waymo-open-dataset-tf-1-15-0==1.2.0
```

--------------------------------

### Install nuScenes dev-kit

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/INSTALL.md

Clones the nuScenes dev-kit repository and adds its Python SDK to the PYTHONPATH. Ensure the path is updated correctly.

```bash
git clone https://github.com/tianweiy/nuscenes-devkit

# add the following line to ~/.bashrc and reactivate bash (remember to change the PATH_TO_NUSCENES_DEVKIT value)
export PYTHONPATH="${PYTHONPATH}:PATH_TO_NUSCENES_DEVKIT/python-sdk"
```

--------------------------------

### Configure CUDA Environment Variables

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/INSTALL.md

Sets up essential CUDA environment variables for compilation. Adjust the CUDA path to match your installation.

```bash
# set the cuda path(change the path to your own cuda location) 
export PATH=/usr/local/cuda-10.0/bin:$PATH
export CUDA_PATH=/usr/local/cuda-10.0
export CUDA_HOME=/usr/local/cuda-10.0
export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64:$LD_LIBRARY_PATH
```

--------------------------------

### Distributed Training

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Start a distributed training job using multiple GPUs. Models and logs are saved to the specified work directory.

```bash
python -m torch.distributed.launch --nproc_per_node=4 ./tools/train.py CONFIG_PATH
```

--------------------------------

### Basic CenterPoint Installation

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/INSTALL.md

Installs core Python libraries and CenterPoint using conda and pip. Remember to update the PYTHONPATH environment variable.

```bash
conda create --name centerpoint python=3.6
conda activate centerpoint
conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=10.0 -c pytorch
git clone https://github.com/tianweiy/CenterPoint.git
cd CenterPoint
pip install -r requirements.txt

# add CenterPoint to PYTHONPATH by adding the following line to ~/.bashrc (change the path accordingly)
export PYTHONPATH="${PYTHONPATH}:PATH_TO_CENTERPOINT"
```

--------------------------------

### Distributed Training with CenterPoint

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Launch distributed training for CenterPoint models using PyTorch's distributed launcher. Examples provided for VoxelNet and PointPillars on the nuScenes dataset.

```bash
# Train VoxelNet on nuScenes with 4 GPUs
python -m torch.distributed.launch --nproc_per_node=4 ./tools/train.py \
    configs/nusc/voxelnet/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z.py
```

```bash
# Train PointPillars on nuScenes
python -m torch.distributed.launch --nproc_per_node=4 ./tools/train.py \
    configs/nusc/pp/nusc_centerpoint_pp_02voxel_two_pfn_10sweep.py
```

--------------------------------

### Define CenterPoint Model Configuration

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Example configuration file structure for a VoxelNet-based CenterPoint model. Defines task groupings, model architecture, training settings, and data pipeline parameters.

```python
# configs/nusc/voxelnet/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z.py

import itertools
import logging
from det3d.utils.config_tool import get_downsample_factor

# Task definitions (class groupings for multi-head detection)
tasks = [
    dict(num_class=1, class_names=["car"]),
    dict(num_class=2, class_names=["truck", "construction_vehicle"]),
    dict(num_class=2, class_names=["bus", "trailer"]),
    dict(num_class=1, class_names=["barrier"]),
    dict(num_class=2, class_names=["motorcycle", "bicycle"]),
    dict(num_class=2, class_names=["pedestrian", "traffic_cone"]),
]
class_names = list(itertools.chain(*[t["class_names"] for t in tasks]))

# Model architecture
model = dict(
    type="VoxelNet",
    pretrained=None,
    reader=dict(type="VoxelFeatureExtractorV3", num_input_features=5),
    backbone=dict(type="SpMiddleResNetFHD", num_input_features=5, ds_factor=8),
    neck=dict(
        type="RPN",
        layer_nums=[5, 5],
        ds_layer_strides=[1, 2],
        ds_num_filters=[128, 256],
        us_layer_strides=[1, 2],
        us_num_filters=[256, 256],
        num_input_features=256,
    ),
    bbox_head=dict(
        type="CenterHead",
        in_channels=512,  # sum of us_num_filters
        tasks=tasks,
        dataset='nuscenes',
        weight=0.25,
        code_weights=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.2, 0.2, 1.0, 1.0],
        common_heads={'reg': (2, 2), 'height': (1, 2), 'dim': (3, 2), 'rot': (2, 2), 'vel': (2, 2)},
    ),
)

# Training settings
train_cfg = dict(assigner=dict(
    target_assigner=dict(tasks=tasks),
    out_size_factor=8, dense_reg=1, gaussian_overlap=0.1, max_objs=500, min_radius=2,
))

# Optimizer and scheduler
optimizer = dict(type="adam", amsgrad=0.0, wd=0.01, fixed_wd=True)
lr_config = dict(type="one_cycle", lr_max=0.001, moms=[0.95, 0.85], div_factor=10.0, pct_start=0.4)
total_epochs = 20

# Data configuration
data = dict(
    samples_per_gpu=4,
    workers_per_gpu=6,
    train=dict(type="NuScenesDataset", root_path="data/nuScenes", nsweeps=10, pipeline=train_pipeline),
    val=dict(type="NuScenesDataset", root_path="data/nuScenes", nsweeps=10, pipeline=test_pipeline, test_mode=True),
)
```

--------------------------------

### Initialize and Run PubTracker

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Configures the PubTracker for center-based tracking and demonstrates the frame-by-frame processing loop for detection results.

```python
import numpy as np
from tools.nusc_tracking.pub_tracker import PubTracker

# Initialize tracker
tracker = PubTracker(
    hungarian=False,  # Use greedy or Hungarian matching
    max_age=3         # Keep unmatched tracks for N frames
)

# Tracking class configuration
NUSCENES_TRACKING_NAMES = [
    'bicycle', 'bus', 'car', 'motorcycle',
    'pedestrian', 'trailer', 'truck'
]

# Velocity error thresholds per class (meters per 0.5 second)
VELOCITY_ERROR = {
    'car': 4, 'truck': 4, 'bus': 5.5, 'trailer': 3,
    'pedestrian': 1, 'motorcycle': 13, 'bicycle': 3,
}

# Process detections frame by frame
def run_tracking(detection_results, frame_timestamps):
    tracker.reset()
    tracking_results = []

    for i, (detections, timestamp) in enumerate(zip(detection_results, frame_timestamps)):
        # Calculate time lag from previous frame
        if i == 0:
            time_lag = 0.5  # Default time interval
            tracker.reset()
        else:
            time_lag = timestamp - frame_timestamps[i-1]

        # Format detections for tracker
        formatted_dets = []
        for det in detections:
            formatted_dets.append({
                'detection_name': det['class_name'],
                'detection_score': det['score'],
                'translation': det['translation'],  # [x, y, z]
                'size': det['size'],                # [w, l, h]
                'rotation': det['rotation'],        # quaternion
                'velocity': det['velocity'],        # [vx, vy]
            })

        # Run tracking step
        tracked_objects = tracker.step_centertrack(formatted_dets, time_lag)

        # Collect active tracks
        frame_results = []
        for obj in tracked_objects:
            if obj['active'] > 0:  # Only output active tracks
                frame_results.append({
                    'tracking_id': obj['tracking_id'],
                    'tracking_name': obj['detection_name'],
                    'tracking_score': obj['detection_score'],
                    'translation': obj['translation'],
                    'size': obj['size'],
                    'rotation': obj['rotation'],
                    'velocity': obj['velocity'],
                })
        tracking_results.append(frame_results)

    return tracking_results
```

--------------------------------

### Execute Two-Stage Detection Training and Initialization

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Commands for training a two-stage detection model and Python code for building the detector instance. The process involves freezing the backbone after the first stage.

```bash
# Two-stage model configuration
# First: train one-stage model
python -m torch.distributed.launch --nproc_per_node=4 ./tools/train.py \
    configs/waymo/voxelnet/waymo_centerpoint_voxelnet_3x.py

# Second: train two-stage refinement (freezes backbone)
python -m torch.distributed.launch --nproc_per_node=4 ./tools/train.py \
    configs/waymo/voxelnet/two_stage/waymo_centerpoint_voxelnet_two_stage_bev_5point_ft_6epoch_freeze.py
```

```python
from det3d.models import build_detector
from det3d.torchie import Config

# Two-stage configuration
cfg = Config.fromfile(
    'configs/waymo/voxelnet/two_stage/waymo_centerpoint_voxelnet_two_stage_bev_5point_ft_6epoch_freeze.py'
)

# Build two-stage detector
model = build_detector(cfg.model, train_cfg=cfg.train_cfg, test_cfg=cfg.test_cfg)

# Two-stage architecture:
# 1. First stage: VoxelNet produces initial detections
# 2. Second stage modules extract features at proposal locations
# 3. ROI head refines boxes and re-scores detections

# The model uses 5-point sampling (center + 4 corners) for ROI features
# Features are extracted from BEV feature map at proposal locations
# Final predictions combine first-stage scores with refinement confidence
```

--------------------------------

### Create Info Files for Waymo Dataset

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/WAYMO.md

Generate info files for Waymo dataset, supporting one or two sweep configurations. These files are crucial for model training.

```bash
# One Sweep Infos 
python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split train --nsweeps=1

python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split val --nsweeps=1

python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split test --nsweeps=1

# Two Sweep Infos (for two sweep detection and tracking models)
python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split train --nsweeps=2

python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split val --nsweeps=2

python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split test --nsweeps=2
```

--------------------------------

### Initialize and Run PointPillars Detector

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Configures and executes the PointPillars detector using a specified configuration file. Requires pre-processed pillar data in the expected dictionary format.

```python
import torch
from det3d.models import build_detector
from det3d.torchie import Config

# Load PointPillars configuration
cfg = Config.fromfile('configs/nusc/pp/nusc_centerpoint_pp_02voxel_two_pfn_10sweep.py')

# Build PointPillars detector
model = build_detector(
    cfg.model,
    train_cfg=cfg.train_cfg,
    test_cfg=cfg.test_cfg
)

# PointPillars architecture:
# - reader: PillarFeatureNet (PointNet-style pillar encoding)
# - backbone: PointPillarsScatter (scatter pillars to BEV pseudo-image)
# - neck: RPN (2D feature extraction)
# - bbox_head: CenterHead (same as VoxelNet)

# Input format for PointPillars
example = {
    'voxels': pillars,           # [N, max_points_per_pillar, num_features]
    'coordinates': pillar_coords, # [N, 3] (batch_idx, y, x)
    'num_points': num_points,     # [N] points per pillar
    'num_voxels': num_pillars,    # [batch_size] pillars per sample
    'shape': spatial_shape,       # [H, W] BEV grid dimensions
}

# Forward pass
model.eval()
with torch.no_grad():
    detections = model(example, return_loss=False)

# Output format
for det in detections:
    boxes = det['box3d_lidar']    # [K, 9] (x, y, z, w, l, h, vx, vy, yaw)
    scores = det['scores']        # [K] confidence scores
    labels = det['label_preds']   # [K] class indices
    token = det['metadata']['token']  # sample identifier
```

--------------------------------

### Prepare Waymo Dataset

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Convert Waymo Open Dataset tfrecord files to pickle format and generate info files for training, validation, and testing. Supports single and two-sweep configurations.

```bash
# Organize dataset directory structure
# WAYMO_DATASET_ROOT/
#   ├── tfrecord_training/
#   ├── tfrecord_validation/
#   └── tfrecord_testing/

# Convert tfrecord to pickle files
# Training set
CUDA_VISIBLE_DEVICES=-1 python det3d/datasets/waymo/waymo_converter.py \
    --record_path 'WAYMO_DATASET_ROOT/tfrecord_training/*.tfrecord' \
    --root_path 'WAYMO_DATASET_ROOT/train/'

# Validation set
CUDA_VISIBLE_DEVICES=-1 python det3d/datasets/waymo/waymo_converter.py \
    --record_path 'WAYMO_DATASET_ROOT/tfrecord_validation/*.tfrecord' \
    --root_path 'WAYMO_DATASET_ROOT/val/'

# Testing set
CUDA_VISIBLE_DEVICES=-1 python det3d/datasets/waymo/waymo_converter.py \
    --record_path 'WAYMO_DATASET_ROOT/tfrecord_testing/*.tfrecord' \
    --root_path 'WAYMO_DATASET_ROOT/test/'
```

```bash
# Create symlink
mkdir data && cd data
ln -s WAYMO_DATASET_ROOT Waymo
```

```bash
# Generate info files (single sweep)
python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split train --nsweeps=1
python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split val --nsweeps=1
python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split test --nsweeps=1

# Generate info files (two sweeps for tracking)
python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split train --nsweeps=2
python tools/create_data.py waymo_data_prep --root_path=data/Waymo --split val --nsweeps=2
```

--------------------------------

### Waymo 3D Detection Configuration Links

Source: https://github.com/tianweiy/centerpoint/blob/master/configs/waymo/README.md

Links to Python configuration files for various VoxelNet and PointPillars models used in Waymo 3D detection experiments.

```python
voxelnet/waymo_centerpoint_voxelnet_3x.py
```

```python
voxelnet/waymo_centerpoint_voxelnet_1x.py
```

```python
voxelnet/waymo_centerpoint_voxelnet_6epoch.py
```

```python
voxelnet/waymo_centerpoint_voxelnet_3epoch.py
```

```python
voxelnet/two_stage/waymo_centerpoint_voxelnet_two_stage_bev_5point_ft_6epoch_freeze.py
```

```python
voxelnet/waymo_centerpoint_voxelnet_two_sweeps_3x_with_velo.py
```

```python
voxelnet/two_stage/waymo_centerpoint_voxelnet_two_sweep_two_stage_bev_5point_ft_6epoch_freeze_with_vel.py
```

```python
pp/waymo_centerpoint_pp_two_pfn_stride1_3x.py
```

```python
pp/two_stage/waymo_centerpoint_pp_two_pfn_stride1_two_stage_bev_6epoch.py
```

```python
pp/waymo_centerpoint_pp_two_cls_two_pfn_stride1_3x.py
```

```python
pp/two_stage/waymo_centerpoint_pp_two_cls_two_pfn_stride1_two_stage_bev_6epoch.py
```

--------------------------------

### Configure VoxelGenerator

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Initialize voxelization parameters for different datasets like nuScenes and Waymo. Ensure the point cloud range and voxel dimensions match the specific dataset requirements.

```python
from det3d.core.input.voxel_generator import VoxelGenerator
import numpy as np

# nuScenes voxelization settings
voxel_generator = VoxelGenerator(
    voxel_size=[0.075, 0.075, 0.2],           # Voxel dimensions in meters
    point_cloud_range=[-54, -54, -5.0, 54, 54, 3.0],  # [x_min, y_min, z_min, x_max, y_max, z_max]
    max_num_points=10,                         # Max points per voxel
    max_voxels=120000                          # Max total voxels
)

# Access grid properties
print(f"Voxel size: {voxel_generator.voxel_size}")
print(f"Grid size: {voxel_generator.grid_size}")  # [1440, 1440, 40]
print(f"Point cloud range: {voxel_generator.point_cloud_range}")

# Generate voxels from point cloud
points = np.random.randn(100000, 5).astype(np.float32)  # [x, y, z, intensity, ring]
voxels, coordinates, num_points = voxel_generator.generate(points)

# voxels: [M, max_num_points, 5] - point features in each voxel
# coordinates: [M, 3] - voxel indices (z, y, x)
# num_points: [M] - actual number of points in each voxel

# Waymo voxelization (larger voxels for faster inference)
waymo_voxel_gen = VoxelGenerator(
    voxel_size=[0.1, 0.1, 0.15],
    point_cloud_range=[-75.2, -75.2, -2, 75.2, 75.2, 4],
    max_num_points=5,
    max_voxels=150000
)
```

--------------------------------

### Create nuScenes Data

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Use this command to create data and info files for the nuScenes dataset. Ensure this is run in a GPU environment.

```bash
# nuScenes
python tools/create_data.py nuscenes_data_prep --root_path=NUSCENES_TRAINVAL_DATASET_ROOT --version="v1.0-trainval" --nsweeps=10
```

--------------------------------

### Define Data Pipelines

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Set up training and testing pipelines using dictionary-based configurations. The training pipeline includes data augmentation steps like ground truth sampling, while the test pipeline focuses on inference-ready preprocessing.

```python
# Training pipeline configuration
train_pipeline = [
    dict(type="LoadPointCloudFromFile", dataset="NuScenesDataset"),
    dict(type="LoadPointCloudAnnotations", with_bbox=True),
    dict(type="Preprocess", cfg=dict(
        mode="train",
        shuffle_points=True,
        global_rot_noise=[-0.78539816, 0.78539816],  # Random rotation [-45, 45] degrees
        global_scale_noise=[0.9, 1.1],               # Random scaling
        global_translate_std=0.5,                     # Random translation
        db_sampler=dict(                             # Ground truth augmentation
            type="GT-AUG",
            enable=True,
            db_info_path="data/nuScenes/dbinfos_train_10sweeps_withvelo.pkl",
            sample_groups=[
                dict(car=2), dict(truck=3), dict(bus=4),
                dict(pedestrian=2), dict(motorcycle=6), dict(bicycle=6),
            ],
        ),
    )),
    dict(type="Voxelization", cfg=dict(
        range=[-54, -54, -5.0, 54, 54, 3.0],
        voxel_size=[0.075, 0.075, 0.2],
        max_points_in_voxel=10,
        max_voxel_num=[120000, 160000],  # [train, test]
    )),
    dict(type="AssignLabel", cfg=dict(
        target_assigner=dict(tasks=tasks),
        out_size_factor=8,
        dense_reg=1,
        gaussian_overlap=0.1,
        max_objs=500,
        min_radius=2,
    )),
    dict(type="Reformat"),
]

# Test pipeline (no augmentation)
test_pipeline = [
    dict(type="LoadPointCloudFromFile", dataset="NuScenesDataset"),
    dict(type="LoadPointCloudAnnotations", with_bbox=True),
    dict(type="Preprocess", cfg=dict(mode="val", shuffle_points=False)),
    dict(type="Voxelization", cfg=voxel_generator_cfg),
    dict(type="AssignLabel", cfg=assigner_cfg),
    dict(type="Reformat"),
]
```

--------------------------------

### Create Data Symlink

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/WAYMO.md

Create a symbolic link for the Waymo dataset within the 'data' directory. Remember to replace WAYMO_DATASET_ROOT with the actual path.

```bash
mkdir data && cd data
ln -s WAYMO_DATASET_ROOT Waymo
```

--------------------------------

### Prepare nuScenes Dataset

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Organize the nuScenes dataset and generate necessary info files and ground truth database. This involves creating symlinks and running a data preparation script.

```bash
# Organize dataset directory structure
# NUSCENES_DATASET_ROOT/
#   ├── samples/       <- key frames
#   ├── sweeps/        <- frames without annotation
#   ├── maps/          <- unused
#   └── v1.0-trainval/ <- metadata

# Create symlink to data directory
mkdir data && cd data
ln -s /path/to/NUSCENES_DATASET_ROOT nuScenes
```

```bash
# Generate info files and ground truth database
python tools/create_data.py nuscenes_data_prep \
    --root_path=data/nuScenes \
    --version="v1.0-trainval" \
    --nsweeps=10
```

--------------------------------

### Run nuScenes 3D Tracking

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Commands for executing center-based tracking on nuScenes detection results with various configuration options.

```bash
python tools/nusc_tracking/pub_test.py \
    --work_dir work_dirs/tracking_result \
    --checkpoint work_dirs/detection_result/infos_val_10sweeps_withvelo_filter_True.json
```

```bash
python tools/nusc_tracking/pub_test.py \
    --work_dir work_dirs/tracking_result \
    --checkpoint work_dirs/detection_result/detection.json \
    --version v1.0-test \
    --root data/nuScenes/v1.0-test
```

```bash
python tools/nusc_tracking/pub_test.py \
    --work_dir work_dirs/tracking_result \
    --checkpoint work_dirs/detection_result/detection.json \
    --hungarian
```

```bash
python tools/nusc_tracking/pub_test.py \
    --work_dir work_dirs/tracking_result \
    --checkpoint work_dirs/detection_result/detection.json \
    --max_age 5
```

--------------------------------

### Build and Run VoxelNet Model

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Programmatic interface for loading configurations, building the detector, and performing forward passes for training or inference.

```python
import torch
from det3d.models import build_detector
from det3d.torchie import Config

# Load model configuration
cfg = Config.fromfile('configs/nusc/voxelnet/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z.py')

# Build VoxelNet detector
model = build_detector(
    cfg.model,
    train_cfg=cfg.train_cfg,
    test_cfg=cfg.test_cfg
)

# Model architecture:
# - reader: VoxelFeatureExtractorV3 (encodes point features within voxels)
# - backbone: SpMiddleResNetFHD (sparse 3D ResNet with 8x downsampling)
# - neck: RPN (feature pyramid network for BEV features)
# - bbox_head: CenterHead (center heatmap + regression heads)

# Forward pass for training
example = {
    'voxels': voxels,           # [N, max_points, num_features]
    'coordinates': coordinates,  # [N, 4] (batch_idx, z, y, x)
    'num_points': num_points,    # [N] points per voxel
    'points': points,            # list of point clouds
    'shape': shape,              # input spatial shape
    'hm': heatmaps,              # ground truth heatmaps
    'anno_box': annotations,     # ground truth boxes
    'ind': indices,              # center indices
    'mask': masks,               # valid object masks
    'cat': categories,           # object categories
}

# Training forward pass (returns loss dict)
loss_dict = model(example, return_loss=True)
# loss_dict contains: 'loss', 'hm_loss', 'loc_loss', 'loc_loss_elem', 'num_positive'

# Inference forward pass (returns detections)
model.eval()
with torch.no_grad():
    detections = model(example, return_loss=False)
# detections: list of dicts with 'box3d_lidar', 'scores', 'label_preds', 'metadata'
```

--------------------------------

### Train VoxelNet Models

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Commands for training, resuming, and auto-scaling learning rates for VoxelNet models.

```bash
python -m torch.distributed.launch --nproc_per_node=4 ./tools/train.py \
    configs/waymo/voxelnet/waymo_centerpoint_voxelnet_3x.py
```

```bash
python -m torch.distributed.launch --nproc_per_node=4 ./tools/train.py \
    configs/nusc/voxelnet/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z.py \
    --resume_from work_dirs/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z/latest.pth
```

```bash
python -m torch.distributed.launch --nproc_per_node=8 ./tools/train.py \
    configs/waymo/voxelnet/waymo_centerpoint_voxelnet_3x.py \
    --autoscale-lr
```

--------------------------------

### Organize nuScenes Dataset

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Organize the nuScenes dataset files in the specified directory structure. Create a symbolic link to your actual dataset root.

```bash
# For nuScenes Dataset         
└── NUSCENES_DATASET_ROOT
       ├── samples       <-- key frames
       ├── sweeps        <-- frames without annotation
       ├── maps          <-- unused
       ├── v1.0-trainval <-- metadata
```

```bash
mkdir data && cd data
ln -s DATA_ROOT 
mv DATA_ROOT nuScenes # rename to nuScenes
```

--------------------------------

### Generate Ground Truth Prediction Bin Files

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/WAYMO.md

Generate ground truth prediction bin files for local evaluation using Waymo's evaluation script. Requires the info path and result path.

```bash
python det3d/datasets/waymo/waymo_common.py --info_path data/Waymo/infos_val_01sweeps_filter_zero_gt.pkl --result_path data/Waymo/ --gt
```

--------------------------------

### Evaluate VoxelNet Models

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Commands for distributed and single-GPU evaluation, including speed testing and test set submission generation.

```bash
python -m torch.distributed.launch --nproc_per_node=4 ./tools/dist_test.py \
    configs/nusc/voxelnet/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z.py \
    --work_dir work_dirs/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z \
    --checkpoint work_dirs/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z/latest.pth
```

```bash
python ./tools/dist_test.py \
    configs/nusc/voxelnet/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z.py \
    --work_dir work_dirs/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z \
    --checkpoint work_dirs/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z/latest.pth \
    --speed_test
```

```bash
python ./tools/dist_test.py \
    configs/nusc/voxelnet/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z_flip.py \
    --work_dir work_dirs/nusc_centerpoint_voxelnet_dcn_0075voxel_flip_testset \
    --checkpoint work_dirs/nusc_0075_flip/voxelnet_converted.pth \
    --testset
```

```bash
python ./tools/dist_test.py \
    configs/waymo/voxelnet/waymo_centerpoint_voxelnet_3x.py \
    --work_dir work_dirs/waymo_centerpoint_voxelnet_3x \
    --checkpoint work_dirs/waymo_centerpoint_voxelnet_3x/latest.pth \
    --testset
```

--------------------------------

### Initialize SpMiddleResNetFHD Backbone

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Initializes the sparse 3D convolutional backbone with specified input features and normalization configuration.

```python
import torch
from det3d.models.backbones.scn import SpMiddleResNetFHD

# Initialize backbone
backbone = SpMiddleResNetFHD(
    num_input_features=5,  # x, y, z, intensity, time
    norm_cfg=dict(type="BN1d", eps=1e-3, momentum=0.01),
)

# Architecture overview:
# conv_input: SubMConv3d(5, 16) + BN + ReLU
# conv1: 2x SparseBasicBlock(16, 16)         -> [1440, 1440, 40]
# conv2: SparseConv3d(16, 32, stride=2) + 2x SparseBasicBlock -> [720, 720, 20]
# conv3: SparseConv3d(32, 64, stride=2) + 2x SparseBasicBlock -> [360, 360, 10]
# conv4: SparseConv3d(64, 128, stride=2) + 2x SparseBasicBlock -> [180, 180, 5]
# extra_conv: SparseConv3d(128, 128) -> [180, 180, 2]
# Output: Dense tensor [B, 256, 180, 180] (2*128 channels after flattening z)
```

--------------------------------

### Configure and Use CenterHead

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Initializes the CenterHead detection module with specific task groupings and regression parameters. Provides methods for loss calculation and inference.

```python
import torch
from det3d.models.bbox_heads.center_head import CenterHead

# Define detection tasks (class groupings)
tasks = [
    dict(num_class=1, class_names=["car"]),
    dict(num_class=2, class_names=["truck", "construction_vehicle"]),
    dict(num_class=2, class_names=["bus", "trailer"]),
    dict(num_class=1, class_names=["barrier"]),
    dict(num_class=2, class_names=["motorcycle", "bicycle"]),
    dict(num_class=2, class_names=["pedestrian", "traffic_cone"]),
]

# Initialize CenterHead
head = CenterHead(
    in_channels=512,                    # BEV feature channels
    tasks=tasks,                        # Detection tasks
    dataset='nuscenes',
    weight=0.25,                        # Loss weight for regression
    code_weights=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.2, 0.2, 1.0, 1.0],  # Per-attribute weights
    common_heads={
        'reg': (2, 2),    # xy offset regression (channels, num_conv)
        'height': (1, 2), # z height regression
        'dim': (3, 2),    # lwh dimension regression
        'rot': (2, 2),    # sin/cos rotation regression
        'vel': (2, 2),    # velocity regression (nuScenes)
    },
    share_conv_channel=64,
    dcn_head=False,  # Use deformable convolution
)

# Forward pass
bev_features = torch.randn(2, 512, 180, 180)  # [B, C, H, W]
predictions, shared_features = head(bev_features)

# predictions is a list of dicts (one per task), each containing:
# - 'hm': [B, num_classes, H, W] center heatmap
# - 'reg': [B, 2, H, W] xy offset from cell center
# - 'height': [B, 1, H, W] object height
# - 'dim': [B, 3, H, W] object dimensions (log scale)
# - 'rot': [B, 2, H, W] rotation (sin, cos)
# - 'vel': [B, 2, H, W] velocity (if enabled)

# Compute loss during training
loss_dict = head.loss(example, predictions, test_cfg)

# Run prediction during inference
detections = head.predict(example, predictions, test_cfg)
```

--------------------------------

### Manage NuScenes Dataset

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Builds the dataset configuration and executes evaluation on detection results.

```python
from det3d.datasets.nuscenes.nuscenes import NuScenesDataset
from det3d.datasets import build_dataset

# Build dataset from configuration
dataset_cfg = dict(
    type="NuScenesDataset",
    root_path="data/nuScenes",
    info_path="data/nuScenes/infos_val_10sweeps_withvelo_filter_True.pkl",
    test_mode=True,
    nsweeps=10,
    class_names=[
        'car', 'truck', 'construction_vehicle', 'bus', 'trailer',
        'barrier', 'motorcycle', 'bicycle', 'pedestrian', 'traffic_cone'
    ],
    pipeline=test_pipeline,
    version="v1.0-trainval",
)

dataset = build_dataset(dataset_cfg)

# Get sample
sample = dataset[0]
# Returns dict with: 'voxels', 'coordinates', 'num_points',
#                    'points', 'shape', 'metadata', etc.

# Evaluate detections
predictions = {
    'sample_token_1': {
        'box3d_lidar': boxes,      # [N, 9]
        'scores': scores,           # [N]
        'label_preds': labels,      # [N]
        'metadata': {'token': 'sample_token_1'}
    },
    # ... more samples
}

results, _ = dataset.evaluation(
    predictions,
    output_dir='work_dirs/eval_results',
    testset=False  # True for test set submission
)

# Results contain mAP and NDS metrics
print(results['results']['nusc'])
```

--------------------------------

### nuScenes Tracking Validation Set

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Run tracking on the validation set after downloading detection files. Specify the work directory and checkpoint path.

```bash
# val set 
python tools/nusc_tracking/pub_test.py --work_dir WORK_DIR_PATH  --checkpoint DETECTION_PATH  
```

--------------------------------

### Configure Test Settings

Source: https://context7.com/tianweiy/centerpoint/llms.txt

Define NMS and post-processing parameters for model evaluation. Adjust thresholds and ranges based on the specific dataset and performance requirements.

```python
# nuScenes test configuration
test_cfg = dict(
    # Limit detection range
    post_center_limit_range=[-61.2, -61.2, -10.0, 61.2, 61.2, 10.0],
    max_per_img=500,                    # Max detections per sample

    # NMS settings
    nms=dict(
        use_rotate_nms=True,            # Use rotated NMS for oriented boxes
        use_multi_class_nms=False,      # Single NMS across all classes
        nms_pre_max_size=1000,          # Keep top-K before NMS
        nms_post_max_size=83,           # Keep top-K after NMS
        nms_iou_threshold=0.2,          # IoU threshold for suppression
    ),

    score_threshold=0.1,                # Minimum confidence score
    pc_range=[-54, -54],               # Point cloud range (xy)
    out_size_factor=8,                  # Downsample factor
    voxel_size=[0.075, 0.075],         # Voxel size for coordinate conversion

    # Optional: circular NMS for specific classes
    circular_nms=False,
    min_radius=[4, 12, 10, 1, 0.85, 0.175],  # Per-task min radius

    # Optional: double flip test-time augmentation
    double_flip=False,
)

# Waymo test configuration (different range and parameters)
waymo_test_cfg = dict(
    post_center_limit_range=[-80, -80, -10.0, 80, 80, 10.0],
    max_per_img=500,
    nms=dict(
        use_rotate_nms=True,
        use_multi_class_nms=False,
        nms_pre_max_size=4096,
        nms_post_max_size=500,
        nms_iou_threshold=0.7,
    ),
    score_threshold=0.1,
    pc_range=[-75.2, -75.2],
    out_size_factor=8,
    voxel_size=[0.1, 0.1],
)
```

--------------------------------

### Single GPU Testing and Inference Time

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Test with a single GPU and measure inference time. This command is useful for performance analysis.

```bash
python ./tools/dist_test.py CONFIG_PATH --work_dir work_dirs/CONFIG_NAME --checkpoint work_dirs/CONFIG_NAME/latest.pth --speed_test 
```

--------------------------------

### Distributed Testing Command

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/WAYMO.md

Perform distributed testing with 4 GPUs. Specify the configuration path, work directory, and checkpoint.

```bash
python -m torch.distributed.launch --nproc_per_node=4 ./tools/dist_test.py CONFIG_PATH --work_dir work_dirs/CONFIG_NAME --checkpoint work_dirs/CONFIG_NAME/latest.pth
```

--------------------------------

### Single GPU Testing and Inference Time Measurement

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/WAYMO.md

Test with a single GPU and measure inference time. Generates a 'my_preds.bin' file for submission.

```bash
python ./tools/dist_test.py CONFIG_PATH --work_dir work_dirs/CONFIG_NAME --checkpoint work_dirs/CONFIG_NAME/latest.pth --speed_test
```

--------------------------------

### Convert Waymo TFRecord Data to Pickle

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/WAYMO.md

Convert Waymo dataset TFRecord files to pickle format for training, validation, and testing sets. Ensure WAYMO_DATASET_ROOT is set correctly.

```bash
# train set 
CUDA_VISIBLE_DEVICES=-1 python det3d/datasets/waymo/waymo_converter.py --record_path 'WAYMO_DATASET_ROOT/tfrecord_training/*.tfrecord'  --root_path 'WAYMO_DATASET_ROOT/train/'

# validation set 
CUDA_VISIBLE_DEVICES=-1 python det3d/datasets/waymo/waymo_converter.py --record_path 'WAYMO_DATASET_ROOT/tfrecord_validation/*.tfrecord'  --root_path 'WAYMO_DATASET_ROOT/val/'

# testing set 
CUDA_VISIBLE_DEVICES=-1 python det3d/datasets/waymo/waymo_converter.py --record_path 'WAYMO_DATASET_ROOT/tfrecord_testing/*.tfrecord'  --root_path 'WAYMO_DATASET_ROOT/test/'
```

--------------------------------

### Distributed Testing

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Perform distributed testing with multiple GPUs. Specify the configuration path, work directory, and checkpoint.

```bash
python -m torch.distributed.launch --nproc_per_node=4 ./tools/dist_test.py CONFIG_PATH --work_dir work_dirs/CONFIG_NAME --checkpoint work_dirs/CONFIG_NAME/latest.pth 
```

--------------------------------

### Waymo 3D Tracking Shell Scripts

Source: https://github.com/tianweiy/centerpoint/blob/master/configs/waymo/README.md

Shell scripts used for executing center-based tracking on top of detection results.

```bash
../../tracking_scripts/centerpoint_voxel_two_sweep_val.sh
```

```bash
../../tracking_scripts/centerpoint_voxel_two_sweep_test.sh
```

--------------------------------

### Run Test Set Evaluation

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/WAYMO.md

Execute the distribution test script with the test set flag enabled.

```bash
python ./tools/dist_test.py CONFIG_PATH --work_dir work_dirs/CONFIG_NAME --checkpoint work_dirs/CONFIG_NAME/latest.pth --testset
```

--------------------------------

### Compile Deformable Convolution CUDA Extension

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/INSTALL.md

Builds the Deformable Convolution CUDA extension. This is optional and may only work with older PyTorch versions (e.g., 1.1).

```bash
# Deformable Convolution (Optional and only works with old torch versions e.g. 1.1)
cd ROOT_DIR/det3d/ops/dcn
python setup.py build_ext --inplace
```

--------------------------------

### nuScenes Tracking Test Set

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Run tracking on the test set. Requires specifying the version and root path for the test data.

```bash
# test set 
python tools/nusc_tracking/pub_test.py --work_dir WORK_DIR_PATH  --checkpoint DETECTION_PATH  --version v1.0-test  --root data/nuScenes/v1.0-test    
```

--------------------------------

### Compile Rotated NMS CUDA Extension

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/INSTALL.md

Builds the Rotated Non-Maximum Suppression CUDA extension. Ensure you are in the correct directory and CUDA is configured.

```bash
# Rotated NMS 
cd ROOT_DIR/det3d/ops/iou3d_nms
python setup.py build_ext --inplace
```

--------------------------------

### Organized CenterPoint Data Structure

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

The expected directory structure for CenterPoint data and generated info files after preparation.

```bash
# For nuScenes Dataset 
└── CenterPoint
       └── data    
              └── nuScenes 
                     ├── samples       <-- key frames
                     ├── sweeps        <-- frames without annotation
                     ├── maps          <-- unused
                     |── v1.0-trainval <-- metadata and annotations
                     |── infos_train_10sweeps_withvelo_filter_True.pkl <-- train annotations
                     |── infos_val_10sweeps_withvelo_filter_True.pkl <-- val annotations
                     |── dbinfos_train_10sweeps_withvelo.pkl <-- GT database info files
                     |── gt_database_10sweeps_withvelo <-- GT database 
```

--------------------------------

### Test Set Data Organization

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Directory structure for the nuScenes test set, including main test folder and info files.

```bash
# For nuScenes Dataset 
└── CenterPoint
       └── data    
              └── nuScenes 
                     ├── samples       <-- key frames
                     ├── sweeps        <-- frames without annotation
                     ├── maps          <-- unused
                     |── v1.0-trainval <-- metadata and annotations
                     |── infos_train_10sweeps_withvelo_filter_True.pkl <-- train annotations
                     |── infos_val_10sweeps_withvelo_filter_True.pkl <-- val annotations
                     |── dbinfos_train_10sweeps_withvelo.pkl <-- GT database info files
                     |── gt_database_10sweeps_withvelo <-- GT database 
                     └── v1.0-test <-- main test folder 
                            ├── samples       <-- key frames
                            ├── sweeps        <-- frames without annotation
                            ├── maps          <-- unused
                            |── v1.0-test <-- metadata and annotations
                            |── infos_test_10sweeps_withvelo.pkl <-- test info
```

--------------------------------

### Test Set Prediction

Source: https://github.com/tianweiy/centerpoint/blob/master/docs/NUSC.md

Generate detection predictions for the test set using a pre-trained checkpoint. Ensure the checkpoint is placed in the correct directory.

```bash
python tools/dist_test.py configs/nusc/voxelnet/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z_flip.py --work_dir work_dirs/nusc_centerpoint_voxelnet_dcn_0075voxel_flip_testset  --checkpoint work_dirs/nusc_0075_flip/voxelnet_converted.pth  --testset 
```