MMDetection3D

https://mmdetection3d.readthedocs.io/en/latest/
Admin
MMDetection3D is an open-source 3D object detection toolbox built on OpenMMLab's foundational...
Tokens:107,553
Snippets:797
Trust Score:9.6
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install Core Dependencies

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Install MMEngine, MMCV, and MMDetection using MIM.

```bash
pip install -U openmim
mim install mmengine
mim install 'mmcv>=2.0.0rc4'
mim install 'mmdet>=3.0.0'
```

--------------------------------

### Setup Multi-Processing Environment

Source: https://mmdetection3d.readthedocs.io/en/latest/_modules/mmdet3d/utils/setup_env.html

Configures multi-processing start methods, OpenCV thread counts, and environment variables for OMP and MKL to prevent system overload.

```python
def setup_multi_processes(cfg):
    """Setup multi-processing environment variables."""
    # set multi-process start method as `fork` to speed up the training
    if platform.system() != 'Windows':
        mp_start_method = cfg.get('mp_start_method', 'fork')
        current_method = mp.get_start_method(allow_none=True)
        if current_method is not None and current_method != mp_start_method:
            warnings.warn(
                f'Multi-processing start method `{mp_start_method}` is '
                f'different from the previous setting `{current_method}`.'
                f'It will be force set to `{mp_start_method}`. You can change '
                f'this behavior by changing `mp_start_method` in your config.')
        mp.set_start_method(mp_start_method, force=True)

    # disable opencv multithreading to avoid system being overloaded
    opencv_num_threads = cfg.get('opencv_num_threads', 0)
    cv2.setNumThreads(opencv_num_threads)

    # setup OMP threads
    # This code is referred from https://github.com/pytorch/pytorch/blob/master/torch/distributed/run.py  # noqa
    workers_per_gpu = cfg.data.get('workers_per_gpu', 1)
    if 'train_dataloader' in cfg.data:
        workers_per_gpu = \
            max(cfg.data.train_dataloader.get('workers_per_gpu', 1),
                workers_per_gpu)

    if 'OMP_NUM_THREADS' not in os.environ and workers_per_gpu > 1:
        omp_num_threads = 1
        warnings.warn(
            f'Setting OMP_NUM_THREADS environment variable for each process '
            f'to be {omp_num_threads} in default, to avoid your system being '
            f'overloaded, please further tune the variable for optimal '
            f'performance in your application as needed.')
        os.environ['OMP_NUM_THREADS'] = str(omp_num_threads)

    # setup MKL threads
    if 'MKL_NUM_THREADS' not in os.environ and workers_per_gpu > 1:
        mkl_num_threads = 1
        warnings.warn(
            f'Setting MKL_NUM_THREADS environment variable for each process '
            f'to be {mkl_num_threads} in default, to avoid your system being '
            f'overloaded, please further tune the variable for optimal '
            f'performance in your application as needed.')
        os.environ['MKL_NUM_THREADS'] = str(mkl_num_threads)
```

--------------------------------

### Install Torchsparse

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Install Torchsparse as a sparse convolution backend.

```bash
sudo apt-get install libsparsehash-dev
pip install --upgrade git+https://github.com/mit-han-lab/torchsparse.git@v1.4.0
```

```bash
conda install -c bioconda sparsehash
export CPLUS_INCLUDE_PATH=CPLUS_INCLUDE_PATH:${YOUR_CONDA_ENVS_DIR}/include
# replace ${YOUR_CONDA_ENVS_DIR} to your anaconda environment path e.g. `/home/username/anaconda3/envs/openmmlab`.
pip install --upgrade git+https://github.com/mit-han-lab/torchsparse.git@v1.4.0
```

--------------------------------

### Install MMDetection3D from Source

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Install in editable mode for development purposes.

```bash
git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
# "-b dev-1.x" means checkout to the `dev-1.x` branch.
cd mmdetection3d
pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in edtiable mode,
# thus any local modifications made to the code will take effect without reinstallation.
```

--------------------------------

### Install pre-commit hooks

Source: https://mmdetection3d.readthedocs.io/en/latest/notes/contribution_guides.html

Install and initialize pre-commit hooks to ensure code style compliance before committing.

```bash
pip install -U pre-commit
pre-commit install
```

--------------------------------

### Install Minkowski Engine

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Install Minkowski Engine as a sparse convolution backend.

```bash
conda install openblas-devel -c anaconda
export CPLUS_INCLUDE_PATH=CPLUS_INCLUDE_PATH:${YOUR_CONDA_ENVS_DIR}/include
# replace ${YOUR_CONDA_ENVS_DIR} to your anaconda environment path e.g. `/home/username/anaconda3/envs/openmmlab`.
pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=/opt/conda/include" --install-option="--blas=openblas"
```

--------------------------------

### Install MMDetection3D from Source on Google Colab

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Clones the repository and installs it in editable mode.

```bash
!git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
%cd mmdetection3d
!pip install -e .

```

--------------------------------

### Configure Learning Rate Schedules

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html

Examples of setting up Poly and CosineAnnealing learning rate schedules in the configuration.

```python
param_scheduler = [
    dict(
        type='PolyLR',
        power=0.9,
        eta_min=1e-4,
        begin=0,
        end=8,
        by_epoch=True)]
```

```python
param_scheduler = [
    dict(
        type='CosineAnnealingLR',
        T_max=8,
        eta_min=lr * 1e-5,
        begin=0,
        end=8,
        by_epoch=True)]
```

--------------------------------

### Install spconv 2.0

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Install spconv 2.0 for optimized GPU memory usage.

```bash
pip install cumm-cuxxx
pip install spconv-cuxxx
```

--------------------------------

### Install MMDetection3D via MIM

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Install as a third-party package dependency.

```bash
mim install "mmdet3d>=1.1.0"
```

--------------------------------

### Clone and configure upstream repository

Source: https://mmdetection3d.readthedocs.io/en/latest/notes/contribution_guides.html

Initial setup steps for first-time contributors to fork, clone, and link the upstream repository.

```bash
git clone git@github.com:XXX/mmdetection3d.git
```

```bash
git remote add upstream git@github.com:open-mmlab/mmdetection3d
```

--------------------------------

### Install pypcd library

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_dataset.html

Command to install the pypcd library for handling PCD files.

```bash
pip install git+https://github.com/DanielPollithy/pypcd.git
```

--------------------------------

### Run mmdet3d-serve container

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/useful_tools.html

Starts the TorchServe container. Requires nvidia-docker for GPU support.

```bash
docker run --rm \
--cpus 8 \
--gpus device=0 \
-p8080:8080 -p8081:8081 -p8082:8082 \
--mount type=bind,source=$MODEL_STORE,target=/home/model-server/model-store \
mmdet3d-serve:latest
```

--------------------------------

### setup_multi_processes(cfg)

Source: https://mmdetection3d.readthedocs.io/en/latest/_modules/mmdet3d/utils/setup_env.html

Configures multi-processing environment variables, including start methods, OpenCV threading, and OMP/MKL thread limits to prevent system overload.

```APIDOC
## setup_multi_processes(cfg)

### Description
Sets up multi-processing environment variables based on the provided configuration. It manages the process start method, limits OpenCV threads, and sets OMP/MKL thread counts to optimize performance and prevent system overloading.

### Parameters
- **cfg** (dict) - Configuration object containing settings like `mp_start_method`, `opencv_num_threads`, and data loader worker counts.
```

--------------------------------

### Install MMEngine via pip

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Standard command to install MMEngine using pip without MIM.

```bash
pip install mmengine

```

--------------------------------

### Configure Custom Hooks

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html

Examples of adding custom hooks to the configuration with optional priority settings.

```python
custom_hooks = [
    dict(type='MyHook', a=a_value, b=b_value)
]
```

```python
custom_hooks = [
    dict(type='MyHook', a=a_value, b=b_value, priority='NORMAL')
]
```

--------------------------------

### Install PyTorch

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Installs PyTorch and torchvision dependencies for either GPU or CPU environments.

```bash
conda install pytorch torchvision -c pytorch
```

```bash
conda install pytorch torchvision cpuonly -c pytorch
```

--------------------------------

### Configure Training Loops

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html

Examples of defining standard epoch-based training loops and dynamic iteration-based training intervals.

```python
train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=12, val_begin=1, val_interval=1)
```

```python
# Before 365001th iteration, we do evaluation every 5000 iterations.
# After 365000th iteration, we do evaluation every 368750 iterations,
# which means that we do evaluation at the end of training.

interval = 5000
max_iters = 368750
dynamic_intervals = [(max_iters // interval * interval + 1, max_iters)]
train_cfg = dict(
    type='IterBasedTrainLoop',
    max_iters=max_iters,
    val_interval=interval,
    dynamic_intervals=dynamic_intervals)
```

--------------------------------

### Install Waymo Dataset Dependencies

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/waymo.html

Install the official Waymo Open Dataset package or optional requirements.

```bash
pip install waymo-open-dataset-tf-2-6-0
```

```bash
pip install -r requirements/optional.txt
```

--------------------------------

### Install pycocotools from source

Source: https://mmdetection3d.readthedocs.io/en/latest/notes/faq.html

Compile and install the latest pycocotools from source to resolve binary incompatibility errors with numpy.

```bash
pip install -e "git+https://github.com/cocodataset/cocoapi#egg=pycocotools&subdirectory=PythonAPI"
```

```bash
pip install -e "git+https://github.com/ppwwyyxx/cocoapi#egg=pycocotools&subdirectory=PythonAPI"
```

--------------------------------

### Configure PartAggregationROIHead

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_models.html

Example configuration for the PartAggregationROIHead, including semantic and bounding box head definitions.

```python
model = dict(
    ...
    roi_head=dict(
        type='PartAggregationROIHead',
        num_classes=3,
        semantic_head=dict(
            type='PointwiseSemanticHead',
            in_channels=16,
            extra_width=0.2,
            seg_score_thr=0.3,
            num_classes=3,
            loss_seg=dict(
                type='mmdet.FocalLoss',
                use_sigmoid=True,
                reduction='sum',
                gamma=2.0,
                alpha=0.25,
                loss_weight=1.0),
            loss_part=dict(
                type='mmdet.CrossEntropyLoss',
                use_sigmoid=True,
                loss_weight=1.0)),
        seg_roi_extractor=dict(
            type='Single3DRoIAwareExtractor',
            roi_layer=dict(
                type='RoIAwarePool3d',
                out_size=14,
                max_pts_per_voxel=128,
                mode='max')),
        bbox_roi_extractor=dict(
            type='Single3DRoIAwareExtractor',
            roi_layer=dict(
                type='RoIAwarePool3d',
                out_size=14,
                max_pts_per_voxel=128,
                mode='avg')),
        bbox_head=dict(
            type='PartA2BboxHead',
            num_classes=3,
            seg_in_channels=16,
            part_in_channels=4,
            seg_conv_channels=[64, 64],
            part_conv_channels=[64, 64],
            merge_conv_channels=[128, 128],
            down_conv_channels=[128, 256],
            bbox_coder=dict(type='DeltaXYZWLHRBBoxCoder'),
            shared_fc_channels=[256, 512, 512, 512],
            cls_channels=[256, 256],
            reg_channels=[256, 256],
            dropout_ratio=0.1,
            roi_feat_size=14,
            with_corner_loss=True,
            loss_bbox=dict(
                type='mmdet.SmoothL1Loss',
                beta=1.0 / 9.0,
                reduction='sum',
                loss_weight=1.0),
            loss_cls=dict(
                type='mmdet.CrossEntropyLoss',
                use_sigmoid=True,
                reduction='sum',
                loss_weight=1.0))),
    ...
)
```

--------------------------------

### Build Waymo evaluation binary

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/waymo.html

Commands to clone the Waymo repository, install Bazel, and compile the detection metrics tool.

```bash
# download the code and enter the base directory
git clone https://github.com/waymo-research/waymo-open-dataset.git waymo-od
# git clone https://github.com/Abyssaledge/waymo-open-dataset-master waymo-od # if you want to use faster multi-thread version.
cd waymo-od
git checkout remotes/origin/master

# use the Bazel build system
sudo apt-get install --assume-yes pkg-config zip g++ zlib1g-dev unzip python3 python3-pip
BAZEL_VERSION=3.1.0
wget https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh
sudo bash bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh
sudo apt install build-essential

# configure .bazelrc
./configure.sh
# delete previous bazel outputs and reset internal caches
bazel clean

bazel build waymo_open_dataset/metrics/tools/compute_detection_metrics_main
cp bazel-bin/waymo_open_dataset/metrics/tools/compute_detection_metrics_main ../mmdetection3d/mmdet3d/evaluation/functional/waymo_utils/
```

--------------------------------

### Evaluate PointPillars on Waymo

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/waymo.html

Example command to run distributed testing for PointPillars on the Waymo dataset.

```bash
./tools/dist_test.sh configs/pointpillars/pointpillars_hv_secfpn_sbn-all_16xb2-2x_waymo-3d-car.py checkpoints/hv_pointpillars_secfpn_sbn-2x16_2x_waymo-3d-car_latest.pth
```

--------------------------------

### Configure PointPillars Model Parameters

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_dataset.html

Example configuration for a PointPillars model, highlighting parameters like voxel_size, point_cloud_range, and anchor settings that require adjustment for custom datasets.

```python
voxel_size = [0.16, 0.16, 4]  # adjust according to your dataset
point_cloud_range = [0, -39.68, -3, 69.12, 39.68, 1]  # adjust according to your dataset
model = dict(
    type='VoxelNet',
    data_preprocessor=dict(
        type='Det3DDataPreprocessor',
        voxel=True,
        voxel_layer=dict(
            max_num_points=32,
            point_cloud_range=point_cloud_range,
            voxel_size=voxel_size,
            max_voxels=(16000, 40000))),
    voxel_encoder=dict(
        type='PillarFeatureNet',
        in_channels=4,
        feat_channels=[64],
        with_distance=False,
        voxel_size=voxel_size,
        point_cloud_range=point_cloud_range),
    # the `output_shape` should be adjusted according to `point_cloud_range`
    # and `voxel_size`
    middle_encoder=dict(
        type='PointPillarsScatter', in_channels=64, output_shape=[496, 432]),
    backbone=dict(
        type='SECOND',
        in_channels=64,
        layer_nums=[3, 5, 5],
        layer_strides=[2, 2, 2],
        out_channels=[64, 128, 256]),
    neck=dict(
        type='SECONDFPN',
        in_channels=[64, 128, 256],
        upsample_strides=[1, 2, 4],
        out_channels=[128, 128, 128]),
    bbox_head=dict(
        type='Anchor3DHead',
        num_classes=3,
        in_channels=384,
        feat_channels=384,
        use_direction_classifier=True,
        assign_per_class=True,
        # adjust the `ranges` and `sizes` according to your dataset
        anchor_generator=dict(
            type='AlignedAnchor3DRangeGenerator',
            ranges=[
                [0, -39.68, -0.6, 69.12, 39.68, -0.6],
                [0, -39.68, -0.6, 69.12, 39.68, -0.6],
                [0, -39.68, -1.78, 69.12, 39.68, -1.78],
            ],
            sizes=[[0.8, 0.6, 1.73], [1.76, 0.6, 1.73], [3.9, 1.6, 1.56]],
            rotations=[0, 1.57],
            reshape_out=False),
        diff_rad_by_sin=True,
        bbox_coder=dict(type='DeltaXYZWLHRBBoxCoder'),
        loss_cls=dict(
            type='mmdet.FocalLoss',
            use_sigmoid=True,
            gamma=2.0,
            alpha=0.25,
            loss_weight=1.0),
        loss_bbox=dict(
            type='mmdet.SmoothL1Loss', beta=1.0 / 9.0, loss_weight=2.0),
        loss_dir=dict(
            type='mmdet.CrossEntropyLoss', use_sigmoid=False,
            loss_weight=0.2)),
    # model training and testing settings
    train_cfg=dict(
        assigner=[
            dict(  # for Pedestrian
                type='Max3DIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.5,
                neg_iou_thr=0.35,
                min_pos_iou=0.35,
                ignore_iof_thr=-1),
            dict(  # for Cyclist
                type='Max3DIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.5,
                neg_iou_thr=0.35,
                min_pos_iou=0.35,
                ignore_iof_thr=-1),
            dict(  # for Car
                type='Max3DIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.6,
                neg_iou_thr=0.45,
                min_pos_iou=0.45,
                ignore_iof_thr=-1),
        ],
        allowed_border=0,
        pos_weight=-1,
        debug=False),
    test_cfg=dict(
        use_rotate_nms=True,
        nms_across_levels=False,
        nms_thr=0.01,
        score_thr=0.1,
        min_bbox_size=0,
        nms_pre=100,
        max_num=50))
```

--------------------------------

### Test models using Slurm

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/train_test.html

Examples for testing models on clusters using Slurm with various evaluation output configurations.

```bash
./tools/slurm_test.sh ${PARTITION} ${JOB_NAME} configs/second/second_hv_secfpn_8xb6-80e_kitti-3d-3class.py \
    checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-3class_20200620_230238-9208083a.pth
```

```bash
./tools/slurm_test.sh ${PARTITION} ${JOB_NAME} configs/pointpillars/pointpillars_hv_secfpn_sbn-all_8xb4-2x_nus-3d.py \
    checkpoints/hv_pointpillars_fpn_sbn-all_4x8_2x_nus-3d_20200620_230405-2fa62f3d.pth \
   --cfg-options 'test_evaluator.jsonfile_prefix=./pointpillars_nuscenes_results'
```

```bash
./tools/slurm_test.sh ${PARTITION} ${JOB_NAME} configs/second/second_hv_secfpn_8xb6-80e_kitti-3d-3class.py \
    checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-3class_20200620_230238-9208083a.pth \
    --cfg-options 'test_evaluator.pklfile_prefix=./second_kitti_results' 'submission_prefix=./second_kitti_results'
```

--------------------------------

### Use PyTorch Optimizers

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html

Example of configuring the optimizer field within optim_wrapper to use a standard PyTorch Adam optimizer.

```python
optim_wrapper = dict(
    type='OptimWrapper',
    optimizer=dict(type='Adam', lr=0.0003, weight_decay=0.0001))
```

--------------------------------

### Update Data Infos to v2

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/dataset_prepare.html

Convert legacy data info files to the v1.1.0 format. The second example demonstrates updating KITTI dataset info files.

```bash
python tools/dataset_converters/update_infos_to_v2.py --dataset ${DATA_SET} --pkl-path ${PKL_PATH} --out-dir ${OUT_DIR}
```

```bash
python tools/dataset_converters/update_infos_to_v2.py --dataset kitti --pkl-path ./data/kitti/kitti_infos_trainval.pkl --out-dir ./data/kitti
```

--------------------------------

### Verify MMDetection3D Installation

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Checks the installed version of the mmdet3d package.

```python
import mmdet3d
print(mmdet3d.__version__)
# Example output: 1.1.0, or an another version.

```

--------------------------------

### Prepare KITTI Data Splits

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/dataset_prepare.html

Create necessary directories and download data split text files for KITTI.

```bash
mkdir ./data/kitti/ && mkdir ./data/kitti/ImageSets

# Download data split
wget -c  https://raw.githubusercontent.com/traveller59/second.pytorch/master/second/data/ImageSets/test.txt --no-check-certificate --content-disposition -O ./data/kitti/ImageSets/test.txt
wget -c  https://raw.githubusercontent.com/traveller59/second.pytorch/master/second/data/ImageSets/train.txt --no-check-certificate --content-disposition -O ./data/kitti/ImageSets/train.txt
wget -c  https://raw.githubusercontent.com/traveller59/second.pytorch/master/second/data/ImageSets/val.txt --no-check-certificate --content-disposition -O ./data/kitti/ImageSets/val.txt
wget -c  https://raw.githubusercontent.com/traveller59/second.pytorch/master/second/data/ImageSets/trainval.txt --no-check-certificate --content-disposition -O ./data/kitti/ImageSets/trainval.txt
```

--------------------------------

### mmdet3d.utils.setup_multi_processes(_cfg_)

Source: https://mmdetection3d.readthedocs.io/en/latest/api.html

Sets up multi-processing environment variables.

```APIDOC
## mmdet3d.utils.setup_multi_processes(_cfg_)

### Description
Setup multi-processing environment variables.
```

--------------------------------

### Run Inference Demo from Source

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Executes the point cloud inference demo using the downloaded model files.

```bash
python demo/pcd_demo.py demo/data/kitti/000008.bin pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth --show
```

--------------------------------

### Download MMDetection3D Config and Checkpoint

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Downloads the necessary configuration and model weights for the PointPillars demo.

```bash
mim download mmdet3d --config pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car --dest .
```

--------------------------------

### Create KITTI Dataset and Download Splits

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/kitti.html

Commands to download data splits and run the creation script to generate ground truth databases and info files.

```bash
mkdir ./data/kitti/ && mkdir ./data/kitti/ImageSets

# Download data split
wget -c  https://raw.githubusercontent.com/traveller59/second.pytorch/master/second/data/ImageSets/test.txt --no-check-certificate --content-disposition -O ./data/kitti/ImageSets/test.txt
wget -c  https://raw.githubusercontent.com/traveller59/second.pytorch/master/second/data/ImageSets/train.txt --no-check-certificate --content-disposition -O ./data/kitti/ImageSets/train.txt
wget -c  https://raw.githubusercontent.com/traveller59/second.pytorch/master/second/data/ImageSets/val.txt --no-check-certificate --content-disposition -O ./data/kitti/ImageSets/val.txt
wget -c  https://raw.githubusercontent.com/traveller59/second.pytorch/master/second/data/ImageSets/trainval.txt --no-check-certificate --content-disposition -O ./data/kitti/ImageSets/trainval.txt

python tools/create_data.py kitti --root-path ./data/kitti --out-dir ./data/kitti --extra-tag kitti --with-plane
```

--------------------------------

### Install MMDetection3D Dependencies on Google Colab

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Uses MIM to install core OpenMMLab packages in a Google Colab environment.

```bash
!pip3 install openmim
!mim install mmengine
!mim install "mmcv>=2.0.0rc4,<2.1.0"
!mim install "mmdet>=3.0.0,<3.1.0"

```

--------------------------------

### Download KITTI via OpenDataLab

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/dataset_prepare.html

Use OpenDataLab CLI tools and MIM to download and preprocess the KITTI dataset.

```bash
# install OpenDataLab CLI tools
pip install -U opendatalab
# log in OpenDataLab. Note that you should register an account on [OpenDataLab](https://opendatalab.com/) before.
pip install odl
odl login
# download and preprocess by MIM
mim download mmdet3d --dataset kitti
```

--------------------------------

### Install MMCV via pip

Source: https://mmdetection3d.readthedocs.io/en/latest/get_started.html

Installs a specific version of MMCV by manually specifying the find-url based on PyTorch and CUDA versions.

```bash
pip install "mmcv>=2.0.0rc4" -f https://download.openmmlab.com/mmcv/dist/cu116/torch1.12.0/index.html

```

--------------------------------

### Use Built-in Hooks

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html

Example of configuring an existing MMDetection3D hook.

```python
custom_hooks = [dict(type='DisableObjectSampleHook', disable_after_epoch=15)]
```

--------------------------------

### Implement PartAggregationROIHead

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_models.html

Example of a specific RoI head implementation inheriting from Base3DRoIHead.

```python
from typing import Dict, List, Tuple

from mmdet.models.task_modules import AssignResult, SamplingResult
from mmengine import ConfigDict
from torch import Tensor
from torch.nn import functional as F

from mmdet3d.registry import MODELS
from mmdet3d.structures import bbox3d2roi
from mmdet3d.utils import InstanceList
from ...structures.det3d_data_sample import SampleList
from .base_3droi_head import Base3DRoIHead


@MODELS.register_module()
class PartAggregationROIHead(Base3DRoIHead):
    """Part aggregation roi head for PartA2."""
```

--------------------------------

### Test VoteNet on ScanNet

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/train_test.html

Examples for testing VoteNet with various visualization and evaluation settings.

```bash
python tools/test.py configs/votenet/votenet_8xb8_scannet-3d.py \
    checkpoints/votenet_8x8_scannet-3d-18class_20200620_230238-2cea9c3a.pth \
    --show --show-dir ./data/scannet/show_results
```

```bash
python tools/test.py configs/votenet/votenet_8xb8_scannet-3d.py \
    checkpoints/votenet_8x8_scannet-3d-18class_20200620_230238-2cea9c3a.pth \
    --show --show-dir ./data/scannet/show_results
```

```bash
python tools/test.py configs/votenet/votenet_8xb8_scannet-3d.py \
    checkpoints/votenet_8x8_scannet-3d-18class_20200620_230238-2cea9c3a.pth
```

--------------------------------

### Test and inference

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/new_data_model.html

Run the testing script with a configuration file and a pre-trained model checkpoint.

```bash
python tools/test.py configs/pointpillars/pointpillars_hv_secfpn_sbn-all_16xb2-2x_waymoD5-3d-3class.py work_dirs/pointpillars_hv_secfpn_sbn-all_16xb2-2x_waymoD5-3d-3class/latest.pth
```

--------------------------------

### Configure Det3DVisualizationHook and backends

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html

Enables visualization and configures multiple backends like Local and Tensorboard.

```python
default_hooks = dict(
    visualization=dict(type='Det3DVisualizationHook', draw=True))

vis_backends = [dict(type='LocalVisBackend'),
                dict(type='TensorboardVisBackend')]
visualizer = dict(
    type='Det3DLocalVisualizer', vis_backends=vis_backends, name='visualizer')
```

--------------------------------

### Config Inheritance via _base_

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/config.html

Demonstrates inheriting from a single file or a list of multiple configuration files.

```python
_base_ = './pointpillars_hv_secfpn_8xb6-160e_kitti-3d-3class.py'
```

```python
_base_ = [
    '../_base_/models/pointpillars_hv_secfpn_kitti.py',
    '../_base_/datasets/kitti-3d-3class.py',
    '../_base_/schedules/cyclic-40e.py', '../_base_/default_runtime.py'
]
```

--------------------------------

### Build mmdet3d-serve Docker image

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/useful_tools.html

Builds the Docker image required for serving MMDetection3D models.

```bash
docker build -t mmdet3d-serve:latest docker/serve/
```

--------------------------------

### Example nuScenes Evaluation Output

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/nuscenes.html

Sample output showing the nuScenes detection score (NDS) and per-class metrics.

```text
mAP: 0.3197
mATE: 0.7595
mASE: 0.2700
mAOE: 0.4918
mAVE: 1.3307
mAAE: 0.1724
NDS: 0.3905
Eval time: 170.8s

Per-class results:
Object Class    AP      ATE     ASE     AOE     AVE     AAE
car     0.503   0.577   0.152   0.111   2.096   0.136
truck   0.223   0.857   0.224   0.220   1.389   0.179
bus     0.294   0.855   0.204   0.190   2.689   0.283
trailer 0.081   1.094   0.243   0.553   0.742   0.167
construction_vehicle    0.058   1.017   0.450   1.019   0.137   0.341
pedestrian      0.392   0.687   0.284   0.694   0.876   0.158
motorcycle      0.317   0.737   0.265   0.580   2.033   0.104
bicycle 0.308   0.704   0.299   0.892   0.683   0.010
traffic_cone    0.555   0.486   0.309   nan     nan     nan
barrier 0.466   0.581   0.269   0.169   nan     nan
```

--------------------------------

### Prepare Model for Publishing

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/useful_tools.html

Prepares model weights for release by converting to CPU tensors, removing optimizer states, and appending a hash ID to the filename.

```bash
python tools/model_converters/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME}
```

```bash
python tools/model_converters/publish_model.py work_dirs/faster_rcnn/latest.pth faster_rcnn_r50_fpn_1x_20190801.pth
```

--------------------------------

### Custom dataset label format

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_dataset.html

Example of the required text annotation format for 3D bounding boxes and categories.

```text
# format: [x, y, z, dx, dy, dz, yaw, category_name]
1.23 1.42 0.23 3.96 1.65 1.55 1.56 Car
3.51 2.15 0.42 1.05 0.87 1.86 1.23 Pedestrian
...
```

--------------------------------

### Generate SemanticKITTI Dataset Info

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/semantickitti.html

Command to create the .pkl info files required for training and testing using the provided conversion script.

```bash
python ./tools/create_data.py semantickitti --root-path ./data/semantickitti --out-dir ./data/semantickitti --extra-tag semantickitti
```

--------------------------------

### Run Point Cloud 3D Detection Demo

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/inference.html

Executes inference on point cloud data. Use the --show flag to visualize results online via Open3D.

```bash
python demo/pcd_demo.py ${PCD_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--score-thr ${SCORE_THR}] [--out-dir ${OUT_DIR}] [--show]
```

```bash
python demo/pcd_demo.py demo/data/kitti/000008.bin configs/pointpillars/pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py ${CHECKPOINT_FILE} --show
```

```bash
python demo/pcd_demo.py demo/data/sunrgbd/sunrgbd_000017.bin configs/votenet/votenet_8xb16_sunrgbd-3d.py ${CHECKPOINT_FILE} --show
```

--------------------------------

### Train a new model

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/new_data_model.html

Execute the training script using a specific configuration file.

```bash
python tools/train.py configs/pointpillars/pointpillars_hv_secfpn_sbn-all_16xb2-2x_waymoD5-3d-3class.py
```

--------------------------------

### Launch Slurm jobs with pre-configured config files

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/train_test.html

Execute training jobs using Slurm after ports have been manually set in the respective configuration files.

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config1.py ${WORK_DIR}
CUDA_VISIBLE_DEVICES=4,5,6,7 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py ${WORK_DIR}
```

--------------------------------

### Plot training curves with analyze_logs.py

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/useful_tools.html

Use this script to visualize training metrics from log files. Requires the seaborn library to be installed.

```bash
python tools/analysis_tools/analyze_logs.py plot_curve [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}] [--mode ${MODE}] [--interval ${INTERVAL}]
```

```bash
python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys loss_cls --legend loss_cls
```

```bash
python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys loss_cls loss_bbox --out losses.pdf
```

```bash
# evaluate PartA2 and second on KITTI according to Car_3D_moderate_strict
python tools/analysis_tools/analyze_logs.py plot_curve tools/logs/PartA2.log.json tools/logs/second.log.json --keys KITTI/Car_3D_moderate_strict --legend PartA2 second --mode eval --interval 1
# evaluate PointPillars for car and 3 classes on KITTI according to Car_3D_moderate_strict
python tools/analysis_tools/analyze_logs.py plot_curve tools/logs/pp-3class.log.json tools/logs/pp.log.json --keys KITTI/Car_3D_moderate_strict --legend pp-3class pp --mode eval --interval 2
```

--------------------------------

### Lyft Evaluation Results Format

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/lyft.html

Example output format showing mAP@0.5:0.95 metrics per class for the Lyft dataset.

```text
+mAPs@0.5:0.95------+--------------+
| class             | mAP@0.5:0.95 |
+-------------------+--------------+
| animal            | 0.0          |
| bicycle           | 0.099        |
| bus               | 0.177        |
| car               | 0.422        |
| emergency_vehicle | 0.0          |
| motorcycle        | 0.049        |
| other_vehicle     | 0.359        |
| pedestrian        | 0.066        |
| truck             | 0.176        |
| Overall           | 0.15         |
+-------------------+--------------+
```

--------------------------------

### Print expanded configuration

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/useful_tools.html

Displays the full configuration file content with all imports expanded. Useful for debugging complex configuration hierarchies.

```bash
python tools/misc/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}]
```

--------------------------------

### Use array_converter for data-type agnostic processing

Source: https://mmdetection3d.readthedocs.io/en/latest/_modules/mmdet3d/utils/array_converter.html

Examples demonstrating how to use the array_converter decorator to handle different input types and control output recovery.

```python
import torch
import numpy as np

# Use torch addition for a + b,
# and convert return values to the type of a
@array_converter(apply_to=('a', 'b'))
def simple_add(a, b):
    return a + b

a = np.array([1.1])
b = np.array([2.2])
simple_add(a, b)

# Use numpy addition for a + b,
# and convert return values to the type of b
@array_converter(to_torch=False, apply_to=('a', 'b'),
                 template_arg_name_='b')
def simple_add(a, b):
    return a + b

simple_add(a, b)

# Use torch funcs for floor(a) if flag=True else ceil(a),
# and return the torch tensor
@array_converter(apply_to=('a',), recover=False)
def floor_or_ceil(a, flag=True):
    return torch.floor(a) if flag else torch.ceil(a)

floor_or_ceil(a, flag=False)
```

--------------------------------

### Generate KITTI Data Infos

Source: https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/lidar_det3d.html

Run the creation script to generate necessary data information files for the KITTI dataset.

```bash
python tools/create_data.py kitti --root-path ./data/kitti --out-dir ./data/kitti --extra-tag kitti
```

--------------------------------

### Define PointPillars Data Pipelines

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/data_pipeline.html

Example configuration for training and testing pipelines used in PointPillars, defining the sequence of operations for data loading, augmentation, and formatting.

```python
train_pipeline = [
    dict(
        type='LoadPointsFromFile',
        load_dim=5,
        use_dim=5,
        backend_args=backend_args),
    dict(
        type='LoadPointsFromMultiSweeps',
        sweeps_num=10,
        backend_args=backend_args),
    dict(type='LoadAnnotations3D', with_bbox_3d=True, with_label_3d=True),
    dict(
        type='GlobalRotScaleTrans',
        rot_range=[-0.3925, 0.3925],
        scale_ratio_range=[0.95, 1.05],
        translation_std=[0, 0, 0]),
    dict(type='RandomFlip3D', flip_ratio_bev_horizontal=0.5),
    dict(type='PointsRangeFilter', point_cloud_range=point_cloud_range),
    dict(type='ObjectRangeFilter', point_cloud_range=point_cloud_range),
    dict(type='ObjectNameFilter', classes=class_names),
    dict(type='PointShuffle'),
    dict(type='DefaultFormatBundle3D', class_names=class_names),
    dict(type='Collect3D', keys=['points', 'gt_bboxes_3d', 'gt_labels_3d'])
]
test_pipeline = [
    dict(
        type='LoadPointsFromFile',
        load_dim=5,
        use_dim=5,
        backend_args=backend_args),
    dict(
        type='LoadPointsFromMultiSweeps',
        sweeps_num=10,
        backend_args=backend_args),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(1333, 800),
        pts_scale_ratio=1.0,
        flip=False,
        pcd_horizontal_flip=False,
        pcd_vertical_flip=False,
        transforms=[
            dict(
                type='GlobalRotScaleTrans',
                rot_range=[0, 0],
                scale_ratio_range=[1., 1.],
                translation_std=[0, 0, 0]),
            dict(type='RandomFlip3D'),
            dict(
                type='PointsRangeFilter', point_cloud_range=point_cloud_range),
            dict(
                type='DefaultFormatBundle3D',
                class_names=class_names,
                with_label=False),
            dict(type='Collect3D', keys=['points'])
        ])
]
```

--------------------------------

### Generate KITTI Info Files

Source: https://mmdetection3d.readthedocs.io/en/latest/user_guides/dataset_prepare.html

Generate info files for KITTI using standard or Slurm-based execution.

```bash
python tools/create_data.py kitti --root-path ./data/kitti --out-dir ./data/kitti --extra-tag kitti
```

```bash
sh tools/create_data.sh <partition> kitti
```
Tokens:
Raw