CrowdDet

https://github.com/xg-chu/crowddet
Admin
CrowdDet is a PyTorch-based object detection framework for detecting highly-overlapped instances in...
Tokens:5,508
Snippets:48
Trust Score:8.6
License:MIT
Update:6 months ago
Show doc for...
Tokens:
Raw
### Setup Docker Environment

Source: https://github.com/xg-chu/crowddet/blob/master/README.md

Commands to build and run the CrowdDet Docker container.

```shell
sudo docker build . -t crowddet
```

```shell
sudo docker run --gpus all --shm-size=8g -it --rm crowddet
```

--------------------------------

### Initialize and Run RPN

Source: https://context7.com/xg-chu/crowddet/llms.txt

Shows how to initialize the Region Proposal Network and perform both inference and training passes using simulated FPN features.

```python
import torch
import sys
sys.path.insert(0, 'lib')
sys.path.insert(0, 'model/rcnn_emd_simple')

from module.rpn import RPN
from config import config

# Initialize RPN
rpn = RPN(rpn_channel=256)

# Simulated FPN features (5 scale levels)
fpn_features = [
    torch.randn(1, 256, 13, 21),   # P6
    torch.randn(1, 256, 25, 42),   # P5
    torch.randn(1, 256, 50, 84),   # P4
    torch.randn(1, 256, 100, 167), # P3
    torch.randn(1, 256, 200, 334), # P2
]

# Image info tensor
im_info = torch.tensor([[800, 1333, 0.75, 1067, 1777, 0]])

# Inference mode - get proposals only
rpn.eval()
with torch.no_grad():
    rpn_rois = rpn(fpn_features, im_info)
    # rpn_rois shape: (num_proposals, 5) - [batch_idx, x1, y1, x2, y2]
    print(f"Number of proposals: {rpn_rois.shape[0]}")
    print(f"Sample proposal: {rpn_rois[0].tolist()}")

# Training mode - get proposals and losses
rpn.train()
gt_boxes = torch.tensor([[[100, 100, 200, 300, 1],
                          [300, 150, 400, 350, 1],
                          [0, 0, 0, 0, 0]]])  # (batch, max_boxes, 5)
rpn_rois, loss_dict = rpn(fpn_features, im_info, gt_boxes)
print(f"RPN cls loss: {loss_dict['loss_rpn_cls'].item():.4f}")
print(f"RPN loc loss: {loss_dict['loss_rpn_loc'].item():.4f}")
```

--------------------------------

### Initialize Network Architecture

Source: https://context7.com/xg-chu/crowddet/llms.txt

Load the Network class and pre-trained weights for inference.

```python
import torch
import sys
sys.path.insert(0, 'lib')
sys.path.insert(0, 'model/rcnn_emd_simple')

from network import Network
from config import config

# Initialize network
net = Network()
net.eval()

# Load pre-trained weights
checkpoint = torch.load('model/rcnn_emd_simple/outputs/model_dump/dump-40.pth',
                        map_location='cpu')
net.load_state_dict(checkpoint['state_dict'])

# Prepare input image (batch_size, channels, height, width)
image = torch.randn(1, 3, 800, 1333)  # Resized image tensor
im_info = torch.tensor([[800, 1333, 0.75, 1067, 1777, 0]])  # [h, w, scale, orig_h, orig_w, num_gt]
```

--------------------------------

### Manage Docker Environment

Source: https://context7.com/xg-chu/crowddet/llms.txt

Provides commands for building the Docker image and running containers with GPU support and volume mounts.

```bash
# Build Docker image
sudo docker build . -t crowddet

# Run Docker container with GPU support
sudo docker run --gpus all --shm-size=8g -it --rm crowddet

# Run with mounted data directory
sudo docker run --gpus all --shm-size=8g -it --rm \
    -v /path/to/CrowdHuman:/data/CrowdHuman \
    crowddet

# Run with mounted model directory for saving checkpoints
sudo docker run --gpus all --shm-size=8g -it --rm \
    -v /path/to/CrowdHuman:/data/CrowdHuman \
    -v /path/to/models:/workspace/model/outputs \
    crowddet
```

--------------------------------

### Train and Test Models

Source: https://github.com/xg-chu/crowddet/blob/master/README.md

Commands to execute training and testing scripts within the tools directory.

```shell
cd tools
python3 train.py -md rcnn_fpn_baseline
```

```shell
cd tools
python3 test.py -md rcnn_fpn_baseline -r 40
```

--------------------------------

### Initialize ResNet50 Backbone and FPN

Source: https://context7.com/xg-chu/crowddet/llms.txt

Sets up a ResNet50 backbone and a Feature Pyramid Network (FPN) module. FPN builds multi-scale feature pyramids essential for detecting objects of various sizes.

```python
import torch
import sys
sys.path.insert(0, 'lib')
sys.path.insert(0, 'model/rcnn_emd_simple')

from backbone.resnet50 import ResNet50
from backbone.fpn import FPN

# Initialize backbone and FPN
resnet = ResNet50(freeze_at=2, use_gn=False)
fpn = FPN(resnet, layers_begin=2, layers_end=6)
```

--------------------------------

### Compute Evaluation Metrics

Source: https://context7.com/xg-chu/crowddet/llms.txt

Sets up the paths and imports required to compute AP, MR, and JI metrics for model evaluation.

```python
import sys
sys.path.insert(0, 'lib')

from evaluate.compute_APMR import compute_APMR
from evaluate.compute_JI import evaluation_all

# Paths to detection results and ground truth
detection_file = 'model/rcnn_emd_simple/outputs/eval_dump/dump-40.json'
gt_file = '/data/CrowdHuman/annotation_val.odgt'
```

--------------------------------

### Test and Evaluate Models

Source: https://context7.com/xg-chu/crowddet/llms.txt

Run inference on validation datasets and compute metrics like AP, MR, and JI.

```bash
# Navigate to tools directory
cd tools

# Test model at epoch 40 using GPU 0
python3 test.py -md rcnn_fpn_baseline -r 40

# Test using multiple GPUs (0-3)
python3 test.py -md rcnn_emd_simple -r 40 -d 0-3

# Test RetinaNet EMD model
python3 test.py -md retina_emd_simple -r 30 -d 0,1

# Output: Creates dump-{epoch}.json in model/{model_dir}/outputs/eval_dump/
# Automatically evaluates and prints: AP:{value}, MR:{value}, JI:{value}
```

--------------------------------

### Evaluate and Visualize Results

Source: https://github.com/xg-chu/crowddet/blob/master/README.md

Commands for evaluating JSON results, running inference, and visualizing outputs.

```shell
cd tools
python3 eval_json.py -f your_json_path.json
python3 inference.py -md rcnn_fpn_baseline -r 40 -i your_image_path.png 
python3 visulize_json.py -f your_json_path.json -n 3
```

--------------------------------

### Train CrowdDet Models

Source: https://context7.com/xg-chu/crowddet/llms.txt

Use the train.py script to initiate distributed multi-GPU training for various model architectures.

```bash
# Navigate to tools directory
cd tools

# Train RCNN FPN Baseline model
python3 train.py -md rcnn_fpn_baseline

# Train RCNN EMD Simple model (with EMD Loss)
python3 train.py -md rcnn_emd_simple

# Train RCNN EMD with Refinement Module
python3 train.py -md rcnn_emd_refine

# Train RetinaNet baseline
python3 train.py -md retina_fpn_baseline

# Resume training from epoch 20
python3 train.py -md rcnn_emd_simple -r 20
```

--------------------------------

### Visualize Detection Results

Source: https://context7.com/xg-chu/crowddet/llms.txt

Generate images displaying detection boxes and ground truth from JSON result files.

```bash
# Navigate to tools directory
cd tools

# Visualize 3 images from results (default)
python3 visulize_json.py -f /path/to/results.json

# Visualize 10 images with custom confidence threshold
python3 visulize_json.py -f detection_results.json -n 10 -v 0.5

# Output: Creates outputs/{image_id}.png files with detection visualization
```

--------------------------------

### Evaluate JSON Results

Source: https://context7.com/xg-chu/crowddet/llms.txt

Compute performance metrics from previously generated detection JSON files.

```bash
# Navigate to tools directory
cd tools

# Evaluate a detection JSON file
python3 eval_json.py -f /path/to/detection_results.json

# Example with model output
python3 eval_json.py -f ../model/rcnn_emd_simple/outputs/eval_dump/dump-40.json

# Output format:
# AP:0.9052, MR:0.4196, JI:0.8209
# Results appended to outputs/result_eval.md
```

--------------------------------

### Load CrowdHuman Dataset and DataLoader

Source: https://context7.com/xg-chu/crowddet/llms.txt

Initializes the CrowdHuman dataset for training or validation. Includes automatic image resizing and ground truth parsing. A DataLoader is configured with a custom collate function for batch processing.

```python
import torch
import sys
sys.path.insert(0, 'lib')
sys.path.insert(0, 'model/rcnn_emd_simple')

from data.CrowdHuman import CrowdHuman
from config import config

# Create training dataset
train_dataset = CrowdHuman(config, if_train=True)
print(f"Training samples: {len(train_dataset)}")

# Create validation dataset
val_dataset = CrowdHuman(config, if_train=False)
print(f"Validation samples: {len(val_dataset)}")

# Get a single sample (training mode)
image, gt_boxes, im_info = train_dataset[0]
# image: numpy array (H, W, 3)
# gt_boxes: numpy array (num_boxes, 5) - [x1, y1, x2, y2, class_id]
# im_info: numpy array [h, w, scale, orig_h, orig_w, num_boxes]

# Create DataLoader with custom collate function
train_loader = torch.utils.data.DataLoader(
    dataset=train_dataset,
    batch_size=2,
    num_workers=2,
    collate_fn=train_dataset.merge_batch,
    shuffle=True
)

# Iterate through batches
for images, gt_boxes, im_info in train_loader:
    if images is not None:
        print(f"Batch images: {images.shape}")  # (batch, 3, H, W)
        print(f"Batch gt_boxes: {gt_boxes.shape}")  # (batch, max_boxes, 5)
        break
```

--------------------------------

### Define Model Configuration

Source: https://context7.com/xg-chu/crowddet/llms.txt

Defines the hyperparameters for the model architecture, training process, and data processing via a configuration class.

```python
import numpy as np

class Config:
    # Output directories
    output_dir = 'outputs'
    model_dir = 'outputs/model_dump'
    eval_dir = 'outputs/eval_dump'
    init_weights = '/data/model/resnet50_fbaug.pth'

    # Data configuration
    image_mean = np.array([103.530, 116.280, 123.675])
    image_std = np.array([57.375, 57.120, 58.395])
    train_image_short_size = 800
    train_image_max_size = 1400
    eval_image_short_size = 800
    eval_image_max_size = 1400

    # Class configuration
    class_names = ['background', 'person']
    num_classes = 2

    # Training configuration
    train_batch_per_gpu = 2
    base_lr = 1e-3 * 1.25
    momentum = 0.9
    weight_decay = 1e-4
    warm_iter = 800
    max_epoch = 30
    lr_decay = [24, 27]  # Epochs to decay learning rate

    # RPN configuration
    anchor_base_size = 32
    anchor_aspect_ratios = [1, 2, 3]
    rpn_nms_threshold = 0.7
    train_prev_nms_top_n = 12000
    train_post_nms_top_n = 2000
    test_prev_nms_top_n = 6000
    test_post_nms_top_n = 1000

    # RCNN configuration
    num_rois = 512
    fg_ratio = 0.5
    fg_threshold = 0.5

    # Testing configuration
    test_nms = 0.5
    test_nms_method = 'set_nms'  # 'set_nms', 'normal_nms', or 'none'
    pred_cls_threshold = 0.01
    visulize_threshold = 0.3

config = Config()
```

--------------------------------

### Apply Set NMS for Crowded Scenes

Source: https://context7.com/xg-chu/crowddet/llms.txt

Implements Set NMS to preserve detections from the same proposal while suppressing duplicates from different proposals. Compares Set NMS with standard NMS.

```python
import numpy as np
import sys
sys.path.insert(0, 'lib')

from utils.nms_utils import set_cpu_nms, cpu_nms

# Detection boxes: [x1, y1, x2, y2, score, tag, proposal_id]
# proposal_id identifies which proposal generated this detection
detections = np.array([
    [100, 100, 200, 300, 0.95, 1, 0],  # proposal 0, high score
    [105, 102, 205, 302, 0.90, 1, 0],  # proposal 0, overlapping (same proposal)
    [102, 101, 202, 301, 0.85, 1, 1],  # proposal 1, overlapping (different proposal)
    [300, 100, 400, 300, 0.80, 1, 2],  # proposal 2, no overlap
])

# Standard NMS - suppresses overlapping boxes regardless of proposal
keep_standard = cpu_nms(detections[:, :5], thresh=0.5)
print(f"Standard NMS kept indices: {keep_standard}")

# Set NMS - preserves same-proposal overlapping boxes
keep_set = set_cpu_nms(detections, thresh=0.5)
print(f"Set NMS kept mask: {keep_set}")

# Apply Set NMS filter
filtered_detections = detections[keep_set]
print(f"Detections after Set NMS: {len(filtered_detections)}")
```

--------------------------------

### Perform Single Image Inference

Source: https://context7.com/xg-chu/crowddet/llms.txt

Detect persons in a single image and save the visualization with bounding boxes.

```bash
# Navigate to tools directory
cd tools

# Run inference on a single image
python3 inference.py -md rcnn_emd_simple -r 40 -i /path/to/your/image.png

# Using RCNN EMD with refinement
python3 inference.py -md rcnn_emd_refine -r 40 -i crowd_scene.jpg

# Output: Creates outputs/{image_name}.png with detection boxes drawn
```

--------------------------------

### Extract FPN Features

Source: https://context7.com/xg-chu/crowddet/llms.txt

Demonstrates how to process an input image tensor through the FPN module and iterate over the resulting multi-scale feature maps.

```python
# Input image tensor
image = torch.randn(1, 3, 800, 1333)

# Get FPN features
fpn_features = fpn(image)

# FPN outputs features at multiple scales
# Stride: 64, 32, 16, 8, 4 (P6 -> P2)
for i, feat in enumerate(fpn_features):
    stride = 64 // (2 ** i) if i > 0 else 64
    print(f"P{6-i}: shape={feat.shape}, stride={stride}")
```

--------------------------------

### Run Inference with EMD Model

Source: https://context7.com/xg-chu/crowddet/llms.txt

Performs inference using a trained EMD model on an image. Outputs bounding boxes, scores, and tags. Ensure the model and image data are correctly loaded.

```python
with torch.no_grad():
    pred_boxes = net(image, im_info)
    # Output shape: (num_proposals, 12) for EMD models
    # Each row: [x1, y1, x2, y2, score, tag] * 2 (two predictions per proposal)
    print(f"Predictions shape: {pred_boxes.shape}")
    print(f"Sample detection: bbox={pred_boxes[0, :4].tolist()}, score={pred_boxes[0, 4]:.3f}")
```

--------------------------------

### Cite CrowdDet

Source: https://github.com/xg-chu/crowddet/blob/master/README.md

BibTeX citation for the CrowdDet research paper.

```bibtex
@InProceedings{Chu_2020_CVPR,
author = {Chu, Xuangeng and Zheng, Anlin and Zhang, Xiangyu and Sun, Jian},
title = {Detection in Crowded Scenes: One Proposal, Multiple Predictions},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}
```

--------------------------------

### Compute AP and MR metrics

Source: https://context7.com/xg-chu/crowddet/llms.txt

Calculates Average Precision and Miss Rate for detection results against ground truth files.

```python
AP, MR = compute_APMR(detection_file, gt_file, target_key='box', mode=0)
print(f"Average Precision (AP): {AP:.4f}")
print(f"Miss Rate (MR): {MR:.4f}")
```

--------------------------------

### Compute and display Jaccard Index

Source: https://context7.com/xg-chu/crowddet/llms.txt

Evaluates the Jaccard Index across various score thresholds and prints the resulting metrics.

```python
# Compute Jaccard Index across different score thresholds
res_lines, max_JI = evaluation_all(detection_file, target_key='box')
print(f"Maximum Jaccard Index (JI): {max_JI:.4f}")

# Print results at each threshold
for line in res_lines:
    print(line)
```

--------------------------------

### Compute EMD Loss for Set Predictions

Source: https://context7.com/xg-chu/crowddet/llms.txt

Calculates the Earth Mover's Distance loss for set predictions, handling assignment permutations between two prediction heads. Requires ground truth targets and labels.

```python
import torch
import sys
sys.path.insert(0, 'lib')
sys.path.insert(0, 'model/rcnn_emd_simple')

from det_oprs.loss_opr import emd_loss_softmax, softmax_loss, smooth_l1_loss

# Simulated predictions from two heads (batch_size=4, num_classes=2)
pred_bbox_0 = torch.randn(4, 8)   # bbox deltas: 4 coords * 2 classes
pred_cls_0 = torch.randn(4, 2)    # classification scores
pred_bbox_1 = torch.randn(4, 8)   # second prediction head
pred_cls_1 = torch.randn(4, 2)

# Ground truth targetsbox_targets = torch.randn(4, 2, 4)  # (batch, 2 instances, 4 coords)
labels = torch.tensor([[1, 0], [1, 1], [0, 1], [1, 0]])  # (batch, 2 instances)

# Compute EMD loss - tries both assignment permutations
loss0 = emd_loss_softmax(pred_bbox_0, pred_cls_0, pred_bbox_1, pred_cls_1,
                         bbox_targets, labels)
loss1 = emd_loss_softmax(pred_bbox_1, pred_cls_1, pred_bbox_0, pred_cls_0,
                         bbox_targets, labels)

# Select minimum loss assignment
loss = torch.cat([loss0, loss1], axis=1)
_, min_indices = loss.min(axis=1)
loss_emd = loss[torch.arange(loss.shape[0]), min_indices].mean()
print(f"EMD Loss: {loss_emd.item():.4f}")
```
Tokens:
Raw