### Install MaskDINO and Dependencies

Source: https://context7.com/idea-research/maskdino/llms.txt

Setup a conda environment, install Detectron2 dependencies, and compile the CUDA MSDeformAttn kernel. Ensure PyTorch and CUDA versions are compatible.

```bash
conda create --name maskdino python=3.8 -y
conda activate maskdino
conda install pytorch==1.9.0 torchvision==0.10.0 cudatoolkit=11.1 -c pytorch -c nvidia
pip install -U opencv-python

git clone git@github.com:facebookresearch/detectron2.git
cd detectron2
pip install -e .
pip install git+https://github.com/cocodataset/panopticapi.git
pip install git+https://github.com/mcordts/cityscapesScripts.git

cd ..
git clone git@github.com:facebookresearch/MaskDINO.git
cd MaskDINO
pip install -r requirements.txt

cd maskdino/modeling/pixel_decoder/ops
sh make.sh

TORCH_CUDA_ARCH_LIST='8.0' FORCE_CUDA=1 python setup.py build install
```

--------------------------------

### Install Cityscapes Scripts

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Install the cityscapesScripts library from its GitHub repository. This is needed for processing the Cityscapes dataset.

```bash
pip install git+https://github.com/mcordts/cityscapesScripts.git
```

--------------------------------

### Example Conda Environment Setup for MaskDINO

Source: https://github.com/idea-research/maskdino/blob/main/INSTALL.md

Set up a Conda environment named 'maskdino' with Python 3.8, install PyTorch, torchvision, and cudatoolkit. Then, install OpenCV, Detectron2, panopticapi, and cityscapesScripts. Finally, clone and install MaskDINO and compile its CUDA kernels.

```bash
conda create --name maskdino python=3.8 -y
conda activate maskdino
conda install pytorch==1.9.0 torchvision==0.10.0 cudatoolkit=11.1 -c pytorch -c nvidia
pip install -U opencv-python

# under your working directory
git clone git@github.com:facebookresearch/detectron2.git
cd detectron2
pip install -e .
pip install git+https://github.com/cocodataset/panopticapi.git
pip install git+https://github.com/mcordts/cityscapesScripts.git

cd ..
git clone git@github.com:facebookresearch/MaskDINO.git
cd MaskDINO
pip install -r requirements.txt
cd maskdino/modeling/pixel_decoder/ops
sh make.sh
```

--------------------------------

### Prepare Cityscapes Datasets

Source: https://context7.com/idea-research/maskdino/llms.txt

Installs cityscapesScripts and provides commands to generate label images and panoptic annotations for the Cityscapes dataset.

```bash
pip install git+https://github.com/mcordts/cityscapesScripts.git

# Generate label images:
CITYSCAPES_DATASET=$DETECTRON2_DATASETS/cityscapes \
  python cityscapesscripts/preparation/createTrainIdLabelImgs.py

# Generate panoptic annotations:
CITYSCAPES_DATASET=$DETECTRON2_DATASETS/cityscapes \
  python cityscapesscripts/preparation/createPanopticImgs.py

# Or use symbolic link if datasets exist elsewhere:
ln -s /existing/coco ./datasets/coco
```

--------------------------------

### Fine-tune MaskDINO with Mask-Enhanced Box Initialization

Source: https://context7.com/idea-research/maskdino/llms.txt

This command-line example shows how to fine-tune an existing MaskDINO model using mask-enhanced box initialization. Ensure the specified configuration file and model weights path are correct.

```bash
# python train_net.py \
#   --num-gpus 8 \
#   --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
#   MODEL.WEIGHTS /path/to/model_iter_032000.pth \
#   MODEL.MaskDINO.INITIALIZE_BOX_TYPE mask2box \
#   SOLVER.MAX_ITER 50000 \
#   SOLVER.STEPS "(45000,)"
```

--------------------------------

### Install panopticapi for COCO and ADE20k

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Install the panopticapi library from its GitHub repository. This is required for processing COCO and ADE20k datasets.

```bash
pip install git+https://github.com/cocodataset/panopticapi.git
```

--------------------------------

### Prepare COCO Instance and Panoptic Datasets

Source: https://context7.com/idea-research/maskdino/llms.txt

Installs the panopticapi package and provides a script to generate semantic annotations from panoptic annotations for COCO dataset evaluation.

```bash
pip install git+https://github.com/cocodataset/panopticapi.git
# Generate semantic annotations from panoptic for evaluation:
python datasets/prepare_coco_semantic_annos_from_panoptic_annos.py
```

--------------------------------

### Build MaskDINO Model and Run Inference

Source: https://context7.com/idea-research/maskdino/llms.txt

This snippet demonstrates how to build a MaskDINO model from a configuration file, load weights, prepare an image input, and run inference to get instance, semantic, or panoptic segmentation outputs.

```python
cfg = get_cfg()
add_deeplab_config(cfg)
add_maskdino_config(cfg)
cfg.merge_from_file("configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml")
cfg.merge_from_list(["MODEL.WEIGHTS", "maskdino_r50_50ep_instance.pth"])
cfg.freeze()

model = build_model(cfg)
DetectionCheckpointer(model).load(cfg.MODEL.WEIGHTS)
model.eval()

# --- Prepare input (BGR numpy array, shape H x W x 3) ---
image = read_image("image.jpg", format="BGR")    # np.ndarray (H, W, 3)
height, width = image.shape[:2]

# Detectron2 batched input format
batched_inputs = [{"image": torch.as_tensor(image.transpose(2, 0, 1).copy()),  # (3, H, W)
"height": height,
"width": width,
}]

# --- Run inference ---
with torch.no_grad():
    outputs = model(batched_inputs)

# outputs is a list[dict], one dict per image
result = outputs[0]

# Instance segmentation output
if "instances" in result:
    instances = result["instances"]
    print("pred_classes:", instances.pred_classes)   # Tensor[N] - class indices
    print("scores:",       instances.scores)          # Tensor[N] - confidence scores
    print("pred_boxes:",   instances.pred_boxes)      # Boxes[N]  - xyxy format
    print("pred_masks:",   instances.pred_masks.shape)# Tensor[N, H, W] - binary masks

# Semantic segmentation output: Tensor[C, H, W] logits per class
if "sem_seg" in result:
    semseg = result["sem_seg"]   # shape (num_classes, H, W)
    pred_labels = semseg.argmax(dim=0)  # shape (H, W)

# Panoptic segmentation output
if "panoptic_seg" in result:
    panoptic_seg, segments_info = result["panoptic_seg"]
    # panoptic_seg: Tensor[H, W] of integer segment IDs
    # segments_info: list[dict] with keys "id", "isthing", "category_id"
    for seg in segments_info:
        print(f"segment {seg['id']}: class={seg['category_id']}, isthing={seg['isthing']}")
```

--------------------------------

### Download and Evaluate ADE20K Semantic Segmentation (ResNet-50)

Source: https://context7.com/idea-research/maskdino/llms.txt

This bash script downloads a pretrained MaskDINO model for ADE20K semantic segmentation using ResNet-50 and then evaluates it. Ensure you have wget and python installed.

```bash
# Download and evaluate ADE20K semantic segmentation - ResNet-50 (mIoU=48.7)
wget https://github.com/IDEA-Research/detrex-storage/releases/download/maskdino-v0.1.0/\nmaskdino_r50_50ep_100q_celoss_hid1024_3s_semantic_ade20k_48.7miou.pth

python train_net.py --eval-only --num-gpus 4 \
  --config-file configs/ade20k/semantic-segmentation/maskdino_R50_bs16_160k_steplr.yaml \
  MODEL.WEIGHTS maskdino_r50_50ep_100q_celoss_hid1024_3s_semantic_ade20k_48.7miou.pth
```

--------------------------------

### Build CUDA Kernel on a System Without GPU

Source: https://github.com/idea-research/maskdino/blob/main/INSTALL.md

Build the CUDA kernel for MSDeformAttn on a system without a GPU but with drivers installed. Specify the CUDA architecture and force CUDA compilation.

```python
TORCH_CUDA_ARCH_LIST='8.0' FORCE_CUDA=1 python setup.py build install
```

--------------------------------

### Download and Evaluate COCO Instance Segmentation (ResNet-50)

Source: https://context7.com/idea-research/maskdino/llms.txt

This bash script downloads a pretrained MaskDINO model for COCO instance segmentation using ResNet-50 and then evaluates it. Ensure you have wget and python installed.

```bash
# Download and evaluate COCO instance segmentation - ResNet-50 (Mask AP=46.3, Box AP=51.7)
wget https://github.com/IDEA-Research/detrex-storage/releases/download/maskdino-v0.1.0/\nmaskdino_r50_50ep_300q_hid2048_3sd1_instance_maskenhanced_mask46.3ap_box51.7ap.pth

python train_net.py --eval-only --num-gpus 8 \
  --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
  MODEL.WEIGHTS maskdino_r50_50ep_300q_hid2048_3sd1_instance_maskenhanced_mask46.3ap_box51.7ap.pth
```

--------------------------------

### Convert Swin Transformer to Detectron2

Source: https://github.com/idea-research/maskdino/blob/main/tools/README.md

This tool converts pre-trained Swin Transformer weights to Detectron2 format. Install the 'timm' library first. Multiple Swin model sizes are supported.

```bash
pip install timm

wget https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_tiny_patch4_window7_224.pth
python tools/convert-pretrained-swin-model-to-d2.py swin_tiny_patch4_window7_224.pth swin_tiny_patch4_window7_224.pkl

wget https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_small_patch4_window7_224.pth
python tools/convert-pretrained-swin-model-to-d2.py swin_small_patch4_window7_224.pth swin_small_patch4_window7_224.pkl

wget https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_base_patch4_window12_384_22k.pth
python tools/convert-pretrained-swin-model-to-d2.py swin_base_patch4_window12_384_22k.pth swin_base_patch4_window12_384_22k.pkl

wget https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_large_patch4_window12_384_22k.pth
python tools/convert-pretrained-swin-model-to-d2.py swin_large_patch4_window12_384_22k.pth swin_large_patch4_window12_384_22k.pkl
```

--------------------------------

### Download and Evaluate COCO Instance Segmentation (Swin-L)

Source: https://context7.com/idea-research/maskdino/llms.txt

This bash script downloads a pretrained MaskDINO model for COCO instance segmentation using Swin-L and then evaluates it. Ensure you have wget and python installed.

```bash
# Download and evaluate COCO instance segmentation - Swin-L (Mask AP=52.3, Box AP=59.0)
wget https://github.com/IDEA-Research/detrex-storage/releases/download/maskdino-v0.1.0/\nmaskdino_swinl_50ep_300q_hid2048_3sd1_instance_maskenhanced_mask52.3ap_box59.0ap.pth

python train_net.py --eval-only --num-gpus 8 \
  --config-file configs/coco/instance-segmentation/swin/maskdino_R50_bs16_50ep_4s_dowsample1_2048.yaml \
  MODEL.WEIGHTS maskdino_swinl_50ep_300q_hid2048_3sd1_instance_maskenhanced_mask52.3ap_box59.0ap.pth
```

--------------------------------

### Download and Evaluate COCO Panoptic Segmentation (ResNet-50)

Source: https://context7.com/idea-research/maskdino/llms.txt

This bash script downloads a pretrained MaskDINO model for COCO panoptic segmentation using ResNet-50 and then evaluates it. Ensure you have wget and python installed.

```bash
# Download and evaluate COCO panoptic segmentation - ResNet-50 (PQ=53.0)
wget https://github.com/IDEA-Research/detrex-storage/releases/download/maskdino-v0.1.0/\nmaskdino_r50_50ep_300q_hid2048_3sd1_panoptic_pq53.0.pth

python train_net.py --eval-only --num-gpus 8 \
  --config-file configs/coco/panoptic-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
  MODEL.WEIGHTS maskdino_r50_50ep_300q_hid2048_3sd1_panoptic_pq53.0.pth
```

--------------------------------

### Prepare ADE20K Datasets

Source: https://context7.com/idea-research/maskdino/llms.txt

Scripts for preparing ADE20K dataset, including downloading instance annotations and generating semantic, panoptic, and instance segmentation annotations for Detectron2.

```bash
# Download instance annotations:
wget http://sceneparsing.csail.mit.edu/data/ChallengeData2017/annotations_instance.tar
tar -xf annotations_instance.tar -C $DETECTRON2_DATASETS/ADEChallengeData2016/

python datasets/prepare_ade20k_sem_seg.py   # generates annotations_detectron2/
python datasets/prepare_ade20k_pan_seg.py   # generates ade20k_panoptic_{train,val}.json
python datasets/prepare_ade20k_ins_seg.py   # generates ade20k_instance_{train,val}.json
```

--------------------------------

### Download ADE20k Instance Annotations

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Download the instance annotation tarball for the ADE20k dataset from the provided URL.

```bash
wget http://sceneparsing.csail.mit.edu/data/ChallengeData2017/annotations_instance.tar
```

--------------------------------

### Run Inference with Pre-trained MaskDINO Model

Source: https://github.com/idea-research/maskdino/blob/main/demo/README.md

Execute inference on input images using a specified configuration file and pre-trained model weights. Ensure you are in the 'demo/' directory. The --opts argument is used to specify the model weights.

```bash
cd demo/
python demo.py --config-file /configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s.yaml \
  --input input1.jpg input2.jpg \
  [--other-options] \
  --opts MODEL.WEIGHTS /path/to/model_file
```

--------------------------------

### MaskDINO Command-Line Demo Script

Source: https://context7.com/idea-research/maskdino/llms.txt

Execute MaskDINO inference from the command line for images, videos, or webcam streams. Customize confidence thresholds and specify model weights and device.

```bash
# --- Command-line demo script ---
# python demo/demo.py \
#   --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
#   --input input1.jpg input2.jpg \
#   --output ./output_dir \
#   --confidence-threshold 0.5 \
#   --opts MODEL.WEIGHTS /path/to/model.pth MODEL.DEVICE cpu
#
# Webcam:
# python demo/demo.py --config-file <cfg> --webcam --opts MODEL.WEIGHTS <ckpt>
#
# Video file:
# python demo/demo.py --config-file <cfg> --video-input video.mp4 --output out.mkv \
#   --opts MODEL.WEIGHTS <ckpt>
```

--------------------------------

### Run Inference on CPU with MaskDINO

Source: https://github.com/idea-research/maskdino/blob/main/demo/README.md

Force inference to run on the CPU by adding 'MODEL.DEVICE cpu' to the --opts argument.

```bash
python demo.py --config-file /configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s.yaml --input input1.jpg --opts MODEL.WEIGHTS /path/to/model_file MODEL.DEVICE cpu
```

--------------------------------

### MaskDINO Training Commands

Source: https://context7.com/idea-research/maskdino/llms.txt

Command-line arguments for training MaskDINO models with different configurations, including Swin-L backbone, single-GPU training, resuming from checkpoints, panoptic segmentation, and semantic segmentation on ADE20K.

```bash
python train_net.py \
  --num-gpus 8 \
  --config-file configs/coco/instance-segmentation/swin/maskdino_R50_bs16_50ep_4s_dowsample1_2048.yaml \
  MODEL.WEIGHTS /path/to/swin_large_patch4_window12_384_22k.pkl
```

```bash
python train_net.py \
  --num-gpus 1 \
  --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
  SOLVER.IMS_PER_BATCH 2 \
  SOLVER.BASE_LR 0.000025
```

```bash
python train_net.py \
  --num-gpus 8 \
  --resume \
  --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
  OUTPUT_DIR ./output/maskdino_r50_instance
```

```bash
python train_net.py \
  --num-gpus 8 \
  --config-file configs/coco/panoptic-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
  MODEL.WEIGHTS ""
```

```bash
python train_net.py \
  --num-gpus 4 \
  --config-file configs/ade20k/semantic-segmentation/maskdino_R50_bs16_160k_steplr.yaml \
  MODEL.WEIGHTS ""
```

```bash
python train_net.py \
  --eval-only --num-gpus 8 \
  --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
  MODEL.WEIGHTS /path/to/checkpoint.pth \
  TEST.AUG.ENABLED True
```

--------------------------------

### Train MaskDINO Model from Scratch

Source: https://context7.com/idea-research/maskdino/llms.txt

Train a MaskDINO model from scratch using multiple GPUs. This command requires significant GPU memory and time, and it initializes weights from scratch by setting MODEL.WEIGHTS to an empty string.

```bash
# --- Train from scratch (8 GPUs, ResNet-50) ---
# Requires ~15G GPU memory per card, ~3 days for 50 epochs
python train_net.py \
  --num-gpus 8 \
  --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
  MODEL.WEIGHTS ""
```

--------------------------------

### Run Inference on Video with MaskDINO

Source: https://github.com/idea-research/maskdino/blob/main/demo/README.md

Process a video file for inference by replacing '--input files' with '--video-input video.mp4'.

```bash
python demo.py --config-file /configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s.yaml --video-input video.mp4 --opts MODEL.WEIGHTS /path/to/model_file
```

--------------------------------

### Prepare ADE20k Panoptic Annotations

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Combine semantic and instance annotations to create panoptic annotations for the ADE20k dataset.

```python
python datasets/prepare_ade20k_pan_seg.py
```

--------------------------------

### Run Inference on Webcam with MaskDINO

Source: https://github.com/idea-research/maskdino/blob/main/demo/README.md

Adapt the inference command to use your webcam as input by replacing '--input files' with '--webcam'.

```bash
python demo.py --config-file /configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s.yaml --webcam --opts MODEL.WEIGHTS /path/to/model_file
```

--------------------------------

### Interactive Demo with VisualizationDemo

Source: https://context7.com/idea-research/maskdino/llms.txt

Use VisualizationDemo for interactive inference on images and videos. It wraps DefaultPredictor and supports optional async multi-GPU parallel inference for video throughput. Ensure the 'demo/' directory is in the Python path.

```python
import cv2
from detectron2.config import get_cfg
from detectron2.projects.deeplab import add_deeplab_config
from maskdino import add_maskdino_config

# Add demo/ directory to path and import
import sys
sys.path.insert(0, "demo/")
from predictor import VisualizationDemo

# Setup config
cfg = get_cfg()
add_deeplab_config(cfg)
add_maskdino_config(cfg)
cfg.merge_from_file("configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml")
cfg.merge_from_list(["MODEL.WEIGHTS", "/path/to/model.pth"])
cfg.freeze()

# Create demo (set parallel=True to use all available GPUs for video)
demo = VisualizationDemo(cfg, parallel=False)

# --- Run on a single image ---
image = cv2.imread("input.jpg")   # BGR format, np.ndarray (H, W, 3)
predictions, visualized_output = demo.run_on_image(image)
visualized_output.save("output.jpg")

# predictions["instances"].pred_classes  -> detected class IDs
# predictions["instances"].scores        -> confidence scores
# predictions["instances"].pred_masks    -> binary instance masks

# --- Run on a video (yields BGR frames) ---
video = cv2.VideoCapture("input.mp4")
for vis_frame in demo.run_on_video(video):
    cv2.imshow("MaskDINO", vis_frame)
    if cv2.waitKey(1) == 27:
        break
video.release()
```

--------------------------------

### Train MaskDINO with Swin Backbone on 8 GPUs

Source: https://github.com/idea-research/maskdino/blob/main/README.md

Use this command to train MaskDINO with Swin backbones. Specify the path to pretrained backbones using MODEL.WEIGHTS. Ensure sufficient GPU memory for larger models like Swin-L.

```sh
python train_net.py --num-gpus 8 --config-file config_path MODEL.WEIGHTS /path/to/checkpoint_file
```

--------------------------------

### Evaluate Pretrained MaskDINO Models

Source: https://github.com/idea-research/maskdino/blob/main/README.md

Use this command to evaluate pretrained MaskDINO models. Ensure you have downloaded the model checkpoint and set the DETECTRON2_DATASETS environment variable if your dataset is not in the repository. Replace `/path/to/checkpoint_file` with the actual path to your downloaded model weights.

```sh
python train_net.py --eval-only --num-gpus 8 --config-file config_path MODEL.WEIGHTS /path/to/checkpoint_file
```

```sh
python train_net.py --eval-only --num-gpus 8 --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml MODEL.WEIGHTS /path/to/checkpoint_file
```

--------------------------------

### Prepare ADE20k Instance Annotations in COCO Format

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Extract instance annotations in COCO format for the ADE20k dataset.

```python
python datasets/prepare_ade20k_ins_seg.py
```

--------------------------------

### Compile CUDA Kernel for MSDeformAttn

Source: https://github.com/idea-research/maskdino/blob/main/INSTALL.md

Navigate to the ops directory and run the make script to compile the CUDA kernel for MSDeformAttn. Ensure CUDA_HOME is defined.

```bash
cd maskdino/modeling/pixel_decoder/ops
sh make.sh
```

--------------------------------

### Analyze Model FLOPs for Semantic Segmentation

Source: https://github.com/idea-research/maskdino/blob/main/tools/README.md

Use this command to analyze model parameters and FLOPs for semantic segmentation tasks on ADE20K. It uses a dummy input image of a fixed size. Do not use `--use-fixed-input-size` for other datasets like Cityscapes.

```bash
python tools/analyze_model.py --num-inputs 1 --tasks flop --use-fixed-input-size --config-file CONFIG_FILE
```

--------------------------------

### Create Cityscapes TrainId Label Images

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Generate labelTrainIds.png files for Cityscapes dataset. These are not needed for instance segmentation.

```bash
CITYSCAPES_DATASET=/path/to/abovementioned/cityscapes python cityscapesscripts/preparation/createTrainIdLabelImgs.py
```

--------------------------------

### Analyze Model FLOPs for Panoptic/Instance Segmentation

Source: https://github.com/idea-research/maskdino/blob/main/tools/README.md

This command analyzes model parameters and FLOPs for panoptic and instance segmentation tasks. It computes the average FLOPs over 100 real validation images.

```bash
python tools/analyze_model.py --num-inputs 100 --tasks flop --config-file CONFIG_FILE
```

--------------------------------

### Programmatic Mask to Box Conversion in MaskDINO

Source: https://context7.com/idea-research/maskdino/llms.txt

This Python code demonstrates the programmatic conversion of mask logits to bounding boxes, showcasing both a fast conversion and an accurate bitmask conversion. It requires PyTorch and Detectron2.

```python
import torch
from maskdino.utils import box_ops
from detectron2.structures import BitMasks

outputs_mask = torch.randn(2, 300, 100, 100)  # (B, Q, H, W) mask logits
h, w = outputs_mask.shape[-2:]
flat_mask = outputs_mask.detach().flatten(0, 1)   # (B*Q, H, W)

# Fast mask2box conversion
boxes_fast = box_ops.masks_to_boxes(flat_mask > 0)           # (B*Q, 4) xyxy
boxes_fast = box_ops.box_xyxy_to_cxcywh(boxes_fast)
boxes_fast = boxes_fast / torch.tensor([w, h, w, h], dtype=torch.float)

# Accurate bitmask conversion
boxes_accurate = BitMasks(flat_mask > 0).get_bounding_boxes().tensor  # (B*Q, 4) xyxy
```

--------------------------------

### Train MaskDINO on 1 GPU with Custom Batch Size and LR

Source: https://github.com/idea-research/maskdino/blob/main/README.md

When training on a single GPU, you need to manually determine appropriate values for the learning rate and batch size. Adjust SOLVER.IMS_PER_BATCH and SOLVER.BASE_LR accordingly.

```sh
python train_net.py --num-gpus 1 --config-file config_path SOLVER.IMS_PER_BATCH SET_TO_SOME_REASONABLE_VALUE SOLVER.BASE_LR SET_TO_SOME_REASONABLE_VALUE
```

--------------------------------

### Prepare ADE20k Semantic Segmentations

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Generate the annotations_detectron2 directory for ADE20k dataset by running this script.

```python
python datasets/prepare_ade20k_sem_seg.py
```

--------------------------------

### Convert Torchvision ResNet to Detectron2

Source: https://github.com/idea-research/maskdino/blob/main/tools/README.md

Use this script to convert pre-trained ResNet weights from torchvision to a format compatible with Detectron2. Ensure the torchvision model file is downloaded before running.

```bash
wget https://download.pytorch.org/models/resnet101-63fe2227.pth
python tools/convert-torchvision-to-d2.py resnet101-63fe2227.pth R-101.pkl
```

--------------------------------

### Set DETECTRON2_DATASETS Environment Variable

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Specify the directory where datasets are located. If unset, it defaults to './datasets'.

```bash
export DETECTRON2_DATASETS=/path/to/datasets
```

--------------------------------

### Mask-Enhanced Box Initialization Configuration

Source: https://context7.com/idea-research/maskdino/llms.txt

Configuration option in YAML to enable mask-enhanced box initialization. 'mask2box' uses the bounding rectangle of a thresholded mask for initialization.

```yaml
# In config YAML, set INITIALIZE_BOX_TYPE to one of:
# 'no'       - standard DINO-style anchor initialization (no mask enhancement)
# 'mask2box' - fast conversion using bounding rect of thresholded mask
```

--------------------------------

### Evaluate MaskDINO Pretrained Model

Source: https://context7.com/idea-research/maskdino/llms.txt

Evaluate a pretrained MaskDINO model using multiple GPUs. This command runs the evaluation-only mode and reports AP metrics on the specified dataset split.

```bash
# --- Evaluate a pretrained model (8 GPUs) ---
python train_net.py \
  --eval-only \
  --num-gpus 8 \
  --config-file configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml \
  MODEL.WEIGHTS /path/to/maskdino_r50_50ep_instance.pth

# Expected output: AP=46.3 (mask), AP=51.7 (box) on COCO val2017
```

--------------------------------

### Save Inference Outputs with MaskDINO

Source: https://github.com/idea-research/maskdino/blob/main/demo/README.md

Specify a directory or file path for saving the inference outputs using the '--output' argument.

```bash
python demo.py --config-file /configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s.yaml --input input1.jpg --output output_dir/ --opts MODEL.WEIGHTS /path/to/model_file
```

--------------------------------

### Prepare COCO Semantic Annotations

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Run this script to extract semantic annotations from panoptic annotations for COCO dataset evaluation.

```python
python datasets/prepare_coco_semantic_annos_from_panoptic_annos.py
```

--------------------------------

### Generate Cityscapes Panoptic Images

Source: https://github.com/idea-research/maskdino/blob/main/datasets/README.md

Generate panoptic annotations for the Cityscapes dataset. These are not needed for semantic and instance segmentation.

```bash
CITYSCAPES_DATASET=/path/to/abovementioned/cityscapes python cityscapesscripts/preparation/createPanopticImgs.py
```

--------------------------------

### MaskDINODecoder Initialization and Usage

Source: https://context7.com/idea-research/maskdino/llms.txt

Instantiating and configuring the MaskDINODecoder, which is the transformer decoder component for MaskDINO. It produces class logits, masks, and bounding boxes.

```python
from detectron2.config import get_cfg
from detectron2.projects.deeplab import add_deeplab_config
from maskdino import add_maskdino_config
from maskdino.modeling.transformer_decoder.maskdino_decoder import MaskDINODecoder

cfg = get_cfg()
add_deeplab_config(cfg)
add_maskdino_config(cfg)
cfg.merge_from_file("configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml")
cfg.freeze()

# Instantiated automatically by build_sem_seg_head; shown here for clarity:
decoder = MaskDINODecoder.from_config(cfg, in_channels=256, mask_classification=True)
# Key architectural hyperparameters from config:
# NUM_OBJECT_QUERIES = 300   (matching queries)
# DEC_LAYERS         = 9    (decoder layers + 1 initial prediction)
# TWO_STAGE          = True  (encoder-based query selection)
# DN                 = "seg" (denoising with both boxes and masks)
# DN_NUM             = 100   (number of denoising groups)
# INITIALIZE_BOX_TYPE= 'mask2box'  (mask-enhanced box initialization)

# Forward signature:
# x             : list of multi-scale features from pixel decoder (len = NUM_FEATURE_LEVELS)
# mask_features : per-pixel embeddings at 1/4 resolution, shape (B, C, H/4, W/4)
# masks         : padding masks, list of bool tensors
```

--------------------------------

### Register MaskDINO Config Defaults

Source: https://context7.com/idea-research/maskdino/llms.txt

Register MaskDINO-specific configuration keys into a Detectron2 CfgNode. This function must be called before loading any MaskDINO YAML config file. It sets defaults for model architecture, loss weights, inference modes, and training hyperparameters.

```python
from detectron2.config import get_cfg
from detectron2.projects.deeplab import add_deeplab_config
from maskdino import add_maskdino_config

cfg = get_cfg()
add_deeplab_config(cfg)       # required for poly LR schedule
add_maskdino_config(cfg)      # registers all MODEL.MaskDINO.* keys

# Load a task-specific YAML config
cfg.merge_from_file("configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml")

# Override individual options at runtime
cfg.merge_from_list([
    "MODEL.WEIGHTS", "/path/to/checkpoint.pth",
    "MODEL.MaskDINO.TEST.INSTANCE_ON", "True",
    "MODEL.MaskDINO.TEST.PANOPTIC_ON", "False",
    "MODEL.MaskDINO.INITIALIZE_BOX_TYPE", "mask2box",  # 'no', 'bitmask', or 'mask2box'
])
cfg.freeze()

# Key config nodes and their defaults:
# cfg.MODEL.MaskDINO.HIDDEN_DIM         = 256
# cfg.MODEL.MaskDINO.NUM_OBJECT_QUERIES = 100
# cfg.MODEL.MaskDINO.DEC_LAYERS         = 6
# cfg.MODEL.MaskDINO.NHEADS             = 8
# cfg.MODEL.MaskDINO.TWO_STAGE          = True
# cfg.MODEL.MaskDINO.DN                 = "seg"   # denoising mode
# cfg.MODEL.MaskDINO.DN_NUM             = 100
# cfg.MODEL.MaskDINO.DN_NOISE_SCALE     = 0.4
# cfg.MODEL.MaskDINO.CLASS_WEIGHT       = 4.0
# cfg.MODEL.MaskDINO.MASK_WEIGHT        = 5.0
# cfg.MODEL.MaskDINO.DICE_WEIGHT        = 5.0
# cfg.MODEL.MaskDINO.BOX_WEIGHT         = 5.0
# cfg.MODEL.MaskDINO.GIOU_WEIGHT        = 2.0
# cfg.MODEL.MaskDINO.TEST.SEMANTIC_ON   = True
# cfg.MODEL.MaskDINO.TEST.INSTANCE_ON   = False
# cfg.MODEL.MaskDINO.TEST.PANOPTIC_ON   = False
```

--------------------------------

### MaskDINO Trainer Programmatic Usage

Source: https://context7.com/idea-research/maskdino/llms.txt

Programmatic usage of the custom MaskDINO Trainer class, extending Detectron2's DefaultTrainer. It handles dataset mappers and evaluators based on configuration.

```python
from detectron2.config import get_cfg
from detectron2.projects.deeplab import add_deeplab_config
from maskdino import add_maskdino_config

# train_net.py Trainer usage (programmatic API)
import sys
sys.path.insert(0, ".")
from train_net import Trainer, setup
from detectron2.engine import default_argument_parser

# Simulate args
args = default_argument_parser().parse_args([
    "--config-file", "configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml",
    "--num-gpus", "1",
])
cfg = setup(args)

# Build evaluator for a specific dataset
evaluator = Trainer.build_evaluator(cfg, "coco_2017_val")

# Build train loader (mapper dispatched by cfg.INPUT.DATASET_MAPPER_NAME)
# "coco_instance_lsj"    -> COCOInstanceNewBaselineDatasetMapper (LSJ augmentation)
# "coco_instance_detr"   -> DetrDatasetMapper
# "coco_panoptic_lsj"    -> COCOPanopticNewBaselineDatasetMapper
# "mask_former_semantic" -> MaskFormerSemanticDatasetMapper
train_loader = Trainer.build_train_loader(cfg)

# Build optimizer with backbone LR multiplier and per-layer weight decay
optimizer = Trainer.build_optimizer(cfg, model)
# Backbone params use: lr * cfg.SOLVER.BACKBONE_MULTIPLIER (default 0.1)
# Norm module params: weight_decay = cfg.SOLVER.WEIGHT_DECAY_NORM
# Embedding params:   weight_decay = cfg.SOLVER.WEIGHT_DECAY_EMBED

# Run full training loop
trainer = Trainer(cfg)
trainer.resume_or_load(resume=False)
trainer.train()
```

--------------------------------

### add_maskdino_config

Source: https://context7.com/idea-research/maskdino/llms.txt

Registers MaskDINO-specific configuration defaults into a Detectron2 CfgNode. This function must be called before loading any MaskDINO YAML config files to set up defaults for model architecture, loss weights, inference modes, and training hyperparameters.

```APIDOC
## add_maskdino_config — Register MaskDINO Config Defaults

Registers all MaskDINO-specific configuration keys into a Detectron2 `CfgNode`. Must be called before loading any MaskDINO YAML config file. Sets defaults for the model architecture, loss weights, inference modes, Swin Transformer backbone parameters, and denoising training hyperparameters.

```python
from detectron2.config import get_cfg
from detectron2.projects.deeplab import add_deeplab_config
from maskdino import add_maskdino_config

cfg = get_cfg()
add_deeplab_config(cfg)       # required for poly LR schedule
add_maskdino_config(cfg)      # registers all MODEL.MaskDINO.* keys

# Load a task-specific YAML config
cfg.merge_from_file("configs/coco/instance-segmentation/maskdino_R50_bs16_50ep_3s_dowsample1_2048.yaml")

# Override individual options at runtime
cfg.merge_from_list([
    "MODEL.WEIGHTS", "/path/to/checkpoint.pth",
    "MODEL.MaskDINO.TEST.INSTANCE_ON", "True",
    "MODEL.MaskDINO.TEST.PANOPTIC_ON", "False",
    "MODEL.MaskDINO.INITIALIZE_BOX_TYPE", "mask2box",  # 'no', 'bitmask', or 'mask2box'
])
cfg.freeze()

# Key config nodes and their defaults:
# cfg.MODEL.MaskDINO.HIDDEN_DIM         = 256
# cfg.MODEL.MaskDINO.NUM_OBJECT_QUERIES = 100
# cfg.MODEL.MaskDINO.DEC_LAYERS         = 6
# cfg.MODEL.MaskDINO.NHEADS             = 8
# cfg.MODEL.MaskDINO.TWO_STAGE          = True
# cfg.MODEL.MaskDINO.DN                 = "seg"   # denoising mode
# cfg.MODEL.MaskDINO.DN_NUM             = 100
# cfg.MODEL.MaskDINO.DN_NOISE_SCALE     = 0.4
# cfg.MODEL.MaskDINO.CLASS_WEIGHT       = 4.0
# cfg.MODEL.MaskDINO.MASK_WEIGHT        = 5.0
# cfg.MODEL.MaskDINO.DICE_WEIGHT        = 5.0
# cfg.MODEL.MaskDINO.BOX_WEIGHT         = 5.0
# cfg.MODEL.MaskDINO.GIOU_WEIGHT        = 2.0
# cfg.MODEL.MaskDINO.TEST.SEMANTIC_ON   = True
# cfg.MODEL.MaskDINO.TEST.INSTANCE_ON   = False
# cfg.MODEL.MaskDINO.TEST.PANOPTIC_ON   = False
```
```

--------------------------------

### MaskDINO.forward

Source: https://context7.com/idea-research/maskdino/llms.txt

The main forward pass of the MaskDINO model. During training, it returns a dictionary of weighted losses. During inference, it returns per-image result dictionaries containing 'instances', 'sem_seg', and/or 'panoptic_seg' keys based on the enabled inference modes.

```APIDOC
## MaskDINO.forward — Unified Detection and Segmentation Inference

The main forward pass of the `MaskDINO` model (registered as `META_ARCHITECTURE: "MaskDINO"`). During training it returns a dict of weighted losses. During inference it returns per-image result dicts containing `"instances"`, `"sem_seg"`, and/or `"panoptic_seg"` keys depending on the enabled inference modes.

```python
import torch
from detectron2.config import get_cfg
from detectron2.modeling import build_model
from detectron2.checkpoint import DetectionCheckpointer
from detectron2.projects.deeplab import add_deeplab_config
from detectron2.data.detection_utils import read_image
from maskdino import add_maskdino_config
```
```

--------------------------------

### SetCriterion for Hungarian Matching Loss

Source: https://context7.com/idea-research/maskdino/llms.txt

Computes combined classification, mask, and bounding box losses using bipartite matching. Handles auxiliary and denoising losses. Supports focal and cross-entropy classification losses.

```python
from maskdino.modeling.criterion import SetCriterion
from maskdino.modeling.matcher import HungarianMatcher

# Build matcher with combined cost function
matcher = HungarianMatcher(
    cost_class=4.0,   # classification cost weight
    cost_mask=5.0,    # binary mask focal loss cost
    cost_dice=5.0,    # dice loss cost
    cost_box=5.0,     # L1 box regression cost
    cost_giou=2.0,    # GIoU cost
    num_points=12544, # points sampled for mask cost (112*112)
)

weight_dict = {
    "loss_ce": 4.0, "loss_mask": 5.0, "loss_dice": 5.0,
    "loss_bbox": 5.0, "loss_giou": 2.0,
}
# Add intermediate and denoising loss keys
for i in range(9):   # DEC_LAYERS
    weight_dict.update({f"{k}_{i}": v for k, v in weight_dict.items()})
weight_dict.update({f"{k}_dn": v for k, v in weight_dict.items()})

criterion = SetCriterion(
    num_classes=80,            # COCO categories
    matcher=matcher,
    weight_dict=weight_dict,
    eos_coef=0.1,              # no-object class weight
    losses=["labels", "masks", "boxes"],
    num_points=12544,
    oversample_ratio=3.0,
    importance_sample_ratio=0.75,
    dn="seg",                  # "no", "standard", or "seg"
    dn_losses=["labels", "masks", "boxes"],
    panoptic_on=False,
    semantic_ce_loss=False,
)

# Training step usage:
# outputs, mask_dict = model.sem_seg_head(features, targets=targets)
# losses = criterion(outputs, targets, mask_dict)
# total_loss = sum(losses[k] * criterion.weight_dict[k]
#                  for k in losses if k in criterion.weight_dict)
# total_loss.backward()

# Loss components returned:
# losses["loss_ce"]       - focal classification loss
# losses["loss_mask"]     - binary cross-entropy on sampled points
# losses["loss_dice"]     - DICE loss on sampled points
# losses["loss_bbox"]     - L1 bounding box regression loss
# losses["loss_giou"]     - generalized IoU loss
# + auxiliary layer variants: loss_ce_0 ... loss_ce_8
# + denoising variants:       loss_ce_dn, loss_mask_dn, etc.
```

--------------------------------

### MaskDINO Forward Pass for Inference

Source: https://context7.com/idea-research/maskdino/llms.txt

The main forward pass for the MaskDINO model. During training, it returns weighted losses. During inference, it returns per-image results including 'instances', 'sem_seg', and/or 'panoptic_seg' based on enabled modes.

```python
import torch
from detectron2.config import get_cfg
from detectron2.modeling import build_model
from detectron2.checkpoint import DetectionCheckpointer
from detectron2.projects.deeplab import add_deeplab_config
from detectron2.data.detection_utils import read_image
from maskdino import add_maskdino_config
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.