RF-DETR

https://rfdetr.roboflow.com/
Admin
RF-DETR is a real-time, transformer-based object detection model developed by Roboflow, offering...
Tokens:176,632
Snippets:1,049
Trust Score:10
Update:6 days ago
Show doc for...
Tokens:
Raw
### Install RF-DETR with LoRA extra

Source: https://rfdetr.roboflow.com/latest/getting-started/migration

Update your install command to include the `lora` extra if you use LoRA adapters during training. The `peft` package is no longer installed by default.

```bash
# Before
pip install rfdetr

# After
pip install 'rfdetr[lora]'

```

--------------------------------

### Install RF-DETR augment extra

Source: https://rfdetr.roboflow.com/latest/getting-started/migration

Installs the `augment` extra, which is required to use custom augmentation configurations or the GPU backend. Note that installing this extra alone does not pin the training resize backend; the backend is auto-selected based on what is installed.

```bash
pip install 'rfdetr[augment]'

```

--------------------------------

### Install RF-DETR with LiteRT and ExecuTorch extras

Source: https://rfdetr.roboflow.com/latest/cookbooks/export-mobile

Installs RF-DETR from the develop branch with litert and executorch extras, along with pinned torch, executorch, and other dependencies. Also force-reinstalls pillow and uninstalls torchaudio to avoid conflicts. The commented line shows an alternative PyPI release install.

```bash
RFDETR_SPEC = "rfdetr[litert,executorch] @ https://github.com/roboflow/rf-detr/archive/refs/heads/develop.zip"
# RFDETR_SPEC = "rfdetr[litert,executorch]>=1.11.2,<2"
!pip install -q "{RFDETR_SPEC}" "torch<2.13" "executorch>=1.3.1,<1.4" psutil supervision pandas
!pip install -q --force-reinstall --no-deps "pillow==11.3.0"
!pip uninstall -q -y torchaudio
```

--------------------------------

### Install RF-DETR development version from source

Source: https://rfdetr.roboflow.com/latest/getting-started/install

Install the latest development version directly from the develop branch archive without cloning the repository.

```bash
pip install https://github.com/roboflow/rf-detr/archive/refs/heads/develop.zip
```

--------------------------------

### Install RF-DETR and Dependencies

Source: https://rfdetr.roboflow.com/latest/cookbooks/export-cpu

Installs the RF-DETR library from the develop branch along with essential dependencies like psutil, supervision, and pandas. The installation source can be toggled to a specific PyPI release version if needed.

```bash
RFDETR_SPEC = "rfdetr[onnx,openvino] @ https://github.com/roboflow/rf-detr/archive/refs/heads/develop.zip"
# RFDETR_SPEC = "rfdetr[onnx,openvino]>=1.11.2,<2"
!pip install -q "{RFDETR_SPEC}" psutil supervision pandas
```

--------------------------------

### Install rfdetr[openvino] extra

Source: https://rfdetr.roboflow.com/latest/exports/basics

Install the OpenVINO export dependency. Required before exporting to the 'openvino' format.

```bash
pip install "rfdetr[openvino]"

```

--------------------------------

### Install rfdetr[coreml] extra

Source: https://rfdetr.roboflow.com/latest/exports/basics

Install the CoreML export dependency. Required before exporting to the 'coreml' format.

```bash
pip install "rfdetr[coreml]"

```

--------------------------------

### Install rfdetr[litert] extra

Source: https://rfdetr.roboflow.com/latest/exports/basics

Install the LiteRT export dependency. Required before exporting to the 'litert' format.

```bash
pip install "rfdetr[litert]"

```

--------------------------------

### Install rfdetr[coreai] extra

Source: https://rfdetr.roboflow.com/latest/exports/basics

Install the Core AI export dependency. Required before exporting to the 'coreai' format.

```bash
pip install "rfdetr[coreai]"

```

--------------------------------

### Set up RF-DETR development environment with uv

Source: https://rfdetr.roboflow.com/latest/getting-started/install

Clone the repository, pin Python 3.11, sync the environment, and install the package with all extras.

```bash
# Clone the repository and navigate to the root directory
git clone --depth 1 -b develop https://github.com/roboflow/rf-detr.git
cd rf-detr

# Pin Python version (optional but recommended)
uv python pin 3.11

# Sync environment (creates .venv, installs pinned Python, and installs dependencies)
uv sync

# Install the package in development mode with all extras
uv pip install -e . --all-extras
```

--------------------------------

### Run training with custom optimizer and scheduler using trainer.fit

Source: https://rfdetr.roboflow.com/latest/cookbooks/custom-optimizer-scheduler

Use this snippet to start the training loop with the custom optimizer and scheduler. The optimizer and scheduler are built by RFDETRModelModule.configure_optimizers() during setup, which requires the connected trainer to size the schedule. On this small dataset, 20 epochs is enough to observe cosine LR decay; scale EPOCHS for a real run.

```python
trainer.fit(model, datamodule=datamodule)
```

--------------------------------

### Setup: imports, environment check, and configuration

Source: https://rfdetr.roboflow.com/latest/cookbooks/export-apple

Imports required libraries, checks for Apple Silicon (macOS arm64), sets device to MPS if available, creates export directory, defines thresholds and batch sizes, and initializes results list. Also defines helper functions for batching images and targets.

```python
import importlib.metadata
import platform
from functools import partial
from pathlib import Path

import numpy as np
import supervision as sv
import torch
from PIL import Image

from rfdetr.export._benchmark import (
    BenchmarkResult,
    _artifact_size_mb,
    _enable_notebook_inline_matplotlib,
    _result_row,
    measure_latency,
    measure_memory,
    visualize_detections,
)

if platform.system() != "Darwin" or platform.machine() != "arm64":
    raise RuntimeError("This notebook requires Apple Silicon (macOS, arm64); none is available here.")

print(f"macOS {platform.mac_ver()[0]}, {platform.machine()}")
print(f"rfdetr: {importlib.metadata.version('rfdetr')}")

PYTORCH_DEVICE = "mps" if torch.backends.mps.is_available() else "cpu"
print(f"PyTorch device: {PYTORCH_DEVICE}")

EXPORT_DIR = Path("export_apple")
EXPORT_DIR.mkdir(exist_ok=True)

CONFIDENCE_THRESHOLD = 0.5
WARMUP_RUNS = 5
MEASURE_RUNS = 30
BATCH_SIZES = (1, 4, 8)

#: One row per (format, config, batch) benchmark, appended to as each section below runs; turned
#: into the summary table and DataFrame in §8.
results: list[dict] = []

def _batched_pixel_values(transform, image: Image.Image, batch: int) -> np.ndarray:
    """Repeat one preprocessed sample image into a ``(batch, C, H, W)`` float32 NCHW array.

    A plain latency benchmark, not a detection-diversity test — every fixed-batch format below
    times against N copies of the same image rather than N distinct images.
    """
    tensor, _ = transform(image, None)
    return np.repeat(tensor[None].numpy(), batch, axis=0)

def _batched_pixel_tensor(transform, image: Image.Image, batch: int) -> torch.Tensor:
    """Repeat one preprocessed sample image into a contiguous ``(batch, C, H, W)`` float32 tensor."""
    tensor, _ = transform(image, None)
    return tensor[None].float().contiguous().repeat(batch, 1, 1, 1)

def _batched_target_sizes(image: Image.Image, batch: int) -> torch.Tensor:
    """Repeat the sample image's ``(height, width)`` once per batch element, for ``post_process``."""
    return torch.tensor([[image.height, image.width]] * batch)

_enable_notebook_inline_matplotlib()
```

--------------------------------

### Initialize Benchmarking Environment

Source: https://rfdetr.roboflow.com/latest/cookbooks/export-cpu

Imports necessary benchmarking utilities from rfdetr.export._benchmark and sets up configuration constants for inference testing. This includes defining batch sizes, confidence thresholds, and warmup/measurement run counts.

```python
from pathlib import Path

import numpy as np
import supervision as sv
from PIL import Image

from rfdetr.export._benchmark import (
    BenchmarkResult,
    _artifact_size_mb,
    _decode_batch,
    _enable_notebook_inline_matplotlib,
    _result_row,
    _tile_batch,
    cpu_brand,
    measure_latency,
    measure_memory,
    visualize_detections,
)

EXPORT_DIR = Path("export_cpu")
EXPORT_DIR.mkdir(exist_ok=True)
CONFIDENCE_THRESHOLD = 0.5
WARMUP_RUNS = 15
MEASURE_RUNS = 50
BATCH_SIZES = (1, 4)

_enable_notebook_inline_matplotlib()
```

--------------------------------

### Set up RF-DETR development environment with virtualenv

Source: https://rfdetr.roboflow.com/latest/getting-started/install

Clone the repository, create a Python 3.10 virtual environment, and install the package in development mode.

```bash
# Clone the repository and navigate to the root directory
git clone --depth 1 -b develop https://github.com/roboflow/rf-detr.git
cd rf-detr

# Set up a Python virtual environment with a specific Python version (e.g., 3.10)
python3.10 -m venv venv

# Activate the virtual environment
source venv/bin/activate

# Upgrade pip
pip install --upgrade pip

# Install the package in development mode
pip install -e "."
```

--------------------------------

### Install coremltools for ExecuTorch CoreML backend

Source: https://rfdetr.roboflow.com/latest/exports/executorch

Install the coremltools package required for the CoreML backend. This package is not included in the rfdetr[executorch] extra and must be installed separately.

```bash
pip install coremltools

```

--------------------------------

### Install inference-models with pip

Source: https://rfdetr.roboflow.com/latest/exports/basics

Install the inference-models library. The first command installs CPU/PyTorch support only. The second command adds TensorRT 10 support and requires an NVIDIA GPU.

```bash
# CPU / PyTorch only
pip install inference-models

# With TensorRT support (NVIDIA GPU required)
pip install "inference-models[trt10]"  # TensorRT 10

```

--------------------------------

### Setup imports and environment for Apple export

Source: https://rfdetr.roboflow.com/latest/cookbooks/export-apple

Imports required libraries, checks for Apple Silicon, sets up export directory, and defines shared benchmarking helpers and constants.

```python
import importlib.metadata
import platform
from functools import partial
from pathlib import Path

import numpy as np
import supervision as sv
import torch
from PIL import Image

from rfdetr.export._benchmark import (
    BenchmarkResult,
    _artifact_size_mb,
    _enable_notebook_inline_matplotlib,
    _result_row,
    measure_latency,
    measure_memory,
    visualize_detections,
)

if platform.system() != "Darwin" or platform.machine() != "arm64":
    raise RuntimeError("This notebook requires Apple Silicon (macOS, arm64); none is available here.")
print(f"macOS {platform.mac_ver()[0]}, {platform.machine()}")
print(f"rfdetr: {importlib.metadata.version('rfdetr')}")

PYTORCH_DEVICE = "mps" if torch.backends.mps.is_available() else "cpu"
print(f"PyTorch device: {PYTORCH_DEVICE}")

EXPORT_DIR = Path("export_apple")
EXPORT_DIR.mkdir(exist_ok=True)
CONFIDENCE_THRESHOLD = 0.5
WARMUP_RUNS = 5
MEASURE_RUNS = 30
BATCH_SIZES = (1, 4, 8)

#: One row per (format, config, batch) benchmark, appended to as each section below runs; turned
#: into the summary table and DataFrame in §8.
results: list[dict] = []


def _batched_pixel_values(transform, image: Image.Image, batch: int) -> np.ndarray:
    """Repeat one preprocessed sample image into a ``(batch, C, H, W)`` float32 NCHW array.

    A plain latency benchmark, not a detection-diversity test — every fixed-batch format below times against N copies of
    the same image rather than N distinct images.
    """
    tensor, _ = transform(image, None)
    return np.repeat(tensor[None].numpy(), batch, axis=0)


def _batched_pixel_tensor(transform, image: Image.Image, batch: int) -> torch.Tensor:
    """Repeat one preprocessed sample image into a contiguous ``(batch, C, H, W)`` float32 tensor."""
    tensor, _ = transform(image, None)
    return tensor[None].float().contiguous().repeat(batch, 1, 1, 1)


def _batched_target_sizes(image: Image.Image, batch: int) -> torch.Tensor:
    """Repeat the sample image's ``(height, width)`` once per batch element, for ``post_process``."""
    return torch.tensor([[image.height, image.width]] * batch)


_enable_notebook_inline_matplotlib()

```

--------------------------------

### Install fuse-augmentations package

Source: https://rfdetr.roboflow.com/latest/getting-started/migration

Install the replacement package for the removed synthetic dataset generator.

```bash
pip install fuse-augmentations

```

--------------------------------

### Setup and Configuration for Keypoint Fine-Tuning

Source: https://rfdetr.roboflow.com/latest/cookbooks/fine-tune_keypoints

Defines dataset metadata, training hyperparameters, and output paths for fine-tuning RF-DETR keypoint models. Selects a dataset via DATASET_KEY and prints configuration details. Resolution must be divisible by the model patch_size (12).

```python
PROJECT_ROOT = Path(__file__).resolve().parent if "__file__" in globals() else Path.cwd()
DATASETS_DIR = PROJECT_ROOT / "datasets"

# Resolution must be divisible by the model patch_size (12 for RFDETRKeypointPreview).
# Larger resolutions improve fine-grained keypoint localisation at the cost of GPU memory
# and training time. Recommended choices per dataset are annotated below.
DATASETS: dict[str, dict[str, Any]] = {
    "dart": {
        "name": "Darts Detection",
        "workspace": "dartdetection-vfgjd",
        "project": "darts_detection-yp7lt",
        "version": 5,
        "source_url": "https://universe.roboflow.com/dartdetection-vfgjd/darts_detection-yp7lt",
        "output_name": "keypoint_darts_detection_demo",
        "keypoint_flip_pairs": [],
        # 576 — close-up shots where the dart tip occupies a large fraction of the frame;
        # default resolution provides sufficient detail for tip localisation.
        "resolution": 576,
    },
    "human_pose": {
        "name": "Dataset Ridimensionato",
        "workspace": "poseestimation-wzidb",
        "project": "dataset-ridimensionato",
        "version": 65,
        "source_url": "https://universe.roboflow.com/poseestimation-wzidb/dataset-ridimensionato/dataset/65",
        "output_name": "keypoint_dataset_ridimensionato_demo",
        "format": "yolo",
        "keypoint_flip_pairs": [],
        # 768 — body joints (wrists, ankles) require fine localisation across a range of
        # person scales; a larger canvas reduces positional error for distant people.
        "resolution": 768,
    },
    "basketball_court": {
        "name": "Basketball Court Detection 2",
        "workspace": "roboflow-jvuqo",
        "project": "basketball-court-detection-2",
        "version": 19,
        "source_url": "https://universe.roboflow.com/roboflow-jvuqo/basketball-court-detection-2",
        "output_name": "keypoint_basketball_court_detection_demo",
        "keypoint_flip_pairs": [],
        # 576 — court lines are coarse features that fill the frame; the default resolution
        # is sufficient to locate corners and centre-circle keypoints accurately.
        "resolution": 576,
    },
    "football_field": {
        "name": "Football Field Detection",
        "workspace": "roboflow-jvuqo",
        "project": "football-field-detection-f07vi",
        "version": 18,
        "source_url": "https://universe.roboflow.com/roboflow-jvuqo/football-field-detection-f07vi",
        "output_name": "keypoint_football_field_detection_demo",
        "keypoint_flip_pairs": [],
        # 768 — a full football pitch captured in a single frame compresses many line
        # intersections into a small pixel area; the higher resolution separates
        # closely spaced markings and reduces keypoint confusion near the centre circle.
        "resolution": 768,
    },
    "tennis_court": {
        "name": "Tennis Court Keypoint",
        "workspace": "shj-pk1wt",
        "project": "tennis_court_keypoint",
        "version": 6,
        "source_url": "https://universe.roboflow.com/shj-pk1wt/tennis_court_keypoint",
        "output_name": "keypoint_tennis_court_demo",
        "format": "yolo",
        "keypoint_flip_pairs": [],
        # 768 — a full court viewed from broadcast angles packs many line intersections
        # into a small pixel area; the higher resolution reduces confusion between
        # closely spaced baseline and service-box corners.
        "resolution": 768,
    },
}

DATASET_KEY = "dart"
DATASET_INFO = DATASETS[DATASET_KEY]
DATASET_FORMAT = DATASET_INFO.get("format", "coco")  # "coco" or "yolo"

OUTPUT_DIR = PROJECT_ROOT / "output" / str(DATASET_INFO["output_name"])
METRICS_CSV = OUTPUT_DIR / "metrics.csv"
VALIDATION_METRICS_JSON = OUTPUT_DIR / "validation_metrics.json"
FINAL_CHECKPOINT_PATH = OUTPUT_DIR / "checkpoint_final_demo.pth"

RESOLUTION = int(DATASET_INFO["resolution"])

SEED = 7
EPOCHS = 50
BATCH_SIZE = 8
GRAD_ACCUM_STEPS = 2
NUM_WORKERS = 8
LR = 1e-4
LR_ENCODER = 1e-4
SAMPLE_PREVIEW_COUNT = 6
SAMPLE_PREVIEW_COLUMNS = 3
SAMPLE_PREVIEW_FIGURE_SIZE: tuple[float, float] | None = None
INFERENCE_COUNT = 6
INFERENCE_COLUMNS = 3
INFERENCE_THRESHOLD = 0.25
KEYPOINT_THRESHOLD = 0.1
PLOT_LOSS_LOG_SCALE = False
DRAW_UNCERTAINTY_ELLIPSES = True
ELLIPSE_SIGMA = 1.0
MAX_ELLIPSE_RADIUS = 36.0

print(f"dataset_key={DATASET_KEY}")
print(f"dataset={DATASET_INFO['name']}")
print(f"dataset_format={DATASET_FORMAT}")
print(f"source_url={DATASET_INFO['source_url']}")
print(f"resolution={RESOLUTION}")
```

--------------------------------

### Install RF-DETR with Apple extras

Source: https://rfdetr.roboflow.com/latest/cookbooks/export-apple

Installs RF-DETR from the develop branch with CoreML, Core AI, and ExecuTorch extras, plus psutil, supervision, and pandas. A commented alternative installs the 1.11.2 release from PyPI.

```python
RFDETR_SPEC = "rfdetr[coreml,coreai,executorch] @ https://github.com/roboflow/rf-detr/archive/refs/heads/develop.zip"
# RFDETR_SPEC = "rfdetr[coreml,coreai,executorch]>=1.11.2,<2"
!pip install -q "{RFDETR_SPEC}" psutil supervision pandas

```

--------------------------------

### Setup shared helpers and constants for Part B

Source: https://rfdetr.roboflow.com/latest/cookbooks/export-mobile

Imports benchmarking utilities and defines export directory, confidence threshold, warmup/measure runs, and batch sizes. This is a fresh import after runtime restart, not a redefinition of Part A's imports.

```python
from pathlib import Path

import numpy as np
import pandas as pd
import supervision as sv
from IPython.display import display
from PIL import Image

from rfdetr.export._benchmark import (
    BenchmarkResult,
    _artifact_size_mb,
    _enable_notebook_inline_matplotlib,
    _result_row,
    cpu_brand,
    measure_latency,
    measure_memory,
    visualize_detections,
)
from rfdetr.export.benchmark import (  # noqa: F811 -- Part B's own kernel, not a re-import of Part A's
    infer_transforms,
    post_process,
)

EXPORT_DIR = Path("export_mobile")
EXPORT_DIR.mkdir(exist_ok=True)
CONFIDENCE_THRESHOLD = 0.5
WARMUP_RUNS = 5
MEASURE_RUNS = 30
BATCH_SIZES = (1, 4)

_enable_notebook_inline_matplotlib()
```

--------------------------------

### Install rfdetr with pip

Source: https://rfdetr.roboflow.com/

Install the rfdetr package in a Python >=3.10 environment using pip.

```bash
pip install rfdetr
```

--------------------------------

### Set up dataset directory structure

Source: https://rfdetr.roboflow.com/latest/cookbooks/custom-augmentations

Creates a coco_demo directory, moves the train and validation image folders into it, and copies the annotation JSON files to the expected _annotations.coco.json names. This structure is required by model.train().

```bash
!mkdir -p coco_demo
!mv train2017 coco_demo/train
!mv val2017   coco_demo/valid
!cp annotations/instances_train2017.json coco_demo/train/_annotations.coco.json
!cp annotations/instances_val2017.json   coco_demo/valid/_annotations.coco.json

```
Tokens:
Raw