Caliscope

https://mprib.github.io/caliscope/
Admin
Caliscope is a permissively licensed multicamera calibration tool that produces intrinsic and...
Tokens:10,037
Snippets:142
Trust Score:9.7
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install uv on Windows

Source: https://mprib.github.io/caliscope/installation

Install uv on Windows using PowerShell. This command downloads and executes the installation script.

```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

```

--------------------------------

### Install GUI Prerequisites on Ubuntu/Debian

Source: https://mprib.github.io/caliscope/installation

Install necessary packages for GUI display on Ubuntu/Debian systems before creating the virtual environment. This ensures all graphical dependencies are met.

```bash
# Install prerequisite packages for GUI display (Ubuntu/Debian)
sudo apt-get update
sudo apt-get install --fix-missing libgl1-mesa-dev

```

--------------------------------

### Install uv on macOS & Linux

Source: https://mprib.github.io/caliscope/installation

Install uv on macOS and Linux systems using curl. This command fetches and runs the installation script.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh

```

--------------------------------

### Check uv Installation

Source: https://mprib.github.io/caliscope/installation

Verify if uv is already installed on your system. If a version number is displayed, you can proceed to the next step.

```bash
uv --version

```

--------------------------------

### Install Caliscope Library

Source: https://mprib.github.io/caliscope/scripting

Install the Caliscope library for scripting or with GUI and visualization tools.

```bash
uv pip install caliscope        # library only
uv pip install caliscope[gui]   # adds desktop app and 3D visualization
```

--------------------------------

### Install Caliscope with GUI

Source: https://mprib.github.io/caliscope/installation

Install the Caliscope desktop application, including GUI and 3D visualization dependencies. This command should be run after activating your virtual environment.

```bash
uv pip install caliscope[gui]

```

--------------------------------

### Install Caliscope for Scripting

Source: https://mprib.github.io/caliscope/installation

Install Caliscope for scripting and library use, without GUI dependencies. This is a lighter installation suitable for backend tasks.

```bash
uv pip install caliscope

```

--------------------------------

### Launch Caliscope GUI

Source: https://mprib.github.io/caliscope/installation

Launch the main Caliscope graphical user interface from the command line. Ensure your virtual environment is activated and Caliscope is installed.

```bash
caliscope

```

--------------------------------

### Example ONNX Model Card Configuration

Source: https://mprib.github.io/caliscope/onnx_trackers

This TOML file defines the configuration for an ONNX pose estimation model, including display name, model path, input size, output format, confidence threshold, and keypoint mappings.

```toml
[model]
# Display name in the GUI (optional, defaults to the .onnx filename if omitted)
name = "RTMPose-t Halpe26"

# Absolute path to your ONNX model file (required)
model_path = "/home/user/models/rtmpose-t-halpe26.onnx"

# Output format: "simcc" or "heatmap" (required)
format = "simcc"

# Model input dimensions as [width, height] (required)
input_size = [192, 256]

# Minimum confidence to report a point (optional, default 0.3)
# Lower values include more detections but may increase false positives
confidence_threshold = 0.3

# Keypoint name-to-index mapping (required)
# Maps human-readable names to the model's output indices
[points]
nose = 0
left_eye = 1
right_eye = 2
left_ear = 3
right_ear = 4
left_shoulder = 5
right_shoulder = 6
left_elbow = 7
right_elbow = 8
left_wrist = 9
right_wrist = 10
left_hip = 11
right_hip = 12
left_knee = 13
right_knee = 14
left_ankle = 15
right_ankle = 16
head = 17
neck = 18
hip = 19
left_big_toe = 20
right_big_toe = 21
left_small_toe = 22
right_small_toe = 23
left_heel = 24
right_heel = 25

# Wireframe segments for 3D visualization (optional)

```

--------------------------------

### Get Caliscope Models Directory Path

Source: https://mprib.github.io/caliscope/onnx_trackers

Use the MODELS_DIR constant from the caliscope library to programmatically retrieve the path to the models directory.

```python
from caliscope import MODELS_DIR
print(MODELS_DIR)
```

--------------------------------

### Create and Activate Virtual Environment on Windows

Source: https://mprib.github.io/caliscope/installation

Set up a Python virtual environment using uv on Windows. Navigate to your project directory, create the environment with a specified Python version, and then activate it.

```bash
# Navigate to the directory that will hold your project
cd path\to\your\project

# Create a virtual environment using Python 3.12
uv venv --python 3.12

# Activate the virtual environment
.\.venv\Scripts\activate

```

--------------------------------

### Step 5: Bootstrap and optimize

Source: https://mprib.github.io/caliscope/scripting

Bootstrap the CaptureVolume using the extracted points and calibrated cameras to estimate initial positions. Then, optimize the volume using bundle adjustment. Outliers can be filtered by error percentile, followed by a re-optimization step for improved accuracy.

```APIDOC
volume = CaptureVolume.bootstrap(ext_points, cameras)
volume = volume.optimize(strict=False)

# Filter outliers and re-optimize
volume = volume.filter_by_percentile_error(2.5)
volume = volume.optimize(strict=False)
```

--------------------------------

### Create and Activate Virtual Environment on macOS/Linux

Source: https://mprib.github.io/caliscope/installation

Set up a Python virtual environment using uv on macOS or Linux. Navigate to your project directory, create the environment with a specified Python version, and then activate it.

```bash
# Navigate to the directory that will hold your project
cd path/to/your/project

# Create a virtual environment using Python 3.12
uv venv --python 3.12

# Activate the virtual environment
source .venv/bin/activate

```

--------------------------------

### Bootstrap and Optimize Capture Volume

Source: https://mprib.github.io/caliscope/scripting

Estimate initial camera positions and triangulate 3D points using bootstrap, then optimize the capture volume with bundle adjustment. Set strict=False for less restrictive optimization.

```python
volume = CaptureVolume.bootstrap(ext_points, cameras)
volume = volume.optimize(strict=False)
```

--------------------------------

### Define Charuco Board and Tracker

Source: https://mprib.github.io/caliscope/scripting

Create a Charuco board object with specified dimensions and square size, then initialize a CharucoTracker. Ensure square_size_cm matches your physical board for accurate scaling.

```python
charuco = Charuco.from_squares(columns=4, rows=5, square_size_cm=3.0)
tracker = CharucoTracker(charuco)
```

--------------------------------

### Initial Caliscope Project Structure

Source: https://mprib.github.io/caliscope/project_setup

This is the default directory structure created by Caliscope when a new project is initialized. It includes directories for calibration targets, intrinsic and extrinsic calibration videos, and motion capture recordings.

```bash
workspace/
├── calibration/
│   ├── targets/          # Auto-created calibration target configurations
│   ├── intrinsic/        # Per-camera calibration videos (unsynchronized)
│   └── extrinsic/        # Multi-camera calibration videos (synchronized)
└── recordings/           # Motion capture sessions

```

--------------------------------

### Intrinsic Calibration Directory Structure

Source: https://mprib.github.io/caliscope/project_setup

Place individual camera calibration videos in the `calibration/intrinsic/` directory. These videos do not need to be synchronized and should each display a calibration target.

```bash
workspace/
└── calibration/
    └── intrinsic/
        ├── cam_0.mp4     # Individual camera recordings
        ├── cam_1.mp4     # No synchronization required
        └── cam_2.mp4

```

--------------------------------

### Extrinsic Calibration Directory Structure

Source: https://mprib.github.io/caliscope/project_setup

Place synchronized videos for extrinsic calibration in the `calibration/extrinsic/` directory. This directory can also optionally contain a `timestamps.csv` file for precise frame synchronization.

```bash
workspace/
└── calibration/
    └── extrinsic/
        ├── cam_0.mp4
        ├── cam_1.mp4
        ├── cam_2.mp4
        └── timestamps.csv      # Optional: per-frame timing data

```

--------------------------------

### Import Caliscope API Modules

Source: https://mprib.github.io/caliscope/scripting

Import necessary classes and functions from the caliscope.api and caliscope.reporting modules for calibration tasks.

```python
from pathlib import Path
from caliscope.api import (
    Charuco, CharucoTracker, CameraArray,
    CaptureVolume, extract_image_points, extract_image_points_multicam,
    calibrate_intrinsics,
)
from caliscope.reporting import (
    print_intrinsic_report, print_extrinsic_report, print_camera_pair_coverage,
)
```

--------------------------------

### Model Source for In-App Download

Source: https://mprib.github.io/caliscope/onnx_trackers

Configure automatic downloading of model weights. Specify the URL, extraction method, and optional license and integrity information.

```toml
[source]
url = "https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/onnx_sdk/rtmpose-t_simcc-body7_pt-body7-halpe26_700e-256x192-6020f8a6_20230605.zip"
extraction = "zip_end2end"
license = "Apache-2.0"
license_url = "https://github.com/open-mmlab/mmpose/blob/main/LICENSE"
file_size_mb = 13
sha256 = "de5fa6ef754e1b19a0f8199d53affef122813e30c580c48be87fcf86c4ec47a7"
```

--------------------------------

### Extrinsic Calibration Output Structure

Source: https://mprib.github.io/caliscope/project_setup

This structure shows the output files generated after successful extrinsic calibration. It includes raw video, timestamps, extracted feature points, and the final calibration result in camera_array.toml.

```bash
workspace/
└── calibration/
    └── extrinsic/
        ├── cam_0.mp4
        ├── cam_1.mp4
        ├── cam_2.mp4
        ├── timestamps.csv           # If per-frame timing was provided
        ├── inferred_timestamps.csv  # Caliscope's timing assumptions when no timestamps.csv provided (not read back)
        ├── CHARUCO/                 # Extraction output (tracker name varies)
        │   └── image_points.csv
        └── capture_volume/          # Calibration result
            ├── camera_array.toml
            ├── image_points.csv
            └── world_points.csv
```

--------------------------------

### Recording Session Structure

Source: https://mprib.github.io/caliscope/project_setup

This is the directory structure for a motion capture recording session. It includes synchronized video files and an optional timestamps.csv for per-frame timing.

```bash
workspace/
└── recordings/
    └── walking_trial/              # Name the folder descriptively
        ├── cam_0.mp4
        ├── cam_1.mp4
        ├── cam_2.mp4
        └── timestamps.csv          # Optional: same format as extrinsic
```

--------------------------------

### Build Camera Array from Video Metadata

Source: https://mprib.github.io/caliscope/scripting

Initialize a CameraArray by providing a dictionary of intrinsic video paths mapped to camera IDs. This reads resolution and frame rate, creating uncalibrated CameraData entries.

```python
intrinsic_videos = {0: Path("intrinsic/cam_0.mp4"), 1: Path("intrinsic/cam_1.mp4")}
cameras = CameraArray.from_video_metadata(intrinsic_videos)
```

--------------------------------

### Motion Tracking Output Structure

Source: https://mprib.github.io/caliscope/project_setup

This structure displays the output files after processing a recording session with a motion tracking system. It includes the calibration snapshot, various coordinate files, and a TRC file for biomechanical software.

```bash
workspace/
└── recordings/
    └── walking_trial/
        ├── cam_0.mp4
        ├── cam_1.mp4
        ├── cam_2.mp4
        ├── timestamps.csv                  # If provided
        └── ONNX_rtmpose_t_halpe26/          # Output subdirectory (tracker name)
            ├── camera_array.toml            # Snapshot of calibration used
            ├── xy_ONNX_rtmpose_t_halpe26.csv
            ├── xyz_ONNX_rtmpose_t_halpe26.csv
            ├── xyz_ONNX_rtmpose_t_halpe26_labelled.csv
            └── xyz_ONNX_rtmpose_t_halpe26.trc
```

--------------------------------

### Check Camera Pair Coverage

Source: https://mprib.github.io/caliscope/scripting

Print a coverage report for camera pairs using the extracted extrinsic points to identify potential overlap issues before calibration.

```python
print_camera_pair_coverage(ext_points)
```

--------------------------------

### Inspect Extrinsic Calibration Results

Source: https://mprib.github.io/caliscope/scripting

Print a detailed report of the extrinsic calibration results, including optimization status, reprojection errors, and scale accuracy metrics.

```python
print_extrinsic_report(volume)
```

--------------------------------

### Control Extraction Progress Reporting

Source: https://mprib.github.io/caliscope/scripting

Demonstrates different ways to control progress bar output during data extraction. Use 'None' for silent operation or a custom callback object.

```python
# Default: Rich progress bar (auto-created, auto-cleaned-up)
points = extract_image_points(video, cam_id, tracker)
```

```python
# Silent: no output
points = extract_image_points(video, cam_id, tracker, progress=None)
```

```python
# Custom: implement the ProgressCallback protocol
points = extract_image_points(video, cam_id, tracker, progress=my_callback)
```

--------------------------------

### Extract Time-Aligned Points with Timestamps File

Source: https://mprib.github.io/caliscope/scripting

Extract image points from multiple extrinsic videos, using a provided timestamps CSV file for frame synchronization if hardware synchronization is not available.

```python
ext_points = extract_image_points_multicam(extrinsic_videos, tracker, timestamps="timestamps.csv")
```

--------------------------------

### Step 4: Extract time-aligned points for extrinsic calibration

Source: https://mprib.github.io/caliscope/scripting

Extract image points from multiple videos simultaneously, aligning frames by timestamp. This function produces a DataFrame with observations from all cameras, which is essential for extrinsic calibration. Optionally, a timestamps file can be provided for non-hardware-synchronized cameras.

```APIDOC
extrinsic_videos = {0: Path("extrinsic/cam_0.mp4"), 1: Path("extrinsic/cam_1.mp4")}
ext_points = extract_image_points_multicam(extrinsic_videos, tracker)

# If cameras were not hardware-synchronized:
ext_points = extract_image_points_multicam(extrinsic_videos, tracker, timestamps="timestamps.csv")
```

--------------------------------

### Align Capture Volume to Object Coordinates

Source: https://mprib.github.io/caliscope/scripting

Align the capture volume to object coordinates using a similarity transform based on a chosen sync index. This sets the world coordinate frame relative to the calibration board.

```python
sync_idx = volume.unique_sync_indices[len(volume.unique_sync_indices) // 2]
volume = volume.align_to_object(sync_idx)
```

--------------------------------

### Save and Load Capture Volume

Source: https://mprib.github.io/caliscope/scripting

Saves the current capture volume to disk, creating necessary files. Reloads a previously saved capture volume and its associated camera array.

```python
volume.save("capture_volume")
```

```python
volume = CaptureVolume.load("capture_volume")
cameras = CameraArray.from_toml("capture_volume/camera_array.toml")
```

--------------------------------

### timestamps.csv Format for Frame Synchronization

Source: https://mprib.github.io/caliscope/project_setup

The `timestamps.csv` file provides per-frame timing data for synchronization. It requires `cam_id` and `frame_time` columns, allowing for asynchronous cameras and varying frame counts.

```csv
cam_id,frame_time
0,927387.33536115
1,927387.50128975
2,927387.3530109001
0,927387.50643105
1,927387.51819965
2,927387.5063038999
0,927387.6684489499
1,927387.66848565
...

```

--------------------------------

### Perform Intrinsic Calibration

Source: https://mprib.github.io/caliscope/scripting

Iterate through intrinsic videos to extract image points using the tracker, then calibrate intrinsics for each camera. Process every Nth frame using frame_step for efficiency.

```python
for cam_id, video in intrinsic_videos.items():
    points = extract_image_points(video, cam_id, tracker, frame_step=5)
    output = calibrate_intrinsics(points, cameras[cam_id])
    cameras[cam_id] = output.camera

    print_intrinsic_report(output)
```

--------------------------------

### Extract Time-Aligned Points for Extrinsic Calibration

Source: https://mprib.github.io/caliscope/scripting

Extract image points from multiple extrinsic videos concurrently, aligning frames by timestamp. This creates a single DataFrame with observations from all cameras.

```python
extrinsic_videos = {0: Path("extrinsic/cam_0.mp4"), 1: Path("extrinsic/cam_1.mp4")}
ext_points = extract_image_points_multicam(extrinsic_videos, tracker)
```

--------------------------------

### Wireframe Segment Definitions

Source: https://mprib.github.io/caliscope/onnx_trackers

Define connections between keypoints for 3D visualization. Each segment requires a color and a list of two point names.

```toml
[segments.shoulders]
color = "y"  # Matplotlib color string (single char or full name)
points = ["left_shoulder", "right_shoulder"]

[segments.left_arm]
color = "g"
points = ["left_shoulder", "left_elbow"]

[segments.left_forearm]
color = "g"
points = ["left_elbow", "left_wrist"]

[segments.right_arm]
color = "r"
points = ["right_shoulder", "right_elbow"]

[segments.right_forearm]
color = "r"
points = ["right_elbow", "right_wrist"]

[segments.pelvis]
color = "y"
points = ["left_hip", "right_hip"]

[segments.left_flank]
color = "y"
points = ["left_hip", "left_shoulder"]

[segments.right_flank]
color = "y"
points = ["right_hip", "right_shoulder"]

[segments.left_thigh]
color = "g"
points = ["left_hip", "left_knee"]

[segments.left_shank]
color = "g"
points = ["left_knee", "left_ankle"]

[segments.right_thigh]
color = "r"
points = ["right_hip", "right_knee"]

[segments.right_shank]
color = "r"
points = ["right_knee", "right_ankle"]

[segments.neck_segment]
color = "y"
points = ["neck", "head"]
```

--------------------------------

### Filter Outliers and Re-optimize

Source: https://mprib.github.io/caliscope/scripting

Remove a specified percentile of outlier observations from the volume and re-optimize to improve accuracy. This is typically done after an initial optimization pass.

```python
volume = volume.filter_by_percentile_error(2.5)
volume = volume.optimize(strict=False)
```
Tokens:
Raw