### Install Dependencies

Source: https://github.com/nicolasdiolez/360extractor/blob/main/README.md

Installs project dependencies using pip. Ensure you have a requirements.txt file.

```bash
pip install -r requirements.txt
```

--------------------------------

### Launch GUI Mode

Source: https://github.com/nicolasdiolez/360extractor/blob/main/README.md

Starts the graphical user interface for interactive video and image processing. This is the recommended mode for ease of use.

```bash
python3 src/main.py
```

--------------------------------

### Configuration File Execution Example

Source: https://github.com/nicolasdiolez/360extractor/blob/main/docs/CLI.md

Run the extraction process using a predefined configuration stored in a JSON file. This is useful for complex or repeatable jobs.

```bash
python src/main.py --config my_job.json
```

--------------------------------

### JSON Configuration File Example

Source: https://github.com/nicolasdiolez/360extractor/blob/main/docs/SETTINGS.md

Use this JSON structure to define job settings for batch processing or CLI mode. CLI arguments will override any settings present in this file.

```json
{
    "input": "videos/holiday.mp4",
    "output": "processed/holiday",
    "interval": 2.0,
    "format": "png",
    "camera_count": 6,
    "active_cameras": [0, 1, 2, 3],
    "resolution": 2048,
    "quality": 100,
    "ai_mask": true,
    "adaptive": false,
    "naming_mode": "realityscan"
}
```

--------------------------------

### Basic Extraction Example

Source: https://github.com/nicolasdiolez/360extractor/blob/main/docs/CLI.md

Extract frames from a video file at a specified interval. Ensure the input video path and output directory are correctly provided.

```bash
python src/main.py --input videos/trip.mp4 --output frames/trip --interval 1.0
```

--------------------------------

### Verify Environment Dependencies

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Runs the `check_env.py` script to verify that all necessary dependencies and external tools are installed and correctly configured. The expected output indicates a ready environment.

```bash
python3 check_env.py
# Expected output (all passing):
# [OK] Python 3.11.4
# [OK] PySide6 6.7.0
# [OK] OpenCV 4.9.0
# [OK] NumPy 1.26.4
# [OK] PyTorch 2.3.0
# [OK] Ultralytics (YOLO26)
# [OK] piexif
# [OK] tqdm
# [OK] ffmpeg  (found at /usr/local/bin/ffmpeg)
# [OK] ffprobe (found at /usr/local/bin/ffprobe)
# ✅ Environment ready.
```

--------------------------------

### Selective Camera Extraction Example

Source: https://github.com/nicolasdiolez/360extractor/blob/main/docs/CLI.md

Extract frames from specific cameras within a multi-camera setup. Camera indices are 0-based.

```bash
python src/main.py --input videos/trip.mp4 --output frames/trip --active-cameras "0,2"
```

--------------------------------

### 360 Extractor Pro JSON Configuration

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Example of a JSON configuration file for defining job settings. CLI flags can override these settings.

```json
{
    "input": "videos/holiday.mp4",
    "output": "processed/holiday",
    "interval": 2.0,
    "format": "png",
    "camera_count": 6,
    "active_cameras": [0, 1, 2, 3],
    "resolution": 2048,
    "quality": 100,
    "layout_mode": "cube",
    "fov": 90,
    "pitch_offset": -20,
    "ai_mask": true,
    "ai_mode": "Generate Mask",
    "ai_confidence": 0.3,
    "ai_detect_humans": true,
    "ai_detect_vehicles": true,
    "ai_detect_plants": false,
    "adaptive": false,
    "adaptive_threshold": 5.0,
    "blur_filter_enabled": true,
    "blur_threshold": 100.0,
    "smart_blur_enabled": true,
    "export_telemetry": true,
    "interpolation_mode": "lanczos",
    "feather_mask": true,
    "naming_mode": "realityscan"
}
```

--------------------------------

### Run 360 Extractor Pro in CLI Mode

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Examples of running the 360 Extractor Pro in headless CLI mode with various options for input, output, interval, layout, AI masking, and telemetry export.

```bash
python3 src/main.py --input videos/tour.mp4 --output frames/tour --interval 1.0
```

```bash
python3 src/main.py \
  --input videos/gopro360.mp4 \
  --output frames/gopro \
  --interval 0.5 \
  --format png \
  --layout cube \
  --resolution 2048 \
  --ai-mask \
  --export-telemetry
```

```bash
python3 src/main.py \
  --input videos/ \
  --output frames/ \
  --adaptive \
  --motion-threshold 5.0 \
  --camera-count 8 \
  --active-cameras "0,2,4,6" \
  --layout ring
```

```bash
python3 src/main.py \
  --input videos/walk.mp4 \
  --output frames/walk \
  --naming-mode custom \
  --image-pattern "{filename}_{frame}_{camera}" \
  --mask-pattern "{filename}_{frame}_{camera}_mask"
```

```bash
python3 src/main.py --config job.json --interval 2.0
```

--------------------------------

### Get Rotation Matrix with GeometryProcessor

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Returns a 3x3 rotation matrix for a given yaw, pitch, and roll. This is used internally by `create_rectilinear_map` but can be used for custom projection math. Requires NumPy.

```python
import numpy as np
from core.geometry import GeometryProcessor

# Rotation looking 45° right and 20° down
R = GeometryProcessor.get_rotation_matrix(yaw_deg=45, pitch_deg=-20, roll_deg=0)
# R is a (3, 3) float64 ndarray

# Apply to a direction vector (looking forward = [0, 0, 1])
forward = np.array([0.0, 0.0, 1.0])
rotated = R @ forward
print(f"Rotated forward vector: {rotated}")
# Output: array representing direction 45° right, 20° down in world space

print(f"Rotation matrix:\n{np.round(R, 4)}")
```

--------------------------------

### Basic CLI Syntax

Source: https://github.com/nicolasdiolez/360extractor/blob/main/docs/CLI.md

The fundamental command structure for running the CLI application. Specify input video and output directory, with optional arguments.

```bash
python src/main.py --input <video_path> --output <output_dir> [options]
```

--------------------------------

### CLI Entry Point

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

The `src/main.py` script serves as the unified entry point for the 360 Extractor Pro. It can be run in GUI mode or headless CLI mode with various options for input, output, and processing.

```APIDOC
## CLI Entry Point

`src/main.py` is the unified entry point. If `--input` or `--config` is passed, it runs in headless CLI mode; otherwise it launches the PySide6 GUI.

```bash
# GUI mode
python3 src/main.py

# CLI mode — basic single video extraction
python3 src/main.py --input videos/tour.mp4 --output frames/tour --interval 1.0

# CLI mode — cube layout, AI mask generation, telemetry export
python3 src/main.py \
  --input videos/gopro360.mp4 \
  --output frames/gopro \
  --interval 0.5 \
  --format png \
  --layout cube \
  --resolution 2048 \
  --ai-mask \
  --export-telemetry

# CLI mode — batch directory, adaptive keyframing, selective cameras
python3 src/main.py \
  --input videos/ \
  --output frames/ \
  --adaptive \
  --motion-threshold 5.0 \
  --camera-count 8 \
  --active-cameras "0,2,4,6" \
  --layout ring

# CLI mode — custom output naming pattern
python3 src/main.py \
  --input videos/walk.mp4 \
  --output frames/walk \
  --naming-mode custom \
  --image-pattern "{filename}_{frame}_{camera}" \
  --mask-pattern "{filename}_{frame}_{camera}_mask"

# CLI mode — load full job from JSON config, override interval
python3 src/main.py --config job.json --interval 2.0
```
```

--------------------------------

### Run 360 Extractor Pro with JSON Config and Expected Output

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Demonstrates running the CLI with a JSON configuration file and shows the expected informational output during processing.

```bash
python3 src/main.py --config job.json
# Expected output:
# [INFO] Loaded configuration from job.json
# [INFO] Found 1 file(s) to process.
# [INFO] Extracting telemetry for holiday.mp4...
# [INFO] Found telemetry stream: gpmd
# [INFO] Extracted 1842 GPS samples.
# [INFO] Loading AI Model: yolo26n-seg.pt on mps...
# [INFO] ✅ ACTIVE AI MODEL: yolo26n-seg.pt
# Processing holiday.mp4 |████████████| 100/100% [02:15<00:00]
```

--------------------------------

### Define a Job with Settings

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Creates a Job dataclass instance, bundling a video file path with a dictionary of processing settings. Accessing settings via typed properties provides safe defaults and clear parameter access.

```python
from core.job import Job

job = Job(
    file_path="videos/insta360_x3.mp4",
    settings={
        'resolution': 3072,
        'camera_count': 8,
        'layout_mode': 'ring',
        'pitch_offset': -20,       # High/Perch (camera on stick)
        'output_format': 'png',
        'adaptive_mode': True,
        'adaptive_threshold': 3.0,
        'export_telemetry': True,
        'active_cameras': [0, 2, 4, 6],  # Front, Back, Left, Right of ring
        'interpolation_mode': 'lanczos',
        'feather_mask': False,
        'naming_mode': 'simple',
        'custom_output_dir': '/mnt/nas/photogrammetry/session_01'
    }
)

print(job.filename)          # 'insta360_x3.mp4'
print(job.resolution)        # 3072
print(job.active_cameras)    # [0, 2, 4, 6]
print(job.export_telemetry)  # True
print(job.adaptive_mode)     # True
print(job.interpolation_mode) # 'lanczos'
print(job.summary())         # 'High (-20°), 8 cams (Ring) [Adaptive]'
```

--------------------------------

### Initialize AIService and Detect GPU

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Initializes the `AIService` for object detection and segmentation using YOLOv26. It automatically detects and selects the best available hardware acceleration (MPS > CUDA > CPU) and loads the specified model. Requires the `core.ai_model` module.

```python
from core.ai_model import AIService

# Check hardware before loading model
device_info = AIService.get_device_info()
# {'device': 'mps', 'device_name': 'Apple Silicon GPU (MPS)', 'is_accelerated': True}

print(f"Running on: {device_info['device_name']}")
print(f"Accelerated: {device_info['is_accelerated']}")

# Initialize — loads yolo26n-seg.pt (nano, NMS-free, fastest)
ai = AIService('yolo26n-seg.pt')
# [INFO] ✓ GPU detected: Apple Silicon GPU (MPS)
# [INFO] Loading AI Model: yolo26n-seg.pt on mps...
# [INFO] ✅ ACTIVE AI MODEL: yolo26n-seg.pt

# Check if GPU is available (class method, no instance required)
if AIService.is_gpu_available():
    print("GPU acceleration active")
```

--------------------------------

### Directory Structure Overview

Source: https://github.com/nicolasdiolez/360extractor/blob/main/ARCHITECTURE.md

Provides a hierarchical view of the project's files and directories, indicating the purpose of key modules and scripts.

```text
360Extractor/
├── src/
│   ├── main.py                 # Entry point (GUI/CLI router)
│   ├── ui/                     # GUI Layer
│   │   ├── main_window.py      # Main window (Persistent Queue/Preview)
│   │   ├── sidebar.py          # Navigation sidebar
│   │   ├── video_card.py       # Job card component
│   │   ├── preview_widget.py   # Persistent preview panel
│   │   ├── log_panel.py        # Log viewer component
│   │   ├── icons.py            # SVG Icon assets
│   │   ├── styles.qss          # Modern dark theme stylesheet
│   │   └── widgets.py          # Shared widgets
│   ├── core/                   # Processing Core
│   │   ├── processor.py        # Extraction Loop & Naming Logic
│   │   ├── geometry.py         # Projection Math
│   │   ├── telemetry.py        # GPS/IMU Manager
│   │   ├── motion_detector.py  # Optical Flow Logic
│   │   └── ai_model.py         # YOLO Wrapper
│   └── utils/
│       ├── gpmf_parser.py      # Binary GPMF Logic
│       ├── camm_parser.py      # Binary CAMM Logic
│       ├── srt_parser.py       # DJI Metadata Logic
│       └── gpx_parser.py       # GPX Sidecar Parser
│   ├── core/                   # Processing Core
│   │   ├── processor.py        # Extraction Loop
│   │   ├── geometry.py         # Projection Math
│   │   ├── telemetry.py        # GPS/IMU Manager (+ GPX sidecar)
│   │   ├── motion_detector.py  # Optical Flow Logic
│   │   └── ai_model.py         # YOLO Wrapper
│   └── utils/
│       ├── gpmf_parser.py      # Binary GPMF Logic
│       ├── camm_parser.py      # Binary CAMM Logic
│       ├── srt_parser.py       # DJI Metadata Logic
│       └── gpx_parser.py       # GPX Sidecar Parser
├── docs/                       # Protocole & Handbooks
├── requirements.txt
└── ARCHITECTURE.md
```

--------------------------------

### Initialize Processing Worker with Job

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Sets up the core extraction pipeline worker with a list of jobs, each defined with comprehensive settings.

```python
from core.job import Job
from core.processor import ProcessingWorker

# Define a job with full settings
job = Job(
    file_path="videos/360_tour.mp4",
    settings={
        'interval_value': 1.0,
        'interval_unit': 'Seconds',
        'output_format': 'jpg',
        'camera_count': 6,
        'layout_mode': 'cube',
        'resolution': 2048,
        'fov': 90,
        'pitch_offset': 0,
        'quality': 95,
        'ai_mode': 'Generate Mask',
        'ai_confidence': 0.25,
        'ai_detect_humans': True,
        'ai_detect_vehicles': False,
        'ai_detect_plants': False,
        'ai_custom_classes': '',
        'ai_invert_mask': True,
        'feather_mask': True,
        'blur_filter_enabled': True,
        'smart_blur_enabled': True,
        'blur_threshold': 100.0,
        'sharpening_enabled': True,
        'sharpening_strength': 0.5,
        'adaptive_mode': True,
        'adaptive_threshold': 5.0,
        'export_telemetry': True,
        'interpolation_mode': 'lanczos',
        'naming_mode': 'realityscan',
        'custom_output_dir': 'output/tour',
        'active_cameras': [0, 1, 2, 3, 4, 5]  # all cube faces
    }
)

worker = ProcessingWorker(jobs=[job])
```

--------------------------------

### Python CLI Entry Point with Argparse

Source: https://github.com/nicolasdiolez/360extractor/blob/main/ARCHITECTURE.md

Defines the command-line interface for the application, supporting custom naming conventions. Leverages argparse for robust argument parsing.

```python
def main():
    parser = argparse.ArgumentParser(
        description="360° Video Preprocessor CLI",
        formatter_class=argparse.ArgumentDefaultsHelpFormatter
    )
    parser.add_argument(
        "--input",
        type=str,
        required=True,
        help="Path to the input 360° video file."
    )
    parser.add_argument(
        "--output",
        type=str,
        default="./output",
        help="Directory to save processed frames and metadata."
    )
    parser.add_argument(
        "--name",
        type=str,
        default=None,
        help="Custom name for the output dataset. If None, derived from input filename."
    )
    # Add other arguments for frame extraction, AI processing, etc.
    args = parser.parse_args()
    
    # Initialize and run the processor
    processor = VideoProcessor(args.input, args.output, args.name)
    processor.process()

if __name__ == "__main__":
    main()
```

--------------------------------

### AIService Initialization and GPU Detection

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Initializes the AIService, which wraps YOLOv26 for object detection and segmentation. It automatically detects and utilizes the best available hardware acceleration (MPS > CUDA > CPU).

```APIDOC
## `AIService` — Initialization and GPU Detection

`AIService` wraps YOLO26 (Ultralytics) for object detection and segmentation. It auto-selects the best available device (Apple MPS > CUDA > CPU) and loads the model at construction time.

### Class Methods

*   `get_device_info()`: Returns information about the detected device.
*   `is_gpu_available()`: Returns `True` if GPU acceleration is available, `False` otherwise.

### Initialization

*   `AIService(model_path: str)`: Initializes the service by loading the specified model.

### Example

```python
from core.ai_model import AIService

# Check hardware before loading model
device_info = AIService.get_device_info()
# {'device': 'mps', 'device_name': 'Apple Silicon GPU (MPS)', 'is_accelerated': True}

print(f"Running on: {device_info['device_name']}")
print(f"Accelerated: {device_info['is_accelerated']}")

# Initialize — loads yolo26n-seg.pt (nano, NMS-free, fastest)
ai = AIService('yolo26n-seg.pt')
# [INFO] ✓ GPU detected: Apple Silicon GPU (MPS)
# [INFO] Loading AI Model: yolo26n-seg.pt on mps...
# [INFO] ✅ ACTIVE AI MODEL: yolo26n-seg.pt

# Check if GPU is available (class method, no instance required)
if AIService.is_gpu_available():
    print("GPU acceleration active")
```
```

--------------------------------

### Manage Application Settings

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

A thread-safe singleton for persisting application settings to a JSON file. Supports reading, updating, and saving settings with priority given to CLI args.

```python
from core.settings_manager import SettingsManager

mgr = SettingsManager()   # Singleton — loads ~/.application360/config.json on first call

# Read individual values
resolution = mgr.get('resolution', 2048)
ai_mode    = mgr.get('ai_mode', 'None')
layout     = mgr.get('layout_mode', 'ring')
print(f"Resolution: {resolution}, AI: {ai_mode}, Layout: {layout}")

# Update and persist
mgr.set('resolution', 4096)
mgr.set('interpolation_mode', 'lanczos')
mgr.save_settings()   # writes to ~/.application360/config.json
```

```python
# Bulk update + save
mgr.save_settings({
    'camera_count': 8,
    'layout_mode': 'fibonacci',
    'adaptive_mode': True,
    'adaptive_threshold': 5.0,
    'blur_filter_enabled': True,
    'blur_threshold': 120.0,
    'export_telemetry': True,
    'naming_mode': 'realityscan'
})

# Read all settings as a dict
all_settings = mgr.get_all()
import json
print(json.dumps(all_settings, indent=2))
```

--------------------------------

### Process Video via CLI

Source: https://github.com/nicolasdiolez/360extractor/blob/main/README.md

Processes a video file using the command-line interface. Specify input, output, and processing interval. Useful for automation.

```bash
python3 src/main.py --input <video_path> --output <output_dir> --interval 1.0
```

--------------------------------

### Verify Environment

Source: https://github.com/nicolasdiolez/360extractor/blob/main/README.md

Checks the Python environment for compatibility. This script helps ensure all necessary packages and configurations are in place.

```bash
python3 check_env.py
```

--------------------------------

### JSON Configuration File

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Job settings can be pre-defined in a JSON config file and passed via `--config`. CLI flags override config values, which in turn override built-in defaults.

```APIDOC
## JSON Configuration File

Job settings can be pre-defined in a JSON config file and passed via `--config`. CLI flags override config values, which in turn override built-in defaults.

```json
{
    "input": "videos/holiday.mp4",
    "output": "processed/holiday",
    "interval": 2.0,
    "format": "png",
    "camera_count": 6,
    "active_cameras": [0, 1, 2, 3],
    "resolution": 2048,
    "quality": 100,
    "layout_mode": "cube",
    "fov": 90,
    "pitch_offset": -20,
    "ai_mask": true,
    "ai_mode": "Generate Mask",
    "ai_confidence": 0.3,
    "ai_detect_humans": true,
    "ai_detect_vehicles": true,
    "ai_detect_plants": false,
    "adaptive": false,
    "adaptive_threshold": 5.0,
    "blur_filter_enabled": true,
    "blur_threshold": 100.0,
    "smart_blur_enabled": true,
    "export_telemetry": true,
    "interpolation_mode": "lanczos",
    "feather_mask": true,
    "naming_mode": "realityscan"
}
```

```bash
python3 src/main.py --config job.json
# Expected output:
# [INFO] Loaded configuration from job.json
# [INFO] Found 1 file(s) to process.
# [INFO] Extracting telemetry for holiday.mp4...
# [INFO] Found telemetry stream: gpmd
# [INFO] Extracted 1842 GPS samples.
# [INFO] Loading AI Model: yolo26n-seg.pt on mps...
# [INFO] ✅ ACTIVE AI MODEL: yolo26n-seg.pt
# Processing holiday.mp4 |████████████| 100/100% [02:15<00:00]
```
```

--------------------------------

### Run Synchronously (CLI Pattern)

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Executes the worker's jobs synchronously, typically used in a command-line interface context. The output shows progress updates and the final generated files.

```python
# Run synchronously (CLI pattern)
worker.run()
```

--------------------------------

### Connect Qt Signals for Progress Tracking

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Connects various Qt signals from a worker object to print status updates, job progress, and errors to the console. This pattern is useful for integrating the worker into applications that require real-time feedback.

```python
worker.progress_updated.connect(lambda val, msg: print(f"[{val:3d}%] {msg}"))
worker.job_started.connect(lambda idx: print(f"--- Job {idx} started ---"))
worker.job_finished.connect(lambda idx: print(f"--- Job {idx} done ---"))
worker.error_occurred.connect(lambda err: print(f"ERROR: {err}"))
worker.finished.connect(lambda: print("All jobs complete."))
```

--------------------------------

### Python AI Service with YOLO26 for Batch Inference

Source: https://github.com/nicolasdiolez/360extractor/blob/main/ARCHITECTURE.md

Wraps YOLO26 for high-throughput GPU inference. Features a `process_batch` method for simultaneous processing across multiple camera views.

```python
from ultralytics import YOLO

class AIService:
    def __init__(self, model_path='yolov8n.pt'):
        # Load the YOLO model
        self.model = YOLO(model_path)
        # Ensure model is on GPU if available
        if self.model.device.type == 'cpu':
            print("Warning: YOLO model loaded on CPU. GPU acceleration recommended.")

    def process_batch(self, image_batch):
        """Processes a batch of images for object detection.
        
        Args:
            image_batch (list): A list of images (e.g., NumPy arrays).
            
        Returns:
            list: A list of detection results, one for each image in the batch.
        """
        # Perform inference on the batch
        results = self.model.predict(image_batch, stream=True)
        
        # Process results (e.g., filter by confidence, class, etc.)
        processed_results = []
        for result in results:
            # Example: Extract bounding boxes and confidences
            boxes = result.boxes  # Boxes object for bounding box outputs
            # Further processing of boxes...
            processed_results.append(result)
            
        return processed_results

    def process_single_image(self, image):
        """Processes a single image.
        
        Args:
            image (np.ndarray): The input image.
            
        Returns:
            object: Detection results for the image.
        """
        results = self.model.predict(image)
        return results[0] # Assuming single image input returns a list with one result
```

--------------------------------

### Batch Image Processing with AIService

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Processes multiple images simultaneously for improved GPU throughput. Returns results in the same order as the input, suitable for main processing loops across multiple camera views.

```python
import cv2
from core.ai_model import AIService
from core.geometry import GeometryProcessor

ai = AIService('yolo26n-seg.pt')

# Build a batch of 6 reprojected views from one equirectangular frame
src = cv2.imread("equirect_frame.jpg")
src_h, src_w = src.shape[:2]
views = GeometryProcessor.generate_views(6, layout_mode='cube')

batch_images = []
for name, yaw, pitch, roll in views:
    mx, my = GeometryProcessor.create_rectilinear_map(src_h, src_w, 2048, 2048, 90, yaw, pitch, roll)
    batch_images.append(cv2.remap(src, mx, my, cv2.INTER_LINEAR, borderMode=cv2.BORDER_WRAP))

# Process full batch — generate masks, detect humans + vehicles
results = ai.process_batch(
    batch_images,
    mode='generate_mask',
    conf=0.25,
    classes=[0, 2, 3, 5, 6, 7],   # persons + vehicles
    invert_mask=True,
    feather_mask=True
)

for (img, mask), (name, *_) in zip(results, views):
    if img is not None and mask is not None:
        cv2.imwrite(f"frame_{name.lower()}.jpg", img)
        cv2.imwrite(f"frame_{name.lower()}.jpg.mask.png", mask)
        print(f"{name}: mask saved")
    # skip_frame mode would yield (None, True) here instead
```

--------------------------------

### Data Flow Sequence Diagram

Source: https://github.com/nicolasdiolez/360extractor/blob/main/ARCHITECTURE.md

Visualizes the sequence of operations and interactions between different components during data extraction and processing.

```mermaid
sequenceDiagram
    participant F as File (360 Video)
    participant T as Telemetry
    participant E as Extractor
    participant P as Projector
    participant A as AI Service
    participant IO as ThreadPool (Disk)

    F->>T: Extract GPMF/CAMM/SRT
    T-->>T: Sync & Interpolate GPS
    F->>E: Decode Frame (t)
    alt Mode: Adaptive (Motion)
        E->>E: Check Optical Flow
        note over E: If score < threshold, skip
    end
    E->>P: Raw Equirectangular Frame
    
    P->>P: Reproject selected cameras (Batch Builders)
    P->>A: Send Batch of Images
    alt AI: Skip Frame
        A->>A: Batch Detect -> Skip frame if any person
    else AI: Generate Mask
        A->>A: Batch Segment -> Return masks
    end
    
    P->>IO: Submit parallel save tasks
    IO->>IO: Save Images & Masks + Embed EXIF
```

--------------------------------

### Process Image with AIService for Detection or Masking

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Runs inference on a BGR image using `AIService`. Supports 'skip_frame' mode to discard frames with detected objects (e.g., people) or 'generate_mask' mode to create segmentation masks. Masks can be inverted and feathered for photogrammetry. Requires OpenCV and NumPy.

```python
import cv2
import numpy as np
from core.ai_model import AIService

ai = AIService('yolo26n-seg.pt')
frame = cv2.imread("rectilinear_view.jpg")  # 2048x2048 BGR

# Mode 1: Skip frames that contain people
result_img, skip_flag = ai.process_image(
    frame, mode='skip_frame', conf=0.25, classes=[0]  # class 0 = person
)
if skip_flag is True:
    print("Person detected — frame discarded")
else:
    print("No person — frame kept")

# Mode 2: Generate a hard-edge segmentation mask
result_img, mask = ai.process_image(
    frame, mode='generate_mask',
    conf=0.3,
    classes=[0],          # Humans only
    invert_mask=True,     # Black=person, White=background (photogrammetry convention)
    feather_mask=False
)
if mask is not None:
    cv2.imwrite("mask_hard.png", mask)
    print(f"Mask shape: {mask.shape}, unique values: {np.unique(mask)}")
    # Mask shape: (2048, 2048), unique values: [  0 255]
```

--------------------------------

### Python Motion Detection with Farneback Optical Flow

Source: https://github.com/nicolasdiolez/360extractor/blob/main/ARCHITECTURE.md

Implements dense optical flow using the Farneback algorithm to calculate scene change magnitude. Used for adaptive frame extraction.

```python
import cv2

class MotionDetector:
    def __init__(self):
        self.prev_gray = None

    def detect(self, frame):
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        if self.prev_gray is None:
            self.prev_gray = gray
            return 0.0
        
        # Calculate dense optical flow using Farneback method
        flow = cv2.calcOpticalFlowFarneback(
            self.prev_gray,
            gray,
            None,
            0.5, 3, 15, 3, 5, 1.2, 0
        )
        
        # Calculate magnitude and angle of 2D vectors
        magnitude, _ = cv2.cartToPolar(flow[..., 0], flow[..., 1])
        
        # Average magnitude as a measure of scene change
        avg_magnitude = cv2.mean(magnitude)[0]
        
        self.prev_gray = gray
        return avg_magnitude
```

--------------------------------

### Generate Camera Views with GeometryProcessor

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Python code to generate camera descriptors for different layouts (ring, cube, fibonacci) using the GeometryProcessor. Supports custom pitch offsets.

```python
from core.geometry import GeometryProcessor

# Ring layout — 8 cameras evenly along the horizon, tilted down 20° (high/perch)
views = GeometryProcessor.generate_views(n=8, pitch_offset=-20, layout_mode='ring')
# [('View_0', 0.0, -20, 0), ('View_1', 45.0, -20, 0), ..., ('View_7', 315.0, -20, 0)]
```

```python
# Cube map — always 6 views, camera count is ignored
views = GeometryProcessor.generate_views(n=6, pitch_offset=0, layout_mode='cube')
# [('Front', 0.0, 0, 0), ('Right', 90.0, 0, 0), ('Back', 180.0, 0, 0),
#  ('Left', 270.0, 0, 0), ('Up', 0.0, 90.0, 0), ('Down', 0.0, -90.0, 0)]
```

```python
# Fibonacci sphere — 12 cameras evenly distributed on a sphere
views = GeometryProcessor.generate_views(n=12, pitch_offset=0, layout_mode='fibonacci')
```

--------------------------------

### Python Telemetry Handler for GoPro, Insta360, DJI

Source: https://github.com/nicolasdiolez/360extractor/blob/main/ARCHITECTURE.md

Detects and parses GPMF (GoPro), CAMM (Insta360), and SRT (DJI) metadata. Supports EXIF injection for processed files.

```python
import piexif

class TelemetryHandler:
    def __init__(self):
        pass

    def parse_gopro(self, filepath):
        # Logic to parse GPMF metadata from GoPro files
        pass

    def parse_insta360(self, filepath):
        # Logic to parse CAMM metadata from Insta360 files
        pass

    def parse_dji(self, filepath):
        # Logic to parse SRT metadata from DJI files
        pass

    def extract_metadata(self, filepath):
        # Detect file type and call appropriate parser
        # Returns a dictionary of metadata
        metadata = {}
        if "GoPro" in filepath: # Simplified detection
            metadata.update(self.parse_gopro(filepath))
        elif "Insta360" in filepath: # Simplified detection
            metadata.update(self.parse_insta360(filepath))
        elif "DJI" in filepath: # Simplified detection
            metadata.update(self.parse_dji(filepath))
        return metadata

    def inject_exif(self, filepath, metadata):
        # Logic to inject extracted metadata into EXIF tags of an output file
        # Example using piexif (requires metadata in a specific format)
        try:
            exif_dict = piexif.load(filepath)
            # Populate exif_dict with metadata
            exif_bytes = piexif.dump(exif_dict)
            piexif.insert(exif_bytes, filepath)
        except Exception as e:
            print(f"Error injecting EXIF data: {e}")
```

--------------------------------

### Generate Rectilinear Maps and Views with GeometryProcessor

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Computes remapping arrays for converting equirectangular frames to rectilinear views using `cv2.remap`. Supports generating maps for single views or batch processing all six cube faces. Requires OpenCV and NumPy.

```python
import cv2
import numpy as np
from core.geometry import GeometryProcessor

# Load equirectangular source (e.g., 5760x2880)
src = cv2.imread("equirect_frame.jpg")
src_h, src_w = src.shape[:2]   # 2880, 5760
out_res = 2048                  # square output

# Generate remap for a 90° FOV front view
map_x, map_y = GeometryProcessor.create_rectilinear_map(
    src_h=src_h, src_w=src_w,
    dest_h=out_res, dest_w=out_res,
    fov_deg=90,
    yaw_deg=0,     # Front
    pitch_deg=0,   # Horizon level
    roll_deg=0
)
# map_x.shape == (2048, 2048), dtype=float32
# map_y.shape == (2048, 2048), dtype=float32

# Reproject with Lanczos for maximum sharpness
front_view = cv2.remap(src, map_x, map_y, cv2.INTER_LANCZOS4, borderMode=cv2.BORDER_WRAP)
cv2.imwrite("front_2048.jpg", front_view, [cv2.IMWRITE_JPEG_QUALITY, 95])
# Output: 2048x2048 pinhole image looking forward

# Batch all 6 cube faces
views = GeometryProcessor.generate_views(6, layout_mode='cube')
for name, yaw, pitch, roll in views:
    mx, my = GeometryProcessor.create_rectilinear_map(src_h, src_w, out_res, out_res, 90, yaw, pitch, roll)
    face = cv2.remap(src, mx, my, cv2.INTER_LANCZOS4, borderMode=cv2.BORDER_WRAP)
    cv2.imwrite(f"cube_{name.lower()}.jpg", face)
```

--------------------------------

### AIService.process_batch

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Runs batched inference on multiple images simultaneously for GPU throughput. Returns a list of (image, mask_or_status) tuples in the same order as the input. This is the method used in the main processing loop across all camera views of a single frame.

```APIDOC
## AIService.process_batch

Runs batched inference on multiple images simultaneously for GPU throughput. Returns a list of `(image, mask_or_status)` tuples in the same order as the input. This is the method used in the main processing loop across all camera views of a single frame.

### Method
```python
ai.process_batch(
    batch_images,
    mode='generate_mask',
    conf=0.25,
    classes=[0, 2, 3, 5, 6, 7],   # persons + vehicles
    invert_mask=True,
    feather_mask=True
)
```

### Parameters
- **batch_images** (list) - A list of images to process.
- **mode** (str) - The processing mode, e.g., 'generate_mask'.
- **conf** (float) - Confidence threshold for detection.
- **classes** (list) - A list of class IDs to detect.
- **invert_mask** (bool) - Whether to invert the generated mask.
- **feather_mask** (bool) - Whether to apply feathering to the mask.
```

--------------------------------

### GeometryProcessor.generate_views

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Generates a list of `(name, yaw, pitch, roll)` camera descriptors for a given count and layout. Supports three spatial distributions: `ring` (equidistant horizon), `cube` (fixed 6-face cube map), and `fibonacci` (Golden Section Spiral for uniform sphere coverage).

```APIDOC
## `GeometryProcessor.generate_views`

Generates a list of `(name, yaw, pitch, roll)` camera descriptors for a given count and layout. Supports three spatial distributions: `ring` (equidistant horizon), `cube` (fixed 6-face cube map), and `fibonacci` (Golden Section Spiral for uniform sphere coverage).

```python
from core.geometry import GeometryProcessor

# Ring layout — 8 cameras evenly along the horizon, tilted down 20° (high/perch)
views = GeometryProcessor.generate_views(n=8, pitch_offset=-20, layout_mode='ring')
# [('View_0', 0.0, -20, 0), ('View_1', 45.0, -20, 0), ..., ('View_7', 315.0, -20, 0)]

# Cube map — always 6 views, camera count is ignored
views = GeometryProcessor.generate_views(n=6, pitch_offset=0, layout_mode='cube')
# [('Front', 0.0, 0, 0), ('Right', 90.0, 0, 0), ('Back', 180.0, 0, 0),
#  ('Left', 270.0, 0, 0), ('Up', 0.0, 90.0, 0), ('Down', 0.0, -90.0, 0)]

# Fibonacci sphere — 12 cameras evenly distributed on a sphere
views = GeometryProcessor.generate_views(n=12, pitch_offset=0, layout_mode='fibonacci')
```
```

--------------------------------

### AIService.process_image

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Runs inference on a single BGR image using the loaded AI model. Supports different modes for frame skipping or generating segmentation masks with optional feathering.

```APIDOC
## `AIService.process_image`

Runs inference on a single BGR image. In `skip_frame` mode it returns `(None, True)` if any target is detected. In `generate_mask` mode it returns `(image, mask_array)` where the mask is a binary PNG-ready uint8 array (black = masked region, white = keep, when `invert_mask=True`). Optionally uses probabilistic soft edges (`feather_mask=True`) for seamless photogrammetry integration.

### Parameters

*   `frame` (numpy.ndarray) - The input BGR image.
*   `mode` (str) - Processing mode: `'skip_frame'` or `'generate_mask'`.
*   `conf` (float) - Confidence threshold for detection (default: 0.25).
*   `classes` (list[int]) - List of class IDs to consider.
*   `invert_mask` (bool) - If `True`, the mask will have the background as black and the object as white (default: `False`).
*   `feather_mask` (bool) - If `True`, applies soft edges to the mask (default: `False`).

### Returns

*   If `mode='skip_frame'`: `(result_img, skip_flag)` where `result_img` is the original frame if no target is detected, and `skip_flag` is `True` if a target was detected, `False` otherwise.
*   If `mode='generate_mask'`: `(result_img, mask)` where `result_img` is the original frame and `mask` is a NumPy array representing the segmentation mask, or `None` if no mask was generated.

### Example

```python
import cv2
import numpy as np
from core.ai_model import AIService

ai = AIService('yolo26n-seg.pt')
frame = cv2.imread("rectilinear_view.jpg")  # 2048x2048 BGR

# Mode 1: Skip frames that contain people
result_img, skip_flag = ai.process_image(
    frame, mode='skip_frame', conf=0.25, classes=[0]  # class 0 = person
)
if skip_flag is True:
    print("Person detected — frame discarded")
else:
    print("No person — frame kept")

# Mode 2: Generate a hard-edge segmentation mask
result_img, mask = ai.process_image(
    frame, mode='generate_mask',
    conf=0.3,
    classes=[0],          # Humans only
    invert_mask=True,     # Black=person, White=background (photogrammetry convention)
    feather_mask=False
)
if mask is not None:
    cv2.imwrite("mask_hard.png", mask)
    print(f"Mask shape: {mask.shape}, unique values: {np.unique(mask)}")
    # Mask shape: (2048, 2048), unique values: [  0 255]
```
```

--------------------------------

### Extract Telemetry Metadata from Video

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Detects and extracts GPS tracks from various binary streams (GPMF, CAMM, SRT) or GPX files. GPS samples can be interpolated and embedded as EXIF data.

```python
from core.telemetry import TelemetryHandler

handler = TelemetryHandler()

# Auto-detect and extract (GPMF, CAMM, SRT, or GPX sidecar)
success = handler.extract_metadata("GS_360_tour.mp4")
# [INFO] Found telemetry stream: gpmd
```

--------------------------------

### Calculate Motion Score with Optical Flow

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Computes a motion score between frames using Farneback Dense Optical Flow, internally downscaling for speed. Used for adaptive keyframing to skip static scenes.

```python
import cv2
from core.motion_detector import MotionDetector

detector = MotionDetector(target_size=(256, 144))  # default

cap = cv2.VideoCapture("timelapse.mp4")
_, frame_prev = cap.read()
_, frame_curr = cap.read()

score = detector.calculate_motion_score(frame_prev, frame_curr)
print(f"Motion score: {score:.4f}")
# e.g., 0.32 = almost static, 8.7 = fast camera movement

# Adaptive threshold example: skip frames with insufficient motion
THRESHOLD = 5.0
last_extracted = frame_prev.copy()

while True:
    ret, frame = cap.read()
    if not ret:
        break

    motion = detector.calculate_motion_score(last_extracted, frame)
    if motion > THRESHOLD:
        print(f"  → Extract frame (motion={motion:.2f})")
        last_extracted = frame.copy()
    else:
        print(f"  → Skip frame   (motion={motion:.2f})")

cap.release()
```

--------------------------------

### Parse Custom AI Classes from String

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Converts comma-separated COCO class names into integer IDs for YOLO models. Supports built-in presets and free-form string parsing, silently ignoring invalid names.

```python
from core.ai_classes import parse_custom_classes, PRESETS, COCO_CLASSES

# Use built-in presets
humans_ids   = PRESETS["Humans"]    # [0]
vehicle_ids  = PRESETS["Vehicles"]  # [2, 3, 4, 5, 6, 7, 8]
plant_ids    = PRESETS["Plants"]    # [58]

# Combine presets
target_classes = humans_ids + vehicle_ids
print(target_classes)  # [0, 2, 3, 4, 5, 6, 7, 8]

# Parse free-form strings (e.g. from CLI --custom-classes)
ids = parse_custom_classes("person, car, truck, bicycle")
print(ids)   # [0, 2, 7, 1]

# Invalid names are silently ignored
ids = parse_custom_classes("person, spaceship, dog")
print(ids)   # [0, 16]   — 'spaceship' ignored

# Full COCO lookup
print(COCO_CLASSES[0])   # 'person'
print(COCO_CLASSES[58])  # 'potted plant'
```

--------------------------------

### Extract and Embed GPS Data

Source: https://context7.com/nicolasdiolez/360extractor/llms.txt

Extracts GPS samples from video metadata and allows embedding this data into image EXIF. Handles automatic GPX sidecar detection.

```python
if handler.has_gps:
    print(f"GPS samples: {len(handler.gps_samples)}")

    # Interpolated GPS at any video timestamp
    gps = handler.get_gps_at_time(timestamp=12.5)   # 12.5 seconds into video
    lat, lon, alt = gps
    print(f"  lat={lat:.6f}, lon={lon:.6f}, alt={alt:.1f}m")
    # lat=48.858844, lon=2.294351, alt=42.3m

    # Embed GPS into an extracted frame image
    ok = handler.embed_exif("frames/frame_000025_front.jpg", lat, lon, alt)
    print(f"  EXIF embedded: {ok}")   # True

# GPX sidecar (Qoocam workflow — auto-detected if same basename)
# Place "video.gpx" next to "video.mp4" — handler picks it up automatically
handler2 = TelemetryHandler()
handler2.extract_metadata("qoocam_video.mp4")  # reads qoocam_video.gpx if present
```