### Install Environment and Dependencies

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Create a conda environment, activate it, and install project dependencies including megapose and bop_toolkit.

```bash
conda env create -f environment.yml
conda activate gigapose
bash src/scripts/install_env.sh

# to install megapose
pip install -e .

# to install bop_toolkit 
pip install git+https://github.com/thodan/bop_toolkit.git
```

--------------------------------

### GigaPose Training Command Line Arguments

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Examples of how to run the GigaPose training script with different dataset configurations using command-line arguments.

```bash
# Train on GSO dataset only (ID=0)
python train.py train_dataset_id=0
```

```bash
# Train on ShapeNet dataset only (ID=1)
python train.py train_dataset_id=1
```

```bash
# Train on both GSO and ShapeNet (ID=2)
python train.py train_dataset_id=2
```

--------------------------------

### Install GigaPose Environment

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Sets up the conda environment and installs required dependencies, including the MegaPose refiner and BOP toolkit.

```bash
# Create conda environment from YAML specification
conda env create -f environment.yml
conda activate gigapose

# Install additional dependencies
bash src/scripts/install_env.sh

# Install megapose package in editable mode
pip install -e .

# Install BOP toolkit for evaluation
pip install git+https://github.com/thodan/bop_toolkit.git
```

--------------------------------

### Download BOP24 Test Datasets

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Installs huggingface_hub and downloads test datasets for BOP challenge 2024. Specify the dataset name using DATASET_NAME.

```bash
pip install -U "huggingface_hub[cli]"
export DATASET_NAME=hope
python -m src.scripts.download_test_bop24 test_dataset_name=$DATASET_NAME
```

--------------------------------

### Evaluate with BOP Toolkit (BOP24)

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Set up environment variables for input directory and file name, then navigate to the BOP toolkit root directory to run the evaluation script for BOP24 pose estimation.

```bash
export INPUT_DIR=DIR_TO_YOUR_PREDICTION_FILE
export FILE_NAME=NAME_PREDICTION_FILE
cd $ROOT_DIR_OF_TOOLKIT
python scripts/eval_bop24_pose.py --results_path $INPUT_DIR --eval_path $INPUT_DIR --result_filenames=$FILE_NAME
```

--------------------------------

### Evaluate with BOP Toolkit (BOP19)

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Set up environment variables for input directory and file name, then execute the BOP19 pose evaluation script using the specified renderer type and paths.

```bash
export INPUT_DIR=DIR_TO_YOUR_PREDICTION_FILE
export FILE_NAME=NAME_PREDICTION_FILE
python bop_toolkit/scripts/eval_bop19_pose.py --renderer_type=vispy --results_path $INPUT_DIR --eval_path $INPUT_DIR --result_filenames=$FILE_NAME
```

--------------------------------

### Download Model Checkpoints

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Downloads the necessary pretrained weights for GigaPose, MegaPose, and default detections.

```bash
# Download CNOS detections for BOP'23 dataset
pip install -U "huggingface_hub[cli]"
python -m src.scripts.download_default_detections

# Download GigaPose checkpoint (gigaPose_v1.ckpt)
python -m src.scripts.download_gigapose

# Download MegaPose refiner checkpoint
python -m src.scripts.download_megapose
```

--------------------------------

### Download and Prepare lmoWonder3d Dataset

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Download reconstructed 3D models and test data for the lmoWonder3d dataset. Create symlinks to integrate it as a new dataset. This step is necessary before onboarding and testing.

```bash
# download the reconstructed 3D models, test images, and test_targets_bop19.json
mkdir $ROOT_DIR/datasets/lmoWonder3d
wget https://huggingface.co/datasets/nv-nguyen/gigaPose/resolve/main/lmoWonder3d.zip -P $ROOT_DIR/datasets/lmoWonder3d
unzip -j $ROOT_DIR/datasets/lmoWonder3d/lmoWonder3d.zip -d $ROOT_DIR/datasets/lmoWonder3d/models -x "*/._*"

# treat lmoWonder3d as a new dataset by creating a symlink 
ln -s $ROOT_DIR/datasets/lmo/test $ROOT_DIR/datasets/lmoWonder3d/test
ln -s $ROOT_DIR/datasets/lmo/test_targets_bop19.json $ROOT_DIR/datasets/lmoWonder3d/test_targets_bop19.json
```

--------------------------------

### Create Dataset Symlink

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Creates a symbolic link to the folder containing pre-downloaded training datasets.

```bash
ln -s /path/to/datasets/gso $ROOT/datasets/gso
```

--------------------------------

### GigaPose Training Script

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

This script sets up and runs the GigaPose training process using PyTorch Lightning and Hydra for configuration. It initializes dataloaders, the model, and the trainer.

```python
import pytorch_lightning as pl
from torch.utils.data import DataLoader
import hydra
from omegaconf import DictConfig, OmegaConf
from hydra.utils import instantiate

train_dataset_names = {0: ["gso"], 1: ["shapenet"], 2: ["gso", "shapenet"]}

@hydra.main(version_base=None, config_path="configs", config_name="train")
def run_train(cfg: DictConfig):
    OmegaConf.set_struct(cfg, False)

    # Initialize trainer with logger
    trainer = instantiate(cfg.machine.trainer)

    # Initialize dataloaders for training datasets
    selected_datasets = train_dataset_names[cfg.train_dataset_id]
    train_dataloaders = []
    for name in selected_datasets:
        cfg.data.train.dataloader.dataset_name = name
        train_dataset = instantiate(cfg.data.train.dataloader)
        train_dataloader = DataLoader(
            train_dataset.web_dataloader.datapipeline,
            batch_size=cfg.machine.batch_size,
            num_workers=cfg.machine.num_workers,
            collate_fn=train_dataset.collate_fn,
        )
        train_dataloaders.append(train_dataloader)

    # Validation dataset (YCBV)
    cfg.data.test.dataloader.dataset_name = "ycbv"
    val_dataset = instantiate(cfg.data.test.dataloader)
    val_dataloader = DataLoader(
        val_dataset.web_dataloader.datapipeline,
        batch_size=1,
        num_workers=cfg.machine.num_workers,
        collate_fn=val_dataset.collate_fn,
    )

    # Initialize model
    model = instantiate(cfg.model)

    # Start training
    trainer.fit(model, train_dataloaders=train_dataloaders, val_dataloaders=val_dataloader)
```

--------------------------------

### Run All Steps for BOP 2023 Core Datasets

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Execute the evaluation script that runs all necessary steps for the 7 core datasets of the BOP challenge 2023. This command automates the entire prediction and evaluation pipeline.

```bash
python -m src.scripts.eval_bop
```

--------------------------------

### Estimate Pose from Single Reference Image

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Commands to download models, render templates, and run estimation for single-image reference workflows.

```bash
# 1. Download reconstructed 3D models and test data
mkdir $ROOT_DIR/datasets/lmoWonder3d
wget https://huggingface.co/datasets/nv-nguyen/gigaPose/resolve/main/lmoWonder3d.zip -P $ROOT_DIR/datasets/lmoWonder3d
unzip -j $ROOT_DIR/datasets/lmoWonder3d/lmoWonder3d.zip -d $ROOT_DIR/datasets/lmoWonder3d/models

# 2. Create symlinks to test data
ln -s $ROOT_DIR/datasets/lmo/test $ROOT_DIR/datasets/lmoWonder3d/test
ln -s $ROOT_DIR/datasets/lmo/test_targets_bop19.json $ROOT_DIR/datasets/lmoWonder3d/test_targets_bop19.json

# 3. Render templates from reconstructed 3D models
python -m src.scripts.render_custom_templates custom_dataset_name=lmoWonder3d

# 4. Run pose estimation
python test.py test_dataset_name=lmoWonder3d run_id=wonder3d_exp

# 5. Run refinement
python refine.py test_dataset_name=lmoWonder3d run_id=wonder3d_exp
```

--------------------------------

### Download BOP23 Test Datasets

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Downloads testing images and CAD models for BOP challenge 2023 core datasets.

```bash
python -m src.scripts.download_test_bop23
```

--------------------------------

### Download Detections and Checkpoints

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Download pre-computed CNOS detections for BOP'23, GigaPose checkpoints, and MegaPose checkpoints using provided Python scripts.

```python
# download cnos detections for BOP'23 dataset
pip install -U "huggingface_hub[cli]"
python -m src.scripts.download_default_detections

# download gigaPose's checkpoints 
python -m src.scripts.download_gigapose

# download megapose's checkpoints
python -m src.scripts.download_megapose
```

--------------------------------

### Onboard and Test Custom Dataset

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Render templates from reconstructed 3D models for a custom dataset and then test the dataset using the GigaPose pipeline. This allows GigaPose to process and evaluate poses on new datasets.

```bash
# Onboarding by rendering templates from reconstructed 3D models
python -m src.scripts.render_custom_templates custom_dataset_name=lmoWonder3d

# now, it can be tested as a normal dataset as in the previous section
python test.py test_dataset_name=lmoWonder3d run_id=$NAME_RUN
python refine.py test_dataset_name=lmoWonder3d run_id=$NAME_RUN
```

--------------------------------

### Run Pose Refinement

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Execute the pose refinement script. Specify the dataset name, run ID, and the test setting (detection or localization).

```bash
python refine.py test_dataset_name=hope run_id=my_experiment test_setting=detection
```

```bash
python refine.py test_dataset_name=lmo run_id=my_experiment test_setting=localization
```

--------------------------------

### Download CNOS Detections

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Download pre-computed GigaPose checkpoints for CNOS detections, which are necessary for certain pose estimation tasks.

```bash
python -m src.scripts.download_cnos_bop23
```

--------------------------------

### Download GigaPose Training Data

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Downloads training images and CAD models for GigaPose. Rendering templates is also included.

```bash
# download training images (> 2TB)
python -m src.scripts.download_train_metaData
python -m src.scripts.download_train_cad 
python -m src.scripts.download_train 
```

```bash
# render templates ( 162 imgs/obj takes ~30mins for gso, ~20hrs for shapenet)
python -m src.scripts.render_gso_templates 
python -m src.scripts.render_shapenet_templates  
```

--------------------------------

### Download BOP Challenge 2024 Datasets

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Downloads core datasets and renders templates for the BOP Challenge 2024.

```bash
# Set dataset name (hope, handal, or hot3d)
export DATASET_NAME=hope

# Download the dataset
pip install -U "huggingface_hub[cli]"
python -m src.scripts.download_test_bop24 test_dataset_name=$DATASET_NAME

# Render templates from CAD models for the downloaded dataset
python -m src.scripts.render_bop_templates test_dataset_name=hope
```

--------------------------------

### Download BOP Challenge 2023 Datasets

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Downloads testing images and CAD models for the BOP Challenge 2023, with options for pre-rendered or manual template generation.

```bash
# Download testing images and CAD models
python -m src.scripts.download_test_bop23

# Option 1: Download pre-rendered templates (faster)
python -m src.scripts.download_bop_templates

# Option 2: Render templates from CAD models manually
python -m src.scripts.render_bop_templates
```

--------------------------------

### Run Refinement on Single Dataset (BOP2023)

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Execute the refinement script for a single dataset, specifying the dataset name and run ID. This step is crucial for reproducing BOP challenge 2023 results.

```bash
python refine.py test_dataset_name=lmo run_id=$NAME_RUN
```

--------------------------------

### Run Refinement (BOP24)

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Executes the refinement step for the 6D detection task on BOP challenge 2024 datasets. Requires specifying the dataset name and a run ID.

```bash
# for both 6D detection task
python refine.py test_dataset_name=hope run_id=$NAME_RUN test_setting=detection
```

--------------------------------

### Run BOP Evaluation Pipeline

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Execute coarse estimation, refinement, and evaluation across multiple BOP datasets.

```python
import os
import hydra
from omegaconf import DictConfig, OmegaConf

@hydra.main(version_base=None, config_path="../../configs", config_name="test")
def run_seven_cores(cfg: DictConfig):
    """Evaluate on all 7 core BOP'23 datasets"""
    OmegaConf.set_struct(cfg, False)
    cfg.save_dir = os.path.join(cfg.machine.root_dir, "results")
    cfg.results_dir = "results/GigaPose"

    for dataset_name in ["lmo", "tudl", "icbin", "tless", "ycbv", "itodd", "hb"]:
        cfg.dataset_name = dataset_name

        # Run coarse estimation
        coarse_cmd = f"python test.py test_dataset_name={dataset_name} run_id=GigaPose"
        os.system(coarse_cmd)

        # Run refinement (top-1)
        refine_cmd = f"python refine.py test_dataset_name={dataset_name} use_multiple=False"
        os.system(refine_cmd)

        # Run refinement (top-5)
        refine_multi_cmd = f"python refine.py test_dataset_name={dataset_name} use_multiple=True"
        os.system(refine_multi_cmd)

        # Evaluate with BOP toolkit
        eval_cmd = f"python bop_toolkit/scripts/eval_bop19_pose.py --results_path {results_path}"
        os.system(eval_cmd)
```

```bash
# Run complete evaluation on all 7 datasets
python -m src.scripts.eval_bop

# Evaluate with BOP toolkit manually
export INPUT_DIR=path/to/predictions
export FILE_NAME=predictions.csv
python bop_toolkit/scripts/eval_bop19_pose.py --renderer_type=vispy --results_path $INPUT_DIR --eval_path $INPUT_DIR --result_filenames=$FILE_NAME
```

--------------------------------

### Initialize and Run Pose Refinement

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Python script to refine coarse pose predictions using the MegaPose refiner. It loads datasets, models, and runs the refinement process.

```python
from pathlib import Path
import hydra
from omegaconf import DictConfig, OmegaConf
from hydra.utils import instantiate
from torch.utils.data import DataLoader
from megapose.datasets.bop_object_datasets import BOPObjectDataset
from src.custom_megapose.refiner_utils import find_init_pose_path
from src.models.refiner import Refiner

@hydra.main(version_base=None, config_path="configs", config_name="test")
def run_refiner(cfg: DictConfig):
    OmegaConf.set_struct(cfg, False)

    # Find initial pose predictions from coarse estimation
    init_loc_path, model_name, run_id = find_init_pose_path(
        cfg.save_dir, cfg.test_dataset_name, cfg.use_multiple
    )

    # Initialize test dataset with initial poses
    cfg.data.test.dataloader.dataset_name = cfg.test_dataset_name
    cfg.data.test.dataloader.init_loc_path = init_loc_path
    cfg.data.test.dataloader.test_setting = cfg.test_setting
    test_dataset = instantiate(cfg.data.test.dataloader)

    dataloader = DataLoader(
        test_dataset.web_dataloader.datapipeline,
        batch_size=1,
        num_workers=10,
        collate_fn=test_dataset.collate_refine_fn,
    )

    # Load CAD models for refinement
    root_dir = Path(cfg.data.test.dataloader.root_dir)
    cad_dir = root_dir / cfg.test_dataset_name / "models"
    object_dataset = BOPObjectDataset(Path(cad_dir), format=".ply")

    # Initialize refiner
    refiner = Refiner(
        object_dataset=object_dataset,
        cfg_refiner_model=cfg.model.refiner,
        use_multiple=cfg.use_multiple,
        log_dir=cfg.save_dir,
        test_dataset_name=cfg.test_dataset_name,
        coarse_model_name=model_name,
        run_id=run_id,
    )

    # Run refinement
    trainer = instantiate(cfg.machine.trainer)
    trainer.test(refiner, dataloaders=dataloader)
```

--------------------------------

### Manage BOP23 Templates

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Provides two options for BOP challenge 2023 core datasets: download pre-rendered templates or render them from CAD models.

```bash
# option 1: download pre-rendered templates 
python -m src.scripts.download_bop_templates
```

```bash
# option 2: render templates from CAD models 
python -m src.scripts.render_bop_templates
```

--------------------------------

### Run 6D Localization on Core19 Datasets

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Execute the refinement script for the 6D localization task on core19 datasets. Ensure the test dataset name and run ID are correctly specified.

```bash
python refine.py test_dataset_name=lmo run_id=$NAME_RUN test_setting=localization
```

--------------------------------

### Initialize Template Dataset

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Load and read templates for specific objects using the TemplateDataset class.

```python
# Initialize template dataset from config
template_dataset = TemplateDataset.from_config(
    model_infos=[{"obj_id": 1}, {"obj_id": 2}],  # Object IDs
    config=template_config,  # Config with paths and parameters
)

# Get templates for a specific object
template_data = template_dataset.get_object_templates(label="1")

# Load all templates for testing
data, poses = template_data.read_test_mode()
# data["rgba"]: (N_templates, 4, H, W) - RGBA images
# data["box"]: (N_templates, 4) - Bounding boxes
# poses: (N_templates, 4, 4) - 6D poses

# Find nearest template for a query pose
template_finder = NearestTemplateFinder(config)
result = template_finder.search_nearest_template(object_rot=query_rotation)
# result["view_id"]: Index of nearest template
# result["inplane"]: In-plane rotation angle (degrees)
```

--------------------------------

### GigaPose Model Initialization

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Defines the GigaPose model class, inheriting from PyTorch Lightning's LightningModule. It initializes networks for appearance encoding and In-plane, Scale, Translation (IST) prediction.

```python
import torch
import pytorch_lightning as pl
from src.models.gigaPose import GigaPose

class GigaPose(pl.LightningModule):
    def __init__(
        self,
        model_name,
        ae_net,
        ist_net,
        training_loss,
        testing_metric,
        optim_config,
        log_interval,
        log_dir,
        max_num_dets_per_forward=None,
        test_setting="localization",
    ):
        super().__init__()
        self.ae_net = ae_net
        self.ist_net = ist_net
        self.template_datasets = {}
        self.pose_recovery = {}
```

--------------------------------

### Template Dataset Imports

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Imports necessary modules for loading and managing pre-rendered object templates for pose estimation.

```python
import numpy as np
import torch
from src.custom_megapose.template_dataset import TemplateDataset, TemplateData, NearestTemplateFinder
from src.dataloader.template import TemplateSet
```

--------------------------------

### Render BOP24 Templates

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Renders templates from CAD models for BOP challenge 2024 core datasets. Specify the dataset name.

```bash
python -m src.scripts.render_bop_templates test_dataset_name=hope
```

--------------------------------

### Checkout Previous Commit for BOP 2023

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Use this command to revert to a previous commit if you encounter issues with the BOP challenge 2023.

```bash
git checkout 388e8bddd8a5443e284a7f70ad103d03f3f461c5
```

--------------------------------

### Run Coarse Pose Estimation

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Executes the coarse pose estimation pipeline using Hydra configuration and PyTorch Lightning.

```python
import hydra
from omegaconf import DictConfig, OmegaConf
from hydra.utils import instantiate
from torch.utils.data import DataLoader

@hydra.main(version_base=None, config_path="configs", config_name="test")
def run_test(cfg: DictConfig):
    OmegaConf.set_struct(cfg, False)

    # Initialize trainer
    trainer = instantiate(cfg.machine.trainer)

    # Initialize model with test setting
    cfg.model.test_setting = cfg.test_setting  # "detection" or "localization"
    model = instantiate(cfg.model)

    # Initialize test dataset
    cfg.data.test.dataloader.dataset_name = cfg.test_dataset_name
    cfg.data.test.dataloader.batch_size = cfg.machine.batch_size
    cfg.data.test.dataloader.test_setting = cfg.test_setting
    test_dataset = instantiate(cfg.data.test.dataloader)

    test_dataloader = DataLoader(
        test_dataset.web_dataloader.datapipeline,
        batch_size=1,
        num_workers=cfg.machine.num_workers,
        collate_fn=test_dataset.collate_fn,
    )

    # Set template dataset for pose recovery
    cfg.data.test.dataloader._target_ = "src.dataloader.template.TemplateSet"
    template_dataset = instantiate(cfg.data.test.dataloader)
    model.template_datasets = {cfg.test_dataset_name: template_dataset}
    model.test_dataset_name = cfg.test_dataset_name

    # Run testing
    trainer.test(model, dataloaders=test_dataloader, ckpt_path=cfg.model.checkpoint_path)
```

```bash
# Run coarse estimation for 6D detection task
python test.py test_dataset_name=hope run_id=my_experiment test_setting=detection

# Run coarse estimation for 6D localization task (core19 datasets only)
python test.py test_dataset_name=lmo run_id=my_experiment test_setting=localization
```

--------------------------------

### LocalSimilarity Module Implementation

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Performs template matching between query and template patches. Requires normalized features and mask inputs to compute similarity and cycle consistency.

```python
import torch
import torch.nn.functional as F
from einops import rearrange
from src.models.matching import LocalSimilarity

class LocalSimilarity(torch.nn.Module):
    def __init__(
        self,
        k=5,                     # Number of top templates to retrieve
        sim_threshold=0.5,       # Minimum similarity threshold
        patch_threshold=2,       # Cycle consistency threshold (patches)
        search_direction="tar2src",
        image_size=224,
        patch_size=14,
        max_batch_size=32,
    ):
        super().__init__()
        self.k = k
        self.num_patches = image_size // patch_size  # 16x16 patches

    def test(self, src_feats, tar_feat, src_masks, tar_mask, max_batch_size=None):
        """
        Find k nearest templates for each query.

        Args:
            src_feats: Template features (B, N_templates, C, H, W)
            tar_feat: Query features (B, C, H, W)
            src_masks: Template masks (B, N_templates, H, W)
            tar_mask: Query mask (B, H, W)

        Returns:
            PandasTensorCollection with:
                - id_src: Selected template indices (B, k)
                - src_pts: Source keypoints (B, k, N_patches, 2)
                - tar_pts: Target keypoints (B, k, N_patches, 2)
        """
        # Normalize features
        tar_feat = F.normalize(tar_feat, dim=1)
        src_feats = F.normalize(src_feats, dim=2)

        # Compute similarity matrix
        sim = torch.einsum("b c t, b n c s -> b n t s", tar_feat, src_feats)
        sim *= src_masks[:, :, None, :]  # Mask invalid regions
        sim *= tar_mask[:, None, :, None]

        # Find nearest neighbor per patch
        score_tar2src, idx_tar2src = torch.max(sim, dim=3)
        score_src2tar, idx_src2tar = torch.max(sim, dim=2)

        # Apply cycle consistency check
        mask_cycle = self.find_consistency_patches(
            sim_src2tar=score_src2tar,
            idx_src2tar=idx_src2tar,
            idx_tar2src=idx_tar2src,
        )

        # Select top-k templates
        sim_avg = torch.sum(score_tar2src * mask_cycle, dim=2) / (self.num_patches**2)
        pred_score_src, pred_id_src = torch.topk(sim_avg, self.k, dim=1)

        return predictions
```

--------------------------------

### Run Coarse Prediction on Single Dataset (BOP2023)

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Execute the test script for coarse prediction on a single dataset, specifying the dataset name and run ID. This is part of the process for reproducing BOP challenge 2023 results.

```bash
python test.py test_dataset_name=lmo run_id=$NAME_RUN
```

--------------------------------

### Object Pose Recovery Module

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Use this module to recover full 6D pose from 2D affine transforms, camera intrinsics, and template pose information. It requires template data and performs calculations for rotation, translation, and depth.

```python
import torch
from einops import rearrange, repeat
from src.models.poses import ObjectPoseRecovery
from src.lib3d.torch import inverse_affine, normalize_affine_transform

class ObjectPoseRecovery(torch.nn.Module):
    def __init__(self, template_K, template_Ms, template_poses, pixel_threshold=14):
        super().__init__()
        self.template_K = template_K       # Template camera intrinsics
        self.template_Ms = template_Ms     # Template crop transforms
        self.template_poses = template_poses  # Template 6D poses (4x4)
        self.ransac = RANSAC(pixel_threshold=pixel_threshold)

    def forward_recovery(self, tar_label, tar_K, tar_M, pred_src_views, pred_M):
        """
        Recover 6D pose from 2D predictions.

        Pipeline:
        1. Rotation = Template viewpoint rotation + In-plane (Kabsch) rotation
        2. 2D translation = Affine transform + Crop transforms
        3. Z depth = 2D scale + Focal length ratio

        Args:
            tar_label: Object labels (B,)
            tar_K: Target camera intrinsics (B, 3, 3)
            tar_M: Target crop transform (B, 3, 3)
            pred_src_views: Predicted template indices (B, k)
            pred_M: Predicted affine transforms (B, k, 3, 3)

        Returns:
            pred_poses: Recovered 6D poses (B, k, 4, 4)
        """
        # Get template data for predicted views
        template_poses = self.template_poses[tar_label - 1]
        template_Ms = self.template_Ms[tar_label - 1]

        # Extract in-plane rotation from affine transform
        pred_R_inplane = normalize_affine_transform(pred_M)

        # Apply in-plane rotation to template pose
        pred_poses = torch.gather(template_poses, 1, pred_src_views)
        pred_poses[:, :, :3, :3] = torch.matmul(pred_R_inplane, pred_poses[:, :, :3, :3])

        # Recover 2D center projection
        inv_query_M = inverse_affine(tar_M)
        affine2d = torch.matmul(torch.matmul(inv_query_M, pred_M), template_Ms)

        # Recover depth: Z_query = (Z_template / scale_2d) * (focal_ratio)
        scale2d = torch.norm(affine2d[:, :, :2, 0], dim=2)
        focal_ratio = tar_K[:, 0, 0] / template_K[:, 0, 0]
        query_z = (template_z / scale2d) * focal_ratio

        # Combine to get 3D translation
        query_translation = torch.matmul(torch.inverse(tar_K), query_center2d)
        pred_poses[:, :, :3, 3] = query_translation * query_z.unsqueeze(-1)

        return pred_poses
```

--------------------------------

### Run Coarse Prediction (BOP24)

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Executes coarse prediction for the 6D detection or localization task on BOP challenge 2024 datasets. Requires specifying the dataset name and a run ID.

```bash
# for 6D detection task
python test.py test_dataset_name=hope run_id=$NAME_RUN test_setting=detection
```

```bash
# for 6D localization task (for only core19 datasets)
python test.py test_dataset_name=lmo run_id=$NAME_RUN test_setting=localization
```

--------------------------------

### GigaPose Pre-compute Template Features

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Method to pre-compute and store appearance and IST features for template images within a given dataset. This optimizes subsequent retrieval operations.

```python
def set_template_data(self, dataset_name):
    """Pre-compute template features for efficient retrieval"""
    template_dataset = self.template_datasets[dataset_name]
    for idx in range(len(template_dataset)):
        templates = template_dataset[idx].rgb.to(self.device)
        ae_features = self.ae_net(templates)
        ist_features = self.ist_net.forward_by_chunk(templates)
        # Store features for nearest neighbor search
```

--------------------------------

### RANSAC Pose Estimation Module

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Estimates affine transforms from predicted correspondences and geometric parameters. Uses pixel-thresholding to identify inliers and select the best transform candidate.

```python
import torch
from src.models.ransac import RANSAC
from src.lib3d.torch import affine_torch, apply_affine

class RANSAC(torch.nn.Module):
    def __init__(self, pixel_threshold=5, patch_size=14):
        super().__init__()
        self.patch_size = patch_size
        self.pixel_threshold = pixel_threshold

    def forward_(self, src_pts, tar_pts, score, relScale, relInplane):
        """
        Find affine transform using RANSAC.

        Args:
            src_pts: Source keypoints (N, 2) in patch coordinates
            tar_pts: Target keypoints (N, 2) in patch coordinates
            score: Confidence scores (N,)
            relScale: Predicted relative scale (N,)
            relInplane: Predicted in-plane rotation as [cos, sin] (N, 2)

        Returns:
            dict with:
                - M: Best affine transform (3, 3)
                - failed: Whether RANSAC failed
                - index_inlier: Indices of inlier correspondences
        """
        N_total = src_pts.shape[0]
        device = src_pts.device

        # Convert to pixel coordinates
        src_pts = src_pts * self.patch_size
        tar_pts = tar_pts * self.patch_size

        # Compute rotation matrix from cos/sin
        cos_theta, sin_theta = relInplane[:, 0], relInplane[:, 1]
        R = torch.stack([cos_theta, -sin_theta, sin_theta, cos_theta], dim=1)
        R = R.reshape(-1, 2, 2)

        # Build affine transform candidates
        M_candidates = affine_torch(scale=relScale, rotation=R)
        aff_train_src_pts = apply_affine(M_candidates, src_pts)
        M_candidates[:, :2, 2] = tar_pts - aff_train_src_pts

        # Find inliers and select best transform
        aff_val_src_pts = apply_affine(M_candidates, src_pts)
        errors = torch.norm(tar_pts - aff_val_src_pts, dim=2)
        inliers = errors <= self.pixel_threshold

        score_inliers = torch.sum(inliers * score, dim=1)
        _, idx_best = torch.max(score_inliers, dim=0)

        return {
            "M": M_candidates[idx_best],
            "failed": score_inliers[idx_best] == 0,
            "index_inlier": inliers[idx_best].nonzero().flatten()
        }
```

--------------------------------

### Train GigaPose Model

Source: https://github.com/nv-nguyen/gigapose/blob/main/README.md

Train the GigaPose model using specified datasets. The `train_dataset_id` parameter selects the dataset: 0 for GSO, 1 for ShapeNet, or 2 for both.

```bash
# train on GSO (ID=0), ShapeNet (ID=1), or both (ID=2)
python train.py train_dataset_id=$ID
```

--------------------------------

### Local Similarity Matching

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

The LocalSimilarity module performs template matching between query image patches and template patches using cosine similarity and cycle consistency checks.

```APIDOC
## LocalSimilarity.test

### Description
Finds the k nearest templates for each query image by computing similarity matrices and applying cycle consistency.

### Parameters
- **src_feats** (torch.Tensor) - Required - Template features (B, N_templates, C, H, W)
- **tar_feat** (torch.Tensor) - Required - Query features (B, C, H, W)
- **src_masks** (torch.Tensor) - Required - Template masks (B, N_templates, H, W)
- **tar_mask** (torch.Tensor) - Required - Query mask (B, H, W)

### Response
- **id_src** (torch.Tensor) - Selected template indices (B, k)
- **src_pts** (torch.Tensor) - Source keypoints (B, k, N_patches, 2)
- **tar_pts** (torch.Tensor) - Target keypoints (B, k, N_patches, 2)
```

--------------------------------

### GigaPose Inference Pipeline

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

The main inference pipeline for the GigaPose model. It computes target features, finds nearest templates, estimates affine transform, and recovers the 6D pose using RANSAC.

```python
def eval_retrieval(self, batch, idx_batch, dataset_name):
    """
    Main inference pipeline:
    1. Compute target features
    2. Find nearest template via similarity search
    3. Estimate affine transform (scale, inplane, translation)
    4. Recover 6D pose using RANSAC
    """
    # Compute target features
    tar_ae_features = self.ae_net(batch.tar_img)

    # Template matching
    predictions = self.testing_metric.test(
        src_feats=template_data.ae_features,
        tar_feat=tar_ae_features,
        src_masks=template_data.mask,
        tar_mask=batch.tar_mask,
    )

    # Estimate affine parameters using IST network
    pred_scales, pred_cosSin_inplanes = self.ist_net.inference(...)

    # Recover pose using RANSAC
    predictions = self.pose_recovery[dataset_name].forward_ransac(predictions)
    pred_poses = self.pose_recovery[dataset_name].forward_recovery(...)

    return predictions
```

--------------------------------

### Compute ADD Metric

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

Calculate the Average Distance of Distinguishable model points between predicted and ground truth poses.

```python
import numpy as np
from src.lib3d.metric import add

def add(pred, gt, pcd, scale):
    """
    Compute ADD metric between predicted and ground truth poses.

    Args:
        pred: Predicted pose (4, 4) transformation matrix
        gt: Ground truth pose (4, 4) transformation matrix
        pcd: Object point cloud (N, 3)
        scale: Scale factor for translation

    Returns:
        dist: Per-point distances (N,)
    """
    # Transform point cloud with predicted pose
    pred_pcd = np.matmul(pred[:3, :3], pcd.T) + pred[:3, 3].reshape(-1, 1) * scale

    # Transform point cloud with ground truth pose
    gt_pcd = np.matmul(gt[:3, :3], pcd.T) + gt[:3, 3].reshape(-1, 1) * scale

    # Compute Euclidean distance
    dist = np.linalg.norm(pred_pcd - gt_pcd, axis=0)
    return dist

# Example usage
pred_pose = np.eye(4)  # Predicted 6D pose
gt_pose = np.eye(4)    # Ground truth pose
point_cloud = np.random.randn(1000, 3)  # Object points

distances = add(pred_pose, gt_pose, point_cloud, scale=1.0)
add_score = np.mean(distances)
print(f"ADD score: {add_score:.4f}")
```

--------------------------------

### RANSAC Pose Estimation

Source: https://context7.com/nv-nguyen/gigapose/llms.txt

The RANSAC module estimates the best affine transform from predicted correspondences, scale, and in-plane rotation.

```APIDOC
## RANSAC.forward_

### Description
Estimates an affine transform using RANSAC based on provided keypoints and geometric parameters.

### Parameters
- **src_pts** (torch.Tensor) - Required - Source keypoints (N, 2) in patch coordinates
- **tar_pts** (torch.Tensor) - Required - Target keypoints (N, 2) in patch coordinates
- **score** (torch.Tensor) - Required - Confidence scores (N,)
- **relScale** (torch.Tensor) - Required - Predicted relative scale (N,)
- **relInplane** (torch.Tensor) - Required - Predicted in-plane rotation as [cos, sin] (N, 2)

### Response
- **M** (torch.Tensor) - Best affine transform (3, 3)
- **failed** (bool) - Whether RANSAC failed
- **index_inlier** (torch.Tensor) - Indices of inlier correspondences
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.