Wildlife Tools

https://github.com/wildlifedatasets/wildlife-tools
Admin
Wildlife Tools is a toolkit for animal individual identification that covers use cases such as...
Tokens:25,146
Snippets:222
Trust Score:7.4
License:MIT
Update:4 months ago
Show doc for...
Tokens:
Raw
### Install wildlife-tools from source

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/docs/index.md

Clone the repository and install the library in editable mode.

```script
git clone git@github.com:WildlifeDatasets/wildlife-tools.git

cd wildlife-tools
pip install -e .
```

--------------------------------

### Install wildlife-tools using pip

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/docs/index.md

Install the library directly from its GitHub repository using pip.

```script
pip install git+https://github.com/WildlifeDatasets/wildlife-tools
```

--------------------------------

### MegaDescriptor-B-224 Training Setup

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/training/MegaDescriptor-B-224.ipynb

This Python script configures and initiates the training process for the MegaDescriptor-B-224 model. It includes dataset preparation, model and loss function setup, and optimizer/scheduler configuration.

```python
from itertools import chain

import pandas as pd
import timm
import torch
import torchvision.transforms as T
from torch.optim import SGD

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.train import ArcFaceLoss, BasicTrainer

# Dataset configuration
metadata = pd.read_csv("../data/metadata/combined/combined_all.csv")
image_root = "../data/images/size-256"
transform = T.Compose(
    [
        T.RandomResizedCrop(size=(224, 224), scale=(0.8, 1.0)),
        T.RandAugment(num_ops=2, magnitude=20),
        T.ToTensor(),
        T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
    ]
)
dataset = WildlifeDataset(metadata=metadata.query('split == "train"'), root=image_root, transform=transform)


# Backbone and loss configuration
backbone = timm.create_model("swin_base_patch4_window7_224", num_classes=0, pretrained=True)
with torch.no_grad():
    dummy_input = torch.randn(1, 3, 224, 224)
    embedding_size = backbone(dummy_input).shape[1]
objective = ArcFaceLoss(num_classes=dataset.num_classes, embedding_size=embedding_size, margin=0.5, scale=64)


# Optimizer and scheduler configuration
params = chain(backbone.parameters(), objective.parameters())
optimizer = SGD(params=params, lr=0.001, momentum=0.9)
min_lr = optimizer.defaults.get("lr") * 1e-3
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=100, eta_min=min_lr)


# Setup training
trainer = BasicTrainer(
    dataset=dataset,
    model=backbone,
    objective=objective,
    optimizer=optimizer,
    scheduler=scheduler,
    batch_size=64,
    accumulation_steps=2,
    num_workers=2,
    epochs=100,
    device="cuda",
)


trainer.train()
```

--------------------------------

### MegaDescriptor-S-224 Training Setup

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/training/MegaDescriptor-S-224.ipynb

Sets up the dataset, model backbone, loss function, optimizer, and scheduler for training. This snippet is used for configuring the training pipeline.

```python
from itertools import chain

import pandas as pd
import timm
import torch
import torchvision.transforms as T
from torch.optim import SGD

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.train import ArcFaceLoss, BasicTrainer

# Dataset configuration
metadata = pd.read_csv("../data/metadata/combined/combined_all.csv")
image_root = "../data/images/size-256"
transform = T.Compose(
    [
        T.RandomResizedCrop(size=(224, 224), scale=(0.8, 1.0)),
        T.RandAugment(num_ops=2, magnitude=20),
        T.ToTensor(),
        T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
    ]
)
dataset = WildlifeDataset(metadata=metadata.query('split == "train"'), root=image_root, transform=transform)


# Backbone and loss configuration
backbone = timm.create_model("swin_small_patch4_window7_224", num_classes=0, pretrained=True)
with torch.no_grad():
    dummy_input = torch.randn(1, 3, 224, 224)
    embedding_size = backbone(dummy_input).shape[1]
objective = ArcFaceLoss(num_classes=dataset.num_classes, embedding_size=embedding_size, margin=0.5, scale=64)


# Optimizer and scheduler configuration
params = chain(backbone.parameters(), objective.parameters())
optimizer = SGD(params=params, lr=0.001, momentum=0.9)
min_lr = optimizer.defaults.get("lr") * 1e-3
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=100, eta_min=min_lr)


# Setup training
trainer = BasicTrainer(
    dataset=dataset,
    model=backbone,
    objective=objective,
    optimizer=optimizer,
    scheduler=scheduler,
    batch_size=64,
    accumulation_steps=2,
    num_workers=2,
    epochs=100,
    device="cuda",
)


trainer.train()
```

--------------------------------

### MegaDescriptor-T-224 Training Setup

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/training/MegaDescriptor-T-224.ipynb

This Python script configures and runs the training process for the MegaDescriptor-T-224 model. It includes dataset loading, model and loss function initialization, optimizer and scheduler setup, and the training execution.

```python
from itertools import chain

import pandas as pd
import timm
import torch
import torchvision.transforms as T
from torch.optim import SGD

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.train import ArcFaceLoss, BasicTrainer

# Dataset configuration
metadata = pd.read_csv("../data/metadata/combined/combined_all.csv")
image_root = "../data/images/size-256"
transform = T.Compose(
    [
        T.RandomResizedCrop(size=(224, 224), scale=(0.8, 1.0)),
        T.RandAugment(num_ops=2, magnitude=20),
        T.ToTensor(),
        T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
    ]
)
dataset = WildlifeDataset(metadata=metadata.query('split == "train"'), root=image_root, transform=transform)


# Backbone and loss configuration
backbone = timm.create_model("swin_tiny_patch4_window7_224", num_classes=0, pretrained=True)
with torch.no_grad():
    dummy_input = torch.randn(1, 3, 224, 224)
    embedding_size = backbone(dummy_input).shape[1]
objective = ArcFaceLoss(num_classes=dataset.num_classes, embedding_size=embedding_size, margin=0.5, scale=64)


# Optimizer and scheduler configuration
params = chain(backbone.parameters(), objective.parameters())
optimizer = SGD(params=params, lr=0.001, momentum=0.9)
min_lr = optimizer.defaults.get("lr") * 1e-3
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=100, eta_min=min_lr)


# Setup training
trainer = BasicTrainer(
    dataset=dataset,
    model=backbone,
    objective=objective,
    optimizer=optimizer,
    scheduler=scheduler,
    batch_size=64,
    accumulation_steps=2,
    num_workers=2,
    epochs=100,
    device="cuda",
)


trainer.train()
```

--------------------------------

### MegaDescriptor-L-384 Training Setup

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/training/MegaDescriptor-L-384.ipynb

This snippet configures and initiates the training process for the MegaDescriptor-L-384 model. It includes dataset loading, model and loss function setup, optimizer and scheduler configuration, and the final training execution.

```python
from itertools import chain

import pandas as pd
import timm
import torch
import torchvision.transforms as T
from torch.optim import SGD

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.train import ArcFaceLoss, BasicTrainer

# Dataset configuration
metadata = pd.read_csv("../data/metadata/combined/combined_all.csv")
image_root = "../data/images/size-518"
transform = T.Compose(
    [
        T.Resize(size=(384, 384)),
        T.RandAugment(num_ops=2, magnitude=20),
        T.ToTensor(),
        T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
    ]
)
dataset = WildlifeDataset(metadata=metadata.query('split == "train"'), root=image_root, transform=transform)


# Backbone and loss configuration
backbone = timm.create_model("swin_large_patch4_window12_384", num_classes=0, pretrained=True)
with torch.no_grad():
    dummy_input = torch.randn(1, 3, 384, 384)
    embedding_size = backbone(dummy_input).shape[1]
objective = ArcFaceLoss(num_classes=dataset.num_classes, embedding_size=embedding_size, margin=0.5, scale=64)


# Optimizer and scheduler configuration
params = chain(backbone.parameters(), objective.parameters())
optimizer = SGD(params=params, lr=0.001, momentum=0.9)
min_lr = optimizer.defaults.get("lr") * 1e-3
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=100, eta_min=min_lr)


# Setup training
trainer = BasicTrainer(
    dataset=dataset,
    model=backbone,
    objective=objective,
    optimizer=optimizer,
    scheduler=scheduler,
    batch_size=16,
    accumulation_steps=8,
    num_workers=2,
    epochs=100,
    device="cuda",
)


trainer.train()
```

--------------------------------

### MegaDescriptor-L-224 Training Setup

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/training/MegaDescriptor-L-224.ipynb

This snippet configures and initiates the training process for the MegaDescriptor-L-224 model. It includes dataset loading, image augmentation, model backbone initialization, ArcFace loss setup, and defines the optimizer and learning rate scheduler.

```python
from itertools import chain

import pandas as pd
import timm
import torch
import torchvision.transforms as T
from torch.optim import SGD

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.train import ArcFaceLoss, BasicTrainer

# Dataset configuration
metadata = pd.read_csv("../data/metadata/combined/combined_all.csv")
image_root = "../data/images/size-256"
transform = T.Compose(
    [
        T.RandomResizedCrop(size=(224, 224), scale=(0.8, 1.0)),
        T.RandAugment(num_ops=2, magnitude=20),
        T.ToTensor(),
        T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
    ]
)
dataset = WildlifeDataset(metadata=metadata.query('split == "train"'), root=image_root, transform=transform)

# Backbone and loss configuration
backbone = timm.create_model("swin_large_patch4_window7_224", num_classes=0, pretrained=True)
with torch.no_grad():
    dummy_input = torch.randn(1, 3, 224, 224)
    embedding_size = backbone(dummy_input).shape[1]
objective = ArcFaceLoss(num_classes=dataset.num_classes, embedding_size=embedding_size, margin=0.5, scale=64)


# Optimizer and scheduler configuration
params = chain(backbone.parameters(), objective.parameters())
optimizer = SGD(params=params, lr=0.001, momentum=0.9)
min_lr = optimizer.defaults.get("lr") * 1e-3
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=100, eta_min=min_lr)


# Setup training
trainer = BasicTrainer(
    dataset=dataset,
    model=backbone,
    objective=objective,
    optimizer=optimizer,
    scheduler=scheduler,
    batch_size=64,
    accumulation_steps=2,
    num_workers=2,
    epochs=100,
    device="cuda",
)


trainer.train()
```

--------------------------------

### Import Libraries and Initialize Model

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-T-224.ipynb

Imports necessary libraries and initializes the MegaDescriptor-T-224 model for feature extraction. Ensure 'timm' and 'torchvision' are installed.

```python
import pandas as pd
from timm import create_model
from torchvision import transforms as T

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.features import DeepFeatures
from wildlife_tools.inference import KnnClassifier
from wildlife_tools.similarity import CosineSimilarity

datasets = [
    "BirdIndividualID",
    "SealID",
    "FriesianCattle2015",
    "ATRW",
    "NDD20",
    "SMALST",
    "SeaTurtleIDHeads",
    "AAUZebraFish",
    "CZoo",
    "CTai",
    "Giraffes",
    "HyenaID2022",
    "MacaqueFaces",
    "OpenCows2020",
    "StripeSpotter",
    "AerialCattle2017",
    "GiraffeZebraID",
    "IPanda50",
    "WhaleSharkID",
    "FriesianCattle2017",
    "Cows2021",
    "LeopardID2022",
    "NOAARightWhale",
    "HappyWhale",
    "HumpbackWhaleID",
    "LionData",
    "NyalaData",
    "ZindiTurtleRecall",
    "BelugaID",
]

model = create_model("hf-hub:BVRA/MegaDescriptor-T-224", pretrained=True)
extractor = DeepFeatures(model, device="cuda")

root_images = "../data/images/size-256"
root_metadata = "../data/metadata/datasets"

```

--------------------------------

### WildFusion with B=1 parameter

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-wildfusion.ipynb

This example demonstrates running WildFusion with the 'B' parameter set to 1. This parameter likely influences the comparison algorithm or output format.

```python
wildfusion(dataset, dataset, B=1)
```

--------------------------------

### Import Libraries and Load Model

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-L-384.ipynb

Imports necessary libraries and creates a MegaDescriptor-L-384 model instance for feature extraction. Ensure PyTorch and timm are installed.

```python
import pandas as pd
from timm import create_model
from torchvision import transforms as T

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.features import DeepFeatures
from wildlife_tools.inference import KnnClassifier
from wildlife_tools.similarity import CosineSimilarity

datasets = [
    "BirdIndividualID",
    "SealID",
    "FriesianCattle2015",
    "ATRW",
    "NDD20",
    "SMALST",
    "SeaTurtleIDHeads",
    "AAUZebraFish",
    "CZoo",
    "CTai",
    "Giraffes",
    "HyenaID2022",
    "MacaqueFaces",
    "OpenCows2020",
    "StripeSpotter",
    "AerialCattle2017",
    "GiraffeZebraID",
    "IPanda50",
    "WhaleSharkID",
    "FriesianCattle2017",
    "Cows2021",
    "LeopardID2022",
    "NOAARightWhale",
    "HappyWhale",
    "HumpbackWhaleID",
    "LionData",
    "NyalaData",
    "ZindiTurtleRecall",
    "BelugaID",
]

model = create_model("hf-hub:BVRA/wildlife-mega-L-384", pretrained=True)
extractor = DeepFeatures(model, device="cuda")

root_images = "../data/images/size-518"
root_metadata = "../data/metadata/datasets"

```

--------------------------------

### Initialize MegaDescriptor-B-224 Model and Feature Extractor

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-B-224.ipynb

This snippet initializes the MegaDescriptor-B-224 model from Hugging Face and sets up a DeepFeatures extractor. Ensure you have the necessary libraries (timm, torchvision, wildlife-tools) installed and the model is accessible.

```python
import pandas as pd
from timm import create_model
from torchvision import transforms as T

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.features import DeepFeatures
from wildlife_tools.inference import KnnClassifier
from wildlife_tools.similarity import CosineSimilarity

datasets = [
    "BirdIndividualID",
    "SealID",
    "FriesianCattle2015",
    "ATRW",
    "NDD20",
    "SMALST",
    "SeaTurtleIDHeads",
    "AAUZebraFish",
    "CZoo",
    "CTai",
    "Giraffes",
    "HyenaID2022",
    "MacaqueFaces",
    "OpenCows2020",
    "StripeSpotter",
    "AerialCattle2017",
    "GiraffeZebraID",
    "IPanda50",
    "WhaleSharkID",
    "FriesianCattle2017",
    "Cows2021",
    "LeopardID2022",
    "NOAARightWhale",
    "HappyWhale",
    "HumpbackWhaleID",
    "LionData",
    "NyalaData",
    "ZindiTurtleRecall",
    "BelugaID",
]

model = create_model("hf-hub:BVRA/MegaDescriptor-B-224", pretrained=True)
extractor = DeepFeatures(model, device="cuda")

root_images = "../data/images/size-256"
root_metadata = "../data/metadata/datasets"
```

--------------------------------

### Inference Output Example 1

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-L-384.ipynb

Sample output from the inference process, likely representing a detected entity or score.

```text
ZindiTurtleRecall 0.7439974042829332
```

--------------------------------

### Inference Output Example 2

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-L-384.ipynb

Another sample output from the inference, possibly indicating a different type of detection or classification.

```text
BelugaID 0.6648394675019577
```

--------------------------------

### Extract Deep Features using MegaDescriptor Tiny

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/README.md

Extracts deep features from an ImageDataset using a pre-trained MegaDescriptor Tiny model from HuggingFace. Ensure the 'timm' library is installed.

```Python
import timm
from wildlife_tools.features import DeepFeatures

name = 'hf-hub:BVRA/MegaDescriptor-T-224'
extractor = DeepFeatures(timm.create_model(name, num_classes=0, pretrained=True))
query, database = extractor(dataset_query), extractor(dataset_database)
```

--------------------------------

### SimilarityPipeline with LightGlue and SuperPoint

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/docs/wildfusion.md

Demonstrates setting up a SimilarityPipeline using LightGlue for matching, SuperPoint for descriptors, and IsotonicRegression for calibration. Images are resized to 512x512 before processing. Calibration must be fitted to the dataset before use.

```Python
import timm
import torchvision.transforms as T
from wildlife_tools.features import SuperPointExtractor
from wildlife_tools.similarity import MatchLightGlue

from wildlife_tools.similarity.wildfusion import SimilarityPipeline
from wildlife_tools.similarity.calibration import IsotonicCalibration


pipeline = SimilarityPipeline(
  matcher = MatchLightGlue(features='superpoint'),
  extractor = SuperPointExtractor(),
  transform = T.Compose([
      T.Resize([512, 512]),
      T.ToTensor()
  ]),
  calibration = IsotonicCalibration()
),
pipeline.fit_calibration(calibration_dataset1, calibration_dataset2)
scores = pipeline(query, database)
```

--------------------------------

### Initialize WildFusion Similarity Pipelines

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-wildfusion.ipynb

Sets up a list of SimilarityPipeline objects, each configured with a specific matcher, feature extractor, image transform, and calibration method. This is used to define the various comparison strategies within WildFusion.

```python
pipelines = [
    SimilarityPipeline(
        matcher=MatchLightGlue(features="superpoint"),
        extractor=SuperPointExtractor(),
        transform=T.Compose([T.Resize([512, 512]), T.ToTensor()]),
        calibration=IsotonicCalibration(),
    ),
    SimilarityPipeline(
        matcher=MatchLightGlue(features="aliked"),
        extractor=AlikedExtractor(),
        transform=T.Compose([T.Resize([512, 512]), T.ToTensor()]),
        calibration=IsotonicCalibration(),
    ),
    SimilarityPipeline(
        matcher=MatchLightGlue(features="disk"),
        extractor=DiskExtractor(),
        transform=T.Compose([T.Resize([512, 512]), T.ToTensor()]),
        calibration=IsotonicCalibration(),
    ),
    SimilarityPipeline(
        matcher=MatchLightGlue(features="sift"),
        extractor=SiftExtractor(),
        transform=T.Compose([T.Resize([512, 512]), T.ToTensor()]),
        calibration=IsotonicCalibration(),
    ),
    SimilarityPipeline(
        matcher=MatchLOFTR(pretrained="outdoor"),
        extractor=None,
        transform=T.Compose(
            [
                T.Resize([512, 512]),
                T.Grayscale(),
                T.ToTensor(),
            ]
        ),
        calibration=IsotonicCalibration(),
    ),
    SimilarityPipeline(
        matcher=CosineSimilarity(),
        extractor=DeepFeatures(
            model=timm.create_model("hf-hub:BVRA/wildlife-mega-L-384", num_classes=0, pretrained=True)
        ),
        transform=T.Compose(
            [
                T.Resize(size=(384, 384)),
                T.ToTensor(),
                T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
            ]
        ),
        calibration=IsotonicCalibration(),
    ),
]


priority_pipeline = SimilarityPipeline(
    matcher=CosineSimilarity(),
    extractor=DeepFeatures(model=timm.create_model("hf-hub:BVRA/wildlife-mega-L-384", num_classes=0, pretrained=True)),
    transform=T.Compose(
        [
            T.Resize(size=(384, 384)),
            T.ToTensor(),
            T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
        ]
    ),
)

wildfusion = WildFusion(calibrated_pipelines=pipelines, priority_pipeline=priority_pipeline)
```

--------------------------------

### Initialize ImageDataset

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-features.ipynb

Sets up the ImageDataset with metadata and a specified transform. This is a prerequisite for using feature extractors.

```python
import pandas as pd
import timm
import torchvision.transforms as T

from wildlife_tools.data import ImageDataset
from wildlife_tools.features import AlikedExtractor, DeepFeatures, DiskExtractor, SiftExtractor, SuperPointExtractor

metadata = {"metadata": pd.read_csv("../tests/TestDataset/metadata.csv"), "root": "../tests/TestDataset"}
transform = T.Compose([T.Resize([224, 224]), T.ToTensor()])
dataset = ImageDataset(**metadata, transform=transform)
```

--------------------------------

### Load MegaDescriptor-L-224 Model and Initialize Feature Extractor

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-L-224.ipynb

Initializes the MegaDescriptor-L-224 model from Hugging Face Hub and sets up the DeepFeatures extractor. Ensure 'cuda' is available for GPU acceleration.

```python
import pandas as pd
from timm import create_model
from torchvision import transforms as T

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.features import DeepFeatures
from wildlife_tools.inference import KnnClassifier
from wildlife_tools.similarity import CosineSimilarity

datasets = [
    "BirdIndividualID",
    "SealID",
    "FriesianCattle2015",
    "ATRW",
    "NDD20",
    "SMALST",
    "SeaTurtleIDHeads",
    "AAUZebraFish",
    "CZoo",
    "CTai",
    "Giraffes",
    "HyenaID2022",
    "MacaqueFaces",
    "OpenCows2020",
    "StripeSpotter",
    "AerialCattle2017",
    "GiraffeZebraID",
    "IPanda50",
    "WhaleSharkID",
    "FriesianCattle2017",
    "Cows2021",
    "LeopardID2022",
    "NOAARightWhale",
    "HappyWhale",
    "HumpbackWhaleID",
    "LionData",
    "NyalaData",
    "ZindiTurtleRecall",
    "BelugaID",
]

model = create_model("hf-hub:BVRA/MegaDescriptor-L-224", pretrained=True)
extractor = DeepFeatures(model, device="cuda")

root_images = "../data/images/size-256"
root_metadata = "../data/metadata/datasets"

```

--------------------------------

### WildFusion with Shortlisting and Budget

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/docs/wildfusion.md

Configures WildFusion to use a specific SimilarityPipeline for initial shortlisting (e.g., CosineSimilarity with MegaDescriptor features) and sets a budget for score calculations per query image. Calibration must be fitted beforehand.

```Python
priority_matcher =  SimilarityPipeline(
    matcher = CosineSimilarity(),
    extractor = DeepFeatures(
        model = timm.create_model(
        'hf-hub:BVRA/wildlife-mega-L-384',
        num_classes=0,
        pretrained=True
        )
    ),
    transform = T.Compose([
        T.Resize(size=(384, 384)),
        T.ToTensor(),
        T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
    ]),
)

wildfusion = WildFusion(calibrated_matchers = matchers)
wildfusion.fit_calibration(calibration_dataset1, calibration_dataset2)
similarity = wildfusion(query, database, B=100)
```

--------------------------------

### WildFusion with Multiple Similarity Pipelines

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/docs/wildfusion.md

Illustrates the creation of a WildFusion object by aggregating multiple SimilarityPipeline instances, each configured with different matching strategies (LightGlue, LOFTR, CosineSimilarity) and feature extractors. Calibration is fitted to the datasets prior to running the fusion.

```Python
import timm
import torchvision.transforms as T
from wildlife_tools.features import *
from wildlife_tools.similarity import CosineSimilarity, MatchLOFTR, MatchLightGlue
from wildlife_tools.similarity.wildfusion import SimilarityPipeline, WildFusion
from wildlife_tools.similarity.calibration import IsotonicCalibration


matchers = [

    SimilarityPipeline(
        matcher = MatchLightGlue(features='superpoint'),
        extractor = SuperPointExtractor(),
        transform = T.Compose([
            T.Resize([512, 512]),
            T.ToTensor()
        ]),
        calibration = IsotonicCalibration()
    ),

    SimilarityPipeline(
        matcher = MatchLightGlue(features='aliked'),
        extractor = AlikedExtractor(),
        transform = T.Compose([
            T.Resize([512, 512]),
            T.ToTensor()
        ]),
        calibration = IsotonicCalibration()
    ),

    SimilarityPipeline(
        matcher = MatchLightGlue(features='disk'),
        extractor = DiskExtractor(),
        transform = T.Compose([
            T.Resize([512, 512]),
            T.ToTensor()
        ]),
        calibration = IsotonicCalibration()
    ),

    SimilarityPipeline(
        matcher = MatchLightGlue(features='sift'),
        extractor = SiftExtractor(),
        transform = T.Compose([
            T.Resize([512, 512]),
            T.ToTensor()
        ]),
        calibration = IsotonicCalibration()
    ),

    SimilarityPipeline(
        matcher = MatchLOFTR(pretrained='outdoor'),
        extractor = None,
        transform = T.Compose([
            T.Resize([512, 512]),
            T.Grayscale(),
            T.ToTensor(),
        ]),
        calibration = IsotonicCalibration()
    ),

    SimilarityPipeline(
        matcher = CosineSimilarity(),
        extractor = DeepFeatures(
            model = timm.create_model(
              'hf-hub:BVRA/wildlife-mega-L-384',
              num_classes=0,
              pretrained=True
              )
        ),
        transform = T.Compose([
            T.Resize(size=(384, 384)),
            T.ToTensor(),
            T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
        ]),
        calibration = IsotonicCalibration()
    ),
]

wildfusion = WildFusion(calibrated_matchers = matchers)
wildfusion.fit_calibration(calibration_dataset1, calibration_dataset2)
similarity = wildfusion(query, database)
```

--------------------------------

### Match Images with Rotation using LoFTR and LightGlue

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-pairwise-sim.ipynb

Compares LoFTR and LightGlue for matching images with significant rotation. A lower init_threshold is used due to the rotation. The visualization shows potential spurious matches.

```python
idx0 = 2
idx1 = 3

img0, _ = dataset_img[idx0]
img1, _ = dataset_img[idx1]

# Match the images - LOFTR
dataset.transform = T.Compose([T.Resize([256, 256]), T.Grayscale(), T.ToTensor()])
matcher = MatchLOFTR(collector=CollectAll(), init_threshold=0.68)
results_loftr = matcher(dataset, dataset, pairs=[(idx0, idx1)])[0]


# Match the images - Light GLue
dataset.transform = T.Compose([T.Resize([256, 256]), T.ToTensor()])
dataset_feat = SuperPointExtractor()(dataset)
matcher = MatchLightGlue(features="superpoint", collector=CollectAll(), init_threshold=0.65)
results_sp = matcher(dataset_feat, dataset_feat, pairs=[(idx0, idx1)])[0]


# Visualise
fig, ax = plt.subplots(2, 1, figsize=(10, 10))
ax[0].set_title("LOFTR")
ax[1].set_title("Superpoint + LightGlue")
visualise_matches(img0, results_loftr["kpts0"], img1, results_loftr["kpts1"], ax=ax[0])
visualise_matches(img0, results_sp["kpts0"], img1, results_sp["kpts1"], ax=ax[1])
```

--------------------------------

### Match Similar Images with LoFTR and LightGlue

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-pairwise-sim.ipynb

Compares LoFTR and LightGlue for matching similar images. LoFTR uses a grayscale transform, while LightGlue uses SuperPoint features. Both use a high init_threshold.

```python
idx0 = 0
idx1 = 1

img0, _ = dataset_img[idx0]
img1, _ = dataset_img[idx1]

# Match the images - LOFTR
dataset.transform = T.Compose([T.Resize([256, 256]), T.Grayscale(), T.ToTensor()])
matcher = MatchLOFTR(collector=CollectAll(), init_threshold=0.99)
results_loftr = matcher(dataset, dataset, pairs=[(idx0, idx1)])[0]


# Match the images - Light GLue
dataset.transform = T.Compose([T.Resize([256, 256]), T.ToTensor()])
dataset_feat = SuperPointExtractor()(dataset)
matcher = MatchLightGlue(features="superpoint", collector=CollectAll(), init_threshold=0.8)
results_sp = matcher(dataset_feat, dataset_feat, pairs=[(idx0, idx1)])[0]


# Visualise
fig, ax = plt.subplots(2, 1, figsize=(10, 10))
ax[0].set_title("LOFTR")
ax[1].set_title("Superpoint + LightGlue")
visualise_matches(img0, results_loftr["kpts0"], img1, results_loftr["kpts1"], ax=ax[0])
visualise_matches(img0, results_sp["kpts0"], img1, results_sp["kpts1"], ax=ax[1])
```

--------------------------------

### Process and Save Datasets

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/data/prepare_datasets.ipynb

Iterates through dataset preparation functions, resizing images to 256x256 and 518x518, and saving them to specified directories. It also asserts that the metadata for both sizes is identical.

```python
datasets_folder = "/mnt/data/turtles/datasets/datasets"  # Path to downloaded datasets

# Create folders with images resized to 256 and 518
for name, prepare in prepare_functions.items():
    print(name)
    prepare(size=256, root=f"{datasets_folder}/{name}", new_root=f"images/size-256/{name}")
    prepare(size=518, root=f"{datasets_folder}/{name}", new_root=f"images/size-518/{name}")

    # Metadata should be the same
    metadata_256 = pd.read_csv(f"images/size-256/{name}/annotations.csv", index_col=0)
    metadata_518 = pd.read_csv(f"images/size-518/{name}/annotations.csv", index_col=0)
    assert metadata_256.equals(metadata_518)
```

--------------------------------

### Initialize MegaDescriptor-S-224 Model and Feature Extractor

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-S-224.ipynb

Loads the pre-trained MegaDescriptor-S-224 model from Hugging Face Hub and initializes a DeepFeatures extractor. Ensure CUDA is available for GPU acceleration.

```python
import pandas as pd
from timm import create_model
from torchvision import transforms as T

from wildlife_tools.data import WildlifeDataset
from wildlife_tools.features import DeepFeatures
from wildlife_tools.inference import KnnClassifier
from wildlife_tools.similarity import CosineSimilarity

datasets = [
    "BirdIndividualID",
    "SealID",
    "FriesianCattle2015",
    "ATRW",
    "NDD20",
    "SMALST",
    "SeaTurtleIDHeads",
    "AAUZebraFish",
    "CZoo",
    "CTai",
    "Giraffes",
    "HyenaID2022",
    "MacaqueFaces",
    "OpenCows2020",
    "StripeSpotter",
    "AerialCattle2017",
    "GiraffeZebraID",
    "IPanda50",
    "WhaleSharkID",
    "FriesianCattle2017",
    "Cows2021",
    "LeopardID2022",
    "NOAARightWhale",
    "HappyWhale",
    "HumpbackWhaleID",
    "LionData",
    "NyalaData",
    "ZindiTurtleRecall",
    "BelugaID",
]

model = create_model("hf-hub:BVRA/MegaDescriptor-S-224", pretrained=True)
extractor = DeepFeatures(model, device="cuda")

root_images = "../data/images/size-256"
root_metadata = "../data/metadata/datasets"

```

--------------------------------

### Initialize ImageDataset

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-pairwise-sim.ipynb

Initializes an ImageDataset with metadata and applies a resize transform.

```python
import pandas as pd
import torchvision.transforms as T

from wildlife_tools.data import ImageDataset
from wildlife_tools.features import SuperPointExtractor
from wildlife_tools.similarity import MatchLightGlue, MatchLOFTR
from wildlife_tools.similarity.pairwise.collectors import CollectAll, CollectCounts, CollectCountsRansac

metadata = {"metadata": pd.read_csv("../tests/TestDataset/metadata.csv"), "root": "../tests/TestDataset"}
dataset = ImageDataset(**metadata)
dataset_img = ImageDataset(**metadata, transform=T.Resize([256, 256]))
```

--------------------------------

### Run WildFusion for all dataset pairs

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-wildfusion.ipynb

This snippet shows how to run WildFusion comparing a dataset against itself. Note that this includes comparisons of the same dataset for both query and database, resulting in a diagonal of 1.0 in the output matrix.

```python
wildfusion(dataset, dataset)
```

--------------------------------

### Create ImageDataset from MacaqueFaces

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/README.md

Initializes an ImageDataset object using metadata from MacaqueFaces and applies specified transformations. Ensure the 'datasets/MacaqueFaces' path is correct.

```Python
from wildlife_datasets.datasets import MacaqueFaces
from wildlife_tools.data import ImageDataset
import torchvision.transforms as T

metadata = MacaqueFaces('datasets/MacaqueFaces')
transform = T.Compose([T.Resize([224, 224]), T.ToTensor(), T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))])
dataset = ImageDataset(metadata.df, metadata.root, transform=transform)
```

--------------------------------

### OpenCows2020 Dataset Performance

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-L-384.ipynb

Displays the progress and final accuracy for the OpenCows2020 dataset.

```text
100%|█████████████████████████████████████████████████████████████████| 2/2 [00:08<00:00,  4.25s/it]
100%|█████████████████████████████████████████████████████████████████| 6/6 [00:21<00:00,  3.66s/it]

Output:
OpenCows2020 1.0
```

--------------------------------

### Dataset Preparation Functions

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/data/prepare_datasets.ipynb

A dictionary mapping dataset names to their respective preparation functions. This is used to systematically process various wildlife datasets.

```python
from prepare_data import *

prepare_functions = {
    "NyalaData": prepare_nyala_data,
    "ZindiTurtleRecall": prepare_zindi_turtle_recall,
    "BelugaID": prepare_beluga_id,
    "BirdIndividualID": prepare_bird_individual_id,
    "SealID": prepare_seal_id,
    "FriesianCattle2015": prepare_friesian_cattle_2015,
    "ATRW": prepare_atrw,
    "NDD20": prepare_ndd20,
    "SMALST": prepare_smalst,
    "SeaTurtleIDHeads": prepare_sea_turtle_id_heads,
    "AAUZebraFish": prepare_zebra_fish,
    "CZoo": prepare_czoo,
    "CTai": prepare_ctai,
    "Giraffes": prepare_giraffes,
    "HyenaID2022": prepare_hyena_id_2022,
    "MacaqueFaces": prepare_macaque_faces,
    "OpenCows2020": prepare_open_cows_2020,
    "StripeSpotter": prepare_stripe_spotter,
    "AerialCattle2017": prepare_aerial_cattle_2017,
    "GiraffeZebraID": prepare_giraffe_zebra_id,
    "IPanda50": prepare_ipanda_50,
    "WhaleSharkID": prepare_whaleshark_id,
    "FriesianCattle2017": prepare_friesian_cattle_2017,
    "Cows2021": prepare_cows2021,
    "LeopardID2022": prepare_leopard_id_2022,
    "NOAARightWhale": prepare_noaa_right_whale,
    "HappyWhale": prepare_happy_whale,
    "HumpbackWhaleID": prepare_humpback_whale_id,
    "LionData": prepare_lion_data,
}
```

--------------------------------

### Initialize and Match with LoFTR

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-pairwise-sim.ipynb

Initializes the LoFTR matcher and performs pairwise matching on a dataset. The default score is the number of significant correspondences (confidence > 0.5).

```python
dataset.transform = T.Compose([T.Resize([256, 256]), T.Grayscale(), T.ToTensor()])

matcher = MatchLOFTR()
output = matcher(dataset, dataset)
output
```

--------------------------------

### Split Dataset into Database and Query Sets

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/docs/inference.md

Splits the loaded dataset into training (database) and testing (query) subsets, ensuring a specific distribution of individuals and images.

```Python
idx_train = list(range(10)) + list(range(190,200))
idx_test = list(range(10,20)) + list(range(200,210))
dataset_database = dataset.get_subset(idx_train)
dataset_query = dataset.get_subset(idx_test)
```

--------------------------------

### Load MacaqueFaces Dataset

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/docs/training.md

Loads the MacaqueFaces dataset with specified transformations and labels. Ensure the dataset is downloaded using MacaqueFaces.get_data(root) before instantiation.

```python
from wildlife_datasets.datasets import MacaqueFaces 
import torchvision.transforms as T

root = "data/MacaqueFaces"
transform = T.Compose([
    T.Resize([384, 384]),
    T.ToTensor(),
    T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))
])

MacaqueFaces.get_data(root)
dataset = MacaqueFaces(
    root,
    transform=transform,
    load_label=True,
    factorize_label=True,
)
```

--------------------------------

### Inference Progress for OpenCows2020

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-T-224.ipynb

This snippet displays the inference progress bars for the OpenCows2020 dataset.

```text
100%|█████████████████████████████████████████████████████████████████| 8/8 [00:16<00:00,  2.00s/it]
100%|███████████████████████████████████████████████████████████████| 30/30 [00:59<00:00,  1.99s/it]
```

--------------------------------

### Fit WildFusion Calibration Models

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-wildfusion.ipynb

Fits the calibration models for all configured similarity pipelines using the provided dataset. This step is crucial for normalizing similarity scores before they are used.

```python
wildfusion.fit_calibration(dataset, dataset)
```

--------------------------------

### Collect All SuperPoint Features and Match

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-pairwise-sim.ipynb

Use CollectAll to gather SuperPoint features and MatchLightGlue to perform pairwise matching. This is suitable for finding all possible correspondences between a query and a database set of features.

```python
collector = CollectAll()
matcher = MatchLightGlue(features="superpoint", collector=collector)
output = matcher(features_query, features_database)
print(len(output))  # = len(query) x len(database)
output
```

--------------------------------

### Calculate Similarity with MatchLightGlue and SuperPoint Features

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/docs/inference.md

Uses the MatchLightGlue model with SuperPoint features to calculate similarity scores based on the count of significant matches. Requires feature extraction and specifies confidence thresholds.

```python
from wildlife_tools.features import SuperPointExtractor
from wildlife_tools.similarity import MatchLightGlue, CollectCounts

transform = T.Compose([T.Resize([224, 224]), T.ToTensor()])
dataset_query.transform, dataset_database.transform = transform, transform
extractor = SuperPointExtractor()
matcher = MatchLightGlue(features='superpoint', collector=CollectCounts(thresholds=[0.25, 0.5, 0.75]))
output = matcher(extractor(dataset_query), extractor(dataset_database))
```

--------------------------------

### LeopardID2022 Dataset Performance

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-L-384.ipynb

Displays the progress and final accuracy for the LeopardID2022 dataset.

```text
100%|█████████████████████████████████████████████████████████████████| 8/8 [00:39<00:00,  4.95s/it]
100%|███████████████████████████████████████████████████████████████| 29/29 [02:13<00:00,  4.59s/it]

Output:
LeopardID2022 0.7557571528262387
```

--------------------------------

### WildFusion with B=2 parameter

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/examples/example-wildfusion.ipynb

This snippet shows the execution of WildFusion with the 'B' parameter set to 2. Different values of 'B' may yield different comparison results or interpretations.

```python
wildfusion(dataset, dataset, B=2)
```

--------------------------------

### Create Split Metadata

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/data/prepare_datasets.ipynb

Splits each dataset's metadata into training and testing sets using a 'ClosedSetSplit' strategy, discarding unknown identities. The split metadata is then saved as a CSV file for each dataset.

```python
# Create dataframe with training / test set splits
from wildlife_datasets import splits

for name in prepare_functions:
    metadata = pd.read_csv(f"images/size-518/{name}/annotations.csv", index_col=0)
    splitter = splits.ClosedSetSplit(0.8, identity_skip="unknown", seed=666)
    idx_train, idx_test = splitter.split(metadata)[0]

    metadata.loc[metadata.index[idx_train], "split"] = "train"
    metadata.loc[metadata.index[idx_test], "split"] = "test"

    os.makedirs(f"metadata/datasets/{name}/", exist_ok=True)
    metadata.to_csv(f"metadata/datasets/{name}/metadata.csv")
```

--------------------------------

### Inference Progress for LeopardID2022

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-T-224.ipynb

This snippet displays the inference progress bars for the LeopardID2022 dataset.

```text
100%|███████████████████████████████████████████████████████████████| 12/12 [00:22<00:00,  1.91s/it]
100%|███████████████████████████████████████████████████████████████| 42/42 [01:24<00:00,  2.01s/it]
```

--------------------------------

### Inference Progress for ZindiTurtleRecall

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-T-224.ipynb

This snippet displays the inference progress bars for the ZindiTurtleRecall dataset.

```text
100%|███████████████████████████████████████████████████████████████| 25/25 [00:30<00:00,  1.21s/it]
100%|███████████████████████████████████████████████████████████████| 76/76 [01:34<00:00,  1.25s/it]
```

--------------------------------

### StripeSpotter Dataset Performance

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-L-384.ipynb

Displays the progress and final accuracy for the StripeSpotter dataset.

```text
100%|███████████████████████████████████████████████████████████████| 73/73 [04:30<00:00,  3.71s/it]
100%|█████████████████████████████████████████████████████████████| 290/290 [17:31<00:00,  3.62s/it]

Output:
StripeSpotter 0.9817073170731707
```

--------------------------------

### Create Aggregated Training Metadata

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/data/prepare_datasets.ipynb

Combines the training sets from all individual datasets into a single aggregated CSV file. It modifies identity and path fields to include the dataset name, preventing collisions and enabling unified loading.

```python
import pandas as pd

results = []
for name in prepare_functions:
    metadata = pd.read_csv(f"metadata/datasets/{name}/metadata.csv", index_col=0)

    df = metadata.query("split == 'train'").copy()
    df["dataset"] = name
    df["identity"] = name + "_" + df["identity"].astype(str)
    df["path"] = name + "/" + df["path"]
    results.append(df)

combined_all = pd.concat(results)

os.makedirs("metadata/combined/", exist_ok=True)
combined_all.to_csv("metadata/combined/combined_all.csv")
```

--------------------------------

### HappyWhale Dataset Performance

Source: https://github.com/wildlifedatasets/wildlife-tools/blob/main/baselines/inference/MegaDescriptor-L-384.ipynb

Displays the progress and final accuracy for the HappyWhale dataset.

```text
100%|███████████████████████████████████████████████████████████████| 30/30 [02:37<00:00,  5.26s/it]
100%|███████████████████████████████████████████████████████████████| 94/94 [08:18<00:00,  5.31s/it]

Output:
HappyWhale 0.3429670639016609
```
Tokens:
Raw