Wildlife Datasets

https://github.com/wildlifedatasets/wildlife-datasets
Admin
Wildlife Datasets provides a pipeline for wildlife re-identification including a dataset zoo,...
Tokens:38,229
Snippets:477
Trust Score:7.4
License:MIT
Update:4 months ago
Show doc for...
Tokens:
Raw
### Install wildlife-datasets Package

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/README.md

Install the wildlife-datasets package using pip. This is the primary method for setting up the library.

```bash
pip install wildlife-datasets
```

--------------------------------

### Download Dataset from HuggingFace

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/adding.md

Example of how to define a dataset class that downloads from HuggingFace. It may be necessary to overwrite the 'get_image' method for specific dataset structures.

```python
from wildlife_datasets.datasets import DownloadHuggingFace, WildlifeDataset

class Chicks4FreeID(DownloadHuggingFace, WildlifeDataset):
    hf_url = 'dariakern/Chicks4FreeID'
```

--------------------------------

### Model Finetuning Setup

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/training.md

Sets up the necessary components for finetuning a model, including defining the loss objective (ArcFaceLoss) and optimizer (SGD). This is part of the WildlifeTools training pipeline.

```python
import itertools
from torch.optim import SGD
from wildlife_tools.train import ArcFaceLoss, BasicTrainer

embedding_size = len(query[0][0])
objective = ArcFaceLoss(
    num_classes=dataset.num_classes,
    embedding_size=embedding_size,
    margin=0.5,
    scale=64
    )

params = itertools.chain(backbone.parameters(), objective.parameters())
optimizer = SGD(params=params, lr=0.001, momentum=0.9)
```

--------------------------------

### Import necessary libraries

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/notebooks/introduction.ipynb

Imports essential libraries for data manipulation and dataset analysis. Ensure these libraries are installed before running.

```python
import sys

sys.path.append("..")

import pandas as pd

from wildlife_datasets.analysis import display_statistics
from wildlife_datasets.datasets import SeaTurtleID2022
from wildlife_datasets.splits import TimeCutoffSplit, TimeProportionSplit, visualize_split
```

--------------------------------

### Download and Get SealID Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/preprocessing.md

Download the SealID dataset using a one-time token URL. Paste the copied URL into the 'url' variable.

```python
url = '' # Paste the URL here
datasets.SealID.get_data('data/SealID', url=url)
```

--------------------------------

### Import necessary libraries

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/split_comparison.ipynb

Imports essential libraries for data manipulation, plotting, and dataset handling. Ensure these libraries are installed.

```python
import sys

sys.path.insert(0, "../..")

import copy
import os
import pickle

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.metrics import auc
from utils import compute_predictions, make_radar_plot2, mean

from wildlife_datasets.datasets import WildlifeReID10k
from wildlife_datasets.metrics import BAKS, BAUS
from wildlife_datasets.splits import BalancedSplit
```

--------------------------------

### Example LaTeX Table for Dataset Metadata

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/notebooks/analysis.ipynb

A sample LaTeX table showcasing structured metadata for the AnimalCLEF2025 dataset. This format is useful for academic papers and reports.

```latex
\begin{tabular}{llrrrllllllr}
\toprule
name & species & year & $n_{\rm img}$ & $n_{\rm ind}$ & span & source & pose & timestamp & wild & more species & size (MB) \\
\midrule
AnimalCLEF2025 \cite{adam2025overview} & multiple & 2025 & 15209 & 1233 & very long & photo & multiple & \cmark & \cmark & \cmark & 1930 \\
\bottomrule
\end{tabular}
```

--------------------------------

### Initialize dataset paths and variables

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2026/prepare.ipynb

Sets up the root directories for datasets and metadata, and defines a flag for copying files.

```python
root_datasets = "/data/wildlife_datasets/data"
root = os.path.join(root_datasets, "AnimalCLEF2026")
root_images = os.path.join(root, "images")
root_metadata = os.path.join(root, "metadata")
copy_files = True

dataset_2025 = AnimalCLEF2025(os.path.join(root_datasets, "AnimalCLEF2025"))
names_2025 = list(dataset_2025.metadata["dataset"].unique())
names_2026 = names_2025 + ["TexasHornedLizards"]
for name in names_2026:
    os.makedirs(f"{root_metadata}/{name}/", exist_ok=True)
```

--------------------------------

### Create Dataset Directories

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/TurtlesTurtles/prepare.ipynb

Sets up the necessary directories for storing dataset components. Ensure the parent directory exists before running.

```python
import sys

sys.path.append("..")
import os

from wildlife_datasets.datasets import SeaTurtleID2022

folder0 = "../data/TurtlesTurtles"
folder1 = f"{folder0}/heads"
folder2 = f"{folder0}/bodies"

os.makedirs(folder0, exist_ok=True)
os.makedirs(folder1, exist_ok=True)
os.makedirs(folder2, exist_ok=True)
```

--------------------------------

### Load and Plot Dataset Sample

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/prepare.ipynb

Loads the prepared dataset and plots a sample grid to verify integrity.

```python
d = AnimalCLEF2025(root)
d.plot_grid();
```

--------------------------------

### Get Unknown Animal Name

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/dataframe.md

Retrieves the default name used for unknown animals in the dataset. This is useful for standardizing entries with unidentified individuals.

```python
from wildlife_datasets import datasets

datasets.WildlifeDataset.unknown_name
print(datasets.WildlifeDataset.unknown_name) # markdown-exec: hide
```

--------------------------------

### Loading and Preparing Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/methods.md

Loads MacaqueFaces and IPanda50 datasets, performs initial cleaning, and prepares them for use. It drops unnecessary columns and selects specific datasets.

```python
from wildlife_datasets import datasets, loader, metrics
import pandas as pd

df = pd.read_csv('docs/csv/MacaqueFaces.csv')
df = df.drop('Unnamed: 0', axis=1)
ds1 = datasets.MacaqueFaces('.', df)
ds1.df.drop(['category'], axis=1, inplace=True)

df = pd.read_csv('docs/csv/IPanda50.csv')
df = df.drop('Unnamed: 0', axis=1)
ds2 = datasets.IPanda50('.', df)

d = ds1
ds = [ds1, ds2]
```

--------------------------------

### Execute Model Finetuning

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/training.md

Initializes and runs the BasicTrainer from WildlifeTools to finetune the model on the dataset. This requires the dataset, model, objective, optimizer, and training parameters.

```python
trainer = BasicTrainer(
    dataset=dataset,
    model=backbone,
    objective=objective,
    optimizer=optimizer,
    epochs=1,
    batch_size=8,
    device='cuda',
)

trainer.train()
```

--------------------------------

### Get Image Size from MacaqueFaces Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/wildlife_dataset.md

Retrieves and prints the size (dimensions) of the first image in the MacaqueFaces dataset. This demonstrates direct image access.

```python
dataset = MacaqueFaces(root0, df)
print(dataset[0].size)
```

--------------------------------

### Download and Initialize MacaqueFaces Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/README.md

This snippet shows how to download the MacaqueFaces dataset to a specified directory and then initialize the dataset object for further use.

```python
from wildlife_datasets import analysis, datasets

datasets.MacaqueFaces.get_data('data/MacaqueFaces')
dataset = datasets.MacaqueFaces('data/MacaqueFaces')
```

--------------------------------

### Initialize Dataset and Feature Roots

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/analyze_wildlife_reid_10k.ipynb

Sets up the root directories for the dataset images and features, and initializes the WildlifeReID-10k dataset object. Verifies the correspondence between dataset and feature orderings.

```python
os.makedirs("results", exist_ok=True)
root = "/data/wildlife_datasets/data/WildlifeReID10k"
root_images = os.path.join(root, "images")
root_features = "features_dino"

d = datasets.WildlifeReID10k(root)
df = d.df
df["date"] = pd.to_datetime(df["date"])
```

--------------------------------

### Initialize WildlifeReID-10k dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/split_comparison.ipynb

Sets up the paths for data, images, features, and clusters, then initializes the WildlifeReID-10k dataset and converts the 'date' column to datetime objects.

```python
root_data = "/data/wildlife_datasets/data/WildlifeReID10k"
root_images = os.path.join(root_data, "images")
root_features = "features_dino"
root_clusters = "clusters"

d = WildlifeReID10k(root_data)
d.df["date"] = pd.to_datetime(d.df["date"])
```

--------------------------------

### Resize and Prepare BrownBearHeads Images

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/scripts/BrownBearHeads/README.md

This script reads cleaned metadata, resolves image paths, and copies/resizes images to create the final prepared dataset.

```bash
python scripts/BrownBearHears/brown_bear_heads_resize_images.py \
  /path/to/Public_release \
  /path/to/metadata_clean.csv \
  /path/to/output_root \
  --max-side 720 \
  --workers 8
```

--------------------------------

### Specify Dataset Roots and Parameters

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/prepare_wildlife_reid_10k.ipynb

Defines the root directories for the original datasets, the target location for the prepared dataset, and configuration parameters such as image transformation, file copying, and split addition. It also initializes directories for clusters and features.

```python
root_datasets = "/data/wildlife_datasets/data"
root = os.path.join(root_datasets, "WildlifeReID10k")
root_images = os.path.join(root, "images")
root_metadata = os.path.join(root, "metadata")
root_clusters = "clusters"
root_features = "features_dino"
os.makedirs(root_clusters, exist_ok=True)

transform = None
copy_files = False
add_split = True

names_permissible = list(names_thresholds.keys())
remove_str = ["[", "]"]
replace_extensions = {'.webp': '.jpg'}
```

--------------------------------

### Prepare LynxID2025 Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/prepare.ipynb

Prepares the LynxID2025 dataset by optionally copying image files and then creating a metadata CSV file from the dataset's metadata attribute.

```python
name = "LynxID2025"

if copy_files:
    shutil.copytree(f"{root_images_original}/{name}/database", f"{root_images}/{name}/database", dirs_exist_ok=True)
    shutil.copytree(f"{root_images_original}/{name}/query", f"{root_images}/{name}/query", dirs_exist_ok=True)

d = AnimalCLEF2025_LynxID2025(f"{root_images_original}/{name}")
os.makedirs(f"{root_metadata}/{name}/", exist_ok=True)
d.metadata.to_csv(f"{root_metadata}/{name}/metadata.csv", index=False)
```

--------------------------------

### Create and Save Dataset Metadata

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/prepare_wildlife_reid_10k.ipynb

Iterates through predefined preparation functions to create and save metadata for each permissible dataset. It configures splitting strategies based on features and thresholds, applies transformations, and saves the resulting metadata to CSV files. This step may require running with `copy_files=True` initially.

```python
for name, prepare in prepare_functions.items():
    if name in names_permissible:
        print(name)
        os.makedirs(f"{root_metadata}/{name}/", exist_ok=True)
        thr, splitter = names_thresholds[name]
        if thr != "time-aware" and thr is not None:
            path_features = f"{root_features}/features_{name}.npy"
            path_clusters = f"{root_clusters}/clusters_{name}_{thr}.npy"
            splitter = SplitterByFeatures(path_features, splitter, thr, file_name=path_clusters)
        metadata_part = prepare(
            f"{root_datasets}/{name}",
            f"{root_images}/{name}",
            transform=transform,
            add_split=add_split,
            splitter=splitter,
            copy_files=copy_files,
            remove_str=remove_str,
            replace_extensions=replace_extensions,
        )
        metadata_part.to_csv(f"{root_metadata}/{name}/metadata.csv", index=False)
```

--------------------------------

### List Small Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/recommended.md

Access and print a list of small datasets suitable for quick testing. These datasets are typically under 1GB.

```python
from wildlife_datasets import datasets

def print_list(xs):
    for x in xs:
        print(x)

def print_list_red(xs):
    for x in xs:
        print(str(x)[8:-2])

datasets.names_small

print_list_red(datasets.names_small) # markdown-exec: hide
```

--------------------------------

### Load SeaTurtleID2022 Dataset with Bounding Box Cropping

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/wildlife_dataset.md

Initializes the SeaTurtleID2022 dataset and configures it to load cropped images based on bounding box information. This is useful for datasets containing object detection annotations.

```python
dataset = SeaTurtleID2022('data/SeaTurtleID2022', img_load='bbox')
dataset[0]
```

--------------------------------

### Build Prepared BrownBearHeads Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/scripts/BrownBearHeads/README.md

This script builds the final prepared BrownBearHeads dataset, including image resizing and metadata preparation.

```bash
python scripts/BrownBearHears/brown_bear_build_prepared.py \
  /path/to/Public_release \
  /path/to/output_root \
  --max-side 720 \
  --workers 8
```

--------------------------------

### Prepare AnimalCLEF2025 datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2026/prepare.ipynb

Iterates through the names of the AnimalCLEF2025 datasets, creates metadata for each, and saves it to a CSV file.

```python
for name in names_2025:
    metadata = create_metadata(dataset_2025, name, copy_files, root)
    metadata.to_csv(f"{root_metadata}/{name}/metadata.csv", index=False)
```

--------------------------------

### Prepare SeaTurtleID2022 Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/prepare.ipynb

Prepares the SeaTurtleID2022 dataset by applying specific preparation functions for database and query sets, concatenating their metadata, and saving it to a CSV file. It also handles image path adjustments and species assignment.

```python
name = "SeaTurtleID2022"

prepare = prepare_functions["SeaTurtleID2022"]
metadata_database = prepare(
    root=f"{root_datasets}/{name}", new_root=f"{root_images}/{name}/database", copy_files=copy_files
)
prepare = prepare_functions["SeaTurtleID2022_AnimalCLEF2025"]
metadata_query = prepare(
    root=f"{root_images_original}/{name}", new_root=f"{root_images}/{name}/query", copy_files=copy_files
)

metadata_database["split"] = "database"
metadata_database["path"] = "database/" + metadata_database["path"]
metadata_query["species"] = "loggerhead turtle"
metadata_query["split"] = "query"
metadata_query["path"] = "query/" + metadata_query["path"]

metadata = pd.concat((metadata_database, metadata_query))
metadata = metadata.reset_index(drop=True)
metadata["image_id"] = range(len(metadata))

os.makedirs(f"{root_metadata}/{name}/", exist_ok=True)
metadata.to_csv(f"{root_metadata}/{name}/metadata.csv", index=False)
```

--------------------------------

### Dataset Download from Kaggle (Competitions)

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/adding.md

Inherit from DownloadKaggle and WildlifeDataset to enable downloads from Kaggle competitions. Specify the kaggle_url and kaggle_type.

```python
from wildlife_datasets.datasets import DownloadKaggle, WildlifeDataset

class AnimalCLEF2025(DownloadKaggle, WildlifeDataset):
    kaggle_url = 'animal-clef-2025'
    kaggle_type = 'competitions'
```

--------------------------------

### List All Available Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/tutorial_datasets.md

Prints a list of all dataset names available in the library. This can be used to iterate and load multiple datasets.

```python
print_array(['wildlife_datasets.datasets.datasets.' + dataset_name.__name__ for dataset_name in datasets.names_all])
```

--------------------------------

### Importing Necessary Libraries

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/methods.md

Imports the core modules from the wildlife_datasets package required for data handling and evaluation.

```python
from wildlife_datasets import datasets, loader, metrics
```

--------------------------------

### Prepare TexasHornedLizards dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2026/prepare.ipynb

Initializes the TexasHornedLizards dataset, processes its metadata, copies images, and saves the metadata to a CSV file.

```python
name = "TexasHornedLizards"

dataset = AnimalCLEF2026_TexasHornedLizards(os.path.join(root_datasets, name), remove_unknown=True)

metadata = dataset.df.copy()
metadata["split"] = "test"
add_public(metadata, 0)
metadata["basename"] = metadata["path"].apply(os.path.basename)
metadata["image_id"] = create_id(metadata["basename"])
for i, row in metadata.iterrows():
    name_new = os.path.join("test", row["image_id"] + os.path.splitext(row["path"])[1])
    path0 = os.path.join(dataset.root, row["path"])
    path1 = os.path.join(root, "images", name, name_new)
    if copy_files:
        os.makedirs(os.path.dirname(path1), exist_ok=True)
        shutil.copy(path0, path1)
    metadata.loc[i, "path"] = name_new

metadata = metadata.sort_values("image_id")
metadata = metadata.reset_index(drop=True)
metadata["image_id"] = range(len(metadata))
metadata["identity"] = name + "_" + metadata["identity"].astype(str)
metadata["species"] = "lizard"
metadata = metadata.drop("basename", axis=1)

metadata.to_csv(f"{root_metadata}/{name}/metadata.csv", index=False)
```

--------------------------------

### List Wild Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/recommended.md

Access and print a list of 'wild' datasets featuring animals in their natural habitats. These are generally more challenging.

```python
from wildlife_datasets import datasets

def print_list(xs):
    for x in xs:
        print(x)

def print_list_red(xs):
    for x in xs:
        print(str(x)[8:-2])

datasets.names_wild

print_list_red(datasets.names_wild) # markdown-exec: hide
```

--------------------------------

### Create Solution CSV

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2026/prepare.ipynb

Generates a solution CSV file containing image IDs, identities, datasets, and usage information. Filters out rows with missing usage data.

```python
df_solution = pd.DataFrame(
    {
        "image_id": dataset.df["image_id"],
        "identity": dataset.df["identity"],
        "dataset": dataset.df["dataset"],
        "Usage": dataset.df["Usage"],
    }
)
df_solution = df_solution[~df_solution["Usage"].isnull()]
df_solution.to_csv(f"{root}/solution.csv", index=False)
```

--------------------------------

### Load and Prepare AnimalCLEF2025 Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/prepare.ipynb

Loads and prepares datasets for LynxID2025, SalamanderID2025, and SeaTurtleID2022. It merges them into a single dataframe, standardizes identity and path formats, and saves the consolidated metadata.

```python
metadata = []
for name in ["LynxID2025", "SalamanderID2025", "SeaTurtleID2022"]:
    metadata_part = pd.read_csv(f"{root_metadata}/{name}/metadata.csv")
    metadata_part["dataset"] = name
    metadata_part["identity"] = name + "_" + metadata_part["identity"].astype(str)
    metadata_part["path"] = "images/" + name + "/" + metadata_part["path"]
    metadata.append(metadata_part)
metadata = pd.concat(metadata).reset_index(drop=True)
metadata["image_id"] = range(len(metadata))
metadata["orientation"] = metadata["orientation"].replace({"below": "down", "up": "top", "above": "top"})
metadata.to_csv(f"{root}/metadata.csv", index=False)
```

--------------------------------

### Summarize Dataset Statistics and Licenses

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/analyze_wildlife_reid_10k.ipynb

Generates a summary for each dataset, including image count, individual count, folder size, wild status, and license information. The results are saved to a LaTeX file.

```python
summary_datasets = {}
for name, df_red in df.groupby("dataset"):
    metadata = eval(f"datasets.{name}.summary")
    if "licenses" in metadata:
        license = metadata["licenses"]
    else:
        license = "Missing"
    summary_datasets[name] = {
        "images": len(df_red),
        "individuals": df_red["identity"].nunique(),
        "size (GB)": np.round(getFolderSize(os.path.join(root_images, name)) / 1024**3, 2),
        "wild": metadata["wild"],
        "license": license_conversion[license],
    }

summary_datasets = pd.DataFrame(summary_datasets).T.sort_index(key=lambda x: x.str.lower())
summary_datasets.loc["TOTAL"] = summary_datasets.sum()
summary_datasets.loc["TOTAL", ["license", "wild"]] = ""
for col in ["images", "individuals"]:
    summary_datasets[col] = summary_datasets[col].astype(int)
for col in ["size (GB)"]:
    summary_datasets[col] = summary_datasets[col].astype(float)
with open("results/summary_datasets.txt", "w") as text_file:
    text_file.write(
        rename_index(summary_datasets).replace({True: "\\cmark", False: "\\xmark"}).to_latex(float_format="%.2f")
    )
summary_datasets["ratio"] = np.round(summary_datasets["images"] / summary_datasets["individuals"], 1)
summary_datasets
```

--------------------------------

### Download MacaqueFaces Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/tutorial_datasets.md

Downloads the complete MacaqueFaces dataset to a specified directory. Ensure the directory exists before running.

```python
datasets.MacaqueFaces.get_data('data/MacaqueFaces')
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/prepare.ipynb

Imports essential libraries for dataset preparation, including system path manipulation, file operations, data handling with pandas and numpy, and image transformations from torchvision.

```python
import sys

sys.path.insert(0, "../..")

import os
import shutil

import numpy as np
import pandas as pd
import torchvision.transforms.functional as F

from wildlife_datasets.datasets import AnimalCLEF2025, AnimalCLEF2025_LynxID2025, AnimalCLEF2025_SalamanderID2025
from wildlife_datasets.preparation import prepare_functions, resize_dataset
from wildlife_datasets.splits import OpenSetSplit, analyze_split
```

--------------------------------

### Dataset Download from Kaggle (Datasets)

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/adding.md

Inherit from DownloadKaggle and WildlifeDataset to enable downloads from Kaggle datasets. Specify the kaggle_url and kaggle_type.

```python
from wildlife_datasets.datasets import DownloadKaggle, WildlifeDataset

class SeaTurtleID2022(DownloadKaggle, WildlifeDataset):
    kaggle_url = 'wildlifedatasets/seaturtleid2022'
    kaggle_type = 'datasets'
```

--------------------------------

### Create Pretrained Model with Timm

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/training.md

Loads a pretrained model from Hugging Face Hub using the timm library. This model can be used for feature extraction or finetuning.

```python
import timm

model_name = "hf-hub:BVRA/MegaDescriptor-L-384"
backbone = timm.create_model(model_name, num_classes=0, pretrained=True)
```

--------------------------------

### Download StripeSpotter Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/preprocessing.md

Download the StripeSpotter dataset. Extraction must be done manually afterwards. This function is only available on Linux.

```python
datasets.StripeSpotter.download('data/StripeSpotter')
```

--------------------------------

### Load MacaqueFaces Dataset with Labels

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/wildlife_dataset.md

Initializes the MacaqueFaces dataset and loads associated labels. Accessing an item will now return both the image and its label.

```python
dataset = MacaqueFaces(root, load_label=True)
dataset[0]
```

--------------------------------

### Perform and analyze an open-set split by sample ratio

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/tutorial_splits.md

Creates an open-set split where approximately 10% of samples are directly assigned to the testing set, and the training set aims for 80% of samples. Analyzes the resulting split.

```python
splitter = splits.OpenSetSplit(0.8, 0.1)
for idx_train, idx_test in splitter.split(df):
    splits.analyze_split(df, idx_train, idx_test)
    print(run('splits.analyze_split(df, idx_train, idx_test)'))
```

--------------------------------

### Initialize MacaqueFaces Dataset with DataFrame

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/wildlife_dataset.md

Initializes the MacaqueFaces dataset using a DataFrame loaded from a CSV file. Ensures the 'Unnamed: 0' column is dropped before dataset creation.

```python
from wildlife_datasets.datasets import MacaqueFaces
import pandas as pd

df = pd.read_csv('docs/csv/MacaqueFaces.csv')
df = df.drop('Unnamed: 0', axis=1)
root0 = 'docs/images'
dataset = MacaqueFaces(root0, df)
```

--------------------------------

### Access First Image in MacaqueFaces Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/wildlife_dataset.md

Demonstrates accessing the first image from the MacaqueFaces dataset by its index. This assumes the dataset has been initialized.

```python
from wildlife_datasets.datasets import MacaqueFaces, SeaTurtleID2022

root = 'data/MacaqueFaces'
dataset = MacaqueFaces(root)
dataset[0]
```

--------------------------------

### Define Dataset Root Directories

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/prepare.ipynb

Sets up the root directory paths for storing and accessing original and processed wildlife dataset images and metadata.

```python
root_datasets = "/data/wildlife_datasets/data"
root = os.path.join(root_datasets, "AnimalCLEF2025")
root_images = os.path.join(root, "images")
root_images_original = os.path.join(root, "images-unprocessed")
root_metadata = os.path.join(root, "metadata")
copy_files = False
```

--------------------------------

### Load MacaqueFaces Dataset with Factorized Labels

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/wildlife_dataset.md

Initializes the MacaqueFaces dataset, loads labels, and converts them into numerical values using factorize. This is useful for machine learning models that require numerical input.

```python
dataset_labels = MacaqueFaces(root0, df, load_label=True, factorize_label=True)
print(dataset_labels[0])
```

--------------------------------

### Load Submission Data

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/submissions.ipynb

Loads the solution file, sample submission, and finds all submission CSV files within a specified directory. It also identifies public and private split indices from the solution.

```python
root = "/data/wildlife_datasets/data/AnimalCLEF2025"
n_max = 50

solution = pd.read_csv(f"{root}/solution.csv")
sample_submission = pd.read_csv(f"{root}/sample_submission.csv")
idx_public = solution["Usage"] == "Public"

root_submission = f"{root}/submissions"
df_submission = find_images(root_submission, img_extensions=".csv")
```

--------------------------------

### Display Metadata Summary for Benchmark Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/notebooks/analysis.ipynb

Creates and displays a pandas DataFrame summarizing metadata for benchmark datasets (AnimalCLEF2025, AnimalCLEF2026, WildlifeReID10k). Replaces boolean values with checkmark and cross symbols for readability.

```python
names = [d.display_name() for d in ds_meta]
df_meta = pd.DataFrame([create_dict(d) for d in ds_meta], index=names)
df_print = df_meta.copy()
df_print.replace(True, "\u2714", inplace=True)
df_print.replace(False, "\u2716", inplace=True)
df_print
```

--------------------------------

### Load and prepare MacaqueFaces dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/default_splits.md

Loads the MacaqueFaces dataset from a CSV file and prepares it for use with the wildlife_datasets library. It removes an unnecessary column and creates a dataset object.

```python
from wildlife_datasets import datasets, splits
import pandas as pd

df = pd.read_csv('docs/csv/MacaqueFaces.csv')
df = df.drop('Unnamed: 0', axis=1)
dataset = datasets.MacaqueFaces('.', df)
df = dataset.df
```

--------------------------------

### Preparing the Testing Set

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/methods.md

Filters the dataset to create the testing set based on the previously generated indices. Photos where the animal was not recognized are ignored.

```python
df_test = d.df.loc[idx_test]
print(df_test)
```

--------------------------------

### Perform and analyze an open-set split by number of individuals

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/tutorial_splits.md

Creates an open-set split by specifying the number of individuals (5) to be placed directly into the testing set. Analyzes the resulting split.

```python
splitter = splits.OpenSetSplit(0.8, n_class_test=5)
for idx_train, idx_test in splitter.split(df):
    splits.analyze_split(df, idx_train, idx_test)
    print(run('splits.analyze_split(df, idx_train, idx_test)'))
```

--------------------------------

### Load and inspect SeaTurtleID2022 dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/notebooks/introduction.ipynb

Loads the SeaTurtleID2022 dataset and displays its dataframe, which contains image metadata. The dataframe is then processed to add 'date' and 'year' columns.

```python
d = SeaTurtleID2022(root, img_load="bbox")
df = d.df
df["date"] = pd.to_datetime(df["date"])
df["year"] = df["date"].dt.year
df
```

--------------------------------

### List Dog Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/recommended.md

Access and print a list of datasets containing images of various dog breeds, suitable for developing and testing dog identification models.

```python
from wildlife_datasets import datasets

def print_list(xs):
    for x in xs:
        print(x)

def print_list_red(xs):
    for x in xs:
        print(str(x)[8:-2])

datasets.names_dogs

print_list_red(datasets.names_dogs) # markdown-exec: hide
```

--------------------------------

### Load and Process Features for Performance Evaluation

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/split_comparison.ipynb

Loads pre-computed features, computes predictions, and calculates performance metrics (BAKS and BAUS) for different similarity thresholds. Results are saved to a pickle file if not already present.

```python
roots = ["features_mega", "features_miew"]
cols = ["split", "split_random"]
file_name = "results/results.pickle"

step = 0.01
ts = [-np.inf] + list(np.arange(0, 1 + step / 10, step)) + [np.inf]
new_individual = "new_individual"

baks = {root: {col: {t: {} for t in ts} for col in cols} for root in roots}
baus = {root: {col: {t: {} for t in ts} for col in cols} for root in roots}
if not os.path.exists(file_name):
    for root in roots:
        for col in cols:
            print(root, col)
            result = {}
            for name, df_dataset in d.df.groupby("dataset"):
                features = np.load(f"{root}/features_{name}.npy")

                idx_train = np.where(df_dataset[col] == "train")[0]
                idx_test = np.where(df_dataset[col] == "test")[0]

                idx_true, idx_pred, similarity = compute_predictions(
                    features[idx_test], features[idx_train], return_score=True
                )
                idx_true = idx_test[idx_true]
                idx_pred = idx_train[idx_pred]
                idx_pred = idx_pred[:, 0]
                similarity = similarity[:, 0]

                y_true = df_dataset["identity"].iloc[idx_true]
                y_pred_closed = df_dataset["identity"].iloc[idx_pred]

                identity_test_only = list(
                    set(df_dataset["identity"].iloc[idx_test]) - set(df_dataset["identity"].iloc[idx_train])
                )

                for t in ts:
                    y_pred = copy.copy(y_pred_closed)
                    with np.errstate(invalid="ignore"):
                        y_pred[similarity < t] = new_individual
                    baks[root][col][t][name] = BAKS(y_true, y_pred, identity_test_only)
                    baus[root][col][t][name] = BAUS(y_true, y_pred, identity_test_only, new_individual)
    with open(file_name, "wb") as handle:
        pickle.dump([baks, baus], handle, protocol=pickle.HIGHEST_PROTOCOL)
else:
    with open(file_name, "rb") as handle:
        baks, baus = pickle.load(handle)
```

--------------------------------

### Dataset Download from Multiple URLs

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/adding.md

For datasets requiring multiple file downloads, use DownloadURL and WildlifeDataset. Specify the downloads attribute as a list of (url, archive_name) tuples.

```python
from wildlife_datasets.datasets import DownloadURL, WildlifeDataset

class MacaqueFaces(DownloadURL, WildlifeDataset):
    downloads = [
        ('https://github.com/clwitham/MacaqueFaces/raw/master/ModelSet/MacaqueFaces.zip', 'MacaqueFaces.zip'),
        ('https://github.com/clwitham/MacaqueFaces/raw/master/ModelSet/MacaqueFaces_ImageInfo.csv', 'MacaqueFaces_ImageInfo.csv'),
    ]
```

--------------------------------

### Load MacaqueFaces Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/tutorial_datasets.md

Loads the MacaqueFaces dataset from a local directory. This initializes a dataset object for further operations.

```python
d = datasets.MacaqueFaces('data/MacaqueFaces')
```

--------------------------------

### List Cow Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/recommended.md

Access and print a list of datasets featuring cows, particularly Friesian cows, for transfer learning experiments across different cow datasets.

```python
from wildlife_datasets import datasets

def print_list(xs):
    for x in xs:
        print(x)

def print_list_red(xs):
    for x in xs:
        print(str(x)[8:-2])

datasets.names_cows

print_list_red(datasets.names_cows) # markdown-exec: hide
```

--------------------------------

### Print Split Information

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/analyze.ipynb

Prints the number of samples and unique identities for each split ('database' and 'query') of the loaded dataset. This provides a summary of the dataset's structure.

```python
for name, df_split in d.df.groupby("split"):
    print(name, len(df_split), df_split["identity"].nunique())
```

--------------------------------

### List Marine Mammal Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/recommended.md

Retrieves and prints a list of available datasets for whales, dolphins, and sharks. Use this to identify marine-related datasets.

```python
datasets.names_whales

print_list_red(datasets.names_whales) # markdown-exec: hide
```

--------------------------------

### Download BrownBearHeads Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/preprocessing.md

Use this snippet to download the BrownBearHeads dataset. The dataset includes a pre-prepared metadata.csv and reorganized image paths.

```python
datasets.BrownBearHeads.get_data('data/BrownBearHeads')
```

--------------------------------

### Initial DataFrame Preparation

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/tutorial_datasets.md

Loads a CSV file into a pandas DataFrame and removes an unnecessary column. This is a common preprocessing step before dataset initialization.

```python
from wildlife_datasets import analysis, datasets, loader
import pandas as pd

df = pd.read_csv('docs/csv/MacaqueFaces.csv')
df = df.drop('Unnamed: 0', axis=1)
d = datasets.MacaqueFaces('.', df)
d.df.drop(['category'], axis=1, inplace=True)
```

--------------------------------

### Dataset Download from a Single URL

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/adding.md

Use DownloadURL and WildlifeDataset for datasets stored as a single archive file. Define the url and archive attributes.

```python
from wildlife_datasets.datasets import DownloadURL, WildlifeDataset

class CTai(DownloadURL, WildlifeDataset):
    url = 'https://github.com/cvjena/chimpanzee_faces/archive/refs/heads/master.zip'
    archive = 'master.zip'
```

--------------------------------

### List Sea Turtle Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/recommended.md

Access and print a list of datasets specifically related to sea turtles, useful for transfer learning tasks involving different sea turtle species.

```python
from wildlife_datasets import datasets

def print_list(xs):
    for x in xs:
        print(x)

def print_list_red(xs):
    for x in xs:
        print(str(x)[8:-2])

datasets.names_turtles

print_list_red(datasets.names_turtles) # markdown-exec: hide
```

--------------------------------

### Import Necessary Packages

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/prepare_wildlife_reid_10k.ipynb

Imports Python packages required for dataset preparation, including system path manipulation, file system operations, numerical computation, data manipulation, and custom dataset utilities.

```python
import sys

sys.path.append("../..")

import os

import numpy as np
import pandas as pd
from thresholds import names_thresholds
from utils import SplitterByFeatures

from wildlife_datasets.datasets import WildlifeReID10k
from wildlife_datasets.preparation import prepare_functions, species_conversion
```

--------------------------------

### Accessing Dataset Element

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/training.md

Demonstrates how to access a single element from the prepared dataset, which returns a tuple of the transformed image and its factorized identity.

```python
dataset[0]
```

--------------------------------

### Loading Multiple Datasets for Mass Evaluation

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/methods.md

Loads multiple wildlife re-identification datasets (IPanda50, MacaqueFaces) for comprehensive method evaluation.

```python
ds = loader.load_datasets(
    [datasets.IPanda50, datasets.MacaqueFaces],
    'data',
    'dataframes'
)
```

--------------------------------

### Load WildlifeReID10k Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/analyze.ipynb

Loads the WildlifeReID10k dataset and applies a resizing transform. Ensure the dataset root path is correctly specified.

```python
import sys

sys.path.append("../..")

import os

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import torchvision.transforms as T

from wildlife_datasets.datasets import AnimalCLEF2025, WildlifeReID10k

os.makedirs("results", exist_ok=True)
n_rows = 4
transform1 = T.Compose([T.Resize(size=(512, 512))])
transform2 = T.Compose([T.Resize(size=(512, 256))])
root = "/data/wildlife_datasets/data/WildlifeReID10k"
d = WildlifeReID10k(root, transform=transform1)
```

--------------------------------

### Load MacaqueFaces dataset for splitting

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/tutorial_splits.md

Loads the MacaqueFaces dataset and its DataFrame, preparing it for subsequent data splitting operations.

```python
from wildlife_datasets import datasets, splits

dataset = datasets.MacaqueFaces('data/MacaqueFaces')
df = dataset.df
```

--------------------------------

### Generate Solution File and Hide Identities

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/prepare.ipynb

Generates a solution CSV file with image IDs, identities, and usage flags (Public/Private). It also modifies the main dataframe to hide query identities and saves it to a new metadata file.

```python
df_database = d.get_subset(d.df["split"] == "database").df
df_query = d.get_subset(d.df["split"] == "query").df
new_identities = np.setdiff1d(df_query["identity"], df_database["identity"])

np.random.seed(666)
p = np.random.rand(len(df_query))
usage = np.repeat(["Private"], len(df_query))
usage[p <= 0.3] = "Public"

df_solution = pd.DataFrame(
    {
        "image_id": df_query["image_id"],
        "identity": df_query["identity"],
        "new_identity": df_query["identity"].apply(lambda x: x in new_identities),
        "dataset": df_query["dataset"],
        "Usage": usage,
    }
)
df_solution.to_csv("solution.csv", index=False)

d.df.loc[d.df["split"] == "query", "identity"] = np.nan
d.df.to_csv(f"{root}/metadata_hidden.csv", index=False)
```

--------------------------------

### Load and verify dataset integrity

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2026/prepare.ipynb

Loads the consolidated AnimalCLEF2026 dataset and plots a sample grid for each dataset subset to verify integrity.

```python
dataset = AnimalCLEF2026(root)
for name in dataset.metadata["dataset"].unique():
    dataset_part = dataset.get_subset(dataset.metadata["dataset"] == name)
    dataset_part.plot_grid()
```

--------------------------------

### List Giraffe Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/recommended.md

Retrieves and prints a list of available giraffe datasets. Use this to identify datasets related to giraffes.

```python
datasets.names_giraffes

print_list_red(datasets.names_giraffes) # markdown-exec: hide
```

--------------------------------

### Loading a Single Dataset

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/methods.md

Loads a specified dataset (MacaqueFaces) into a pandas DataFrame. Ensure datasets are downloaded first.

```python
d = loader.load_dataset(datasets.MacaqueFaces, 'data', 'dataframes')
```

--------------------------------

### Specify dataset root directory

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/notebooks/introduction.ipynb

Sets the root directory where the dataset is stored. This path must be valid and accessible.

```python
root = "/data/wildlife_datasets/data/SeaTurtleID2022"
```

--------------------------------

### Display Metadata Summary for All Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/notebooks/analysis.ipynb

Creates and displays a pandas DataFrame summarizing metadata for all loaded wildlife re-identification datasets. Replaces boolean values with checkmark and cross symbols for readability.

```python
names = [d.display_name() for d in ds]
df = pd.DataFrame([create_dict(d) for d in ds], index=names)
df_print = df.copy()
df_print.replace(True, "\u2714", inplace=True)
df_print.replace(False, "\u2716", inplace=True)
df_print
```

--------------------------------

### Display Dataset DataFrame

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/tutorial_datasets.md

Prints the pandas DataFrame containing all samples of the loaded dataset. This is useful for inspecting the dataset's contents.

```python
print(d.df)
```

--------------------------------

### Define a Basic WildlifeDataset Subclass

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/docs/adding.md

Create a new dataset by subclassing WildlifeDataset and implementing the create_catalogue method to return a pandas DataFrame. The DataFrame must adhere to specific requirements.

```python
import pandas as pd
from wildlife_datasets.datasets import WildlifeDataset

class Test(WildlifeDataset):
    def create_catalogue(self) -> pd.DataFrame:
        df = pd.DataFrame({
            'image_id': [1, 2, 3, 4],
            'identity': ['Lukas', 'Vojta', 'Lukas', 'Vojta'],
            'path': ['img1.jpg', 'img2.jpg', 'img3.jpg', 'img4.jpg'],
        })
        return df
```

--------------------------------

### Load and Plot AnimalCLEF2025 Datasets

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/AnimalCLEF2025/analyze.ipynb

Loads the AnimalCLEF2025 dataset and iterates through its datasets to plot grids of sample images. It applies specific transforms and filters based on dataset information, saving each grid to a file.

```python
n_rows = 3
n_cols = 3
root = "/data/wildlife_datasets/data/AnimalCLEF2025"
d = AnimalCLEF25(root, transform=transform1)
info = {"LynxID2025": (10, transform1), "SalamanderID2025": (5, transform2), "SeaTurtleID2022": (10, transform1)}
for name, df_dataset in d.df.groupby("dataset"):
    i = 0
    idx = []
    k_start, d.transform = info[name]
    for k, (identity, df_identity) in enumerate(df_dataset.groupby("identity")):
        if k < k_start:
            continue
        if name in ["LynxID2025", "SeaTurtleID2022"]:
            df_identity = df_identity[df_identity["orientation"] == "left"]
        if len(df_identity) >= n_rows:
            idx = idx + [df_identity.index[:n_rows]]
            i += 1
            if i == n_rows:
                break
    d.plot_grid(idx=idx, n_rows=n_rows, n_cols=n_cols, background_color=(255, 255, 255))
    plt.savefig(f"results/grid_{name}.png", bbox_inches="tight", dpi=1200)
```

--------------------------------

### Summarizing WildlifeReID-10k Dataset Splits

Source: https://github.com/wildlifedatasets/wildlife-datasets/blob/main/baselines/WildlifeReID-10k/analyze_wildlife_reid_10k.ipynb

This code calculates and summarizes split ratios (train, validation, test) for each dataset in the WildlifeReID-10k collection. It raises an exception if any split is not closed-set. The summary includes counts, unique individuals, and various ratio metrics.

```python
summary = {}
for name, df_dataset in metadata.groupby("dataset"):
    idx_train = df_dataset[df_dataset["split"] == "train"].index
    idx_val = df_dataset[df_dataset["split"] == "val"].index
    idx_test = df_dataset[df_dataset["split"] == "test"].index

    split_data_test = splits.extract_data_split(df_dataset, list(idx_train) + list(idx_val), idx_test)
    split_data_val = splits.extract_data_split(df_dataset, list(idx_train), idx_val)

    if split_data["id_split"] != "open-set":
        raise Exception("split is not closed set")
    summary[name] = {
        "n": len(df_dataset),
        "n_individuals": df_dataset["identity"].nunique(),
        "ratio_train": split_data_val["n_train"] / len(df_dataset),
        "ratio_val": split_data_val["n_test"] / len(df_dataset),
        "ratio_test": split_data_test["n_test"] / len(df_dataset),
        "ratio_val_only": split_data_val["n_test_only"] / split_data_val["n_train"],
        "ratio_test_only": split_data_test["n_test_only"] / len(df_dataset),
        #'ratio_train_only': split_data2['n_train_only'] / split_data2['n'],
    }
    # wpefkpowef
summary = pd.DataFrame(summary).T
summary[["n", "n_individuals"]] = summary[["n", "n_individuals"]].astype(int)
summary
```
Tokens:
Raw