### Install Dependencies

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Installs necessary libraries including numpy, datasets, pybioclip, scikit-learn, and matplotlib. Use this command to set up your environment.

```python
!pip install -q numpy datasets pybioclip scikit-learn matplotlib
```

--------------------------------

### Install pybioclip

Source: https://github.com/imageomics/pybioclip/blob/main/examples/TOL-Subsetting.ipynb

Installs the pybioclip library quietly. Ensure you have pip installed.

```python
!pip install pybioclip --quiet
```

--------------------------------

### Install pybioclip via pip

Source: https://github.com/imageomics/pybioclip/blob/main/docs/index.md

Standard installation command for the pybioclip package.

```console
pip install pybioclip
```

```console
pip install --upgrade pip
```

--------------------------------

### Install Dependencies

Source: https://github.com/imageomics/pybioclip/blob/main/examples/iNaturalistPredict.ipynb

Install the required pybioclip and pyinaturalist packages.

```python
!pip install pybioclip pyinaturalist --quiet
```

--------------------------------

### Install PyBioCLIP and Dependencies

Source: https://github.com/imageomics/pybioclip/blob/main/examples/PredictImages.ipynb

Installs the pybioclip library and ipywidgets for interactive use. Use --quiet to suppress output.

```python
!pip install pybioclip ipywidgets --quiet
```

--------------------------------

### Download example images

Source: https://github.com/imageomics/pybioclip/blob/main/docs/apptainer.md

Downloads sample images from the bioclip-demo repository for testing.

```console
wget https://huggingface.co/spaces/imageomics/bioclip-demo/resolve/main/examples/Ursus-arctos.jpeg
wget https://huggingface.co/spaces/imageomics/bioclip-demo/resolve/main/examples/Felis-catus.jpeg
```

--------------------------------

### Download example image

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Downloads a sample image ('Ursus-arctos.jpeg') from a specified URL using wget. This image will be used for demonstration purposes.

```bash
!wget -q https://huggingface.co/spaces/imageomics/bioclip-demo/resolve/main/examples/Ursus-arctos.jpeg
```

--------------------------------

### Install pybioclip and grad-cam

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Installs the pybioclip library and grad-cam package using pip. This is a prerequisite for using BioCLIP with GradCAM.

```python
!pip install -q git+https://github.com/imageomics/pybioclip.git grad-cam
```

--------------------------------

### Setup GradCAM object

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Initializes the BioCLIP classifier and sets up the GradCAM object. The target layer is specified as the last layer's normalization in the ViT transformer, and the reshape_transform function is provided for ViT compatibility.

```python
model = TreeOfLifeClassifier()
model.eval()
target_layers = [model.model.visual.transformer.resblocks[-1].ln_1]    
cam = GradCAM(model=model,
              target_layers=target_layers,
              reshape_transform=reshape_transform)
```

--------------------------------

### Setup RidgeClassifier Pipeline

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotRidgeClassifier.ipynb

Configures a scikit-learn pipeline that includes feature transformation and a RidgeClassifier.

```python
def init_clf():
    return sklearn.pipeline.make_pipeline(
            sklearn.preprocessing.StandardScaler(),
            sklearn.linear_model.RidgeClassifier(),
        )
```

```python
# Create a model that preprocesses images into feature embeddings that are passed to the RidgeClassifier
model = sklearn.pipeline.make_pipeline(
    FunctionTransformer(create_image_features),
    init_clf(),
)
```

--------------------------------

### Display Dataset Information

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Prints the available labels and an example image from the dataset. Useful for verifying dataset integrity and understanding its structure.

```python
print("Labels:", ",".join(dataset[TRAIN_NAME].features[LABEL_NAME].names), "\n")
print("Example Image:", dataset[TRAIN_NAME][IMAGE_NAME][0])
dataset[TRAIN_NAME][IMAGE_NAME][0]
```

--------------------------------

### Import Libraries

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotRidgeClassifier.ipynb

Imports essential Python libraries for data manipulation, machine learning, deep learning, and visualization. Ensure all these libraries are installed before executing this code.

```python
import itertools
import numpy as np
from tqdm.notebook import tqdm

import torch
from datasets import load_dataset

from bioclip.predict import BaseClassifier

import sklearn.model_selection
import sklearn.pipeline
import sklearn.preprocessing
import sklearn.experimental.enable_halving_search_cv
import sklearn.linear_model
import scipy.stats
from sklearn.preprocessing import FunctionTransformer

from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
from sklearn.metrics import accuracy_score
import matplotlib.pyplot as plt
```

--------------------------------

### View command line help

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-tutorial.md

Displays help information for the CLI or specific commands.

```console
bioclip -h
bioclip <command> -h
```

--------------------------------

### Define Image URLs and Configuration

Source: https://github.com/imageomics/pybioclip/blob/main/examples/PredictImages.ipynb

Sets up a list of image URLs to download and defines constants for the image directory and thumbnail size.

```python
IMAGE_URLS = [
    'https://huggingface.co/spaces/imageomics/bioclip-demo/resolve/main/examples/Ursus-arctos.jpeg',
    'https://huggingface.co/spaces/imageomics/bioclip-demo/resolve/main/examples/Actinostola-abyssorum.png',
    'https://huggingface.co/spaces/imageomics/bioclip-demo/resolve/main/examples/Felis-catus.jpeg',
]
IMAGE_DIR = "images"
THUMBNAIL_SIZE = (256,256)
```

--------------------------------

### List available models with bioclip list-models

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-help.md

Use this command to view available models and their associated pretrained checkpoints.

```bash
usage: bioclip list-models [-h] [--model MODEL]

Note that this will only list models known to open_clip; any model identifier
loadable by open_clip, such as from hf-hub, file, etc should also be usable for
--model in the embed and predict commands.
(The default model hf-hub:imageomics/bioclip-2 is one example.)

options:
  -h, --help     show this help message and exit
  --model MODEL  list available tags for pretrained model checkpoint(s) for
                 specified model
```

--------------------------------

### Open-Ended Species Prediction

Source: https://github.com/imageomics/pybioclip/blob/main/docs/geo-restricted-taxa.md

Perform open-ended species prediction to get a broad set of possible classifications. This is useful for anomaly detection, as it can identify species not typically found in a region.

```console
bioclip predict --k 20 nene.jpg > bioclip_species_predictions_oe.csv
```

--------------------------------

### Few-Shot SVM Classification with PyBioCLIP Embeddings

Source: https://github.com/imageomics/pybioclip/blob/main/docs/python-tutorial.md

Trains a Support Vector Machine (SVM) classifier on BioCLIP image embeddings using a small number of labeled examples per class. Requires scikit-learn.

```python
from bioclip import CLIP
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np

# Load BioCLIP model
model = CLIP()

# Assume you have image paths and corresponding labels
# image_paths = [...] 
# labels = [...] 

# Generate embeddings for images
# embeddings = []
# for img_path in image_paths:
#     image = Image.open(img_path).convert('RGB')
#     embeddings.append(model.embed_image(image).detach().numpy())
# embeddings = np.array(embeddings)

# For demonstration, using dummy data:
def generate_dummy_data(num_samples=100, num_classes=5):
    embeddings = np.random.rand(num_samples, 512) # Assuming 512-dim embeddings
    labels = np.random.randint(0, num_classes, num_samples)
    return embeddings, labels

embeddings, labels = generate_dummy_data()

# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(embeddings, labels, test_size=0.2, random_state=42)

# Train an SVM classifier
svm_classifier = SVC(kernel='linear', C=1.0, random_state=42) # Example parameters
svm_classifier.fit(X_train, y_train)

# Predict and evaluate
y_pred = svm_classifier.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print(f"SVM Accuracy: {accuracy:.4f}")
```

--------------------------------

### Configure Environment Variables

Source: https://github.com/imageomics/pybioclip/blob/main/examples/iNaturalistPredict.ipynb

Set global configuration variables for the iNaturalist user, device, and local storage paths.

```python
DEVICE = "cpu"
INAT_USER = "john4866"

MAX_IMAGES = 10
IMAGES_DIR = "images"
```

--------------------------------

### List Models and Taxa with CLI

Source: https://context7.com/imageomics/pybioclip/llms.txt

Query available models and export taxonomic labels for subset filtering.

```bash
bioclip list-models
```

```bash
bioclip list-models --model ViT-B-16
```

```bash
bioclip list-tol-taxa > tol_taxa.csv
```

```bash
bioclip list-tol-taxa --model hf-hub:imageomics/bioclip > tol_v1_taxa.csv
```

--------------------------------

### Configure Inference Device with Python API

Source: https://context7.com/imageomics/pybioclip/llms.txt

Set the compute device to CPU, NVIDIA GPU, or Apple Silicon for optimized inference performance.

```python
from bioclip import TreeOfLifeClassifier, CustomLabelsClassifier, Rank

# CPU (default)
classifier = TreeOfLifeClassifier(device='cpu')

# NVIDIA GPU
classifier = TreeOfLifeClassifier(device='cuda')

# Specific GPU
classifier = TreeOfLifeClassifier(device='cuda:1')

# Apple Silicon (MPS)
classifier = TreeOfLifeClassifier(device='mps')

# Check current device
print(classifier.device)

# Same for CustomLabelsClassifier
custom_classifier = CustomLabelsClassifier(
    ["cat", "dog"],
    device='cuda'
)
```

--------------------------------

### Define reshape_transform utility function

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

A utility function to reshape the output tensor from a Vision Transformer (ViT) model, adapting it for use with CAM methods. This function is based on the logic used in the pytorch-grad-cam ViT example.

```python
def reshape_transform(tensor, height=14, width=14):
    result = tensor[:, 1:, :].reshape(tensor.size(0),
                                        height, width, tensor.size(2))
    # Bring the channels to the first dimension,
    # like in CNNs.
    result = result.transpose(2, 3).transpose(1, 2)
    return result
```

--------------------------------

### Run Pybioclip Docker Container (Windows CPU)

Source: https://github.com/imageomics/pybioclip/blob/main/docs/docker.md

Execute this command for CPU-based predictions with Pybioclip on Windows. The cache directory will be created in the current directory if not explicitly mounted.

```bash
docker run --rm -v %cd%:/home/bcuser ghcr.io/imageomics/pybioclip:1.0.0 bioclip predict Ursus-arctos.jpeg
```

--------------------------------

### Run Pybioclip Docker Container (Mac/Linux CPU)

Source: https://github.com/imageomics/pybioclip/blob/main/docs/docker.md

Use this command to run Pybioclip with CPU support on Mac or Linux. It mounts the current directory and the cache directory into the container.

```bash
docker run --platform linux/amd64 \
           -v $(pwd):/home/bcuser \
           -v ~/.cache:/home/bcuser/.cache \
           --rm ghcr.io/imageomics/pybioclip:1.1.0 \
           bioclip predict Ursus-arctos.jpeg
```

--------------------------------

### Use Original BioCLIP Model

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-tutorial.md

Switch from the default BioCLIP 2 model to the original BioCLIP model using the --model argument.

```console
bioclip predict --model hf-hub:imageomics/bioclip Ursus-arctos.jpeg
```

```console
bioclip list-tol-taxa --model hf-hub:imageomics/bioclip
```

--------------------------------

### Initialize SVM with Randomized Search

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Creates a new SVM model with a randomized hyperparameter search for kernel, C, and gamma. It uses a pipeline that includes StandardScaler and SVC. The search is limited to 100 iterations and uses 16 parallel jobs.

```python
def init_svc():
    """Create a new, randomly initialized SVM with a random hyperparameter search over kernel, C and gamma. It uses only 16 jobs in parallel to prevent overloading the CPUs on a shared machine."""
    return sklearn.model_selection.RandomizedSearchCV(
        sklearn.pipeline.make_pipeline(
            sklearn.preprocessing.StandardScaler(),
            sklearn.svm.SVC(C=1.0, kernel="rbf"),
        ),
        {
            "svc__C": scipy.stats.loguniform(a=1e-3, b=1e1),
            "svc__kernel": ["rbf", "linear", "sigmoid", "poly"],
            "svc__gamma": scipy.stats.loguniform(a=1e-4, b=1e-3),
        },
        n_iter=100,
        n_jobs=16,
        random_state=42,
    )
```

--------------------------------

### Execution Results

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotRidgeClassifier.ipynb

Console output logs showing dataset resolution and file download progress.

```text
Resolving data files:   0%|          | 0/1000 [00:00<?, ?it/s]
```

```text
Resolving data files:   0%|          | 0/50 [00:00<?, ?it/s]
```

```text
Resolving data files:   0%|          | 0/403 [00:00<?, ?it/s]
```

```text
Downloading data:   0%|          | 0/1000 [00:00<?, ?files/s]
```

```text
025.jpg:   0%|          | 0.00/15.1k [00:00<?, ?B/s]
```

```text
023.jpg:   0%|          | 0.00/15.0k [00:00<?, ?B/s]
```

```text
027.jpg:   0%|          | 0.00/16.6k [00:00<?, ?B/s]
```

```text
021.jpg:   0%|          | 0.00/24.0k [00:00<?, ?B/s]
```

```text
012.jpg:   0%|          | 0.00/19.2k [00:00<?, ?B/s]
```

```text
020.jpg:   0%|          | 0.00/22.8k [00:00<?, ?B/s]
```

```text
017.jpg:   0%|          | 0.00/21.3k [00:00<?, ?B/s]
```

```text
029.jpg:   0%|          | 0.00/21.5k [00:00<?, ?B/s]
```

```text
028.jpg:   0%|          | 0.00/23.1k [00:00<?, ?B/s]
```

```text
024.jpg:   0%|          | 0.00/19.9k [00:00<?, ?B/s]
```

```text
026.jpg:   0%|          | 0.00/16.9k [00:00<?, ?B/s]
```

```text
018.jpg:   0%|          | 0.00/19.4k [00:00<?, ?B/s]
```

```text
011.jpg:   0%|          | 0.00/29.3k [00:00<?, ?B/s]
```

```text
015.jpg:   0%|          | 0.00/24.3k [00:00<?, ?B/s]
```

```text
022.jpg:   0%|          | 0.00/16.4k [00:00<?, ?B/s]
```

```text
019.jpg:   0%|          | 0.00/17.1k [00:00<?, ?B/s]
```

```text
030.jpg:   0%|          | 0.00/26.7k [00:00<?, ?B/s]
```

```text
031.jpg:   0%|          | 0.00/21.2k [00:00<?, ?B/s]
```

```text
033.jpg:   0%|          | 0.00/25.9k [00:00<?, ?B/s]
```

```text
035.jpg:   0%|          | 0.00/27.5k [00:00<?, ?B/s]
```

```text
041.jpg:   0%|          | 0.00/17.6k [00:00<?, ?B/s]
```

```text
039.jpg:   0%|          | 0.00/16.9k [00:00<?, ?B/s]
```

```text
038.jpg:   0%|          | 0.00/22.9k [00:00<?, ?B/s]
```

```text
037.jpg:   0%|          | 0.00/26.2k [00:00<?, ?B/s]
```

```text
044.jpg:   0%|          | 0.00/21.0k [00:00<?, ?B/s]
```

```text
034.jpg:   0%|          | 0.00/19.2k [00:00<?, ?B/s]
```

```text
042.jpg:   0%|          | 0.00/21.5k [00:00<?, ?B/s]
```

```text
036.jpg:   0%|          | 0.00/22.6k [00:00<?, ?B/s]
```

```text
045.jpg:   0%|          | 0.00/18.5k [00:00<?, ?B/s]
```

```text
032.jpg:   0%|          | 0.00/25.3k [00:00<?, ?B/s]
```

```text
047.jpg:   0%|          | 0.00/16.9k [00:00<?, ?B/s]
```

```text
046.jpg:   0%|          | 0.00/16.9k [00:00<?, ?B/s]
```

```text
040.jpg:   0%|          | 0.00/25.0k [00:00<?, ?B/s]
```

```text
043.jpg:   0%|          | 0.00/20.4k [00:00<?, ?B/s]
```

```text
049.jpg:   0%|          | 0.00/18.7k [00:00<?, ?B/s]
```

```text
050.jpg:   0%|          | 0.00/19.0k [00:00<?, ?B/s]
```

```text
048.jpg:   0%|          | 0.00/20.9k [00:00<?, ?B/s]
```

```text
056.jpg:   0%|          | 0.00/16.3k [00:00<?, ?B/s]
```

```text
055.jpg:   0%|          | 0.00/29.2k [00:00<?, ?B/s]
```

```text
051.jpg:   0%|          | 0.00/17.1k [00:00<?, ?B/s]
```

```text
053.jpg:   0%|          | 0.00/20.6k [00:00<?, ?B/s]
```

```text
052.jpg:   0%|          | 0.00/16.3k [00:00<?, ?B/s]
```

```text
054.jpg:   0%|          | 0.00/18.0k [00:00<?, ?B/s]
```

```text
057.jpg:   0%|          | 0.00/15.1k [00:00<?, ?B/s]
```

```text
058.jpg:   0%|          | 0.00/29.4k [00:00<?, ?B/s]
```

```text
062.jpg:   0%|          | 0.00/17.7k [00:00<?, ?B/s]
```

```text
059.jpg:   0%|          | 0.00/12.8k [00:00<?, ?B/s]
```

```text
061.jpg:   0%|          | 0.00/17.6k [00:00<?, ?B/s]
```

--------------------------------

### List pretrained models for a specific model

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-tutorial.md

Lists the available pretrained weights for a specified model architecture.

```console
bioclip list-models --model  ViT-B-16
```

--------------------------------

### Configure Custom Labels Classifiers

Source: https://context7.com/imageomics/pybioclip/llms.txt

Initialize classifiers with specific labels using either built-in BioCLIP models or custom OpenCLIP configurations.

```python
from bioclip import BIOCLIP_V1_MODEL_STR

classifier_v1 = CustomLabelsClassifier(
    cls_ary=["cat", "dog", "bird"],
    model_str=BIOCLIP_V1_MODEL_STR  # Use original BioCLIP
)
```

```python
custom_classifier = CustomLabelsClassifier(
    cls_ary=["indoor", "outdoor", "underwater"],
    model_str='ViT-B-16',
    pretrained_str='laion2b_s34b_b88k'
)
```

--------------------------------

### List available models

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-tutorial.md

Displays all available models for use with BioCLIP.

```console
bioclip list-models
```

--------------------------------

### Download pybioclip container

Source: https://github.com/imageomics/pybioclip/blob/main/docs/apptainer.md

Pulls the pybioclip container image from the GitHub Container Registry.

```console
apptainer pull oras://ghcr.io/imageomics/pybioclip-sif:1.0.0
```

--------------------------------

### Run Pybioclip Docker Container (Windows GPU)

Source: https://github.com/imageomics/pybioclip/blob/main/docs/docker.md

This command utilizes GPU acceleration for Pybioclip on Windows. It requires Docker to be configured for GPU usage.

```bash
docker run --rm --gpus all -v %cd%:/home/bcuser ghcr.io/imageomics/pybioclip bioclip:1.0.0 predict --device cuda Ursus-arctos.jpeg
```

--------------------------------

### Run Pybioclip Docker Container (Linux GPU)

Source: https://github.com/imageomics/pybioclip/blob/main/docs/docker.md

This command enables GPU acceleration for Pybioclip on Linux. Ensure your NVIDIA drivers and Docker are configured for GPU access.

```bash
docker run --gpus all \
           --platform linux/amd64 \
           -v $(pwd):/home/bcuser \
           -v ~/.cache:/home/bcuser/.cache \
           --rm ghcr.io/imageomics/pybioclip:1.1.0 \
           bioclip predict --device cuda Ursus-arctos.jpeg
```

--------------------------------

### Select image path

Source: https://github.com/imageomics/pybioclip/blob/main/examples/PredictImages.ipynb

Extracts the first image path from the image_paths collection.

```python
example_image_path = image_paths[0]
```

--------------------------------

### Display highlighted image

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Uses the show_cam_on_image utility to overlay the generated grayscale activation map onto the original RGB image, creating a highlighted image. This highlighted image is then converted to a PIL Image object and displayed.

```python
img_ary = show_cam_on_image(rgb_img, grayscale_cam)

image = ary_to_image(img_ary)

display(image)
```

--------------------------------

### Import Libraries

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSimpleShot.ipynb

Import required modules for data handling, model classification, and metrics.

```python
import itertools
import numpy as np
from tqdm.notebook import tqdm

import torch
from datasets import load_dataset

from bioclip.predict import BaseClassifier

import sklearn.neighbors

from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
from sklearn.metrics import accuracy_score
import matplotlib.pyplot as plt
```

--------------------------------

### Train SimpleShot model

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSimpleShot.ipynb

Initializes and fits the SimpleShot model using training dataset columns.

```python
simpleshot = SimpleShot(device)
# Convert dataset columns to lists for Colab compatibility (sklearn requires list-like objects)
simpleshot.fit(list(dataset[TRAIN_NAME][IMAGE_NAME]), list(dataset[TRAIN_NAME][LABEL_NAME]))
```

--------------------------------

### Predict top 3 genera for an image and display output as a table

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-tutorial.md

Use the --format table, --k, and --rank flags to customize the prediction output.

```console
bioclip predict --format table --k 3 --rank=genus Ursus-arctos.jpeg
```

--------------------------------

### Display image and predictions

Source: https://github.com/imageomics/pybioclip/blob/main/examples/PredictImages.ipynb

Loads an image, creates a thumbnail, and displays both the path and the image, followed by filtering the dataframe for the specific file.

```python
image = PIL.Image.open(example_image_path)
image.thumbnail(THUMBNAIL_SIZE)
display(example_image_path)
display(image)
df[df['file_name'] == example_image_path]
```

--------------------------------

### Download Images and Create DataFrame

Source: https://github.com/imageomics/pybioclip/blob/main/examples/iNaturalistPredict.ipynb

Iterate through observations to download images locally and compile metadata into a pandas DataFrame.

```python
ids = []
uris = []
paths = []
taxons = []

os.makedirs(IMAGES_DIR, exist_ok=True)

for observation in observations:
    path = os.path.join(IMAGES_DIR, f"{observation.id}.jpg")

    with observation.photos[0].open(size='medium') as infile:
        with open(path, 'wb') as outfile:
            outfile.write(infile.read())

    ids.append(observation.id)
    uris.append(observation.uri)
    paths.append(path)
    taxons.append(observation.taxon.name)

df = pd.DataFrame({
    'id': ids,
    'uri': uris,
    'path': paths,
    'taxon': taxons
})
df.head()
```

--------------------------------

### Dataset and Embedding Settings

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Configures settings for the Hugging Face dataset and image embedding batch size. Adjust BATCH_SIZE for memory and performance trade-offs.

```python
# Dataset settings
HF_DATASET_NAME = "Somnath01/Birds_Species"
TRAIN_NAME = 'train'
TEST_NAME = 'test'
IMAGE_NAME = 'image'
LABEL_NAME = 'label'

# Image embedding settings
BATCH_SIZE = 30
```

--------------------------------

### Import necessary libraries

Source: https://github.com/imageomics/pybioclip/blob/main/examples/TOL-Subsetting.ipynb

Imports the TreeOfLifeClassifier and Rank enum from pybioclip, and pandas for data manipulation.

```python
from bioclip import TreeOfLifeClassifier, Rank
import pandas as pd
```

--------------------------------

### Import Libraries

Source: https://github.com/imageomics/pybioclip/blob/main/examples/iNaturalistPredict.ipynb

Import necessary modules for data handling, image processing, and BioCLIP classification.

```python
from pyinaturalist import get_observations, Observation
import os
import io
import pandas as pd
from PIL import Image
from bioclip import TreeOfLifeClassifier, Rank
```

--------------------------------

### Import GradCAM functions

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Imports various CAM (Class Activation Mapping) methods from pytorch-grad-cam, including GradCAM, ScoreCAM, and others, along with utility functions for image preprocessing and model targets.

```python
from pytorch_grad_cam import GradCAM, \
    ScoreCAM, \
    GradCAMPlusPlus, \
    AblationCAM, \
    XGradCAM, \
    EigenCAM, \
    EigenGradCAM, \
    LayerCAM, \
    FullGrad
from pytorch_grad_cam.ablation_layer import AblationLayerVit
from pytorch_grad_cam.utils.image import preprocess_image
from pytorch_grad_cam.utils.model_targets import ClassifierOutputTarget
from pytorch_grad_cam.utils.image import show_cam_on_image
```

--------------------------------

### Usage of bioclip list-tol-taxa

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-help.md

Displays the command-line interface options for listing TreeOfLife taxon labels.

```bash
usage: bioclip list-tol-taxa [-h] [--model MODEL]

options:
  -h, --help     show this help message and exit
  --model MODEL  model identifier (see command list-models); default: hf-hub:imageomics/bioclip-2
```

--------------------------------

### Experiment with Grad-CAM on BioCLIP

Source: https://github.com/imageomics/pybioclip/blob/main/docs/python-tutorial.md

This notebook applies Grad-CAM AI explainability techniques to BioCLIP, allowing visualization of image regions important for predictions.

```python
from bioclip import CLIP
from pytorch_grad_cam import GradCAM, ScoreCAM, GradCAMPlusPlus, AblationCAM, XGradCAM, LayerCAM
from pytorch_grad_cam.utils.image import show_cam_on_image
from pytorch_grad_cam.utils.model_targets import ClassifierOutputTarget
import torchvision.models as models
from torchvision.transforms import Compose, ToTensor, Normalize, Resize
from PIL import Image
import numpy as np

# Load BioCLIP model
cam_model = CLIP()

# Load an image
img_path = "path/to/your/image.jpg"
input_image = Image.open(img_path).convert('RGB')

# Preprocess the image for the model
preprocess = Compose([
    Resize((224, 224)),
    ToTensor(),
    Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.245]),
])
input_tensor = preprocess(input_image).unsqueeze(0)

# Define the target layer for Grad-CAM
# This will depend on the specific model architecture within BioCLIP
# For demonstration, let's assume a common structure
target_layer = cam_model.visual.layer4 # Example target layer

# Initialize Grad-CAM
cam = GradCAM(model=cam_model.visual, target_layers=[target_layer], use_cuda=torch.cuda.is_available())

# Define the target for the prediction (e.g., a specific class)
# This would typically be derived from the model's prediction for the image
targets = [ClassifierOutputTarget(0)] # Example target class index

# Generate the CAM mask
grayscale_cam = cam(input_tensor=input_tensor, targets=targets)

# Display the CAM mask on the original image
visualization = show_cam_on_image(np.array(input_image), grayscale_cam[0], use_rgb=True)

# You can then display 'visualization' using matplotlib or save it
# import matplotlib.pyplot as plt
# plt.imshow(visualization)
# plt.axis('off')
# plt.show()

```

--------------------------------

### Create a prediction

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-tutorial.md

Runs a classification prediction on an image using a specified model and pretrained weights.

```console
bioclip predict --cls duck,fish,bear --model ViT-B-16 --pretrained laion2b_s34b_b88k Ursus-arctos.jpeg
```

--------------------------------

### Determine Device (GPU/CPU/MPS)

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Automatically detects and selects the optimal device for computation, prioritizing CUDA (GPU), then MPS (Apple Silicon), and falling back to CPU. Ensures efficient processing.

```python
if torch.cuda.is_available():
    device = torch.device("cuda")
elif torch.backends.mps.is_available():
    device = torch.device("mps")
else:
    device = torch.device("cpu")
print(f"Using device: {device}")
```

--------------------------------

### Predict Species with CLI

Source: https://context7.com/imageomics/pybioclip/llms.txt

Perform species predictions using various models, hardware acceleration, and subset filtering.

```bash
bioclip predict --subset hawaii_birds.csv nene.jpeg
```

```bash
bioclip predict --model hf-hub:imageomics/bioclip image.jpeg
```

```bash
bioclip predict --device cuda image.jpeg
```

```bash
bioclip predict --log predictions.json --k 10 image.jpeg
```

```bash
bioclip predict --batch-size 32 --output results.csv images/*.jpeg
```

--------------------------------

### Execute Predictions via Command Line

Source: https://context7.com/imageomics/pybioclip/llms.txt

Run taxonomic or custom label predictions directly from the terminal with various output formats.

```bash
bioclip predict bear.jpeg
```

```bash
bioclip predict --rank genus --format table --k 3 bear.jpeg
```

```bash
bioclip predict --output results.csv bear.jpeg cat.jpeg bird.jpeg
```

```bash
bioclip predict --cls cat,dog,bear,bird image.jpeg
```

```bash
bioclip predict --cls species_list.txt image.jpeg
```

```bash
bioclip predict --bins bins.csv image.jpeg
```

--------------------------------

### Initialize CustomLabelsClassifier

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Initializes the `CustomLabelsClassifier` from `bioclip.predict` for zero-shot predictions. Requires a list of label names and the device to be specified.

```python
from bioclip.predict import CustomLabelsClassifier
classifier = CustomLabelsClassifier(dataset[TEST_NAME].features[LABEL_NAME].names, device=device)
predicted_labels = []
```

--------------------------------

### Import Libraries

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Imports essential Python libraries for data manipulation, machine learning, and deep learning. Includes PyBiCLIP, Hugging Face datasets, PyTorch, and scikit-learn.

```python
import itertools
import numpy as np
from tqdm.notebook import tqdm

import torch
from datasets import load_dataset

from bioclip.predict import BaseClassifier

import sklearn.model_selection
import sklearn.pipeline
import sklearn.preprocessing
import sklearn.svm
import scipy.stats
from sklearn.preprocessing import FunctionTransformer

from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
from sklearn.metrics import accuracy_score
import matplotlib.pyplot as plt
```

--------------------------------

### Import core libraries

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Imports essential libraries for image manipulation and numerical operations, including OpenCV, NumPy, and Pillow.

```python
import cv2
import numpy as np
from PIL import Image
```

--------------------------------

### Perform zero-shot classification with PyBioCLIP

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotRidgeClassifier.ipynb

Initializes a CustomLabelsClassifier and iterates through a batched dataset to generate predictions. Ensure the dataset features and batch size are correctly defined before execution.

```python
from bioclip.predict import CustomLabelsClassifier
classifier = CustomLabelsClassifier(dataset[TEST_NAME].features[LABEL_NAME].names, device=device)
predicted_labels = []
# Convert dataset columns to lists for Colab compatibility
for images in batched(list(dataset[TEST_NAME][IMAGE_NAME]), BATCH_SIZE):
    predictions = classifier.predict(images, k=1)
    for pred in predictions:
        label_str = pred['classification']
        label = dataset[TEST_NAME].features[LABEL_NAME].str2int(label_str)
        predicted_labels.append(label)
# Convert dataset columns to lists for Colab compatibility
show_metrics(predicted_labels, list(dataset[TEST_NAME][LABEL_NAME]))
```

--------------------------------

### Predict from Binning CSV

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-tutorial.md

Use a CSV file to map classes to specific bins for aggregated prediction results.

```text
cls,bin
cat,one
bird,one
bear,two
```

```console
bioclip predict --bins bins.csv Ursus-arctos.jpeg
```

--------------------------------

### Predict Species for Images with PyBioCLIP

Source: https://github.com/imageomics/pybioclip/blob/main/docs/python-tutorial.md

Use this notebook to download images and predict their species using PyBioCLIP.

```python
import torch
from PIL import Image
from bioclip import CLIP

model = CLIP()

# Load an image
image = Image.open("path/to/your/image.jpg")

# Predict species
results = model.predict_image(image)
print(results)
```

--------------------------------

### bioclip list-models

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-help.md

Lists available models and pretrained model checkpoints.

```APIDOC
## CLI COMMAND: bioclip list-models

### Description
List available models and pretrained model checkpoints.

### Parameters
#### Options
- **--model** (string) - Optional - List available tags for pretrained model checkpoint(s) for specified model
```

--------------------------------

### Set image path

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Defines the path to the image file that will be used for analysis.

```python
image_path = "Ursus-arctos.jpeg"
```

--------------------------------

### Initialize BaseClassifier

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Initializes the BaseClassifier for image embedding generation. Ensure the device is correctly set.

```python
classifier = BaseClassifier(device=device)
```

--------------------------------

### Initialize Classifier and Batching Utility

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotRidgeClassifier.ipynb

Initializes the BaseClassifier and defines a generator function for batching items.

```python
classifier = BaseClassifier(device=device)
```

```python
def batched(items, batch_size):
    it = iter(items)
    while (batch := list(itertools.islice(it, batch_size))):
          yield batch
```

--------------------------------

### Predict Species for iNaturalist Images with PyBioCLIP

Source: https://github.com/imageomics/pybioclip/blob/main/docs/python-tutorial.md

This notebook demonstrates how to download images from iNaturalist.org and predict their species using PyBioCLIP.

```python
from bioclip import CLIP

model = CLIP()

# Predict species for iNaturalist images
# This will download images from iNaturalist and predict species
results = model.predict_inaturalist_images(query="dog")
print(results)
```

--------------------------------

### Create and Apply Custom Taxa Filter

Source: https://github.com/imageomics/pybioclip/blob/main/examples/TOL-Subsetting.ipynb

Demonstrates creating a custom filter to exclude specific species and then applying it to predictions. This is useful for focusing predictions on desired taxonomic groups.

```python
label_data = classifier.get_label_data()
taxa_filter = ~label_data.species.isin(["Ursus arctos", "Ursus arctos syriacus"])
```

```python
classifier.apply_filter(taxa_filter)
preds = classifier.predict("Ursus-arctos.jpeg", Rank.SPECIES)
pd.DataFrame(preds)
```

```text
Result:
           file_name   kingdom    phylum     class         order    family  \
0  Ursus-arctos.jpeg  Animalia  Chordata  Mammalia     Carnivora   Ursidae   
1  Ursus-arctos.jpeg  Animalia  Chordata  Mammalia     Carnivora   Ursidae   
2  Ursus-arctos.jpeg  Animalia  Chordata  Mammalia     Carnivora   Ursidae   
3  Ursus-arctos.jpeg  Animalia  Chordata  Mammalia  Artiodactyla  Cervidae   
4  Ursus-arctos.jpeg  Animalia  Chordata  Mammalia     Carnivora   Ursidae   

    genus       species_epithet                      species  \
0   Ursus      arctos bruinosus       Ursus arctos bruinosus   
1   Ursus                arctus                 Ursus arctus   
2   Ursus            americanus             Ursus americanus   
3  Cervus  canadensis sibericus  Cervus canadensis sibericus   
4   Ursus        arctos nelsoni         Ursus arctos nelsoni   

            common_name     score  
0                        0.501536  
1                        0.303386  
2  Louisiana black bear  0.060895  
3                        0.038811  
4                        0.022875  
```

--------------------------------

### Subset Predictions by TOL Labels

Source: https://github.com/imageomics/pybioclip/blob/main/docs/command-line-tutorial.md

Create a CSV file to restrict predictions to specific taxonomic orders and run the prediction command.

```text
order
Artiodactyla
Rodentia
```

```console
bioclip predict --subset orders.csv Ursus-arctos.jpeg
```

--------------------------------

### Apply GradCAM and generate highlighted image

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Applies the initialized GradCAM object to the input tensor to generate a grayscale activation map. Options for eigen_smooth and aug_smooth are used to refine the CAM output. The resulting grayscale map is then overlaid on the original image.

```python
grayscale_cam = cam(input_tensor=input_tensor,
                    targets=None,
                    eigen_smooth=True,
                    aug_smooth=False)
# Here grayscale_cam has only one image in the batch
grayscale_cam = grayscale_cam[0, :]
```

--------------------------------

### Download and Print Image Paths

Source: https://github.com/imageomics/pybioclip/blob/main/examples/PredictImages.ipynb

Downloads images from the predefined URLs using the download_image function and prints the list of saved image paths.

```python
image_paths = [download_image(url) for url in IMAGE_URLS]
print("Image paths:")
print(image_paths)
```

--------------------------------

### Create SVM Pipeline with Feature Extraction

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Constructs an SVM pipeline that first transforms images into feature embeddings using `create_image_features` and then passes these features to an SVM model initialized by `init_svc`.

```python
# Create a model that preprocesses images into feature embeddings that are passed to the SVM
svc = sklearn.pipeline.make_pipeline(
    FunctionTransformer(create_image_features),
    init_svc(),
)
```

--------------------------------

### Run predictions on CPU

Source: https://github.com/imageomics/pybioclip/blob/main/docs/apptainer.md

Executes the pybioclip prediction command using the local container image on a CPU.

```console
./pybioclip_sif_1.0.0.sif bioclip predict Ursus-arctos.jpeg Felis-catus.jpeg
```

--------------------------------

### Initialize TreeOfLifeClassifier

Source: https://github.com/imageomics/pybioclip/blob/main/examples/TOL-Subsetting.ipynb

Creates an instance of the TreeOfLifeClassifier. No arguments are needed for basic initialization.

```python
classifier = TreeOfLifeClassifier()
```

--------------------------------

### Load Hugging Face Dataset

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Loads the specified dataset from Hugging Face Hub. This may take several minutes on the first run due to image downloads. Ensure HF_TOKEN is set in Colab secrets to avoid rate limiting.

```python
dataset = load_dataset(HF_DATASET_NAME)
dataset
```

--------------------------------

### Convert image to GradCAM tensor

Source: https://github.com/imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb

Reads the specified image, converts it to an RGB array, and then preprocesses it into a tensor suitable for GradCAM analysis, using specific mean and standard deviation values.

```python
rgb_img = read_image_ary(image_path)
input_tensor = make_input_tensor(rgb_img,
                                 mean=[0.5, 0.5, 0.5],
                                 std=[0.5, 0.5, 0.5])
```

--------------------------------

### Generate Image Features (Embeddings)

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Converts a list of PIL images into numerical feature embeddings using a batched approach. Progress is shown using tqdm. Requires `classifier` and `BATCH_SIZE` to be defined.

```python
def create_image_features(pil_image_ary):
    # batch PIL -> features transformation
    all_features = []
    with tqdm(total=len(pil_image_ary), desc='Creating image embeddings', unit='image') as progress_bar:
        for images in batched(pil_image_ary, BATCH_SIZE):
            features = classifier.create_image_features(images, normalize=True)
            all_features.append(features.cpu())
            progress_bar.update(len(images))
    return torch.cat(all_features, dim=0).cpu().numpy()
```

--------------------------------

### Track Prediction Metadata with Recorders

Source: https://context7.com/imageomics/pybioclip/llms.txt

Attach a recorder to a classifier to log prediction metadata for reproducibility.

```python
from bioclip import TreeOfLifeClassifier, Rank
from bioclip.recorder import attach_prediction_recorder, save_recorded_predictions

classifier = TreeOfLifeClassifier()

# Attach recorder with custom settings
attach_prediction_recorder(
    classifier,
    tree_of_life_version="imageomics/TreeOfLife-200M",
    subset=None
)

# Make predictions (automatically recorded)
predictions = classifier.predict("bear.jpeg", Rank.SPECIES)
predictions = classifier.predict(["cat.jpeg", "dog.jpeg"], Rank.GENUS)

# Save to JSON for provenance chain
save_recorded_predictions(classifier, "predictions_log.json")

# Or save as human-readable text (appends to file)
save_recorded_predictions(classifier, "predictions_log.txt")
```

--------------------------------

### Predict species using TreeOfLifeClassifier

Source: https://github.com/imageomics/pybioclip/blob/main/examples/PredictImages.ipynb

Initializes the classifier on the CPU and predicts species for provided image paths. Set the HF_TOKEN environment variable in Colab to prevent rate limiting.

```python
classifier = TreeOfLifeClassifier(device='cpu')
prediction_ary = classifier.predict(image_paths, rank=Rank.SPECIES)
df = pd.DataFrame(prediction_ary)
df
```

--------------------------------

### Generate Embeddings with CLI

Source: https://context7.com/imageomics/pybioclip/llms.txt

Create semantic image embeddings for downstream tasks like clustering or similarity search.

```bash
bioclip embed bear.jpeg
```

```bash
bioclip embed --output embeddings.json image1.jpeg image2.jpeg
```

```bash
bioclip embed --device cuda images/*.jpeg
```

```bash
bioclip embed --model hf-hub:imageomics/bioclip image.jpeg
```

--------------------------------

### Perform Batch Predictions with Progress Callback

Source: https://context7.com/imageomics/pybioclip/llms.txt

Use a callback function to monitor progress during batch image processing.

```python
def progress_callback(processed, total):
    print(f"Processed {processed}/{total} images")

predictions = classifier.predict(
    ["img1.jpeg", "img2.jpeg", "img3.jpeg"],
    k=3,
    batch_size=2,
    callback=progress_callback
)
```

--------------------------------

### Download Image Function

Source: https://github.com/imageomics/pybioclip/blob/main/examples/PredictImages.ipynb

Defines a function to download an image from a given URL, save it to a specified directory, and return its path. Ensures the image directory exists.

```python
def download_image(url):
    filename = os.path.basename(url)
    os.makedirs(IMAGE_DIR, exist_ok=True)
    path = os.path.join(IMAGE_DIR, filename)
    response = requests.get(url, stream=True)
    response.raise_for_status()
    with open(path, 'wb') as out_file:
        shutil.copyfileobj(response.raw, out_file)
    return path
```

--------------------------------

### Batch Process Images and Evaluate Predictions

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotSVM.ipynb

Iterates through a dataset in batches to generate predictions and converts them into integer labels for metric calculation.

```python
# Convert dataset columns to lists for Colab compatibility
for images in batched(list(dataset[TEST_NAME][IMAGE_NAME]), BATCH_SIZE):
    predictions = classifier.predict(images, k=1)
    for pred in predictions:
        label_str = pred['classification']
        label = dataset[TEST_NAME].features[LABEL_NAME].str2int(label_str)
        predicted_labels.append(label)
# Convert dataset columns to lists for Colab compatibility
show_metrics(predicted_labels, list(dataset[TEST_NAME][LABEL_NAME]))
```

--------------------------------

### Inspect Dataset Structure

Source: https://github.com/imageomics/pybioclip/blob/main/examples/FewShotRidgeClassifier.ipynb

Displays the dataset dictionary structure including train, validation, and test splits with their respective features and row counts.

```python
DatasetDict({
    train: Dataset({
        features: ['image', 'label'],
        num_rows: 1000
    })
    validation: Dataset({
        features: ['image', 'label'],
        num_rows: 50
    })
    test: Dataset({
        features: ['image', 'label'],
        num_rows: 403
    })
})
```