### Install THINGSvision

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Installs the THINGSvision library and ensures it's up-to-date. Run this command in your environment.

```python
!pip install --upgrade thingsvision
```

--------------------------------

### Install THINGSvision and CLIP

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Installs the THINGSvision library and the CLIP model from its GitHub repository. Run these commands in your environment.

```python
!pip install --upgrade thingsvision
!pip install git+https://github.com/openai/CLIP.git
```

--------------------------------

### Install ThingsVision and Dependencies

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Install the 'thingsvision' package and its dependencies, including CLIP and Harmonization, using pip.

```bash
$ pip install --upgrade thingsvision
$ pip install git+https://github.com/openai/CLIP.git
$ pip install git+https://github.com/serre-lab/Harmonization.git
```

--------------------------------

### Install ThingsVision and CLIP

Source: https://github.com/vicco-group/thingsvision/blob/master/README.md

Install the ThingsVision library and the CLIP model using pip. Ensure your conda environment is activated.

```bash
$ pip install --upgrade thingsvision
$ pip install git+https://github.com/openai/CLIP.git
```

--------------------------------

### Install ipywidgets

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Installs the ipywidgets package, which may be required for interactive features in environments like Jupyter notebooks.

```python
!pip install ipywidgets
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/vicco-group/thingsvision/blob/master/README.md

Create a new conda environment with a specific Python version and activate it. This is the first step before installing the library.

```bash
$ conda create -n thingsvision python=3.10
$ conda activate thingsvision
```

--------------------------------

### Get Torchvision AlexNet Extractor

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Instantiates an extractor for the AlexNet model from torchvision. Ensures the model is loaded with default pretrained weights if available. Requires PyTorch and ThingsVision to be installed.

```python
import torch
from thingsvision import get_extractor

model_name = 'alexnet'
source = 'torchvision'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {'weights': 'DEFAULT'}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)
```

--------------------------------

### Initialize CLIP Extractor with GPU

Source: https://github.com/vicco-group/thingsvision/blob/master/README.md

Instantiate a ThingsVision extractor for the CLIP model, loading it to the GPU if available for faster inference. Ensure PyTorch is installed and CUDA is configured.

```python
import torch
from thingsvision import get_extractor
from thingsvision.utils.storing import save_features
from thingsvision.utils.data import ImageDataset, DataLoader

model_name = 'clip'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT-L/14'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Create a new conda environment named 'thingsvision' with Python 3.9 and activate it. This is the first step for local setup.

```bash
$ conda create -n thingsvision python=3.9
$ conda activate thingsvision
```

--------------------------------

### Install DreamSim Dependencies

Source: https://github.com/vicco-group/thingsvision/blob/master/README.md

Install the DreamSim library for extracting features from DreamSim models. Ensure you are in the 'thingsvision' conda environment.

```bash
$ pip install dreamsim==0.1.3
```

--------------------------------

### Get DreamSim OpenCLIP Variant Extractor

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Extract features from the OpenCLIP variant of DreamSim. Ensure the 'dreamsim' package is installed. Set `pretrained=True` for pretraining.

```bash
$ pip install dreamsim==0.1.2
```

```python
import torch
from thingsvision import get_extractor

model_name = 'DreamSim'
module_name = 'model.mlp'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'open_clip_vitb32'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)
```

--------------------------------

### Install Harmonization Dependencies

Source: https://github.com/vicco-group/thingsvision/blob/master/README.md

Install additional packages required for extracting features from harmonized models. This includes keras-cv-attention-models and vit-keras.

```bash
$ pip install "keras-cv-attention-models>=1.3.5" "vit-keras==0.1.2"
$ pip install git+https://github.com/serre-lab/Harmonization.git
```

--------------------------------

### Get Torchvision RegNet Extractor with Specific Weights

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Instantiates an extractor for the RegNet Y 32GF model from torchvision, using specific SWAG and ImageNet finetuned weights. This demonstrates how to select particular weight initializations for a model. Requires PyTorch and Thingsvision.

```python
import torch
from thingsvision import get_extractor

model_name = 'regnet_y_32gf'
source = 'torchvision'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {'weights': 'IMAGENET1K_SWAG_LINEAR_V1'}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)
```

--------------------------------

### Get OpenCLIP Model Extractor

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Extract features using OpenCLIP models. You can specify the training dataset, such as 'laion400m_e32'. Set `pretrained=True` for pretraining.

```python
import torch
from thingsvision import get_extractor

model_name = 'OpenCLIP'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT-B/32',
    'dataset': 'laion400m_e32'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)
```

--------------------------------

### Extract Features from ALIGN Model

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Use this snippet to extract features from the Kakaobrain ALIGN model. This example shows how to initialize the extractor for this specific model.

```python
import torch
from thingsvision import get_extractor

model_name = 'Kakaobrain_Align'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
)
```

--------------------------------

### Get Module Names from Layers

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Retrieves a list of names for all modules (layers) within a given model.

```python
def get_module_names(modules: List[Any]) -> List[str]:
    """Yield module names associated with layers."""
    return list(map(lambda m: m.name, modules))
```

--------------------------------

### Extract Features from Harmonization Models

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Use this snippet to extract features from models available in the Harmonization repository. Ensure you have installed the necessary packages.

```python
import torch
from thingsvision import get_extractor

model_name = 'Harmonization'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT_B16'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)
```

--------------------------------

### Get Extractor from PyTorch Model

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/CustomModels.md

Use `get_extractor_from_model` to create an extractor for a PyTorch model. Pass the model, device, backend, and optional preprocessing transforms. This is an alternative to creating a custom class.

```python
from thingsvision import get_extractor_from_model
from torchvision.models import alexnet, AlexNet_Weights
import torch

# initialize a model of your choice, here we use AlexNet from torchvision 
# and load the ImageNet weights
model_weights = AlexNet_Weights.DEFAULT
model = alexnet(model_weights)

# you can also pass a custom preprocessing function that is applied to every 
# image before extraction
transforms = model_weights.transforms()

# provide the backend of the model (either 'pt' or 'tf')
backend = 'pt'

# set the device to 'cuda' if you have a GPU available
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor_from_model(
  model=model, 
  device=device,
  transforms=transforms,
  backend=backend
)
```

--------------------------------

### Get Official CLIP Model Extractor

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Extract features using official CLIP models. The default variant is 'ViT-B/32'. Set `pretrained=True` to use ImageNet pretraining.

```python
import torch
from thingsvision import get_extractor

model_name = 'clip'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT-B/32'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)
```

--------------------------------

### Python: Initialize Feature Extractor

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Import necessary components and initialize a ThingsVision extractor using a CLIP model. The model is moved to GPU if available.

```python
import torch
from thingsvision import get_extractor
from thingsvision.utils.storing import save_features
from thingsvision.utils.data import ImageDataset, DataLoader

model_name = 'clip'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT-B/32'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)
```

--------------------------------

### CLI: Show Model Information

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Use the 'thingsvision show-model' command to display information about available models. Specify the model name and its source.

```bash
thingsvision show-model --model-name "alexnet" --source "torchvision"
```

--------------------------------

### Configuration for Feature Extraction

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Sets up configuration parameters for feature extraction, including model pretraining, batch size, image cropping, and activation flattening.

```python
pretrained = True # use pretrained model weights
model_path = None # if pretrained = False (i.e., randomly initialized weights) set path to model weights
batch_size = 32 # use a power of two (this can be any size, depending on the number of images for which you aim to extract features)
apply_center_crop = True # center crop images (set to False, if you don't want to center-crop images)
flatten_activations = True # whether or not features (e.g., of Conv layers) should be flattened
device = 'cuda' if torch.cuda.is_available() else 'cpu'
class_names = None  # optional list of class names for class dataset
file_names = None # optional list of file names according to which features should be sorted
```

--------------------------------

### Construct Full Paths

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Combines the mounted Google Drive directory with the user-defined image and output directories to create absolute paths.

```python
full_image_path = os.path.join(mounted_dir, 'MyDrive', image_dir)
full_output_path = os.path.join(mounted_dir, 'MyDrive', output_dir)
```

--------------------------------

### Configuration Parameters for Feature Extraction

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Sets up configuration parameters for feature extraction. These include options for using pretrained models, specifying model paths, batch sizes, image cropping, feature flattening, class names, file names, file formats, and the computation device.

```python
pretrained = True # use pretrained model weights
model_path = None # if pretrained = False (i.e., randomly initialized weights) set path to model weights
batch_size = 32 # use a power of two (this can be any size, depending on the number of images for which you aim to extract features)
apply_center_crop = True # center crop images (set to False, if you don't want to center-crop images)
flatten_activations = True # whether or not features (e.g., of Conv layers) should be flattened
class_names = None  # optional list of class names for class dataset
file_names = None # optional list of file names according to which features should be sorted
file_format = "npy" # format with which to save features to disk (can be set to "mat", "txt", "npy", "hdf5")
device = 'cuda' if torch.cuda.is_available() else 'cpu'
```

--------------------------------

### Get Keras Model Extractor

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Use this snippet to get a feature extractor for models available in Keras applications. Ensure `pretrained=True` for ImageNet pretraining.

```python
import torch
from thingsvision import get_extractor

model_name = 'VGG16'
source = 'keras'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True
)
```

--------------------------------

### Python: Create Dataset and DataLoader

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Create an ImageDataset and DataLoader for your images. The dataset is configured with the extractor's backend and transformations, and the DataLoader uses the same backend.

```python
root='path/to/your/image/directory' # (e.g., './images/)
batch_size = 32

dataset = ImageDataset(
    root=root,
    out_path='path/to/features',
    backend=extractor.get_backend(), # backend framework of model
    transforms=extractor.get_transformations(resize_dim=256, crop_dim=224) # set the input dimensionality to whichever values are required for your pretrained model
)

batches = DataLoader(
    dataset=dataset,
    batch_size=batch_size,
    backend=extractor.get_backend() # backend framework of model
)
```

--------------------------------

### Load VGG-16 from Custom Source (Ecoset)

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Loads the VGG-16 model pretrained on Ecoset from a custom source. Ensure the 'source' parameter is set to 'custom' and 'model_path' points to your model weights.

```python
## load model
model_name = 'VGG16_ecoset'
# specifiy model source
# the model's source here is custom
source = 'custom'
extractor = get_extractor(
    model_name=model_name,
    pretrained=pretrained,
    model_path=model_path,
    device=device,
    source=source,
)
```

--------------------------------

### Import PyTorch and THINGSvision Libraries

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Imports essential libraries for PyTorch-based feature extraction, including THINGSvision's extractor, data utilities, and core extraction functions. Also imports Google Colab drive utilities and typing hints.

```python
import os
import torch
import torch.nn as nn
import numpy as np
from thingsvision import get_extractor
from thingsvision.utils.storing import save_features
from thingsvision.utils.data import ImageDataset, DataLoader
from thingsvision.core.extraction import center_features

from google.colab import drive
from typing import Any, Dict, List, Optional, Union
```

--------------------------------

### Define Image and Feature Directories

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Specifies the paths for input images and output feature storage within Google Drive. Ensure these paths are correctly set for your GDrive structure.

```python
image_dir = 'path/to/images'  # path/to/images in GDrive
output_dir = 'path/to/features' # path/to/output  in GDrive
```

--------------------------------

### Initialize Centered Kernel Alignment (CKA)

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Initializes the CKA object with the number of samples (m) and the desired kernel type (e.g., 'linear'). CKA measures the similarity between representations.

```python
from thingsvision.core.cka import CKA
```

```python
features_i = features_fc_layers[f'fc_01']
features_j = features_fc_layers[f'fc_02']

assert features_i.shape[0] == features_j.shape[0]
m = features_i.shape[0]
cka = CKA(m=m, kernel='linear')
```

--------------------------------

### Unzipping Object Images

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/index.md

This bash script demonstrates how to unzip multiple object image zip files using a provided password. Ensure all zip files are in the current directory.

```bash
for fn in object_images_*.zip; do unzip -P the_password $fn; done
```

--------------------------------

### Implement Custom Model Class

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/CustomModels.md

Define a class that inherits from `Custom` and implements the `create_model` method to load your PyTorch model and weights. Ensure the `backend` attribute is set correctly.

```python
from thingsvision.custom_models.custom import Custom
import torchvision.models as torchvision_models
import torch

class VGG16_ecoset(Custom):
    def __init__(self, device, **kwargs) -> None:
        super().__init__(device)
        self.backend = 'pt'
        self.preprocess = None

    def create_model(self):
        model = torchvision_models.vgg16(pretrained=False, num_classes=565)
        path_to_weights = 'https://osf.io/fe7s5/download'
        state_dict = torch.hub.load_state_dict_from_url(path_to_weights, map_location=self.device)
        model.load_state_dict(state_dict)
        return model, self.preprocess
```

--------------------------------

### Select Module for CLIP

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Selects the 'visual' module for image feature extraction from the loaded CLIP model. Uncomment 'extractor.show_model()' to inspect available module names.

```python
## select layer

# NOTE: uncomment the line below, if you are uncertain about layer naming of CLIP
# extractor.show_model()
module_name = 'visual' # image features
```

--------------------------------

### Load Images from Folder with ImageDataset

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/LoadingYourData.md

Use ImageDataset to load images from a directory. Labels are inferred from subfolder names. Requires an extractor backend and transformations.

```python
from thingsvision.utils.data import ImageDataset

# load extractor beforehand
extractor = ...

dataset = ImageDataset(
    root='path/to/root/img/directory' # (e.g., './root/')
    out_path='path/to/features',
    backend=extractor.get_backend(),
    transforms=extractor.get_transformations()
)
```

--------------------------------

### Load CLIP Model

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Loads a CLIP model with a specified variant. Ensure 'pretrained' and 'model_path' are correctly set for your environment.

```python
# load model
model_name = 'clip'
variant = 'ViT-B/32'
# specifiy model source
# the model's source here is custom
source = 'custom'
extractor = get_extractor( 
    model_name=model_name,
    pretrained=pretrained,
    model_path=model_path,
    device=device,
    source=source,
    model_parameters={'variant': variant}
)
```

--------------------------------

### Load VGG-16 with Batch Norm (TorchVision)

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Loads the VGG-16 model with batch normalization, pretrained on ImageNet, from the torchvision source. Ensure the 'source' parameter is correctly set.

```python
# load model
model_name = 'vgg16_bn' 
# specify model source 
# we use torchvision here (https://pytorch.org/vision/stable/models.html)
source = 'torchvision'
# initialize the extractor
extractor = get_extractor(
    model_name=model_name,
    pretrained=pretrained,
    model_path=model_path,
    device=device,
    source=source
)
```

--------------------------------

### Load Timm Model

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Use this snippet to load any model available in the 'timm' library. Ensure the model name is spelled correctly as per the timm documentation. Set `pretrained=True` to load pre-trained weights.

```python
import torch
from thingsvision import get_extractor

model_name = 'tf_efficientnet_b0'
source = 'timm'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True
)
```

--------------------------------

### Load Images from HDF5 with HDF5Dataset

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/LoadingYourData.md

Use HDF5Dataset to load images stored in HDF5 files. Specify the HDF5 file path, the key for the image dataset, and optionally image indices. Requires an extractor backend and transformations.

```python
from thingsvision.utils.data import HDF5Dataset

# load extractor beforehand
extractor = ...

dataset = HDF5Dataset(
    hdf5_fp='path/to/hdf5/file' # (e.g., './nsd_stimuli.hdf5')
    img_ds_key='imgBrick',
    out_path='path/to/features',
    backend=extractor.get_backend(),
    transforms=extractor.get_transformations()
)
```

--------------------------------

### Load SSL DINO Model with CLS Token Extraction

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

Load a DINO model (ViT variant) from the 'ssl' source and extract features specifically for the CLS token. This is useful for tasks where the CLS token represents the global image representation.

```python
import torch
from thingsvision import get_extractor

model_name = 'dino-vit-base-p16'
source = 'ssl'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_paramters = {"token_extraction": "cls_token"} # extract DINO features exclusively for the [cls] token

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)
```

--------------------------------

### Mount Google Drive

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Mounts Google Drive to the specified directory, allowing access to files stored on your drive. Use `force_remount=True` to ensure a fresh connection.

```python
mounted_dir = '/thingsvision'
drive.mount(mounted_dir, force_remount=True)
```

--------------------------------

### Load SSL MAE Model with Average Pooling

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

This snippet loads a MAE model (ViT variant) from the 'ssl' source and extracts features using average pooling over tokens. This is an alternative method for obtaining global image features.

```python
import torch
from thingsvision import get_extractor

model_name = 'mae-vit-large-p16'
source = 'ssl'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_paramters = {"token_extraction": "avg_pool"} # average-pool tokens before extracting the MAE features

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)
```

--------------------------------

### Load SSL SimCLR Model

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

This snippet loads a SimCLR model from the 'ssl' source. The model name indicates the pre-training objective (e.g., 'simclr-rn50'). `pretrained=True` loads weights pre-trained on ImageNet-1K.

```python
import torch
from thingsvision import get_extractor

model_name = 'simclr-rn50'
source = 'ssl'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True
)
```

--------------------------------

### Display AlexNet Model Architecture

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Show the architecture of the AlexNet model to identify module names for feature extraction. The output details layers like 'features' and 'classifier'.

```python
extractor.show_model()

# Output:
AlexNet(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))
    (1): ReLU(inplace=True)
    (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
    (4): ReLU(inplace=True)
    (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (7): ReLU(inplace=True)
    (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (9): ReLU(inplace=True)
    (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(6, 6))
  (classifier): Sequential(
    (0): Dropout(p=0.5, inplace=False)
    (1): Linear(in_features=9216, out_features=4096, bias=True)
    (2): ReLU(inplace=True)
    (3): Dropout(p=0.5, inplace=False)
    (4): Linear(in_features=4096, out_features=4096, bias=True)
    (5): ReLU(inplace=True)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)
```

--------------------------------

### Load VGG-16 Model from Keras

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Loads a pre-trained VGG-16 model from the Keras source. Ensure the 'keras' source is correctly specified for downloading weights.

```python
## load model
model_name = 'VGG16' 
# specify model source 
# we use keras (backend is TensorFlow) here (https://keras.io/api/applications/)
source = 'keras' 

extractor = get_extractor(
            model_name=model_name,
            pretrained=pretrained,
            model_path=model_path,
            device=device,
            source=source
)
```

--------------------------------

### Initialize and Compare CKA

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Initializes the Centered Kernel Alignment (CKA) object and compares feature representations from two models. Ensure the feature dimensions match.

```python
from thingsvision.core.cka import CKA
```

```python
assert vgg_ecoset_features.shape[0] == clip_features.shape[0]
m = clip_features.shape[0]
cka = CKA(m=m, kernel='linear')
rho = cka.compare(X=vgg_ecoset_features, Y=clip_features)
```

--------------------------------

### Select Layer for Feature Extraction

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Specifies the layer from which to extract features. Use 'module_name.show()' to inspect available layer names if uncertain.

```python
## select layer

# NOTE: uncomment the line below, if you are uncertain about layer naming in TensorFlow
# module_name = model.show() 
module_name = 'fc1' # 'block1_conv1'
```

--------------------------------

### Select Layer for Feature Extraction (Ecoset VGG-16)

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Selects a specific module name (layer) from the loaded VGG-16 (Ecoset) model. Uncomment 'extractor.show_model()' to view available layer names.

```python
## select layer

# NOTE: uncomment the line below, if you are uncertain about layer naming in PyTorch and look at architecture
# extractor.show_model()
module_name = 'features.23' 
```

--------------------------------

### Extract features to disk

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/LowMemOptions.md

Use this when running out of RAM during feature extraction for a large number of images. Features are written to disk as they are extracted, freeing up RAM. The `step_size` parameter controls how many batches are processed before writing to disk.

```python
extractor = ...
batches = ...

output_dir = '/path/to/output/directory'
extractor.extract_features(
    batches=batches,
    module_name=...,
    flatten_acts=True,
    output_dir=output_dir
) # returns None if output_dir is set
```

--------------------------------

### Import Libraries

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Imports necessary libraries for THINGSvision, including TensorFlow, NumPy, and utility functions for data handling and feature storage.

```python
import os
import torch
import tensorflow
import re
import numpy as np
from thingsvision import get_extractor
from thingsvision.utils.storing import save_features
from thingsvision.utils.data import ImageDataset, DataLoader

from google.colab import drive
from typing import Any, Dict, List
```

--------------------------------

### Extract and Save Image Features

Source: https://github.com/vicco-group/thingsvision/blob/master/README.md

Extract features from the specified module (e.g., 'visual') of the model and save them to disk. Features can be saved in various formats like 'npy', 'txt', 'mat', 'pt', or 'hdf5'.

```python
module_name = 'visual'

features = extractor.extract_features(
    batches=batches,
    module_name=module_name,
    flatten_acts=True,
    output_type="ndarray", # or "tensor" (only applicable to PyTorch models of which CLIP and DINO are ones!)
)

save_features(features, out_path='path/to/features', file_format='npy') # file_format can be set to "npy", "txt", "mat", "pt", or "hdf5"
```

--------------------------------

### Custom Forward Function for PyTorch Extractor

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/CustomModels.md

Provide a `custom_forward_fn` to `get_extractor_from_model` when your PyTorch model's forward method has a different signature or requires custom handling. The function must accept `self` (the extractor) and `batch`.

```python
from thingsvision import get_extractor_from_model
import torch.nn as nn

class ComplexForwardModel(nn.Module):
    def __init__(self):
        super().__init__()
        # model definition 
        # ...

    def forward(self, x, y):
        # forward function with custom signature
        # ...
        return x

def custom_forward_fn(self, batch):
    # custom forward function that passes the batch to the model's forward function
    return self.model(batch, y=None)

model = ComplexForwardModel()
backend = 'pt'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor_from_model(
  model=model, 
  device=device,
  transforms=transforms,
  backend=backend,
  forward_fn=custom_forward_fn
)
```

--------------------------------

### Custom Batch Extraction with PyTorch

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Extract features from a custom PyTorch dataloader using a batch extraction context manager. This allows for preprocessing before extraction and postprocessing of features.

```python
module_name = 'visual'

# your custom dataset and dataloader classes come here (for example, a PyTorch data loader)
my_dataset = ...
my_dataloader = ...

with extractor.batch_extraction(module_name, output_type="tensor") as e: 
  for batch in my_dataloader:
    ... # whatever preprocessing you want to add to the batch
    feature_batch = e.extract_batch(
      batch=batch,
      flatten_acts=True, # flatten 2D feature maps from an early convolutional or attention layer
      )
    ... # whatever post-processing you want to add to the extracted features
```

--------------------------------

### Perform Centered Kernel Alignment (CKA)

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/RSA.md

CKA is used to compare image features between different model architectures or layers. It supports both 'torch' and 'numpy' backends, with 'torch' allowing GPU acceleration.

```python
from thingsvision.core.cka import get_cka

backend = "torch" # can be set to either 'torch' or 'numpy' 
m = # number of images (e.g., features_i.shape[0])
kernel = 'linear' # linear or rbf kernel (for rbf kernel define sigma, i.e., the width of the Gaussian)
unbiased = True # whether to compute an unbiased version of CKA
device = "cuda" # only necessary to be defined for the 'torch' backend (NumPy runs on CPU only)
sigma = None # needs to be defined for 'rbf' kernel

cka = get_cka(backend=backend, m=m, kernel=kernel, unbiased=unbiased, device=device, sigma=sigma)
rho = cka.compare(X=features_i, Y=features_j)
```

--------------------------------

### Plot Representational Dissimilarity Matrix (RSA)

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Plots the computed RDM to visualize representational dissimilarities. Allows customization of output path, format, colormap, and display.

```python
from thingsvision.core.rsa import plot_rdm
```

```python
# plot rdm
plot_rdm(
            full_output_path,
            features,
            method='correlation',
            format='.png', # '.jpg'
            colormap='cividis',
            show_plot=True,
)
```

--------------------------------

### Display ResNet50 Model Architecture

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Show the architecture of the ResNet50 model to identify module names for feature extraction. The output details layers like 'conv1' and 'layer4.2.conv3'.

```python
extractor.show_model()

# Output:
ResNet(
  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace=True)
  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  (layer1): Sequential(
    (0): Bottleneck(
      (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (downsample): Sequential(
        (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): Bottleneck(
      (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size [...]
```

--------------------------------

### Align Features using gLocal

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/Alignment.md

Post-aligns extracted features using the gLocal method. This is computationally inexpensive as it applies an affine transformation. It is currently supported for specific models like CLIP and DINOv2 variants.

```python
aligned_features = extractor.align(
    features=features,
    module_name=module_name,
    alignment_type="gLocal",
)
```

--------------------------------

### Multi-Module Feature Extraction with PyTorch

Source: https://github.com/vicco-group/thingsvision/blob/master/README.md

Extract features from multiple modules of a model simultaneously using a custom PyTorch pipeline. The extracted features are returned as a dictionary.

```python
module_names = ['visual', ...] # add more module_names here

# your custom dataset and dataloader classes come here (for example, a PyTorch data loader)
my_dataset = ...
my_dataloader = ...

with extractor.batch_extraction(module_names=module_names, output_type="tensor") as e: 
  for batch in my_dataloader:
    ... # whatever preprocessing you want to add to the batch
    feature_batch_dict = e.extract_batch(
      batch=batch,
      flatten_acts=True, # flatten 2D feature maps from an early convolutional or attention layer
      )
    ... # whatever post-processing you want to add to the extracted features
```

--------------------------------

### Compute Representational Dissimilarity Matrix (RDM)

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Computes the RDM for CLIP features using the correlation method. This is a prerequisite for RSA.

```python
from thingsvision.core.rsa import compute_rdm
```

```python
# compute representational dissimilarity matrix (RDM) for CLIP features
rdm = compute_rdm(clip_features, method='correlation')
```

--------------------------------

### Plot Representational Dissimilarity Matrix (RDM)

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Plots the computed RDM for visualization. Customize the output path, format, and colormap as required.

```python
from thingsvision.core.rsa import plot_rdm
```

```python
# plot rdm
plot_rdm(
        full_output_path,
        clip_features,
        method='correlation',
        format='.png', # '.jpg'
        colormap='cividis',
        show_plot=True,
)
```

--------------------------------

### CLI: Extract Features

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Use the 'thingsvision extract-features' command to extract features from images. Specify image root, model details, batch size, device, file format, and output path.

```bash
thingsvision extract-features --image-root "./data" --model-name "alexnet" --module-name "features.10" --batch-size 32 --device "cuda" --source "torchvision" --file-format "npy" --out-path "./features"
```

--------------------------------

### Extract and Save CLIP Features

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Extracts features from a single image using the specified CLIP module, optionally centers them, and saves them to disk. Adjust batch_size and other parameters as needed.

```python
# extract features
clip_features = extract_features(
    extractor=extractor,
    module_name=module_name,
    image_path=full_image_path,
    out_path=full_output_path,
    batch_size=batch_size,
    flatten_activations=flatten_activations,
    apply_center_crop=apply_center_crop,
    class_names=class_names,
    file_names=file_names,
)

# apply centering (not necessary, but may be desirable, depending on the analysis)
clip_features = center_features(clip_features)

# save features to disk
save_features(clip_features, out_path=f'{full_output_path}/features_{model_name}_{module_name}', file_format=file_format)
```

--------------------------------

### Select Layer for Feature Extraction

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/pytorch.ipynb

Selects a specific module name (layer) from the loaded PyTorch model for feature extraction. Uncomment 'extractor.show_model()' to inspect available layer names.

```python
## select layer

# NOTE: uncomment the line below, if you are uncertain about layer naming in PyTorch
# extractor.show_model() 
module_name = 'features.23' 
```

--------------------------------

### Python: Extract and Save Features

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/GettingStarted.md

Extract features from the image encoder of CLIP ('visual' module) and save them to disk. Features can be flattened and stored as ndarray or tensor.

```python
module_name = 'visual'

features = extractor.extract_features(
  batches=batches,
  module_name=module_name,
  flatten_acts=True, # flatten 2D feature maps from an early convolutional or attention layer
  output_type="ndarray", # or "tensor" (only applicable to PyTorch models of which CLIP is one!)
)

save_features(features, out_path='path/to/features', file_format='npy') # file_format can be set to "npy", "txt", "mat", "pt", or "hdf5"
```

--------------------------------

### Compute Representational Dissimilarity Matrix (RSA)

Source: https://github.com/vicco-group/thingsvision/blob/master/notebooks/tensorflow.ipynb

Computes the Representational Dissimilarity Matrix (RDM) from extracted features using a specified method, typically 'correlation'. This is a core step in RSA.

```python
from thingsvision.core.rsa import compute_rdm
```

```python
# compute representational dissimilarity matrix
rdm = compute_rdm(features, method='correlation')
```

--------------------------------

### Extract Features from Segment Anything Models

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/AvailableModels.md

This code snippet is for extracting features using Segment Anything models. You can specify different variants like 'vit_h', 'vit_l', or 'vit_b'.

```python
import torch
from thingsvision import get_extractor

model_name = 'SegmentAnything'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

model_parameters = {
    'variant': 'vit_h' # also vit_l and vit_b
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)
```

--------------------------------

### Compute RDM and Correlate with Human Representations

Source: https://github.com/vicco-group/thingsvision/blob/master/docs/RSA.md

Use these functions to compute a representational dissimilarity matrix (RDM) from features and then correlate it with human representations using a specified correlation method.

```python
from thingsvision.core.rsa import compute_rdm, correlate_rdms

rdm_dnn = compute_rdm(features, method='correlation')
corr_coeff = correlate_rdms(rdm_dnn, rdm_human, correlation='pearson')
```