Thingsvision

https://vicco-group.github.io/thingsvision/
Admin
Thingsvision is a Python package that simplifies the extraction of image representations from...
Tokens:8,244
Snippets:79
Trust Score:9.7
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install ThingsVision and Dependencies

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Install the ThingsVision library and its dependencies, including CLIP and Harmonization, using pip.

```bash
$ pip install --upgrade thingsvision
$ pip install git+https://github.com/openai/CLIP.git
$ pip install git+https://github.com/serre-lab/Harmonization.git

```

--------------------------------

### Show Model Information via CLI

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Use the 'show-model' command to get information about available models. Specify the model name and source.

```bash
thingsvision show-model --model-name "alexnet" --source "torchvision"

```

--------------------------------

### Install DreamSim Package

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Install the DreamSim package to enable feature extraction. Ensure the version is compatible.

```bash
$ pip install dreamsim==0.1.2

```

--------------------------------

### Install Harmonization Packages

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Install the necessary packages for Harmonization models. This may require specific OS environments like Ubuntu.

```bash
$ pip install git+https://github.com/serre-lab/Harmonization.git
$ pip install keras-cv-attention-models>=1.3.5

```

--------------------------------

### Initialize ThingsVision Extractor in Python

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Import necessary components and instantiate a ThingsVision extractor with a specified model, source, device, and model parameters. This example uses a CLIP model.

```python
import torch
from thingsvision import get_extractor
from thingsvision.utils.storing import save_features
from thingsvision.utils.data import ImageDataset, DataLoader

model_name = 'clip'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT-B/32'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)

```

--------------------------------

### Initialize Ecoset-Trained Model

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Load models trained on the Ecoset dataset. This example shows how to initialize 'Alexnet_ecoset'. Other available models include Resnet50_ecoset, VGG16_ecoset, and Inception_ecoset.

```python
import torch
from thingsvision import get_extractor

model_name = 'Alexnet_ecoset'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True
)

```

--------------------------------

### Initialize CORnet Model

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Use this snippet to load a CORnet model. Ensure PyTorch is installed and specify the desired model name (e.g., 'cornet-s').

```python
import torch
from thingsvision import get_extractor

model_name = 'cornet-s'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True
)

```

--------------------------------

### Create and Activate Conda Environment

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Use this command to create a new conda environment with a specific Python version and activate it for installing ThingsVision.

```bash
$ conda create -n thingsvision python=3.9
$ conda activate thingsvision

```

--------------------------------

### Get Torchvision AlexNet Extractor

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiates an extractor for the AlexNet model from the torchvision source. Uses default pretrained weights if available.

```python
import torch
from thingsvision import get_extractor

model_name = 'alexnet'
source = 'torchvision'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {'weights': 'DEFAULT'}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)

```

--------------------------------

### Get Custom OpenCLIP Model Extractor

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiate an extractor for a custom OpenCLIP model. Specify the `variant` for architecture and `dataset` for the training dataset (e.g., 'laion400m_e32').

```python
import torch
from thingsvision import get_extractor

model_name = 'OpenCLIP'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT-B/32',
    'dataset': 'laion400m_e32'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)

```

--------------------------------

### Get Torchvision RegNet Extractor with Specific Weights

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiates an extractor for the RegNet Y 32GF model from torchvision, specifying custom pretrained weights ('IMAGENET1K_SWAG_LINEAR_V1').

```python
import torch
from thingsvision import get_extractor

model_name = 'regnet_y_32gf'
source = 'torchvision'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {'weights': 'IMAGENET1K_SWAG_LINEAR_V1'}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)

```

--------------------------------

### Get Custom CLIP Model Extractor

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiate an extractor for a custom CLIP model. The `variant` parameter specifies the model architecture (e.g., 'ViT-B/32'). `pretrained=True` loads ImageNet weights.

```python
import torch
from thingsvision import get_extractor

model_name = 'clip'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT-B/32'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)

```

--------------------------------

### Get Custom Harmonization Model Extractor

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiate an extractor for a Harmonization model. The `variant` parameter specifies the model architecture (e.g., 'ViT_B16').

```python
import torch
from thingsvision import get_extractor

model_name = 'Harmonization'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'ViT_B16'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)

```

--------------------------------

### Get Keras Model Extractor

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiate an extractor for a Keras model. Ensure the model name is case-sensitive and matches the `keras.applications` documentation. Set `pretrained=True` to use ImageNet weights.

```python
import torch
from thingsvision import get_extractor

model_name = 'VGG16'
source = 'keras'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True
)

```

--------------------------------

### Get Extractor from Existing Model

Source: https://vicco-group.github.io/thingsvision/CustomModels.html

Use get_extractor_from_model to directly create an extractor for PyTorch or TensorFlow models. Pass the model, device, backend, and optional transforms.

```python
from thingsvision import get_extractor_from_model
from torchvision.models import alexnet, AlexNet_Weights
import torch

# initialize a model of your choice, here we use AlexNet from torchvision 
# and load the ImageNet weights
model_weights = AlexNet_Weights.DEFAULT
model = alexnet(model_weights)

# you can also pass a custom preprocessing function that is applied to every 
# image before extraction
transforms = model_weights.transforms()

# provide the backend of the model (either 'pt' or 'tf')
backend = 'pt'

# set the device to 'cuda' if you have a GPU available
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor_from_model(
  model=model, 
  device=device,
  transforms=transforms,
  backend=backend
)

```

--------------------------------

### Get Custom DreamSim Model Extractor

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiate an extractor for a DreamSim model. The `variant` parameter selects the specific DreamSim model (e.g., 'open_clip_vitb32'). Note that 'dino_vitb16' and 'ensemble' variants only support extraction from the `model.mlp` module.

```python
import torch
from thingsvision import get_extractor

model_name = 'DreamSim'
module_name = 'model.mlp'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_parameters = {
    'variant': 'open_clip_vitb32'
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)

```

--------------------------------

### Initialize ALIGN Model

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Load Kakaobrain's reproduction of the ALIGN model from Hugging Face. This snippet demonstrates the basic initialization process.

```python
import torch
from thingsvision import get_extractor

model_name = 'Kakaobrain_Align'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
)

```

--------------------------------

### Initialize and Compare CKA

Source: https://vicco-group.github.io/thingsvision/RSA.html

Initializes the Centered Kernel Alignment (CKA) calculator with specified backend, kernel, and other parameters. Then, it compares two sets of features (X and Y) to compute the CKA similarity score. Ensure the backend and device are set appropriately for your environment.

```python
from thingsvision.core.cka import get_cka

backend = "torch" # can be set to either 'torch' or 'numpy' 
m = # number of images (e.g., features_i.shape[0])
kernel = 'linear' # linear or rbf kernel (for rbf kernel define sigma, i.e., the width of the Gaussian)
unbiased = True # whether to compute an unbiased version of CKA
device = "cuda" # only necessary to be defined for the 'torch' backend (NumPy runs on CPU only)
sigma = None # needs to be defined for 'rbf' kernel

cka = get_cka(backend=backend, m=m, kernel=kernel, unbiased=unbiased, device=device, sigma=sigma)
rho = cka.compare(X=features_i, Y=features_j)


```

--------------------------------

### Create Dataset and DataLoader for Image Feature Extraction

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Set up an ImageDataset and DataLoader for your images. The dataset class handles images in a root directory, and the DataLoader provides batches for processing.

```python
root='path/to/your/image/directory' # (e.g., './images/)
batch_size = 32

dataset = ImageDataset(
    root=root,
    out_path='path/to/features',
    backend=extractor.get_backend(), # backend framework of model
    transforms=extractor.get_transformations(resize_dim=256, crop_dim=224) # set the input dimensionality to whichever values are required for your pretrained model
)

batches = DataLoader(
    dataset=dataset,
    batch_size=batch_size,
    backend=extractor.get_backend() # backend framework of model
)

```

--------------------------------

### Initialize Segment Anything Model

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Load the Segment Anything model. You can specify different variants like 'vit_h', 'vit_l', or 'vit_b' using the 'model_parameters' argument.

```python
import torch
from thingsvision import get_extractor

model_name = 'SegmentAnything'
source = 'custom'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

model_parameters = {
    'variant': 'vit_h' # also vit_l and vit_b
}

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters
)

```

--------------------------------

### Implement Custom Model Class

Source: https://vicco-group.github.io/thingsvision/CustomModels.html

Inherit from Custom and implement create_model to define your model, weights, and preprocessing. Set the backend attribute to 'pt' or 'tf'.

```python
from thingsvision.custom_models.custom import Custom
import torchvision.models as torchvision_models
import torch

class VGG16_ecoset(Custom):
    def __init__(self, device, **kwargs) -> None:
        super().__init__(device)
        self.backend = 'pt'
        self.preprocess = None

    def create_model(self):
        model = torchvision_models.vgg16(pretrained=False, num_classes=565)
        path_to_weights = 'https://osf.io/fe7s5/download'
        state_dict = torch.hub.load_state_dict_from_url(path_to_weights, map_location=self.device)
        model.load_state_dict(state_dict)
        return model, self.preprocess

```

--------------------------------

### Unzipping Object Images

Source: https://vicco-group.github.io/thingsvision

Use this command to unzip multiple object image zip files. Replace 'the_password' with the actual password.

```bash
for fn in object_images_*.zip; do unzip -P the_password $fn; done
```

--------------------------------

### Load SSL MAE Model with Average Pooling

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiates an extractor for a MAE ViT model from the 'ssl' source, using average pooling for feature extraction. Model names specify architecture and patch resolution (e.g., 'mae-vit-large-p16').

```python
import torch
from thingsvision import get_extractor

model_name = 'mae-vit-large-p16'
source = 'ssl'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_paramters = {"token_extraction": "avg_pool"} # average-pool tokens before extracting the MAE features

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)

```

--------------------------------

### Load SSL DINO Model with CLS Token Extraction

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiates an extractor for a DINO ViT model from the 'ssl' source, specifically extracting features for the CLS token. Model names indicate architecture and patch resolution (e.g., 'dino-vit-base-p16').

```python
import torch
from thingsvision import get_extractor

model_name = 'dino-vit-base-p16'
source = 'ssl'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_paramters = {"token_extraction": "cls_token"} # extract DINO features exclusively for the [cls] token

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True,
  model_parameters=model_parameters,
)

```

--------------------------------

### Load Timm Model

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiates an extractor for a specified model from the 'timm' library. Ensure model names match 'timm' documentation exactly. `pretrained=True` loads weights from the model's documentation.

```python
import torch
from thingsvision import get_extractor

model_name = 'tf_efficientnet_b0'
source = 'timm'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True
)

```

--------------------------------

### Loading Images from a Directory with ImageDataset

Source: https://vicco-group.github.io/thingsvision/LoadingYourData.html

Use `ImageDataset` for images stored in a folder structure. Labels are inferred from subfolder names. Ensure an extractor and transformations are provided.

```python
from thingsvision.utils.data import ImageDataset

# load extractor beforehand
extractor = ...

dataset = ImageDataset(
    root='path/to/root/img/directory' # (e.g., './root/')
    out_path='path/to/features',
    backend=extractor.get_backend(),
    transforms=extractor.get_transformations()
)

```

--------------------------------

### Load SSL SimCLR Model

Source: https://vicco-group.github.io/thingsvision/AvailableModels.html

Instantiates an extractor for a SimCLR model from the 'ssl' source. Models are typically ResNet50 architecture pre-trained on ImageNet-1K. `pretrained=True` loads the pre-trained weights.

```python
import torch
from thingsvision import get_extractor

model_name = 'simclr-rn50'
source = 'ssl'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor(
  model_name=model_name,
  source=source,
  device=device,
  pretrained=True
)

```

--------------------------------

### Custom Forward and Flatten Functions

Source: https://vicco-group.github.io/thingsvision/CustomModels.html

Provide custom forward_fn and flatten_fn to get_extractor_from_model for models with non-standard forward signatures or activation flattening needs. The custom_forward_fn must accept 'self' and 'batch'.

```python
from thingsvision import get_extractor_from_model
import torch.nn as nn

class ComplexForwardModel(nn.Module):
    def __init__(self):
        super().__init__()
        # model definition 
        # ...

    def forward(self, x, y):
        # forward function with custom signature
        # ...
        return x

def custom_forward_fn(self, batch):
    # custom forward function that passes the batch to the model's forward function
    return self.model(batch, y=None)

model = ComplexForwardModel()
backend = 'pt'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

extractor = get_extractor_from_model(
  model=model, 
  device=device,
  transforms=transforms,
  backend=backend,
  forward_fn=custom_forward_fn
)

```

--------------------------------

### Display ResNet50 Model Architecture

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Use `extractor.show_model()` to display the architecture of the ResNet50 model. This helps in identifying module names for feature extraction.

```python
extractor.show_model()

# Output:
ResNet(
  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace=True)
  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  (layer1): Sequential(
    (0): Bottleneck(
      (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (downsample): Sequential(
        (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): Bottleneck(
      (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size [...]
```

--------------------------------

### Display AlexNet Model Architecture

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Use `extractor.show_model()` to display the architecture of the AlexNet model. This helps in identifying module names for feature extraction.

```python
extractor.show_model()

# Output:
AlexNet(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))
    (1): ReLU(inplace=True)
    (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
    (4): ReLU(inplace=True)
    (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (7): ReLU(inplace=True)
    (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (9): ReLU(inplace=True)
    (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(6, 6))
  (classifier): Sequential(
    (0): Dropout(p=0.5, inplace=False)
    (1): Linear(in_features=9216, out_features=4096, bias=True)
    (2): ReLU(inplace=True)
    (3): Dropout(p=0.5, inplace=False)
    (4): Linear(in_features=4096, out_features=4096, bias=True)
    (5): ReLU(inplace=True)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)

```

--------------------------------

### Extract Features to Disk

Source: https://vicco-group.github.io/thingsvision/LowMemOptions.html

Use this when RAM is a concern. Features are written directly to disk as they are extracted, freeing up RAM. The `step_size` parameter controls how many batches are processed before writing to disk.

```python
extractor = ...
batches = ...

output_dir = '/path/to/output/directory'
extractor.extract_features(
    batches=batches,
    module_name=...,
    flatten_acts=True,
    output_dir=output_dir
) # returns None if output_dir is set

```

--------------------------------

### Loading Images from an HDF5 File with HDF5Dataset

Source: https://vicco-group.github.io/thingsvision/LoadingYourData.html

Use `HDF5Dataset` to load images stored within HDF5 files. Specify the HDF5 file path, the key for the image dataset, and provide an extractor backend and transformations.

```python
from thingsvision.utils.data import HDF5Dataset

# load extractor beforehand
extractor = ...

dataset = HDF5Dataset(
    hdf5_fp='path/to/hdf5/file' # (e.g., './nsd_stimuli.hdf5')
    img_ds_key='imgBrick',
    out_path='path/to/features',
    backend=extractor.get_backend(),
    transforms=extractor.get_transformations()
)

```

--------------------------------

### Extract Features via CLI

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Use the 'extract-features' command to extract features from images in a directory. Specify image root, model, module, batch size, device, and output path.

```bash
thingsvision extract-features --image-root "./data" --model-name "alexnet" --module-name "features.10" --batch-size 32 --device "cuda" --source "torchvision" --file-format "npy" --out-path "./features"

```

--------------------------------

### Citation for THINGSvision Paper

Source: https://vicco-group.github.io/thingsvision

This is the citation format to use when referencing the THINGSvision paper in academic work.

```bibtex
@article{Muttenthaler_2021,
	author = {Muttenthaler, Lukas and Hebart, Martin N.},
	title = {THINGSvision: A Python Toolbox for Streamlining the Extraction of Activations From Deep Neural Networks},
	journal ={Frontiers in Neuroinformatics},
	volume = {15},
	pages = {45},
	year = {2021},
	url = {https://www.frontiersin.org/article/10.3389/fninf.2021.679838},
	doi = {10.3389/fninf.2021.679838},
	issn = {1662-5196},
}
```

--------------------------------

### Extract and Save Image Features in Python

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Extract features from image batches using the specified module name and save them to disk. Features can be saved in various formats like 'npy', 'txt', 'mat', 'pt', or 'hdf5'.

```python
module_name = 'visual'

features = extractor.extract_features(
  batches=batches,
  module_name=module_name,
  flatten_acts=True, # flatten 2D feature maps from an early convolutional or attention layer
  output_type="ndarray", # or "tensor" (only applicable to PyTorch models of which CLIP is one!)
)

save_features(features, out_path='path/to/features', file_format='npy') # file_format can be set to "npy", "txt", "mat", "pt", or "hdf5"

```

--------------------------------

### Custom Data Pipeline Extraction with PyTorch

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

Use this snippet for feature extraction with a custom PyTorch dataloader. It allows for preprocessing before extraction and postprocessing of features.

```python
module_name = 'visual'

# your custom dataset and dataloader classes come here (for example, a PyTorch data loader)
my_dataset = ...
my_dataloader = ...

with extractor.batch_extraction(module_name, output_type="tensor") as e: 
  for batch in my_dataloader:
    ... # whatever preprocessing you want to add to the batch
    feature_batch = e.extract_batch(
      batch=batch,
      flatten_acts=True, # flatten 2D feature maps from an early convolutional or attention layer
      )
    ... # whatever post-processing you want to add to the extracted features

```

--------------------------------

### Custom Data Pipeline Extraction with TensorFlow/Keras

Source: https://vicco-group.github.io/thingsvision/GettingStarted.html

This snippet demonstrates feature extraction using a custom TensorFlow/Keras dataloader. It supports preprocessing of batches and postprocessing of extracted features.

```python
module_name = 'visual'

# your custom dataset and dataloader classes come here (for example, TFRecords files)
my_dataset = ...
my_dataloader = ...

for batch in my_dataloader:
  ... # whatever preprocessing you want to add to the batch
  feature_batch = extractor.extract_batch(
    batch=batch,
    module_name=module_name,
    flatten_acts=True, # flatten 2D feature maps from an early convolutional or attention layer
    )
  ... # whatever post-processing you want to add to the extracted features

```

--------------------------------

### Compute RDM and Correlate RDMs

Source: https://vicco-group.github.io/thingsvision/RSA.html

Computes a representational dissimilarity matrix (RDM) from features and then correlates it with a human RDM using a specified correlation method. Ensure features and human representations are properly prepared.

```python
from thingsvision.core.rsa import compute_rdm, correlate_rdms

rdm_dnn = compute_rdm(features, method='correlation')
corr_coeff = correlate_rdms(rdm_dnn, rdm_human, correlation='pearson')


```

--------------------------------

### Align Features with gLocal

Source: https://vicco-group.github.io/thingsvision/Alignment.html

Use this method to post-align extracted features with human object similarity using the gLocal alignment type. This applies an affine transformation to the representation space.

```python
aligned_features = extractor.align(
    features=features,
    module_name=module_name,
    alignment_type="gLocal",
)
```
Tokens:
Raw