PyTorch Out-of-Distribution Detection

https://pytorch-ood.readthedocs.io
Admin
PyTorch-OOD is a PyTorch-based library offering modular, tested, and well-documented implementations...
Tokens:230,973
Snippets:1,060
Trust Score:8.3
Update:11 months ago
Show doc for...
Tokens:
Raw
### PyTorch-OOD Readme Basic Example

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/9f1924e867e30011153740205b1cba72/newsgroups_oe

A basic example demonstrating the usage of PyTorch-OOD. This snippet shows how to load a model and perform OOD detection. It assumes PyTorch and PyTorch-OOD are installed.

```python
import torch
import torchvision.models as models
from torchood.model import OODModel

# Load a pre-trained model (e.g., ResNet-50)
model = models.resnet50(pretrained=True)

# Wrap the model with OODModel
ood_model = OODModel(model)

# Example input data (batch size 1)
data = torch.randn(1, 3, 224, 224)

# Get OOD scores
scores = ood_model.get_ood_scores(data)

print(f"OOD Scores: {scores}")
```

--------------------------------

### Setup Preprocessing and Data Loaders - Python

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/loss/supervised/scone

Configures image transformations, datasets (CIFAR10, TinyImages300k, Textures), and data loaders for training and testing. It applies resizing andToTensor transforms, and uses `ToUnknown` to label OOD data. This setup is essential for preparing the data for the OOD detection task.

```python
 76 trans = tvt.Compose([tvt.Resize(size=(32, 32)), tvt.ToTensor()])
 77
 78 # setup ID training data
 79 dataset_in_train = CIFAR10(root="data", train=True, download=True, transform=trans)
 80
 81 # setup OOD training data, use ToUnknown() to mark labels as OOD
 82 # this way, outlier exposure can automatically decide if the training samples are ID or OOD
 83 dataset_out_train = TinyImages300k(
 84     root="data", download=True, transform=trans, target_transform=ToUnknown()
 85 )
 86
 87 # setup ID test data
 88 dataset_in_test = CIFAR10(root="data", train=False, transform=trans)
 89
 90 # TODO: Add Covariate-shifted Data
 91
 92 # setup OOD test data, use ToUnknown() to mark labels as OOD
 93 dataset_out_test = Textures(
 94     root="data", download=True, transform=trans, target_transform=ToUnknown()
 95 )
 96
 97 # create data loaders
 98 train_loader = DataLoader(dataset_in_train + dataset_out_train, batch_size=64, shuffle=True)
 99 test_loader = DataLoader(dataset_in_test + dataset_out_test, batch_size=128)
100
101 train_loader_in = DataLoader(dataset_in_train, batch_size=128)
```

--------------------------------

### Setup Data and Model for OpenMax - Python

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/detectors/openmax

Initializes data transformations, datasets (CIFAR10 for in-distribution and Textures for out-of-distribution), and data loaders. It also sets up a pre-trained WideResNet model for CIFAR-10.

```Python
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10

from pytorch_ood.dataset.img import Textures
from pytorch_ood.detector import OpenMax
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

fix_random_seed(123)

device = "cuda:0"

trans = WideResNet.transform_for("cifar10-pt")

dataset_train = CIFAR10(root="data", train=True, download=True, transform=trans)
dataset_in_test = CIFAR10(root="data", train=False, download=True, transform=trans)
dataset_out_test = Textures(
    root="data", download=True, transform=trans, target_transform=ToUnknown()
)

train_loader = DataLoader(dataset_train, batch_size=128, shuffle=True)

# create data loaders
test_loader = DataLoader(dataset_in_test + dataset_out_test, batch_size=128)

model = WideResNet(num_classes=10, pretrained="cifar10-pt").to(device).eval()
```

--------------------------------

### Setup and Imports for CIFAR10 OOD Detection

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/benchmarks/interface/cifar10_openood

Imports necessary libraries like pandas and torch, and specific modules from pytorch-ood for benchmark, detectors, models, and utilities. It also sets the random seed and defines device and loader configurations.

```python
13 import pandas as pd  # additional dependency, used here for convenience
14 import torch
15
16 from pytorch_ood.benchmark import CIFAR10_OpenOOD
17 from pytorch_ood.detector import MaxSoftmax, ReAct, ASH
18 from pytorch_ood.model import WideResNet
19 from pytorch_ood.utils import fix_random_seed
20
21 fix_random_seed(123)
22
23 device = "cuda:0"
24 loader_kwargs = {"batch_size": 64}
```

--------------------------------

### Import Libraries and Setup Device for OpenMax

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/91e3c482d6be20a674d72492c494353a/openmax

Imports necessary libraries from PyTorch and PyTorch-OOD for OOD detection. It also sets the random seed for reproducibility and defines the device (e.g., 'cuda:0') for computations. This is the initial setup for using the OpenMax detector.

```python
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10

from pytorch_ood.dataset.img import Textures
from pytorch_ood.detector import OpenMax
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

fix_random_seed(123)

device = "cuda:0"
```

--------------------------------

### Install PyTorch-OOD from PyPI

Source: https://pytorch-ood.readthedocs.io/en/latest/info

Installs the latest stable version of the PyTorch-OOD library directly from the Python Package Index (PyPI). This is the standard method for users who want to use the library in their projects.

```shell
pip install pytorch-ood

```

--------------------------------

### Install PyTorch-OOD from PyPI

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/info

Installs the latest stable version of the pytorch-ood library directly from the Python Packaging Index (PyPI). This is the standard method for installing Python packages.

```shell
pip install pytorch-ood
```

--------------------------------

### Setup Data Loaders for MCHAD - PyTorch

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/loss/supervised/mchad

This code sets up data loaders for CIFAR10 (in-distribution) and TinyImages300k (out-of-distribution) datasets. It also includes setup for Textures dataset as out-of-distribution test data. Transforms are applied using WideResNet's pre-defined transform for 'imagenet32-nocifar'.

```python
28 import torch
29 from torch.optim.lr_scheduler import CosineAnnealingLR
30 from torch.optim import Adam
31 from torch.utils.data import DataLoader, random_split
32 from torchmetrics import Accuracy
33 from torchvision.datasets import CIFAR10
34 from tqdm import tqdm
35 import math
36
37 from pytorch_ood.dataset.img import Textures, TinyImages300k
38 from pytorch_ood.loss import MCHADLoss
39 from pytorch_ood.model import WideResNet
40 from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed, is_known
41
42 fix_random_seed(123)
43
44 data_root = "data"
45 n_epochs = 10
46 device = "cuda:0"
47 embedding_dim = 7  # dimensionality of output space
48 margin = math.sqrt(embedding_dim)
49 batch_size = 256
50
51 trans = WideResNet.transform_for("imagenet32-nocifar")
52
53 # setup ID training data
54 data_in_train = CIFAR10(root=data_root, train=True, download=True, transform=trans)
55
56 # setup OOD training data, same size as ID training data
57 tiny300k = TinyImages300k(
58     root=data_root, download=True, transform=trans, target_transform=ToUnknown()
59 )
60
61 # setup ID test data
62 data_in_test = CIFAR10(root=data_root, train=False, transform=trans)
63
64 # setup OOD test data, use ToUnknown() to mark labels as OOD
65 data_out_test = Textures(
66     root=data_root, download=True, transform=trans, target_transform=ToUnknown()
67 )
68
69
70 # create data loaders
71 test_loader = DataLoader(data_in_test + data_out_test, batch_size=batch_size, num_workers=16)
72
73 data_out_train, _ = random_split(
74     tiny300k, [len(data_in_train), len(tiny300k) - len(data_in_train)]
75 )
76 train_loader = DataLoader(
77     data_in_train + data_out_train, batch_size=batch_size, shuffle=True, num_workers=16
78 )
```

--------------------------------

### Install PyTorch-OOD from Git Repository

Source: https://pytorch-ood.readthedocs.io/en/latest/info

Installs the latest development version of the PyTorch-OOD library directly from its Git repository. This is useful for developers or users who need access to the newest features or bug fixes before they are released to PyPI.

```shell
pip install git+ssh://git@github.com/kkirchheim/pytorch-ood.git@dev

```

--------------------------------

### Setup DataLoaders for CIFAR-10 and Textures

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/72639fd1157990859b80a45ebb3981bf/gram

Configures data transformations for CIFAR-10 and loads training and testing datasets, including in-distribution CIFAR-10 and out-of-distribution Textures datasets. It then creates DataLoaders for efficient batch processing.

```python
trans = WideResNet.transform_for("cifar10-pt")

dataset_train = CIFAR10(root="data", train=True, download=True, transform=trans)
dataset_in_test = CIFAR10(root="data", train=False, download=True, transform=trans)
dataset_out_test = Textures(
    root="data", download=True, transform=trans, target_transform=ToUnknown()
)

train_loader = DataLoader(dataset_train, batch_size=128, shuffle=True, num_workers=10)

# create data loaders
test_loader = DataLoader(dataset_in_test + dataset_out_test, batch_size=128, num_workers=10)
```

--------------------------------

### Setup Data Loaders for CIFAR 10

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/detectors/gram

Configures data transformations and creates DataLoaders for training, in-distribution testing, and out-of-distribution testing using CIFAR10 and Textures datasets. Dependencies include torchvision, pytorch_ood.dataset.img, and pytorch_ood.utils.

```python
from torch import nn
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10

import torch.nn.functional as F
from pytorch_ood.dataset.img import Textures
from pytorch_ood.detector import Gram
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

logging.basicConfig(level=logging.INFO)

fix_random_seed(123)

device = "cuda"

trans = WideResNet.transform_for("cifar10-pt")

dataset_train = CIFAR10(root="data", train=True, download=True, transform=trans)
dataset_in_test = CIFAR10(root="data", train=False, download=True, transform=trans)
dataset_out_test = Textures(
    root="data", download=True, transform=trans, target_transform=ToUnknown()
)

train_loader = DataLoader(dataset_train, batch_size=128, shuffle=True, num_workers=10)

# create data loaders
test_loader = DataLoader(dataset_in_test + dataset_out_test, batch_size=128, num_workers=10)
```

--------------------------------

### Shell Command to Run MCHAD Example

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/57ecaed3de1f6ee19ff9ebd9da9ecd8e/mchad

This is a shell command to execute the MCHAD example script. It assumes the script is located in the 'examples/loss/supervised/' directory.

```shell
python examples/loss/supervised/mchad.py
```

--------------------------------

### Setup CIFAR10 Dataset and DataLoader

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/loss/supervised/cifar_vos

Prepares the CIFAR10 dataset for training and testing, including setup for in-distribution and out-of-distribution test data. It configures transformations and uses a DataLoader for efficient batch processing. The `ToUnknown` transform is used to mark OOD samples.

```Python
import numpy as np
import torch
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10

from pytorch_ood.dataset.img import Textures
from pytorch_ood.detector import EnergyBased, WeightedEBO
from pytorch_ood.loss import VirtualOutlierSynthesizingRegLoss
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

device = "cuda:0"
batch_size = 128
num_epochs = 10
lr = 0.1
num_classes = 10

fix_random_seed(12345)
g = torch.Generator()
g.manual_seed(0)

trans = WideResNet.transform_for("cifar10-pt")

dataset = CIFAR10(root="data", train=True, transform=trans, download=True)

# setup ID test data
dataset_in_test = CIFAR10(root="data", train=False, transform=trans)

# setup OOD test data, use ToUnknown() to mark labels as OOD
dataset_out_test = Textures(
    root="data", download=True, transform=trans, target_transform=ToUnknown()
)


loader = DataLoader(
    dataset,
    batch_size=batch_size,
    shuffle=True,
    num_workers=10,
    worker_init_fn=fix_random_seed,
    generator=g,
)
```

--------------------------------

### Import Libraries for OSR

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/osr/osr

Imports necessary libraries for PyTorch, dataset handling, model definition, OSR utilities, and progress tracking. These are essential for setting up and running the OSR experiment.

```Python
import torch.nn
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10
from torch.nn import CrossEntropyLoss
from tqdm import tqdm
from torchmetrics import Accuracy

from pytorch_ood.dataset.ossim import DynamicOSS
from pytorch_ood.model import WideResNet
from pytorch_ood.detector import MaxSoftmax
from pytorch_ood.utils import fix_random_seed, TargetMapping, OODMetrics, is_known
```

--------------------------------

### Install PyTorch-OOD from Git (Dev Branch)

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/info

Installs the latest development branch of pytorch-ood directly from its GitHub repository using SSH. This allows users to access the newest features and changes.

```shell
pip install git+ssh://git@github.com/kkirchheim/pytorch-ood.git@dev
```

--------------------------------

### Setup Datasets - Python

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/segmentation/street_vos

Initializes the `StreetHazards` dataset for both training and testing. The `my_transform` function is applied to the dataset during initialization. The dataset is downloaded if not already present.

```python
72 dataset = StreetHazards(root="data", subset="test", transform=my_transform, download=True)
73 dataset_test = StreetHazards(root="data", subset="test", transform=my_transform, download=True)
```

--------------------------------

### Build PyTorch-OOD Documentation

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/info

Installs necessary Sphinx packages for building documentation and then uses the 'make html' command in the docs directory to generate the HTML documentation. This is useful for developers who want to view or contribute to the documentation.

```shell
pip install sphinx_gallery sphinx_rtd_theme sphinx
cd docs
make html
```

--------------------------------

### Cityscapes Dataset Setup

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/segmentation/roadanomaly

Initializes the Cityscapes dataset for training. Requires the dataset to be downloaded and placed in the 'data/cityscapes' folder. Uses 'fine' mode and 'semantic' target type with custom transformations.

```Python
# Please download Citiscapes Dataset, for example, from https://www.cityscapes-dataset.com/ or https://github.com/mcordts/cityscapesScripts/blob/master/cityscapesscripts/download/downloader.py
# and put it in the data/cityscapes folder
dataset = Cityscapes(
    root="data/cityscapes",
    split="train",
    transforms=cityscapes_transform,
    mode="fine",
    target_type="semantic",
)
```

--------------------------------

### Import Libraries for Gram Detector

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/72639fd1157990859b80a45ebb3981bf/gram

Imports necessary libraries from PyTorch, torchvision, and PyTorch-OOD for setting up the Gram detector and handling data. It also configures logging and fixes random seeds for reproducibility.

```python
import logging

from torch import nn
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10

import torch.nn.functional as F
from pytorch_ood.dataset.img import Textures
from pytorch_ood.detector import Gram
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

logging.basicConfig(level=logging.INFO)

fix_random_seed(123)

device = "cuda"
```

--------------------------------

### Install PyTorch with CUDA Support (Conda)

Source: https://pytorch-ood.readthedocs.io/en/latest/info

Installs PyTorch, torchvision, torchaudio, torchtext, and PyTorch-CUDA with specific versions using Conda. This command is typically used for setting up the environment for deep learning tasks that leverage GPU acceleration.

```shell
conda install pytorch torchvision torchaudio torchtext==0.14.0 pytorch-cuda=11.7 -c pytorch -c nvidia

```

--------------------------------

### Initialize and Fit Gram Detector

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/72639fd1157990859b80a45ebb3981bf/gram

Instantiates the Gram detector with the extracted model components and specifies the number of classes and poles. It then fits the detector to the training data.

```python
detector = Gram(
    head,
    [layer1, layer2, layer3, layer4, layer5],
    num_classes=10,
    num_poles_list=[1, 2, 3, 4, 5],
)

print("Fitting...")
detector.fit(train_loader, device=device)
```

--------------------------------

### Setup and Imports for ImageNet OOD Benchmark

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/97a3fefbd74860f608c23bed4f4b771b/imagenet_openood

Imports necessary libraries including pandas, torch, torchvision models, and OpenOOD components. It also sets up random seed, device, and loader arguments for the benchmark.

```python
import pandas as pd  # additional dependency, used here for convenience
import torch
from torchvision.models import resnet50
from torchvision.models.resnet import ResNet50_Weights

from pytorch_ood.benchmark import ImageNet_OpenOOD
from pytorch_ood.detector import MaxSoftmax
from pytorch_ood.utils import fix_random_seed

fix_random_seed(123)

device = "cuda:0"
loader_kwargs = {"batch_size": 16, "num_workers": 12}
```

--------------------------------

### Setup Model - Python

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/segmentation/street_vos

Configures and initializes a Feature Pyramid Network (FPN) model with a ResNet-50 encoder pre-trained on ImageNet. The model is moved to the specified device (e.g., CUDA).

```python
78 model = smp.FPN(
79     encoder_name="resnet50",
80     encoder_weights="imagenet",
81     in_channels=3,
82     classes=num_classes,
83 ).to(device)
```

--------------------------------

### Setup CIFAR10 Preprocessing and Datasets

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/84fdcd664ed0798686d0d35c174ac7ce/osr

Configures the necessary transformations for the CIFAR10 dataset using WideResNet's built-in methods and loads both training and testing datasets, concatenating them for use in the open-set simulation.

```python
trans = WideResNet.transform_for("cifar10-pt")
norm_std = WideResNet.norm_std_for("cifar10-pt")

dataset_1 = CIFAR10(root="data", train=True, transform=trans, download=True)
dataset_2 = CIFAR10(root="data", train=False, transform=trans, download=True)
dataset = dataset_1 + dataset_2
```

--------------------------------

### Build PyTorch-OOD Documentation

Source: https://pytorch-ood.readthedocs.io/en/latest/info

Installs necessary packages for building documentation (Sphinx, Sphinx Gallery, Read the Docs theme) and then runs the documentation build process using make. This is primarily for developers contributing to the documentation.

```shell
pip install sphinx_gallery sphinx_rtd_theme sphinx
cd docs
make html

```

--------------------------------

### Model Definition and Optimizer Setup - Python

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/loss/supervised/scone

Defines and initializes a WideResNet model, pre-trained on ImageNet (excluding CIFAR10 classes), and modifies its final layer for CIFAR10's 10 classes. It also initializes a logistic regression model and sets up an SGD optimizer with momentum and weight decay. The `evaluate_classification_loss_training` function is called to get the initial loss for `EnergyMarginLoss`.

```python
105 model = WideResNet(num_classes=1000, pretrained="imagenet32-nocifar")
106 # we have to replace the final layer to account for the lower number of
107 # classes in the CIFAR10 dataset
108 model.fc = torch.nn.Linear(model.fc.in_features, 10)
109
110 model.to(device)
111
112 logistic_regression = nn.Linear(1, 1)
113
114 logistic_regression.to(device)
115
116 opti = SGD(
117     list(model.parameters()) + list(logistic_regression.parameters()),
118     lr=0.0001,
119     momentum=0.9,
120     weight_decay=0.0005,
121     nesterov=True,
122 )
123
124 # Calculate ID Classification Loss Before Fine-Tuning
125 full_train_loss = evaluate_classification_loss_training(
126     model=model, train_loader_in=train_loader_in
127 )
128
129 criterion = EnergyMarginLoss(full_train_loss=full_train_loss)
130
131 scheduler = torch.optim.lr_scheduler.MultiStepLR(
132     optimizer=opti,
133     milestones=[int(n_epochs * 0.5), int(n_epochs * 0.75), int(n_epochs * 0.9)],
134     gamma=0.5,
135 )
```

--------------------------------

### Setup and Initialization for ODIN Benchmark

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/benchmarks/interface/cifar100_odin

This code block initializes necessary libraries and sets up the environment for the ODIN benchmark. It imports required modules from pytorch-ood, defines the device to be used (GPU if available), and sets up loader keyword arguments for batch processing. It also fixes the random seed for reproducibility.

```python
import pandas as pd
import torch

from pytorch_ood.benchmark import CIFAR100_ODIN
from pytorch_ood.detector import ODIN, MaxSoftmax
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import fix_random_seed

fix_random_seed(123)

device = "cuda:0"
loader_kwargs = {"batch_size": 64}
```

--------------------------------

### Setup PyTorch OOD Environment and Configurations

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/878b76c441037b8698577976f3605e71/newsgroups

Imports necessary libraries for PyTorch OOD, dataset handling, and model components. Sets up fixed random seed, training epochs, learning rate, device, and root directory for data.

```python
import pandas as pd
import torch
import torch.nn.functional as F
from torch.utils.data import DataLoader
from torchtext.data.utils import get_tokenizer
from torchtext.vocab import build_vocab_from_iterator
from tqdm import tqdm

from pytorch_ood.dataset.txt import Multi30k, NewsGroup20, Reuters52, WMT16Sentences
from pytorch_ood.detector import (
    EnergyBased,
    Entropy,
    KLMatching,
    Mahalanobis,
    MaxLogit,
    MaxSoftmax,
    ViM,
)
from pytorch_ood.model import GRUClassifier
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

fix_random_seed(123)

n_epochs = 10
lr = 0.001
device = "cuda:0"
root = "data"
```

--------------------------------

### Install Pandas and Scikit-learn for Benchmarks

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/benchmarks/index

Installs the pandas and scikit-learn libraries, which are required as additional dependencies for running the PyTorch OOD benchmark examples. These libraries are commonly used for data manipulation and evaluation metrics in machine learning.

```shell
pip install pandas scikit-learn
```

--------------------------------

### Setup CIFAR10 Benchmark

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/benchmarks/manual/cifar10_baseline

Imports necessary libraries for PyTorch OOD benchmarking, including dataset handling, neural network models, and OOD detection algorithms. Sets up the device and fixes random seeds for reproducibility.

```Python
import pandas as pd  # additional dependency, used here for convenience
import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10, CIFAR100, MNIST, FashionMNIST

from pytorch_ood.dataset.img import (
    LSUNCrop,
    LSUNResize,
    Textures,
    TinyImageNetCrop,
    TinyImageNetResize,
    Places365,
)
from pytorch_ood.detector import (
    ODIN,
    EnergyBased,
    Entropy,
    KLMatching,
    Mahalanobis,
    MaxLogit,
    MaxSoftmax,
    ViM,
    RMD,
    DICE,
    SHE,
    Gram,
    MultiMahalanobis,
)
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

device = "cuda:0"

fix_random_seed(123)
```

--------------------------------

### Setup ODIN Benchmark for CIFAR10

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/benchmarks/interface/cifar10_odin

Initializes the necessary components for the ODIN benchmark on CIFAR10. This includes setting up the random seed, device, data loader arguments, and importing required libraries like pandas, torch, and specific modules from pytorch_ood.

```python
12 import pandas as pd  # additional dependency, used here for convenience
13 import torch
14
15 from pytorch_ood.benchmark import CIFAR10_ODIN
16 from pytorch_ood.detector import ODIN, MaxSoftmax
17 from pytorch_ood.model import WideResNet
18 from pytorch_ood.utils import fix_random_seed
19
20 fix_random_seed(123)
21
22 device = "cuda:0"
23 loader_kwargs = {"batch_size": 64, "num_workers": 12}
```

--------------------------------

### Model Initialization and Optimizer Setup

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/text/newsgroups

Initializes the GRUClassifier model with the vocabulary size and number of output classes (20 for Newsgroups 20). It moves the model to the specified device (GPU if available) and sets up the Adam optimizer and a CosineAnnealingLR learning rate scheduler.

```python
108 print("STAGE 1: Train Model")
109 model = GRUClassifier(num_classes=20, n_vocab=len(vocab))
110 model.to(device)
111
112 optimizer = torch.optim.Adam(model.parameters(), lr=lr)
113 scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=n_epochs)
```

--------------------------------

### Model and Loss Function Setup for VOS

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/loss/supervised/cifar_vos

Initializes a WideResNet model pre-trained on CIFAR10 and sets up the `VirtualOutlierSynthesizingRegLoss`. This involves defining auxiliary neural network layers (`phi` and `weights_energy`) and configuring the loss function parameters.

```Python
model = WideResNet(num_classes=10, pretrained="cifar10-pt").to(device)

phi = torch.nn.Linear(1, 2).to(device)
weights_energy = torch.nn.Linear(num_classes, 1).to(device)
torch.nn.init.uniform_(weights_energy.weight)

criterion = VirtualOutlierSynthesizingRegLoss(
    phi,
    weights_energy,
    device=device,
    num_classes=num_classes,
    num_input_last_layer=128,
    fc=model.fc,
    sample_number=10000,
    select=1,
    sample_from=1000,
    alpha=0.1,
)
```

--------------------------------

### Setup and Data Loading for OpenMax

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/detectors/openmax

This code snippet demonstrates setting up data transformations, loading CIFAR10 for training and testing, and incorporating a custom out-of-distribution dataset (Textures). It configures PyTorch DataLoaders for efficient batch processing. Dependencies include `torch`, `torchvision`, and `pytorch_ood`.

```python
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10

from pytorch_ood.dataset.img import Textures
from pytorch_ood.detector import OpenMax
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

fix_random_seed(123)

device = "cuda:0"

trans = WideResNet.transform_for("cifar10-pt")

dataset_train = CIFAR10(root="data", train=True, download=True, transform=trans)
dataset_in_test = CIFAR10(root="data", train=False, download=True, transform=trans)
dataset_out_test = Textures(
    root="data", download=True, transform=trans, target_transform=ToUnknown()
)

train_loader = DataLoader(dataset_train, batch_size=128, shuffle=True)

# create data loaders
test_loader = DataLoader(dataset_in_test + dataset_out_test, batch_size=128)
```

--------------------------------

### WideResNet: Get Data Transformation Pipeline for Pretrained Models

Source: https://pytorch-ood.readthedocs.io/en/latest/_modules/pytorch_ood/model/wrn

Provides the specific data preprocessing pipeline (transformations) used during the evaluation of pre-trained WideResNet models. This ensures consistency with the original training setup, supporting models like 'cifar10-pt', 'cifar100-pt', and 'er-cifar10-tune'.

```python
    @staticmethod
    def transform_for(pretrained: str) -> tvt.Compose:
        """
        Return pre-processing used for the evaluation of a pretrained model
        """
        if pretrained in ["cifar10-pt", "cifar100-pt", "er-cifar10-tune"]:
            # Setup preprocessing
            mean = [x / 255 for x in [125.3, 123.0, 113.9]]
            std = [x / 255 for x in [63.0, 62.1, 66.7]]
            trans = tvt.Compose(
                [
                    tvt.Resize(size=(32, 32)),
                    ToRGB(),
                    tvt.ToTensor(),
                    tvt.Normalize(std=std, mean=mean),
                ]
            )
            return trans
        elif pretrained in ["imagenet32-nocifar"]:
            mean = [0.5] * 3
            std = [0.5] * 3
            trans = tvt.Compose(
                [
                    tvt.Resize(size=(32, 32)),
                    ToRGB(),
                    tvt.ToTensor(),
                    tvt.Normalize(std=std, mean=mean),
                ]
            )
            return trans

        raise ValueError("Unknown Model")
```

--------------------------------

### Model Training Loop with VOS Loss

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/loss/supervised/cifar_vos

Implements the training loop for the WideResNet model using the configured Virtual Outlier Synthesizing Reg Loss. It includes optimizer setup, learning rate scheduling with CosineAnnealingLR, and iterative updates of model parameters.

```Python
optimizer = torch.optim.SGD(
    list(model.parameters()) + list(phi.parameters()) + list(weights_energy.parameters()),
    lr=lr,
    momentum=0.9,
    weight_decay=5e-4,
)


# setup scheduler for optimizer (recommended)
scheduler = CosineAnnealingLR(
    optimizer,
    T_max=num_epochs * len(loader),
)
loss_ema = 0

for epoch in range(num_epochs):
    for n, (x, y) in enumerate(loader):
        optimizer.zero_grad()
        y, x = y.to(device), x.to(device)

        features = model.features(x)
        y_hat = model.fc(features)
        loss = criterion(y_hat, features, y)

        loss.backward()
        optimizer.step()
        scheduler.step()

        loss_ema = 0.8 * loss_ema + 0.2 * loss.item()

        if n % 10 == 0:
            print(f"Epoch {epoch:03d} [{n:05d}/{len(loader):05d}] 	 Loss: {loss_ema:02.2f}")
```

--------------------------------

### Create PyTorch DataLoader for Training

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/loss/supervised/mchad

This snippet demonstrates how to create a PyTorch DataLoader for training a model. It combines in-distribution and out-of-distribution data, specifies batch size, shuffling, and the number of worker processes for data loading. This is a common setup for training deep learning models, especially when dealing with OOD detection.

```python
train_loader = DataLoader(
    data_in_train + data_out_train,
    batch_size=batch_size,
    shuffle=True,
    num_workers=16,
)
```

--------------------------------

### Setup CIFAR10 datasets

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/osr/osr

Loads the CIFAR10 training and testing datasets using torchvision. It combines both datasets into a single dataset object for further processing. The 'download=True' argument ensures datasets are downloaded if not already present.

```python
dataset_1 = CIFAR10(root="data", train=True, transform=trans, download=True)
dataset_2 = CIFAR10(root="data", train=False, transform=trans, download=True)
dataset = dataset_1 + dataset_2
```

--------------------------------

### Install PyTorch-OOD Editable Version

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/info

Installs a developer version of pytorch-ood in editable mode. This involves cloning the repository and then installing it using pip, allowing for direct modifications to the source code.

```shell
git clone https://github.com/kkirchheim/pytorch-ood
cd pytorch-ood
pip install -e .
```

--------------------------------

### Install PyTorch with CUDA using Conda

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/info

Installs PyTorch, torchvision, torchaudio, torchtext, and pytorch-cuda with a specific version for CUDA 11.7. This command uses conda and specifies channels for PyTorch and NVIDIA.

```shell
conda install pytorch torchvision torchaudio torchtext==0.14.0 pytorch-cuda=11.7 -c pytorch -c nvidia
```

--------------------------------

### MCHAD Loss Setup and Training

Source: https://pytorch-ood.readthedocs.io/en/latest/_sources/auto_examples/loss/supervised/mchad

This Python snippet sets up the model, loss function, optimizer, and data loaders for training with MCHADLoss. It initializes a WideResNet model, configures MCHADLoss with specified dimensions and weights, and prepares data from CIFAR10 and TinyImages300k.

```Python
import torch
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.optim import Adam
from torch.utils.data import DataLoader, random_split
from torchmetrics import Accuracy
from torchvision.datasets import CIFAR10
from tqdm import tqdm
import math

from pytorch_ood.dataset.img import Textures, TinyImages300k
from pytorch_ood.loss import MCHADLoss
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed, is_known

fix_random_seed(123)

data_root = "data"
n_epochs = 10
device = "cuda:0"
embedding_dim = 7  # dimensionality of output space
margin = math.sqrt(embedding_dim)
batch_size = 256

trans = WideResNet.transform_for("imagenet32-nocifar")

# setup ID training data
data_in_train = CIFAR10(root=data_root, train=True, download=True, transform=trans)

# setup OOD training data, same size as ID training data
tiny300k = TinyImages300k(
    root=data_root, download=True, transform=trans, target_transform=ToUnknown()
)

# setup ID test data
data_in_test = CIFAR10(root=data_root, train=False, transform=trans)

# setup OOD test data, use ToUnknown() to mark labels as OOD
data_out_test = Textures(
    root=data_root, download=True, transform=trans, target_transform=ToUnknown()
)


# create data loaders
test_loader = DataLoader(data_in_test + data_out_test, batch_size=batch_size, num_workers=16)

data_out_train, _ = random_split(
    tiny300k, [len(data_in_train), len(tiny300k) - len(data_in_train)]
)
train_loader = DataLoader(
    data_in_train + data_out_train, batch_size=batch_size, shuffle=True, num_workers=16
)


# Create DNN, pretrained on the imagenet excluding cifar10 classes.
# We have to replace the final layer to match the number of classes.
model = WideResNet(num_classes=1000, pretrained="imagenet32-nocifar")
model.fc = torch.nn.Linear(model.fc.in_features, embedding_dim)
model.to(device)

opti = Adam(model.parameters())
criterion = MCHADLoss(
    n_classes=10, n_dim=embedding_dim, weight_oe=0.01, weight_center=2, margin=margin
).to(device)
scheduler = CosineAnnealingLR(opti, T_max=n_epochs * len(train_loader))
```

--------------------------------

### Initialize PyTorch OOD Benchmark and Detectors

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/40e479a2fdb68dd86e6899aa32186d93/cifar10_odin

This snippet initializes the necessary libraries and sets up the environment for the ODIN benchmark. It imports PyTorch, Pandas, and specific components from `pytorch-ood` for defining the model, detectors, and benchmark. It also fixes random seeds for reproducibility and sets up device and loader arguments.

```python
import pandas as pd
import torch

from pytorch_ood.benchmark import CIFAR10_ODIN
from pytorch_ood.detector import ODIN, MaxSoftmax
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import fix_random_seed

fix_random_seed(123)

device = "cuda:0"
loader_kwargs = {"batch_size": 64, "num_workers": 12}
```

--------------------------------

### Example Out-of-Distribution Metrics Output

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/72639fd1157990859b80a45ebb3981bf/gram

This code snippet represents the typical output format for the OODMetrics after evaluating the detector. It shows key performance indicators like AUROC, AUTC, and FPR95TPR.

```text
# {'AUROC': 0.8175439834594727, 'AUTC': 0.4554872214794159, 'AUPR-IN': 0.8401336073875427, 'AUPR-OUT': 0.7695250511169434, 'FPR95TPR': 0.8087999820709229}
```

--------------------------------

### Install PyTorch-OOD in Editable Mode

Source: https://pytorch-ood.readthedocs.io/en/latest/info

Installs the PyTorch-OOD library in editable mode, allowing for direct modifications to the source code to take effect without needing to reinstall. This is commonly used during development to test changes.

```shell
git clone https://github.com/kkirchheim/pytorch-ood
cd pytorch-ood
pip install -e .

```

--------------------------------

### Import Libraries and Setup Parameters - Python

Source: https://pytorch-ood.readthedocs.io/en/latest/auto_examples/segmentation/street_vos

Imports necessary libraries such as numpy, PyTorch, and PyTorch-OOD utilities. It also defines key parameters for training like device, batch size, number of epochs, learning rate, and number of classes. Random seed is fixed for reproducibility.

```python
25 import numpy as np
26 import segmentation_models_pytorch as smp
27 import torch
28 from segmentation_models_pytorch.encoders import get_preprocessing_fn
29 from segmentation_models_pytorch.metrics import iou_score
30 from torch.utils.data import DataLoader
31 from torchvision.transforms.functional import pad, to_tensor
32
33 from pytorch_ood.dataset.img import StreetHazards
34 from pytorch_ood.detector import WeightedEBO
35 from pytorch_ood.loss import VOSRegLoss
36 from pytorch_ood.utils import OODMetrics, fix_random_seed
37
38 device = "cuda:0"
39 batch_size = 4
40 num_epochs = 1
41 lr = 0.0001
42 num_classes = 13
43
44 fix_random_seed(12345)
45 g = torch.Generator()
46 g.manual_seed(0)
```

--------------------------------

### Setup CIFAR10 Datasets and DataLoaders with Transformations

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/491e63114423d8b73855475222c8e1da/cifar_vos

Configures the CIFAR10 dataset for training and testing, including setup for in-distribution (ID) and out-of-distribution (OOD) test data. It applies specified transformations and uses a DataLoader for efficient batching during training, ensuring reproducibility with a manual seed.

```python
trans = WideResNet.transform_for("cifar10-pt")

dataset = CIFAR10(root="data", train=True, transform=trans, download=True)

# setup ID test data
dataset_in_test = CIFAR10(root="data", train=False, transform=trans)

# setup OOD test data, use ToUnknown() to mark labels as OOD
dataset_out_test = Textures(
    root="data", download=True, transform=trans, target_transform=ToUnknown()
)


loader = DataLoader(
    dataset,
    batch_size=batch_size,
    shuffle=True,
    num_workers=10,
    worker_init_fn=fix_random_seed,
    generator=g,
)
```

--------------------------------

### Setup PyTorch OOD Environment and Preprocessing

Source: https://pytorch-ood.readthedocs.io/en/latest/_downloads/60172d03ca513c12edf3cd7c57ceef90/cifar100_baseline

Initializes the PyTorch OOD environment, imports necessary libraries, and sets up preprocessing transformations and normalization parameters for the CIFAR 100 dataset. This code configures the device for computation and ensures reproducibility by fixing random seeds.

```python
import pandas as pd
import torch
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR100, CIFAR10, MNIST, FashionMNIST
from torch import nn

from pytorch_ood.dataset.img import (
    LSUNCrop,
    LSUNResize,
    Textures,
    TinyImageNetCrop,
    TinyImageNetResize,
    Places365,
)
from pytorch_ood.detector import (
    ODIN,
    EnergyBased,
    Entropy,
    KLMatching,
    Mahalanobis,
    MaxLogit,
    MaxSoftmax,
    ViM,
    RMD,
    DICE,
    SHE,
    Gram,
    MultiMahalanobis,
)
from pytorch_ood.model import WideResNet
from pytorch_ood.utils import OODMetrics, ToUnknown, fix_random_seed

device = "cuda:0"

fix_random_seed(123)

# setup preprocessing
trans = WideResNet.transform_for("cifar100-pt")
norm_std = WideResNet.norm_std_for("cifar100-pt")
```

--------------------------------

### CIFAR100_ODIN Benchmark Setup - Python

Source: https://pytorch-ood.readthedocs.io/en/latest/_modules/pytorch_ood/benchmark/img/cifar100

Initializes the CIFAR100 OOD benchmark replicating the ODIN paper's setup. It defines in-distribution datasets (CIFAR100 train/test) and a list of outlier datasets including TinyImageNet, LSUN, UniformNoise, and GaussianNoise. The 'evaluate' method uses a provided detector to compute OOD metrics on these datasets.

```python
from typing import Dict, List

from torch.utils.data import DataLoader, Dataset
from torchvision.datasets import CIFAR100
from torchvision.transforms import Compose

from pytorch_ood.api import Detector
from pytorch_ood.benchmark import Benchmark
from pytorch_ood.dataset.img import (
    GaussianNoise,
    LSUNCrop,
    LSUNResize,
    TinyImageNetCrop,
    TinyImageNetResize,
    UniformNoise,
)
from pytorch_ood.utils import OODMetrics, ToUnknown


class CIFAR100_ODIN(Benchmark):
    """
    Replicates the OOD detection benchmark from the ODIN paper for CIFAR 100.

    :see Paper: `ArXiv <https://arxiv.org/abs/1706.02690>`__

    Outlier datasets are

     * TinyImageNetCrop
     * TinyImageNetResize
     * LSUNResize
     * LSUNCrop
     * Uniform
     * Gaussian
    """

    def __init__(self, root, transform):
        """
        :param root: where to store datasets
        :param transform: transform to apply to images
        """
        self.transform = transform
        self.train_in = CIFAR100(root, download=True, transform=transform, train=True)
        self.test_in = CIFAR100(root, download=True, transform=transform, train=False)

        self.test_oods = [
            TinyImageNetCrop(
                root, download=True, transform=transform, target_transform=ToUnknown()
            ),
            TinyImageNetResize(
                root, download=True, transform=transform, target_transform=ToUnknown()
            ),
            LSUNResize(root, download=True, transform=transform, target_transform=ToUnknown()),
            LSUNCrop(root, download=True, transform=transform, target_transform=ToUnknown()),
            UniformNoise(
                1000,
                size=(32, 32, 3),
                transform=transform,
                target_transform=ToUnknown(),
            ),
            GaussianNoise(
                1000,
                size=(32, 32, 3),
                transform=transform,
                target_transform=ToUnknown(),
            ),
        ]

        self.ood_names: List[str] = []  #: OOD Dataset names
        self.ood_names = [
            "TinyImageNetCrop",
            "TinyImageNetResize",
            "LSUNResize",
            "LSUNCrop",
            "Uniform",
            "Gaussian",
        ]


    def train_set(self) -> Dataset:
        """
        Training dataset
        """
        return self.train_in


    def test_sets(self, known=True, unknown=True) -> List[Dataset]:
        """
        List of the different test datasets.
        If known and unknown are true, each dataset contains ID and OOD data.

        :param known: include ID
        :param unknown: include OOD
        """

        if known and unknown:
            return [self.test_in + other for other in self.test_oods]

        if known and not unknown:
            return [self.train_in]

        if not known and unknown:
            return self.ood_datasets

        raise ValueError()


    def evaluate(
        self, detector: Detector, loader_kwargs: Dict = None, device: str = "cpu"
    ) -> List[Dict]:
        """
        Evaluates the given detector on all datasets and returns a list with the results

        :param detector: the detector to evaluate
        :param loader_kwargs: keyword arguments to give to the data loader
        :param device: the device to move batches to
        """
        if loader_kwargs is None:
            loader_kwargs = {}

        metrics = []

        for name, dataset in zip(self.ood_names, self.test_sets()):
            loader = DataLoader(dataset=dataset, **loader_kwargs)

            m = OODMetrics()

            for x, y in loader:
                m.update(detector(x.to(device)), y)

            r = m.compute()
            r.update({"Dataset": name})

            metrics.append(r)

        return metrics

```

--------------------------------

### FishyScapes Dataset Initialization and Download (Python)

Source: https://pytorch-ood.readthedocs.io/en/latest/_modules/pytorch_ood/dataset/img/fishyscapes

Initializes the FishyScapes dataset, handling download and integrity checks. It requires the root directory for the dataset, the root directory for CityScapes, and the desired version. Optionally, transformations can be applied.

```python
def __init__(self, root, cs_root, version="3.0.0", download: bool = False, transforms=None):
        """

        :param root: dataset root
        :param cs_root: directory with cityscapes validation images
        :param version: can be one of ``1.0.0``, ``2.0.0``, ``3.0.0``
        :param download: whether to download the dataset
        :param transforms: transformations to apply to image and target mask
        """
        assert version in self.dataset_links.keys(), f"Unknown dataset version: '{version}'"

        self.root = root
        self.cs_root = cs_root
        self.transforms = transforms
        self.version = version
        self.dirname = f"fishyscapes-{version}"

        if download:
            self.download()

        if not self._check_integrity():
            raise RuntimeError(
                "Dataset not found or corrupted."
                " You can use download=True to download it"
            )

        self.files = [
            f for f in os.listdir(join(self.root, self.dirname)) if f.endswith("_rgb.npz")
        ]
```
Tokens:
Raw