### Install Dependencies

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Install Python dependencies from the requirements.txt file. Ensure you have Python 3.6 or later.

```bash
pip install -r requirements.txt
```

--------------------------------

### Train Segmentation Network (Python)

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Python code for training a ResNet38-based segmentation network, including model loading, optimizer setup, and training loop.

```python
# Equivalent Python usage
import torch
import importlib
from pathlib import Path

# Load segmentation network
model = getattr(
    importlib.import_module('network.resnet38_seg'),
    'Net'
)(num_classes=21)

# Initialize with classification weights
weights = torch.load('res38_cls.pth')
model.load_state_dict(weights, strict=False)

# Setup optimizer with different learning rates
optimizer = torch.optim.SGD([
    {'params': model.get_1x_lr_params(), 'lr': 0.0007},
    {'params': model.get_10x_lr_params(), 'lr': 0.007}  # 10x for new layers
], momentum=0.9, weight_decay=1e-5)

model = torch.nn.DataParallel(model).cuda()
model.train()

# Training loop
criterion = torch.nn.CrossEntropyLoss(ignore_index=255)
for epoch in range(30):
    for images, seg_labels in dataloader:
        pred = model(images.cuda())
        pred = torch.nn.functional.interpolate(pred, size=seg_labels.shape[1:], mode='bilinear')
        loss = criterion(pred, seg_labels.cuda())

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

    torch.save(model.module.state_dict(), f'seg_models/model_{epoch}.pth')
```

--------------------------------

### MCTformer Forward Pass for CAM Generation

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Performs a forward pass through the MCTformer model in inference mode to obtain class logits and attention maps for CAM generation. Specify `return_att=True` and `attention_type` to get localization cues.

```python
import torch
import torch.nn.functional as F
from timm.models import create_model
import models

model = create_model('deit_small_MCTformerPlus', num_classes=20, input_size=448)
model.cuda().eval()

# Prepare input image (batch_size=1, channels=3, height=448, width=448)
image = torch.randn(1, 3, 448, 448).cuda()

# Training mode: returns [cls_logits, all_cls_embeddings, patch_logits]
model.train()
outputs = model(image)
cls_logits = outputs[0]           # Shape: (1, 20)
cls_embeddings = outputs[1]       # Shape: (12, 1, 20, 384) - all layer embeddings
patch_logits = outputs[2]         # Shape: (1, 20)

# Inference mode with attention maps
model.eval()
with torch.no_grad():
    logits, cams, patch_attn = model(
        image,
        return_att=True,
        n_layers=3,                # Use last 3 layers for attention
        attention_type='fused'     # Options: 'fused', 'patchcam', 'mct'
    )

# cams: Class Activation Maps - Shape: (1, 20, 28, 28)
# patch_attn: Patch self-attention weights for refinement
print(f"Logits shape: {logits.shape}")        # (1, 20)
print(f"CAMs shape: {cams.shape}")            # (1, 20, 28, 28)
print(f"Patch attention shape: {patch_attn.shape}")  # (12, 1, 784, 784)

# Apply sigmoid to get class probabilities
probs = torch.sigmoid(logits)
predicted_classes = (probs > 0.5).squeeze()
print(f"Predicted classes: {predicted_classes.nonzero().squeeze().tolist()}")
```

--------------------------------

### Download and Extract PASCAL VOC 2012

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Download the PASCAL VOC 2012 development kit and extract it. This dataset is used for training and evaluation.

```bash
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
tar –xvf VOCtrainval_11-May-2012.tar
```

--------------------------------

### Run Main Training Script

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Run the main run.sh script for training MCTformer. This script also visualizes and evaluates generated localization maps.

```bash
bash run.sh
```

--------------------------------

### Train Affinity Network (CLI)

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Command-line interface for training the affinity network from PSA to refine CAMs into dense pseudo-labels using random walk.

```bash
python psa/train_aff.py \
    --weights res38_cls.pth \
    --voc12_root VOCdevkit/VOC2012 \
    --la_crf_dir cam-npy-crf_1/ \
    --ha_crf_dir cam-npy-crf_12/ \
    --batch_size 8 \
    --max_epoches 5 \
    --lr 0.01 \
    --crop_size 448
```

--------------------------------

### Download Augmented Annotations for PASCAL VOC 2012

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Download the augmented annotations for PASCAL VOC 2012 from the SBD dataset. These are required for training.

```bash
wget https://www.dropbox.com/s/oeu149j8qtbs1x0/SegmentationClassAug.zip?dl=0 -O SegmentationClassAug.zip
unzip SegmentationClassAug.zip
```

--------------------------------

### Create and Load MCTformer+ Model

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Instantiates the deit_small_MCTformerPlus model, loads pretrained DeiT weights, and adapts positional embeddings for a larger input size. Use this to set up the model for training or inference.

```python
import torch
from timm.models import create_model
import models  # registers MCTformer models

# Create MCTformer+ model for VOC (20 classes)
model = create_model(
    'deit_small_MCTformerPlus',
    pretrained=False,
    num_classes=20,
    drop_rate=0.0,
    drop_path_rate=0.1,
    input_size=448
)

# Load pretrained DeiT weights and adapt to MCTformer
checkpoint = torch.hub.load_state_dict_from_url(
    url="https://dl.fbaipublicfiles.com/deit/deit_small_patch16_224-cd65a155.pth",
    map_location="cpu", check_hash=True
)['model']

# Interpolate position embeddings for larger input size
pos_embed = checkpoint['pos_embed']
embedding_size = pos_embed.shape[-1]
num_extra_tokens = 1
orig_size = int((pos_embed.shape[-2] - num_extra_tokens) ** 0.5)

# Expand cls_token for multiple classes
cls_tokens = checkpoint['cls_token'].repeat(1, 20, 1)  # 20 classes
pos_embed_cls = pos_embed[:, :num_extra_tokens].repeat(1, 20, 1)

# Resize patch position embeddings for 448x448 input
pos_tokens = pos_embed[:, num_extra_tokens:]
pos_tokens = pos_tokens.reshape(-1, orig_size, orig_size, embedding_size).permute(0, 3, 1, 2)
pos_tokens = torch.nn.functional.interpolate(pos_tokens, scale_factor=2, mode='bicubic')
pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(1, 2)

checkpoint['cls_token'] = cls_tokens
checkpoint['pos_embed_cls'] = pos_embed_cls
checkpoint['pos_embed_pat'] = pos_tokens

model.load_state_dict(checkpoint, strict=False)
model.cuda()
print(f"Model parameters: {sum(p.numel() for p in model.parameters() if p.requires_grad):,}")
# Output: Model parameters: 21,674,260
```

--------------------------------

### Train and Test Segmentation Model

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Run the run_seg.sh script for training and testing the segmentation model. The model is initialized with pre-trained classification weights when training on VOC.

```bash
bash run_seg.sh
```

--------------------------------

### Train MCTformer+

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Execute the run_mct_plus.sh script to train the MCTformer+ model. This script handles the training process.

```bash
bash run_mct_plus.sh
```

--------------------------------

### Full MCTformer+ Pipeline for VOC2012

Source: https://context7.com/xulianuwa/mctformer/llms.txt

A bash script orchestrating the complete MCTformer+ pipeline for VOC2012, including training, CAM generation, and evaluation. Ensure all paths and parameters are correctly set for each step.

```bash
#!/bin/bash
# Complete MCTformer+ pipeline for VOC2012

# Step 1: Train MCTformer+
python main.py --data-path VOCdevkit/VOC2012 \
    --finetune https://dl.fbaipublicfiles.com/deit/deit_small_patch16_224-cd65a155.pth

# Step 2: Generate CAMs
python main.py --data-set VOC12MS \
    --img-list VOCdevkit/VOC2012/ImageSets/Segmentation \
    --data-path VOCdevkit/VOC2012 \
    --gen_attention_maps \
    --resume saved_model/checkpoint.pth

# Step 3: Evaluate CAMs
python evaluation.py --list voc12/train_id.txt \
    --gt_dir VOCdevkit/VOC2012/SegmentationClassAug \
    --logfile cam-npy/evallog.txt \
    --type npy \
    --curve True \
    --predict_dir cam-npy \
    --comment "MCTformer+ CAMs"
```

--------------------------------

### VOC12Dataset Initialization and Usage

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Initializes the VOC12Dataset for PASCAL VOC 2012 multi-label classification. Requires specifying paths to image lists, VOC root, and applying transformations.

```python
import torch
from torch.utils.data import DataLoader
from datasets import VOC12Dataset, build_transform
import argparse

args = argparse.Namespace(
    input_size=448,
    color_jitter=0.4,
    aa='rand-m9-mstd0.5-inc1',
    train_interpolation='bicubic',
    reprob=0.25,
    remode='pixel',
    recount=1,
    gen_attention_maps=False,
)

# Create training dataset with augmentations
transform = build_transform(is_train=True, args=args)
dataset = VOC12Dataset(
    img_name_list_path='voc12',              # Directory containing train_aug_id.txt
    voc12_root='VOCdevkit/VOC2012',
    train=True,
    transform=transform
)

print(f"Dataset size: {len(dataset)}")  # Output: Dataset size: 10582

# Get a sample
image, label = dataset[0]
print(f"Image shape: {image.shape}")    # Output: Image shape: torch.Size([3, 448, 448])
print(f"Label shape: {label.shape}")    # Output: Label shape: torch.Size([20])
print(f"Present classes: {label.nonzero().squeeze().tolist()}")  # e.g., [0, 14]

# Create dataloader
dataloader = DataLoader(dataset, batch_size=32, shuffle=True, num_workers=4)
for images, labels in dataloader:
    print(f"Batch images: {images.shape}")   # (32, 3, 448, 448)
    print(f"Batch labels: {labels.shape}")   # (32, 20)
    break
```

--------------------------------

### Train Segmentation Network (CLI)

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Command-line interface for training a ResNet38-based segmentation network using pseudo ground-truth masks.

```bash
python seg/train_seg.py \
    --network resnet38_seg \
    --num_epochs 30 \
    --seg_pgt_path pgt-psa-rw/ \
    --init_weights res38_cls.pth \
    --save_path seg_models/ \
    --list_path voc12/train_aug_id.txt \
    --img_path VOCdevkit/VOC2012/JPEGImages \
    --num_classes 21 \
    --batch_size 4 \
    --lr 0.0007 \
    --crop_size 321 \
    --gpu_ids 0
```

--------------------------------

### Post-process Seeds with PSA

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Run the run_psa.sh script to use PSA for post-processing seeds and generating pseudo ground-truth segmentation masks. Pre-trained classification weights are used for initialization.

```bash
bash run_psa.sh
```

--------------------------------

### Evaluate MCTformer with Search Thresholds

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Iterates through thresholds to find the best mIoU for MCTformer evaluation. Includes early stopping.

```python
for i in range(0, 60):
    threshold = i / 100.0
    results = do_python_eval(
        predict_folder='cam-npy/',
        gt_folder='VOCdevkit/VOC2012/SegmentationClassAug/',
        name_list=name_list,
        num_cls=21,
        input_type='npy',
        threshold=threshold,
        printlog=False
    )

    print(f"Threshold: {threshold:.2f}, mIoU: {results['mIoU']:.2f}%")

    if results['mIoU'] > best_miou:
        best_miou = results['mIoU']
        best_threshold = threshold
    else:
        break  # Early stopping

print(f"\nBest threshold: {best_threshold:.2f}, Best mIoU: {best_miou:.2f}%")

# Log results
writelog('evallog.txt',
         {'mIoU': best_miou, 'threshold': best_threshold},
         'MCTformer+ CAM evaluation')
```

--------------------------------

### Generate Attention Maps with Patch Attention Refinement

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Generates multi-scale CAMs using patch attention refinement. Requires model, dataset, scales, image list, data path, resume checkpoint, and specifies attention generation, type, layer index, patch attention refinement, visualization, and output directories.

```bash
# Generate multi-scale CAMs with patch attention refinement
python main.py \
    --model deit_small_MCTformerPlus \
    --data-set VOC12MS \
    --scales 1.0 0.75 1.25 \
    --img-list VOCdevkit/VOC2012/ImageSets/Segmentation \
    --data-path VOCdevkit/VOC2012 \
    --resume saved_model/checkpoint.pth \
    --gen_attention_maps \
    --attention-type fused \
    --layer-index 3 \
    --patch-attn-refine True \
    --visualize-cls-attn True \
    --attention-dir cam-png \
    --cam-npy-dir cam-npy
```

--------------------------------

### Train MCTformer+ on VOC2012

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Trains the MCTformer+ model on the VOC2012 dataset. Specify model, batch size, epochs, dataset, image list, data path, input size, output directory, finetune weights, learning rate, warmup epochs, loss weight, and number of CCTs.

```bash
# Train MCTformer+ on VOC2012
python main.py \
    --model deit_small_MCTformerPlus \
    --batch-size 32 \
    --epochs 45 \
    --data-set VOC12 \
    --img-list voc12 \
    --data-path VOCdevkit/VOC2012 \
    --input-size 448 \
    --output_dir saved_model \
    --finetune https://dl.fbaipublicfiles.com/deit/deit_small_patch16_224-cd65a155.pth \
    --lr 5e-4 \
    --warmup-epochs 5 \
    --loss-weight 1.0 \
    --num-cct 12
```

--------------------------------

### Load Model and Generate Attention Maps

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Loads a pre-trained MCTformer model and generates attention maps for a given dataset. Ensure the model checkpoint and dataset are correctly specified.

```python
model = create_model('deit_small_MCTformerPlus', num_classes=num_classes, input_size=448)
checkpoint = torch.load('saved_model/checkpoint.pth', map_location='cpu')
model.load_state_dict(checkpoint['model'])
model.cuda()

generate_attention_maps_ms(data_loader, model, torch.device('cuda'), args)
```

--------------------------------

### deit_small_MCTformerPlus Model Creation

Source: https://context7.com/xulianuwa/mctformer/llms.txt

This snippet demonstrates how to create an instance of the MCTformer+ model with a DeiT-small architecture, load pretrained weights, and adapt positional embeddings for a custom input size.

```APIDOC
## deit_small_MCTformerPlus Model Creation

### Description
Creates an MCTformer+ model instance using the DeiT-small architecture. It includes steps for loading pretrained DeiT weights and adapting positional embeddings for different input sizes.

### Method
```python
create_model(
    'deit_small_MCTformerPlus',
    pretrained=False,
    num_classes=20,
    drop_rate=0.0,
    drop_path_rate=0.1,
    input_size=448
)
```

### Parameters
- **model_name** (string) - Required - The name of the model architecture ('deit_small_MCTformerPlus').
- **pretrained** (bool) - Optional - Whether to load pretrained weights.
- **num_classes** (int) - Required - The number of classes for the segmentation task.
- **drop_rate** (float) - Optional - Dropout rate for the model.
- **drop_path_rate** (float) - Optional - Stochastic depth rate.
- **input_size** (int) - Optional - The input image size.

### Request Example
```python
import torch
from timm.models import create_model
import models  # registers MCTformer models

# Create MCTformer+ model for VOC (20 classes)
model = create_model(
    'deit_small_MCTformerPlus',
    pretrained=False,
    num_classes=20,
    drop_rate=0.0,
    drop_path_rate=0.1,
    input_size=448
)

# Load pretrained DeiT weights and adapt to MCTformer
checkpoint = torch.hub.load_state_dict_from_url(
    url="https://dl.fbaipublicfiles.com/deit/deit_small_patch16_224-cd65a155.pth",
    map_location="cpu", check_hash=True
)['model']

# Interpolate position embeddings for larger input size
pos_embed = checkpoint['pos_embed']
embedding_size = pos_embed.shape[-1]
num_extra_tokens = 1
orig_size = int((pos_embed.shape[-2] - num_extra_tokens) ** 0.5)

# Expand cls_token for multiple classes
cls_tokens = checkpoint['cls_token'].repeat(1, 20, 1)  # 20 classes
pos_embed_cls = pos_embed[:, :num_extra_tokens].repeat(1, 20, 1)

# Resize patch position embeddings for 448x448 input
pos_tokens = pos_embed[:, num_extra_tokens:]
pos_tokens = pos_tokens.reshape(-1, orig_size, orig_size, embedding_size).permute(0, 3, 1, 2)
pos_tokens = torch.nn.functional.interpolate(pos_tokens, scale_factor=2, mode='bicubic')
pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(1, 2)

checkpoint['cls_token'] = cls_tokens
checkpoint['pos_embed_cls'] = pos_embed_cls
checkpoint['pos_embed_pat'] = pos_tokens

model.load_state_dict(checkpoint, strict=False)
model.cuda()
print(f"Model parameters: {sum(p.numel() for p in model.parameters() if p.requires_grad):,}")
```

### Response
#### Success Response (200)
- **model** (torch.nn.Module) - The initialized MCTformer+ model.

#### Response Example
```
Model parameters: 21,674,260
```
```

--------------------------------

### Download MS COCO 2014 Dataset

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Download the training and validation sets for the MS COCO 2014 dataset. This dataset is used for training and evaluation.

```bash
wget http://images.cocodataset.org/zips/train2014.zip
wget http://images.cocodataset.org/zips/val2014.zip
```

--------------------------------

### Threshold Curve Search for CAM Pseudo-labels

Source: https://context7.com/xulianuwa/mctformer/llms.txt

This utility helps find the optimal background threshold for generating pseudo-labels from CAM predictions. It iterates through thresholds to maximize mIoU.

```python
from evaluation import do_python_eval, writelog
import pandas as pd

df = pd.read_csv('voc12/train_id.txt', names=['filename'])
name_list = df['filename'].values

best_miou = 0.0
best_threshold = 0.0

# The actual loop for threshold search is not provided in the source, 
# but this setup indicates the intention to use do_python_eval within such a loop.
```

--------------------------------

### COCOClsDataset Initialization

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Initializes the COCOClsDataset for MS COCO 2014 multi-label classification. Requires specifying paths to image lists, COCO root, and label files.

```python
from datasets import COCOClsDataset, build_transform
import argparse

args = argparse.Namespace(
    input_size=448,
    gen_attention_maps=False,
    color_jitter=0.4,
    aa='rand-m9-mstd0.5-inc1',
    train_interpolation='bicubic',
    reprob=0.25,
    remode='pixel',
    recount=1,
)

transform = build_transform(is_train=True, args=args)
dataset = COCOClsDataset(
    img_name_list_path='coco',               # Directory with train_id.txt
    coco_root='/path/to/COCO',               # Contains train2014/, val2014/
    label_file_path='coco_cls_labels.npy',   # Pre-computed labels
    train=True,
    transform=transform
)

print(f"COCO dataset size: {len(dataset)}")  # Output: COCO dataset size: 82081

image, label = dataset[0]
print(f"Image shape: {image.shape}")         # (3, 448, 448)
print(f"Label shape: {label.shape}")         # (80,) - 80 COCO classes
```

--------------------------------

### Train One Epoch with MCTformer

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Performs one epoch of training using multi-label soft margin loss, contrastive loss, and optional patch-level supervision. Supports mixed-precision training.

```python
import torch
from torch.utils.data import DataLoader
from timm.models import create_model
from timm.optim import create_optimizer
from timm.utils import NativeScaler
from datasets import build_dataset
from engine import train_one_epoch
import argparse
import models

# Setup training configuration
args = argparse.Namespace(
    model='deit_small_MCTformerPlus',
    data_set='VOC12',
    data_path='VOCdevkit/VOC2012',
    img_list='voc12',
    input_size=448,
    batch_size=32,
    lr=5e-4,
    weight_decay=0.05,
    opt='adamw',
    opt_eps=1e-8,
    opt_betas=None,
    momentum=0.9,
    clip_grad=None,
    loss_weight=1.0,
    num_cct=12,           # Number of contrastive class token layers
    num_workers=10,
    gen_attention_maps=False,
    color_jitter=0.4,
    aa='rand-m9-mstd0.5-inc1',
    smoothing=0.1,
    train_interpolation='bicubic',
    reprob=0.25,
    remode='pixel',
    recount=1,
)

# Build dataset and dataloader
dataset_train, num_classes = build_dataset(is_train=True, args=args)
args.nb_classes = num_classes  # 20 for VOC, 80 for COCO

data_loader = DataLoader(
    dataset_train,
    batch_size=args.batch_size,
    shuffle=True,
    num_workers=args.num_workers,
    pin_memory=True,
    drop_last=True
)

# Create model and optimizer
model = create_model(
    args.model,
    num_classes=num_classes,
    drop_rate=0.0,
    drop_path_rate=0.1,
    input_size=args.input_size
)
model.cuda()

optimizer = create_optimizer(args, model)
loss_scaler = NativeScaler()

# Train one epoch
train_stats = train_one_epoch(
    model=model,
    data_loader=data_loader,
    optimizer=optimizer,
    device=torch.device('cuda'),
    epoch=0,
    loss_scaler=loss_scaler,
    max_norm=args.clip_grad,
    args=args
)

print(f"Training stats: {train_stats}")
# Output: {'mct_loss': 0.234, 'attn_loss': 0.156, 'pat_loss': 0.189, 'loss': 0.579, 'lr': 0.0005}
```

--------------------------------

### Train MCTformer on MS COCO 2014

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Execute run_coco.sh to train MCTformer and generate class-specific localization maps on the MS COCO 2014 dataset. Download class label numpy file and trained model separately.

```bash
bash run_coco.sh
```

--------------------------------

### Infer Segmentation Predictions (CLI)

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Command-line interface for performing multi-scale inference with optional CRF post-processing for segmentation predictions.

```bash
python seg/infer_seg.py \
    --weights seg_models/model_29.pth \
    --network resnet38_seg \
    --list_path voc12/val_id.txt \
    --gt_path VOCdevkit/VOC2012/SegmentationClass \
    --img_path VOCdevkit/VOC2012/JPEGImages \
    --save_path val_results/ \
    --save_path_c val_results_color/ \
    --scales 0.5 0.75 1.0 1.25 1.5 \
    --use_crf True \
    --num_classes 21
```

--------------------------------

### Generate Multi-Scale Attention Maps

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Generates multi-scale class-specific localization maps using a trained MCTformer model. Supports horizontal flipping and patch attention refinement.

```python
import torch
from torch.utils.data import DataLoader
from timm.models import create_model
from datasets import build_dataset
from engine import generate_attention_maps_ms
import argparse
import models

args = argparse.Namespace(
    model='deit_small_MCTformerPlus',
    data_set='VOC12MS',
    data_path='VOCdevkit/VOC2012',
    img_list='VOCdevkit/VOC2012/ImageSets/Segmentation',
    input_size=448,
    num_workers=4,
    gen_attention_maps=True,
    scales=[1.0, 0.75, 1.25],     # Multi-scale inference
    patch_size=16,
    layer_index=3,                 # Use last 3 transformer layers
    attention_type='fused',        # 'fused', 'patchcam', or 'mct'
    patch_attn_refine=True,        # Refine with patch self-attention
    visualize_cls_attn=True,
    attention_dir='cam-png',       # Save visualization
    cam_npy_dir='cam-npy',         # Save numpy CAMs
)

# Build multi-scale dataset
dataset_train, num_classes = build_dataset(is_train=False, gen_attn=True, args=args)
args.nb_classes = num_classes

data_loader = DataLoader(
    dataset_train,
    batch_size=1,
    shuffle=False,
    num_workers=args.num_workers,
    pin_memory=True
)

```

--------------------------------

### Infer Affinity Network

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Applies a trained affinity network to propagate CAM seeds and generate pseudo ground-truth segmentation masks. Requires weights, an inference list, CAM directory, VOC root, and an output directory.

```bash
python psa/infer_aff.py \
    --weights resnet38_aff.pth \
    --infer_list voc12/train_id.txt \
    --cam_dir cam-npy/ \
    --voc12_root VOCdevkit/VOC2012 \
    --out_rw pgt-psa-rw/
```

--------------------------------

### Infer Segmentation Predictions (Python)

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Python code for multi-scale inference using a trained segmentation model, including normalization and softmax application.

```python
import torch
import torch.nn.functional as F
import numpy as np
import cv2
import importlib
from PIL import Image

# Load model
model = getattr(importlib.import_module('network.resnet38_seg'), 'Net')(num_classes=21)
model.load_state_dict(torch.load('seg_models/model_29.pth'))
model.eval().cuda()

# Multi-scale inference
scales = [0.5, 0.75, 1.0, 1.25, 1.5]
image = cv2.imread('image.jpg')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
H, W = image.shape[:2]

# Normalize
img_norm = image.astype(np.float32)
img_norm = (img_norm / 255. - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]
input_tensor = torch.from_numpy(img_norm.transpose(2, 0, 1)[None]).float().cuda()

probs = torch.zeros((1, 21, H, W)).cuda()
with torch.no_grad():
    for scale in scales:
        new_size = [int(H * scale), int(W * scale)]
        scaled = F.interpolate(input_tensor, new_size, mode='bilinear', align_corners=True)
        prob = model(scaled)
        prob = F.interpolate(prob, (H, W), mode='bilinear', align_corners=False)
        prob = F.softmax(prob, dim=1)
        probs = torch.max(probs, prob)

prediction = probs.argmax(dim=1).squeeze().cpu().numpy()
print(f"Prediction shape: {prediction.shape}")  # (H, W)
print(f"Unique classes: {np.unique(prediction)}")  # e.g., [0, 15, 20]
```

--------------------------------

### Evaluate MCTformer Model Performance

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Evaluates the trained MCTformer model on a validation set, computing mean Average Precision (mAP) for multi-label classification.

```python
import torch
from torch.utils.data import DataLoader
from timm.models import create_model
from datasets import build_dataset
from engine import evaluate
import argparse
import models

args = argparse.Namespace(
    model='deit_small_MCTformerPlus',
    data_set='VOC12',
    data_path='VOCdevkit/VOC2012',
    img_list='voc12',
    input_size=448,
    batch_size=48,
    num_workers=10,
    gen_attention_maps=False,
)

# Build validation dataset
dataset_val, num_classes = build_dataset(is_train=False, args=args)
data_loader_val = DataLoader(
    dataset_val,
    batch_size=args.batch_size,
    shuffle=False,
    num_workers=args.num_workers,
    pin_memory=True
)

# Load trained model
model = create_model('deit_small_MCTformerPlus', num_classes=num_classes, input_size=448)
checkpoint = torch.load('saved_model/checkpoint.pth', map_location='cpu')
model.load_state_dict(checkpoint['model'])
model.cuda()

# Evaluate
test_stats = evaluate(data_loader_val, model, torch.device('cuda'))
print(f"Validation mAP: {test_stats['mAP']*100:.1f}%")
# Output: * mAP 0.912 loss 0.156
# Output: Validation mAP: 91.2%
```

--------------------------------

### MCTformerPlus Forward Pass

Source: https://context7.com/xulianuwa/mctformer/llms.txt

This snippet details the forward pass of the MCTformerPlus model, showing how to obtain class logits and attention maps for Class Activation Maps (CAM) generation.

```APIDOC
## MCTformerPlus.forward

### Description
Performs the forward pass of the MCTformerPlus model. It can return class logits and attention maps for CAM generation. Supports different attention types for localization cues.

### Method
```python
model(image, return_att=True, n_layers=3, attention_type='fused')
```

### Parameters
- **image** (torch.Tensor) - Required - Input image tensor.
- **return_att** (bool) - Optional - Whether to return attention maps.
- **n_layers** (int) - Optional - Number of transformer layers to use for attention maps.
- **attention_type** (str) - Optional - Type of attention to compute. Options: 'fused', 'patchcam', 'mct'. Defaults to 'fused'.

### Request Example
```python
import torch
import torch.nn.functional as F
from timm.models import create_model
import models

model = create_model('deit_small_MCTformerPlus', num_classes=20, input_size=448)
model.cuda().eval()

# Prepare input image (batch_size=1, channels=3, height=448, width=448)
image = torch.randn(1, 3, 448, 448).cuda()

# Training mode: returns [cls_logits, all_cls_embeddings, patch_logits]
model.train()
outputs = model(image)
cls_logits = outputs[0]           # Shape: (1, 20)
cls_embeddings = outputs[1]       # Shape: (12, 1, 20, 384) - all layer embeddings
patch_logits = outputs[2]         # Shape: (1, 20)

# Inference mode with attention maps
model.eval()
with torch.no_grad():
    logits, cams, patch_attn = model(
        image,
        return_att=True,
        n_layers=3,                # Use last 3 layers for attention
        attention_type='fused'     # Options: 'fused', 'patchcam', 'mct'
    )

# cams: Class Activation Maps - Shape: (1, 20, 28, 28)
# patch_attn: Patch self-attention weights for refinement
print(f"Logits shape: {logits.shape}")        # (1, 20)
print(f"CAMs shape: {cams.shape}")            # (1, 20, 28, 28)
print(f"Patch attention shape: {patch_attn.shape}")  # (12, 1, 784, 784)

# Apply sigmoid to get class probabilities
probs = torch.sigmoid(logits)
predicted_classes = (probs > 0.5).squeeze()
print(f"Predicted classes: {predicted_classes.nonzero().squeeze().tolist()}")
```

### Response
#### Success Response (200)
- **logits** (torch.Tensor) - Class logits for each class. Shape: (batch_size, num_classes).
- **cams** (torch.Tensor) - Class Activation Maps. Shape: (batch_size, num_classes, height, width).
- **patch_attn** (torch.Tensor) - Patch self-attention weights. Shape: (num_layers, batch_size, num_patches, num_patches).

#### Response Example
```
Logits shape: torch.Size([1, 20])
CAMs shape: torch.Size([1, 20, 28, 28])
Patch attention shape: torch.Size([12, 1, 784, 784])
Predicted classes: [3, 7, 15]
```
```

--------------------------------

### Evaluate Segmentation Masks with do_python_eval

Source: https://context7.com/xulianuwa/mctformer/llms.txt

Computes mean Intersection over Union (mIoU) for predicted segmentation masks against ground truth. Supports PNG or NPY input types and multi-processing.

```python
from evaluation import do_python_eval
import pandas as pd

# Load image list
df = pd.read_csv('voc12/train_id.txt', names=['filename'])
name_list = df['filename'].values

# Evaluate PNG predictions
results = do_python_eval(
    predict_folder='predictions/',           # Directory with predicted masks
    gt_folder='VOCdevkit/VOC2012/SegmentationClassAug/',
    name_list=name_list,
    num_cls=21,                              # 20 classes + background
    input_type='png',
    printlog=True
)
# Output:
# background: 88.234%    aeroplane: 76.543%
# bicycle:    45.678%    bird:      82.345%
# ...
# ======================================================
# mIoU:       67.234%

print(f"mIoU: {results['mIoU']:.2f}%")

# Evaluate numpy CAM predictions with threshold search
results = do_python_eval(
    predict_folder='cam-npy/',
    gt_folder='VOCdevkit/VOC2012/SegmentationClassAug/',
    name_list=name_list,
    num_cls=21,
    input_type='npy',
    threshold=0.35,                          # Background threshold
    printlog=True
)
```

--------------------------------

### MCTformer Citation

Source: https://github.com/xulianuwa/mctformer/blob/main/README.md

Citation details for the MCTformer paper. Use this when referencing the model in your research.

```bibtex
@inproceedings{xu2022multi,
  title={Multi-class Token Transformer for Weakly Supervised Semantic Segmentation},
  author={Xu, Lian and Ouyang, Wanli and Bennamoun, Mohammed and Boussaid, Farid and Xu, Dan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={4310--4319},
  year={2022}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.