### Full Environment Setup Script

Source: https://github.com/nvlabs/segformer/blob/master/docs/get_started.md

Comprehensive scripts for setting up the environment from scratch, including symlinking or linking datasets on Linux and Windows.

```shell
# Linux setup
mkdir data
ln -s $DATA_ROOT data
```

```shell
# Windows setup
set PATH=full\path\to\your\cpp\compiler;%PATH%
mklink /D data %DATA_ROOT%
```

--------------------------------

### SegFormer Project Installation and Setup (Bash)

Source: https://context7.com/nvlabs/segformer/llms.txt

Provides bash commands for setting up the SegFormer project environment. This includes creating a conda environment, installing PyTorch with CUDA support, and installing necessary Python dependencies and the MMSegmentation library.

```bash
# Create conda environment
conda create -n segformer python=3.8 -y
conda activate segformer

# Install PyTorch (CUDA 10.1 example)
pip install torch==1.7.1 torchvision==0.8.2

# Install dependencies
pip install timm==0.3.2
pip install mmcv-full==1.2.7
pip install opencv-python==4.5.1.48

# Install MMSegmentation (SegFormer)
cd SegFormer
pip install -e . --user

# Verify installation
python -c "import mmseg; print(mmseg.__version__)"
python -c "import mmcv; print(mmcv.__version__)"
```

--------------------------------

### Verify environment installation

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Imports PyTorch, Torchvision, and MMSegmentation to verify successful installation. It outputs the current versions and the availability of CUDA GPU acceleration.

```python
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())

import mmseg
print(mmseg.__version__)
```

--------------------------------

### Verify MMSegmentation Installation via Python API

Source: https://github.com/nvlabs/segformer/blob/master/docs/get_started.md

Initializes a SegFormer model using a configuration and checkpoint file, then performs inference on static images or video streams. This script confirms that dependencies are met and the model architecture is functional.

```python
from mmseg.apis import inference_segmentor, init_segmentor
import mmcv

config_file = 'configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py'
checkpoint_file = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'

# build the model from a config file and a checkpoint file
model = init_segmentor(config_file, checkpoint_file, device='cuda:0')

# test a single image and show the results
img = 'test.jpg'
result = inference_segmentor(model, img)
model.show_result(img, result, show=True)
model.show_result(img, result, out_file='result.jpg')

# test a video and show the results
video = mmcv.VideoReader('video.mp4')
for frame in video:
   result = inference_segmentor(model, frame)
   model.show_result(frame, result, wait_time=1)
```

--------------------------------

### Install PyTorch and Dependencies

Source: https://github.com/nvlabs/segformer/blob/master/docs/get_started.md

Installs the required PyTorch and torchvision libraries. The specific versions can be adjusted based on the required CUDA compatibility.

```shell
conda install pytorch=1.6.0 torchvision cudatoolkit=10.1 -c pytorch
```

--------------------------------

### Install MMCV and MMSegmentation

Source: https://github.com/nvlabs/segformer/blob/master/docs/get_started.md

Installs the MMCV framework required for CUDA operations and the core MMSegmentation library. Options include installing from PyPI or cloning the repository for development mode.

```shell
pip install mmcv-full==latest+torch1.5.0+cu101 -f https://download.openmmlab.com/mmcv/dist/index.html
pip install mmsegmentation
git clone https://github.com/open-mmlab/mmsegmentation.git
cd mmsegmentation
pip install -e .
```

--------------------------------

### Prepare environment and download model checkpoints

Source: https://github.com/nvlabs/segformer/blob/master/demo/inference_demo.ipynb

Sets up the local directory structure and downloads a pre-trained PSPNet model weight file from remote cloud storage using system commands.

```bash
!mkdir ../checkpoints
!wget https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmsegmentation/models/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth -P ../checkpoints
```

--------------------------------

### SegFormer B1 ADE20K Configuration File Example (Python)

Source: https://context7.com/nvlabs/segformer/llms.txt

An example Python configuration file for training SegFormer B1 on the ADE20K dataset. It details model architecture, optimizer settings, learning rate schedule, data configuration, and evaluation intervals, inheriting from base configurations.

```python
# local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py

# Inherit base configurations
_base_ = [
    '../../_base_/models/segformer.py',
    '../../_base_/datasets/ade20k_repeat.py',
    '../../_base_/default_runtime.py',
    '../../_base_/schedules/schedule_160k_adamw.py'
]

# Model settings
norm_cfg = dict(type='SyncBN', requires_grad=True)
find_unused_parameters = True

model = dict(
    type='EncoderDecoder',
    pretrained='pretrained/mit_b1.pth',
    backbone=dict(
        type='mit_b1',
        style='pytorch'
    ),
    decode_head=dict(
        type='SegFormerHead',
        in_channels=[64, 128, 320, 512],
        in_index=[0, 1, 2, 3],
        feature_strides=[4, 8, 16, 32],
        channels=128,
        dropout_ratio=0.1,
        num_classes=150,
        norm_cfg=norm_cfg,
        align_corners=False,
        decoder_params=dict(embed_dim=256),
        loss_decode=dict(type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)
    ),
    train_cfg=dict(),
    test_cfg=dict(mode='whole')
)

# Optimizer configuration
optimizer = dict(
    _delete_=True,
    type='AdamW',
    lr=0.00006,
    betas=(0.9, 0.999),
    weight_decay=0.01,
    paramwise_cfg=dict(
        custom_keys={
            'pos_block': dict(decay_mult=0.),
            'norm': dict(decay_mult=0.),
            'head': dict(lr_mult=10.)
        }
    )
)

# Learning rate configuration
lr_config = dict(
    _delete_=True,
    policy='poly',
    warmup='linear',
    warmup_iters=1500,
    warmup_ratio=1e-6,
    power=1.0,
    min_lr=0.0,
    by_epoch=False
)

# Data configuration
data = dict(samples_per_gpu=2)

# Evaluation configuration
evaluation = dict(interval=16000, metric='mIoU')
```

--------------------------------

### Initialize and Run Inference with SegFormer

Source: https://context7.com/nvlabs/segformer/llms.txt

Demonstrates loading a pre-trained SegFormer model from configuration and checkpoint files. Includes examples for performing inference on images and accessing the resulting segmentation mask.

```python
from mmseg.apis import init_segmentor, inference_segmentor, show_result_pyplot
from mmseg.core.evaluation import get_palette
import mmcv
import numpy as np

config_file = 'local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py'
checkpoint_file = 'pretrained/segformer_b1_ade20k_160k.pth'
device = 'cuda:0'

model = init_segmentor(config_file, checkpoint_file, device=device)

img_path = 'demo/demo.png'
result = inference_segmentor(model, img_path)

palette = get_palette('cityscapes')
show_result_pyplot(model, img_path, result, palette=palette, fig_size=(15, 10))

img = mmcv.imread(img_path)
result = inference_segmentor(model, img)

seg_mask = result[0]
print(f"Segmentation mask shape: {seg_mask.shape}")
print(f"Unique classes detected: {np.unique(seg_mask)}")
```

--------------------------------

### Clone and install MMSegmentation

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Removes any existing repository directory, clones the latest MMSegmentation codebase, and installs it in editable mode. This setup allows for local modification and execution of the segmentation library.

```bash
!rm -rf mmsegmentation
!git clone https://github.com/open-mmlab/mmsegmentation.git 
%cd mmsegmentation
!pip install -e .
```

--------------------------------

### Display Training Configuration

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Prints the pretty-formatted configuration object to the console. This is essential for debugging model parameters and pipeline transformations before starting a training run.

```python
print(f'Config:\n{cfg.pretty_text}')
```

--------------------------------

### Run Inference Demo via CLI

Source: https://github.com/nvlabs/segformer/blob/master/docs/get_started.md

Executes image segmentation inference using the provided demo script. Requires an input image, a configuration file, and a model checkpoint, with optional flags for device selection and palette thresholds.

```shell
python demo/image_demo.py ${IMAGE_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${DEVICE_NAME}] [--palette-thr ${PALETTE}]

# Example execution:
python demo/image_demo.py demo/demo.jpg configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py \
    checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth --device cuda:0 --palette cityscapes
```

--------------------------------

### Visualize Segmentation Results on Custom Images (Bash)

Source: https://context7.com/nvlabs/segformer/llms.txt

Provides bash commands to run the image visualization demo script. It shows examples for basic visualization, using a specific palette like ADE20K, and running inference on the CPU.

```bash
# Basic visualization
python demo/image_demo.py \
    demo/demo.png \
    local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py \
    work_dirs/segformer_b1/iter_160000.pth \
    --device cuda:0 \
    --palette cityscapes

# Visualize with ADE20K palette
python demo/image_demo.py \
    path/to/your/image.jpg \
    local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py \
    pretrained/segformer_b1_ade20k_160k.pth \
    --device cuda:0 \
    --palette ade20k

# Use CPU for inference
python demo/image_demo.py \
    demo/demo.png \
    local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py \
    work_dirs/segformer_b1/iter_160000.pth \
    --device cpu \
    --palette cityscapes
```

--------------------------------

### Initialize and execute segmentation inference

Source: https://github.com/nvlabs/segformer/blob/master/demo/inference_demo.ipynb

Imports necessary segmentation APIs, initializes the model with specific configuration and checkpoint paths, performs inference on a target image, and displays the visualized segmentation result.

```python
from mmseg.apis import init_segmentor, inference_segmentor, show_result_pyplot
from mmseg.core.evaluation import get_palette

config_file = '../configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py'
checkpoint_file = '../checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'

# build the model from a config file and a checkpoint file
model = init_segmentor(config_file, checkpoint_file, device='cuda:0')

# test a single image
img = 'demo.png'
result = inference_segmentor(model, img)

# show the results
show_result_pyplot(model, img, result, get_palette('cityscapes'))
```

--------------------------------

### Dataset Folder Structure for Reorganization

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/customize_datasets.md

This example illustrates the required folder structure for customizing datasets by reorganizing data. It shows how to place image directories and annotation directories, with subfolders for training and validation splits.

```none
├── data
│   ├── my_dataset
│   │   ├── img_dir
│   │   │   ├── train
│   │   │   │   ├── xxx{img_suffix}
│   │   │   │   ├── yyy{img_suffix}
│   │   │   │   ├── zzz{img_suffix}
│   │   │   ├── val
│   │   ├── ann_dir
│   │   │   ├── train
│   │   │   │   ├── xxx{seg_map_suffix}
│   │   │   │   ├── yyy{seg_map_suffix}
│   │   │   │   ├── zzz{seg_map_suffix}
│   │   │   ├── val
```

--------------------------------

### Install SegFormer dependencies

Source: https://github.com/nvlabs/segformer/blob/master/README.md

Shell commands to install the required Python packages and the local SegFormer package. This setup includes torchvision, timm, mmcv, and OpenCV.

```bash
pip install torchvision==0.8.2
pip install timm==0.3.2
pip install mmcv-full==1.2.7
pip install opencv-python==4.5.1.48
cd SegFormer && pip install -e . --user
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/nvlabs/segformer/blob/master/docs/get_started.md

Initializes a dedicated Python environment for SegFormer to manage package dependencies cleanly. This is the first step in the installation process for both platforms.

```shell
conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab
```

--------------------------------

### Using Split Files for Dataset Customization

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/customize_datasets.md

This example shows how to use text files to specify which image-annotation pairs should be included in a dataset split. Only files with names listed in the split file will be loaded.

```none
xxx
zzz
```

--------------------------------

### Register and Use Custom Pipeline (Python)

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/data_pipeline.md

Demonstrates how to define a custom data transformation pipeline and register it with MMCV for use in configuration files. The example shows adding a dummy key to the results dictionary.

```python
from mmseg.datasets import PIPELINES

@PIPELINES.register_module()
class MyTransform:

    def __call__(self, results):
        results['dummy'] = True
        return results
```

--------------------------------

### Complex Dataset Mixing Example

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/customize_datasets.md

This comprehensive Python configuration example shows how to mix datasets by repeating Dataset_A N times and Dataset_B M times, and then concatenating these repeated datasets for training. It also includes configurations for validation and testing sets.

```python
dataset_A_train = dict(
    type='RepeatDataset',
    times=N,
    dataset=dict(
        type='Dataset_A',
        ...
        pipeline=train_pipeline
    )
)
dataset_A_val = dict(
    ...
    pipeline=test_pipeline
)
dataset_A_test = dict(
    ...
    pipeline=test_pipeline
)
dataset_B_train = dict(
    type='RepeatDataset',
    times=M,
    dataset=dict(
        type='Dataset_B',
        ...
        pipeline=train_pipeline
    )
)
data = dict(
    imgs_per_gpu=2,
    workers_per_gpu=2,
    train = [
        dataset_A_train,
        dataset_B_train
    ],
    val = dataset_A_val,
    test = dataset_A_test
)
```

--------------------------------

### Train SegFormer Models

Source: https://context7.com/nvlabs/segformer/llms.txt

Provides methods for training SegFormer using Python scripts for configuration-based setup or command-line interface tools for single and distributed multi-GPU training.

```python
from mmseg.apis import set_random_seed, train_segmentor
from mmseg.datasets import build_dataset
from mmseg.models import build_segmentor
from mmcv.utils import Config

cfg = Config.fromfile('local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py')
set_random_seed(0, deterministic=True)

model = build_segmentor(cfg.model, train_cfg=cfg.get('train_cfg'), test_cfg=cfg.get('test_cfg'))
datasets = [build_dataset(cfg.data.train)]
model.CLASSES = datasets[0].CLASSES

train_segmentor(model, datasets, cfg, distributed=False, validate=True, timestamp='20240101_120000')
```

```bash
# Single-GPU training
python tools/train.py local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py

# Multi-GPU distributed training
./tools/dist_train.sh local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py 4

# Training with custom options
python tools/train.py local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py --work-dir work_dirs/segformer_b1_custom --load-from pretrained/mit_b1.pth
```

--------------------------------

### SegFormer Configuration Schema

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

An example dictionary structure representing a complete training configuration for an Encoder-Decoder model, including backbone, head, and data loading pipeline definitions.

```python
model = dict(
    type='EncoderDecoder',
    pretrained='open-mmlab://resnet50_v1c',
    backbone=dict(
        type='ResNetV1c',
        depth=50,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        norm_cfg=dict(type='BN', requires_grad=True)
    ),
    decode_head=dict(
        type='PSPHead',
        num_classes=8
    )
)
data = dict(
    samples_per_gpu=8,
    train=dict(type='StandfordBackgroundDataset')
)
```

--------------------------------

### Enable Online Hard Example Mining (OHEM) for PSPNet (Python)

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/training_tricks.md

This configuration demonstrates how to enable Online Hard Example Mining (OHEM) for training a PSPNet model in MMSegmentation. OHEM focuses training on pixels with low confidence scores (below a specified threshold) or high loss, improving performance on challenging examples. It requires specifying a `type`, `thresh`, and `min_kept` for the pixel sampler.

```python
_base_ = './pspnet_r50-d8_512x1024_40k_cityscapes.py'
model=dict(
    decode_head=dict(
        sampler=dict(type='OHEMPixelSampler', thresh=0.7, min_kept=100000)) )
```

--------------------------------

### Define a Custom Optimizer Constructor

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/customize_runtime.md

Provides an example of defining a custom optimizer constructor, `MyOptimizerConstructor`, which allows for parameter-specific optimization settings, such as different weight decay for BatchNorm layers.

```python
from mmcv.utils import build_from_cfg

from mmcv.runner.optimizer import OPTIMIZER_BUILDERS, OPTIMIZERS
from mmseg.utils import get_root_logger
from .my_optimizer import MyOptimizer


@OPTIMIZER_BUILDERS.register_module()
class MyOptimizerConstructor(object):

    def __init__(self, optimizer_cfg, paramwise_cfg=None):

    def __call__(self, model):

        return my_optimizer
```

--------------------------------

### Install MMCV dependency

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Installs the required MMCV-full package using pip from the OpenMMLab distribution index. This ensures compatibility with the specific PyTorch and CUDA versions used in the project.

```bash
!pip install mmcv-full==latest+torch1.5.0+cu101 -f https://download.openmmlab.com/mmcv/dist/index.html
```

--------------------------------

### Concatenating Annotation Directories and Split Files Simultaneously

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/customize_datasets.md

This Python configuration example shows how to simultaneously use multiple annotation directories and split files. Each split file corresponds to a specific annotation directory, allowing for complex dataset subsetting.

```python
dataset_A_train = dict(
          type='Dataset_A',
          img_dir = 'img_dir',
          ann_dir = ['anno_dir_1', 'anno_dir_2'],
          split = ['split_1.txt', 'split_2.txt'],
          pipeline=train_pipeline
      )
```

--------------------------------

### Define configuration inheritance

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Demonstrates how to inherit settings from a base model configuration file. This allows developers to reuse complex architectures while overriding only the necessary fields.

```python
_base_ = "../deeplabv3/deeplabv3_r50_512x1024_40ki_cityscapes.py"

# Modification of fields follows here
```

--------------------------------

### Run Inference with MMSegmentation Models

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Sets up the environment by downloading a pretrained model, initializing the segmentor, and performing inference on a sample image. Requires the mmsegmentation library.

```python
!mkdir checkpoints
!wget https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmsegmentation/models/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth -P checkpoints

from mmseg.apis import inference_segmentor, init_segmentor, show_result_pyplot
from mmseg.core.evaluation import get_palette

config_file = 'configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py'
checkpoint_file = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'

# build the model from a config file and a checkpoint file
model = init_segmentor(config_file, checkpoint_file, device='cuda:0')

# test a single image
img = 'demo/demo.png'
result = inference_segmentor(model, img)

# show the results
show_result_pyplot(model, img, result, get_palette('cityscapes'))
```

--------------------------------

### Inspect configuration via CLI

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Uses the provided Python script to print the merged configuration settings to the terminal. Supports dynamic overrides using the --options flag to modify specific parameters during runtime.

```bash
python tools/print_config.py /PATH/TO/CONFIG
python tools/print_config.py /PATH/TO/CONFIG --options xxx.yyy=zzz
```

--------------------------------

### Configure PYTHONPATH for MMSegmentation

Source: https://github.com/nvlabs/segformer/blob/master/docs/get_started.md

Adjusts the shell environment variable to prioritize the local MMSegmentation directory. Removing this line in training and testing scripts allows the system to default to the globally installed environment version.

```shell
PYTHONPATH="$(dirname $0)/..":$PYTHONPATH
```

--------------------------------

### PSPNet Training and Testing Pipelines (Python)

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/data_pipeline.md

Defines the data preparation pipelines for training and testing a PSPNet model. Includes operations like image loading, annotation loading, resizing, cropping, flipping, photometric distortion, normalization, padding, formatting, and collection.

```python
img_norm_cfg = dict(
    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
crop_size = (512, 1024)
train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations'),
    dict(type='Resize', img_scale=(2048, 1024), ratio_range=(0.5, 2.0)),
    dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.75),
    dict(type='RandomFlip', flip_ratio=0.5),
    dict(type='PhotoMetricDistortion'),
    dict(type='Normalize', **img_norm_cfg),
    dict(type='Pad', size=crop_size, pad_val=0, seg_pad_val=255),
    dict(type='DefaultFormatBundle'),
    dict(type='Collect', keys=['img', 'gt_semantic_seg']),
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(2048, 1024),
        # img_ratios=[0.5, 0.75, 1.0, 1.25, 1.5, 1.75],
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=True),
            dict(type='RandomFlip'),
            dict(type='Normalize', **img_norm_cfg),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img']),
        ])
]
```

--------------------------------

### Initialize and execute SegFormer model training

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

This snippet creates the necessary working directory using MMCV utilities and launches the training process. It utilizes the train_segmentor function, which requires a pre-defined model, dataset configuration, and optional training parameters.

```python
import mmcv
import os.path as osp

# Create work_dir
mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))

# Run the segmentation training process
train_segmentor(model, datasets, cfg, distributed=False, validate=True, meta=dict())
```

--------------------------------

### Build Dataloaders for Training and Testing (Python)

Source: https://context7.com/nvlabs/segformer/llms.txt

Demonstrates how to build dataloaders for both training and testing datasets. For training, it uses specified batch sizes, GPU workers, and distribution settings. For testing, it defines a custom dataset configuration with a specific image loading and augmentation pipeline before building the dataloader.

```python
train_loader = build_dataloader(
    train_dataset,
    samples_per_gpu=2,
    workers_per_gpu=4,
    num_gpus=1,
    dist=False,
    seed=0,
    drop_last=True
)

test_dataset_config = dict(
    type='ADE20KDataset',
    data_root='data/ade/ADEChallengeData2016',
    img_dir='images/validation',
    ann_dir='annotations/validation',
    test_mode=True,
    pipeline=[
        dict(type='LoadImageFromFile'),
        dict(
            type='MultiScaleFlipAug',
            img_scale=(2048, 512),
            flip=False,
            transforms=[
                dict(type='Resize', keep_ratio=True),
                dict(type='RandomFlip'),
                dict(type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True),
                dict(type='ImageToTensor', keys=['img']),
                dict(type='Collect', keys=['img'])
            ]
        )
    ]
)

test_dataset = build_dataset(test_dataset_config)
test_loader = build_dataloader(
    test_dataset,
    samples_per_gpu=1,
    workers_per_gpu=4,
    dist=False,
    shuffle=False
)
```

--------------------------------

### Modifying Training and Testing Pipelines (Python)

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Shows how to define and utilize intermediate variables like `train_pipeline` and `test_pipeline` in configuration files. Users must redefine these variables and then pass them into the `data` dictionary to modify the data loading and augmentation strategies for training and testing.

```python
_base_ = '../pspnet/psp_r50_512x1024_40ki_cityscapes.py'
crop_size = (512, 1024)
img_norm_cfg = dict(
    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations'),
    dict(type='Resize', img_scale=(2048, 1024), ratio_range=(1.0, 2.0)),  # change to [1., 2.]
    dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.75),
    dict(type='RandomFlip', flip_ratio=0.5),
    dict(type='PhotoMetricDistortion'),
    dict(type='Normalize', **img_norm_cfg),
    dict(type='Pad', size=crop_size, pad_val=0, seg_pad_val=255),
    dict(type='DefaultFormatBundle'),
    dict(type='Collect', keys=['img', 'gt_semantic_seg']),
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(2048, 1024),
        img_ratios=[0.5, 0.75, 1.0, 1.25, 1.5, 1.75],  # change to multi scale testing
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=True),
            dict(type='RandomFlip'),
            dict(type='Normalize', **img_norm_cfg),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img']),
        ])
]
data = dict(
    train=dict(pipeline=train_pipeline),
    val=dict(pipeline=test_pipeline),
    test=dict(pipeline=test_pipeline))

```

--------------------------------

### Evaluate SegFormer models

Source: https://github.com/nvlabs/segformer/blob/master/README.md

Commands to evaluate trained models using single or multi-GPU setups. These scripts support standard testing as well as multi-scale augmentation testing.

```bash
# Single-gpu testing
python tools/test.py local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py /path/to/checkpoint_file

# Multi-gpu testing
./tools/dist_test.sh local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py /path/to/checkpoint_file <GPU_NUM>

# Multi-gpu, multi-scale testing
tools/dist_test.sh local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py /path/to/checkpoint_file <GPU_NUM> --aug-test
```

--------------------------------

### Specify Pre-trained Model and Working Directory

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Sets the path to a pre-trained model checkpoint to be loaded and defines the working directory where training logs and saved files will be stored.

```python
# We can still use the pre-trained Mask RCNN model though we do not need to
# use the mask branch
cfg.load_from = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'

# Set up working dir to save files and logs.
cfg.work_dir = './work_dirs/tutorial'
```

--------------------------------

### Train Model via Command Line Interface

Source: https://github.com/nvlabs/segformer/blob/master/docs/train.md

Commands to initiate model training across different distributed environments including single GPU, multi-GPU, and Slurm-managed clusters.

```shell
python tools/train.py ${CONFIG_FILE} [optional arguments]
```

```shell
./tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments]
```

```shell
[GPUS=${GPUS}] ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} ${CONFIG_FILE} --work-dir ${WORK_DIR}
```

```shell
GPUS=16 ./tools/slurm_train.sh dev pspr50 configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py /nfs/xxxx/psp_r50_512x1024_40ki_cityscapes
```

--------------------------------

### Import Custom Optimizer via __init__.py

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/customize_runtime.md

Illustrates how to make a custom optimizer discoverable by importing it within the `mmseg/core/optimizer/__init__.py` file. This ensures the optimizer is registered.

```python
from .my_optimizer import MyOptimizer
```

--------------------------------

### Configure Training and Dataset Parameters in SegFormer

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Defines the project's configuration dictionary including dataset loading, optimizer settings, learning rate schedules, and runner execution details. This configuration is used by the MMCV-based training pipeline to manage iterative training loops and model evaluation.

```python
test = dict(
    type='CityscapesDataset',
    data_root='data/cityscapes/',
    img_dir='leftImg8bit/val',
    ann_dir='gtFine/val',
    pipeline=[
        dict(type='LoadImageFromFile'),
        dict(
            type='MultiScaleFlipAug',
            img_scale=(2048, 1024),
            flip=False,
            transforms=[
                dict(type='Resize', keep_ratio=True),
                dict(type='RandomFlip'),
                dict(
                    type='Normalize',
                    mean=[123.675, 116.28, 103.53],
                    std=[58.395, 57.12, 57.375],
                    to_rgb=True),
                dict(type='ImageToTensor', keys=['img']),
                dict(type='Collect', keys=['img'])
            ])
    ])

optimizer = dict(
    type='SGD',
    lr=0.01,
    momentum=0.9,
    weight_decay=0.0005)

runner = dict(
    type='IterBasedRunner',
    max_iters=40000)
```

--------------------------------

### Configure Dataset Type and Path

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Sets the dataset type to 'StandfordBackgroundDataset' and defines the root directory for the dataset. It also configures data loading parameters such as samples per GPU and workers per GPU.

```python
cfg.dataset_type = 'StandfordBackgroundDataset'
cfg.data_root = data_root

cfg.data.samples_per_gpu = 8
cfg.data.workers_per_gpu=8
```

--------------------------------

### Switching Normalization Configuration (Python)

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Illustrates how to change the normalization layer type, such as switching from `SyncBN` to `BN`, by redefining the `norm_cfg` variable. This change needs to be applied to all relevant model components like the backbone, decode head, and auxiliary head where normalization is used.

```python
_base_ = '../pspnet/psp_r50_512x1024_40ki_cityscpaes.py'
norm_cfg = dict(type='BN', requires_grad=True)
model = dict(
    backbone=dict(norm_cfg=norm_cfg),
    decode_head=dict(norm_cfg=norm_cfg),
    auxiliary_head=dict(norm_cfg=norm_cfg))

```

--------------------------------

### Training Data Augmentation Pipeline

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Defines the sequence of image and annotation transformations applied during training. Includes resizing, random cropping, flipping, photometric distortion, normalization, padding, and data formatting. This pipeline ensures data diversity and robustness for model training.

```python
[
    dict(type='Resize',
        img_scale=(2048, 1024),
        ratio_range=(0.5, 2.0)),
    dict(type='RandomCrop',
        crop_size=(512, 1024),
        cat_max_ratio=0.75),
    dict(
        type='RandomFlip',
        flip_ratio=0.5),
    dict(type='PhotoMetricDistortion'),
    dict(
        type='Normalize',
        mean=[123.675, 116.28, 103.53],
        std=[58.395, 57.12, 57.375],
        to_rgb=True),
    dict(type='Pad',
        size=(512, 1024),
        pad_val=0,
        seg_pad_val=255),
    dict(type='DefaultFormatBundle'),
    dict(type='Collect',
        keys=['img', 'gt_semantic_seg'])
]
```

--------------------------------

### Ignoring Fields in Base Configs (Python)

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Demonstrates how to ignore specific fields from base configuration files using `_delete_=True`. This is useful for completely replacing sections like the model backbone with new definitions. It requires understanding the structure of the base config and the new desired structure.

```python
_base_ = '../pspnet/psp_r50_512x1024_40ki_cityscpaes.py'
norm_cfg = dict(type='SyncBN', requires_grad=True)
model = dict(
    pretrained='open-mmlab://msra.hrnetv2_w32',
    backbone=dict(
        _delete_=True,
        type='HRNet',
        norm_cfg=norm_cfg,
        extra=dict(
            stage1=dict(
                num_modules=1,
                num_branches=1,
                block='BOTTLENECK',
                num_blocks=(4, ),
                num_channels=(64, )),
            stage2=dict(
                num_modules=1,
                num_branches=2,
                block='BASIC',
                num_blocks=(4, 4),
                num_channels=(32, 64)),
            stage3=dict(
                num_modules=4,
                num_branches=3,
                block='BASIC',
                num_blocks=(4, 4, 4),
                num_channels=(32, 64, 128)),
            stage4=dict(
                num_modules=3,
                num_branches=4,
                block='BASIC',
                num_blocks=(4, 4, 4, 4),
                num_channels=(32, 64, 128, 256)))),
    decode_head=dict(...),
    auxiliary_head=dict(...))

```

--------------------------------

### Prepare Custom Datasets for Training

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Automates the download, extraction, and initial inspection of a custom dataset. Provides a template for converting raw annotations into semantic segmentation maps.

```python
# download and unzip
!wget http://dags.stanford.edu/data/iccv09Data.tar.gz -O standford_background.tar.gz
!tar xf standford_background.tar.gz

# Let's take a look at the dataset
import mmcv
import matplotlib.pyplot as plt

img = mmcv.imread('iccv09Data/images/6000124.jpg')
plt.figure(figsize=(8, 6))
plt.imshow(mmcv.bgr2rgb(img))
plt.show()

import os.path as osp
import numpy as np
from PIL import Image
# convert dataset annotation to semantic segmentation map
data_root = 'iccv09Data'
img_dir = 'images'
ann_dir = 'labels'
```

--------------------------------

### Configure PSPNet Model Architecture

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Defines the complete configuration for a PSPNet segmentor, including the ResNet50V1c backbone, PSPHead, and auxiliary FCNHead. It specifies hyperparameters for normalization, loss functions, and data pipelines necessary for model training and evaluation.

```python
norm_cfg = dict(type='SyncBN', requires_grad=True)
model = dict(
    type='EncoderDecoder',
    pretrained='open-mmlab://resnet50_v1c',
    backbone=dict(
        type='ResNetV1c',
        depth=50,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        dilations=(1, 1, 2, 4),
        strides=(1, 2, 1, 1),
        norm_cfg=dict(type='SyncBN', requires_grad=True),
        norm_eval=False,
        style='pytorch',
        contract_dilation=True),
    decode_head=dict(
        type='PSPHead',
        in_channels=2048,
        in_index=3,
        channels=512,
        pool_scales=(1, 2, 3, 6),
        dropout_ratio=0.1,
        num_classes=19,
        norm_cfg=dict(type='SyncBN', requires_grad=True),
        align_corners=False,
        loss_decode=dict(
            type='CrossEntropyLoss',
            use_sigmoid=False,
            loss_weight=1.0)),
    auxiliary_head=dict(
        type='FCNHead',
        in_channels=1024,
        in_index=2,
        channels=256,
        num_convs=1,
        concat_input=False,
        dropout_ratio=0.1,
        num_classes=19,
        norm_cfg=dict(type='SyncBN', requires_grad=True),
        align_corners=False,
        loss_decode=dict(
            type='CrossEntropyLoss',
            use_sigmoid=False,
            loss_weight=0.4)))
train_cfg = dict()
test_cfg = dict(mode='whole')
dataset_type = 'CityscapesDataset'
data_root = 'data/cityscapes/'
img_norm_cfg = dict(
    mean=[123.675, 116.28, 103.53],
    std=[58.395, 57.12, 57.375],
    to_rgb=True)
crop_size = (512, 1024)
train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations')]
```

--------------------------------

### Use PyTorch Adam Optimizer in Config

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/customize_runtime.md

Demonstrates how to switch to the Adam optimizer by modifying the `optimizer` field in configuration files. Note that this may lead to a performance drop.

```python
optimizer = dict(type='Adam', lr=0.0003, weight_decay=0.0001)
```

--------------------------------

### Test Dataset with Single GPU (Shell)

Source: https://github.com/nvlabs/segformer/blob/master/docs/inference.md

Execute inference on a dataset using a single GPU. This command requires the configuration file and checkpoint file. Optional arguments allow specifying output file, evaluation metrics, and visualization.

```shell
python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--out ${RESULT_FILE}] [--eval ${EVAL_METRICS}] [--show]
```

--------------------------------

### Set Training, Validation, and Test Data Configurations

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Assigns the dataset type, root path, image directory, annotation directory, pipeline, and split file for the training, validation, and testing datasets.

```python
cfg.data.train.type = cfg.dataset_type
cfg.data.train.data_root = cfg.data_root
cfg.data.train.img_dir = img_dir
cfg.data.train.ann_dir = ann_dir
cfg.data.train.pipeline = cfg.train_pipeline
cfg.data.train.split = 'splits/train.txt'

cfg.data.val.type = cfg.dataset_type
cfg.data.val.data_root = cfg.data_root
cfg.data.val.img_dir = img_dir
cfg.data.val.ann_dir = ann_dir
cfg.data.val.pipeline = cfg.test_pipeline
cfg.data.val.split = 'splits/val.txt'

cfg.data.test.type = cfg.dataset_type
cfg.data.test.data_root = cfg.data_root
cfg.data.test.img_dir = img_dir
cfg.data.test.ann_dir = ann_dir
cfg.data.test.pipeline = cfg.test_pipeline
cfg.data.test.split = 'splits/val.txt'
```

--------------------------------

### Cityscapes Dataset Configuration

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Defines the configuration for the Cityscapes dataset, including data root, image and annotation directories for both training and validation sets. It specifies the batch size per GPU and the number of workers per GPU for data loading.

```python
{
    'samples_per_gpu': 2,
    'workers_per_gpu': 2,
    'train': {
        'type': 'CityscapesDataset',
        'data_root': 'data/cityscapes/',
        'img_dir': 'leftImg8bit/train',
        'ann_dir': 'gtFine/train',
        'pipeline': [
            dict(type='LoadImageFromFile'),
            dict(type='LoadAnnotations'),
            dict(
                type='Resize', img_scale=(2048, 1024), ratio_range=(0.5, 2.0)),
            dict(type='RandomCrop', crop_size=(512, 1024), cat_max_ratio=0.75),
            dict(type='RandomFlip', flip_ratio=0.5),
            dict(type='PhotoMetricDistortion'),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=True),
            dict(type='Pad', size=(512, 1024), pad_val=0, seg_pad_val=255),
            dict(type='DefaultFormatBundle'),
            dict(type='Collect', keys=['img', 'gt_semantic_seg'])
        ]
    },
    'val': {
        'type': 'CityscapesDataset',
        'data_root': 'data/cityscapes/',
        'img_dir': 'leftImg8bit/val',
        'ann_dir': 'gtFine/val',
        'pipeline': [
            dict(type='LoadImageFromFile'),
            dict(
                type='MultiScaleFlipAug',
                img_scale=(2048, 1024),
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=True),
                    dict(type='RandomFlip'),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[58.395, 57.12, 57.375],
                        to_rgb=True),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]
    }
}
```

--------------------------------

### Configure Training Iterations, Logging, and Evaluation Intervals

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

Sets the total number of training iterations, the interval for logging training information, and the interval for evaluating the model's performance. Checkpoint saving interval is also configured.

```python
cfg.total_iters = 200
cfg.log_config.interval = 10
cfg.evaluation.interval = 200
cfg.checkpoint_config.interval = 200
```

--------------------------------

### Visualize Segmentation Results (Shell)

Source: https://github.com/nvlabs/segformer/blob/master/docs/inference.md

Test a model and visualize the segmentation results in real-time. This command is applicable to single GPU testing and requires a GUI environment. Results are displayed as they are generated, allowing for interactive inspection.

```shell
python tools/test.py configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py \
    checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth \
    --show
```

--------------------------------

### Testing Data Augmentation Pipeline

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/config.md

Configures the data augmentation pipeline for testing. It utilizes MultiScaleFlipAug to apply augmentations across different scales and includes resizing, flipping (though disabled in this config), normalization, and conversion to tensors. This pipeline prepares data for model evaluation.

```python
[
    dict(type='LoadImageFromFile'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(2048, 1024),
        flip=False,
        transforms=[
            dict(type='Resize',
                 keep_ratio=True),
            dict(type='RandomFlip'),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=True),
            dict(type='ImageToTensor',
                keys=['img']),
            dict(type='Collect',
                keys=['img'])
        ])
]
```

--------------------------------

### Specify Custom Optimizer in Config

Source: https://github.com/nvlabs/segformer/blob/master/docs/tutorials/customize_runtime.md

Shows how to use a custom-implemented optimizer, `MyOptimizer`, in the configuration file by specifying its type and constructor arguments.

```python
optimizer = dict(type='MyOptimizer', a=a_value, b=b_value, c=c_value)
```

--------------------------------

### Load and Modify Training Configuration

Source: https://github.com/nvlabs/segformer/blob/master/demo/MMSegmentation_Tutorial.ipynb

This snippet demonstrates how to load a base training configuration file using `mmcv.Config` and subsequently modify it for a new dataset. It specifically adjusts normalization configuration (`norm_cfg`) and the number of output classes for the decode and auxiliary heads of the segmentation model.

```python
from mmcv import Config
from mmseg.apis import set_random_seed

# Load the base configuration file
cfg = Config.fromfile('configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py')

# Since we use ony one GPU, BN is used instead of SyncBN
cfg.norm_cfg = dict(type='BN', requires_grad=True)
cfg.model.backbone.norm_cfg = cfg.norm_cfg
cfg.model.decode_head.norm_cfg = cfg.norm_cfg
cfg.model.auxiliary_head.norm_cfg = cfg.norm_cfg

# modify num classes of the model in decode/auxiliary head
# Assuming 'classes' is defined and has length 8
num_classes = 8 
cfg.model.decode_head.num_classes = num_classes
cfg.model.auxiliary_head.num_classes = num_classes

# Optional: Set random seed for reproducibility
# set_random_seed(42, deterministic=False)

```