### Install Libraries

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/segformer_inference_pretrained.ipynb

Ensure you have the latest versions of the required libraries installed.

```python
# make sure you have the latest version of the libraries
!pip install -U segmentation-models-pytorch
!pip install albumentations matplotlib requests pillow
```

--------------------------------

### Install Latest Version from Source

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/install.md

Install the most recent version directly from the GitHub repository.

```bash
pip install -U git+https://github.com/qubvel/segmentation_models.pytorch
```

--------------------------------

### Install ONNX and ONNX Runtime

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/convert_to_onnx.ipynb

Install the required libraries for ONNX export and runtime.

```python
# to make onnx export work
!pip install onnx onnxruntime
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/README.md

Set up the development environment by creating a virtual environment and installing the library in development mode.

```bash
make install_dev  # Create .venv, install SMP in dev mode
```

--------------------------------

### Install PyPI Version

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/README.md

Install the latest stable version of the segmentation-models-pytorch library from PyPI.

```bash
pip install segmentation-models-pytorch
```

--------------------------------

### Install Segmentation Models PyTorch

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Installs the segmentation-models-pytorch library along with lightning and albumentations. Use this to set up your environment.

```python
%%capture
%pip install --upgrade segmentation-models-pytorch lightning albumentations
```

--------------------------------

### Install PyPI Version

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/install.md

Install the latest stable version from the Python Package Index (PyPI).

```bash
pip install -U segmentation-models-pytorch
```

--------------------------------

### Install Latest GitHub Version

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/README.md

Install the most recent version of the segmentation-models-pytorch library directly from its GitHub repository.

```bash
pip install git+https://github.com/qubvel/segmentation_models.pytorch
```

--------------------------------

### Install Segmentation Models PyTorch and Dependencies

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/cars segmentation (camvid).ipynb

Installs the segmentation-models-pytorch library, lightning, and albumentations. Use this command in your environment to set up the required packages.

```bash
!pip install segmentation-models-pytorch lightning albumentations
```

--------------------------------

### Install Segmentation Models PyTorch and Dependencies

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Installs the segmentation-models-pytorch library and other necessary packages like lightning and albumentations. Use this command to set up your environment.

```bash
%%capture
!pip install -U git+https://github.com/qubvel-org/segmentation_models.pytorch
!pip install lightning albumentations
```

--------------------------------

### Model Initialization Parameters Example

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/save_load_model_and_share_with_hf_hub.ipynb

Example dictionary defining initialization parameters for a segmentation model. This can be useful for documenting or recreating model configurations.

```python
model_init_params = {
    "encoder_name": "resnet34",
    "encoder_depth": 5,
    "encoder_weights": "imagenet",
    "decoder_use_batchnorm": True,
    "decoder_channels": (256, 128, 64, 32, 16),
    "decoder_attention_type": None,
    "in_channels": 3,
    "classes": 1,
    "activation": None,
    "aux_params": None
}
```

--------------------------------

### Install Albumentations and NumPy

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/save_load_model_and_share_with_hf_hub.ipynb

Install or upgrade the albumentations library and ensure numpy is at version 1.x. This is a prerequisite for using albumentations with segmentation models.

```bash
!pip install -U albumentations numpy==1.*
```

--------------------------------

### Model Metrics Example

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/save_load_model_and_share_with_hf_hub.ipynb

Example JSON object representing performance metrics for a trained model. This is typically saved alongside the model.

```json
{
    "mIoU": 0.95,
    "accuracy": 0.96
}
```

--------------------------------

### Import SimpleOxfordPetDataset

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Imports the predefined SimpleOxfordPetDataset class for easy data loading. This class is useful for quick setup and testing.

```python
from segmentation_models_pytorch.datasets import SimpleOxfordPetDataset
```

--------------------------------

### Compute and use segmentation metrics

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/metrics.md

Example demonstrating how to compute statistics (TP, FP, FN, TN) and then calculate various metrics like IoU, F1, F2, accuracy, and recall for multilabel predictions. Ensure `torch` is imported and tensors are prepared.

```python
import segmentation_models_pytorch as smp

# lets assume we have multilabel prediction for 3 classes
output = torch.rand([10, 3, 256, 256])
target = torch.rand([10, 3, 256, 256]).round().long()

# first compute statistics for true positives, false positives, false negative and
# true negative "pixels"
tp, fp, fn, tn = smp.metrics.get_stats(output, target, mode='multilabel', threshold=0.5)

# then compute metrics with required reduction (see metric docs)
iou_score = smp.metrics.iou_score(tp, fp, fn, tn, reduction="micro")
f1_score = smp.metrics.f1_score(tp, fp, fn, tn, reduction="micro")
f2_score = smp.metrics.fbeta_score(tp, fp, fn, tn, beta=2, reduction="micro")
accuracy = smp.metrics.accuracy(tp, fp, fn, tn, reduction="macro")
recall = smp.metrics.recall(tp, fp, fn, tn, reduction="micro-imagewise")
```

--------------------------------

### Download CamVid Dataset

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/cars segmentation (camvid).ipynb

Clones the SegNet-Tutorial repository to download the CamVid dataset. Ensure you have git installed and accessible in your environment.

```bash
!git clone https://github.com/alexgkendall/SegNet-Tutorial ./data
```

--------------------------------

### Get Training Augmentation Configuration

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Defines a set of augmentations for the training dataset, including flips, affine transforms, padding, cropping, noise, perspective, and color manipulations. Always apply padding and random crop to ensure consistent input size.

```python
def get_training_augmentation():
    train_transform = [
        A.HorizontalFlip(p=0.5),
        A.ShiftScaleRotate(
            scale_limit=0.5, rotate_limit=0, shift_limit=0.1, p=1, border_mode=0
        ),
        A.PadIfNeeded(min_height=320, min_width=320, always_apply=True),
        A.RandomCrop(height=320, width=320, always_apply=True),
        A.GaussNoise(p=0.2),
        A.Perspective(p=0.5),
        A.OneOf(
            [
                A.CLAHE(p=1),
                A.RandomBrightnessContrast(p=1),
                A.RandomGamma(p=1),
            ],
            p=0.9,
        ),
        A.OneOf(
            [
                A.Sharpen(p=1),
                A.Blur(blur_limit=3, p=1),
                A.MotionBlur(blur_limit=3, p=1),
            ],
            p=0.9,
        ),
        A.OneOf(
            [
                A.RandomBrightnessContrast(p=1),
                A.HueSaturationValue(p=1),
            ],
            p=0.9,
        ),
    ]
    return A.Compose(train_transform)
```

--------------------------------

### Instantiate Dataset and Visualize Sample

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Creates an instance of the custom Dataset class and visualizes the first image and its mask. This helps verify data loading and preprocessing.

```python
dataset = Dataset(x_train_dir, y_train_dir)
image, mask = dataset[0]
print(f"Mask shape: {mask.shape}")
visualize(image=image, mask=mask)
```

--------------------------------

### Instantiate Dataset and Visualize Sample

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/cars segmentation (camvid).ipynb

Creates an instance of the custom Dataset class for the 'car' class and visualizes the first sample's image and its corresponding mask. This helps in verifying data loading and preprocessing.

```python
dataset = Dataset(x_train_dir, y_train_dir, classes=["car"])
# get some sample
image, mask = dataset[0]
visualize(
    image=image,
    cars_mask=mask.squeeze(),
)
```

--------------------------------

### Create and Load Datasets

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/cars segmentation (camvid).ipynb

Sets up training, validation, and test datasets using the defined augmentation pipelines and loads them into DataLoaders. Specifies batch size, shuffling, and number of workers for efficient data handling.

```python
CLASSES = ["car"]

train_dataset = Dataset(
    x_train_dir,
    y_train_dir,
    augmentation=get_training_augmentation(),
    classes=CLASSES,
)

valid_dataset = Dataset(
    x_valid_dir,
    y_valid_dir,
    augmentation=get_validation_augmentation(),
    classes=CLASSES,
)

test_dataset = Dataset(
    x_test_dir,
    y_test_dir,
    augmentation=get_validation_augmentation(),
    classes=CLASSES,
)

train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True, num_workers=4)
valid_loader = DataLoader(valid_dataset, batch_size=32, shuffle=False, num_workers=4)
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=False, num_workers=4)
```

--------------------------------

### Visualize Segmentation Predictions

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Fetches a batch of data, switches the model to evaluation mode, gets raw logits, applies softmax to get probabilities, and converts probabilities to predicted class labels. It then visualizes the original image, ground truth mask, and predicted mask for the first 5 samples in the batch.

```python
import numpy as np

# Fetch a batch from the test loader
images, masks = next(iter(test_loader))

# Switch the model to evaluation mode
with torch.inference_mode():
    model.eval()
    logits = model(images)  # Get raw logits from the model

# Apply softmax to get class probabilities
# Shape: [batch_size, num_classes, H, W]

pr_masks = logits.softmax(dim=1)
# Convert class probabilities to predicted class labels
pr_masks = pr_masks.argmax(dim=1)  # Shape: [batch_size, H, W]

# Visualize a few samples (image, ground truth mask, and predicted mask)
for idx, (image, gt_mask, pr_mask) in enumerate(zip(images, masks, pr_masks)):
    if idx <= 4:  # Visualize first 5 samples
        plt.figure(figsize=(12, 6))

        # Original Image
        plt.subplot(1, 3, 1)
        plt.imshow(
            image.cpu().numpy().transpose(1, 2, 0)
        )  # Convert CHW to HWC for plotting
        plt.title("Image")
        plt.axis("off")

        # Ground Truth Mask
        plt.subplot(1, 3, 2)
        plt.imshow(gt_mask.cpu().numpy(), cmap="tab20")  # Visualize ground truth mask
        plt.title("Ground truth")
        plt.axis("off")

        # Predicted Mask
        plt.subplot(1, 3, 3)
        plt.imshow(pr_mask.cpu().numpy(), cmap="tab20")  # Visualize predicted mask
        plt.title("Prediction")
        plt.axis("off")

        # Show the figure
        plt.show()
    else:
        break
```

--------------------------------

### Create Datasets and DataLoaders

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Sets up training, validation, and test datasets using specified augmentation configurations. It then creates DataLoaders for each dataset, specifying batch size, shuffle, and number of workers.

```python
train_dataset = Dataset(
    x_train_dir,
    y_train_dir,
    augmentation=get_training_augmentation(),
)

valid_dataset = Dataset(
    x_valid_dir,
    y_valid_dir,
    augmentation=get_validation_augmentation(),
)

test_dataset = Dataset(
    x_test_dir,
    y_test_dir,
    augmentation=get_validation_augmentation(),
)

# Change to > 0 if not on Windows machine
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True, num_workers=0)
valid_loader = DataLoader(valid_dataset, batch_size=32, shuffle=False, num_workers=0)
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=False, num_workers=0)
```

--------------------------------

### Load Model from Hub

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Load a pre-trained segmentation model from the hub using its identifier. Ensure the 'segmentation_models' library is installed.

```python
import segmentation_models_pytorch as smp

loaded_model = smp.from_pretrained("ytzfhqs/oxford-pet-segmentation")
print("Loaded model:", loaded_model.__class__.__name__)
```

--------------------------------

### Initialize Datasets and DataLoaders

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Initializes training, validation, and test datasets using SimpleOxfordPetDataset. It also creates DataLoaders with specified batch sizes and number of workers. Ensure datasets do not overlap.

```python
# init train, val, test sets
train_dataset = SimpleOxfordPetDataset(root, "train")
valid_dataset = SimpleOxfordPetDataset(root, "valid")
test_dataset = SimpleOxfordPetDataset(root, "test")

# It is a good practice to check datasets don`t intersects with each other
assert set(test_dataset.filenames).isdisjoint(set(train_dataset.filenames))
assert set(test_dataset.filenames).isdisjoint(set(valid_dataset.filenames))
assert set(train_dataset.filenames).isdisjoint(set(valid_dataset.filenames))

print(f"Train size: {len(train_dataset)}")
print(f"Valid size: {len(valid_dataset)}")
print(f"Test size: {len(test_dataset)}")

n_cpu = os.cpu_count()
train_dataloader = DataLoader(
    train_dataset, batch_size=64, shuffle=True, num_workers=n_cpu
)
valid_dataloader = DataLoader(
    valid_dataset, batch_size=64, shuffle=False, num_workers=n_cpu
)
test_dataloader = DataLoader(
    test_dataset, batch_size=64, shuffle=False, num_workers=n_cpu
)
```

--------------------------------

### Initialize PyTorch Lightning Trainer

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Initialize the Trainer with basic configurations like max epochs and logging frequency. Ensure necessary imports are present.

```python
trainer = pl.Trainer(max_epochs=EPOCHS, log_every_n_steps=1)
```

--------------------------------

### Download CamVid Dataset

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Clones the SegNet-Tutorial repository to download the CamVid dataset. This dataset is used for semantic segmentation examples.

```python
%%capture
# Download Data
!git clone https://github.com/alexgkendall/SegNet-Tutorial ./data
```

--------------------------------

### Visualize Training Dataset Sample

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Displays the first image and its corresponding mask from the training dataset. Note that images are transposed to HWC for visualization, and masks have their channel dimension squeezed.

```python
# lets look at some samples

sample = train_dataset[0]
plt.subplot(1, 2, 1)
# for visualization we have to transpose back to HWC
plt.imshow(sample["image"].transpose(1, 2, 0))
plt.subplot(1, 2, 2)
# for visualization we have to remove 3rd dimension of mask
plt.imshow(sample["mask"].squeeze())
plt.show()
```

--------------------------------

### Custom Decoder Normalization in Unet++

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/models.md

Example of configuring custom normalization layers within the Unet++ decoder. This allows for fine-grained control over the normalization process.

```python
decoder_use_norm={"type": "layernorm", "eps": 1e-2}
```

--------------------------------

### Load Pretrained Model and Perform Inference

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/upernet_inference_pretrained.ipynb

Loads a pretrained UperNet model (e.g., swin-tiny) from Hugging Face, sets up the appropriate preprocessing, and performs inference on a sample image. The model is moved to the appropriate device (CUDA or CPU).

```python
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# More checkpoints can be found here:
# https://huggingface.co/collections/smp-hub/upernet-67fadcdbe08418c6ea94f768
checkpoint = "smp-hub/upernet-swin-tiny"

# Load pretrained model and preprocessing function
model = smp.from_pretrained(checkpoint).eval().to(device)
preprocessing = A.Compose.from_pretrained(checkpoint)
print("Preprocessing:", preprocessing)

# Load image
url = "https://huggingface.co/datasets/hf-internal-testing/fixtures_ade20k/resolve/main/ADE_val_00000001.jpg"
image = Image.open(requests.get(url, stream=True).raw)

# Preprocess image
image = np.array(image)
normalized_image = preprocessing(image=image)["image"]
input_tensor = torch.as_tensor(normalized_image)
input_tensor = input_tensor.permute(2, 0, 1).unsqueeze(0)  # HWC -> BCHW
input_tensor = input_tensor.to(device)

# Perform inference
with torch.inference_mode():
    output_mask = model(input_tensor)

# Postprocess mask
mask = torch.nn.functional.interpolate(
    output_mask, size=image.shape[:2], mode="bilinear", align_corners=False
)
mask = mask[0].argmax(0).cpu().numpy()
```

--------------------------------

### Get Data Preprocessing Function

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/quickstart.md

Retrieve the appropriate preprocessing function for a given encoder and its weights. This is recommended for optimal results when using pre-trained weights.

```python
from segmentation_models_pytorch.encoders import get_preprocessing_fn

preprocess_input = get_preprocessing_fn('resnet18', pretrained='imagenet')
```

--------------------------------

### Get Validation Augmentation Configuration

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Applies padding to ensure the image shape is divisible by 32 for the validation dataset. This is a simpler transform compared to the training set.

```python
def get_validation_augmentation():
    """Add paddings to make image shape divisible by 32"""
    test_transform = [
        A.PadIfNeeded(384, 480),
    ]
    return A.Compose(test_transform)
```

--------------------------------

### Custom PyTorch Dataset Class

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Defines a custom Dataset class for loading images and segmentation masks. It handles class remapping and augmentation. Ensure albumentations is installed for augmentation.

```python
import os
import cv2
import numpy as np
import albumentations as A


class Dataset(BaseDataset):
    CLASSES = [
        "sky",
        "building",
        "pole",
        "road",
        "pavement",
        "tree",
        "signsymbol",
        "fence",
        "car",
        "pedestrian",
        "bicyclist",
        "unlabelled",
    ]

    def __init__(self, images_dir, masks_dir, classes=None, augmentation=None):
        self.ids = os.listdir(images_dir)
        self.images_fps = [os.path.join(images_dir, image_id) for image_id in self.ids]
        self.masks_fps = [os.path.join(masks_dir, image_id) for image_id in self.ids]

        # Always map background ('unlabelled') to 0
        self.background_class = self.CLASSES.index("unlabelled")

        # If specific classes are provided, map them dynamically
        if classes:
            self.class_values = [self.CLASSES.index(cls.lower()) for cls in classes]
        else:
            self.class_values = list(range(len(self.CLASSES)))  # Default to all classes

        # Create a remapping dictionary: class value in dataset -> new index (0, 1, 2, ...)
        # Background will always be 0, other classes will be remapped starting from 1.
        self.class_map = {self.background_class: 0}
        self.class_map.update(
            {
                v: i + 1
                for i, v in enumerate(self.class_values)
                if v != self.background_class
            }
        )

        self.augmentation = augmentation

    def __getitem__(self, i):
        # Read the image
        image = cv2.imread(self.images_fps[i])
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)  # Convert BGR to RGB

        # Read the mask in grayscale mode
        mask = cv2.imread(self.masks_fps[i], 0)

        # Create a blank mask to remap the class values
        mask_remap = np.zeros_like(mask)

        # Remap the mask according to the dynamically created class map
        for class_value, new_value in self.class_map.items():
            mask_remap[mask == class_value] = new_value

        if self.augmentation:
            sample = self.augmentation(image=image, mask=mask_remap)
            image, mask_remap = sample["image"], sample["mask"]
        image = image.transpose(2, 0, 1)
        return image, mask_remap

    def __len__(self):
        return len(self.ids)
```

--------------------------------

### Unet Model

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/models.md

Documentation for the Unet model, a fully convolutional neural network for semantic image segmentation. It details the class parameters, return types, and provides a usage example.

```APIDOC
## Unet Model

### Description
U-Net is a fully convolutional neural network architecture designed for semantic image segmentation. It consists of an encoder (downsampling path) and a decoder (upsampling path) with skip connections to preserve spatial details.

### Class
`segmentation_models_pytorch.Unet`

### Parameters
- **encoder_name** (str) – Name of the classification model used as an encoder (backbone).
- **encoder_depth** (int) – Number of stages in the encoder, in range [3, 5]. Default is 5.
- **encoder_weights** (str | None) – Encoder weights. Options: None (random initialization), "imagenet" (pre-training on ImageNet), or other available pretrained weights. For `tu-` encoders, set to True to download weights or None for random initialization.
- **decoder_channels** (Sequence[int]) – List of integers specifying `in_channels` for decoder convolutions. Length must match `encoder_depth`.
- **decoder_use_norm** (bool | str | Dict[str, Any]) – Normalization type between Conv2D and activation. Accepts True (defaults to "batchnorm"), False (nn.Identity), string (e.g., "batchnorm", "instancenorm"), or a dictionary for custom settings like `{"type": "layernorm", "eps": 1e-2}`.
- **decoder_attention_type** (str | None) – Attention module for the decoder. Available options: None, "scse".
- **decoder_interpolation** (str) – Interpolation mode for the decoder. Available options: "nearest", "bilinear", "bicubic", "area", "nearest-exact". Default is "nearest".
- **in_channels** (int) – Number of input channels for the model. Default is 3 (RGB images).
- **classes** (int) – Number of output classes for the segmentation mask.
- **activation** (str | Callable | None) – Activation function after the final convolution. Options: "sigmoid", "softmax", "logsoftmax", "tanh", "identity", callable, or None. Default is None.
- **aux_params** (dict | None) – Dictionary for auxiliary output (classification head). Supported keys: `classes` (int), `pooling` (str: "max" or "avg"), `dropout` (float), `activation` (str).
- **kwargs** (Dict[str, Any]) – Arguments passed to the encoder class `__init__()`. Applies only to `timm` models.

### Returns
An instance of the Unet model.

### Return type
`torch.nn.Module`

### Example
```python
import torch
import segmentation_models_pytorch as smp

model = smp.Unet("resnet18", encoder_weights="imagenet", classes=5)
model.eval()
```
```

--------------------------------

### Visualize Test Dataset Sample

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Displays the first image and its corresponding mask from the test dataset. The image is transposed to HWC and the mask's channel dimension is squeezed for proper visualization.

```python
sample = test_dataset[0]
plt.subplot(1, 2, 1)
# for visualization we have to transpose back to HWC
plt.imshow(sample["image"].transpose(1, 2, 0))
plt.subplot(1, 2, 2)

```

--------------------------------

### Train Segmentation Model

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/cars segmentation (camvid).ipynb

Configures and starts the training process for the segmentation model using PyTorch Lightning's Trainer. It specifies the maximum number of epochs and logging frequency.

```python
trainer = pl.Trainer(max_epochs=EPOCHS, log_every_n_steps=1)

trainer.fit(
    model,
    train_dataloaders=train_loader,
    val_dataloaders=valid_loader,
)
```

--------------------------------

### Import Libraries for Preprocessing

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/save_load_model_and_share_with_hf_hub.ipynb

Import the necessary libraries, albumentations for image transformations and segmentation_models_pytorch for the model.

```python
import albumentations as A
import segmentation_models_pytorch as smp
```

--------------------------------

### Visualize Augmented Images and Masks

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Demonstrates how to apply training augmentations to a dataset and visualize the resulting augmented images and their corresponding masks. This helps in understanding the effect of the augmentations.

```python
# Visualize resulted augmented images and masks
augmented_dataset = Dataset(
    x_train_dir,
    y_train_dir,
    augmentation=get_training_augmentation(),
)

# Visualizing the same image with different random transforms
for i in range(3):
    image, mask = augmented_dataset[3]
    print(f"Mask shape: {mask.shape}")
    print(np.unique(mask))
    visualize(image=image, mask=mask)
```

--------------------------------

### Visualize Augmented Images

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/cars segmentation (camvid).ipynb

Demonstrates how to apply the training augmentation to a dataset and visualize the resulting augmented images and masks. This helps in understanding the effect of the chosen augmentations.

```python
# Visualize resulted augmented images and masks
augmented_dataset = Dataset(
    x_train_dir,
    y_train_dir,
    augmentation=get_training_augmentation(),
    classes=["car"],
)

# same image with different random transforms
for i in range(3):
    image, mask = augmented_dataset[3]
    visualize(image=image, mask=mask.squeeze())
```

--------------------------------

### Understanding Clouds Dataset Solution

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/HALLOFFAME.md

Code repository for a 34th place solution in the Understanding Clouds from Satellite Images competition.

```python
https://github.com/khornlund/understanding-cloud-organization
```

--------------------------------

### Instantiate CamVidModel

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Creates an instance of the CamVidModel with specified architecture, encoder, and channel configurations. Use this to initialize the model before training.

```python
model = CamVidModel("FPN", "resnext50_32x4d", in_channels=3, out_classes=OUT_CLASSES)
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Imports core libraries for PyTorch, visualization, PyTorch Lightning, and segmentation models. Ensure these are imported before use.

```python
import os

import torch
import matplotlib.pyplot as plt
import pytorch_lightning as pl
from torch.optim import lr_scheduler
import segmentation_models_pytorch as smp
from torch.utils.data import DataLoader
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Imports core libraries for data handling, visualization, and PyTorch dataset management.

```python
import os
import matplotlib.pyplot as plt
from torch.utils.data import DataLoader
from torch.utils.data import Dataset as BaseDataset
```

--------------------------------

### Download PyTorch Model Weights

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/cars segmentation (camvid).ipynb

Shows the progress of downloading pre-trained weights for the resnext50_32x4d model from PyTorch's model hub.

```text
Downloading: "https://download.pytorch.org/models/resnext50_32x4d-7cdf4587.pth" to /root/.cache/torch/hub/checkpoints/resnext50_32x4d-7cdf4587.pth
100%|██████████| 95.8M/95.8M [00:00<00:00, 127MB/s]
```

--------------------------------

### Configure Optimizer and Learning Rate Scheduler

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Sets up the Adam optimizer and CosineAnnealingLR scheduler for model training. The scheduler is configured to anneal over steps.

```python
def configure_optimizers(self):
        optimizer = torch.optim.Adam(self.parameters(), lr=2e-4)
        scheduler = lr_scheduler.CosineAnnealingLR(optimizer, T_max=T_MAX, eta_min=1e-5)
        return {
            
```

--------------------------------

### Save Model and Push to Hugging Face Hub

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/save_load.md

Saves the model and simultaneously pushes it to the Hugging Face Hub. Requires authentication with Hugging Face CLI.

```python
import segmentation_models_pytorch as smp

model = smp.Unet('resnet34', encoder_weights='imagenet')

# Or saved and pushed to the Hub simultaneously
model.save_pretrained('username/my-model', push_to_hub=True)
```

--------------------------------

### Import segmentation_models_pytorch

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/save_load_model_and_share_with_hf_hub.ipynb

Import the library for use in your project.

```python
import segmentation_models_pytorch as smp
```

--------------------------------

### Unet++ Model Initialization

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/models.md

This shows how to initialize the Unet++ model with specified encoder and decoder configurations. It's suitable for semantic segmentation tasks.

```python
segmentation_models_pytorch.UnetPlusPlus(
    encoder_name="resnet34",
    encoder_depth=5,
    encoder_weights="imagenet",
    decoder_use_norm="batchnorm",
    decoder_channels=(256, 128, 64, 32, 16),
    decoder_attention_type=None,
    decoder_interpolation="nearest",
    in_channels=3,
    classes=1,
    activation=None,
    aux_params=None,
    **kwargs
)
```

--------------------------------

### Create or Load Model

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/convert_to_onnx.ipynb

Instantiate a segmentation model (e.g., Unet with ResNet34 encoder) and set it to evaluation mode.

```python
model = smp.Unet("resnet34", encoder_weights="imagenet", classes=1)
model = model.eval()
```

--------------------------------

### Initialize Segmentation Model

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Instantiates a segmentation model from the segmentation_models.pytorch library. Specify the encoder, backbone, and number of output classes.

```python
model = PetModel("FPN", "resnet34", in_channels=3, out_classes=1)
```

--------------------------------

### Load Model from Local Directory

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/save_load.md

Loads a previously saved model's weights and configuration from a local directory. Ensure the path points to the directory where the model was saved.

```python
import segmentation_models_pytorch as smp

# Load the model from the local directory
model = smp.from_pretrained('./my_model')
```

--------------------------------

### Define Data Directories

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb

Sets up the directory paths for training, validation, and testing images and their corresponding segmentation masks.

```python
DATA_DIR = "./data/CamVid/"

x_train_dir = os.path.join(DATA_DIR, "train")
y_train_dir = os.path.join(DATA_DIR, "trainannot")

x_valid_dir = os.path.join(DATA_DIR, "val")
y_valid_dir = os.path.join(DATA_DIR, "valannot")

x_test_dir = os.path.join(DATA_DIR, "test")
y_test_dir = os.path.join(DATA_DIR, "testannot")
```

--------------------------------

### Download Oxford-IIIT Pet Dataset

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Downloads the Oxford-IIIT Pet Dataset to the specified root directory. This is a prerequisite for using the SimpleOxfordPetDataset.

```python
# download data
root = "."
SimpleOxfordPetDataset.download(root)
```

--------------------------------

### Basic Training Loop

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/docs/quickstart.md

A simplified PyTorch training loop demonstrating model inference, loss calculation, backpropagation, and optimizer step.

```python
for images, gt_masks in dataloader:

    predicted_mask = model(images)
    loss = loss_fn(predicted_mask, gt_masks)

    loss.backward()
    optimizer.step()
```

--------------------------------

### Train the Model

Source: https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/binary_segmentation_intro.ipynb

Initializes a PyTorch Lightning Trainer and fits the model to the training and validation data. Training is set to run for a specified number of epochs with logging every N steps.

```python
trainer = pl.Trainer(max_epochs=EPOCHS, log_every_n_steps=1)

trainer.fit(
    model,
    train_dataloaders=train_dataloader,
    val_dataloaders=valid_dataloader,
)
```