### Install Documentation Requirements

Source: https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md

Navigate to the 'docs' directory and install the necessary Python packages for building the documentation using pip and the requirements.txt file.

```bash
cd docs
pip install -r requirements.txt
```

--------------------------------

### Build and install torchvision library

Source: https://github.com/pytorch/vision/blob/main/examples/cpp/README.md

Compiles and installs the torchvision C++ library from source.

```bash
mkdir build
cd build
cmake .. -DCMAKE_PREFIX_PATH=/path/to/libtorch  # or -DTorch_DIR= if you installed PyTorch normally, see above
cmake --build .
cmake --install .
```

--------------------------------

### Install Optional Dependencies for Image Formats

Source: https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md

Install libpng, libjpeg-turbo, and libwebp to enable native encoding/decoding for PNG, JPEG, and WebP formats in torchvision.io. This is an optional step.

```bash
conda install libpng libjpeg-turbo libwebp -c pytorch
```

--------------------------------

### Serve Built Documentation Locally

Source: https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md

Start a simple Python HTTP server in the 'docs/build/html' directory to serve the generated documentation files. This is useful for local development and previewing changes.

```bash
python -m http.server 8000 # or any free port
```

--------------------------------

### Build Documentation Locally

Source: https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md

Navigate to the 'docs' directory and use 'make' to build the HTML documentation. The 'html-noplot' target excludes building gallery examples to save time.

```bash
cd docs
make html-noplot
```

--------------------------------

### List and Initialize Models by Name

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Provides examples of using `list_models` to find available models and `get_model` to initialize specific models by their string names, with or without weights.

```python
# List available models
all_models = list_models()
classification_models = list_models(module=torchvision.models)

# Initialize models
m1 = get_model("mobilenet_v3_large", weights=None)
m2 = get_model("quantized_mobilenet_v3_large", weights="DEFAULT")
```

--------------------------------

### Apply Preprocessing Transforms

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Illustrates how to get and apply the necessary preprocessing transforms bundled with a model's weights to an input image.

```python
# Initialize the Weight Transforms
weights = ResNet50_Weights.DEFAULT
preprocess = weights.transforms()

# Apply it to the input image
img_transformed = preprocess(img)
```

--------------------------------

### Configure CMake with PyTorch installation

Source: https://github.com/pytorch/vision/blob/main/examples/cpp/README.md

Sets the Torch_DIR variable and runs CMake for a standard PyTorch installation.

```bash
TORCH_PATH=$(python -c "import pathlib, torch; print(pathlib.Path(torch.__path__[0]))")
Torch_DIR="${TORCH_PATH}/share/cmake/Torch"   # there should be .cmake files in there

cmake .. -DTorch_DIR=$Torch_DIR
```

--------------------------------

### Evaluation Results for Author Provided Weights

Source: https://github.com/pytorch/vision/blob/main/references/depth/stereo/README.md

Example log output for author-provided MEGVII_V1 weights. These results can be used to benchmark against other pre-trained models.

```text
Dataset: middlebury2014-train @size: [384, 512]:
{
	1: {
		2: {'mae': 1.704, 'rmse': 3.738, '1px': 0.738, '3px': 0.896, '5px': 0.933, 'relepe': 0.157, 'fl-all': 76.464}
		5: {'mae': 0.956, 'rmse': 2.963, '1px': 0.88, '3px': 0.948, '5px': 0.965, 'relepe': 0.124, 'fl-all': 88.186}
		10: {'mae': 0.792, 'rmse': 2.765, '1px': 0.905, '3px': 0.958, '5px': 0.97, 'relepe': 0.114, 'fl-all': 90.429}
		20: {'mae': 0.749, 'rmse': 2.706, '1px': 0.907, '3px': 0.961, '5px': 0.972, 'relepe': 0.113, 'fl-all': 90.807}
	},
}
{
	2: {
		2: {'mae': 1.702, 'rmse': 3.784, '1px': 0.784, '3px': 0.894, '5px': 0.924, 'relepe': 0.172, 'fl-all': 80.313}
		5: {'mae': 0.932, 'rmse': 2.907, '1px': 0.877, '3px': 0.944, '5px': 0.963, 'relepe': 0.125, 'fl-all': 87.979}
		10: {'mae': 0.773, 'rmse': 2.768, '1px': 0.901, '3px': 0.958, '5px': 0.972, 'relepe': 0.117, 'fl-all': 90.43}
		20: {'mae': 0.854, 'rmse': 2.971, '1px': 0.9, '3px': 0.957, '5px': 0.97, 'relepe': 0.122, 'fl-all': 90.269}
	},
}
```

--------------------------------

### Install Other Development Dependencies

Source: https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md

Install additional Python packages required for development and testing of Torchvision, including testing frameworks and linters.

```bash
pip install expecttest flake8 typing mypy pytest pytest-mock scipy requests
```

--------------------------------

### Clone and Install Torchvision for Development

Source: https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md

Clone the Torchvision repository and install it in editable mode for development. Use the -v flag for verbose output. For macOS, specific environment variables might be needed. For C++ debugging, set DEBUG=1.

```bash
git clone https://github.com/pytorch/vision.git
cd vision
pip install -e . -v --no-build-isolation
```

```bash
# or, for OSX
# MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ pip install -e . -v --no-build-isolation
# for C++ debugging, use DEBUG=1
# DEBUG=1 pip install -e . -v --no-build-isolation
```

--------------------------------

### Configure project with torchvision support

Source: https://github.com/pytorch/vision/blob/main/examples/cpp/README.md

Configures CMake to link against the installed torchvision library using the USE_TORCHVISION flag.

```bash
cmake .. -DTorch_DIR=$Torch_DIR -DTorchVision_DIR=path/to/libtorchvision -DUSE_TORCHVISION=ON
cmake --build .
```

--------------------------------

### Evaluation Results for CREStereo Finetuned Weights

Source: https://github.com/pytorch/vision/blob/main/references/depth/stereo/README.md

Example log output for finetuned CREStereo weights, showing MAE and other metrics. Compare these results to the base weights to assess performance improvements.

```text
Dataset: middlebury2014-train @size: [384, 512]:
{
	1: {
		2: {'mae': 1.85, 'rmse': 3.797, '1px': 0.673, '3px': 0.862, '5px': 0.917, 'relepe': 0.171, 'fl-all': 69.736}
		5: {'mae': 1.111, 'rmse': 3.166, '1px': 0.838, '3px': 0.93, '5px': 0.957, 'relepe': 0.134, 'fl-all': 84.596}
		10: {'mae': 1.02, 'rmse': 3.073, '1px': 0.854, '3px': 0.938, '5px': 0.96, 'relepe': 0.129, 'fl-all': 86.042}
		20: {'mae': 0.993, 'rmse': 3.059, '1px': 0.855, '3px': 0.942, '5px': 0.967, 'relepe': 0.126, 'fl-all': 85.784}
	},
}
{
	2: {
		2: {'mae': 1.667, 'rmse': 3.867, '1px': 0.78, '3px': 0.891, '5px': 0.922, 'relepe': 0.165, 'fl-all': 78.89}
		5: {'mae': 1.158, 'rmse': 3.278, '1px': 0.843, '3px': 0.926, '5px': 0.955, 'relepe': 0.135, 'fl-all': 84.556}
		10: {'mae': 1.046, 'rmse': 3.13, '1px': 0.85, '3px': 0.934, '5px': 0.96, 'relepe': 0.13, 'fl-all': 85.464}
		20: {'mae': 1.021, 'rmse': 3.102, '1px': 0.85, '3px': 0.935, '5px': 0.963, 'relepe': 0.129, 'fl-all': 85.417}
	},
}
```

--------------------------------

### Evaluation Results for CREStereo Base Weights

Source: https://github.com/pytorch/vision/blob/main/references/depth/stereo/README.md

Example log output showing Mean Absolute Error (MAE) and other metrics for different cascade and iteration counts when evaluating CREStereo base weights.

```text
Dataset: middlebury2014-train @size: [384, 512]:
{
	1: {
		2: {'mae': 2.363, 'rmse': 4.352, '1px': 0.611, '3px': 0.828, '5px': 0.891, 'relepe': 0.176, 'fl-all': 64.511}
		5: {'mae': 1.618, 'rmse': 3.71, '1px': 0.761, '3px': 0.879, '5px': 0.918, 'relepe': 0.154, 'fl-all': 77.128}
		10: {'mae': 1.416, 'rmse': 3.53, '1px': 0.777, '3px': 0.896, '5px': 0.933, 'relepe': 0.148, 'fl-all': 78.388}
		20: {'mae': 1.448, 'rmse': 3.583, '1px': 0.771, '3px': 0.893, '5px': 0.931, 'relepe': 0.145, 'fl-all': 77.7}
	},
}
{
	2: {
		2: {'mae': 1.972, 'rmse': 4.125, '1px': 0.73, '3px': 0.865, '5px': 0.908, 'relepe': 0.169, 'fl-all': 74.396}
		5: {'mae': 1.403, 'rmse': 3.448, '1px': 0.793, '3px': 0.905, '5px': 0.937, 'relepe': 0.151, 'fl-all': 80.186}
		10: {'mae': 1.312, 'rmse': 3.368, '1px': 0.799, '3px': 0.912, '5px': 0.943, 'relepe': 0.148, 'fl-all': 80.379}
		20: {'mae': 1.376, 'rmse': 3.542, '1px': 0.796, '3px': 0.91, '5px': 0.942, 'relepe': 0.149, 'fl-all': 80.054}
	},
}
```

--------------------------------

### Object Detection with Faster R-CNN

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Perform object detection using a pre-trained Faster R-CNN model. This example demonstrates initializing the model with default weights, applying preprocessing, running inference, and drawing bounding boxes on the image.

```python
from torchvision.io.image import decode_image
from torchvision.models.detection import fasterrcnn_resnet50_fpn_v2, FasterRCNN_ResNet50_FPN_V2_Weights
from torchvision.utils import draw_bounding_boxes
from torchvision.transforms.functional import to_pil_image

img = decode_image("test/assets/encode_jpeg/grace_hopper_517x606.jpg")

# Step 1: Initialize model with the best available weights
weights = FasterRCNN_ResNet50_FPN_V2_Weights.DEFAULT
model = fasterrcnn_resnet50_fpn_v2(weights=weights, box_score_thresh=0.9)
model.eval()

# Step 2: Initialize the inference transforms
preprocess = weights.transforms()

# Step 3: Apply inference preprocessing transforms
batch = [preprocess(img)]

# Step 4: Use the model and visualize the prediction
prediction = model(batch)[0]
labels = [weights.meta["categories"][i] for i in prediction["labels"]]
box = draw_bounding_boxes(img, boxes=prediction["boxes"],
                          labels=labels,
                          colors="red",
                          width=4, font_size=30)
im = to_pil_image(box.detach())
im.show()
```

--------------------------------

### Compose V2 Transforms Pipeline

Source: https://github.com/pytorch/vision/blob/main/docs/source/transforms.md

Example of a typical transform pipeline using torchvision.transforms.v2.Compose. It demonstrates converting to tensor, setting dtype, resizing, and normalizing. This pipeline is optimized for performance in training environments with multiple workers.

```python
from torchvision.transforms import v2
transforms = v2.Compose([
    v2.ToImage(),  # Convert to tensor, only needed if you had a PIL image
    v2.ToDtype(torch.uint8, scale=True),  # optional, most input are already uint8 at this point
    # ...
    v2.RandomResizedCrop(size=(224, 224), antialias=True),  # Or Resize(antialias=True)
    # ...
    v2.ToDtype(torch.float32, scale=True),  # Normalize expects float input
    v2.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
```

--------------------------------

### Create build directory

Source: https://github.com/pytorch/vision/blob/main/examples/cpp/README.md

Initializes a build directory for the project.

```bash
mkdir build
cd build
```

--------------------------------

### Initialize ResNet50 with Different Weights

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Demonstrates initializing a ResNet50 model with various pre-trained weight options, including the latest default, specific versions, and no weights. Supports string aliases for weights.

```python
from torchvision.models import resnet50, ResNet50_Weights

# Old weights with accuracy 76.130%
resnet50(weights=ResNet50_Weights.IMAGENET1K_V1)

# New weights with accuracy 80.858%
resnet50(weights=ResNet50_Weights.IMAGENET1K_V2)

# Best available weights (currently alias for IMAGENET1K_V2)
# Note that these weights may change across versions
resnet50(weights=ResNet50_Weights.DEFAULT)

# Strings are also supported
resnet50(weights="IMAGENET1K_V2")

# No weights - random initialization
resnet50(weights=None)
```

--------------------------------

### Video Backend Management

Source: https://github.com/pytorch/vision/blob/main/docs/source/index.md

Functions to get and set the video decoding backend for torchvision.

```APIDOC
## GET torchvision.get_video_backend()

### Description
Returns the currently active video backend used to decode videos.

### Method
GET

### Endpoint
/torchvision/get_video_backend

### Response
#### Success Response (200)
- **return** (str) - Name of the video backend. Currently only ‘pyav’ is supported.

## POST torchvision.set_video_backend(backend)

### Description
Specifies the package used to decode videos.

### Method
POST

### Endpoint
/torchvision/set_video_backend

### Parameters
#### Path Parameters
- **backend** (string) - Required - Name of the video backend. Only ‘pyav’ is supported.
The `pyav` package uses the 3rd party PyAv library. It is a Pythonic binding for the FFmpeg libraries.
```

--------------------------------

### Image Backend Management

Source: https://github.com/pytorch/vision/blob/main/docs/source/index.md

Functions to get and set the image loading backend for torchvision.

```APIDOC
## GET torchvision.get_image_backend()

### Description
Gets the name of the package used to load images.

### Method
GET

### Endpoint
/torchvision/get_image_backend

## POST torchvision.set_image_backend(backend)

### Description
Specifies the package used to load images.

### Method
POST

### Endpoint
/torchvision/set_image_backend

### Parameters
#### Path Parameters
- **backend** (string) - Required - Name of the image backend. one of {‘PIL’, ‘accimage’}.
The `accimage` package uses the Intel IPP library. It is generally faster than PIL, but does not support as many operations.
```

--------------------------------

### Configure Video ResNet training arguments

Source: https://github.com/pytorch/vision/blob/main/references/video_classification/README.md

Command-line arguments for training Video ResNet models, including clip length, learning rate, and augmentation settings.

```bash
# number of frames per clip
--clip_len 16 \ 
--frame-rate 15 \
# allow for temporal jittering
--clips_per_video 5 \
--batch-size 24 \
--epochs 45 \
--lr 0.64 \
# we use 10 epochs for linear warmup
--lr-warmup-epochs 10 \
# learning rate is decayed at 20, 30, and 40 epoch by a factor of 10
--lr-milestones 20, 30, 40 \
--lr-gamma 0.1 \
--train-resize-size 128 171 \
--train-crop-size 112 112 \
--val-resize-size 128 171 \
--val-crop-size 112 112
```

--------------------------------

### Build and upload torchvision wheels

Source: https://github.com/pytorch/vision/wiki/Home

Use these commands to generate distribution files and upload them to PyPI. Ensure you are on the correct version branch before executing.

```bash
# checkout to relevant version branch
python setup.py sdist bdist_wheel
twine upload dist/torchvision-0.1.7-py2.py3-none-any.whl
```

--------------------------------

### Initialize and Preprocess Semantic Segmentation Model

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Demonstrates loading a pre-trained FCN_ResNet50 model, initializing its default weights, and applying the required inference transforms to an input image.

```python
from torchvision.io.image import decode_image
from torchvision.models.segmentation import fcn_resnet50, FCN_ResNet50_Weights
from torchvision.transforms.functional import to_pil_image

img = decode_image("gallery/assets/dog1.jpg")

# Step 1: Initialize model with the best available weights
weights = FCN_ResNet50_Weights.DEFAULT
model = fcn_resnet50(weights=weights)
model.eval()

# Step 2: Initialize the inference transforms
preprocess = weights.transforms()

# Step 3: Apply inference preprocessing transforms
batch = preprocess(img).unsqueeze(0)
```

--------------------------------

### Quantize ShuffleNet V2 Models

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Use these commands for post-training quantization of shufflenet_v2_x1_5 and shufflenet_v2_x2_0 models on CPU. Ensure the correct weights and data path are specified.

```bash
python train_quantization.py --device='cpu' --post-training-quantize --qbackend='fbgemm'     --model=shufflenet_v2_x1_5 --weights="ShuffleNet_V2_X1_5_Weights.IMAGENET1K_V1"     --train-crop-size 176 --val-resize-size 232 --data-path /datasets01_ontap/imagenet_full_size/061417/
```

```bash
python train_quantization.py --device='cpu' --post-training-quantize --qbackend='fbgemm'     --model=shufflenet_v2_x2_0 --weights="ShuffleNet_V2_X2_0_Weights.IMAGENET1K_V1"     --train-crop-size 176 --val-resize-size 232 --data-path /datasets01_ontap/imagenet_full_size/061417/
```

--------------------------------

### Run multi-GPU training with torchrun

Source: https://github.com/pytorch/vision/blob/main/references/video_classification/README.md

Executes training on a single node with 8 GPUs using the torchrun utility.

```bash
torchrun --nproc_per_node=8 train.py --data-path=/data/kinectics400 --kinetics-version="400" --lr 0.08 --cache-dataset --sync-bn --amp
```

--------------------------------

### Format Torchvision Code with ufmt

Source: https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md

Format the Torchvision codebase using ufmt, black, and usort. This command formats all files in the torchvision directory. Ensure these tools are installed first.

```bash
ufmt format torchvision
```

--------------------------------

### Migrate to New Multi-weight API

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Shows equivalent model initialization calls between the old deprecated API (using `pretrained` parameter) and the new multi-weight support API.

```python
from torchvision.models import resnet50, ResNet50_Weights

# Using pretrained weights:
resnet50(weights=ResNet50_Weights.IMAGENET1K_V1)
resnet50(weights="IMAGENET1K_V1")
resnet50(pretrained=True)  # deprecated
resnet50(True)  # deprecated

# Using no weights:
resnet50(weights=None)
resnet50()
resnet50(pretrained=False)  # deprecated
resnet50(False)  # deprecated
```

--------------------------------

### List Optional Arguments for Training Script

Source: https://github.com/pytorch/vision/blob/main/references/similarity/README.md

View all available optional arguments for the training script. This helps in customizing the training process.

```bash
python train.py -h    # Lists all optional arguments
```

--------------------------------

### Get Graph Node Names for ResNet50

Source: https://github.com/pytorch/vision/blob/main/docs/source/feature_extraction.md

Retrieves the names of all graph nodes for a given model in both training and evaluation modes. This is useful for designing feature extractors by identifying available nodes.

```python
import torch
from torchvision.models import resnet50
from torchvision.models.feature_extraction import get_graph_node_names
from torchvision.models.feature_extraction import create_feature_extractor
from torchvision.models.detection.mask_rcnn import MaskRCNN
from torchvision.models.detection.backbone_utils import LastLevelMaxPool
from torchvision.ops.feature_pyramid_network import FeaturePyramidNetwork


# To assist you in designing the feature extractor you may want to print out
# the available nodes for resnet50.
m = resnet50()
train_nodes, eval_nodes = get_graph_node_names(resnet50())
```

--------------------------------

### Run Training Script with Default Arguments

Source: https://github.com/pytorch/vision/blob/main/references/similarity/README.md

Execute the training script with default arguments. This script trains ResNet50 on FashionMNIST to learn image embeddings.

```bash
python train.py 		# Runs training script with default args
```

--------------------------------

### Train Faster R-CNN ResNet-50 FPN

Source: https://github.com/pytorch/vision/blob/main/references/detection/README.md

Use this command to train a Faster R-CNN model with a ResNet-50 FPN backbone on the COCO dataset. Ensure you have the necessary dependencies installed and data path configured.

```bash
torchrun --nproc_per_node=8 train.py\
    --dataset coco --model fasterrcnn_resnet50_fpn --epochs 26\
    --lr-steps 16 22 --aspect-ratio-group-factor 3 --weights-backbone ResNet50_Weights.IMAGENET1K_V1
```

--------------------------------

### Evaluate Author Provided Weights (MEGVII_V1)

Source: https://github.com/pytorch/vision/blob/main/references/depth/stereo/README.md

Run evaluation using the author-provided MEGVII_V1 weights on the Middlebury 2014 dataset. This allows comparison with external weight sources.

```bash
torchrun --nproc_per_node 1 --nnodes 1 cascade_evaluation.py --dataset middlebury2014-train --batch-size 1 --dataset-root $dataset_root --model crestereo_base --weights CREStereo_Base_Weights.MEGVII_V1
```

--------------------------------

### Train FCOS ResNet-50 FPN

Source: https://github.com/pytorch/vision/blob/main/references/detection/README.md

Command to train an FCOS model with ResNet-50 FPN backbone on the COCO dataset. This example includes settings for learning rate, automatic mixed precision (AMP), and backbone weights.

```bash
torchrun --nproc_per_node=8 train.py\
    --dataset coco --model fcos_resnet50_fpn --epochs 26\
    --lr-steps 16 22 --aspect-ratio-group-factor 3  --lr 0.01 --amp --weights-backbone ResNet50_Weights.IMAGENET1K_V1
```

--------------------------------

### S3D Model Training Configuration

Source: https://github.com/pytorch/vision/blob/main/references/video_classification/README.md

Use this configuration for training the S3D model. It specifies parameters like batch size, learning rate, clip length, and resizing dimensions.

```bash
--batch-size=12 --lr 0.2 --clip-len 64 --clips-per-video 5 --sync-bn \
--train-resize-size 256 256 --train-crop-size 224 224 --val-resize-size 256 256 --val-crop-size 224 224
```

--------------------------------

### Run single-GPU training

Source: https://github.com/pytorch/vision/blob/main/references/video_classification/README.md

Executes training on a single GPU. Note that this process can be extremely slow.

```bash
python train.py --data-path=/data/kinectics400 --kinetics-version="400" --batch-size=8 --cache-dataset
```

--------------------------------

### Quantization Aware Training for MobileNetV2

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Train MobileNetV2 with quantization aware training using torchrun for distributed training. Key settings include batch size, learning rate, and number of epochs.

```bash
torchrun --nproc_per_node=8 train_quantization.py --model='mobilenet_v2'
```

--------------------------------

### Evaluate Quantized Models

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Evaluate quantized models using pre-trained weights on CPU. Replace '<qbackend>' and '<model_name>' with specific values. Additional parameters are needed for inception_v3.

```bash
python train_quantization.py --device='cpu' --test-only --qbackend='<qbackend>' --model='<model_name>'
```

```bash
--val-resize-size 342 --val-crop-size 299 --train-crop-size 299
```

--------------------------------

### Quantization Aware Training for MobileNetV3 Large

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Train MobileNetV3 Large with quantization aware training using torchrun. Specify the model, weight decay, and learning rate. Training converges around 10 epochs.

```bash
torchrun --nproc_per_node=8 train_quantization.py --model='mobilenet_v3_large'     --wd 0.00001 --lr 0.001
```

--------------------------------

### Post-Training Quantization

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Command to perform post-training quantization on CPU. Settings include num_calibration_batches: 32, num_workers: 16, batch_size: 32, eval_batch_size: 128, and qbackend: 'fbgemm'. $MODEL can be one of several supported models.

```bash
python train_quantization.py --device='cpu' --post-training-quantize --qbackend='fbgemm' --model='$MODEL'
```

--------------------------------

### Train Vision Transformer Models

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Commands for training various Vision Transformer (ViT) architectures.

```bash
torchrun --nproc_per_node=8 train.py\
    --model vit_b_16 --epochs 300 --batch-size 512 --opt adamw --lr 0.003 --wd 0.3\
    --lr-scheduler cosineannealinglr --lr-warmup-method linear --lr-warmup-epochs 30\
    --lr-warmup-decay 0.033 --amp --label-smoothing 0.11 --mixup-alpha 0.2 --auto-augment ra\
    --clip-grad-norm 1 --ra-sampler --cutmix-alpha 1.0 --model-ema
```

```bash
torchrun --nproc_per_node=8 train.py\
    --model vit_b_32 --epochs 300 --batch-size 512 --opt adamw --lr 0.003 --wd 0.3\
    --lr-scheduler cosineannealinglr --lr-warmup-method linear --lr-warmup-epochs 30\
    --lr-warmup-decay 0.033 --amp --label-smoothing 0.11 --mixup-alpha 0.2 --auto-augment imagenet\
    --clip-grad-norm 1 --ra-sampler --cutmix-alpha 1.0 --model-ema
```

```bash
torchrun --nproc_per_node=8 train.py\
    --model vit_l_16 --epochs 600 --batch-size 128 --lr 0.5 --lr-scheduler cosineannealinglr\
    --lr-warmup-method linear --lr-warmup-epochs 5 --label-smoothing 0.1 --mixup-alpha 0.2\
    --auto-augment ta_wide --random-erase 0.1 --weight-decay 0.00002 --norm-weight-decay 0.0\
    --clip-grad-norm 1 --ra-sampler --cutmix-alpha 1.0 --model-ema --val-resize-size 232
```

```bash
torchrun --nproc_per_node=8 train.py\
    --model vit_l_32 --epochs 300 --batch-size 512 --opt adamw --lr 0.003 --wd 0.3\
    --lr-scheduler cosineannealinglr --lr-warmup-method linear --lr-warmup-epochs 30\
    --lr-warmup-decay 0.033 --amp --label-smoothing 0.11 --mixup-alpha 0.2 --auto-augment ra\
    --clip-grad-norm 1 --ra-sampler --cutmix-alpha 1.0 --model-ema
```

--------------------------------

### List model weights via PyTorch Hub

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Retrieve and print all available weights for a specific model using PyTorch Hub.

```python
import torch

weight_enum = torch.hub.load("pytorch/vision", "get_model_weights", name="resnet50")
print([weight for weight in weight_enum])
```

--------------------------------

### Execute the model

Source: https://github.com/pytorch/vision/blob/main/examples/cpp/README.md

Runs the compiled executable with the specified TorchScript model file.

```bash
./run_model resnet18.pt
```

--------------------------------

### Perform inference with classification models

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Initialize a model with pre-trained weights, apply inference transforms, and process an image.

```python
from torchvision.io import decode_image
from torchvision.models import resnet50, ResNet50_Weights

img = decode_image("test/assets/encode_jpeg/grace_hopper_517x606.jpg")

# Step 1: Initialize model with the best available weights
weights = ResNet50_Weights.DEFAULT
model = resnet50(weights=weights)
model.eval()

# Step 2: Initialize the inference transforms
preprocess = weights.transforms()

# Step 3: Apply inference preprocessing transforms
batch = preprocess(img).unsqueeze(0)

# Step 4: Use the model and print the predicted category
prediction = model(batch).squeeze(0).softmax(0)
class_id = prediction.argmax().item()
score = prediction[class_id].item()
category_name = weights.meta["categories"][class_id]
print(f"{category_name}: {100 * score:.1f}%")
```

--------------------------------

### Train AlexNet or VGG models

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Use this command to train AlexNet or VGG models when batch normalization is not included, requiring a lower learning rate.

```bash
torchrun --nproc_per_node=8 train.py\
    --model $MODEL --lr 1e-2
```

--------------------------------

### Train EfficientNet-V2

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Command to train EfficientNet-V2 models. Requires setting $MODEL, $TRAIN_SIZE, and $EVAL_SIZE variables.

```bash
torchrun --nproc_per_node=8 train.py \
--model $MODEL --batch-size 128 --lr 0.5 --lr-scheduler cosineannealinglr \
--lr-warmup-epochs 5 --lr-warmup-method linear --auto-augment ta_wide --epochs 600 --random-erase 0.1 \
--label-smoothing 0.1 --mixup-alpha 0.2 --cutmix-alpha 1.0 --weight-decay 0.00002 --norm-weight-decay 0.0 \
--train-crop-size $TRAIN_SIZE --model-ema --val-crop-size $EVAL_SIZE --val-resize-size $EVAL_SIZE \
--ra-sampler --ra-reps 4
```

--------------------------------

### Train MaxViT Model

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Command to train MaxViT models, with $MODEL set to maxvit_t. Note that --val-resize-size was not optimized post-training for this model.

```bash
torchrun --nproc_per_node=8 --n_nodes=4 train.py\
--model $MODEL --epochs 400 --batch-size 128 --opt adamw --lr 3e-3 --weight-decay 0.05 --lr-scheduler cosineannealinglr --lr-min 1e-5 --lr-warmup-method linear  --lr-warmup-epochs 32  --label-smoothing 0.1 --mixup-alpha 0.8 --clip-grad-norm 1.0 --interpolation bicubic --auto-augment ta_wide --policy-magnitude 15 --model-ema --val-resize-size 224\
--val-crop-size 224 --train-crop-size 224 --amp  --model-ema-steps 32 --transformer-embedding-decay 0 --sync-bn
```

--------------------------------

### Train SwinTransformer Model

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Command for training SwinTransformer models. Supported models include swin_t, swin_s, and swin_b. Note that --val-resize-size was optimized post-training.

```bash
torchrun --nproc_per_node=8 train.py \
--model $MODEL --epochs 300 --batch-size 128 --opt adamw --lr 0.001 --weight-decay 0.05 --norm-weight-decay 0.0  --bias-weight-decay 0.0 --transformer-embedding-decay 0.0 --lr-scheduler cosineannealinglr --lr-min 0.00001 --lr-warmup-method linear  --lr-warmup-epochs 20 --lr-warmup-decay 0.01 --amp --label-smoothing 0.1 --mixup-alpha 0.8 --clip-grad-norm 5.0 --cutmix-alpha 1.0 --random-erase 0.25 --interpolation bicubic --auto-augment ta_wide --model-ema --ra-sampler --ra-reps 4  --val-resize-size 224
```

--------------------------------

### Train SwinTransformer V2 Model

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Training command for SwinTransformer V2 models. Available models are swin_v2_t, swin_v2_s, and swin_v2_b. The --val-resize-size parameter was optimized post-training.

```bash
torchrun --nproc_per_node=8 train.py\
--model $MODEL --epochs 300 --batch-size 128 --opt adamw --lr 0.001 --weight-decay 0.05 --norm-weight-decay 0.0  --bias-weight-decay 0.0 --transformer-embedding-decay 0.0 --lr-scheduler cosineannealinglr --lr-min 0.00001 --lr-warmup-method linear  --lr-warmup-epochs 20 --lr-warmup-decay 0.01 --amp --label-smoothing 0.1 --mixup-alpha 0.8 --clip-grad-norm 5.0 --cutmix-alpha 1.0 --random-erase 0.25 --interpolation bicubic --auto-augment ta_wide --model-ema --ra-sampler --ra-reps 4  --val-resize-size 256 --val-crop-size 256 --train-crop-size 256 
```

--------------------------------

### Inference with Pre-trained Video Classification Model

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Use this snippet to perform inference on a video using pre-trained video classification models. Ensure you have the necessary video file and that the model and weights are correctly initialized.

```python
from torchvision.io.video import read_video
from torchvision.models.video import r3d_18, R3D_18_Weights

vid, _, _ = read_video("test/assets/videos/v_SoccerJuggling_g23_c01.avi", output_format="TCHW")
vid = vid[:32]  # optionally shorten duration

# Step 1: Initialize model with the best available weights
weights = R3D_18_Weights.DEFAULT
model = r3d_18(weights=weights)
model.eval()

# Step 2: Initialize the inference transforms
preprocess = weights.transforms()

# Step 3: Apply inference preprocessing transforms
batch = preprocess(vid).unsqueeze(0)

# Step 4: Use the model and print the predicted category
prediction = model(batch).squeeze(0).softmax(0)
label = prediction.argmax().item()
score = prediction[label].item()
category_name = weights.meta["categories"][label]
print(f"{category_name}: {100 * score}%")
```

--------------------------------

### Train SSD300 VGG16

Source: https://github.com/pytorch/vision/blob/main/references/detection/README.md

Command for training an SSD300 model with a VGG16 backbone. This script is optimized for the COCO dataset and includes parameters for epochs, learning rate steps, batch size, weight decay, data augmentation, and backbone weights.

```bash
torchrun --nproc_per_node=8 train.py\
    --dataset coco --model ssd300_vgg16 --epochs 120\
    --lr-steps 80 110 --aspect-ratio-group-factor 3 --lr 0.002 --batch-size 4\
    --weight-decay 0.0005 --data-augmentation ssd --weights-backbone VGG16_Weights.IMAGENET1K_FEATURES
```

--------------------------------

### Train ConvNeXt Model

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Use this command to train ConvNeXt models. Adjust the --model argument for different variants like convnext_tiny, convnext_small, convnext_base, and convnext_large. Training was performed on a single node with 8 GPUs.

```bash
torchrun --nproc_per_node=8 train.py \
--model $MODEL --batch-size 128 --opt adamw --lr 1e-3 --lr-scheduler cosineannealinglr \
--lr-warmup-epochs 5 --lr-warmup-method linear --auto-augment ta_wide --epochs 600 --random-erase 0.1 \
--label-smoothing 0.1 --mixup-alpha 0.2 --cutmix-alpha 1.0 --weight-decay 0.05 --norm-weight-decay 0.0 \
--train-crop-size 176 --model-ema --val-resize-size 232 --ra-sampler --ra-reps 4
```

--------------------------------

### Train RegNet Models

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Commands for training RegNet variants. Hyperparameters vary based on model size.

```bash
torchrun --nproc_per_node=8 train.py\
     --model $MODEL --epochs 100 --batch-size 128 --wd 0.00005 --lr=0.8\
     --lr-scheduler=cosineannealinglr --lr-warmup-method=linear\
     --lr-warmup-epochs=5 --lr-warmup-decay=0.1
```

```bash
torchrun --nproc_per_node=8 train.py\
     --model $MODEL --epochs 100 --batch-size 64 --wd 0.00005 --lr=0.4\
     --lr-scheduler=cosineannealinglr --lr-warmup-method=linear\
     --lr-warmup-epochs=5 --lr-warmup-decay=0.1
```

```bash
torchrun --nproc_per_node=8 train.py\
     --model $MODEL --epochs 100 --batch-size 32 --wd 0.00005 --lr=0.2\
     --lr-scheduler=cosineannealinglr --lr-warmup-method=linear\
     --lr-warmup-epochs=5 --lr-warmup-decay=0.1
```

--------------------------------

### Enable Mixed Precision Training

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Enable Automatic Mixed Precision (AMP) training on GPU for PyTorch Vision models using the --amp flag. This can improve throughput and allow larger batch sizes by utilizing FP16 operations on compatible NVIDIA GPUs.

```bash
torchrun --nproc_per_node=8 train.py\
    --model resnext50_32x4d --epochs 100 --amp
```

--------------------------------

### Run Mypy for Type Checking

Source: https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md

Use the mypy tool with the provided configuration file to check for type annotation errors in the codebase. Ensure type hints are added where required.

```bash
mypy --config-file mypy.ini
```

--------------------------------

### Train ResNeXt models

Source: https://github.com/pytorch/vision/blob/main/references/classification/README.md

Command to train ResNeXt models for a single node with 8 GPUs. Learning rate scaling is recommended for different GPU counts or batch sizes.

```bash
torchrun --nproc_per_node=8 train.py\
    --model $MODEL --epochs 100
```

--------------------------------

### Load ImageNet dataset with DataLoader

Source: https://github.com/pytorch/vision/blob/main/docs/source/datasets.md

Instantiate a built-in dataset and wrap it in a DataLoader for parallel processing.

```python
imagenet_data = torchvision.datasets.ImageNet('path/to/imagenet_root/')
data_loader = torch.utils.data.DataLoader(imagenet_data,
                                          batch_size=4,
                                          shuffle=True,
                                          num_workers=args.nThreads)
```

--------------------------------

### Perform inference with a quantized ResNet50 model

Source: https://github.com/pytorch/vision/blob/main/docs/source/models.md

Demonstrates loading a pre-trained quantized ResNet50 model, applying necessary transforms, and performing inference on an image.

```python
from torchvision.io import decode_image
from torchvision.models.quantization import resnet50, ResNet50_QuantizedWeights

img = decode_image("test/assets/encode_jpeg/grace_hopper_517x606.jpg")

# Step 1: Initialize model with the best available weights
weights = ResNet50_QuantizedWeights.DEFAULT
model = resnet50(weights=weights, quantize=True)
model.eval()

# Step 2: Initialize the inference transforms
preprocess = weights.transforms()

# Step 3: Apply inference preprocessing transforms
batch = preprocess(img).unsqueeze(0)

# Step 4: Use the model and print the predicted category
prediction = model(batch).squeeze(0).softmax(0)
class_id = prediction.argmax().item()
score = prediction[class_id].item()
category_name = weights.meta["categories"][class_id]
print(f"{category_name}: {100 * score}%")
```