### Installation

Source: https://github.com/apple/ml-cvnets/blob/main/README.md

Instructions for installing CVNets using Conda.

```bash
git clone git@github.com:apple/ml-cvnets.git
cd ml-cvnets

conda create -n cvnets python=3.10.8
conda activate cvnets

pip install -r requirements.txt -c constraints.txt
pip install --editable .
```

--------------------------------

### Install CVNets

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/installation.md

Commands to clone the repository, install dependencies, and install the library in editable mode.

```bash
git clone git@github.com:apple/ml-cvnets.git
cd ml-cvnets
pip install -r requirements.txt
pip install --editable .
```

--------------------------------

### Example: Download image and detect objects with SSD w/ ResNet-50

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/detection/README-detection-SSD-tutorial.md

An example command demonstrating how to download an image from the internet and then detect objects using an SSD with a ResNet-50 backbone.

```bash
export IMG_PATH="http://farm3.staticflickr.com/2700/4532763002_7f4eafc5b0_z.jpg"
export CFG_FILE="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/detection/coco-ssd-resnet-50.yaml"
export MODEL_WEIGHTS="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/detection/coco-ssd-resnet-50.pt"
cvnets-eval-det --common.config-file $CFG_FILE --common.results-loc ssdlite_results --model.detection.pretrained $MODEL_WEIGHTS --model.detection.n-classes 81 --evaluation.detection.resize-input-images --evaluation.detection.mode single_image --evaluation.detection.path "${IMG_PATH}" --model.detection.ssd.conf-threshold 0.3
```

--------------------------------

### Training CLIP Command

Source: https://github.com/apple/ml-cvnets/blob/main/examples/range_augment/README-clip.md

An example command to initiate CLIP model training on multiple nodes and GPUs, including environment variable setup.

```bash
export CFG_FILE="PATH_TO_MODEL_CONFIGURATION_FILE"
export RANK=<NODE_ID> * <NUM_GPUS_PER_NODE> # For Node-0, RANK=0; For Node-1, Rank=8, For Node-2, RANK=16, and so on.
export WORLD_SIZE=<NUM_NODES> * <NUM_GPUS_PER_NODE> # WORLD_SIZE=32 nodes * 8 GPUS per node = 256
cvnets-train --common.config-file $CFG_FILE --common.results-loc results_clip --ddp.rank $RANK --ddp.world-size $WORLD_SIZE --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT'
```

--------------------------------

### Dataset Configuration

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-new-dataset.md

Example YAML configuration for a dataset, specifying name, category, and data roots.

```yaml
dataset:
  name: "ade20k"
  category: "segmentation"
  # Where the data is stored for train/validation (can be different)
  root_train: "/mnt/vision_datasets/ADEChallengeData2016/"
  root_val: "/mnt/vision_datasets/ADEChallengeData2016/"
```

--------------------------------

### Registering a New Dataset

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-new-dataset.md

Example of registering a new dataset class using the DATASET_REGISTRY decorator.

```python
from data.datasets import DATASET_REGISTRY
from data.datasets.dataset_base import BaseImageDataset

@DATASET_REGISTRY.register(name="ade20k", type="segmentation")
class ADE20KDataset(BaseImageDataset):
    # PyTorch Dataset type.
    pass
```

--------------------------------

### Configuration for Extended Dataset

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-new-dataset.md

Example YAML configuration for an extended dataset, specifying name, category, and data roots.

```yaml
dataset:
  name: "my-new-dataset"
  category: "classification"
  root_train: "<path-to-training-data>"
  root_val: "<path-to-validation-data>"
```

--------------------------------

### Example: Segmenting an Internet Image with DeepLabv3 w/ MobileViT

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/segmentation/README-segmentation-deeplabv3-tutorial.md

Command to download an image from the internet and segment objects using DeepLabv3 with a MobileViT backbone.

```bash
export IMG_PATH="http://farm7.staticflickr.com/6206/6118204766_b1c9a39153_z.jpg"
export CFG_FILE="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/segmentation/pascalvoc/deeplabv3-mobilevitv1.yaml"
export MODEL_WEIGHTS="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/segmentation/pascalvoc/deeplabv3-mobilevitv1.pt"
cvnets-eval-seg --common.config-file $CFG_FILE --common.results-loc deeplabv3_results --model.segmentation.pretrained $MODEL_WEIGHTS --model.segmentation.n-classes 21 \
 --evaluation.segmentation.resize-input-images --evaluation.segmentation.mode single_image --evaluation.segmentation.path "${IMG_PATH}" --evaluation.segmentation.save-masks \
 --evaluation.segmentation.apply-color-map --evaluation.segmentation.save-overlay-rgb-pred
```

--------------------------------

### Qualitative evaluation on a single image

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/segmentation/README-segmentation-mobilevit-v2.md

This example demonstrates qualitative evaluation by downloading an image from the internet and segmenting objects using a DeepLabv3 with a MobileViTv2-1.0 backbone. It sets environment variables for the image path, configuration file, and model weights, then executes the evaluation command.

```bash
export IMG_PATH="http://farm7.staticflickr.com/6206/6118204766_b1c9a39153_z.jpg"
 export CFG_FILE="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/segmentation/pascalvoc/mobilevitv2/deeplabv3-mobilevitv2-1.0.yaml"
 export MODEL_WEIGHTS="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/segmentation/pascalvoc/mobilevitv2/deeplabv3-mobilevitv2-1.0.pt"
 cvnets-eval-seg 
 python main_eval.py --common.config-file $CFG_FILE --common.results-loc deeplabv3_mobilevitv2_results --model.segmentation.pretrained $MODEL_WEIGHTS --model.segmentation.n-classes 21 \
 --evaluation.segmentation.resize-input-images --evaluation.segmentation.mode single_image --evaluation.segmentation.path "${IMG_PATH}" --evaluation.segmentation.save-masks \
 --evaluation.segmentation.apply-color-map --evaluation.segmentation.save-overlay-rgb-pred
```

--------------------------------

### Example: MobileViTv2 model conversion

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-pytorch-to-coreml.md

Example commands to convert a MobileViTv2-1.0 classification model trained on ImageNet-1k to CoreML format.

```bash
export CONFIG_FILE="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/classification/mobilevitv2/imagenet1k/256x256/mobilevitv2-1.0.yaml"
export MODEL_WEIGHTS="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/classification/mobilevitv2/imagenet1k/256x256/mobilevitv2-1.0.pt"
cvnets-convert --common.config-file $CONFIG_FILE --common.results-loc coreml_models_cls --model.classification.pretrained $MODEL_WEIGHTS  --conversion.coreml-extn mlmodel
```

--------------------------------

### Qualitative evaluation on a folder of images

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/detection/README-SSDLite-mobilevit-v2.md

Example command to run object detection on multiple images stored in a folder using the SSDLite-MobileViTv2 model.

```bash
export IMG_FOLDER_PATH="PATH_TO_FOLDER_CONTAINING_IMAGES"
 export CFG_FILE="PATH_TO_CONFIG_FILE"
 export MODEL_WEIGHTS="PATH_TO_MODEL_WEIGHTS"
 cvnets-eval-det --common.config-file $CFG_FILE --common.results-loc ssdlite_mobilevitv2_results --model.detection.pretrained $MODEL_WEIGHTS --model.detection.n-classes 81 --evaluation.detection.resize-input-images --evaluation.detection.mode image_folder --evaluation.detection.path $IMG_FOLDER_PATH --model.detection.ssd.conf-threshold 0.3
```

--------------------------------

### Qualitative evaluation on images in a folder

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/detection/README-detection-SSD-tutorial.md

An example command to run detection on multiple images stored in a folder using the SSDLite-MobileViTv2 model.

```bash
export IMG_FOLDER_PATH="PATH_TO_FOLDER_CONTAINING_IMAGES"
export CFG_FILE="PATH_TO_CONFIG_FILE"
export MODEL_WEIGHTS="PATH_TO_MODEL_WEIGHTS"
cvnets-eval-det --common.config-file $CFG_FILE --common.results-loc ssdlite_mobilevitv2_results --model.detection.pretrained $MODEL_WEIGHTS --model.detection.n-classes 81 --evaluation.detection.resize-input-images --evaluation.detection.mode image_folder --evaluation.detection.path $IMG_FOLDER_PATH --model.detection.ssd.conf-threshold 0.3
```

--------------------------------

### Qualitative evaluation on a single image

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/detection/README-SSDLite-mobilevit-v2.md

Example command to run object detection on a single image using the SSDLite-MobileViTv2 model.

```bash
export IMG_PATH="LOCATION_OF_IMAGE_FILE"
 export CFG_FILE="PATH_TO_CONFIG_FILE"
 export MODEL_WEIGHTS="PATH_TO_MODEL_WEIGHTS"
 cvnets-eval-det --common.config-file $CFG_FILE --common.results-loc ssdlite_mobilevitv2_results --model.detection.pretrained $MODEL_WEIGHTS --model.detection.n-classes 81 --evaluation.detection.resize-input-images --evaluation.detection.mode single_image --evaluation.detection.path "${IMG_PATH}" --model.detection.ssd.conf-threshold 0.3
```

--------------------------------

### Extending an Existing Dataset

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-new-dataset.md

Example of extending an existing dataset (ImagenetDataset) and customizing transforms.

```python
from data.datasets import DATASET_REGISTRY
from data.datasets.classification.imagenet import ImagenetDataset

@DATASET_REGISTRY.register(name="my-new-dataset", type="classification")
class AmazingDataset(ImagenetDataset):
    def training_transforms(self, size: tuple or int):
        # My amazing new training-time transforms
        pass
```

--------------------------------

### Qualitative evaluation on a single image

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/detection/README-detection-SSD-tutorial.md

An example command to run detection on a single image using the SSDLite-MobileViTv2 model.

```bash
export IMG_PATH="LOCATION_OF_IMAGE"
export CFG_FILE="PATH_TO_CONFIG_FILE"
export MODEL_WEIGHTS="PATH_TO_MODEL_WEIGHTS"
cvnets-eval-det --common.config-file $CFG_FILE --common.results-loc ssdlite_mobilevitv2_results --model.detection.pretrained $MODEL_WEIGHTS --model.detection.n-classes 81 --evaluation.detection.resize-input-images --evaluation.detection.mode single_image --evaluation.detection.path "${IMG_PATH}" --model.detection.ssd.conf-threshold 0.3
```

--------------------------------

### Qualitative Evaluation on a Folder of Images

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/segmentation/README-segmentation-deeplabv3-tutorial.md

Example command to run segmentation on multiple images stored in a folder using the DeepLabv3-MobileNetv2 model trained on PASCAL VOC 2012.

```bash
export IMG_FOLDER_PATH="PATH_TO_FOLDER_CONTAINING_IMAGES"
export CFG_FILE="PATH_TO_CONFIG_FILE"
export MODEL_WEIGHTS="PATH_TO_MODEL_WEIGHTS"
cvnets-eval-seg --common.config-file $CFG_FILE --common.results-loc deeplabv3_results --model.segmentation.pretrained $MODEL_WEIGHTS --model.segmentation.n-classes 21 --evaluation.segmentation.resize-input-images --evaluation.segmentation.mode image_folder --evaluation.segmentation.path $IMG_FOLDER_PATH \
 --evaluation.segmentation.save-masks \
 --evaluation.segmentation.apply-color-map \
 --evaluation.segmentation.save-overlay-rgb-pred
```

--------------------------------

### Single node 4-GPU training of PSPNet-MobileViTv2-1.0 on PASCAL VOC 2012

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/segmentation/README-segmentation-mobilevit-v2.md

This command starts the training for the PSPNet-MobileViTv2-1.0 model on the PASCAL VOC 2012 dataset, utilizing a single node with 4 GPUs. It requires the path to the ImageNet-1k pre-trained checkpoint.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/segmentation/pascal_voc/pspnet_mobilevitv2.yaml --common.results-loc pspnet_mobilevitv2_results/width_1_0_0 --common.override-kwargs model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Example Data Organization

Source: https://github.com/apple/ml-cvnets/blob/main/examples/range_augment/README-clip.md

This illustrates the expected directory structure for storing image-text pairs in tar files for the CLIP dataset.

```text
img_text_tar_dataset/00000000_0_1000.tar.gz
|--- 00000000_0_image
|--- 00000000_0_text
|--- 00000000_1_image
|--- 00000000_1_text
|--- ...

img_text_tar_dataset/00000000_1000_2000.tar.gz
|--- 00000000_1000_image
|--- 00000000_1000_text
|--- 00000000_1001_image
|--- 00000000_1001_text
|--- ...
```

--------------------------------

### Example Metadata File Format

Source: https://github.com/apple/ml-cvnets/blob/main/examples/range_augment/README-clip.md

This shows the format of a metadata file used for sharding data, mapping ID ranges to tar file names.

```json
Example {'0-1000': '00000000_0_1000.tar.gz', 1000-2000', '00000000_1000_2000.tar.gz'}
```

--------------------------------

### Qualitative Evaluation on a Single Image

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/segmentation/README-segmentation-deeplabv3-tutorial.md

Example command to run segmentation on a single image using the DeepLabv3-MobileNetv2 model trained on PASCAL VOC 2012.

```bash
export IMG_PATH="LOCATION_OF_IMAGE"
export CFG_FILE="PATH_TO_CONFIG_FILE"
export MODEL_WEIGHTS="PATH_TO_MODEL_WEIGHTS"
cvnets-eval-seg --common.config-file $CFG_FILE --common.results-loc deeplabv3_results --model.segmentation.pretrained $MODEL_WEIGHTS --model.segmentation.n-classes 21 --evaluation.segmentation.resize-input-images --evaluation.segmentation.mode single_image --evaluation.segmentation.path "${IMG_PATH}" \
 --evaluation.segmentation.save-masks \
 --evaluation.segmentation.apply-color-map \
 --evaluation.segmentation.save-overlay-rgb-pred
```

--------------------------------

### Dataset Configuration

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-config-files-intro.md

Defines dataset parameters including root directories, names, batch sizes, workers, and GPU memory handling.

```yaml
dataset:
  root_train: "/mnt/imagenet/training"
  root_val: "/mnt/imagenet/validation"
  name: "imagenet"
  category: "classification"
  train_batch_size0: 128
  val_batch_size0: 100
  eval_batch_size0: 100
  workers: 8
  persistent_workers: true
  pin_memory: true
```

--------------------------------

### Sampler Configuration

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-config-files-intro.md

Configures the data sampler, including the type and parameters for variable batch sampling, crop dimensions, and scale variations.

```yaml
sampler:
  name: "variable_batch_sampler"
  vbs:
    crop_size_width: 224
    crop_size_height: 224
    max_n_scales: 5
    min_crop_size_width: 128
    max_crop_size_width: 320
    min_crop_size_height: 128
    max_crop_size_height: 320
    check_scale: 32
```

--------------------------------

### Optimizer and LR Scheduler Configuration

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-config-files-intro.md

Defines the optimizer (SGD with momentum and weight decay) and the learning rate scheduler (Cosine annealing with warm-up).

```yaml
optim:
  name: "sgd"
  weight_decay: 1.e-4
  no_decay_bn_filter_bias: true
  sgd:
    momentum: 0.9
scheduler:
  name: "cosine"
  is_iteration_based: false
  max_epochs: 150
  warmup_iterations: 7500
  warmup_init_lr: 0.05
  cosine:
    max_lr: 0.4
    min_lr: 2.e-4
```

--------------------------------

### Model Configuration

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-config-files-intro.md

Specifies the model architecture, including the type (ResNet-50), activation functions, normalization, and initialization strategies.

```yaml
model:
  classification:
    name: "resnet"
    activation:
      name: "relu"
    resnet:
      depth: 50
  normalization:
    name: "batch_norm"
    momentum: 0.1
  activation:
    name: "relu"
    inplace: true
  layer:
    global_pool: "mean"
    conv_init: "kaiming_normal"
    linear_init: "normal"
```

--------------------------------

### Multi-Node Training (Node-3)

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-classification-tutorial.md

Command for Node-3 in a multi-node training setup, specifying distributed training parameters.

```bash
export CFG_FILE="PATH_TO_MODEL_CONFIGURATION_FILE"
cvnets-train --common.config-file $CFG_FILE --common.results-loc classification_results --ddp.rank 24 --ddp.world-size 32 --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT' --ddp.backend nccl
```

--------------------------------

### Multi-Node Training (Node-2)

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-classification-tutorial.md

Command for Node-2 in a multi-node training setup, specifying distributed training parameters.

```bash
export CFG_FILE="PATH_TO_MODEL_CONFIGURATION_FILE"
cvnets-train --common.config-file $CFG_FILE --common.results-loc classification_results --ddp.rank 16 --ddp.world-size 32 --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT' --ddp.backend nccl
```

--------------------------------

### Multi-Node Training (Node-1)

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-classification-tutorial.md

Command for Node-1 in a multi-node training setup, specifying distributed training parameters.

```bash
export CFG_FILE="PATH_TO_MODEL_CONFIGURATION_FILE"
cvnets-train --common.config-file $CFG_FILE --common.results-loc classification_results --ddp.rank 8 --ddp.world-size 32 --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT' --ddp.backend nccl
```

--------------------------------

### Multi-Node Training (Node-0)

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-classification-tutorial.md

Command for Node-0 in a multi-node training setup, specifying distributed training parameters.

```bash
export CFG_FILE="PATH_TO_MODEL_CONFIGURATION_FILE"
cvnets-train --common.config-file $CFG_FILE --common.results-loc classification_results --ddp.rank 0 --ddp.world-size 32 --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT' --ddp.backend nccl
```

--------------------------------

### EMA and Training Statistics Configuration

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-config-files-intro.md

Enables Exponential Moving Average (EMA) for the model and defines the statistics to compute and report during training and validation, including the checkpoint metric.

```yaml
ema:
  enable: true
  momentum: 0.0005
stats:
  val: [ "loss", "top1", "top5" ]
  train: ["loss"]
  checkpoint_metric: "top1"
  checkpoint_metric_max: true
```

--------------------------------

### Finetune MobileViTv2-1.5 from ImageNet-21k-P+ImageNet-1k pre-training at 384x384 resolution

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to finetune MobileViTv2-1.5 with 4 A100 GPUs.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in21k_in1k.yaml --common.results-loc mobilevitv2_results_in21k_1k_ft_384/width_1_5_0 --common.override-kwargs model.classification.mitv2.width_multiplier=1.5 model.classification.pretrained="LOCATION_OF_IMAGENET_21k_1k_CHECKPOINT"
```

--------------------------------

### Finetune from ImageNet-21k-P+ImageNet-1k pre-training at 384x384 resolution

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to finetune MobileViTv2-2.0 with 2 A100 GPUs from ImageNet-21k-P+ImageNet-1k pre-training.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in21k_in1k.yaml --common.results-loc mobilevitv2_results_in21k_1k_ft_384/width_2_0_0 --common.override-kwargs model.classification.mitv2.width_multiplier=2.0 model.classification.pretrained="LOCATION_OF_IMAGENET_21k_1k_CHECKPOINT"
```

--------------------------------

### Single node finetuning of MobileViTv2-0.5 with 2 A100 GPUs

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command for fine-tuning MobileViTv2-0.5 with a specified width multiplier.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in1k.yaml --common.results-loc mobilevitv2_results_in1k_ft_384/width_0_5_0 --common.override-kwargs model.classification.mitv2.width_multiplier=0.5 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Single node finetuning of MobileViTv2-1.5 with 2 A100 GPUs

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command for fine-tuning MobileViTv2-1.5 with a specified width multiplier.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in1k.yaml --common.results-loc mobilevitv2_results_in1k_ft_384/width_1_5_0 --common.override-kwargs model.classification.mitv2.width_multiplier=1.5 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Image Augmentation Configuration

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/general/README-config-files-intro.md

Specifies data augmentation techniques to be used during training, such as random cropping, flipping, resizing, and center cropping.

```yaml
image_augmentation:
  random_resized_crop:
    enable: true
    interpolation: "bicubic"
  random_horizontal_flip:
    enable: true
  resize:
    enable: true
    size: 256 # shorter size is 256
    interpolation: "bicubic"
  center_crop:
    enable: true
    size: 224
```

--------------------------------

### Single node finetuning of MobileViTv2-1.00 with 2 A100 GPUs

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command for fine-tuning MobileViTv2-1.00 with a specified width multiplier.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in1k.yaml --common.results-loc mobilevitv2_results_in1k_ft_384/width_1_0_0 --common.override-kwargs model.classification.mitv2.width_multiplier=1.00 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Finetune MobileViTv2-1.5 from ImageNet-21k-P pre-training at 256x256 resolution

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to finetune MobileViTv2-1.5 with 4 A100 GPUs.

```bash
cvnets-train --common.config-file config/classification/finetune_in21k_to_1k/mobilevit_v2.yaml --common.results-loc mobilevitv2_results_in21k_ft_256/width_1_5_0 --common.override-kwargs model.classification.mitv2.width_multiplier=1.5 model.classification.pretrained="LOCATION_OF_IMAGENET_21k_CHECKPOINT"
```

--------------------------------

### MS-COCO Object Detection Training

Source: https://github.com/apple/ml-cvnets/blob/main/examples/vit/README.md

Example command for training a ViT-B/16 model with Mask R-CNN on the MS-COCO dataset, distributed across multiple nodes and GPUs.

```bash
export CFG_FILE="examples/vit/detection/mask_rcnn_vit_base_clip.yaml"
export RANK=<NODE_ID> * <NUM_GPUS_PER_NODE> # For Node-0, RANK=0; For Node-1, Rank=8, For Node-2, RANK=16, and so on.
export WORLD_SIZE=<NUM_NODES> * <NUM_GPUS_PER_NODE> # WORLD_SIZE=8 nodes * 8 GPUS per node = 64
cvnets-train --common.config-file $CFG_FILE --common.results-loc results_detection --ddp.rank $RANK --ddp.world-size $WORLD_SIZE --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT'
```

--------------------------------

### Single node finetuning of MobileViTv2-1.25 with 2 A100 GPUs

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command for fine-tuning MobileViTv2-1.25 with a specified width multiplier.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in1k.yaml --common.results-loc mobilevitv2_results_in1k_ft_384/width_1_2_5 --common.override-kwargs model.classification.mitv2.width_multiplier=1.25 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Finetune from ImageNet-21k-P pre-training at 256x256 resolution

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to finetune MobileViTv2-2.0 with 4 A100 GPUs from ImageNet-21k-P pre-training.

```bash
cvnets-train --common.config-file config/classification/finetune_in21k_to_1k/mobilevit_v2.yaml --common.results-loc mobilevitv2_results_in21k_ft_256/width_2_0_0 --common.override-kwargs model.classification.mitv2.width_multiplier=2.0 model.classification.pretrained="LOCATION_OF_IMAGENET_21k_CHECKPOINT"
```

--------------------------------

### Finetune MobileViTv2-1.75 from ImageNet-21k-P+ImageNet-1k pre-training at 384x384 resolution

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to finetune MobileViTv2-1.75 with 4 A100 GPUs.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in21k_in1k.yaml --common.results-loc mobilevitv2_results_in21k_1k_ft_384/width_1_7_5 --common.override-kwargs model.classification.mitv2.width_multiplier=1.75 model.classification.pretrained="LOCATION_OF_IMAGENET_21k_1k_CHECKPOINT"
```

--------------------------------

### Single node finetuning of MobileViTv2-0.75 with 2 A100 GPUs

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command for fine-tuning MobileViTv2-0.75 with a specified width multiplier.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in1k.yaml --common.results-loc mobilevitv2_results_in1k_ft_384/width_0_7_5 --common.override-kwargs model.classification.mitv2.width_multiplier=0.75 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Single node finetuning of MobileViTv2-1.75 with 2 A100 GPUs

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command for fine-tuning MobileViTv2-1.75 with a specified width multiplier.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in1k.yaml --common.results-loc mobilevitv2_results_in1k_ft_384/width_1_7_5 --common.override-kwargs model.classification.mitv2.width_multiplier=1.75 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Object Detection Demo Command

Source: https://github.com/apple/ml-cvnets/blob/main/examples/range_augment/README-object-detection.md

This command demonstrates how to run object detection with a pre-trained Mask R-CNN model, specifying image path, configuration file, model weights, and evaluation parameters.

```shell
export IMG_PATH="http://farm2.staticflickr.com/1173/678795137_bb1a91f659_z.jpg"
export CFG_FILE="examples/range_augment/detection/maskrcnn_resnet_50.yaml"
export MODEL_WEIGHTS="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/examples/range_augment/detection/maskrcnn_resnet_50.pt"
cvnets-eval-det --common.config-file $CFG_FILE \
 --common.results-loc results \
 --model.detection.pretrained $MODEL_WEIGHTS \
 --model.detection.n-classes 81 \
 --evaluation.detection.resize-input-images \
 --evaluation.detection.mode single_image \
 --evaluation.detection.path "${IMG_PATH}" \
 --model.detection.mask-rcnn.box-score-thresh 0.7
```

--------------------------------

### Finetune MobileViTv2-1.75 from ImageNet-21k-P pre-training at 256x256 resolution

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to finetune MobileViTv2-1.75 with 4 A100 GPUs.

```bash
cvnets-train --common.config-file config/classification/finetune_in21k_to_1k/mobilevit_v2.yaml --common.results-loc mobilevitv2_results_in21k_ft_256/width_1_7_5 --common.override-kwargs model.classification.mitv2.width_multiplier=1.75 model.classification.pretrained="LOCATION_OF_IMAGENET_21k_CHECKPOINT"
```

--------------------------------

### MobileViTv2-0.5 Training

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Single node training command for MobileViTv2-0.5 with 8 A100 GPUs.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/classification/imagenet/mobilevit_v2.yaml --common.results-loc mobilevitv2_results/width_0_5_0 --common.override-kwargs scheduler.cosine.max_lr=0.0090 scheduler.cosine.min_lr=0.00090 optim.weight_decay=0.004 model.classification.mitv2.width_multiplier=0.50
```

--------------------------------

### Citation for SSD: Single Shot Multibox Detector

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/detection/README-detection-SSD-tutorial.md

BibTeX citation for the SSD paper.

```bibtex
@inproceedings{liu2016ssd,
  title={Ssd: Single shot multibox detector},
  author={Liu, Wei and Anguelov, Dragomir and Erhan, Dumitru and Szegedy, Christian and Reed, Scott and Fu, Cheng-Yang and Berg, Alexander C},
  booktitle={European conference on computer vision},
  pages={21--37},
  year={2016},
  organization={Springer}
}
```

--------------------------------

### MobileViTv2-1.5 Training

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Single node training command for MobileViTv2-1.5 with 8 A100 GPUs.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/classification/imagenet/mobilevit_v2.yaml --common.results-loc mobilevitv2_results/width_1_5_0 --common.override-kwargs scheduler.cosine.max_lr=0.0035 scheduler.cosine.min_lr=0.00035 optim.weight_decay=0.029 model.classification.mitv2.width_multiplier=1.50
```

--------------------------------

### Single node 4-GPU training of PSPNet-MobileViTv2-1.0 on ADE20k

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/segmentation/README-segmentation-mobilevit-v2.md

This command initiates training for the PSPNet-MobileViTv2-1.0 model on the ADE20k dataset, using a single node with 4 GPUs. It requires the path to the ImageNet-1k pre-trained checkpoint.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/segmentation/ade20k/pspnet_mobilevitv2.yaml --common.results-loc pspnet_ade20k_results/width_1_0_0 --common.override-kwargs model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Single node 4-GPU training of DeepLabv3-MobileViTv2-1.0 on PASCAL VOC 2012

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/segmentation/README-segmentation-mobilevit-v2.md

This command initiates the training process for the DeepLabv3-MobileViTv2-1.0 model on the PASCAL VOC 2012 dataset using a single node with 4 GPUs. It specifies the configuration file, results location, and requires a pre-trained ImageNet-1k checkpoint.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/segmentation/pascal_voc/deeplabv3_mobilevitv2.yaml --common.results-loc deeplabv3_mobilevitv2_results/width_1_0_0 --common.override-kwargs model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Training Command

Source: https://github.com/apple/ml-cvnets/blob/main/examples/byteformer/README.md

Command to initiate training for ByteFormer models.

```bash
cvnets-train --common.config-file $CONFIG_FILE
```

--------------------------------

### MobileViTv2-1.00 Training

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Single node training command for MobileViTv2-1.00 with 8 A100 GPUs.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/classification/imagenet/mobilevit_v2.yaml --common.results-loc mobilevitv2_results/width_1_0_0 --common.override-kwargs scheduler.cosine.max_lr=0.0075 scheduler.cosine.min_lr=0.00075 optim.weight_decay=0.013 model.classification.mitv2.width_multiplier=1.00
```

--------------------------------

### Finetune MobileViTv2-2.0 from ImageNet-1k pre-training at 384x384 resolution

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to fine-tune MobileViTv2-2.0 with 2 A100 GPUs on ImageNet-1k at 384x384 resolution. Requires specifying the location of the pre-trained checkpoint.

```bash
cvnets-train --common.config-file config/classification/finetune_higher_res/mobilevit_v2_in1k.yaml --common.results-loc mobilevitv2_results_in1k_ft_384/width_2_0_0 --common.override-kwargs model.classification.mitv2.width_multiplier=2.0 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### MobileViTv2-1.25 Training

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Single node training command for MobileViTv2-1.25 with 8 A100 GPUs.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/classification/imagenet/mobilevit_v2.yaml --common.results-loc mobilevitv2_results/width_1_2_5 --common.override-kwargs scheduler.cosine.max_lr=0.0049 scheduler.cosine.min_lr=0.00049 optim.weight_decay=0.020 model.classification.mitv2.width_multiplier=1.25
```

--------------------------------

### ImageNet-21k-P Pre-training Command (Node 3)

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to initiate ImageNet-21k-P pre-training for MobileViT-v2 on Node 3.

```bash
cvnets-train --common.config-file config/classification/imagenet_21k/mobilevit_v2.yaml --common.results-loc mobilevitv2_results_in21k/width_1_5_0 --ddp.rank 24 --ddp.world-size 32 --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT' --ddp.backend nccl --common.override-kwargs optim.weight_decay=0.029 model.classification.mitv2.width_multiplier=1.5 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### ImageNet-21k-P Pre-training Command (Node 2)

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to initiate ImageNet-21k-P pre-training for MobileViT-v2 on Node 2.

```bash
cvnets-train --common.config-file config/classification/imagenet_21k/mobilevit_v2.yaml --common.results-loc mobilevitv2_results_in21k/width_1_5_0 --ddp.rank 16 --ddp.world-size 32 --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT' --ddp.backend nccl --common.override-kwargs optim.weight_decay=0.029 model.classification.mitv2.width_multiplier=1.5 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### ImageNet-21k-P Pre-training Command (Node 1)

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to initiate ImageNet-21k-P pre-training for MobileViT-v2 on Node 1.

```bash
cvnets-train --common.config-file config/classification/imagenet_21k/mobilevit_v2.yaml --common.results-loc mobilevitv2_results_in21k/width_1_5_0 --ddp.rank 8 --ddp.world-size 32 --ddp.dist-url 'tcp://IP_OF_NODE0:FREEPORT' --ddp.backend nccl --common.override-kwargs optim.weight_decay=0.029 model.classification.mitv2.width_multiplier=1.5 model.classification.pretrained="LOCATION_OF_IMAGENET_1k_CHECKPOINT"
```

--------------------------------

### Evaluation command for ImageNet-21k-P finetuning

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Command to evaluate the finetuned MobileViTv2 model.

```bash
CUDA_VISIBLE_DEVICES=0 cvnets-eval --common.config-file config/classification/finetune_in21k_to_1k/mobilevit_v2.yaml --common.results-loc mobilevitv2_results_in21k_ft_256/width_2_0_0 --model.classification.pretrained mobilevitv2_results_in21k_ft_256/width_2_0_0/checkpoint_ema_best.pt --common.override-kwargs model.classification.mitv2.width_multiplier=2.00 model.classification.finetune_pretrained_model=false
```

--------------------------------

### MobileViTv2-0.75 Training

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Single node training command for MobileViTv2-0.75 with 8 A100 GPUs.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/classification/imagenet/mobilevit_v2.yaml --common.results-loc mobilevitv2_results/width_0_7_5 --common.override-kwargs scheduler.cosine.max_lr=0.0090 scheduler.cosine.min_lr=0.00090 optim.weight_decay=0.008 model.classification.mitv2.width_multiplier=0.75
```

--------------------------------

### MobileViTv2-1.75 Training

Source: https://github.com/apple/ml-cvnets/blob/main/docs/source/en/models/classification/README-mobilevit-v2.md

Single node training command for MobileViTv2-1.75 with 8 A100 GPUs.

```bash
PYTHONWARNINGS="ignore" cvnets-train --common.config-file config/classification/imagenet/mobilevit_v2.yaml --common.results-loc mobilevitv2_results/width_1_7_5 --common.override-kwargs scheduler.cosine.max_lr=0.0026 scheduler.cosine.min_lr=0.00026 optim.weight_decay=0.039 model.classification.mitv2.width_multiplier=1.75
```