### Install MMRazor and Dependencies

Source: https://context7.com/open-mmlab/mmrazor/llms.txt

Install MMCV and MMEngine using MIM, then install MMRazor from source for development or as a package.

```shell
pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0"

# Step 1a — editable install from source (recommended for development)
git clone -b main https://github.com/open-mmlab/mmrazor.git
cd mmrazor
pip install -v -e .

# Step 1b — install as a package
pip install "mmrazor>=1.0.0"
```

--------------------------------

### ConfigurableDistiller Example Configuration

Source: https://context7.com/open-mmlab/mmrazor/llms.txt

Example configuration for distilling ResNet-50 to ResNet-18 using feature-map losses. Specifies teacher/student recorders, distillation losses, loss-to-feature mappings, and optional connectors.

```python
# Example config: distill ResNet-50 → ResNet-18 using feature-map losses
distiller = dict(
    type='ConfigurableDistiller',
    teacher_recorders=dict(
        fc=dict(type='ModuleOutputs', source='head.fc'),
        feat=dict(type='ModuleOutputs', source='backbone.layer4')),
    student_recorders=dict(
        fc=dict(type='ModuleOutputs', source='head.fc'),
        feat=dict(type='ModuleOutputs', source='backbone.layer4')),
    distill_losses=dict(
        loss_kl=dict(type='KLDivergence', tau=1, loss_weight=1.0),
        loss_feat=dict(type='L2Loss', loss_weight=0.5)),
    loss_forward_mappings=dict(
        loss_kl=dict(
            preds_S=dict(recorder='fc', from_student=True),
            preds_T=dict(recorder='fc', from_student=False)),
        loss_feat=dict(
            s_feature=dict(recorder='feat', from_student=True),
            t_feature=dict(recorder='feat', from_student=False))),
    connectors=dict(
        # project student 256-ch features to match teacher 512-ch
        feat_connector=dict(
            type='ConvModuleConnector',
            in_channel=256, out_channel=512, kernel_size=1)))

model = dict(
    type='mmrazor.SingleTeacherDistill',
    architecture=dict(       # student
        type='mmcls.ImageClassifier',
        backbone=dict(type='ResNet', depth=18), ...),
    teacher=dict(            # teacher (frozen)
        type='mmcls.ImageClassifier',
        backbone=dict(type='ResNet', depth=50), ...),
    teacher_ckpt='checkpoints/resnet50_in1k.pth',
    distiller=distiller)
```

--------------------------------

### Install and Initialize Pre-commit Hook

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/notes/contribution_guide.md

Install the pre-commit package and then initialize the pre-commit hook within the repository. This ensures code linters and formatters are enforced on every commit.

```shell
pip install -U pre-commit
```

```shell
pre-commit install
```

--------------------------------

### Run LLaMA GPTQ Example

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/examples/language_models/LLaMA/README.md

Execute the GPTQ example for LLaMA models. Specify the model path and the dataset to use for calibration.

```shell
# example for decapoda-research/llama-7b-hf
python projects/mmrazor_large/examples/language_models/LLaMA/llama_gptq.py decapoda-research/llama-7b-hf c4
```

```shell
# help
usage: llama_gptq.py [-h] [--seed SEED] [--nsamples NSAMPLES] [--batch_size BATCH_SIZE] [--save SAVE] [-m M] model {wikitext2,ptb,c4}

positional arguments:
  model                 Llama model to load
  {wikitext2,ptb,c4}    Where to extract calibration data from.

optional arguments:
  -h, --help            show this help message and exit
  --seed SEED           Seed for sampling the calibration data.
  --nsamples NSAMPLES   Number of calibration data samples.
  --batch_size BATCH_SIZE
                        Batchsize for calibration and evaluation.
  --save SAVE           Path to saved model.
  -m M                  Whether to enable memory efficient forward
```

--------------------------------

### Run LLaMA SparseGPT Example

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/examples/language_models/LLaMA/README.md

Execute the SparseGPT example for LLaMA models. Specify the model path and the dataset to use for calibration.

```shell
# example for decapoda-research/llama-7b-hf
python projects/mmrazor_large/examples/language_models/LLaMA/llama_sparse_gpt.py decapoda-research/llama-7b-hf c4
```

```shell
# help
usage: llama_sparse_gpt.py [-h] [--seed SEED] [--nsamples NSAMPLES] [--batch_size BATCH_SIZE] [--save SAVE] [-m M] model {wikitext2,ptb,c4}

positional arguments:
  model                 Llama model to load
  {wikitext2,ptb,c4}    Where to extract calibration data from.

optional arguments:
  -h, --help            show this help message and exit
  --seed SEED           Seed for sampling the calibration data.
  --nsamples NSAMPLES   Number of calibration data samples.
  --batch_size BATCH_SIZE
                        Batchsize for calibration and evaluation.
  --save SAVE           Path to saved model.
  -m M                  Whether to enable memory efficient forward
```

--------------------------------

### Install MMRazor from Source

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/get_started/installation.md

Clone the MMRazor repository and install it in editable mode for development. The '-v' flag provides verbose output, and '-e' installs the package in editable mode, meaning local changes are reflected without reinstallation.

```shell
git clone -b main https://github.com/open-mmlab/mmrazor.git
cd mmrazor
pip install -v -e .
```

--------------------------------

### Run GPTQ on OPT Model

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/examples/language_models/OPT/README.md

Execute the GPTQ example script for OPT models. Specify the OPT model name and the dataset for calibration.

```shell
python projects/mmrazor_large/examples/language_models/OPT/opt_gptq.py facebook/opt-125m c4
```

--------------------------------

### Install MMCV and MMengine with MIM

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/get_started/installation.md

Install the necessary dependencies, MMCV and MMengine, using the MIM package manager.

```shell
pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0"
```

--------------------------------

### Run SparseGPT on OPT Model

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/examples/language_models/OPT/README.md

Execute the SparseGPT example script for OPT models. Specify the OPT model name and the dataset for calibration.

```shell
python projects/mmrazor_large/examples/language_models/OPT/opt_sparse_gpt.py facebook/opt-125m c4
```

--------------------------------

### Install PyTorch on CPU

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/get_started/installation.md

Install PyTorch and Torchvision for CPU-only platforms using conda.

```shell
conda install pytorch torchvision cpuonly -c pytorch
```

--------------------------------

### Run ResNet18 with GPTQ

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/examples/ResNet/README.md

Execute the GPTQ example for ResNet18. The imagenet folder must be in torch format. You can modify the batch size and number of samples.

```shell
python projects/mmrazor_large/examples/ResNet/resnet18_gptq.py --data {imagenet_path} --batchsize 128 --num_samples 512
```

--------------------------------

### Default Search Groups Example

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/mutator.md

Demonstrates how `search_groups` are automatically determined when each `OneShotMutableModule` is treated as a separate group.

```Python
from mmrazor.models import OneShotModuleMutator, OneShotMutableModule

class SearchableModel(nn.Module):
    def __init__(self, one_shot_op_cfg):
        # assume `OneShotMutableModule` contains 4 choices:
        # choice1, choice2, choice3 and choice4
        self.choice_block1 = OneShotMutableModule(**one_shot_op_cfg)
        self.choice_block2 = OneShotMutableModule(**one_shot_op_cfg)
        self.choice_block3 = OneShotMutableModule(**one_shot_op_cfg)

    def forward(self, x: Tensor) -> Tensor:
        x = self.choice_block1(x)
        x = self.choice_block2(x)
        x = self.choice_block3(x)

        return x

supernet = SearchableModel(one_shot_op_cfg)
mutator1 = OneShotModuleMutator()
# build mutator1 from supernet.
mutator1.prepare_from_supernet(supernet)
>>> mutator1.search_groups.keys()
dict_keys([0, 1, 2])
```

--------------------------------

### Basic Augmentation Example

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/delivery.md

Demonstrates the use of Augments for data augmentation. The output shows that the results are different due to the randomness of mixup.

```python
from mmcls.models.utils import Augments
from mmrazor.core import MethodOutputsDelivery

augments_cfg = dict(type='BatchMixup', alpha=1., num_classes=10, prob=1.0)
augments = Augments(augments_cfg)

imgs = torch.randn(2, 3, 32, 32)
label = torch.randint(0, 10, (2,))

imgs_teacher, label_teacher = augments(imgs, label)
imgs_student, label_student = augments(imgs, label)

print(torch.equal(label_teacher, label_student))
print(torch.equal(imgs_teacher, imgs_student))
```

--------------------------------

### Get Channel Units with Init Args

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/tutorials/how_to_use_config_tool_of_pruning.md

Retrieves the channel unit configuration including initialization arguments by using the `-i` flag with the `get_channel_units.py` script.

```shell
python ./tools/pruning/get_channel_units.py -i ./configs/pruning/mmcls/autoslim/autoslim_mbv2_1.5x_slimmable_subnet_8xb256_in1k.py
```

--------------------------------

### Distributed Training on 2 Machines x 8 GPUs (Machine 0)

Source: https://context7.com/open-mmlab/mmrazor/llms.txt

Command for machine 0 in a distributed setup involving two machines, each with 8 GPUs. Requires setting environment variables for node count, rank, port, and master address.

```shell
NNODES=2 NODE_RANK=0 PORT=29500 MASTER_ADDR=192.168.1.1 \
  sh tools/dist_train.sh configs/distill/mmcls/kd/kd_logits_resnet34_resnet18_8xb32_in1k.py 8
```

--------------------------------

### Get L1 Prune Config Help

Source: https://github.com/open-mmlab/mmrazor/blob/main/demo/pruning/config_pruning.ipynb

Displays the help message for the 'get_l1_prune_config.py' script, outlining its usage and available arguments.

```bash
! python ./tools/pruning/get_l1_prune_config.py -h
```

--------------------------------

### Configure Quantization Components

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_quantization_algorithms.md

Configure global quantization settings, the model architecture with a quantizer, and the training loop for custom QAT algorithms. This setup is typically done within a Python configuration file.

```Python
global_qconfig = dict(
    w_observer=dict(),
    a_observer=dict(),
    w_fake_quant=dict(),
    a_fake_quant=dict(),
    w_qscheme=dict(),
    a_qscheme=dict(),
)
model = dict(
    type='mmrazor.MMArchitectureQuant',
    architecture=resnet,
    quantizer=dict(
        type='mmrazor.OpenvinoQuantizer',
        global_qconfig=global_qconfig,
        tracer=dict()))
train_cfg = dict(type='mmrazor.LSQEpochBasedLoop')
```

--------------------------------

### Run ResNet18 with SparseGPT

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/examples/ResNet/README.md

Execute the SparseGPT example for ResNet18. Ensure the imagenet folder follows the torch format. Adjust batch size and number of samples as needed.

```shell
python projects/mmrazor_large/examples/ResNet/resnet18_sparse_gpt.py --data {imagenet_path} --batchsize 128 --num_samples 512
```

--------------------------------

### Distributed Training on 2 Machines x 8 GPUs (Machine 1)

Source: https://context7.com/open-mmlab/mmrazor/llms.txt

Command for machine 1 in a distributed setup involving two machines, each with 8 GPUs. Similar environment variables are required, with NODE_RANK set to 1.

```shell
NNODES=2 NODE_RANK=1 PORT=29500 MASTER_ADDR=192.168.1.1 \
  sh tools/dist_train.sh configs/distill/mmcls/kd/kd_logits_resnet34_resnet18_8xb32_in1k.py 8
```

--------------------------------

### Launch multiple jobs on a single machine (dist_train.sh)

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/user_guides/3_train_with_different_devices.md

Launch multiple training jobs on a single machine by specifying different ports to avoid communication conflicts. This example uses `dist_train.sh`.

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=29500 sh tools/xxx/dist_train.sh ${CONFIG_FILE} 4 --work_dir tmp_work_dir_1
```

```bash
CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=29501 sh tools/xxx/dist_train.sh ${CONFIG_FILE} 4 --work_dir tmp_work_dir_2
```

--------------------------------

### Prepare and Load Pretrain Configuration

Source: https://github.com/open-mmlab/mmrazor/blob/main/demo/pruning/config_pruning.ipynb

Sets up a base configuration for pre-training by writing a config string to a file and then loading it using `Config.fromfile`. This demonstrates how to integrate base configurations.

```python
# Prepare pretrain config
pretrain_config_path = f'{work_dir}/pretrain.py'
config_string = """
_base_ = ['mmcls::resnet/resnet34_8xb32_in1k.py']
"""
write_config(config_string, pretrain_config_path)
print(Config.fromfile(pretrain_config_path)['model'])
```

--------------------------------

### Build and Prepare Model with ChannelMutator

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/notes/changelog.md

Demonstrates how to build a model using the registry and prepare it for pruning using ChannelMutator. This is useful for understanding channel dependencies in a supernet.

```python
from mmrazor.registry import MODELS

ARCHITECTURE_CFG = dict(
    _scope_='mmcls',
    type='ImageClassifier',
    backbone=dict(type='MobileNetV2', widen_factor=1.5),
    neck=dict(type='GlobalAveragePooling'),
    head=dict(type='mmcls.LinearClsHead', num_classes=1000, in_channels=1920))
model = MODELS.build(ARCHITECTURE_CFG)
from mmrazor.models.mutators import ChannelMutator

channel_mutator = ChannelMutator()
channel_mutator.prepare_from_supernet(model)
units = channel_mutator.mutable_units
print(units[0])
```

--------------------------------

### Install PyTorch on GPU

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/get_started/installation.md

Install PyTorch and Torchvision for GPU platforms using conda.

```shell
conda install pytorch torchvision -c pytorch
```

--------------------------------

### Initialize and Prepare GPTQ Compressor

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/algorithms/GPTQ.md

Initializes the GPTQ compressor and prepares the model for quantization. This is the first step in applying the GPTQ algorithm.

```python
from mmrazor.implementations.quantization import gptq

# initial model, dataloaders
model
train_loader, test_loader

## init gptq compressor and prepare for quantization
compressor = gptq.GPTQCompressor()
compressor.prepare(model)
```

--------------------------------

### Install MMRazor with Pip

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/get_started/installation.md

Install MMRazor as a dependency or third-party package using pip. Ensure you have MMRazor version 1.0.0 or higher.

```shell
pip install "mmrazor>=1.0.0"
```

--------------------------------

### Step 2: Search for Subnet

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/nas/mmcls/autoformer/README.md

Search for an optimal subnet using the pre-trained supernet. Load the checkpoint from the supernet pre-training step using `--cfg-options load_from`.

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=29500 ./tools/dist_train.sh \
  configs/nas/mmcls/autoformer/autoformer_search_8xb128_in1k.py 4 \
  --work-dir $WORK_DIR --cfg-options load_from=$STEP1_CKPT
```

--------------------------------

### Train Pretrained Model with QAT

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/quantization/qat/base/README.md

Use this command to initiate the training process for a pre-trained model using Quantization-Aware Training. Ensure the CONFIG variable points to the correct QAT configuration file.

```bash
python tools/train.py ${CONFIG}
```

--------------------------------

### Update Model Config for Quantization

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/user_guides/quantization_user_guide.md

Example of modifying the '_base_' and 'float_checkpoint' to switch from ResNet18 to ResNet50 for quantization.

```python
# before
_base_ = ['mmcls::resnet/resnet18_8xb32_in1k.py']
float_checkpoint = 'https://download.openmmlab.com/mmclassification/v0/resnet/resnet18_8xb32_in1k_20210831-fbbb1da6.pth'

# after
_base_ = ['mmcls::resnet/resnet50_8xb32_in1k.py']
float_checkpoint = 'https://download.openmmlab.com/mmclassification/v0/resnet/resnet50_8xb32_in1k_20210831-ea4938fc.pth'
```

--------------------------------

### Prepare Working Directory

Source: https://github.com/open-mmlab/mmrazor/blob/main/demo/pruning/config_pruning.ipynb

Creates a temporary directory for storing configuration files and other outputs. This prevents cluttering the root directory.

```python
# prepare work_dir
work_dir = './demo/tmp/'
if not os.path.exists(work_dir):
    os.mkdir(work_dir)
```

--------------------------------

### Import Custom Algorithm: __init__

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_kd_algorithms.md

To use a newly defined algorithm, import it in the algorithm's __init__.py file. This makes the algorithm available for use in configurations.

```Python
from .single_teacher_distill import SingleTeacherDistill

__all__ = [..., 'SingleTeacherDistill']
```

--------------------------------

### Clean Temporary Directory

Source: https://github.com/open-mmlab/mmrazor/blob/main/demo/pruning/config_pruning.ipynb

Removes the working directory to clean up temporary files before starting a new run.

```shell
! rm -r $work_dir
```

--------------------------------

### Initialize MutableChannelUnits from a Model

Source: https://github.com/open-mmlab/mmrazor/blob/main/mmrazor/models/mutables/mutable_channel/units/mutable_channel_unit.ipynb

Initializes a list of SequentialMutableChannelUnits from a given PyTorch model. This is a common starting point for channel pruning.

```python
from mmrazor.models.mutables.mutable_channel.units import SequentialMutableChannelUnit
from mmrazor.structures.graph import ModuleGraph
from typing import List

model = MyModel()
units: List[
    SequentialMutableChannelUnit] = SequentialMutableChannelUnit.init_from_channel_analyzer(model)  # type: ignore
print(
    f'This model has {len(units)} MutableChannelUnit(SequentialMutableChannelUnit).')

```

--------------------------------

### Multi-machine distributed training (First machine)

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/user_guides/3_train_with_different_devices.md

Command to initiate distributed training across multiple machines. This is executed on the first machine, setting up the master node.

```bash
NNODES=2 NODE_RANK=0 PORT=$MASTER_PORT MASTER_ADDR=$MASTER_ADDR sh tools/dist_train.sh $CONFIG $GPUS
```

--------------------------------

### BYOT Distillation Training Command

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/distill/mmcls/byot/README.md

Use this command to start the distillation training process for BYOT with a ResNet18 model on CIFAR100.

```bash
sh tools/dist_train.sh \
  configs/distill/mmcls/byot/byot_logits_resnet18_cifar100_8xb16_in1k.py 8
```

--------------------------------

### Import LSQEpochBasedLoop

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_quantization_algorithms.md

Import the custom LSQEpochBasedLoop into the mmrazor.engine.runner package.

```python
from .quantization_loops import LSQEpochBasedLoop

__all__ = ['LSQEpochBasedLoop']
```

--------------------------------

### GPTQ OPT Script Help

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/examples/language_models/OPT/README.md

Display the help message for the GPTQ OPT script, showing available arguments and their descriptions.

```shell
usage: opt_gptq.py [-h] [--seed SEED] [--nsamples NSAMPLES] [--batch_size BATCH_SIZE] [--save SAVE] [-m M] model {wikitext2,ptb,c4}

positional arguments:
  model                 OPT model to load; pass `facebook/opt-X`.
  {wikitext2,ptb,c4}    Where to extract calibration data from.

optional arguments:
  -h, --help            show this help message and exit
  --seed SEED           Seed for sampling the calibration data.
  --nsamples NSAMPLES   Number of calibration data samples.
  --batch_size BATCH_SIZE
                        Batchsize for calibration and evaluation.
  --save SAVE           Path to saved model.
  -m M                  Whether to enable memory efficient forward
```

--------------------------------

### Registering a New Mutable: OneShotMutableOP

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/mutable.md

Example of registering a new mutable class `OneShotMutableOP` by inheriting from `OneShotMutableModule` and using the `MODELS.register_module()` decorator.

```python
# Copyright (c) OpenMMLab. All rights reserved.
import random
from abc import abstractmethod
from typing import Any, Dict, List, Optional, Union

import numpy as np
import torch.nn as nn
from torch import Tensor

from mmrazor.registry import MODELS
from ..base_mutable import CHOICE_TYPE, CHOSEN_TYPE
from .mutable_module import MutableModule

@MODELS.register_module()
class OneShotMutableOP(OneShotMutableModule[str, str]):
    ...

```

--------------------------------

### SparseGPT OPT Script Help

Source: https://github.com/open-mmlab/mmrazor/blob/main/projects/mmrazor_large/examples/language_models/OPT/README.md

Display the help message for the SparseGPT OPT script, showing available arguments and their descriptions.

```shell
usage: opt_sparse_gpt.py [-h] [--seed SEED] [--nsamples NSAMPLES] [--batch_size BATCH_SIZE] [--save SAVE] [-m M] model {wikitext2,ptb,c4}

positional arguments:
  model                 OPT model to load; pass `facebook/opt-X`.
  {wikitext2,ptb,c4}    Where to extract calibration data from.

optional arguments:
  -h, --help            show this help message and exit
  --seed SEED           Seed for sampling the calibration data.
  --nsamples NSAMPLES   Number of calibration data samples.
  --batch_size BATCH_SIZE
                        Batchsize for calibration and evaluation.
  --save SAVE           Path to saved model.
  -m M                  Whether to enable memory efficient forward
```

--------------------------------

### Configure Training, Validation, and Testing Settings

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_quantization_algorithms.md

Define configurations for training, validation, and testing loops. This includes specifying the training loop type, such as mmrazor.LSQEpochBasedLoop, and setting up validation and testing configurations.

```python
optim_wrapper = dict()
param_scheduler = dict()
model_wrapper_cfg = dict()

# train, val, test setting
train_cfg = dict(type='mmrazor.LSQEpochBasedLoop')
val_cfg = dict()
test_cfg = val_cfg
```

--------------------------------

### Initialize SearchableShuffleNetV2 Backbone

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_architectures.md

Defines the `__init__` method for `SearchableShuffleNetV2`, setting up parameters like `arch_setting`, `widen_factor`, and configuration for convolutional layers, normalization, and activation functions. It includes validation for `out_indices` and `frozen_stages`.

```python
from typing import Dict, List, Optional, Sequence, Union

import torch.nn as nn
from mmcls.models.backbones.base_backbone import BaseBackbone
from mmcv.cnn import ConvModule, constant_init, normal_init
from mmcv.runner import ModuleList, Sequential
from torch import Tensor
from torch.nn.modules.batchnorm import _BatchNorm

from mmrazor.registry import MODELS

@MODELS.register_module()
class SearchableShuffleNetV2(BaseBackbone):

    def __init__(self, 
                 arch_setting: List[List], 
                 stem_multiplier: int = 1, 
                 widen_factor: float = 1.0, 
                 out_indices: Sequence[int] = (4, ), 
                 frozen_stages: int = -1, 
                 with_last_layer: bool = True, 
                 conv_cfg: Optional[Dict] = None, 
                 norm_cfg: Dict = dict(type='BN'), 
                 act_cfg: Dict = dict(type='ReLU'), 
                 norm_eval: bool = False, 
                 with_cp: bool = False, 
                 init_cfg: Optional[Union[Dict, List[Dict]]] = None) -> None:
            layers_nums = 5 if with_last_layer else 4
            for index in out_indices:
                if index not in range(0, layers_nums):
                    raise ValueError('the item in out_indices must in ' 
                                     f'range(0, 5). But received {index}')

            self.frozen_stages = frozen_stages
            if frozen_stages not in range(-1, layers_nums):
                raise ValueError('frozen_stages must be in range(-1, 5). ' 
                                 f'But received {frozen_stages}')

            super().__init__(init_cfg)

            self.arch_setting = arch_setting
            self.widen_factor = widen_factor
            self.out_indices = out_indices
            self.conv_cfg = conv_cfg
            self.norm_cfg = norm_cfg
            self.act_cfg = act_cfg
            self.norm_eval = norm_eval
            self.with_cp = with_cp

            last_channels = 1024
            self.in_channels = 16 * stem_multiplier

            # build the first layer
            self.conv1 = ConvModule(
                in_channels=3,
                out_channels=self.in_channels,
                kernel_size=3,
                stride=2,
                padding=1,
                conv_cfg=conv_cfg,
                norm_cfg=norm_cfg,
                act_cfg=act_cfg)

            # build the middle layers
            self.layers = ModuleList()
            for channel, num_blocks, mutable_cfg in arch_setting:
                out_channels = round(channel * widen_factor)
                layer = self._make_layer(out_channels, num_blocks, 
                                         copy.deepcopy(mutable_cfg))
                self.layers.append(layer)

            # build the last layer
            if with_last_layer:
                self.layers.append(
                    ConvModule(
                        in_channels=self.in_channels,
                        out_channels=last_channels,
                        kernel_size=1,
                        conv_cfg=conv_cfg,
                        norm_cfg=norm_cfg,
                        act_cfg=act_cfg))

```

--------------------------------

### Use Custom Algorithm in Config

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_pruning_algorithms.md

Configure your custom pruning algorithm in the model configuration. This example shows how to specify the `AutoSlim` type and its associated mutator.

```python
model = dict(
    type='AutoSlim',
    architecture=...,
    mutator=dict(type='OneShotChannelMutator', ...),
    )
```

--------------------------------

### Step 1: Supernet Pre-training

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/nas/mmcls/autoformer/README.md

Pre-train the AutoFormer supernet on ImageNet. Ensure distributed training is configured correctly with specified devices and port.

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=29500 ./tools/dist_train.sh \
  configs/nas/mmcls/autoformer/autoformer_supernet_32xb256_in1k.py 4 \
  --work-dir $WORK_DIR
```

--------------------------------

### Finetune Configuration Parameters

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/pruning/base/group_fisher/README.md

These parameters can be set within the finetuning configuration file to specify paths and learning rates.

```python
"""
_base_(str): The path to your pruning config file.
pruned_path (str): The path to the checkpoint of the pruned model.
finetune_lr (float): The lr rate to finetune. Usually, we directly use the lr
    rate of the pretrain.
"""
```

--------------------------------

### Custom Search Groups Example

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/mutator.md

Illustrates how to define custom search groups by passing a `custom_group` argument to the mutator, allowing modules to share search spaces.

```Python
custom_group = [
    ['op1', 'op2'],
    ['op3']
]
mutator2 = OneShotMutator(custom_group)
mutator2.prepare_from_supernet(supernet)
```

```Python
>>> mutator2.search_groups.keys()
dict_keys([0, 1])
```

--------------------------------

### Configure AutoSlim in Config File

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_mixed_algorithms.md

Example configuration for using the AutoSlim algorithm in a project. It specifies the type and essential components like architecture, mutator, and distiller.

```python
model= dict(
    type='mmrazor.AutoSlim',
    architecture=...,
    mutator=dict(
        type='OneShotChannelMutator',
        ...),
    distiller=dict(
        type='ConfigurableDistiller',
        ...),
    ...)
```

--------------------------------

### Pre-training Backbone

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/distill/mmcls/abloss/README.md

Execute the pre-training script for the backbone network. Ensure the configuration file path and the number of GPUs are correctly specified.

```bash
sh tools/dist_train.sh configs/distill/mmcls/abloss/abloss_pretrain_backbone_resnet50_resnet18_in1k.py 8
```

--------------------------------

### Get MutableChannelUnits Using Tracer

Source: https://github.com/open-mmlab/mmrazor/blob/main/mmrazor/models/mutables/mutable_channel/units/mutable_channel_unit.ipynb

Demonstrates how to obtain MutableChannelUnits by using a tracer, which converts the model to a graph and then to units. This method automatically returns all available units.

```python
# 1. using tracer
def get_mutable_channel_units_using_tracer(model):
    units = SequentialMutableChannelUnit.init_from_channel_analyzer(model)
    return units


model = MyModel()
units = get_mutable_channel_units_using_tracer(model)
print(f'The model has {len(units)} MutableChannelUnits.')
```

--------------------------------

### Distillation: Knowledge Distillation

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/user_guides/2_train_different_types_algorithms.md

Initiate knowledge distillation by running the training script with the appropriate configuration file and optional arguments.

```bash
python tools/train.py ${CONFIG_FILE} [optional arguments]
```

```bash
python ./tools/train.py \
  configs/distill/mmcls/kd/kd_logits_r34_r18_8xb32_in1k.py \
  --work-dir your_work_dir
```

--------------------------------

### Define a Sample Model Architecture

Source: https://github.com/open-mmlab/mmrazor/blob/main/mmrazor/models/mutators/channel_mutator/channel_mutator.ipynb

Defines a simple PyTorch model with sequential layers, pooling, and a linear head. This model serves as an example for demonstrating ChannelMutator functionality.

```python
from mmengine.model import BaseModel
from torch import nn
import torch
from collections import OrderedDict

class MyModel(nn.Module):

    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            OrderedDict([('conv0', nn.Conv2d(3, 8, 3, 1, 1)),
                         ('relu', nn.ReLU()),
                         ('conv1', nn.Conv2d(8, 16, 3, 1, 1))]))
        self.pool = nn.AdaptiveAvgPool2d(1)
        self.head = nn.Linear(16, 1000)

    def forward(self, x):
        feature = self.net(x)
        pool = self.pool(feature).flatten(1)
        return self.head(pool)
```

--------------------------------

### Run PTQ for Pre-trained Model

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/quantization/ptq/base/README.md

Execute Post-Training Quantization on a pre-trained model using the specified configuration file.

```bash
python tools/ptq.py ${CONFIG}
```

--------------------------------

### Import Custom Algorithm Class

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/algorithm.md

Make your custom algorithm available by importing its class in the '__init__.py' file of its directory and then in the main 'mmrazor/models/algorithms/__init__.py'.

```coffeescript
from .xxx import XXX

__all__ = ['XXX']
```

--------------------------------

### Develop New Algorithm Component (OneShotChannelMutator)

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_pruning_algorithms.md

Create a custom mutator by inheriting from `ChannelMutator` and registering it. This example shows the `OneShotChannelMutator` with essential methods like `sample_choices` and `set_choices`.

```python
from mmrazor.registry import MODELS
from .channel_mutator import ChannelMutator


@MODELS.register_module()
class OneShotChannelMutator(ChannelMutator):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    def sample_choices(self):
        pass

    def set_choices(self, choice_dict):
        pass

    # supernet is a kind of architecture in `mmrazor/models/architectures/`
    def build_search_groups(self, supernet):
        pass
```

--------------------------------

### Distillation Training

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/distill/mmcls/abloss/README.md

Run the distillation training process using the modified configuration file and specified number of GPUs.

```bash
sh tools/dist_train.sh configs/distill/mmcls/abloss/abloss_logits_resnet50_resnet18_in1k.py 8
```

--------------------------------

### Develop Custom Loss: L1Loss

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_kd_algorithms.md

Create a custom loss function by inheriting from nn.Module and registering it with @MODELS.register_module(). This example shows how to implement an L1 loss for feature distillation.

```Python
from mmrazor.registry import MODELS

@MODELS.register_module()
class L1Loss(nn.Module):
    def __init__( 
        self, 
        loss_weight: float = 1.0, 
        size_average: Optional[bool] = None, 
        reduce: Optional[bool] = None, 
        reduction: str = 'mean',
    ) -> None:
        super().__init__()
        ...

    def forward(self, s_feature, t_feature):
        loss = F.l1_loss(s_feature, t_feature, self.size_average, self.reduce,
                         self.reduction)
        return self.loss_weight * loss
```

--------------------------------

### Get Channel Units Config

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/tutorials/how_to_use_config_tool_of_pruning.md

Generates a configuration for channel units of a model using the `get_channel_units.py` script. The output includes the mutator type and channel unit configuration.

```shell
python ./tools/pruning/get_channel_units.py ./configs/pruning/mmcls/autoslim/autoslim_mbv2_1.5x_slimmable_subnet_8xb256_in1k.py
```

--------------------------------

### Execute Training with Pruning Configuration

Source: https://github.com/open-mmlab/mmrazor/blob/main/demo/pruning/config_pruning.ipynb

This command executes the training process using the specified pruning configuration file.

```bash
! timeout 2 python ./tools/train.py $prune_config_path
```

--------------------------------

### Analyze Channel Dependency with ChannelAnalyzer

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/notes/changelog.md

Use ChannelAnalyzer to automatically analyze channel dependency in CNN models. This example demonstrates analyzing a RetinaNet model from mmdet and printing the configuration.

```python
from mmrazor.models.task_modules import ChannelAnalyzer
from mmengine.hub import get_model
import json

model = get_model('mmdet::retinanet/retinanet_r18_fpn_1x_coco.py')
unit_configs: dict = ChannelAnalyzer().analyze(model)
unit_config0 = list(unit_configs.values())[0]
print(json.dumps(unit_config0, indent=4))
```

--------------------------------

### 4 GPUs Distributed Training Command

Source: https://context7.com/open-mmlab/mmrazor/llms.txt

Command to start distributed training across 4 GPUs using the provided script. The --work_dir argument specifies the output directory.

```shell
sh tools/dist_train.sh \
  configs/nas/mmcls/spos/spos_shufflenet_supernet_8xb128_in1k.py \
  4 --work_dir work_dirs/spos_4gpu
```

--------------------------------

### Get FLOPs and Parameters of Pruned Model

Source: https://github.com/open-mmlab/mmrazor/blob/main/configs/pruning/mmcls/l1-norm/README.md

Calculate the FLOPs and parameters of a pruned model using its deploy configuration file. Ensure `{pruning_deploy_config}.py` points to the correct configuration.

```bash
python ./tools/pruning/get_flops.py \
    {pruning_deploy_config}.py
```

--------------------------------

### Train with CPU

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/user_guides/3_train_with_different_devices.md

Use this command to train a model on a CPU. Note that CPU training is significantly slower and may not be compatible with all algorithms due to dependencies like SyncBN.

```bash
export CUDA_VISIBLE_DEVICES=-1
python tools/train.py ${CONFIG_FILE}
```

--------------------------------

### Import Custom Module to __init__.py

Source: https://github.com/open-mmlab/mmrazor/blob/main/docs/en/advanced_guides/customize_architectures.md

Add this line to `mmrazor/models/architectures/backbones/__init__.py` to make your custom module available.

```python
from .searchable_shufflenet_v2 import SearchableShuffleNetV2

__all__ = ['SearchableShuffleNetV2']
```