MMTracking

https://mmtracking.readthedocs.io/en/latest/
Admin
MMTracking is an open-source video object tracking library that provides implementations of video...
Tokens:159,458
Snippets:831
Trust Score:9.6
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install MMTracking

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Clone the repository and install the package in development mode.

```bash
git clone https://github.com/open-mmlab/mmtracking.git
cd mmtracking
```

```bash
pip install -r requirements/build.txt
pip install -v -e .  # or "python setup.py develop"
```

--------------------------------

### Install mmcv-full

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Install the pre-built mmcv-full package from the OpenMMLab distribution server.

```bash
# pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.10.0/index.html
```

--------------------------------

### Example of Preparing Model for Publishing

Source: https://mmtracking.readthedocs.io/en/latest/useful_tools_scripts.html

Example usage of the publish_model.py script to convert and prepare a specific model checkpoint for publishing.

```python
python tools/analysis/publish_model.py work_dirs/dff_faster_rcnn_r101_dc5_1x_imagenetvid/latest.pth dff_faster_rcnn_r101_dc5_1x_imagenetvid.pth
```

--------------------------------

### Install VOT toolkit

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Optional dependency for VOT evaluation, recommended to install before mmcv and mmdetection.

```bash
pip install git+https://github.com/votchallenge/toolkit.git
```

--------------------------------

### Run MOT Demo

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Verify the installation by running the MOT demo script.

```bash
python demo/demo_mot_vis.py configs/mot/deepsort/sort_faster-rcnn_fpn_4e_mot17-private.py --input demo/demo.mp4 --output mot.mp4
```

--------------------------------

### Resume or Load Checkpoint and Run Training in MMTracking

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/apis/train.html

Handles resuming training from a checkpoint or loading initial weights. After setup, it starts the training process using the specified workflow and total epochs.

```python
    if cfg.resume_from:
        runner.resume(cfg.resume_from)
    elif cfg.load_from:
        runner.load_checkpoint(cfg.load_from)
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
```

--------------------------------

### Full Setup Script

Source: https://mmtracking.readthedocs.io/en/latest/install.html

A complete script for setting up MMTracking with conda and CUDA 10.1.

```bash
conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab

conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.1 -c pytorch -y

pip install git+https://github.com/votchallenge/toolkit.git (optional)
# install the latest mmcv
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.6.0/index.html

# install mmdetection
pip install mmdet

# install mmtracking
git clone https://github.com/open-mmlab/mmtracking.git
cd mmtracking
pip install -r requirements/build.txt
pip install -v -e .
pip install git+https://github.com/JonathonLuiten/TrackEval.git
pip install git+https://github.com/lvis-dataset/lvis-api.git
pip install git+https://github.com/TAO-Dataset/tao.git
```

--------------------------------

### Install MMDetection

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Install MMDetection via pip or build from source for development.

```bash
pip install mmdet
```

```bash
git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection
pip install -r requirements/build.txt
pip install -v -e .  # or "python setup.py develop"
```

--------------------------------

### Test SOT Models

Source: https://mmtracking.readthedocs.io/en/latest/quick_run.html

Examples for testing SiameseRPN++ on LaSOT.

```bash
python tools/test.py configs/sot/siamese_rpn/siamese_rpn_r50_20e_lasot.py \
    --checkpoint checkpoints/siamese_rpn_r50_1x_lasot_20211203_151612-da4b3c66.pth \
    --out results.pkl \
    --eval track
```

```bash
./tools/dist_test.sh configs/sot/siamese_rpn/siamese_rpn_r50_20e_lasot.py 8 \
    --checkpoint checkpoints/siamese_rpn_r50_1x_lasot_20211203_151612-da4b3c66.pth \
    --out results.pkl \
    --eval track
```

--------------------------------

### Install MMCV

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Install the required MMCV package using the provided index URL.

```bash
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.10/index.html
```

--------------------------------

### Install Evaluation Dependencies

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Install additional dependencies required for specific dataset evaluations.

```bash
pip install git+https://github.com/JonathonLuiten/TrackEval.git
```

```bash
pip install git+https://github.com/lvis-dataset/lvis-api.git
```

```bash
pip install git+https://github.com/TAO-Dataset/tao.git
```

--------------------------------

### Test VID Models

Source: https://mmtracking.readthedocs.io/en/latest/quick_run.html

Examples for testing DFF models on ImageNet VID.

```bash
python tools/test.py configs/vid/dff/dff_faster_rcnn_r101_dc5_1x_imagenetvid.py \
    --checkpoint checkpoints/dff_faster_rcnn_r101_dc5_1x_imagenetvid_20201218_172720-ad732e17.pth \
    --out results.pkl \
    --eval bbox
```

```bash
./tools/dist_test.sh configs/vid/dff/dff_faster_rcnn_r101_dc5_1x_imagenetvid.py 8 \
    --checkpoint checkpoints/dff_faster_rcnn_r101_dc5_1x_imagenetvid_20201218_172720-ad732e17.pth \
    --out results.pkl \
    --eval bbox
```

--------------------------------

### Test MOT Models

Source: https://mmtracking.readthedocs.io/en/latest/quick_run.html

Examples for testing Tracktor on MOT17, including configuration overrides.

```bash
python tools/test.py configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot17-public-half.py \
    --eval track
```

```bash
./tools/dist_test.sh configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot17-public-half.py 8 \
    --eval track
```

```python
model = dict(
    detector=dict(
        init_cfg=dict(
            type='Pretrained',
            checkpoint='/path/to/detector_model')),
    reid=dict(
        init_cfg=dict(
            type='Pretrained',
            checkpoint='/path/to/reid_model'))
    )
```

--------------------------------

### Print Entire Configuration

Source: https://mmtracking.readthedocs.io/en/latest/useful_tools_scripts.html

Print the complete configuration file, expanding all imported settings. Supports specifying options to override default settings.

```python
python tools/analysis/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}]
```

--------------------------------

### Get Root Logger

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Gets the root logger, optionally with a specified log file and level.

```APIDOC
## get_root_logger

### Description
Get root logger.

### Method
`get_root_logger(log_file=None, log_level=20)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **log_file** (str) - Optional - File path of log. Defaults to None.
- **log_level** (int) - Optional - The level of logger. Defaults to logging.INFO (20).

### Request Example
```json
{
  "log_file": "train.log",
  "log_level": 10
}
```

### Response
#### Success Response (200)
- **logger** (logging.Logger) - The obtained logger instance.

#### Response Example
```json
{
  "logger": "<logging.Logger object>"
}
```
```

--------------------------------

### init(img, bbox)

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Initializes the single object tracker in the first frame.

```APIDOC
## init(img, bbox)

### Description
Initialize the single object tracker in the first frame.

### Parameters
#### Request Body
- **img** (Tensor) - Required - input image of shape (1, C, H, W).
- **bbox** (list | Tensor) - Required - in [cx, cy, w, h] format.
```

--------------------------------

### Initialize a Model from Config

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/apis/inference.html

Initializes a model using a configuration file and an optional checkpoint. Handles configuration merging and model weight loading. Ensure the device is correctly specified for CUDA or CPU.

```python
import logging
import os
import tempfile

import mmcv
import numpy as np
import torch
from mmcv.ops import RoIPool
from mmcv.parallel import collate, scatter
from mmcv.runner import load_checkpoint
from mmdet.datasets.pipelines import Compose

from mmtrack.models import build_model




def init_model(config,
               checkpoint=None,
               device='cuda:0',
               cfg_options=None,
               verbose_init_params=False):
    """Initialize a model from config file.

    Args:
        config (str or :obj:`mmcv.Config`): Config file path or the config
            object.
        checkpoint (str, optional): Checkpoint path. Default as None.
        cfg_options (dict, optional): Options to override some settings in
            the used config. Default to None.
        verbose_init_params (bool, optional): Whether to print the information
            of initialized parameters to the console. Default to False.

    Returns:
        nn.Module: The constructed detector.
    """
    if isinstance(config, str):
        config = mmcv.Config.fromfile(config)
    elif not isinstance(config, mmcv.Config):
        raise TypeError('config must be a filename or Config object, '
                        f'but got {type(config)})
    if cfg_options is not None:
        config.merge_from_dict(cfg_options)
    if 'detector' in config.model:
        config.model.detector.pretrained = None
    model = build_model(config.model)

    if not verbose_init_params:
        # Creating a temporary file to record the information of initialized
        # parameters. If not, the information of initialized parameters will be
        # printed to the console because of the call of
        # `mmcv.runner.BaseModule.init_weights`.
        tmp_file = tempfile.NamedTemporaryFile(delete=False)
        file_handler = logging.FileHandler(tmp_file.name, mode='w')
        model.logger.addHandler(file_handler)
        # We need call `init_weights()` to load pretained weights in MOT
        # task.
        model.init_weights()
        file_handler.close()
        model.logger.removeHandler(file_handler)
        tmp_file.close()
        os.remove(tmp_file.name)
    else:
        # We need call `init_weights()` to load pretained weights in MOT task.
        model.init_weights()

    if checkpoint is not None:
        checkpoint = load_checkpoint(model, checkpoint, map_location='cpu')
        if 'meta' in checkpoint and 'CLASSES' in checkpoint['meta']:
            model.CLASSES = checkpoint['meta']['CLASSES']
    if not hasattr(model, 'CLASSES'):
        if hasattr(model, 'detector') and hasattr(model.detector, 'CLASSES'):
            model.CLASSES = model.detector.CLASSES
        else:
            print("Warning: The model doesn't have classes")
            model.CLASSES = None
    model.cfg = config  # save the config in the model for convenience
    model.to(device)
    model.eval()
    return model

```

--------------------------------

### Install PyTorch and torchvision

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Install PyTorch packages using conda. Specific versions may be required based on your CUDA environment.

```bash
conda install pytorch torchvision -c pytorch
```

```bash
conda install pytorch==1.5 cudatoolkit=10.1 torchvision -c pytorch
```

```bash
conda install pytorch=1.3.1 cudatoolkit=9.2 torchvision=0.4.2 -c pytorch
```

--------------------------------

### Build mmtrack-serve Docker image

Source: https://mmtracking.readthedocs.io/en/latest/useful_tools_scripts.html

Build the Docker image using the provided Dockerfile.

```bash
docker build -t mmtrack-serve:latest docker/serve/
```

--------------------------------

### GET /extract_feats

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Extracts features for an image during the testing phase.

```APIDOC
## GET /extract_feats

### Description
Extract features for the input image during testing.

### Method
GET

### Parameters
#### Query Parameters
- **img** (Tensor) - Required - Shape (1, C, H, W) encoding input image.
- **img_metas** (list[dict]) - Required - List of image information dictionaries.
- **ref_img** (list[Tensor]) - Required - Reference images.
- **ref_img_metas** (list[list[list[dict]]]) - Required - Reference image metadata.
```

--------------------------------

### Inspect Configuration File

Source: https://mmtracking.readthedocs.io/en/latest/tutorials/config.html

Use this command to view the fully resolved configuration after inheritance and modifications are applied.

```bash
python tools/analysis/print_config.py /PATH/TO/CONFIG
```

--------------------------------

### Import Custom Tracker via __init__

Source: https://mmtracking.readthedocs.io/en/latest/tutorials/customize_mot_model.html

Register a custom tracker by importing it into the `mmtrack/models/mot/trackers/__init__.py` file.

```python
from .my_tracker import MyTracker

```

--------------------------------

### Test deployment with curl

Source: https://mmtracking.readthedocs.io/en/latest/useful_tools_scripts.html

Send a video file to the inference API to verify the deployment.

```bash
curl http://127.0.0.1:8080/predictions/${MODEL_NAME} -T demo/demo.mp4 -o result.mp4
```

--------------------------------

### Annotation Information Retrieval

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Functions to get specific annotation details.

```APIDOC
## Function: get_ann_info

### Description
Get COCO annotations by the information of image.

### Parameters
* **img_info** (int) - Information of image.

### Returns
* dict - Annotation information of img_info.
```

--------------------------------

### Video ID Retrieval

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Functions to get and filter video IDs.

```APIDOC
## Function: get_vid_ids

### Description
Get video ids that satisfy given filter conditions. Default return all video ids.

### Parameters
* **vidIds** (list[int]) - The given video ids. Defaults to [].

### Returns
* list[int] - Video ids.
```

```APIDOC
## Function: load_vids

### Description
Get video information of given video ids. Default return all videos information.

### Parameters
* **ids** (list[int]) - The given video ids. Defaults to [].

### Returns
* list[dict] - List of video information.
```

--------------------------------

### Prepare Training Data

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/base_sot_dataset.html

Samples two videos from the dataset and prepares them for the training pipeline.

```python
    def prepare_train_data(self, video_ind):
        """Get training data sampled from some videos. We firstly sample two
        videos from the dataset and then parse the data information. The first
        operation in the training pipeline is frames sampling.

        Args:
            video_ind (int): video index

        Returns:
            dict: training data pairs, triplets or groups.
        """
        while True:
            video_inds = random.choices(list(range(len(self))), k=2)
            pair_video_infos = []
            for video_index in video_inds:
                ann_infos = self.get_ann_infos_from_video(video_index)
                img_infos = self.get_img_infos_from_video(video_index)
                video_infos = dict(**ann_infos, **img_infos)
                self.pre_pipeline(video_infos)
                pair_video_infos.append(video_infos)
```

--------------------------------

### Run mmtrack-serve container

Source: https://mmtracking.readthedocs.io/en/latest/useful_tools_scripts.html

Execute the container with GPU support or omit --gpus for CPU-only execution.

```bash
docker run --rm \
--cpus 8 \
--gpus device=0 \
-p8080:8080 -p8081:8081 -p8082:8082 \
--mount type=bind,source=$MODEL_STORE,target=/home/model-server/model-store \
mmtrack-serve:latest
```

--------------------------------

### Get Available Device

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Retrieves an available device (CPU, CUDA, or NPU).

```APIDOC
## get_device

### Description
Returns an available device, cpu, cuda or npu.

### Method
`get_device()`

### Parameters
None

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **device** (str) - The name of the available device ('cpu', 'cuda', or 'npu').

#### Response Example
```json
{
  "device": "cuda"
}
```
```

--------------------------------

### Sample key and reference images

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/coco_video_dataset.html

Methods to sample key frames and reference frames within a video sequence.

```python
    def key_img_sampling(self, img_ids, interval=1):
        """Sampling key images."""
        return img_ids[::interval]
```

```python
    def ref_img_sampling(self,
                         img_info,
                         frame_range,
                         stride=1,
                         num_ref_imgs=1,
                         filter_key_img=True,
                         method='uniform',
                         return_key_img=True):
        """Sampling reference frames in the same video for key frame.

        Args:
            img_info (dict): The information of key frame.
            frame_range (List(int) | int): The sampling range of reference
                frames in the same video for key frame.
            stride (int): The sampling frame stride when sampling reference
                images. Default: 1.
            num_ref_imgs (int): The number of sampled reference images.
                Default: 1.
            filter_key_img (bool): If False, the key image will be in the
                sampling reference candidates, otherwise, it is exclude.
                Default: True.
            method (str): The sampling method. Options are 'uniform',
                'bilateral_uniform', 'test_with_adaptive_stride',
                'test_with_fix_stride'. 'uniform' denotes reference images are
                randomly sampled from the nearby frames of key frame.
                'bilateral_uniform' denotes reference images are randomly
                sampled from the two sides of the nearby frames of key frame.
                'test_with_adaptive_stride' is only used in testing, and
                denotes the sampling frame stride is equal to (video length /
                the number of reference images). test_with_fix_stride is only
```

--------------------------------

### GET /simple_test

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Performs video object detection testing without augmentation.

```APIDOC
## GET /simple_test

### Description
Test the video object detector without augmentation.

### Method
GET

### Parameters
#### Query Parameters
- **img** (Tensor) - Required - Shape (1, C, H, W) encoding input image.
- **img_metas** (list[dict]) - Required - List of image information dictionaries.
- **ref_img** (list[Tensor] | None) - Optional - List containing one Tensor of shape (1, N, C, H, W) encoding input reference images.
- **ref_img_metas** (list[list[list[dict]]] | None) - Optional - Image information for reference images.
- **proposals** (Tensor | None) - Optional - Override rpn proposals with custom proposals.
- **rescale** (bool) - Optional - If False, bboxes/masks fit img scale; otherwise, original image scale. Defaults to False.

### Response
#### Success Response (200)
- **result** (list[ndarray]) - The detection results.
```

--------------------------------

### Prepare GOT10k dataset

Source: https://mmtracking.readthedocs.io/en/latest/dataset.html

Unzips GOT10k data archives and generates annotation information.

```bash
bash ./tools/convert_datasets/got10k/unzip_got10k.sh ./data/got10k
python ./tools/convert_datasets/got10k/gen_got10k_infos.py -i ./data/got10k -o ./data/got10k/annotations
```

--------------------------------

### Create dataset index

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/parsers/coco_video_parser.html

Builds internal mappings for videos, images, and instances from the loaded dataset.

```python
def createIndex(self):
    """Create index."""
    print('creating index...')
    anns, cats, imgs, vids = {}, {}, {}, {}
    (imgToAnns, catToImgs, vidToImgs, vidToInstances,
     instancesToImgs) = defaultdict(list), defaultdict(list), defaultdict(
         list), defaultdict(list), defaultdict(list)

    if 'videos' not in self.dataset and self.load_img_as_vid:
        self.dataset = self.convert_img_to_vid(self.dataset)

    if 'videos' in self.dataset:
        for video in self.dataset['videos']:
            vids[video['id']] = video

    if 'annotations' in self.dataset:
        for ann in self.dataset['annotations']:
            imgToAnns[ann['image_id']].append(ann)
            anns[ann['id']] = ann
            if 'instance_id' in ann:
                instancesToImgs[ann['instance_id']].append(ann['image_id'])
                if 'video_id' in ann and \
                    ann['instance_id'] not in \
                        vidToInstances[ann['video_id']]:
                    vidToInstances[ann['video_id']].append(
                        ann['instance_id'])

    if 'images' in self.dataset:
        for img in self.dataset['images']:
            vidToImgs[img['video_id']].append(img)
            imgs[img['id']] = img

    if 'categories' in self.dataset:
        for cat in self.dataset['categories']:
            cats[cat['id']] = cat

    if 'annotations' in self.dataset and 'categories' in self.dataset:
        for ann in self.dataset['annotations']:
            catToImgs[ann['category_id']].append(ann['image_id'])

    print('index created!')

    self.anns = anns
    self.imgToAnns = imgToAnns
    self.catToImgs = catToImgs
    self.imgs = imgs
    self.cats = cats
    self.videos = vids
    self.vidToImgs = vidToImgs
    self.vidToInstances = vidToInstances
    self.instancesToImgs = instancesToImgs
```

--------------------------------

### GET /get_ann_info

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/coco_video_dataset.html

Retrieves COCO annotations associated with a specific image information object.

```APIDOC
## GET /get_ann_info

### Description
Retrieves COCO annotations by the information of the image.

### Method
GET

### Endpoint
/get_ann_info

### Parameters
#### Request Body
- **img_info** (int) - Required - Information of the image.
```

--------------------------------

### Get Video Visibility Information

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/base_sot_dataset.html

Initializes visibility status for all frames in a video as True.

```python
    def get_visibility_from_video(self, video_ind):
        """Get the visible information of instance in a video."""
        visible = np.array([True] * self.get_len_per_video(video_ind))
        return dict(visible=visible)
```

--------------------------------

### Prepare OTB100 dataset

Source: https://mmtracking.readthedocs.io/en/latest/dataset.html

Unzips OTB100 files and downloads the pre-generated information file.

```bash
bash ./tools/convert_datasets/otb100/unzip_otb100.sh ./data/otb100
wget https://download.openmmlab.com/mmtracking/data/otb100_infos.txt -P data/otb100/annotations
```

--------------------------------

### Get Length of DistributedQuotaSampler

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/samplers/quota_sampler.html

Returns the number of samples per epoch for the current replica.

```python
    def __len__(self):
        return self.num_samples
```

--------------------------------

### QuasiDenseEmbedHead Initialization

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Initializes the QuasiDenseEmbedHead with configurable parameters for embedding channels, softmax temperature, and loss functions. Defaults are provided for most parameters.

```python
class QuasiDenseEmbedHead(_embed_channels =256_, _softmax_temp =- 1_, _loss_track ={'loss_weight': 0.25, 'type': 'MultiPosCrossEntropyLoss'}_, _loss_track_aux ={'hard_mining': True, 'loss_weight': 1.0, 'margin': 0.3, 'sample_ratio': 3, 'type': 'L2Loss'}_, _init_cfg ={'bias': 0, 'distribution': 'uniform', 'layer': 'Linear', 'override': {'bias': 0, 'mean': 0, 'name': 'fc_embed', 'std': 0.01, 'type': 'Normal'}, 'type': 'Xavier'}_, _* args_, _** kwargs_)
```

--------------------------------

### SiameseRPNHead Initialization

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Initializes the SiameseRPNHead with configurations for anchor generation, loss functions, and training/testing parameters.

```APIDOC
## SiameseRPNHead

### Description
Siamese RPN head.

### Parameters
- **anchor_generator** (dict) - Configuration of anchor generator.
- **in_channels** (int) - Number of input channels.
- **kernel_size** (int) - Size of the convolutional kernel. Defaults to 3.
- **norm_cfg** (dict) - Configuration of normalization layer. Defaults to {'type': 'BN'}.
- **weighted_sum** (bool) - Whether to use weighted sum for features. Defaults to False.
- **bbox_coder** (dict) - Configuration of bbox coder. Defaults to {'target_means': [0.0, 0.0, 0.0, 0.0], 'target_stds': [1.0, 1.0, 1.0, 1.0], 'type': 'DeltaXYWHBBoxCoder'}.
- **loss_cls** (dict) - Configuration of classification loss. Defaults to {'loss_weight': 1.0, 'reduction': 'sum', 'type': 'CrossEntropyLoss'}.
- **loss_bbox** (dict) - Configuration of bounding box regression loss. Defaults to {'loss_weight': 1.2, 'reduction': 'sum', 'type': 'L1Loss'}.
- **train_cfg** (dict) - Configuration when training. Defaults to None.
- **test_cfg** (dict) - Configuration when testing. Defaults to None.
- **init_cfg** (dict) - Configuration of initialization. Defaults to None.
```

--------------------------------

### SiameseRPNOptimizerHook

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/core/optimizer/sot_optimizer_hook.html

Optimizer hook for siamese rpn. It controls when to start training the backbone layers.

```APIDOC
## SiameseRPNOptimizerHook

### Description
Optimizer hook for siamese rpn. It controls when to start training the backbone layers.

### Method
`__init__`

### Parameters
- **backbone_start_train_epoch** (int) - Required - Start to train the backbone at `backbone_start_train_epoch`-th epoch.
- **backbone_train_layers** (list(str)) - Required - List of str denoting the stages needed be trained in backbone.

### Method
`before_train_epoch`

### Description
If `runner.epoch >= self.backbone_start_train_epoch`, start to train the backbone.

### Parameters
- **runner** (object) - Required - The runner object containing training state.

### Request Example
```python
# This is a conceptual example as this hook is used internally by the runner
# The actual instantiation would be part of the training configuration.
optimizer_hook_config = {
    "type": "SiameseRPNOptimizerHook",
    "backbone_start_train_epoch": 5,
    "backbone_train_layers": ["layer1", "layer2"]
}
```

### Response
#### Success Response (None)
This method does not return a value. It modifies the `requires_grad` attribute of model parameters.

#### Response Example
(No direct response, modifies model parameters in-place)
```

--------------------------------

### Launch Slurm Training Jobs with Configured Ports

Source: https://mmtracking.readthedocs.io/en/latest/quick_run.html

Launch Slurm training jobs using configuration files that have specific communication ports defined. This example uses `config1.py` and `config2.py` with different ports.

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config1.py ${WORK_DIR}
CUDA_VISIBLE_DEVICES=4,5,6,7 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py ${WORK_DIR}
```

--------------------------------

### SiameseRPNFp16OptimizerHook

Source: https://mmtracking.readthedocs.io/en/latest/api.html

FP16Optimizer hook for Siamese RPN. It controls when to start training the backbone based on the epoch.

```python
class mmtrack.core.optimizer.SiameseRPNFp16OptimizerHook(_backbone_start_train_epoch_ , _backbone_train_layers_ , _** kwargs_)
```

```python
before_train_epoch(_runner_)
```

--------------------------------

### Import Custom ReID Model via __init__

Source: https://mmtracking.readthedocs.io/en/latest/tutorials/customize_mot_model.html

Register a custom ReID model by importing it into the `mmtrack/models/reid/__init__.py` file.

```python
from .my_reid import MyReID

```

--------------------------------

### RandomSampleConcatDataset Class

Source: https://mmtracking.readthedocs.io/en/latest/api.html

A wrapper of concatenated dataset that supports randomly sampling one dataset and then getting samples from it.

```APIDOC
## RandomSampleConcatDataset

### Description
A wrapper of concatenated dataset. Support randomly sampling one dataset from concatenated datasets and then getting samples from the sampled dataset.

### Parameters
- **dataset_cfgs** - Configurations for the concatenated datasets.
- **dataset_sampling_weights** (list[float], optional) - Weights for sampling each dataset. Defaults to None.
```

--------------------------------

### SiameseRPNOptimizerHook

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Optimizer hook for Siamese RPN. Similar to SiameseRPNFp16OptimizerHook, it controls backbone training start epoch.

```python
class mmtrack.core.optimizer.SiameseRPNOptimizerHook(_backbone_start_train_epoch_ , _backbone_train_layers_ , _** kwargs_)
```

```python
before_train_epoch(_runner_)
```

--------------------------------

### Train a New Model with Custom Config

Source: https://mmtracking.readthedocs.io/en/latest/quick_run.html

Use this command to train a new model using a customized configuration file. Ensure the configuration file is correctly prepared.

```python
python tools/train.py ${NEW_CONFIG_FILE}

```

--------------------------------

### SiameseRPNFp16OptimizerHook

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/core/optimizer/sot_optimizer_hook.html

FP16Optimizer hook for siamese rpn. It controls when to start training the backbone layers with FP16 precision.

```APIDOC
## SiameseRPNFp16OptimizerHook

### Description
FP16Optimizer hook for siamese rpn. It controls when to start training the backbone layers with FP16 precision.

### Method
`__init__`

### Parameters
- **backbone_start_train_epoch** (int) - Required - Start to train the backbone at `backbone_start_train_epoch`-th epoch.
- **backbone_train_layers** (list(str)) - Required - List of str denoting the stages needed be trained in backbone.

### Method
`before_train_epoch`

### Description
If `runner.epoch >= self.backbone_start_train_epoch`, start to train the backbone.

### Parameters
- **runner** (object) - Required - The runner object containing training state.

### Request Example
```python
# This is a conceptual example as this hook is used internally by the runner
# The actual instantiation would be part of the training configuration.
optimizer_hook_config = {
    "type": "SiameseRPNFp16OptimizerHook",
    "backbone_start_train_epoch": 5,
    "backbone_train_layers": ["layer1", "layer2"]
}
```

### Response
#### Success Response (None)
This method does not return a value. It modifies the `requires_grad` attribute of model parameters.

#### Response Example
(No direct response, modifies model parameters in-place)
```

--------------------------------

### Define a Custom Track Head

Source: https://mmtracking.readthedocs.io/en/latest/tutorials/customize_mot_model.html

Implement a new track head, inheriting from `mmcv.runner.BaseModule`. This example defines `MyHead`.

```python
from mmcv.runner import BaseModule

from mmdet.models import HEADS

@HEADS.register_module()
class MyHead(BaseModule):

    def __init__(self,
                arg1,
                arg2):
        pass

    def forward(self, inputs):
        # implementation is ignored
        pass

```

--------------------------------

### Define a Custom ReID Model

Source: https://mmtracking.readthedocs.io/en/latest/tutorials/customize_mot_model.html

Implement a new ReID model, inheriting from `mmcv.runner.BaseModule`. This example defines `MyReID`.

```python
from mmcv.runner import BaseModule

from ..builder import REID

@REID.register_module()
class MyReID(BaseModule):

    def __init__(self,
                arg1,
                arg2):
        pass

    def forward(self, inputs):
        # implementation is ignored
        pass

```

--------------------------------

### Prepare Model for Publishing

Source: https://mmtracking.readthedocs.io/en/latest/useful_tools_scripts.html

Convert model weights to CPU tensors, delete optimizer states, and append a hash ID to the checkpoint filename for publishing.

```python
python tools/analysis/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME}
```

--------------------------------

### Configure Custom Dataset in MMTracking

Source: https://mmtracking.readthedocs.io/en/latest/tutorials/customize_dataset.html

Example configuration for using a custom dataset in CocoVID format with 5 classes.

```python
...
# dataset settings
dataset_type = 'CocoVideoDataset'
classes = ('a', 'b', 'c', 'd', 'e')
...
data = dict(
    samples_per_gpu=2,
    workers_per_gpu=2,
    train=dict(
        type=dataset_type,
        classes=classes,
        ann_file='path/to/your/train/data',
        ...),
    val=dict(
        type=dataset_type,
        classes=classes,
        ann_file='path/to/your/val/data',
        ...),
    test=dict(
        type=dataset_type,
        classes=classes,
        ann_file='path/to/your/test/data',
        ...))
...
```

--------------------------------

### Build MMCV from Source

Source: https://mmtracking.readthedocs.io/en/latest/install.html

Compile MMCV from the source repository.

```bash
git clone https://github.com/open-mmlab/mmcv.git
cd mmcv
MMCV_WITH_OPS=1 pip install -e .  # package mmcv-full will be installed after this step
cd ..
```

--------------------------------

### Configuration Naming Convention

Source: https://mmtracking.readthedocs.io/en/latest/tutorials/config.html

The standard format for naming configuration files to ensure consistency across the project.

```text
{model}_[model setting]_{backbone}_{neck}_[norm setting]_[misc]_[gpu x batch_per_gpu]_{schedule}_{dataset}
```

--------------------------------

### Get Number of Frames in a Video

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/sot_imagenet_vid_dataset.html

This function retrieves the total number of frames for a specified video index within the dataset.

```APIDOC
## GET /api/videos/{video_ind}/frames

### Description
Retrieves the number of frames for a specific video.

### Method
GET

### Endpoint
/api/videos/{video_ind}/frames

### Parameters
#### Path Parameters
- **video_ind** (int) - Required - The index of the video for which to retrieve the frame count.

### Response
#### Success Response (200)
- **frame_count** (int) - The total number of frames in the specified video.

#### Response Example
```json
{
  "frame_count": 150
}
```
```

--------------------------------

### STARK Model Initialization

Source: https://mmtracking.readthedocs.io/en/latest/api.html

Initializes the STARK model for visual tracking.

```APIDOC
## STARK Model Initialization

### Description
Initializes the STARK (Spatio-Temporal Transformer for Visual Tracking) model.

### Parameters
- **backbone** (dict) - Required - Configuration of backbone network.
- **neck** (dict) - Optional - Configuration of neck network.
- **head** (dict) - Optional - Configuration of head network.
- **init_cfg** (dict) - Optional - Configuration of initialization.
- **frozen_modules** (str | list | tuple) - Optional - Names of frozen modules.
- **train_cfg** (dict) - Optional - Configuration of training.
- **test_cfg** (dict) - Optional - Configuration of testing.
```

--------------------------------

### GET /load_vids

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/parsers/coco_video_parser.html

Retrieves video information for specified video IDs. If no IDs are provided, it returns information for all available videos.

```APIDOC
## GET /load_vids

### Description
Retrieves video information for a given list of video IDs. If the input is empty, it returns all video information.

### Method
GET

### Endpoint
/load_vids

### Parameters
#### Query Parameters
- **ids** (list[int]) - Optional - The list of video IDs to retrieve. Defaults to an empty list.

### Response
#### Success Response (200)
- **videos** (list[dict]) - A list of dictionaries containing video information.
```

--------------------------------

### Get Length Per Video

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/sot_coco_dataset.html

Returns the number of frames in a video instance. For SOTCocoDataset, each instance corresponds to a single frame.

```python


[docs]    def get_len_per_video(self, video_ind):
        """Get the number of frames in a video."""
        return 1

```

--------------------------------

### Configure DataLoader and Worker Initialization

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/builder.html

Sets up the DataLoader with appropriate samplers and persistent worker settings based on the PyTorch version.

```python
                    num_samples=samples_per_epoch)
            else:
                sampler = GroupSampler(dataset, samples_per_gpu)
        # ----- non-distributed test mode ------
        else:
            sampler = SOTVideoSampler(dataset) if is_sot_dataset else None

        batch_size = num_gpus * samples_per_gpu
        num_workers = num_gpus * workers_per_gpu

    init_fn = partial(
        worker_init_fn, num_workers=num_workers, rank=rank,
        seed=seed) if seed is not None else None

    if (TORCH_VERSION != 'parrots'
            and digit_version(TORCH_VERSION) >= digit_version('1.7.0')):
        kwargs['persistent_workers'] = persistent_workers
    elif persistent_workers is True:
        warnings.warn('persistent_workers is invalid because your pytorch '
                      'version is lower than 1.7.0')

    data_loader = DataLoader(
        dataset,
        batch_size=batch_size,
        sampler=sampler,
        num_workers=num_workers,
        collate_fn=partial(collate, samples_per_gpu=samples_per_gpu),
        pin_memory=False,
        worker_init_fn=init_fn,
        **kwargs)

    return data_loader
```

--------------------------------

### Prepare Test Image Data

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/coco_video_dataset.html

A convenience method to get testing data after the pipeline has been applied. It simply calls the `prepare_data` method.

```python
def prepare_test_img(self, idx):
        """Get testing data after pipeline.

        Args:
            idx (int): Index of data.

        Returns:
            dict: Testing data after pipeline with new keys intorduced by
            pipeline.
        """
        return self.prepare_data(idx)
```

--------------------------------

### Test Models via CLI

Source: https://mmtracking.readthedocs.io/en/latest/quick_run.html

Commands for testing models on single or multiple GPUs.

```bash
# single-gpu testing
python tools/test.py ${CONFIG_FILE} [--checkpoint ${CHECKPOINT_FILE}] [--out ${RESULT_FILE}] [--eval ${EVAL_METRICS}]

# multi-gpu testing
./tools/dist_test.sh ${CONFIG_FILE} ${GPU_NUM} [--checkpoint ${CHECKPOINT_FILE}] [--out ${RESULT_FILE}] [--eval ${EVAL_METRICS}]
```

--------------------------------

### Get Visibility Information for Instances

Source: https://mmtracking.readthedocs.io/en/latest/_modules/mmtrack/datasets/sot_imagenet_vid_dataset.html

This function calculates and returns the visibility status for instances within a given frame, considering occlusion.

```APIDOC
## POST /api/frames/{frame_id}/visibility

### Description
Calculates and returns the visibility status for instances in a frame.

### Method
POST

### Endpoint
/api/frames/{frame_id}/visibility

### Parameters
#### Path Parameters
- **frame_id** (int) - Required - The ID of the frame to process.

#### Request Body
- **instance_id** (int) - Required - The ID of the instance to check visibility for.

### Request Example
```json
{
  "instance_id": 123
}
```

### Response
#### Success Response (200)
- **visible** (array[bool]) - An array indicating the visibility of instances. `True` if visible, `False` if occluded.

#### Response Example
```json
{
  "visible": [true, false, true]
}
```
```

--------------------------------

### Prepare TrackingNet dataset

Source: https://mmtracking.readthedocs.io/en/latest/dataset.html

Unzips TrackingNet files and generates the necessary annotation information.

```bash
bash ./tools/convert_datasets/trackingnet/unzip_trackingnet.sh ./data/trackingnet
python ./tools/convert_datasets/trackingnet/gen_trackingnet_infos.py -i ./data/trackingnet -o ./data/trackingnet/annotations
```
Tokens:
Raw