### YAML Configuration Example

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Example YAML configuration file for training on THUMOS14 with I3D features. This structure defines dataset parameters, model architecture, optimizer settings, and training/testing configurations.

```yaml
# configs/thumos_i3d.yaml — example for THUMOS14 with I3D features
dataset_name: thumos          # maps to @register_dataset("thumos")
train_split: ['validation']   # THUMOS14 uses 'validation' split for training
val_split: ['test']
model_name: LocPointTransformer  # maps to @register_meta_arch(...)

dataset:
  json_file: ./data/thumos/annotations/thumos14.json
  feat_folder: ./data/thumos/i3d_features
  file_ext: .npy
  num_classes: 20
  input_dim: 2048
  feat_stride: 4              # 4 frames per feature vector @ ~30fps = ~0.133s
  num_frames: 16
  trunc_thresh: 0.5           # truncate actions if <50% remains after crop
  crop_ratio: [0.9, 1.0]      # random crop augmentation during training
  max_seq_len: 2304

model:
  fpn_type: identity          # no FPN; direct pass-through
  n_mha_win_size: 19          # local attention window of 19 points
  max_buffer_len_factor: 6.0  # buffer = 6 * max_seq_len for point generator

opt:
  learning_rate: 0.0001       # scaled by num GPUs in train.py
  epochs: 30
  weight_decay: 0.05
  warmup: true
  warmup_epochs: 5
  schedule_type: cosine

train_cfg:
  init_loss_norm: 100         # initial EMA normalizer for loss
  clip_grad_l2norm: 1.0       # gradient clipping
  center_sample: radius
  center_sample_radius: 1.5   # sample points within 1.5 * stride of action center

test_cfg:
  pre_nms_thresh: 0.001
  pre_nms_topk: 2000
  iou_threshold: 0.1
  nms_method: soft            # soft | hard | none
  nms_sigma: 0.5
  voting_thresh: 0.7          # segment voting threshold
  max_seg_num: 200
  multiclass_nms: true

output_folder: ./ckpt/

```

--------------------------------

### Command-Line Training Script

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

The main script for end-to-end training. It handles configuration loading, dataset/model/optimizer/scheduler setup, training loop execution, checkpoint saving, and TensorBoard logging.

```bash
#!/bin/bash
# Example usage of train.py (actual command not provided in source)
# python train.py --config configs/your_config.yaml
```

--------------------------------

### Monitor Training with TensorBoard

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Optionally, monitor the training process by launching TensorBoard and pointing it to the logs directory within your experiment output.

```shell
tensorboard --logdir=./ckpt/thumos_i3d_reproduce/logs
```

--------------------------------

### Monitor Training with TensorBoard

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Optional command to visualize training progress using TensorBoard. Ensure the log directory path is correct.

```shell
tensorboard --logdir=./ckpt/ego4d_omnivore_egovlp_reproduce/logs
```

--------------------------------

### Resume ActionFormer Training

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Continue training from a previously saved checkpoint. Ensure the output directory and resume path are correctly specified.

```bash
python ./train.py ./configs/thumos_i3d.yaml \
    --output reproduce \
    --resume ./ckpt/thumos_i3d_reproduce/epoch_020.pth.tar
```

--------------------------------

### Build Dataset and DataLoader

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Instantiates a dataset class (e.g., THUMOS14Dataset) and wraps it in a PyTorch DataLoader. Each `__getitem__` returns a dictionary containing video features, segment timestamps, labels, fps, and duration. Ensure random seed is fixed for reproducibility.

```python
from libs.datasets import make_dataset, make_data_loader
from libs.core import load_config
from libs.utils import fix_random_seed

cfg = load_config("./configs/thumos_i3d.yaml")
rng = fix_random_seed(cfg['init_rand_seed'], include_cuda=True)

# Training dataset: split = ['validation'] for THUMOS14
train_dataset = make_dataset(
    cfg['dataset_name'],      # "thumos"
    True,                     # is_training=True
    cfg['train_split'],       # ['validation']
    **cfg['dataset']
    # feat_folder, json_file, input_dim, num_classes, feat_stride, etc.
)

# DataLoader with shuffle + worker RNG seeding
train_loader = make_data_loader(
    train_dataset,
    True,                         # shuffle=True
    rng,                          # rng_generator for reproducibility
    **cfg['loader']               # batch_size=2, num_workers=4
)

# Each batch is a list of dicts (auto-collation disabled):
for video_list in train_loader:
    sample = video_list[0]
    print(sample['video_id'])          # e.g., "video_validation_0000051"
    print(sample['feats'].shape)       # torch.Size([2048, T])
    print(sample['segments'].shape)    # torch.Size([N, 2])  — in feature grid coords
    print(sample['labels'].shape)      # torch.Size([N])
    print(sample['fps'])               # e.g., 29.97
    print(sample['duration'])          # e.g., 213.4  seconds
    break
```

--------------------------------

### Initialize ActivityNet mAP and Recall Evaluator

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Sets up the ANETdetection evaluator for computing per-class average precision and top-kx recall. Requires annotation file, split, tIoU thresholds, and optionally top-k values and number of workers.

```python
import numpy as np
from libs.utils.metrics import ANETdetection

evaluator = ANETdetection(
    ant_file='./data/thumos/annotations/thumos14.json',
    split='test',
    tiou_thresholds=np.linspace(0.3, 0.7, 5),
    top_k=(1, 5),
    num_workers=8
)
```

--------------------------------

### Monitor Training with TensorBoard

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Optional command to monitor training progress using TensorBoard. Ensure the log directory matches your experiment output.

```shell
tensorboard --logdir=./ckpt/anet_tsp_reproduce/logs
```

--------------------------------

### Create LR Scheduler with Linear Warmup

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Initializes a learning rate scheduler (cosine annealing or multi-step) with an optional linear warmup phase. The scheduler steps per iteration, so `num_iters_per_epoch` must be provided. The total training duration is determined by `warmup_epochs` and `epochs`.

```python
from libs.utils import make_optimizer, make_scheduler
from libs.core import load_config
from libs.modeling import make_meta_arch
import torch.nn as nn

cfg = load_config("./configs/thumos_i3d.yaml")
model = nn.DataParallel(make_meta_arch(cfg['model_name'], **cfg['model']))
optimizer = make_optimizer(model, cfg['opt'])

num_iters_per_epoch = 500  # e.g., len(train_loader)
scheduler = make_scheduler(optimizer, cfg['opt'], num_iters_per_epoch)
# cfg['opt']['warmup']=True, warmup_epochs=5, epochs=30, schedule_type='cosine'
# Total steps = (5 + 30) * 500 = 17500

# Step every iteration inside the training loop:
for batch in train_loader:
    optimizer.zero_grad()
    loss = model(batch)['final_loss']
    loss.backward()
    optimizer.step()
    scheduler.step()    # <-- per iteration, not per epoch

print(scheduler.get_last_lr())  # current learning rate
```

--------------------------------

### Load and Merge YAML Config with Defaults

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Loads a YAML configuration file and merges it with built-in defaults. It derives model parameters like input dimension and number of classes from the dataset section. This is the initial step for all training scripts.

```python
from libs.core import load_config

# thumos_i3d.yaml overrides only the fields it needs; defaults fill the rest
cfg = load_config("./configs/thumos_i3d.yaml")

# Key fields available after loading:
print(cfg['dataset_name'])          # "thumos"
print(cfg['model_name'])            # "LocPointTransformer"
print(cfg['model']['input_dim'])    # 2048  (derived from dataset.input_dim)
print(cfg['model']['num_classes'])  # 20    (derived from dataset.num_classes)
print(cfg['opt']['learning_rate'])  # 0.0001
print(cfg['devices'])               # ['cuda:0']

# Example: override max segments at test time
cfg['model']['test_cfg']['max_seg_num'] = 500
```

--------------------------------

### Train ActionFormer on EPIC Kitchens (Verbs)

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Use this command to train the ActionFormer model on verbs using SlowFast features. Ensure configuration files and data are correctly set up.

```shell
python ./train.py ./configs/epic_slowfast_verb.yaml --output reproduce
```

--------------------------------

### Instantiate the PtTransformer Model

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Creates the `LocPointTransformer` model, wrapping backbone, neck, point generator, and heads. In training mode, `forward()` returns a loss dictionary; in eval mode, it returns decoded segment predictions. The model is wrapped in `nn.DataParallel` for multi-GPU support.

```python
import torch.nn as nn
from libs.modeling import make_meta_arch
from libs.core import load_config

cfg = load_config("./configs/thumos_i3d.yaml")

model = make_meta_arch(cfg['model_name'], **cfg['model'])
# Wrap for multi-GPU (even with a single device)
model = nn.DataParallel(model, device_ids=cfg['devices'])

# --- Training forward pass ---
model.train()
losses = model(video_list)
# losses = {
#   'cls_loss':   tensor(0.42),   # sigmoid focal loss over all valid points
#   'reg_loss':   tensor(0.17),   # DIoU loss over positive points
#   'final_loss': tensor(0.59)    # cls_loss + loss_weight * reg_loss
# }
losses['final_loss'].backward()

# --- Inference forward pass ---
model.eval()
with torch.no_grad():
    results = model(video_list)   # batch_size must be 1 at inference
```

--------------------------------

### Train ActionFormer on EPIC Kitchens (Nouns)

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Use this command to train the ActionFormer model on nouns using SlowFast features. Ensure configuration files and data are correctly set up.

```shell
python ./train.py ./configs/epic_slowfast_noun.yaml --output reproduce
```

--------------------------------

### Evaluate Pre-trained Model with I3D Features

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the provided pre-trained ActionFormer model using I3D features. Specify the configuration file and the directory containing the pre-trained model artifacts.

```shell
python ./eval.py ./configs/anet_i3d.yaml ./pretrained/anet_i3d_reproduce/
```

--------------------------------

### Train ActionFormer with I3D Features

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Use this command to train the ActionFormer model with I3D features. The --output flag specifies the experiment name.

```shell
python ./train.py ./configs/anet_i3d.yaml --output reproduce
```

--------------------------------

### Evaluate Trained Model with I3D Features

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the performance of the trained ActionFormer model using I3D features. Provide the configuration file and the experiment directory.

```shell
python ./eval.py ./configs/anet_i3d.yaml ./ckpt/anet_i3d_reproduce
```

--------------------------------

### Evaluate Pre-trained ActionFormer Model

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate a pre-trained ActionFormer model. Ensure the pre-trained model and its associated configuration directory are correctly placed.

```shell
python ./eval.py ./configs/ego4d_omnivore_egovlp.yaml ./pretrained/ego4d_omnivore_egovlp_reproduce/
```

--------------------------------

### Evaluate ActionFormer Model (Nouns)

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the trained ActionFormer model for nouns. The command points to the configuration file and the directory containing the trained model checkpoints.

```shell
python ./eval.py ./configs/epic_slowfast_noun.yaml ./ckpt/epic_slowfast_noun_reproduce
```

--------------------------------

### Evaluate Trained ActionFormer Model

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the performance of the trained ActionFormer model using the specified configuration file and the experiment output directory. Expected mAP is provided.

```shell
python ./eval.py ./configs/thumos_i3d.yaml ./ckpt/thumos_i3d_reproduce
```

--------------------------------

### make_scheduler

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Creates a learning rate scheduler with cosine annealing or multi-step decay, featuring an optional linear warmup phase. The scheduler steps per iteration, requiring `num_iters_per_epoch` to be provided. The total training duration is determined by `warmup_epochs` and `epochs`.

```APIDOC
## `make_scheduler` — Learning rate scheduler with linear warmup

Creates a cosine annealing or multi-step LR scheduler with an optional linear warmup phase. Schedulers step every iteration (not epoch), so `num_iters_per_epoch` must be provided. The total number of epochs is `warmup_epochs + epochs`.

```python
from libs.utils import make_optimizer, make_scheduler
from libs.core import load_config
from libs.modeling import make_meta_arch
import torch.nn as nn

cfg = load_config("./configs/thumos_i3d.yaml")
model = nn.DataParallel(make_meta_arch(cfg['model_name'], **cfg['model']))
optimizer = make_optimizer(model, cfg['opt'])

num_iters_per_epoch = 500  # e.g., len(train_loader)
scheduler = make_scheduler(optimizer, cfg['opt'], num_iters_per_epoch)
# cfg['opt']['warmup']=True, warmup_epochs=5, epochs=30, schedule_type='cosine'
# Total steps = (5 + 30) * 500 = 17500

# Step every iteration inside the training loop:
for batch in train_loader:
    optimizer.zero_grad()
    loss = model(batch)['final_loss']
    loss.backward()
    optimizer.step()
    scheduler.step()    # <-- per iteration, not per epoch

print(scheduler.get_last_lr())  # current learning rate
```
```

--------------------------------

### Evaluate ActionFormer Model (Verbs)

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the trained ActionFormer model for verbs. The command points to the configuration file and the directory containing the trained model checkpoints.

```shell
python ./eval.py ./configs/epic_slowfast_verb.yaml ./ckpt/epic_slowfast_verb_reproduce
```

--------------------------------

### Initialize ConvTransformerBackbone

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Instantiates the ConvTransformerBackbone, a multi-scale convolutional-transformer backbone. It processes input features through convolutional layers, positional encoding, and Transformer blocks with local window attention for efficient sequence processing. Use this for feature extraction in video understanding tasks.

```python
import torch
from libs.modeling.backbones import ConvTransformerBackbone

backbone = ConvTransformerBackbone(
    n_in=2048,           # I3D feature dim
    n_embd=512,          # internal embedding dim
    n_head=4,            # attention heads
    n_embd_ks=3,         # conv kernel size
    max_len=2304,        # max sequence length
    arch=(2, 2, 5),      # 2 embd convs, 2 stem transformers, 5 branch transformers
    mha_win_size=[19]*6, # local window size per level (thumos_i3d config)
    scale_factor=2,      # 2x downsampling per branch level
    with_ln=True,
    path_pdrop=0.1
)

B, C, T = 2, 2048, 768
x = torch.randn(B, C, T)
mask = torch.ones(B, 1, T, dtype=torch.bool)

out_feats, out_masks = backbone(x, mask)
# out_feats: tuple of 6 tensors (stem + 5 branch levels)
# shapes: [(B,512,768), (B,512,384), (B,512,192), (B,512,96), (B,512,48), (B,512,24)]
for i, (f, m) in enumerate(zip(out_feats, out_masks)):
    print(f"Level {i}: feats={f.shape}, mask={m.shape}")
```

--------------------------------

### Evaluate Pre-trained ActionFormer Model (Nouns)

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the provided pre-trained ActionFormer model for nouns. This command uses the evaluation script with the noun configuration and the pre-trained model directory.

```shell
python ./eval.py ./configs/epic_slowfast_noun.yaml ./pretrained/epic_slowfast_noun_reproduce/
```

--------------------------------

### Evaluate Pre-trained ActionFormer Model (Verbs)

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the provided pre-trained ActionFormer model for verbs. This command uses the evaluation script with the verb configuration and the pre-trained model directory.

```shell
python ./eval.py ./configs/epic_slowfast_verb.yaml ./pretrained/epic_slowfast_verb_reproduce/
```

--------------------------------

### Train ActionFormer on Ego4D

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Use this command to train ActionFormer with specified features. The experiment details will be saved in the ./ckpt directory.

```shell
python ./train.py ./configs/ego4d_omnivore_egovlp.yaml --output reproduce
```

--------------------------------

### Run Single Epoch Training Loop

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Executes a single training epoch, including forward/backward passes, gradient clipping, optimizer/scheduler steps, EMA updates, and TensorBoard logging. Prints per-iteration statistics.

```python
from libs.utils import train_one_epoch
from torch.utils.tensorboard import SummaryWriter

tb_writer = SummaryWriter('./ckpt/thumos_i3d_reproduce/logs')

train_one_epoch(
    train_loader,
    model,
    optimizer,
    scheduler,
    curr_epoch=0,
    model_ema=model_ema,
    clip_grad_l2norm=1.0,   # from train_cfg; -1 to disable
    tb_writer=tb_writer,
    print_freq=10
)
```

--------------------------------

### Evaluate Trained ActionFormer Model

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the performance of a trained ActionFormer model using the specified configuration and checkpoint directory. Expected results are provided in the documentation.

```shell
python ./eval.py ./configs/ego4d_omnivore_egovlp.yaml ./ckpt/ego4d_omnivore_egovlp_reproduce
```

--------------------------------

### Train ActionFormer on THUMOS14

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Use this command to train the ActionFormer model with I3D features on the THUMOS14 dataset. Specify the configuration file and an output directory for experiment artifacts.

```shell
python ./train.py ./configs/thumos_i3d.yaml --output reproduce
```

--------------------------------

### Initialize and Update ModelEma

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Sets up an Exponential Moving Average (EMA) of model weights with a specified decay rate (default 0.999). The EMA model is updated after each optimizer step and used for inference to improve performance. Ensure to save both the main model's state_dict and the EMA's state_dict_ema for checkpoints.

```python
from libs.utils import ModelEma
from libs.modeling import make_meta_arch
from libs.core import load_config
import torch.nn as nn

cfg = load_config("./configs/thumos_i3d.yaml")
model = nn.DataParallel(make_meta_arch(cfg['model_name'], **cfg['model']))
model_ema = ModelEma(model, decay=0.999)

# During training, update EMA after each optimizer step:
for video_list in train_loader:
    losses = model(video_list)
    losses['final_loss'].backward()
    optimizer.step()
    model_ema.update(model)   # EMA update: ema = 0.999*ema + 0.001*model

# Save both model and EMA weights:
save_states = {
    'epoch': 10,
    'state_dict': model.state_dict(),
    'state_dict_ema': model_ema.module.state_dict(),
    'optimizer': optimizer.state_dict(),
    'scheduler': scheduler.state_dict(),
}

```

--------------------------------

### Evaluate Pre-trained Model with TSP Features

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the provided pre-trained ActionFormer model using TSP features. Specify the configuration file and the directory containing the pre-trained model artifacts.

```shell
python ./eval.py ./configs/anet_tsp.yaml ./pretrained/anet_tsp_reproduce/
```

--------------------------------

### Evaluate Trained Model with TSP Features

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the performance of the trained ActionFormer model using TSP features. Provide the configuration file and the experiment directory.

```shell
python ./eval.py ./configs/anet_tsp.yaml ./ckpt/anet_tsp_reproduce
```

--------------------------------

### Run Validation/Evaluation Loop

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Performs inference over the validation set, collects results in ActivityNet format, and computes mAP using ANETdetection. Can save raw results to a pickle file.

```python
from libs.utils import valid_one_epoch, ANETdetection

val_db_vars = val_dataset.get_attributes()
# val_db_vars = {
#   'dataset_name': 'thumos-14',
#   'tiou_thresholds': array([0.3, 0.4, 0.5, 0.6, 0.7]),
#   'empty_label_ids': []
# }

det_eval = ANETdetection(
    val_dataset.json_file,
    val_dataset.split[0],           # 'test'
    tiou_thresholds=val_db_vars['tiou_thresholds']
)

mAP = valid_one_epoch(
    val_loader,
    model,
    curr_epoch=-1,
    evaluator=det_eval,
    ext_score_file=None,            # or path to external cls scores pkl/json
    tb_writer=None,
    print_freq=10
)
# Console output:
# [RESULTS] Action detection results on thumos14.
# |tIoU = 0.30: mAP = 82.13 (%)
# |tIoU = 0.40: mAP = 77.80 (%)
# |tIoU = 0.50: mAP = 70.95 (%)
# |tIoU = 0.60: mAP = 59.40 (%)
# |tIoU = 0.70: mAP = 43.87 (%)
# Average mAP: 66.83 (%)
print(f"Average mAP: {mAP:.2f}%")
```

--------------------------------

### Evaluate Pre-trained ActionFormer Model

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Evaluate the provided pre-trained ActionFormer model for THUMOS 14. Ensure the model files are unpacked in the specified directory and use this command for evaluation.

```shell
python ./eval.py ./configs/thumos_i3d.yaml ./pretrained/thumos_i3d_reproduce/
```

--------------------------------

### Train ActionFormer with TSP Features

Source: https://github.com/happyharrycn/actionformer_release/blob/main/README.md

Use this command to train the ActionFormer model with TSP features. The --output flag specifies the experiment name.

```shell
python ./train.py ./configs/anet_tsp.yaml --output reproduce
```

--------------------------------

### Evaluate Specific Checkpoint File

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Evaluate a single checkpoint file (.pth.tar). This is useful for testing specific saved states without needing the entire folder.

```bash
python ./eval.py ./configs/thumos_i3d.yaml \
    ./pretrained/thumos_i3d_reproduce/epoch_034.pth.tar
```

--------------------------------

### Load EMA Weights for Inference

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Loads model weights from a checkpoint file, specifically applying the EMA (Exponential Moving Average) weights for inference.

```python
checkpoint = torch.load('epoch_010.pth.tar', map_location='cuda:0')
model.load_state_dict(checkpoint['state_dict_ema'])
```

--------------------------------

### Compile NMS C++ Code

Source: https://github.com/happyharrycn/actionformer_release/blob/main/INSTALL.md

Compile the C++ implementation of NMS. Recompile if PyTorch is updated.

```shell
cd ./libs/utils
python setup.py install --user
cd ../..
```

--------------------------------

### train_one_epoch

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Runs one full pass over the training DataLoader, including forward and backward passes, gradient clipping, optimizer and scheduler stepping, EMA update, and TensorBoard logging. Prints per-iteration stats at a configurable frequency.

```APIDOC
## `train_one_epoch` — Single epoch training loop

Runs one full pass over the training DataLoader: forward pass, backward pass, gradient clipping, optimizer and scheduler stepping, EMA update, and TensorBoard logging. Prints per-iteration stats at a configurable frequency.

```python
from libs.utils import train_one_epoch
from torch.utils.tensorboard import SummaryWriter

tb_writer = SummaryWriter('./ckpt/thumos_i3d_reproduce/logs')

train_one_epoch(
    train_loader,
    model,
    optimizer,
    scheduler,
    curr_epoch=0,
    model_ema=model_ema,
    clip_grad_l2norm=1.0,   # from train_cfg; -1 to disable
    tb_writer=tb_writer,
    print_freq=10
)
# Console output (every 10 iterations):
# [Train]: Epoch 000 started
# Epoch: [000][00010/00500]  Time 0.43 (0.41)  Loss 0.81 (0.94)
#         cls_loss 0.65 (0.78)   reg_loss 0.16 (0.16)
# [Train]: Epoch 000 finished with lr=0.00002000
```
```

--------------------------------

### Create AdamW Optimizer with Selective Weight Decay

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Generates an AdamW or SGD optimizer, partitioning model parameters into decay and no-decay groups. This is crucial for stable Transformer training by applying weight decay only to linear/conv weights and not biases or norm layers. Ensure your config includes 'opt' settings like type, learning_rate, and weight_decay.

```python
from libs.utils import make_optimizer
from libs.core import load_config
from libs.modeling import make_meta_arch
import torch.nn as nn

cfg = load_config("./configs/thumos_i3d.yaml")
model = nn.DataParallel(make_meta_arch(cfg['model_name'], **cfg['model']))

optimizer = make_optimizer(model, cfg['opt'])
# cfg['opt'] = {
#   "type": "AdamW",
#   "learning_rate": 0.0001,
#   "weight_decay": 0.05,
#   "momentum": 0.9
# }

# The optimizer has two param groups:
print(len(optimizer.param_groups))          # 2
print(optimizer.param_groups[0]['weight_decay'])  # 0.05  (decay group)
print(optimizer.param_groups[1]['weight_decay'])  # 0.0   (no-decay group)
```

--------------------------------

### make_optimizer

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Creates an AdamW or SGD optimizer with selective weight decay. It partitions model parameters into decay and no-decay groups, applying weight decay only to linear and convolutional weights, while biases and norm layer weights are excluded. This approach is crucial for stable Transformer training.

```APIDOC
## `make_optimizer` — Create AdamW/SGD optimizer with selective weight decay

Partitions model parameters into decay and no-decay groups: linear/conv weights decay, biases and norm layer weights do not. This follows the pattern from minGPT and is important for stable Transformer training.

```python
from libs.utils import make_optimizer
from libs.core import load_config
from libs.modeling import make_meta_arch
import torch.nn as nn

cfg = load_config("./configs/thumos_i3d.yaml")
model = nn.DataParallel(make_meta_arch(cfg['model_name'], **cfg['model']))

optimizer = make_optimizer(model, cfg['opt'])
# cfg['opt'] = {
#   "type": "AdamW",
#   "learning_rate": 0.0001,
#   "weight_decay": 0.05,
#   "momentum": 0.9
# }

# The optimizer has two param groups:
print(len(optimizer.param_groups))          # 2
print(optimizer.param_groups[0]['weight_decay'])  # 0.05  (decay group)
print(optimizer.param_groups[1]['weight_decay'])  # 0.0   (no-decay group)
```
```

--------------------------------

### Evaluate Predictions with ANETdetection

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Evaluates a set of predictions against ground truth using the initialized ANETdetection evaluator. Predictions can be provided as a dictionary of numpy arrays.

```python
# predictions dict (from valid_one_epoch output structure):
preds = {
    'video-id': ['video_test_0000001', 'video_test_0000001', 'video_test_0000002'],
    't-start':  np.array([10.2, 45.0, 5.1]),
    't-end':    np.array([18.7, 52.3, 12.4]),
    'label':    np.array([3, 7, 3]),
    'score':    np.array([0.92, 0.88, 0.75])
}

mAP_per_thresh, average_mAP, recall = evaluator.evaluate(preds, verbose=True)
# mAP_per_thresh.shape = (5,)  — one value per tIoU threshold
# average_mAP           = scalar mean across thresholds
# recall.shape          = (5, 2) — per tIoU and per top-k
```

--------------------------------

### Evaluate Specific Epoch

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Evaluate a specific epoch from a training run by providing the epoch number. This allows for detailed analysis of intermediate model states.

```bash
python ./eval.py ./configs/thumos_i3d.yaml ./ckpt/thumos_i3d_reproduce -epoch 34
```

--------------------------------

### valid_one_epoch

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Runs inference over the entire validation set, collects results in ActivityNet format, optionally fuses with external classification scores, and computes mAP. Can also save raw results to a pickle file.

```APIDOC
## `valid_one_epoch` — Validation / evaluation loop

Runs inference over the entire validation set, collects results in ActivityNet format (video-id, t-start, t-end, label, score arrays), optionally fuses with external classification scores, and calls `ANETdetection.evaluate()` to compute mAP. Can also save raw results to a pickle file for later evaluation.

```python
from libs.utils import valid_one_epoch, ANETdetection

val_db_vars = val_dataset.get_attributes()
# val_db_vars = {
#   'dataset_name': 'thumos-14',
#   'tiou_thresholds': array([0.3, 0.4, 0.5, 0.6, 0.7]),
#   'empty_label_ids': []
# }

det_eval = ANETdetection(
    val_dataset.json_file,
    val_dataset.split[0],           # 'test'
    tiou_thresholds=val_db_vars['tiou_thresholds']
)

mAP = valid_one_epoch(
    val_loader,
    model,
    curr_epoch=-1,
    evaluator=det_eval,
    ext_score_file=None,            # or path to external cls scores pkl/json
    tb_writer=None,
    print_freq=10
)
# Console output:
# [RESULTS] Action detection results on thumos14.
# |tIoU = 0.30: mAP = 82.13 (%)
# |tIoU = 0.40: mAP = 77.80 (%)
# |tIoU = 0.50: mAP = 70.95 (%)
# |tIoU = 0.60: mAP = 59.40 (%)
# |tIoU = 0.70: mAP = 43.87 (%)
# Average mAP: 66.83 (%)
print(f"Average mAP: {mAP:.2f}%")
```
```

--------------------------------

### Save Raw Predictions for Submission

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Skip evaluation and save only the raw prediction results, typically for submission to leaderboards or further processing. Specify the configuration and checkpoint paths.

```bash
python ./eval.py ./configs/ego4d_omnivore_egovlp.yaml \
    ./ckpt/ego4d_omnivore_egovlp_reproduce --saveonly
```

--------------------------------

### ANETdetection

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Computes per-class average precision at multiple tIoU thresholds and top-kx recall using parallel workers. Accepts predictions in various formats and loads ground truth from ActivityNet JSON.

```APIDOC
## `ANETdetection` — ActivityNet-style mAP and recall evaluator

Computes per-class average precision at multiple tIoU thresholds and top-kx recall using parallel workers. Accepts predictions as a pandas DataFrame, a JSON file, or a dict of numpy arrays. Ground truth is loaded from the standard ActivityNet JSON annotation format.

```python
import numpy as np
from libs.utils.metrics import ANETdetection

evaluator = ANETdetection(
    ant_file='./data/thumos/annotations/thumos14.json',
    split='test',
    tiou_thresholds=np.linspace(0.3, 0.7, 5),
    top_k=(1, 5),
    num_workers=8
)

# predictions dict (from valid_one_epoch output structure):
preds = {
    'video-id': ['video_test_0000001', 'video_test_0000001', 'video_test_0000002'],
    't-start':  np.array([10.2, 45.0, 5.1]),
    't-end':    np.array([18.7, 52.3, 12.4]),
    'label':    np.array([3, 7, 3]),
    'score':    np.array([0.92, 0.88, 0.75])
}

mAP_per_thresh, average_mAP, recall = evaluator.evaluate(preds, verbose=True)
# mAP_per_thresh.shape = (5,)  — one value per tIoU threshold
# average_mAP           = scalar mean across thresholds
# recall.shape          = (5, 2) — per tIoU and per top-k
```
```

--------------------------------

### ModelEma

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Maintains an Exponential Moving Average (EMA) of model weights with a specified decay rate (defaulting to 0.999). The EMA model is updated after each optimizer step and is used exclusively during inference to significantly improve final model quality. EMA weights can be loaded via `state_dict_ema` in checkpoints.

```APIDOC
## `ModelEma` — Exponential Moving Average of model weights

Maintains an EMA copy of the model with `decay=0.999`. The EMA model is updated after every optimizer step and is used exclusively at inference time (loaded via `state_dict_ema` in checkpoints). This significantly improves final model quality.

```python
from libs.utils import ModelEma
from libs.modeling import make_meta_arch
from libs.core import load_config
import torch.nn as nn

cfg = load_config("./configs/thumos_i3d.yaml")
model = nn.DataParallel(make_meta_arch(cfg['model_name'], **cfg['model']))
model_ema = ModelEma(model, decay=0.999)

# During training, update EMA after each optimizer step:
for video_list in train_loader:
    losses = model(video_list)
    losses['final_loss'].backward()
    optimizer.step()
    model_ema.update(model)   # EMA update: ema = 0.999*ema + 0.001*model

# Save both model and EMA weights:
save_states = {
    'epoch': 10,
    'state_dict': model.state_dict(),
    'state_dict_ema': model_ema.module.state_dict(),
    'optimizer': optimizer.state_dict(),
    'scheduler': scheduler.state_dict(),
}


```

--------------------------------

### Limit Evaluation Output Segments

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Limit the number of predicted segments per video during evaluation. This can be useful for managing output size or focusing on top predictions.

```bash
python ./eval.py ./configs/thumos_i3d.yaml ./ckpt/thumos_i3d_reproduce -t 100
```

--------------------------------

### Fuse Model Scores with External Class Scores

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Combines model-predicted segment scores with external video-level classification scores using geometric mean. Improves mAP when the localization model's classification is weaker.

```python
from libs.utils.postprocessing import postprocess_results

# raw_results: dict with arrays from valid_one_epoch (before evaluator call)
raw_results = {
    'video-id': [...],
    't-start':  np.array([...]),
    't-end':    np.array([...]),
    'label':    np.array([...]),   # model's class prediction
    'score':    np.array([...])
}

# ext_score_file: pkl or json mapping video_id -> array of shape (num_classes,)
fused_results = postprocess_results(
    raw_results,
    cls_score_file='./data/thumos/annotations/thumos14_cls_scores.pkl',
    num_pred=200,   # keep top-200 segments per video
    topk=2          # assign top-2 video-level classes to each segment
)
# fused_results has same structure but with re-labeled and re-scored segments
# new score = sqrt(cls_score[k] * original_seg_score)
```

--------------------------------

### ConvTransformerBackbone

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

A multi-scale convolutional-transformer backbone that processes feature tensors through convolutional embedding layers, optional sinusoidal position encoding, stem Transformer blocks, and branching Transformer blocks with strided pooling to produce a multi-scale feature pyramid. It utilizes local window attention for complexity management.

```APIDOC
## `ConvTransformerBackbone` — Multi-scale convolutional-transformer backbone

The primary backbone (`backbone_type='convTransformer'`). Processes `(B, C, T)` feature tensors through convolutional embedding layers, optional sinusoidal position encoding, stem Transformer blocks (no downsampling), then branching Transformer blocks with strided pooling that produce a multi-scale feature pyramid. Local window attention (`n_mha_win_size`) is used to keep complexity linear in sequence length.

```python
import torch
from libs.modeling.backbones import ConvTransformerBackbone

backbone = ConvTransformerBackbone(
    n_in=2048,           # I3D feature dim
    n_embd=512,          # internal embedding dim
    n_head=4,            # attention heads
    n_embd_ks=3,         # conv kernel size
    max_len=2304,        # max sequence length
    arch=(2, 2, 5),      # 2 embd convs, 2 stem transformers, 5 branch transformers
    mha_win_size=[19]*6, # local window size per level (thumos_i3d config)
    scale_factor=2,      # 2x downsampling per branch level
    with_ln=True,
    path_pdrop=0.1
)

B, C, T = 2, 2048, 768
x = torch.randn(B, C, T)
mask = torch.ones(B, 1, T, dtype=torch.bool)

out_feats, out_masks = backbone(x, mask)
# out_feats: tuple of 6 tensors (stem + 5 branch levels)
# shapes: [(B,512,768), (B,512,384), (B,512,192), (B,512,96), (B,512,48), (B,512,24)]
for i, (f, m) in enumerate(zip(out_feats, out_masks)):
    print(f"Level {i}: feats={f.shape}, mask={m.shape}")
```
```

--------------------------------

### postprocess_results

Source: https://context7.com/happyharrycn/actionformer_release/llms.txt

Fuses model-predicted segment scores with external video-level classification scores by taking the geometric mean. Improves mAP when the localization model's classification is weaker than a dedicated video classifier.

```APIDOC
## `postprocess_results` — Score fusion with external classifier

Fuses model-predicted segment scores with top-k external video-level classification scores (e.g., from TSN/VideoMAE) by taking the geometric mean. Each segment is duplicated for the top-k predicted classes, improving mAP when the localization model's classification is weaker than a dedicated video classifier.

```python
from libs.utils.postprocessing import postprocess_results

# raw_results: dict with arrays from valid_one_epoch (before evaluator call)
raw_results = {
    'video-id': [...],
    't-start':  np.array([...]),
    't-end':    np.array([...]),
    'label':    np.array([...]),   # model's class prediction
    'score':    np.array([...])
}

# ext_score_file: pkl or json mapping video_id -> array of shape (num_classes,)
fused_results = postprocess_results(
    raw_results,
    cls_score_file='./data/thumos/annotations/thumos14_cls_scores.pkl',
    num_pred=200,   # keep top-200 segments per video
    topk=2          # assign top-2 video-level classes to each segment
)
# fused_results has same structure but with re-labeled and re-scored segments
# new score = sqrt(cls_score[k] * original_seg_score)
```
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.