MotionAGFormer

https://github.com/taatiteam/motionagformer
Admin
MotionAGFormer is a transformer-based network for enhancing 3D human pose estimation from monocular...
Tokens:7,973
Snippets:65
Trust Score:8.2
License:Apache-2.0
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install Project Dependencies

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Installs all necessary project dependencies using pip. Ensure you have Python 3.8.10 and PyTorch 2.0.0 with CUDA 12.2.

```bash
pip install -r requirements.txt
```

--------------------------------

### Run MotionAGFormer Demo

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Execute the demo script to visualize the model's performance on videos. Place pre-trained YOLOv3, HRNet, and base model checkpoints in the specified directories, and your videos in './demo/video'.

```bash
python demo/vis.py --video sample_video.mp4
```

--------------------------------

### Run Demo Pipeline for In-the-Wild Inference

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Executes an end-to-end inference pipeline using YOLOv3 for detection, HRNet for 2D keypoint estimation, and MotionAGFormer for 3D pose lifting. Requires pre-trained checkpoints for all models and places input videos in a specific directory.

```bash
# Prerequisites:
#   1. Place YOLOv3 + HRNet checkpoints in  ./demo/lib/checkpoint/
#   2. Place MotionAGFormer-Base checkpoint  ./checkpoint/motionagformer-b-h36m.pth.tr
#   3. Place input video in                 ./demo/video/

python demo/vis.py --video sample_video.mp4 --gpu 0

# Outputs written to ./demo/output/sample_video/:
#   input_2D/keypoints.npz   — detected 2D keypoints
#   pose2D/*.png             — per-frame 2D overlay images
#   pose3D/*.png             — per-frame 3D pose renders
#   sample_video.mp4         — final side-by-side video
```

--------------------------------

### Preprocess Human3.6M Data (XSmall)

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Slices motion clips for the MotionAGFormer-XSmall model. Requires downloading preprocessed data and placing it in 'data/motion3d'.

```bash
python h36m.py --n-frames 27
```

--------------------------------

### Evaluate Human3.6M Models with MotionAGFormer

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Use this command to evaluate Human3.6M models. Ensure you have downloaded the weights and specify the correct checkpoint directory, file name, and configuration path.

```bash
python train.py --eval-only --checkpoint <CHECKPOINT-DIRECTORY> --checkpoint-file <CHECKPOINT-FILE-NAME> --config <PATH-TO-CONFIG>
```

```bash
python train.py --eval-only --checkpoint checkpoint --checkpoint-file motionagformer-l-h36m.pth.tr --config configs/h36m/MotionAGFormer-large.yaml
```

--------------------------------

### Evaluate MPI-INF-3DHP Models with MotionAGFormer

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Use this command to evaluate MPI-INF-3DHP models. Ensure you have downloaded the weights and specify the correct checkpoint directory, file name, and configuration path.

```bash
python train_3dhp.py --eval-only --checkpoint <CHECKPOINT-DIRECTORY> --checkpoint-file <CHECKPOINT-FILE-NAME> --config <PATH-TO-CONFIG>
```

--------------------------------

### Preprocess Human3.6M Data (Base/Large)

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Slices motion clips for HumanAGFormer-Base and MotionAGFormer-Large models. Requires downloading preprocessed data and placing it in 'data/motion3d'.

```bash
python h36m.py  --n-frames 243
```

--------------------------------

### MotionAGFormer Main Model

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Builds the main MotionAGFormer model. Use for lifting 2D keypoint sequences to 3D poses. Can also return motion representations for downstream tasks.

```python
import torch
from model.MotionAGFormer import MotionAGFormer

# Build MotionAGFormer-Base equivalent (243-frame, 17-joint, 3-channel input)
model = MotionAGFormer(
    n_layers=16,
    dim_in=3,           # (x, y, confidence)
    dim_feat=128,
    dim_rep=512,
    dim_out=3,          # (X, Y, Z) output
    mlp_ratio=4,
    num_heads=8,
    use_layer_scale=True,
    layer_scale_init_value=1e-5,
    use_adaptive_fusion=True,
    hierarchical=False,
    use_temporal_similarity=True,
    neighbour_num=2,
    temporal_connection_len=1,
    use_tcn=False,
    graph_only=False,
    n_frames=243,
    num_joints=17,
)
model.eval()

# Random inference: batch=2, T=243 frames, J=17 joints, C=3
x = torch.randn(2, 243, 17, 3)
with torch.no_grad():
    out = model(x)
    rep = model(x, return_rep=True)   # motion representation (512-d) for downstream tasks

print(out.shape)   # torch.Size([2, 243, 17, 3])
print(rep.shape)   # torch.Size([2, 243, 17, 512])
```

--------------------------------

### Load Model using Config-Driven Factory

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Loads a MotionAGFormer model variant by reading an EasyDict configuration. This is the primary entry point for training and evaluation scripts.

```python
from utils.tools import get_config
from utils.learning import load_model

# Load Base model config
args = get_config('configs/h36m/MotionAGFormer-base.yaml')
model = load_model(args)

# Count parameters
from utils.tools import count_param_numbers
print(f"Parameters: {count_param_numbers(model):,}")
```

--------------------------------

### Preprocess Human3.6M Data (Small)

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Slices motion clips for the MotionAGFormer-Small model. Requires downloading preprocessed data and placing it in 'data/motion3d'.

```bash
python h36m.py --n-frames 81
```

--------------------------------

### Visualize Human3.6M Data

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Generates a GIF visualization of a human pose sequence from the Human3.6M dataset. Expects 243 frames and saves the output to the 'data' directory.

```bash
python visualize.py --dataset h36m --sequence-number <AN ARBITRARY NUMBER>
```

--------------------------------

### Train MotionAGFormer on Human3.6M

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Trains the MotionAGFormer model on the Human3.6M dataset using a specified configuration file. Optionally enable weight and biases logging.

```bash
python train.py --config <PATH-TO-CONFIG>
```

```bash
python train.py --config configs/h36m/MotionAGFormer-base.yaml --use-wandb --wandb-name MotionAGFormer-base
```

--------------------------------

### Load YAML Configuration

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Parses a YAML configuration file into an EasyDict object, allowing attribute-based access to configuration keys. The config name is stored in `args.name`.

```python
from utils.tools import get_config

args = get_config('configs/h36m/MotionAGFormer-large.yaml')

print(args.n_layers)
print(args.dim_feat)
print(args.n_frames)
print(args.use_adaptive_fusion)
print(args.name)
```

--------------------------------

### Train MotionAGFormer on MPI-INF-3DHP

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

Trains the MotionAGFormer model on the MPI-INF-3DHP dataset using a specified configuration file. Weight and biases logging can also be enabled.

```bash
python train_3dhp.py --config <PATH-TO-CONFIG>
```

--------------------------------

### Initialize Spatial and Temporal Attention Modules

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Initializes multi-head self-attention modules for spatial and temporal data processing. These modules operate on tensors with dimensions [Batch, Time, Joints, Channels].

```python
from model.modules.attention import Attention
import torch

spatial_attn  = Attention(dim_in=128, dim_out=128, num_heads=8, mode='spatial')
temporal_attn = Attention(dim_in=128, dim_out=128, num_heads=8, mode='temporal')

x = torch.randn(4, 243, 17, 128)
print(spatial_attn(x).shape)
print(temporal_attn(x).shape)
```

--------------------------------

### Save and Load Checkpoints in MotionAGFormer

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Persists epoch, optimizer state, model weights, and best MPJPE. `load_pretrained_weights` handles `nn.DataParallel` prefix stripping and skips mismatched keys.

```python
import torch
from torch import optim
from utils.learning import load_model, load_pretrained_weights
from utils.tools import get_config
from train import save_checkpoint

args  = get_config('configs/h36m/MotionAGFormer-base.yaml')
model = load_model(args)
optimizer = optim.AdamW(model.parameters(), lr=5e-4, weight_decay=0.01)

# Save
save_checkpoint(
    checkpoint_path='checkpoint/epoch_10.pth.tr',
    epoch=10, lr=5e-4,
    optimizer=optimizer,
    model=model,
    min_mpjpe=52.3,
    wandb_id='abc123',
)

# Load (handles DataParallel prefix automatically)
ckpt = torch.load('checkpoint/epoch_10.pth.tr', map_location='cpu')
load_pretrained_weights(model, ckpt['model'])
# [INFO] (load_pretrained_weights) 248 layers are loaded
# [INFO] (load_pretrained_weights) 0 layers are discarded
```

--------------------------------

### Programmatic 3D Inference with MotionAGFormer

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Performs 3D pose lifting using a pre-trained MotionAGFormer model. This snippet demonstrates how to load the model, process raw 2D keypoints into clips, normalize coordinates, and apply test-time augmentation by averaging predictions from original and flipped inputs.

```python
import torch, numpy as np
from model.MotionAGFormer import MotionAGFormer
from demo.vis import flip_data, turn_into_clips, normalize_screen_coordinates

model = MotionAGFormer(n_layers=16, dim_in=3, dim_feat=128, dim_rep=512, dim_out=3,
                       num_heads=8, n_frames=243)
model.load_state_dict(torch.load('checkpoint/motionagformer-b-h36m.pth.tr')['model'])
model.eval()

# keypoints: (1, T, 17, 2)  — raw pixel coordinates from HRNet
keypoints = np.random.rand(1, 300, 17, 2).astype(np.float32)
clips, downsample = turn_into_clips(keypoints)

results = []
for clip in clips:
    inp     = torch.tensor(normalize_screen_coordinates(clip, w=1280, h=720)).cuda()
    inp_aug = torch.tensor(flip_data(inp.cpu().numpy())).cuda()
    with torch.no_grad():
        out = (model(inp) + flip_data(model(inp_aug))) / 2
    results.append(out.cpu().numpy())
```

--------------------------------

### Load Human3.6M Dataset with MotionDataset3D

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Loads pre-processed Human3.6M motion data using PyTorch's DataLoader. This dataset can optionally append 2D velocity channels and applies random horizontal flips for augmentation during training.

```python
from utils.tools import get_config
from data.reader.motion_dataset import MotionDataset3D
from torch.utils.data import DataLoader

args = get_config('configs/h36m/MotionAGFormer-base.yaml')

train_dataset = MotionDataset3D(args, args.subset_list, 'train')
test_dataset  = MotionDataset3D(args, args.subset_list, 'test')

train_loader = DataLoader(train_dataset, batch_size=16, shuffle=True,  num_workers=4)
test_loader  = DataLoader(test_dataset,  batch_size=16, shuffle=False, num_workers=4)

motion_2d, motion_3d = next(iter(train_loader))
print(motion_2d.shape)   # torch.Size([16, 243, 17, 3])  (x, y, conf)
print(motion_3d.shape)   # torch.Size([16, 243, 17, 3])  (X, Y, Z)
```

--------------------------------

### Initialize Spatial and Temporal GCNs

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Instantiates spatial and temporal Graph Convolutional Networks (GCNs) with specified input/output dimensions, number of nodes, and operational modes. The temporal GCN can optionally use similarity neighbors.

```python
spatial_gcn = GCN(dim_in=128, dim_out=128, num_nodes=17, mode='spatial')

temporal_gcn = GCN(
    dim_in=128, dim_out=128,
    num_nodes=243, mode='temporal',
    neighbour_num=4, use_temporal_similarity=True,
)

x = torch.randn(4, 243, 17, 128)
out_s = spatial_gcn(x)
out_t = temporal_gcn(x)
print(out_s.shape, out_t.shape)
```

--------------------------------

### Read and Process H36M Data with DataReaderH36M

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Reads raw Human3.6M data, normalizes coordinates, splits sequences into clips, and provides a denormalization function. Use `data_stride_train` for overlapping clips during training and `data_stride_test` for non-overlapping clips during testing.

```python
from data.reader.h36m import DataReaderH36M

reader = DataReaderH36M(
    n_frames=243,
    sample_stride=1,
    data_stride_train=81,   # 1/3 clip overlap during training
    data_stride_test=243,
    dt_root='data/motion3d',
    dt_file='h36m_sh_conf_cam_source_final.pkl',
)

train_2d, test_2d, train_3d, test_3d = reader.get_sliced_data()
print(train_2d.shape)   # (N_clips, 243, 17, 3)
print(test_3d.shape)    # (N_clips, 243, 17, 3)

# After inference: convert normalised predictions → mm
import numpy as np
fake_pred = np.random.randn(*test_3d.shape).astype(np.float32)
denorm = reader.denormalize(fake_pred)  # (N_clips, 243, 17, 3) in mm
```

--------------------------------

### Pose 3D Loss Functions

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Collection of PyTorch differentiable training losses and NumPy evaluation metrics for 3D pose estimation. The total loss is a weighted sum of these components.

```python
import torch
from loss.pose3d import (
    loss_mpjpe, n_mpjpe, loss_velocity,
    loss_limb_var, loss_limb_gt, loss_angle, loss_angle_velocity,
    mpjpe, p_mpjpe, acc_error,
)
import numpy as np

pred = torch.randn(8, 243, 17, 3)   # model output
gt   = torch.randn(8, 243, 17, 3)   # ground truth

# --- Training losses (differentiable) ---
l_pos      = loss_mpjpe(pred, gt)            # mean joint position error
l_scale    = n_mpjpe(pred, gt)               # scale-normalised MPJPE
l_vel      = loss_velocity(pred, gt)         # temporal velocity consistency
l_lv       = loss_limb_var(pred)             # limb length variance over time
l_lg       = loss_limb_gt(pred, gt)          # limb length vs. ground-truth
l_ang      = loss_angle(pred, gt)            # joint angle error
l_ang_vel  = loss_angle_velocity(pred, gt)   # angular velocity error

total = l_pos + 0.5*l_scale + 20.0*l_vel + l_lv + l_lg + l_ang + l_ang_vel
total.backward()
```

--------------------------------

### load_pretrained_weights

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Loads pre-trained weights into a model, automatically handling the stripping of the 'module.' prefix common with nn.DataParallel and silently skipping any layers with mismatched keys.

```APIDOC
## load_pretrained_weights

### Description
Loads pre-trained weights into a model, automatically handling the stripping of the 'module.' prefix common with nn.DataParallel and silently skipping any layers with mismatched keys.

### Parameters
- **model** (torch.nn.Module) - The model to load weights into.
- **state_dict** (dict) - The state dictionary containing the pre-trained weights.
```

--------------------------------

### save_checkpoint

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Persists the current training state, including epoch number, learning rate, optimizer state, model weights, and the best Mean Per Joint Position Error (MPJPE).

```APIDOC
## save_checkpoint

### Description
Persists the current training state, including epoch number, learning rate, optimizer state, model weights, and the best Mean Per Joint Position Error (MPJPE).

### Parameters
- **checkpoint_path** (str) - Path to save the checkpoint file.
- **epoch** (int) - The current epoch number.
- **lr** (float) - The current learning rate.
- **optimizer** (torch.optim.Optimizer) - The optimizer state to save.
- **model** (torch.nn.Module) - The model weights to save.
- **min_mpjpe** (float) - The minimum MPJPE achieved so far.
- **wandb_id** (str) - Optional identifier for Weights & Biases logging.
```

--------------------------------

### Calculate Evaluation Metrics (NumPy)

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Computes MPJPE, P-MPJPE, and acceleration error using NumPy arrays. Ensure predictions and ground truth are detached from the computation graph and converted to NumPy arrays before use.

```python
pred_np = pred.detach().numpy()
gt_np   = gt.detach().numpy()
e1  = mpjpe(pred_np[0], gt_np[0])       # shape (243,), per-frame MPJPE
e2  = p_mpjpe(pred_np[0], gt_np[0])     # Protocol #2 (Procrustes-aligned)
acc = acc_error(pred_np[0], gt_np[0])   # acceleration error
print(f"MPJPE: {e1.mean():.1f} mm  P-MPJPE: {e2.mean():.1f} mm  Acc: {acc.mean():.2f}")
```

--------------------------------

### MotionAGFormerBlock - Dual-Branch Spatiotemporal Block

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Each block runs an attention branch and a graph branch in parallel, then fuses their outputs adaptively.

```APIDOC
## MotionAGFormerBlock - Dual-Branch Spatiotemporal Block

### Description
Each block runs an **attention branch** (spatial → temporal self-attention) and a **graph branch** (spatial GCN → temporal GCN / MS-TCN) on the same input, then fuses their outputs with a learned per-token soft weight via `nn.Linear(dim*2, 2) → softmax`. Setting `hierarchical=True` splits channels between branches instead of processing them independently.

### Usage Example
```python
from model.MotionAGFormer import MotionAGFormerBlock
import torch

# Standard adaptive-fusion block, 128-d features
block = MotionAGFormerBlock(
    dim=128,
    mlp_ratio=4,
    num_heads=8,
    use_adaptive_fusion=True,   # learned alpha per token
    hierarchical=False,
    use_temporal_similarity=True,
    neighbour_num=2,
    use_tcn=False,
    graph_only=False,
    n_frames=243,
)
x = torch.randn(4, 243, 17, 128)
out = block(x)
print(out.shape)   # torch.Size([4, 243, 17, 128])

# Graph-only variant (no AGFormerBlock wrappers, raw GCN + MS-TCN)
block_gcn = MotionAGFormerBlock(
    dim=128,
    graph_only=True,
    use_tcn=True,     # MS-TCN temporal
    n_frames=243,
)
out_gcn = block_gcn(x)
print(out_gcn.shape)  # torch.Size([4, 243, 17, 128])
```
```

--------------------------------

### MotionAGFormer Citation

Source: https://github.com/taatiteam/motionagformer/blob/master/README.md

If MotionAGFormer is useful for your project, please cite the paper using this BibTeX entry.

```bibtex
@inproceedings{motionagformer2024,
  title     =   {MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network},
  author    =   {Soroush Mehraban, Vida Adeli, Babak Taati},
  booktitle =   {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  year      =   {2024}
}
```

--------------------------------

### GCN Graph Convolutional Network Module

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Implements spatial or temporal GCN with adaptive adjacency. Spatial mode uses fixed skeleton adjacency. Temporal mode can build dynamic adjacency based on cosine similarity or use a fixed offset.

```python
from model.modules.graph import GCN
import torch
```

--------------------------------

### MotionAGFormer - Main Model

Source: https://context7.com/taatiteam/motionagformer/llms.txt

The top-level model that accepts a batch of 2D pose sequences and outputs lifted 3D poses. It combines spatial-temporal self-attention transformers with graph convolutional networks.

```APIDOC
## MotionAGFormer - Main Model

### Description
The top-level model that accepts a batch of 2D pose sequences `[B, T, J, C]` and outputs lifted 3D poses `[B, T, J, 3]`. Internally it projects input joints with a linear embedding, adds positional embeddings, passes through a stack of `MotionAGFormerBlock` layers, normalises, projects to a motion representation via a Tanh bottleneck, and finally regresses 3D coordinates.

### Usage Example
```python
import torch
from model.MotionAGFormer import MotionAGFormer

# Build MotionAGFormer-Base equivalent (243-frame, 17-joint, 3-channel input)
model = MotionAGFormer(
    n_layers=16,
    dim_in=3,           # (x, y, confidence)
    dim_feat=128,
    dim_rep=512,
    dim_out=3,          # (X, Y, Z) output
    mlp_ratio=4,
    num_heads=8,
    use_layer_scale=True,
    layer_scale_init_value=1e-5,
    use_adaptive_fusion=True,
    hierarchical=False,
    use_temporal_similarity=True,
    neighbour_num=2,
    temporal_connection_len=1,
    use_tcn=False,
    graph_only=False,
    n_frames=243,
    num_joints=17,
)
model.eval()

# Random inference: batch=2, T=243 frames, J=17 joints, C=3
x = torch.randn(2, 243, 17, 3)
with torch.no_grad():
    out = model(x)
    rep = model(x, return_rep=True)   # motion representation (512-d) for downstream tasks

print(out.shape)   # torch.Size([2, 243, 17, 3])
print(rep.shape)   # torch.Size([2, 243, 17, 512])
```
```

--------------------------------

### GCN - Graph Convolutional Network Module

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Implements a spatial or temporal GCN with adaptive adjacency.

```APIDOC
## GCN - Graph Convolutional Network Module

### Description
Implements a spatial or temporal GCN with adaptive adjacency. In **spatial** mode it uses a fixed skeleton adjacency (17-joint H36M connections). In **temporal** mode with `use_temporal_similarity=True` it builds a dynamic top-k cosine-similarity adjacency per forward pass; otherwise it uses a fixed diagonal-offset adjacency.

### Usage Example
```python
from model.modules.graph import GCN
import torch
```
```

--------------------------------

### Apply Horizontal Flip Augmentation

Source: https://context7.com/taatiteam/motionagformer/llms.txt

Mirrors pose sequences along the x-axis and swaps left/right joint indices. This function works with both NumPy arrays and PyTorch tensors, and is used for both training augmentation and test-time augmentation (TTA).

```python
from utils.data import flip_data
import torch, numpy as np

# Works with NumPy arrays
pose_np = np.random.randn(243, 17, 3)
flipped_np = flip_data(pose_np)   # shape unchanged

# Works with PyTorch tensors in batch form
pose_t = torch.randn(8, 243, 17, 3)
flipped_t = flip_data(pose_t)

# Test-time augmentation (TTA) pattern
model.eval()
with torch.no_grad():
    pred_orig = model(pose_t)
    pred_flip = model(flipped_t)
    pred_tta  = (pred_orig + flip_data(pred_flip)) / 2
```

--------------------------------

### MotionAGFormerBlock Dual-Branch Spatiotemporal Block

Source: https://context7.com/taatiteam/motionagformer/llms.txt

A dual-branch spatiotemporal block with adaptive fusion. Use for standard operation. The graph-only variant can be used for raw GCN + MS-TCN processing.

```python
from model.MotionAGFormer import MotionAGFormerBlock
import torch

# Standard adaptive-fusion block, 128-d features
block = MotionAGFormerBlock(
    dim=128,
    mlp_ratio=4,
    num_heads=8,
    use_adaptive_fusion=True,   # learned alpha per token
    hierarchical=False,
    use_temporal_similarity=True,
    neighbour_num=2,
    use_tcn=False,
    graph_only=False,
    n_frames=243,
)
x = torch.randn(4, 243, 17, 128)
out = block(x)
print(out.shape)   # torch.Size([4, 243, 17, 128])

# Graph-only variant (no AGFormerBlock wrappers, raw GCN + MS-TCN)
block_gcn = MotionAGFormerBlock(
    dim=128,
    graph_only=True,
    use_tcn=True,     # MS-TCN temporal
    n_frames=243,
)
out_gcn = block_gcn(x)
print(out_gcn.shape)  # torch.Size([4, 243, 17, 128])
```

--------------------------------

### AGFormerBlock Single-Branch ST/TS Block

Source: https://context7.com/taatiteam/motionagformer/llms.txt

A residual block for spatial or temporal mixing. Configure mixer type ('attention', 'graph', 'ms-tcn') and mode ('spatial', 'temporal'). Layer scaling and stochastic depth are applied.

```python
from model.MotionAGFormer import AGFormerBlock
import torch

# Spatial attention block
spatial_attn = AGFormerBlock(
    dim=128, num_heads=8, mode='spatial', mixer_type='attention',
    use_layer_scale=True, layer_scale_init_value=1e-5,
)
# Temporal graph block
temporal_gcn = AGFormerBlock(
    dim=128, mode='temporal', mixer_type='graph',
    use_temporal_similarity=True, neighbour_num=4, n_frames=243,
)

x = torch.randn(4, 243, 17, 128)
x = spatial_attn(x)   # spatial mixing
x = temporal_gcn(x)   # temporal mixing
print(x.shape)         # torch.Size([4, 243, 17, 128])
```

--------------------------------

### AGFormerBlock - Single-Branch ST/TS Block

Source: https://context7.com/taatiteam/motionagformer/llms.txt

A residual block whose mixer can be attention, graph, or MS-TCN, and operates in spatial or temporal mode.

```APIDOC
## AGFormerBlock - Single-Branch ST/TS Block

### Description
A residual block whose mixer can be `"attention"`, `"graph"`, or `"ms-tcn"`, and whose mode can be `"spatial"` (operates over the joint dimension) or `"temporal"` (operates over the frame dimension). Layer scaling and stochastic depth are applied to both mixer and MLP sub-layers.

### Usage Example
```python
from model.MotionAGFormer import AGFormerBlock
import torch

# Spatial attention block
spatial_attn = AGFormerBlock(
    dim=128, num_heads=8, mode='spatial', mixer_type='attention',
    use_layer_scale=True, layer_scale_init_value=1e-5,
)
# Temporal graph block
temporal_gcn = AGFormerBlock(
    dim=128, mode='temporal', mixer_type='graph',
    use_temporal_similarity=True, neighbour_num=4, n_frames=243,
)

x = torch.randn(4, 243, 17, 128)
x = spatial_attn(x)   # spatial mixing
x = temporal_gcn(x)   # temporal mixing
print(x.shape)         # torch.Size([4, 243, 17, 128])
```
```
Tokens:
Raw