InterFormer PyTorch Port

https://github.com/westbrooklong/interformer_pytorch
Admin
InterFormer PyTorch Port is a PyTorch implementation of the InterFormer architecture for...
Tokens:9,621
Snippets:83
Trust Score:6.3
Update:5 months ago
Show doc for...
Tokens:
Raw
### Show All Options

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Display all available command-line options for the `run_taac2026_sample.py` script.

```bash
python scripts/run_taac2026_sample.py --help
```

--------------------------------

### Run Training Sample

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/README.md

Initiate a training run using the sample script. Specify the number of epochs and batch size for the training process.

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32
```

--------------------------------

### Run Training Script with Minimal Configuration

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/AGENTS.md

Launches the training script with a small dataset size, limited epochs, and a small batch size for quick validation. Useful for testing the training pipeline's basic functionality.

```bash
python scripts/run_taac2026_sample.py --max-rows 32 --epochs 1 --batch-size 8
```

--------------------------------

### Basic Training Configuration

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Run a basic training session with specified epochs and batch size.

```bash
python scripts/run_taac2026_sample.py --epochs 10 --batch-size 32
```

--------------------------------

### Train with Default Paper-Style Interaction Arch

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/README.md

Run the training sample script using the default paper-style interaction backbone (DHEN). This allows for training with the specific interaction modules reported in the paper.

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32 --interaction-backbone dhen
```

--------------------------------

### Common Tuning Options

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Configure various training parameters including sequence length, model dimensions, attention heads, layers, and optimizer settings.

```bash
python scripts/run_taac2026_sample.py \
  --epochs 5 \
  --batch-size 32 \
  --seq-len 16 \
  --d-model 128 \
  --num-heads 4 \
  --num-layers 3 \
  --num-cls-tokens 4 \
  --num-pma-tokens 2 \
  --num-recent-tokens 2 \
  --interaction-backbone dhen \
  --interaction-layers 2 \
  --dcn-layers 2 \
  --cross-low-rank 32 \
  --lr 1e-3 \
  --weight-decay 1e-4
```

--------------------------------

### Resume Training

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/README.md

Resume a previous training session from a checkpoint file. Specify the total number of epochs, batch size, the checkpoint file to resume from, and enable checkpoint saving.

```bash
python scripts/run_taac2026_sample.py --epochs 10 --batch-size 32 --resume best_model_20260416_120000.pt --save-checkpoint
```

--------------------------------

### Resume Training from Checkpoint

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Resume training from a saved checkpoint. The `--resume` argument accepts a checkpoint filename or an absolute path. This restores model weights, optimizer state, GradScaler state, finished epoch, and best validation AUC.

```bash
python scripts/run_taac2026_sample.py --epochs 10 --batch-size 32 --resume best_model_20260416_120000.pt
```

```bash
python scripts/run_taac2026_sample.py --epochs 10 --batch-size 32 --resume best_model_20260416_120000.pt --save-checkpoint
```

```bash
python scripts/run_taac2026_sample.py --epochs 10 --batch-size 32 --resume D:\Users\WESTBROOK\PycharmProjects\RecAlgo\Interformer_Pytorch\outputs\taac2026_sample\best_model_20260416_120000.pt
```

--------------------------------

### Run Backbone Sanity Check

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/README.md

Execute the main PyTorch script to perform a sanity check on the backbone implementation. No specific arguments are required for this basic check.

```bash
python main_pytorch.py
```

--------------------------------

### Choose Interaction Backbone

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Select the interaction backbone for the model. Available options include 'dhen', 'dot', and 'dcn'.

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32 --interaction-backbone dhen
```

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32 --interaction-backbone dot
```

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32 --interaction-backbone dcn
```

--------------------------------

### Initialize and Use DHENInteraction

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Instantiate the DHENInteraction backbone, which stacks multiple DHENLayer modules for heterogeneous cross-feature interaction. It combines Dot-Product Interaction and DCN outputs.

```python
import torch
from main_pytorch import DHENInteraction

x = torch.randn(4, 14, 64)  # [batch, num_tokens, d_model]

dhen = DHENInteraction(
    num_tokens=14,
    d_model=64,
    num_layers=2,
    dcn_layers=2,
    low_rank=16,
    deep_hidden=256,
    dropout=0.1,
)
out = dhen(x)   # shape: [4, 14, 64]
print(out.shape)
```

--------------------------------

### Train and Save Checkpoint

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Train the model for a specified number of epochs and save the best performing checkpoint. Saved files are timestamped.

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32 --save-checkpoint
```

--------------------------------

### Run TAAC2026 Training Script

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Executes the main training script for TAAC2026, supporting various configurations like epochs, batch size, backbone choice, mixed precision, checkpoint saving/resuming, and limiting rows for quick checks.

```bash
# Basic training run (5 epochs, batch size 32)
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32

# Use DHEN backbone (default, matches paper benchmark)
python scripts/run_taac2026_sample.py \
    --epochs 10 --batch-size 64 \
    --interaction-backbone dhen \
    --d-model 128 --num-layers 3 --num-heads 4 \
    --amp --amp-dtype bf16 \
    --save-checkpoint

# Use a local parquet file instead of downloading
python scripts/run_taac2026_sample.py \
    --local-parquet data/sample.parquet \
    --epochs 5 --batch-size 32

# Resume from a saved checkpoint
python scripts/run_taac2026_sample.py \
    --epochs 20 --batch-size 32 \
    --resume best_model_20260416_120000.pt \
    --save-checkpoint

# Quick sanity check with row limit
python scripts/run_taac2026_sample.py \
    --max-rows 200 --epochs 2 --batch-size 16

# Backbone sanity check (no training, prints tensor shapes)
python main_pytorch.py
```

--------------------------------

### Initialize and Use PersonalizedFFN

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Instantiate PersonalizedFFN, a user-conditioned feed-forward network. It generates dynamic per-sample weights from a non-sequence summary to personalize sequence token transformations.

```python
import torch
from main_pytorch import PersonalizedFFN

pffn = PersonalizedFFN(
    num_summary_tokens=4,
    d_model=64,
    hidden_dim=128,
    dropout=0.1,
)
non_seq_summary = torch.randn(4, 4, 64)  # [batch, num_cls_tokens, d_model]
seq             = torch.randn(4, 20, 64) # [batch, seq_len, d_model]
out = pffn(non_seq_summary, seq)         # shape: [4, 20, 64]
print(out.shape)
```

--------------------------------

### Tune InterFormer Summarization Tokens

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Adjust the number of classification, PMA, and recent tokens for InterFormer summarization.

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32 --num-cls-tokens 4 --num-pma-tokens 2 --num-recent-tokens 2
```

--------------------------------

### Initialize and Use DCNInteraction

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Instantiate the DCNInteraction module, which applies low-rank token-mixing cross layers in parallel with a token-wise deep MLP. It can be used standalone or within DHEN.

```python
import torch
from main_pytorch import DCNInteraction

x = torch.randn(4, 14, 64)

dcn = DCNInteraction(
    num_tokens=14,
    d_model=64,
    num_layers=2,
    low_rank=16,
    deep_hidden=256,
    dropout=0.0,
)
out = dcn(x)   # shape: [4, 14, 64]
print(out.shape)
```

--------------------------------

### Initialize and Use InterFormerBlock

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Instantiate the InterFormerBlock with specified dimensions and parameters. This block performs a single cross-modal update operation on non-sequence and sequence tokens.

```python
import torch
from main_pytorch import InterFormerBlock

block = InterFormerBlock(
    non_seq_tokens=12,
    num_cls_tokens=4,
    num_pma_tokens=2,
    num_recent_tokens=2,
    d_model=64,
    num_heads=4,
    interaction_backbone="dhen",
    interaction_layers=2,
    dcn_layers=2,
    cross_low_rank=16,
    summary_hidden=128,
    interaction_hidden=256,
    sequence_hidden=128,
    dropout=0.0,
)
non_seq = torch.randn(4, 12, 64)   # [batch, non_seq_tokens, d_model]
seq     = torch.randn(4, 20, 64)   # [batch, cls+seq_len, d_model]
next_non_seq, next_seq = block(non_seq, seq)
print(next_non_seq.shape)  # [4, 12, 64]
print(next_seq.shape)      # [4, 20, 64]
```

--------------------------------

### Build Tensors from Flat Schema Rows

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Converts a list of flat schema rows into tensors for model input. Prints the shapes of the resulting tensors and extracted labels/metadata.

```python
rows = [
    {
        "label_type": 1,
        "label_time": 1700000000,
        "domain_watch_time": [12.0, 8.5, 20.1],
        "domain_click_count": [3.0, 1.0, 5.0],
        "play_ratio": 0.75,
        "video_duration": 120.0,
    },
    {
        "label_type": 0,
        "label_time": 1700000010,
        "domain_watch_time": [5.0, 3.0],
        "domain_click_count": [1.0, 0.0],
        "play_ratio": 0.3,
        "video_duration": 60.0,
    },
]
non_seq_x, seq_x, labels, metadata = build_tensors(rows, seq_len=16)
print(non_seq_x.shape)  # [2, <non_seq_dim>]
print(seq_x.shape)      # [2, 16, <seq_feature_dim>]
print(labels)           # tensor([1, 0])
print(metadata["schema"])        # "flat"
print(metadata["non_seq_dim"])   # int
```

--------------------------------

### Read Local Parquet Data

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Specify a local Parquet file for training data using the `--local-parquet` argument.

```bash
python scripts/run_taac2026_sample.py --local-parquet D:\path\to\demo_1000.parquet --epochs 5 --batch-size 32
```

--------------------------------

### Load TAAC2026 Dataset with HuggingFace Fallback

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Loads the TAAC2026 dataset, prioritizing Hugging Face Hub with automatic retries and caching. It can also load from a local parquet file to bypass network I/O.

```python
from pathlib import Path
from utils.taac_data import load_train_split

# Load from Hugging Face (with automatic retry and caching)
dataset = load_train_split(
    dataset_id="TAAC2026/data_sample_1000",
    cache_dir=Path(".cache/taac2026"),
    local_parquet=None,
)
print(len(dataset))  # number of rows

# Or load from a local parquet file
dataset = load_train_split(
    dataset_id="TAAC2026/data_sample_1000",
    cache_dir=Path(".cache/taac2026"),
    local_parquet=Path("data/sample.parquet"),
)
```

--------------------------------

### Initialize and Use TAACInterFormerClassifier

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Instantiate the TAACInterFormerClassifier, an end-to-end trainable classifier. It wraps the InterFormerEncoder pipeline and accepts raw non-sequence and sequence feature tensors.

```python
import torch
from models.taac_interformer import TAACInterFormerClassifier

model = TAACInterFormerClassifier(
    non_seq_dim=12,
    seq_feature_dim=5,
    num_classes=3,
    d_model=64,
    num_heads=4,
    num_layers=3,
    num_cls_tokens=4,
    num_pma_tokens=2,
    num_recent_tokens=2,
    interaction_backbone="dhen",
    dropout=0.1,
)
non_seq_x = torch.randn(8, 12)      # [batch, non_seq_dim]
seq_x     = torch.randn(8, 16, 5)   # [batch, seq_len, seq_feature_dim]
logits = model(non_seq_x, seq_x)    # shape: [8, 3]
print(logits.shape)  # torch.Size([8, 3])
probs = torch.softmax(logits, dim=-1)
```

--------------------------------

### Compile Python Modules

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/AGENTS.md

Use this command to check for syntax errors in the specified Python files. It's a quick way to ensure basic code integrity before running.

```bash
python -m py_compile main_pytorch.py models/taac_interformer.py scripts/run_taac2026_sample.py
```

--------------------------------

### Set Random Seed and Split Indices

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Ensures reproducibility by setting random seeds for Python, PyTorch, and CUDA. Generates shuffled train/validation indices from a fixed seed, useful for consistent data splitting.

```python
import torch
from utils.common import set_seed, split_indices

set_seed(42)

train_idx, val_idx = split_indices(size=1000, val_ratio=0.2, seed=42)
print(len(train_idx), len(val_idx))  # 800 200

# Use with TensorDataset
from torch.utils.data import TensorDataset, DataLoader
X = torch.randn(1000, 32)
y = torch.randint(0, 3, (1000,))
train_loader = DataLoader(TensorDataset(X[train_idx], y[train_idx]), batch_size=32, shuffle=True)
val_loader   = DataLoader(TensorDataset(X[val_idx],   y[val_idx]),   batch_size=32, shuffle=False)
```

--------------------------------

### DHENInteraction

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

DHEN heterogeneous interaction backbone that stacks multiple DHENLayer modules for rich cross-feature interaction.

```APIDOC
## `DHENInteraction` — DHEN heterogeneous interaction backbone

Stacks multiple `DHENLayer` modules, each of which ensembles Dot-Product Interaction and DCN (Deep & Cross Network) outputs through a learned projection and residual shortcut, providing rich cross-feature interaction.

```python
import torch
from main_pytorch import DHENInteraction

x = torch.randn(4, 14, 64)  # [batch, num_tokens, d_model]

dhen = DHENInteraction(
    num_tokens=14,
    d_model=64,
    num_layers=2,
    dcn_layers=2,
    low_rank=16,
    deep_hidden=256,
    dropout=0.1,
)
out = dhen(x)   # shape: [4, 14, 64]
print(out.shape)
```
```

--------------------------------

### InterFormerEncoder: Full heterogeneous interaction encoder

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Stacks `InterFormerBlock` layers to cross-update non-sequence and sequence branches. Returns updated tokens and summary vectors for both branches. Supports various interaction backbones like 'dhen', 'dot', and 'dcn'.

```python
import torch
from main_pytorch import NumericFieldTokenizer, SequenceMaskNet, InterFormerEncoder

torch.manual_seed(0)
batch_size, non_seq_dim, seq_len, seq_feat_dim, d_model = 4, 12, 16, 5, 64

non_seq_tokens = NumericFieldTokenizer(non_seq_dim, d_model)(torch.randn(batch_size, non_seq_dim))
seq_tokens = SequenceMaskNet(seq_feat_dim, d_model)(torch.randn(batch_size, seq_len, seq_feat_dim))

encoder = InterFormerEncoder(
    non_seq_tokens=non_seq_dim,
    d_model=d_model,
    num_heads=4,
    num_layers=3,
    num_cls_tokens=4,
    num_pma_tokens=2,
    num_recent_tokens=2,
    interaction_backbone="dhen",  # "dhen" | "dot" | "dcn"
    interaction_layers=2,
    dcn_layers=2,
    cross_low_rank=16,
    summary_hidden=128,
    interaction_hidden=256,
    sequence_hidden=128,
    dropout=0.1,
)
non_seq_out, seq_out, non_seq_summary, seq_summary = encoder(
    non_seq_tokens=non_seq_tokens,
    seq_tokens=seq_tokens,
)
# non_seq_out:     [4, 12,  64]   — updated non-seq tokens
# seq_out:         [4, 20,  64]   — CLS(4) + seq(16) tokens
# non_seq_summary: [4,  4,  64]   — compressed non-seq summary
# seq_summary:     [4,  8,  64]   — CLS(4)+PMA(2)+recent(2) summary
print(non_seq_summary.shape, seq_summary.shape)
```

--------------------------------

### Enable/Disable Mixed Precision Training

Source: https://github.com/westbrooklong/interformer_pytorch/blob/main/Test.md

Control automatic mixed precision (AMP) training. CUDA enables AMP by default. Use `--amp-dtype bf16` to specify the dtype or `--no-amp` to disable.

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32
```

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32 --amp-dtype bf16
```

```bash
python scripts/run_taac2026_sample.py --epochs 5 --batch-size 32 --no-amp
```

--------------------------------

### SequenceMaskNet: MaskNet-style sequence token encoder

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Encodes multi-channel sequences into dense per-step tokens. Each time-step's multi-field values are embedded, merged, gated through a mask MLP, and projected to `d_model` via an LCE MLP.

```python
import torch
from main_pytorch import SequenceMaskNet

# 4 samples, 16 time steps, 5 features per step
seq_x = torch.randn(4, 16, 5)

seq_tokenizer = SequenceMaskNet(
    num_sequence_fields=5,
    d_model=64,
    hidden_dim=128,
    dropout=0.1,
)
seq_tokens = seq_tokenizer(seq_x)  # shape: [4, 16, 64]
print(seq_tokens.shape)  # torch.Size([4, 16, 64])
```

--------------------------------

### Initialize and Use RotaryMultiHeadAttention

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Instantiate RotaryMultiHeadAttention for self-attention with Rotary Position Embedding (RoPE). This is used within SequenceArch to model temporal dependencies.

```python
import torch
from main_pytorch import RotaryMultiHeadAttention

seq = torch.randn(4, 20, 64)  # [batch, seq_len, d_model]

rope_attn = RotaryMultiHeadAttention(
    d_model=64,
    num_heads=4,
    dropout=0.1,
    rope_base=10000.0,
)
out = rope_attn(seq)   # shape: [4, 20, 64]
print(out.shape)
```

--------------------------------

### InterFormerBlock

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

A single cross-modal update block that performs compression, summarization, and interaction updates between non-sequential and sequential tokens.

```APIDOC
## `InterFormerBlock` — single cross-modal update block

One block performs four operations: compress non-seq tokens into a summary, summarize sequence tokens via CLS+PMA+recent, run the Interaction Arch to update non-seq tokens using the sequence summary, and run the Sequence Arch (PFFN + RoPE attention) to update sequence tokens using the non-seq summary.

```python
import torch
from main_pytorch import InterFormerBlock

block = InterFormerBlock(
    non_seq_tokens=12,
    num_cls_tokens=4,
    num_pma_tokens=2,
    num_recent_tokens=2,
    d_model=64,
    num_heads=4,
    interaction_backbone="dhen",
    interaction_layers=2,
    dcn_layers=2,
    cross_low_rank=16,
    summary_hidden=128,
    interaction_hidden=256,
    sequence_hidden=128,
    dropout=0.0,
)
non_seq = torch.randn(4, 12, 64)   # [batch, non_seq_tokens, d_model]
seq     = torch.randn(4, 20, 64)   # [batch, cls+seq_len, d_model]
next_non_seq, next_seq = block(non_seq, seq)
print(next_non_seq.shape)  # [4, 12, 64]
print(next_seq.shape)      # [4, 20, 64]
```
```

--------------------------------

### Calculate Accuracy and AUC from Logits

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Computes top-1 accuracy and macro-averaged AUC from raw model logits. For binary classification, multiclass_auc_from_logits automatically reduces to binary AUC.

```python
import torch
from utils.metrics import accuracy_from_logits, multiclass_auc_from_logits

logits = torch.tensor([[2.0, 0.5, 0.1], [0.1, 0.3, 3.0], [1.5, 0.2, 0.1]])
labels = torch.tensor([0, 2, 0])

acc = accuracy_from_logits(logits, labels)
print(f"Accuracy: {acc:.4f}")   # 1.0

auc = multiclass_auc_from_logits(logits, labels)
print(f"AUC: {auc:.4f}")        # macro-averaged one-vs-rest AUC

# Binary case: automatically uses binary AUC
binary_logits = torch.tensor([[0.9, 0.1], [0.2, 0.8], [0.6, 0.4]])
binary_labels = torch.tensor([0, 1, 0])
auc_bin = multiclass_auc_from_logits(binary_logits, binary_labels)
print(f"Binary AUC: {auc_bin:.4f}")
```

--------------------------------

### SequenceMaskNet

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Encodes a multi-channel sequence (e.g., per-step behavioral signals) into dense per-step tokens. Each time-step's multi-field values are embedded, merged into a flat vector, gated through a mask MLP, and projected to `d_model` via a separate LCE MLP.

```APIDOC
## SequenceMaskNet

### Description
Encodes a multi-channel sequence (e.g., per-step behavioral signals) into dense per-step tokens. Each time-step's multi-field values are embedded, merged into a flat vector, gated through a mask MLP, and projected to `d_model` via a separate LCE MLP.

### Usage
```python
import torch
from main_pytorch import SequenceMaskNet

# 4 samples, 16 time steps, 5 features per step
seq_x = torch.randn(4, 16, 5)

seq_tokenizer = SequenceMaskNet(
    num_sequence_fields=5,
    d_model=64,
    hidden_dim=128,
    dropout=0.1,
)
seq_tokens = seq_tokenizer(seq_x)  # shape: [4, 16, 64]
print(seq_tokens.shape)
```

### Parameters
- **num_sequence_fields** (int) - The number of features per time step in the sequence.
- **d_model** (int) - The dimensionality of the output token vectors.
- **hidden_dim** (int) - The dimensionality of the hidden layers in the MLPs.
- **dropout** (float) - The dropout rate.
```

--------------------------------

### DCNInteraction

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Deep & Cross Network interaction that applies low-rank token-mixing cross layers in parallel with a token-wise deep MLP.

```APIDOC
## `DCNInteraction` — Deep & Cross Network interaction

Applies low-rank token-mixing cross layers in parallel with a token-wise deep MLP, then combines them with layer normalization. Can be used standalone or as the cross sub-module inside DHEN.

```python
import torch
from main_pytorch import DCNInteraction

x = torch.randn(4, 14, 64)

dcn = DCNInteraction(
    num_tokens=14,
    d_model=64,
    num_layers=2,
    low_rank=16,
    deep_hidden=256,
    dropout=0.0,
)
out = dcn(x)   # shape: [4, 14, 64]
print(out.shape)
```
```

--------------------------------

### Build Tensors for TAAC Data

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Utilize the build_tensors function for automatic schema detection and tensorization of raw dataset rows. It handles scalar, array, and sequence features, applying normalization.

```python
from utils.taac_data import build_tensors
```

--------------------------------

### PersonalizedFFN

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

User-conditioned feed-forward network that generates dynamic per-sample weights from the non-sequence summary and applies them to sequence tokens.

```APIDOC
## `PersonalizedFFN` — user-conditioned feed-forward network

Generates dynamic per-sample weights from the non-sequence summary and applies them to sequence tokens, enabling personalized transformation of behavioral history conditioned on static user/item features.

```python
import torch
from main_pytorch import PersonalizedFFN

pffn = PersonalizedFFN(
    num_summary_tokens=4,
    d_model=64,
    hidden_dim=128,
    dropout=0.1,
)
non_seq_summary = torch.randn(4, 4, 64)  # [batch, num_cls_tokens, d_model]
seq             = torch.randn(4, 20, 64) # [batch, seq_len, d_model]
out = pffn(non_seq_summary, seq)         # shape: [4, 20, 64]
print(out.shape)
```
```

--------------------------------

### NumericFieldTokenizer

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Maps a batch of scalar numeric features into per-field token vectors by learning a weight and bias for each field independently, producing a `[batch, num_fields, d_model]` tensor suitable for downstream transformer processing.

```APIDOC
## NumericFieldTokenizer

### Description
Maps a batch of scalar numeric features into per-field token vectors by learning a weight and bias for each field independently, producing a `[batch, num_fields, d_model]` tensor suitable for downstream transformer processing.

### Usage
```python
import torch
from main_pytorch import NumericFieldTokenizer

# 4 samples, each with 12 scalar numeric features
non_seq_x = torch.randn(4, 12)

tokenizer = NumericFieldTokenizer(num_fields=12, d_model=64)
tokens = tokenizer(non_seq_x)  # shape: [4, 12, 64]
print(tokens.shape)
```

### Parameters
- **num_fields** (int) - The number of input fields.
- **d_model** (int) - The dimensionality of the output token vectors.
```

--------------------------------

### InterFormerEncoder

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

The top-level encoder stacks `L` `InterFormerBlock` layers, each of which cross-updates non-sequence and sequence branches. Returns four tensors: updated non-sequence tokens, updated sequence tokens (with prepended CLS tokens), and final summary vectors for both branches.

```APIDOC
## InterFormerEncoder

### Description
The top-level encoder stacks `L` `InterFormerBlock` layers, each of which cross-updates non-sequence and sequence branches. Returns four tensors: updated non-sequence tokens, updated sequence tokens (with prepended CLS tokens), and final summary vectors for both branches.

### Usage
```python
import torch
from main_pytorch import NumericFieldTokenizer, SequenceMaskNet, InterFormerEncoder

torch.manual_seed(0)
batch_size, non_seq_dim, seq_len, seq_feat_dim, d_model = 4, 12, 16, 5, 64

non_seq_tokens = NumericFieldTokenizer(non_seq_dim, d_model)(torch.randn(batch_size, non_seq_dim))
seq_tokens = SequenceMaskNet(seq_feat_dim, d_model)(torch.randn(batch_size, seq_len, seq_feat_dim))

encoder = InterFormerEncoder(
    non_seq_tokens=non_seq_dim,
    d_model=d_model,
    num_heads=4,
    num_layers=3,
    num_cls_tokens=4,
    num_pma_tokens=2,
    num_recent_tokens=2,
    interaction_backbone="dhen",  # "dhen" | "dot" | "dcn"
    interaction_layers=2,
    dcn_layers=2,
    cross_low_rank=16,
    summary_hidden=128,
    interaction_hidden=256,
    sequence_hidden=128,
    dropout=0.1,
)
non_seq_out, seq_out, non_seq_summary, seq_summary = encoder(
    non_seq_tokens=non_seq_tokens,
    seq_tokens=seq_tokens,
)
# non_seq_out:     [4, 12,  64]   — updated non-seq tokens
# seq_out:         [4, 20,  64]   — CLS(4) + seq(16) tokens
# non_seq_summary: [4,  4,  64]   — compressed non-seq summary
# seq_summary:     [4,  8,  64]   — CLS(4)+PMA(2)+recent(2) summary
print(non_seq_summary.shape, seq_summary.shape)
```

### Parameters
- **non_seq_tokens** (int) - The number of non-sequence input features.
- **d_model** (int) - The dimensionality of the model.
- **num_heads** (int) - The number of attention heads.
- **num_layers** (int) - The number of InterFormer blocks.
- **num_cls_tokens** (int) - The number of classification tokens.
- **num_pma_tokens** (int) - The number of pooling multi-head attention tokens.
- **num_recent_tokens** (int) - The number of recent tokens for sequence processing.
- **interaction_backbone** (str) - The backbone for interaction layers. Options: "dhen", "dot", "dcn".
- **interaction_layers** (int) - The number of interaction layers.
- **dcn_layers** (int) - The number of DCN layers (if `interaction_backbone` is "dcn").
- **cross_low_rank** (int) - The low-rank dimension for cross-attention.
- **summary_hidden** (int) - The hidden dimension for summary projection.
- **interaction_hidden** (int) - The hidden dimension for interaction layers.
- **sequence_hidden** (int) - The hidden dimension for sequence processing.
- **dropout** (float) - The dropout rate.

### Returns
- **non_seq_out** (torch.Tensor) - Updated non-sequence tokens.
- **seq_out** (torch.Tensor) - Updated sequence tokens (with CLS tokens).
- **non_seq_summary** (torch.Tensor) - Compressed non-sequence summary.
- **seq_summary** (torch.Tensor) - Compressed sequence summary.
```

--------------------------------

### NumericFieldTokenizer: Per-field embedding of scalar inputs

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Maps scalar numeric features into per-field token vectors by learning a weight and bias for each field independently. Produces a `[batch, num_fields, d_model]` tensor for downstream transformer processing.

```python
import torch
from main_pytorch import NumericFieldTokenizer

# 4 samples, each with 12 scalar numeric features
non_seq_x = torch.randn(4, 12)

tokenizer = NumericFieldTokenizer(num_fields=12, d_model=64)
tokens = tokenizer(non_seq_x)  # shape: [4, 12, 64]
print(tokens.shape)  # torch.Size([4, 12, 64])
```

--------------------------------

### TAACInterFormerClassifier

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

An end-to-end trainable classifier that wraps the InterFormer encoder pipeline with a two-layer MLP head.

```APIDOC
## `TAACInterFormerClassifier` — end-to-end trainable classifier

Wraps the full encoder pipeline (field tokenization → MaskNet → InterFormerEncoder) into a classification model with a two-layer MLP head. Accepts raw non-sequence and sequence feature tensors directly.

```python
import torch
from models.taac_interformer import TAACInterFormerClassifier

model = TAACInterFormerClassifier(
    non_seq_dim=12,
    seq_feature_dim=5,
    num_classes=3,
    d_model=64,
    num_heads=4,
    num_layers=3,
    num_cls_tokens=4,
    num_pma_tokens=2,
    num_recent_tokens=2,
    interaction_backbone="dhen",
    dropout=0.1,
)
non_seq_x = torch.randn(8, 12)      # [batch, non_seq_dim]
seq_x     = torch.randn(8, 16, 5)   # [batch, seq_len, seq_feature_dim]
logits = model(non_seq_x, seq_x)    # shape: [8, 3]
print(logits.shape)  # torch.Size([8, 3])
probs = torch.softmax(logits, dim=-1)
```
```

--------------------------------

### RotaryMultiHeadAttention

Source: https://context7.com/westbrooklong/interformer_pytorch/llms.txt

Multi-head self-attention with Rotary Position Embedding (RoPE), used for modeling temporal dependencies within sequence branches.

```APIDOC
## `RotaryMultiHeadAttention` — RoPE self-attention for sequences

Multi-head self-attention with Rotary Position Embedding (RoPE). Used inside the `SequenceArch` to model temporal dependencies within the behavioral sequence branch.

```python
import torch
from main_pytorch import RotaryMultiHeadAttention

seq = torch.randn(4, 20, 64)  # [batch, seq_len, d_model]

rope_attn = RotaryMultiHeadAttention(
    d_model=64,
    num_heads=4,
    dropout=0.1,
    rope_base=10000.0,
)
out = rope_attn(seq)   # shape: [4, 20, 64]
print(out.shape)
```
```
Tokens:
Raw