BSARec

https://github.com/yehjin-shin/bsarec
Admin
BSARec is a sequential recommendation model that leverages Fourier transform to balance inductive...
Tokens:11,076
Snippets:65
Trust Score:8
Update:10 months ago
Show doc for...
Tokens:
Raw
### Example Training Command for BSARec on Beauty Dataset

Source: https://github.com/yehjin-shin/bsarec/blob/main/README.md

An example of how to train the BSARec model using the Beauty dataset with specific hyperparameters. This command demonstrates the typical usage for initiating a training run.

```python
python main.py  --data_name Beauty \
                --lr 0.0005 \
                --alpha 0.7 \
                --c 5 \
                --num_attention_heads 1 \
                --train_name BSARec_Beauty
```

--------------------------------

### Set Up BSARec Conda Environment

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Provides commands to set up the conda environment for the BSARec project. It shows how to create an environment from a YAML file, activate it, and lists manual installation steps for PyTorch, CUDA, NumPy, SciPy, and tqdm if needed. It also includes a verification command to check the PyTorch and CUDA installation.

```bash
# Create environment from YAML file
conda env create -f bsarec_env.yaml
conda activate bsarec

# Manual installation
conda create -n bsarec python=3.9.7
conda activate bsarec
conda install pytorch=1.8.1 cudatoolkit=11.1.74 -c pytorch -c nvidia
conda install numpy=1.24.3
pip install scipy==1.11.1 tqdm==1.65.0

# Verify installation
python -c "import torch; print(f'PyTorch {torch.__version__}, CUDA available: {torch.cuda.is_available()}')"
```

--------------------------------

### Example Testing Command for BSARec on Beauty Dataset

Source: https://github.com/yehjin-shin/bsarec/blob/main/README.md

An example of how to test a pretrained BSARec model using the Beauty dataset. This command specifies the dataset, model parameters, and the name of the best model to load for evaluation.

```python
python main.py  --data_name Beauty \
                --alpha 0.7 \
                --c 5 \
                --num_attention_heads 1 \
                --load_model BSARec_Beauty_best \
                --do_eval
```

--------------------------------

### Load Sequential Recommendation Datasets in Python

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

This Python code snippet shows how to load sequential user interaction data from a specified directory and prepare it for training recommendation models. It uses functions from the 'dataset' and 'utils' modules to parse arguments, get sequence dictionaries, create train/validation/test dataloaders with padding and negative sampling, and generate rating matrices. The dataloaders yield batches containing user IDs, input sequences, target items, and negative samples.

```python
from dataset import get_seq_dic, get_dataloder, get_rating_matrix
from utils import parse_args

# Parse configuration
args = parse_args()
args.data_name = 'Beauty'
args.data_dir = './data/'
args.batch_size = 256
args.max_seq_length = 50
args.num_workers = 4

# Load sequence dictionary
seq_dic, max_item, num_users = get_seq_dic(args)
args.item_size = max_item + 1
args.num_users = num_users + 1

# Create dataloaders
train_dataloader, eval_dataloader, test_dataloader = get_dataloder(args, seq_dic)

# Each batch contains:
# - user_ids: [batch_size] tensor of user indices
# - input_ids: [batch_size, max_seq_length] padded item sequences
# - answers: [batch_size] target items
# - neg_answer: [batch_size] negative samples
# - same_target: [batch_size, max_seq_length] for contrastive learning (if applicable)

# Generate rating matrices for evaluation
valid_rating_matrix, test_rating_matrix = get_rating_matrix(args.data_name, seq_dic, max_item)
# Rating matrices are scipy.sparse.csr_matrix with shape [num_users, num_items]
```

--------------------------------

### Configure BSARec Model Training Parameters

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Configures model training settings for BSARec using Python. It parses command-line arguments and sets parameters such as data directories, batch size, learning rate, model architecture details (hidden size, attention heads), dropout probabilities, and BSARec-specific parameters like 'alpha' and 'c'. It also handles seed setting, output directory creation, and logging setup.

```python
import os
from utils import parse_args, set_seed, check_path, set_logger

# Parse arguments with model-specific defaults
args = parse_args()

# Basic configuration
args.data_dir = './data/'
args.output_dir = './output/'
args.data_name = 'Beauty'
args.batch_size = 256
args.epochs = 200
args.lr = 0.001
args.seed = 42
args.gpu_id = '0'
args.no_cuda = False
args.patience = 10
args.log_freq = 1

# Model architecture
args.model_type = 'BSARec'
args.max_seq_length = 50
args.hidden_size = 64
args.num_hidden_layers = 2
args.num_attention_heads = 1
args.hidden_dropout_prob = 0.5
args.attention_probs_dropout_prob = 0.5
args.initializer_range = 0.02

# BSARec-specific parameters
args.alpha = 0.7  # balance between frequency and attention (0=pure attention, 1=pure frequency)
args.c = 5  # frequency cutoff (filters out high frequencies beyond c)

# SASRec uses binary cross-entropy loss with negative sampling
# BSARec uses cross-entropy over full item vocabulary

# Set random seeds for reproducibility
set_seed(args.seed)

# Create output directory if needed
check_path(args.output_dir)

# Setup logging
logger = set_logger(os.path.join(args.output_dir, 'training.log'))
logger.info(f"Configuration: {args}")
```

--------------------------------

### Train Baseline Sequential Recommendation Models

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

This section demonstrates how to train various baseline sequential recommendation models by specifying the 'model_type' argument. Examples include training SASRec with a learning rate, GRU4Rec with its specific hidden size, and BERT4Rec with a mask ratio. The available model types are listed, allowing users to switch between architectures like Caser, GRU4Rec, SASRec, BERT4Rec, FMLPRec, DuoRec, FEARec, and BSARec.

```bash
# Train SASRec baseline
python main.py \
    --model_type SASRec \
    --data_name Beauty \
    --num_attention_heads 1 \
    --lr 0.001 \
    --train_name SASRec_Beauty

# Train GRU4Rec baseline
python main.py \
    --model_type GRU4Rec \
    --data_name Beauty \
    --gru_hidden_size 64 \
    --train_name GRU4Rec_Beauty

# Train BERT4Rec with masking
python main.py \
    --model_type BERT4Rec \
    --data_name Beauty \
    --mask_ratio 0.2 \
    --train_name BERT4Rec_Beauty
```

--------------------------------

### Dataset Selection Prompt

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/data/process/README.md

This is the interactive prompt displayed after running the processing script, listing available datasets and asking for user input. It guides the user to enter a specific dataset name or 'all'.

```text
Available datasets: ['Beauty', 'Sports_and_Outdoors', 'Toys_and_Games', 'LastFM', 'ML-1M']
Enter the name of the dataset (or type 'all' to process all datasets):
```

--------------------------------

### Get Sequence Output for Each Layer

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure3.ipynb

Defines a function `get_seqout` to load and concatenate output sequences from different layers of a model.  It iterates through files in a specified directory, extracts layer-specific output, and concatenates them into a list of output sequences for each layer. The output shape is (data_size, hidden_dim).

```python
def get_seqout(save_path, n_layers=16):
    
    # Dictionary comprehension to initialize the layer data lists
    layer_dict = {i: [] for i in range(n_layers + 1)}

    # npy file name list
    file_lst = [os.path.join(save_path, f) for f in sorted(os.listdir(save_path))]

    # Load and append data for each layer
    for file in sorted(file_lst):
        layer_num = file.split('/')[-1].split('layer_')[0] # format: "0layer_0iter.npy"
        layer_num = eval(layer_num)
        tmp = np.load(file)[:,-1,:]
        layer_dict[layer_num].append(tmp)
    
    # Concatenate the arrays for each layer
    layers = [np.concatenate(layer_dict[i]) for i in sorted(layer_dict.keys())]

    return layers
```

--------------------------------

### Execute Full Training Loop with Early Stopping and Validation

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Sets up and runs a complete training pipeline for a recommendation model. It includes initializing the trainer, setting up early stopping based on validation performance, and performing training, validation, and testing epochs. The best model is saved and loaded for final testing.

```python
from trainers import Trainer
from utils import EarlyStopping, set_logger, check_path, set_seed
import os
import torch
import numpy as np

# Setup
set_seed(42)
check_path('./output')
logger = set_logger('./output/training.log')
args.checkpoint_path = './output/model_best.pt'

# Initialize trainer
trainer = Trainer(
    model=model,
    train_dataloader=train_dataloader,
    eval_dataloader=eval_dataloader,
    test_dataloader=test_dataloader,
    args=args,
    logger=logger
)

# Setup early stopping
early_stopping = EarlyStopping(
    checkpoint_path=args.checkpoint_path,
    logger=logger,
    patience=10,
    verbose=True
)

# Training loop
args.valid_rating_matrix, args.test_rating_matrix = get_rating_matrix(
    args.data_name, seq_dic, max_item
)

for epoch in range(args.epochs):
    # Train one epoch
    trainer.train(epoch)

    # Validate
    scores, _ = trainer.valid(epoch)
    # scores = [HR@5, NDCG@5, HR@10, NDCG@10, HR@20, NDCG@20]

    # Check early stopping on MRR (last metric)
    early_stopping(np.array(scores[-1:]), trainer.model)
    if early_stopping.early_stop:
        logger.info("Early stopping triggered")
        break

# Load best model and test
trainer.model.load_state_dict(torch.load(args.checkpoint_path))
test_scores, result_info = trainer.test(0)
logger.info(f"Test Results: {result_info}")
```

--------------------------------

### Switch Between Different Sequential Recommendation Models

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Demonstrates how to instantiate and use various sequential recommendation models (SASRec, GRU4Rec, BERT4Rec, DuoRec) through a unified interface provided by MODEL_DICT. It shows setting model-specific arguments and performing forward passes or loss calculations.

```python
from model import MODEL_DICT

# SASRec - Self-Attentive Sequential Recommendation
sasrec_model = MODEL_DICT['sasrec'](args)
output = sasrec_model(input_ids, user_ids)

# GRU4Rec - GRU-based sequential recommendation
args.gru_hidden_size = 64
gru_model = MODEL_DICT['gru4rec'](args)
output = gru_model(input_ids, user_ids)

# BERT4Rec - Bidirectional sequential recommendation with masking
args.mask_ratio = 0.2
bert_model = MODEL_DICT['bert4rec'](args)
output = bert_model(input_ids, user_ids)

# DuoRec - Contrastive learning-based model
args.tau = 1.0  # temperature for contrastive loss
args.lmd = 0.1  # contrastive loss weight
args.lmd_sem = 0.1  # semantic contrastive loss weight
duo_model = MODEL_DICT['duorec'](args)
loss = duorec_model.calculate_loss(input_ids, answers, neg_answers, same_target_aug, user_ids)
```

--------------------------------

### Train BSARec Model with Custom Hyperparameters

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

This command-line script initiates the training process for the BSARec model on the 'Beauty' dataset. It configures hyperparameters such as learning rate, alpha (balancing frequency and attention), frequency cutoff (c), number of attention heads, batch size, epochs, hidden layer size, and maximum sequence length. The training pipeline includes data loading, model initialization, optimization, checkpointing, logging, and evaluation.

```bash
python main.py \
    --data_name Beauty \
    --lr 0.0005 \
    --alpha 0.7 \
    --c 5 \
    --num_attention_heads 1 \
    --train_name BSARec_Beauty \
    --batch_size 256 \
    --epochs 200 \
    --hidden_size 64 \
    --num_hidden_layers 2 \
    --max_seq_length 50
```

--------------------------------

### Create Custom Sequential Recommendation Dataset in Python

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

This Python code defines how to create a custom dataset for sequential recommendation using the 'RecDataset' class. It assumes input data is stored in text files where each line represents a user's interaction sequence (user_id item1 item2 ...). The 'RecDataset' supports training with a sliding window approach and automatically handles negative sampling and contrastive learning if configured.

```python
from dataset import RecDataset
import torch

# Data format: each line is "user_id item1 item2 item3 ..."
# Example data/Beauty.txt content:
# 0 45 67 89 12 34
# 1 23 45 67 89
# 2 12 34 56 78 90

# Training dataset with sliding window
train_dataset = RecDataset(args, user_seq, data_type='train')
```

--------------------------------

### Create and Activate Conda Environment for BSARec

Source: https://github.com/yehjin-shin/bsarec/blob/main/README.md

Sets up the necessary Python environment for the BSARec project using Conda. It requires a 'bsarec_env.yaml' file to define the project's dependencies.

```bash
conda env create -f bsarec_env.yaml
conda activate bsarec
```

--------------------------------

### Loading Sequential Data

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Load user interaction sequences from text files and prepare train/validation/test dataloaders with proper padding and negative sampling. This API is fundamental for preparing data for model training.

```APIDOC
## Loading Sequential Data

### Description
Loads user interaction sequences from specified data files and prepares dataloaders for training, validation, and testing, including data padding and negative sampling.

### Method
`from dataset import get_seq_dic, get_dataloder, get_rating_matrix`
`from utils import parse_args`

### Parameters
#### `parse_args()`
Parses command-line arguments. Configuration can be set programmatically after parsing.

#### `get_seq_dic(args)`
- `args` (object) - Configuration object containing parameters like `data_name`, `data_dir`.
- Returns a tuple: `(seq_dic, max_item, num_users)`
  - `seq_dic` (dict): Dictionary mapping user IDs to their interaction sequences.
  - `max_item` (int): The maximum item ID found in the dataset.
  - `num_users` (int): The total number of unique users.

#### `get_dataloder(args, seq_dic)`
- `args` (object) - Configuration object with parameters like `batch_size`, `max_seq_length`, `num_workers`.
- `seq_dic` (dict) - Sequence dictionary returned by `get_seq_dic`.
- Returns a tuple: `(train_dataloader, eval_dataloader, test_dataloader)`

#### `get_rating_matrix(data_name, seq_dic, max_item)`
- `data_name` (string) - Name of the dataset.
- `seq_dic` (dict) - Sequence dictionary.
- `max_item` (int) - Maximum item ID.
- Returns a tuple: `(valid_rating_matrix, test_rating_matrix)`
  - Rating matrices are `scipy.sparse.csr_matrix` objects.

### Request Example
```python
from dataset import get_seq_dic, get_dataloder, get_rating_matrix
from utils import parse_args

# Parse configuration
args = parse_args()
args.data_name = 'Beauty'
args.data_dir = './data/'
args.batch_size = 256
args.max_seq_length = 50
args.num_workers = 4

# Load sequence dictionary
seq_dic, max_item, num_users = get_seq_dic(args)
args.item_size = max_item + 1
args.num_users = num_users + 1

# Create dataloaders
train_dataloader, eval_dataloader, test_dataloader = get_dataloder(args, seq_dic)

# Generate rating matrices for evaluation
valid_rating_matrix, test_rating_matrix = get_rating_matrix(args.data_name, seq_dic, max_item)
```

### Response
#### Output
- `train_dataloader`, `eval_dataloader`, `test_dataloader`: PyTorch DataLoaders providing batches of data.
- Each batch contains:
  - `user_ids` (tensor): User indices.
  - `input_ids` (tensor): Padded input sequences.
  - `answers` (tensor): Target items.
  - `neg_answer` (tensor): Negative samples.
  - `same_target` (tensor): For contrastive learning (if applicable).
- `valid_rating_matrix`, `test_rating_matrix` (scipy.sparse.csr_matrix): Sparse matrices representing user-item interactions.
```

--------------------------------

### Instantiate and Use BSARec Model with Frequency Filtering

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Initializes the BSARec model with specified hyperparameters, including item size, hidden dimensions, attention heads, and parameters for frequency filtering (alpha, c). It demonstrates a forward pass, loss calculation, and prediction steps, utilizing PyTorch tensors for input and output.

```python
from model import MODEL_DICT
import torch

# Initialize BSARec model
args.model_type = 'bsarec'
args.item_size = 1000  # number of items + 1
args.hidden_size = 64
args.num_hidden_layers = 2
args.num_attention_heads = 1
args.alpha = 0.7  # balance between frequency (DSP) and attention (GSP)
args.c = 5  # frequency cutoff parameter
args.max_seq_length = 50
args.hidden_dropout_prob = 0.5
args.attention_probs_dropout_prob = 0.5
args.initializer_range = 0.02

model = MODEL_DICT['bsarec'](args=args)

# Forward pass
input_ids = torch.randint(0, args.item_size, (32, 50))  # [batch_size, seq_length]
sequence_output = model(input_ids)  # [batch_size, seq_length, hidden_size]

# Calculate training loss
answers = torch.randint(1, args.item_size, (32,))
neg_answers = torch.randint(1, args.item_size, (32,))
same_target = torch.zeros(32, 50)
user_ids = torch.arange(32)
loss = model.calculate_loss(input_ids, answers, neg_answers, same_target, user_ids)

# Prediction
predictions = model.predict(input_ids, user_ids)  # [batch_size, seq_length, hidden_size]
# Take last hidden state for recommendation
last_hidden = predictions[:, -1, :]  # [batch_size, hidden_size]
# Compute scores for all items
item_embeddings = model.item_embeddings.weight  # [item_size, hidden_size]
scores = torch.matmul(last_hidden, item_embeddings.t())  # [batch_size, item_size]
top20_items = torch.topk(scores, k=20, dim=1).indices  # [batch_size, 20]
```

--------------------------------

### Import Libraries for Spectral Response Visualization

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure2.ipynb

Imports necessary Python libraries including os, scipy, torch, numpy, matplotlib, and dft from scipy.linalg. These are fundamental for numerical operations, signal processing, and plotting required for spectral analysis.

```python
import os
import scipy
import torch
import numpy as np
from scipy.linalg import dft
import matplotlib.pyplot as plt
```

--------------------------------

### Train Baseline Models

Source: https://github.com/yehjin-shin/bsarec/blob/main/README.md

Command-line interface for training baseline recommendation models. Allows selection of different model types (e.g., Caser, GRU4Rec, SASRec) via the '--model_type' argument. Hyperparameters for baselines can be found in 'src/utils.py'.

```python
python main.py  --model_type SASRec \
                --data_name Beauty \
                --num_attention_heads 1 \
                --train_name SASRec_Beauty
```

--------------------------------

### Test Pretrained BSARec Model

Source: https://github.com/yehjin-shin/bsarec/blob/main/README.md

Command-line interface for evaluating a pretrained BSARec model. Requires the dataset name, model parameters (alpha, C, num_attention_heads), and the name of the pretrained model file (without .pt extension) located in 'src/output'.

```python
python main.py  --data_name [DATASET] \
                --alpha [ALPHA] \
                --c [C] \
                --num_attention_heads [N_HEADS] \
                --load_model [PRETRAINED_MODEL_NAME] \
                --do_eval
```

--------------------------------

### Custom Dataset Format

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Define custom sequential datasets with automatic negative sampling and contrastive learning support using the `RecDataset` class. This allows for flexible data handling.

```APIDOC
## Custom Dataset Format

### Description
Provides a `RecDataset` class for creating custom datasets from user interaction sequences, supporting automatic negative sampling and contrastive learning.

### Method
`from dataset import RecDataset`

### Parameters
#### `RecDataset(args, user_seq, data_type='train')`
- `args` (object) - Configuration object containing dataset parameters.
- `user_seq` (list or dict) - User interaction sequences. If a dictionary, keys are user IDs and values are lists of item IDs.
- `data_type` (string) - Type of dataset ('train', 'eval', 'test'). Affects how sequences are split or processed (e.g., for sliding window in training).

### Data Format
Input data files should have one user's interaction sequence per line, with item IDs separated by spaces.
Example `data/Beauty.txt`:
```
0 45 67 89 12 34
1 23 45 67 89
2 12 34 56 78 90
```

### Request Example
```python
from dataset import RecDataset
import torch

# Assuming 'args' is already configured and 'user_seq' is loaded
# Example: user_seq = [[45, 67, 89, 12, 34], [23, 45, 67, 89], ...]

# Training dataset with sliding window
train_dataset = RecDataset(args, user_seq, data_type='train')

# Example batch retrieval (if using with DataLoader)
# batch = train_dataset[0]
# print(batch)
```

### Response
#### Output
- `train_dataset`: An instance of `RecDataset` configured for training.
- This dataset can be used with PyTorch's `DataLoader` to create batches for model training.
```

--------------------------------

### Load Sequence Output for Models

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure3.ipynb

Loads output sequences for different recommendation models (SASRec, BSARec, FMLPRec, DuoRec) using the `get_seqout` function.  It specifies the base path for the output files and the number of layers. The resulting data is stored in a dictionary, with model names as keys and a list of output sequences as values.

```python
base_path = "./sequence_output/"
seqout = {
    "SASRec": get_seqout(base_path+"LastFM_SASRec", 16),
    "BSARec": get_seqout(base_path+"LastFM_BSARec", 16),
    "FMLPRec": get_seqout(base_path+"LastFM_FMLPRec", 16),
    "DuoRec": get_seqout(base_path+"LastFM_DuoRec", 16),
}
```

--------------------------------

### Training BSARec Model

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Main entry point for training the BSARec recommendation model with configurable hyperparameters. This API facilitates the entire training pipeline from data loading to evaluation.

```APIDOC
## Training BSARec Model

### Description
Main entry point for training the BSARec recommendation model with configurable hyperparameters including learning rate, alpha (balance between frequency and attention), and c (frequency cutoff).

### Method
`python main.py` (command line execution)

### Parameters
#### Command Line Arguments
- `--data_name` (string) - Required - Name of the dataset (e.g., `Beauty`).
- `--lr` (float) - Optional - Learning rate (default: `0.0005`).
- `--alpha` (float) - Optional - Balance between frequency and attention (default: `0.7`).
- `--c` (int) - Optional - Frequency cutoff (default: `5`).
- `--num_attention_heads` (int) - Optional - Number of attention heads (default: `1`).
- `--train_name` (string) - Required - Name for the training run and model checkpoint.
- `--batch_size` (int) - Optional - Batch size for training (default: `256`).
- `--epochs` (int) - Optional - Number of training epochs (default: `200`).
- `--hidden_size` (int) - Optional - Size of hidden layers (default: `64`).
- `--num_hidden_layers` (int) - Optional - Number of hidden layers (default: `2`).
- `--max_seq_length` (int) - Optional - Maximum sequence length (default: `50`).

### Request Example
```bash
python main.py \
    --data_name Beauty \
    --lr 0.0005 \
    --alpha 0.7 \
    --c 5 \
    --num_attention_heads 1 \
    --train_name BSARec_Beauty \
    --batch_size 256 \
    --epochs 200 \
    --hidden_size 64 \
    --num_hidden_layers 2 \
    --max_seq_length 50
```

### Response
#### Output
The training process logs progress, saves the best model checkpoint to `./output/[train_name].pt`, and evaluates on the test set, printing metrics to the console. Logs are saved to `./output/[train_name].log`.
```

--------------------------------

### Train BSARec Model

Source: https://github.com/yehjin-shin/bsarec/blob/main/README.md

Command-line interface for training the BSARec model. It requires specifying dataset name, learning rate, alpha, C value, number of attention heads, and a name for the log file and checkpoint. Pretrained models and logs are saved in 'src/output'.

```python
python main.py  --data_name [DATASET] \
                --lr [LEARNING_RATE] \
                --alpha [ALPHA] \
                --c [C] \
                --num_attention_heads [N_HEADS] \
                --train_name [LOG_NAME]
```

--------------------------------

### Training Baseline Models

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Switch between different sequential recommendation architectures by changing the `model_type` parameter. This allows for comparison with established baseline models.

```APIDOC
## Training Baseline Models

### Description
Train various baseline sequential recommendation models by specifying the `model_type` parameter.

### Method
`python main.py` (command line execution)

### Parameters
#### Command Line Arguments
- `--model_type` (string) - Required - The type of baseline model to train (e.g., `SASRec`, `GRU4Rec`, `BERT4Rec`, `Caser`, `FMLPRec`, `DuoRec`, `FEARec`).
- `--data_name` (string) - Required - Name of the dataset (e.g., `Beauty`).
- `--train_name` (string) - Required - Name for the training run.
- `--num_attention_heads` (int) - Optional - Number of attention heads (used by SASRec, BERT4Rec, etc.).
- `--lr` (float) - Optional - Learning rate (e.g., `0.001` for SASRec).
- `--gru_hidden_size` (int) - Optional - Hidden size for GRU4Rec.
- `--mask_ratio` (float) - Optional - Masking ratio for BERT4Rec (e.g., `0.2`).

### Request Examples
#### Train SASRec
```bash
python main.py \
    --model_type SASRec \
    --data_name Beauty \
    --num_attention_heads 1 \
    --lr 0.001 \
    --train_name SASRec_Beauty
```

#### Train GRU4Rec
```bash
python main.py \
    --model_type GRU4Rec \
    --data_name Beauty \
    --gru_hidden_size 64 \
    --train_name GRU4Rec_Beauty
```

#### Train BERT4Rec
```bash
python main.py \
    --model_type BERT4Rec \
    --data_name Beauty \
    --mask_ratio 0.2 \
    --train_name BERT4Rec_Beauty
```

### Available Model Types
`Caser`, `GRU4Rec`, `SASRec`, `BERT4Rec`, `FMLPRec`, `DuoRec`, `FEARec`, `BSARec`
```

--------------------------------

### Evaluating Pretrained Model

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Load and evaluate a pretrained BSARec model on the test dataset without retraining. This is useful for quickly assessing model performance on unseen data.

```APIDOC
## Evaluating Pretrained Model

### Description
Load and evaluate a pretrained BSARec model on the test dataset without retraining.

### Method
`python main.py` (command line execution)

### Parameters
#### Command Line Arguments
- `--data_name` (string) - Required - Name of the dataset (e.g., `Beauty`).
- `--alpha` (float) - Optional - Alpha parameter used during training (must match saved model, default: `0.7`).
- `--c` (int) - Optional - C parameter used during training (must match saved model, default: `5`).
- `--num_attention_heads` (int) - Optional - Number of attention heads used during training (must match saved model, default: `1`).
- `--load_model` (string) - Required - Name of the pretrained model to load (e.g., `BSARec_Beauty_best`).
- `--do_eval` - Flag to enable evaluation.

### Request Example
```bash
python main.py \
    --data_name Beauty \
    --alpha 0.7 \
    --c 5 \
    --num_attention_heads 1 \
    --load_model BSARec_Beauty_best \
    --do_eval
```

### Response
#### Output
Prints evaluation metrics such as HR@k and NDCG@k for specified k values.

#### Response Example
```json
{
  "Epoch": 0,
  "HR@5": "0.0234",
  "NDCG@5": "0.0156",
  "HR@10": "0.0367",
  "NDCG@10": "0.0198",
  "HR@20": "0.0556",
  "NDCG@20": "0.0243"
}
```
```

--------------------------------

### Compute Recommendation Evaluation Metrics

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Imports and utilizes functions for calculating common recommendation system evaluation metrics, specifically recall@k and NDCG@k. These metrics are essential for assessing the performance of recommendation models.

```python
from metrics import recall_at_k, ndcg_k
import numpy as np
```

--------------------------------

### Run Data Processing Script

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/data/process/README.md

Execute the shell script to initiate the data processing workflow. This script prompts the user to select a dataset or process all available datasets.

```shell
sh process.sh
```

--------------------------------

### Evaluate Pretrained BSARec Model

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

This command loads a previously trained BSARec model (specified by 'load_model') and evaluates its performance on the test dataset without further training. It uses the same dataset and core BSARec hyperparameters (alpha, c, num_attention_heads) as during training. The expected output provides evaluation metrics like Hit Rate (HR) and NDCG at different k values (5, 10, 20).

```bash
python main.py \
    --data_name Beauty \
    --alpha 0.7 \
    --c 5 \
    --num_attention_heads 1 \
    --load_model BSARec_Beauty_best \
    --do_eval
```

--------------------------------

### Import Modules for Visualization

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure3.ipynb

Imports necessary Python modules for data loading, numerical computation, plotting, and similarity calculation. Includes libraries such as os, torch, numpy, matplotlib, and scipy.

```python
import os
import torch
import numpy as np
import matplotlib.pyplot as plt
from torchmetrics.functional.pairwise import pairwise_cosine_similarity
from scipy.linalg import svdvals

color_dict = {
    "BSARec": "blue",
    "SASRec": "darkorange",
    "DuoRec": "green",
    "FMLPRec": "crimson"
}
```

--------------------------------

### Calculate Recommendation Metrics (Hit Rate, NDCG)

Source: https://context7.com/yehjin-shin/bsarec/llms.txt

Calculates Hit Rate (HR) and Normalized Discounted Cumulative Gain (NDCG) for recommendation systems. HR@k measures if the true next item is within the top-k predictions, while NDCG@k evaluates the ranking quality considering the position of the true item. It takes the ground truth 'actual' items and a matrix of 'predicted' top-k items for multiple users as input.

```python
import numpy as np

# Assuming recall_at_k and ndcg_k are defined elsewhere
# from some_metrics_module import recall_at_k, ndcg_k

# Placeholder functions for demonstration if not provided in context
def recall_at_k(actual, predicted, topk):
    hits = 0
    for i in range(len(actual)):
        if actual[i] in predicted[i, :topk]:
            hits += 1
    return hits / len(actual)

def ndcg_k(actual, predicted, topk):
    # Simplified placeholder for NDCG calculation
    return recall_at_k(actual, predicted, topk) # In reality, this would involve ranking scores

actual = np.array([5, 12, 23, 8, 15])  # true next items for 5 users
predicted = np.array([
    [3, 5, 7, 9, 12, 15, 18, 21, 24, 27],  # top-10 for user 0
    [1, 4, 7, 10, 13, 16, 19, 22, 25, 28],  # top-10 for user 1
    [2, 5, 8, 11, 14, 17, 20, 23, 26, 29],  # top-10 for user 2
    [6, 9, 12, 15, 18, 21, 24, 27, 30, 33],  # top-10 for user 3
    [4, 8, 12, 16, 20, 24, 28, 32, 36, 40],  # top-10 for user 4
])

# Calculate Hit Rate @ k
hr_at_5 = recall_at_k(actual, predicted, topk=5)
hr_at_10 = recall_at_k(actual, predicted, topk=10)
print(f"HR@5: {hr_at_5:.4f}, HR@10: {hr_at_10:.4f}")

# Calculate NDCG @ k
ndcg_at_5 = ndcg_k(actual, predicted, topk=5)
ndcg_at_10 = ndcg_k(actual, predicted, topk=10)
print(f"NDCG@5: {ndcg_at_5:.4f}, NDCG@10: {ndcg_at_10:.4f}")

# Hit Rate measures if target item is in top-k predictions
# NDCG measures ranking quality with position-aware scoring
```

--------------------------------

### Calculate FMLPRec Filter Layer Spectral Response (Python)

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure2.ipynb

Calculates the spectral response of the filter layer in FMLPRec. It loads complex weight values, separates real and imaginary parts, computes the magnitude, averages across a dimension, and prepares the data for plotting. Requires a pre-saved complex weight numpy file.

```python
file_path = 'sequence_output/LastFM_Spectral_Response/FMLPRec_complex_weight.npy'
complex_weight_layer = np.load(file_path)[0]

filter_real = complex_weight_layer[:,:,0]
filter_imaginary = complex_weight_layer[:,:,1]
filter_abs = filter_real**2 + filter_imaginary**2

fmlprec_filter = filter_abs.mean(axis=1)
```

--------------------------------

### Plot FMLPRec Filter Layer Spectral Response (Python)

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure2.ipynb

Plots the spectral response of FMLPRec's filter layer. It utilizes the fftshift function for frequency centering and then displays the normalized magnitude of the filter responses against frequency using matplotlib. Requires the output from the FMLPRec spectral response calculation.

```python
x, fmlprec_shifted_filter = fftshift(fmlprec_filter)
plt.title('Spectral Responses of Filter Layer in FMLPRec')
plt.xlabel('Frequency')
plt.ylabel('Normalized Magnitude')
plt.plot(x, fmlprec_shifted_filter / fmlprec_shifted_filter.max())
plt.show()
```

--------------------------------

### FFTShift Function for Rearranging FFT Output

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure2.ipynb

The fftshift function rearranges the output of a Fast Fourier Transform (FFT) to place the zero-frequency component at the center. It takes an array and an optional size 'n' as input and returns the frequency bins and the rearranged spectral data, useful for visualization.

```python
def fftshift(arr, n=50):
    
    freq = scipy.fft.rfftfreq(50)
    x = np.concatenate([np.flip(-freq[1:]), freq])
    y = np.concatenate([np.flip(arr[1:]), arr])

    return x, y
```

--------------------------------

### Calculate SASRec Self-Attention Spectral Response (Python)

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure2.ipynb

Calculates the spectral response of the self-attention matrix in SASRec. It loads an attention matrix, computes the Discrete Fourier Transform (DFT) and its inverse, applies them to the attention matrix, takes the mean across heads, and extracts the absolute values. Requires a pre-saved attention matrix numpy file.

```python
file_path = 'sequence_output/LastFM_Spectral_Response/SASRec_attention_weight.npy'
attention_matrix = np.load(file_path)

dft_matrix = dft(50)
idft_matrix = np.linalg.inv(dft_matrix)

index = 75
head = 0
attention = attention_matrix[index, head]
sasrec_filter = dft_matrix @ attention @ idft_matrix

sasrec_filter = np.abs(sasrec_filter.mean(axis=1)[:26])
```

--------------------------------

### Compute Cosine Similarity for Output Sequences

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure3.ipynb

Calculates the average cosine similarity between the output sequences of each model.  The `compute_cosine_similarity` function takes a list of sequences, converts them to tensors, and computes the pairwise cosine similarity using `torchmetrics`.  The result for each model is then stored in a dictionary.

```python
def compute_cosine_similarity(seqout, return_numpy=True):
    
    seqout = torch.Tensor(seqout) # shape: (n_seq, hidden_dim)
    cos_sim = pairwise_cosine_similarity(seqout) # shape: (n_seq, n_seq)
    cos_sim = cos_sim.mean()
    
    return cos_sim.item()

cos_sim_dict = {}
for model in seqout.keys():
    cos_sim_dict[model] = list(map(lambda x: compute_cosine_similarity(x), seqout[model]))
```

--------------------------------

### Compute and Plot Singular Values

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure3.ipynb

Computes and visualizes the normalized singular values of the output sequences. For each model, the function takes the last layer's output, calculates its singular values using `svdvals`, and normalizes them by dividing by the largest singular value.  The normalized singular values are then plotted for each model, using the `color_dict` for styling.

```python
svs_dict = {}
for model in seqout.keys():
    x = seqout[model][-1]
    svs_dict[model] = svdvals(x)/(svdvals(x).max())
```

```python
for model in ['SASRec', 'DuoRec', 'FMLPRec', 'BSARec']:
    plt.plot(svs_dict[model], label=model, color=color_dict[model])

plt.xlabel('Singular Value Index')
plt.ylabel('Normalized Singular Value')
plt.legend(loc='upper right')
plt.grid(visible=True)

plt.show()
```

--------------------------------

### Plot Cosine Similarity vs. Layers

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure3.ipynb

Generates a plot showing the cosine similarity of the output sequences for different models across layers. The x-axis represents the number of layers, and the y-axis represents the average cosine similarity. Each model's performance is represented by a different color and marker, using the `color_dict` for styling.

```python
for model in ['SASRec', 'DuoRec', 'FMLPRec', 'BSARec']:
    plt.plot(range(0,17), cos_sim_dict[model], label=model, marker='o', color=color_dict[model])

plt.xlabel('Number of Layers')
plt.ylabel('Cosine Similarity')
plt.legend(loc='upper left')
plt.grid(visible=True)

plt.tight_layout()
plt.show()
```

--------------------------------

### Plot SASRec Self-Attention Spectral Response (Python)

Source: https://github.com/yehjin-shin/bsarec/blob/main/src/visualize/figure2.ipynb

Plots the calculated spectral response of SASRec's self-attention matrix. It uses the fftshift function to center the frequency components and then plots the normalized magnitude against the frequency using matplotlib. Requires the output from the SASRec spectral response calculation.

```python
x, sasrec_shifted_filter = fftshift(sasrec_filter)
plt.title('Spectral Responses of Self-Attention in SASRec')
plt.xlabel('Frequency')
plt.ylabel('Normalized Magnitude')
plt.plot(x, sasrec_shifted_filter / sasrec_shifted_filter.max())
plt.show()
```
Tokens:
Raw