### Install Project Dependencies

Source: https://github.com/thuml/openltm/blob/main/README.md

Use this command to install all necessary Python packages listed in the requirements file.

```bash
pip install -r requirements.txt
```

--------------------------------

### Timer Model Initialization and Forward Pass

Source: https://context7.com/thuml/openltm/llms.txt

Shows how to set up and run the Timer model, a decoder-only Transformer for autoregressive time series forecasting. Ensure PyTorch is installed.

```python
import torch
from models.timer import Model

class Args:
    input_token_len = 96
    output_token_len = 96
    d_model = 512
    n_heads = 8
    e_layers = 6
    d_ff = 2048
    dropout = 0.1
    activation = 'relu'
    use_norm = True

configs = Args()
model = Model(configs)

# Univariate time series input
x = torch.randn(32, 672, 1)  # [batch, seq_len, features]
x_mark = torch.zeros(32, 672, 1)
y_mark = torch.zeros(32, 672, 1)

output = model(x, x_mark, y_mark)
print(f"Prediction shape: {output.shape}")  # [32, 672, 1]
```

--------------------------------

### Moirai Model Initialization and Forward Pass

Source: https://context7.com/thuml/openltm/llms.txt

Illustrates the setup and usage of the Moirai model for universal time series forecasting with bidirectional attention. Suitable for multivariate data.

```python
import torch
from models.moirai import Model

class Args:
    input_token_len = 96
    output_token_len = 96
    test_pred_len = 96
    d_model = 512
    n_heads = 8
    e_layers = 5
    d_ff = 2048
    dropout = 0.1
    activation = 'relu'
    use_norm = True

configs = Args()
model = Model(configs)

# Multivariate forecasting
x = torch.randn(16, 672, 321)  # Electricity dataset has 321 variables
x_mark = torch.zeros(16, 672, 1)
y_mark = torch.zeros(16, 672, 1)

output = model(x, x_mark, y_mark)
print(f"Forecast shape: {output.shape}")  # [16, 96, 321]
```

--------------------------------

### Initialize and Load Timer-XL

Source: https://github.com/thuml/openltm/blob/main/load_pth_ckpt.ipynb

Configure model arguments and load weights from a local checkpoint file.

```python
# init the args of pre-trained Timer-XL

args = argparse.Namespace()
args.input_token_len = 96
args.output_token_len = 96
args.d_model = 1024
args.n_heads = 8
args.e_layers = 8
args.d_ff = 2048
args.dropout = 0.1
args.activation = 'relu'
args.use_norm = True
args.flash_attention = False
args.covariate = False
args.output_attention = False

model = timer_xl.Model(args)

# download the checkpoint from https://cloud.tsinghua.edu.cn/f/01c35ca13f474176be7b/
model.load_state_dict(torch.load('checkpoint.pth'))
```

--------------------------------

### Timer-XL Model Initialization and Forward Pass

Source: https://context7.com/thuml/openltm/llms.txt

Demonstrates how to initialize and use the Timer-XL model for time series forecasting. Requires PyTorch and specific model configurations.

```python
import torch
from models.timer_xl import Model

class Args:
    input_token_len = 96
    output_token_len = 96
    d_model = 512
    n_heads = 8
    e_layers = 4
    d_ff = 2048
    dropout = 0.1
    activation = 'relu'
    output_attention = False
    covariate = False
    flash_attention = False
    use_norm = True

configs = Args()
model = Model(configs)

# Input: [batch_size, seq_len, num_features]
batch_size, seq_len, num_vars = 32, 672, 7
x = torch.randn(batch_size, seq_len, num_vars)
x_mark = torch.zeros(batch_size, seq_len, 1)
y_mark = torch.zeros(batch_size, seq_len, 1)

# Forward pass returns predictions
output = model(x, x_mark, y_mark)
print(f"Input shape: {x.shape}")
print(f"Output shape: {output.shape}")  # [32, 672, 7]
```

--------------------------------

### Initialize Univariate and Multivariate Datasets

Source: https://context7.com/thuml/openltm/llms.txt

Configure dataset benchmarks for univariate or multivariate time series data. Univariate treats variables as separate samples, while multivariate processes them together.

```python
# Univariate: each variable is treated as a separate sample
univar_dataset = UnivariateDatasetBenchmark(
    root_path='./dataset/ETT-small/',
    data_path='ETTh1.csv',
    flag='train',  # 'train', 'val', or 'test'
    size=[672, 96, 96],  # [seq_len, input_token_len, output_token_len]
    scale=True,
    nonautoregressive=False,
    subset_rand_ratio=1.0
)

# Multivariate: all variables are processed together
multivar_dataset = MultivariateDatasetBenchmark(
    root_path='./dataset/electricity/',
    data_path='electricity.csv',
    flag='train',
    size=[672, 96, 96],
    scale=True,
    nonautoregressive=True  # For non-autoregressive models like Moirai
)

# Access samples
seq_x, seq_y, seq_x_mark, seq_y_mark = univar_dataset[0]
print(f"Univariate sample - X: {seq_x.shape}, Y: {seq_y.shape}")
```

--------------------------------

### Initialize and Run MOMENT Model

Source: https://context7.com/thuml/openltm/llms.txt

Configures and executes the MOMENT foundation model using a Transformer encoder.

```python
import torch
from models.moment import Model

class Args:
    seq_len = 512
    test_pred_len = 96
    input_token_len = 16  # patch_len
    d_model = 256
    n_heads = 8
    e_layers = 4
    d_ff = 512
    dropout = 0.1
    activation = 'relu'
    use_norm = True

configs = Args()
model = Model(configs)

x = torch.randn(16, 512, 7)
x_mark = torch.zeros(16, 512, 1)
y_mark = torch.zeros(16, 512, 1)

output = model(x, x_mark, y_mark)
print(f"MOMENT output: {output.shape}")  # [16, 96, 7]
```

--------------------------------

### Run Training Command

Source: https://context7.com/thuml/openltm/llms.txt

Use this command to initiate supervised training for time series forecasting tasks. Ensure all paths and model parameters are correctly specified.

```bash
python -u run.py \
  --task_name forecast \
  --is_training 1 \
  --root_path ./dataset/ETT-small/ \
  --data_path ETTh1.csv \
  --model_id ETTh1 \
  --model timer_xl \
  --data UnivariateDatasetBenchmark \
  --seq_len 672 \
  --input_token_len 96 \
  --output_token_len 96 \
  --test_seq_len 672 \
  --test_pred_len 96 \
  --e_layers 8 \
  --d_model 1024 \
  --d_ff 2048 \
  --batch_size 32 \
  --learning_rate 0.0001 \
  --train_epochs 10 \
  --gpu 0 \
  --cosine \
  --tmax 10 \
  --use_norm
```

--------------------------------

### Initialize and Run Time-LLM Model

Source: https://context7.com/thuml/openltm/llms.txt

Configures and executes the Time-LLM model, which aligns time series patches with text prototypes.

```python
import torch
from models.time_llm import Model

class Args:
    seq_len = 512
    test_pred_len = 96
    input_token_len = 16
    stride = 8
    d_model = 32
    d_ff = 128
    n_heads = 8
    ts_vocab_size = 1000
    llm_model = "GPT2"  # Options: LLAMA, GPT2, BERT
    llm_layers = 6
    dropout = 0.1
    use_norm = True
    domain_des = "The Electricity Transformer Temperature (ETT) is a crucial indicator in the electric power long-term deployment."

configs = Args()
model = Model(configs)

x = torch.randn(4, 512, 7)
x_mark = torch.zeros(4, 512, 1)
y_mark = torch.zeros(4, 512, 1)

output = model(x, x_mark, y_mark)
print(f"Time-LLM output: {output.shape}")  # [4, 96, 7]
```

--------------------------------

### Initialize and Run TTM Model

Source: https://context7.com/thuml/openltm/llms.txt

Configures and executes the Tiny Time Mixers model for multivariate forecasting.

```python
import torch
from models.ttm import Model

class Args:
    seq_len = 512
    test_pred_len = 96
    patch_size = 16
    stride = 8
    n_vars = 7
    d_model = 64
    hidden_dim = 16
    e_layers = 4
    layers = 8
    factor = 2
    mode = "mix_channel"
    d_mode = "common_channel"
    AP_levels = 0
    use_decoder = True
    dropout = 0.1
    use_norm = True

configs = Args()
model = Model(configs)

x = torch.randn(16, 512, 7)
x_mark = torch.zeros(16, 512, 1)
y_mark = torch.zeros(16, 512, 1)

output = model(x, x_mark, y_mark)
print(f"TTM output: {output.shape}")  # [16, 96, 7]
```

--------------------------------

### Import Dependencies

Source: https://github.com/thuml/openltm/blob/main/load_pth_ckpt.ipynb

Required libraries for model initialization and data handling.

```python
import torch
import argparse
import pandas as pd
import matplotlib.pyplot as plt

from models import timer_xl
```

--------------------------------

### Run Inference with Pre-trained Model

Source: https://context7.com/thuml/openltm/llms.txt

Execute this command to perform testing or inference using a pre-trained model. Specify the path to the checkpoint file for loading.

```bash
python -u run.py \
  --task_name forecast \
  --is_training 0 \
  --root_path ./dataset/ETT-small/ \
  --data_path ETTh1.csv \
  --model timer_xl \
  --data UnivariateDatasetBenchmark \
  --seq_len 672 \
  --input_token_len 96 \
  --output_token_len 96 \
  --test_seq_len 672 \
  --test_pred_len 96 \
  --test-dir ./checkpoints/timer_xl \
  --test_file_name checkpoint.pth \
  --gpu 0
```

--------------------------------

### Initialize and Run AutoTimes Model

Source: https://context7.com/thuml/openltm/llms.txt

Configures and executes the AutoTimes model for autoregressive forecasting using frozen LLMs.

```python
import torch
from models.autotimes import Model

class Args:
    input_token_len = 96
    llm_model = "GPT2"  # Options: GPT2, LLAMA, OPT
    d_model = 256
    e_layers = 2
    dropout = 0.1
    activation = 'relu'
    use_norm = True
    ddp = False
    local_rank = 0

configs = Args()
model = Model(configs)

x = torch.randn(8, 672, 1)
x_mark = torch.zeros(8, 672, 1)
y_mark = torch.zeros(8, 672, 1)

output = model(x, x_mark, y_mark)
print(f"AutoTimes output: {output.shape}")  # [8, 672, 1]
```

--------------------------------

### Visualize Input and Output

Source: https://github.com/thuml/openltm/blob/main/load_pth_ckpt.ipynb

Plot the input lookback sequence and the model's raw output.

```python
plt.plot(input.squeeze().detach().numpy())
```

```python
plt.plot(output.squeeze().detach().numpy())
```

--------------------------------

### GPT4TS Model Initialization

Source: https://context7.com/thuml/openltm/llms.txt

Shows the basic initialization for the GPT4TS model, which adapts pre-trained GPT-2 for time series tasks by fine-tuning specific layers.

```python
import torch
from models.gpt4ts import Model

class Args:
    seq_len = 512
    test_pred_len = 96
    patch_size = 16
    stride = 8
    gpt_layers = 6
    d_model = 768
    use_norm = True

configs = Args()
model = Model(configs)

```

--------------------------------

### Orchestrate Training with Exp_Forecast

Source: https://context7.com/thuml/openltm/llms.txt

Use the Exp_Forecast class to manage the training loop, testing, and model checkpointing by defining an Args configuration object.

```python
from exp.exp_forecast import Exp_Forecast

class Args:
    # Model configuration
    model = 'timer_xl'
    task_name = 'forecast'

    # Data configuration
    data = 'UnivariateDatasetBenchmark'
    root_path = './dataset/ETT-small/'
    data_path = 'ETTh1.csv'
    checkpoints = './checkpoints/'

    # Sequence lengths
    seq_len = 672
    input_token_len = 96
    output_token_len = 96
    test_seq_len = 672
    test_pred_len = 96

    # Model architecture
    d_model = 512
    n_heads = 8
    e_layers = 4
    d_ff = 2048
    dropout = 0.1
    activation = 'relu'
    use_norm = True
    output_attention = False
    covariate = False
    flash_attention = False
    nonautoregressive = False

    # Training parameters
    learning_rate = 0.0001
    batch_size = 32
    train_epochs = 10
    patience = 3
    weight_decay = 0
    cosine = True
    tmax = 10
    lradj = 'type1'
    valid_last = False
    last_token = False

    # Hardware
    gpu = 0
    ddp = False
    dp = False
    num_workers = 4

    # Adaptation
    adaptation = False
    pretrain_model_path = ''
    subset_rand_ratio = 1.0
    test_flag = 'T'

args = Args()
exp = Exp_Forecast(args)

# Training
setting = 'forecast_etth1_timer_xl'
model = exp.train(setting)

# Testing
exp.test(setting, test=0)
```

--------------------------------

### Run Large-Scale Pre-training Scripts

Source: https://github.com/thuml/openltm/blob/main/README.md

Utilize these bash scripts for large-scale pre-training on different datasets like UTSD and ERA5.

```bash
# Large-scale pre-training
# (a) pre-training on UTSD
bash ./scripts/pretrain/timer_xl_utsd.sh
# (b) pre-training on ERA5
bash ./scripts/pretrain/timer_xl_era5.sh
```

--------------------------------

### Full-Shot Fine-tuning

Source: https://context7.com/thuml/openltm/llms.txt

Fine-tune a pre-trained model on a downstream task using the full dataset. Specify the pre-trained model path.

```bash
python -u run.py \
  --task_name forecast \
  --is_training 1 \
  --root_path ./dataset/ETT-small/ \
  --data_path ETTh1.csv \
  --model timer_xl \
  --data UnivariateDatasetBenchmark \
  --seq_len 2880 \
  --input_token_len 96 \
  --output_token_len 96 \
  --e_layers 8 \
  --d_model 1024 \
  --d_ff 2048 \
  --batch_size 2048 \
  --learning_rate 5e-6 \
  --train_epochs 10 \
  --cosine \
  --tmax 10 \
  --use_norm \
  --adaptation \
  --pretrain_model_path checkpoints/timer_xl_pretrained/checkpoint.pth
```

--------------------------------

### Run Model Adaptation Scripts

Source: https://github.com/thuml/openltm/blob/main/README.md

These bash scripts are used for adapting pre-trained models, supporting both full-shot and few-shot fine-tuning scenarios.

```bash
# Model adaptation
# (a) full-shot fine-tune
bash ./scripts/adaptation/full_shot/timer_xl_etth1.sh
# (b) few-shot fine-tune
bash ./scripts/adaptation/few_shot/timer_xl_etth1.sh
```

--------------------------------

### DataParallel Alternative Training

Source: https://context7.com/thuml/openltm/llms.txt

A simpler, though less efficient, alternative for multi-GPU training using Python directly with the --dp flag.

```bash
python run.py \
  --dp \
  --devices 0,1,2,3,4,5,6,7 \
  ... # other arguments
```

--------------------------------

### Use Data Provider Factory

Source: https://context7.com/thuml/openltm/llms.txt

Loads benchmark datasets using the data provider factory and iterates through batches.

```python
from data_provider.data_factory import data_provider

class Args:
    data = 'UnivariateDatasetBenchmark'  # or 'MultivariateDatasetBenchmark'
    root_path = './dataset/ETT-small/'
    data_path = 'ETTh1.csv'
    seq_len = 672
    input_token_len = 96
    output_token_len = 96
    test_seq_len = 672
    test_pred_len = 96
    nonautoregressive = False
    test_flag = 'T'
    subset_rand_ratio = 1.0
    batch_size = 32
    num_workers = 4
    ddp = False

args = Args()

# Get train, validation, and test data loaders
train_data, train_loader = data_provider(args, flag='train')
val_data, val_loader = data_provider(args, flag='val')
test_data, test_loader = data_provider(args, flag='test')

print(f"Train samples: {len(train_data)}")
print(f"Val samples: {len(val_data)}")
print(f"Test samples: {len(test_data)}")

# Iterate through batches
for batch_x, batch_y, batch_x_mark, batch_y_mark in train_loader:
    print(f"Input batch: {batch_x.shape}")  # [batch, seq_len, features]
    print(f"Target batch: {batch_y.shape}")  # [batch, pred_len, features]
    break
```

--------------------------------

### Implement Early Stopping and Learning Rate Scheduling

Source: https://context7.com/thuml/openltm/llms.txt

Manage training stability using EarlyStopping and adjust_learning_rate utilities within a training loop.

```python
from utils.tools import EarlyStopping, adjust_learning_rate
import torch

class Args:
    patience = 3
    dp = False
    ddp = False
    lradj = 'type1'  # Options: type1, type2, type3
    learning_rate = 0.001

args = Args()

# Early stopping monitors validation loss
early_stopping = EarlyStopping(args, verbose=True, delta=0)

# Simulated training loop
model = torch.nn.Linear(10, 1)
optimizer = torch.optim.Adam(model.parameters(), lr=args.learning_rate)

for epoch in range(100):
    val_loss = 0.5 - epoch * 0.01  # Simulated decreasing loss

    # Check early stopping
    early_stopping(val_loss, model, './checkpoints/')

    if early_stopping.early_stop:
        print(f"Early stopping triggered at epoch {epoch}")
        break

    # Adjust learning rate
    adjust_learning_rate(optimizer, epoch, args)
```

--------------------------------

### Single-Node Multi-GPU Training with DDP

Source: https://context7.com/thuml/openltm/llms.txt

Use torchrun for distributed data parallel training across multiple GPUs on a single node. Ensure CUDA_VISIBLE_DEVICES is set correctly.

```bash
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7

torchrun --nproc_per_node=8 run.py \
  --task_name forecast \
  --is_training 1 \
  --root_path ./dataset/UTSD-full-npy \
  --model timer_xl \
  --data Utsd_Npy \
  --seq_len 2880 \
  --input_token_len 96 \
  --output_token_len 96 \
  --e_layers 8 \
  --d_model 1024 \
  --d_ff 2048 \
  --batch_size 16384 \
  --learning_rate 0.00005 \
  --train_epochs 10 \
  --cosine \
  --tmax 10 \
  --use_norm \
  --ddp
```

--------------------------------

### Import Dataset Classes

Source: https://context7.com/thuml/openltm/llms.txt

Imports the univariate and multivariate dataset classes for manual data handling.

```python
from data_provider.data_loader import UnivariateDatasetBenchmark, MultivariateDatasetBenchmark
```

--------------------------------

### Run Supervised Training Scripts

Source: https://github.com/thuml/openltm/blob/main/README.md

Execute these bash scripts for supervised training tasks, including one-for-one forecasting and one-for-all rolling forecasting.

```bash
# Supervised training
# (a) one-for-one forecasting
bash ./scripts/supervised/forecast/moirai_ecl.sh
# (b) one-for-all (rolling) forecasting
bash ./scripts/supervised/rolling_forecast/timer_xl_ecl.sh
```

--------------------------------

### Custom Model Integration: Registration

Source: https://context7.com/thuml/openltm/llms.txt

Register your custom model in exp/exp_basic.py by adding it to the model_dict. Ensure the model name matches the key.

```python
# 2. Register in exp/exp_basic.py
from models import timer, timer_xl, moirai, moment, gpt4ts, ttm, time_llm, autotimes
from models import my_model  # Add import

class Exp_Basic(object):
    def __init__(self, args):
        self.model_dict = {
            "timer": timer,
            "timer_xl": timer_xl,
            "my_model": my_model,  # Register model
            # ... other models
        }

```

--------------------------------

### Few-Shot Fine-tuning

Source: https://context7.com/thuml/openltm/llms.txt

Adapt a pre-trained model using a small subset of the data (e.g., 10%). Use --subset_rand_ratio to specify the data fraction.

```bash
python -u run.py \
  --task_name forecast \
  --is_training 1 \
  --adaptation \
  --pretrain_model_path checkpoints/timer_xl_pretrained/checkpoint.pth \
  --subset_rand_ratio 0.1 \
  ... # other arguments
```

--------------------------------

### BibTeX Citation for Timer-XL

Source: https://github.com/thuml/openltm/blob/main/README.md

This is the BibTeX entry for the 'Timer-XL: Long-Context Transformers for Unified Time Series Forecasting' paper.

```bibtex
@article{liu2024timer,
  title={Timer-XL: Long-Context Transformers for Unified Time Series Forecasting},
  author={Liu, Yong and Qin, Guo and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
  journal={arXiv preprint arXiv:2410.04803},
  year={2024}
}
```

--------------------------------

### BibTeX Citation for Timer

Source: https://github.com/thuml/openltm/blob/main/README.md

This is the BibTeX entry for the 'Timer: Generative Pre-trained Transformers Are Large Time Series Models' paper.

```bibtex
@inproceedings{liutimer,
  title={Timer: Generative Pre-trained Transformers Are Large Time Series Models},
  author={Liu, Yong and Zhang, Haoran and Li, Chenyu and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
  booktitle={Forty-first International Conference on Machine Learning}
}
```

--------------------------------

### Perform Zero-Shot Prediction

Source: https://github.com/thuml/openltm/blob/main/load_pth_ckpt.ipynb

Prepare input data from a CSV and generate forecasts using the loaded model.

```python
# evaluate zere-shot prediction

df = pd.read_csv("https://raw.githubusercontent.com/WenWeiTHU/TimeSeriesDatasets/refs/heads/main/ETT-small/ETTh2.csv")

lookback_length = 1440      # support the maximum context length up to 2880 
input = torch.tensor(df["OT"][:lookback_length]).unsqueeze(0).float()

# generate forecast 
prediction_length = 96     # forecast the next 96 timestamps, supporting maximum prediction length encompassed in the context length

output =  model(input.unsqueeze(-1), None, None)
```

--------------------------------

### Plot Final Forecast

Source: https://github.com/thuml/openltm/blob/main/load_pth_ckpt.ipynb

Visualize the ground truth, lookback window, and model prediction.

```python
# plot the prediction

plt.figure(figsize=(12, 4))
plt.plot(df["OT"][:lookback_length + prediction_length], color="limegreen", label="Groundtruth")
plt.plot(range(lookback_length, lookback_length + prediction_length), pred, color="tomato", label="Prediction")
plt.plot(df["OT"][:lookback_length], color="royalblue", label="Lookback")
plt.legend()
plt.grid()
plt.show()
```

--------------------------------

### Load UTSD Pre-training Datasets

Source: https://context7.com/thuml/openltm/llms.txt

Load large-scale pre-training data using either CSV or NPY formats. NPY format is recommended for faster data loading.

```python
from data_provider.data_loader import UTSD, UTSD_Npy

# For CSV format (original UTSD)
utsd_dataset = UTSD(
    root_path='./dataset/UTSD/',
    flag='train',
    size=[2880, 96, 96],  # 30 tokens * 96 token_len
    scale=True,
    stride=1,
    split=0.9
)

# For NPY format (faster loading)
utsd_npy = UTSD_Npy(
    root_path='./dataset/UTSD-full-npy/',
    flag='train',
    size=[2880, 96, 96],
    scale=True,
    stride=1,
    split=0.9
)

print(f"Total pre-training windows: {len(utsd_dataset)}")
```

--------------------------------

### Calculate Evaluation Metrics

Source: https://context7.com/thuml/openltm/llms.txt

Compute standard forecasting metrics like MSE, MAE, and RMSE using the provided utility functions.

```python
import numpy as np
from utils.metrics import metric, MAE, MSE, RMSE, MAPE, SMAPE

# Generate sample predictions and ground truth
pred = np.random.randn(100, 96, 7)  # [samples, pred_len, features]
true = np.random.randn(100, 96, 7)

# Calculate all metrics at once
mae, mse, rmse, mape, mspe, smape = metric(pred, true)

print(f"MAE: {mae:.4f}")
print(f"MSE: {mse:.4f}")
print(f"RMSE: {rmse:.4f}")
print(f"MAPE: {mape:.4f}")
print(f"SMAPE: {smape:.4f}")

# Individual metric functions
mae_score = MAE(pred, true)
mse_score = MSE(pred, true)
rmse_score = RMSE(pred, true)
```

--------------------------------

### BibTeX Citation for Sundial

Source: https://github.com/thuml/openltm/blob/main/README.md

This is the BibTeX entry for the 'Sundial: A Family of Highly Capable Time Series Foundation Models' paper.

```bibtex
@article{liu2025sundial,
  title={Sundial: A Family of Highly Capable Time Series Foundation Models},
  author={Liu, Yong and Qin, Guo and Shi, Zhiyuan and Chen, Zhi and Yang, Caiyin and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
  journal={arXiv preprint arXiv:2502.00816},
  year={2025}
}
```

--------------------------------

### Run GPT4TS Model

Source: https://context7.com/thuml/openltm/llms.txt

Processes time series data through a GPT-2 based architecture.

```python
x = torch.randn(8, 512, 7)
x_mark = torch.zeros(8, 512, 1)
y_mark = torch.zeros(8, 512, 1)

output = model(x, x_mark, y_mark)
print(f"GPT4TS output: {output.shape}")  # [8, 96, 7]
```

--------------------------------

### Extract Prediction

Source: https://github.com/thuml/openltm/blob/main/load_pth_ckpt.ipynb

Select the final prediction tokens from the model output sequence.

```python
# Note that the output is the whole sequence of next token prediction
# so we need to select the last token (token_len=96) as the final prediction

pred = output[:, -96:, 0].squeeze().detach().numpy()
```

--------------------------------

### Custom Model Integration: Model Definition

Source: https://context7.com/thuml/openltm/llms.txt

Define a custom time series model by inheriting from nn.Module and implementing the forward pass. Ensure it handles normalization if specified.

```python
# 1. Create models/my_model.py
import torch
from torch import nn

class Model(nn.Module):
    """
    Custom Time Series Model
    """
    def __init__(self, configs):
        super().__init__()
        self.seq_len = configs.seq_len
        self.pred_len = configs.test_pred_len
        self.use_norm = configs.use_norm

        # Your model architecture
        self.encoder = nn.Linear(configs.seq_len, configs.d_model)
        self.decoder = nn.Linear(configs.d_model, configs.test_pred_len)

    def forward(self, x, x_mark, y_mark):
        # x: [batch, seq_len, features]
        if self.use_norm:
            means = x.mean(1, keepdim=True).detach()
            x = x - means
            stdev = torch.sqrt(torch.var(x, dim=1, keepdim=True, unbiased=False) + 1e-5)
            x /= stdev

        # Process
        B, L, C = x.shape
        x = x.permute(0, 2, 1)  # [B, C, L]
        enc_out = self.encoder(x)  # [B, C, D]
        dec_out = self.decoder(enc_out)  # [B, C, pred_len]
        dec_out = dec_out.permute(0, 2, 1)  # [B, pred_len, C]

        if self.use_norm:
            dec_out = dec_out * stdev + means

        return dec_out

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.