### Install Project Dependencies Source: https://github.com/zhouhaoyi/informer2020/blob/main/README.md Installs the required Python libraries for the Informer project using pip and the provided requirements file. ```bash pip install -r requirements.txt ``` -------------------------------- ### Manage Experiments with Docker and Make Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Provides Makefile commands for environment setup, dataset downloading, and executing experiments in a containerized environment. ```makefile make init make dataset make run_module module="bash scripts/ETTh1.sh" make jupyter make bash_docker ``` -------------------------------- ### Build Encoder with Self-Attention Distilling Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Illustrates the construction of an Informer encoder using distillation layers to reduce sequence length. Includes examples for both standard Encoder and EncoderStack configurations. ```python import torch import torch.nn as nn from models.encoder import Encoder, EncoderLayer, ConvLayer, EncoderStack from models.attn import ProbAttention, AttentionLayer d_model = 512 n_heads = 8 d_ff = 2048 e_layers = 2 def create_attn_layer(): return AttentionLayer( ProbAttention(False, factor=5, attention_dropout=0.05, output_attention=False), d_model, n_heads, mix=False ) encoder_layers = [ EncoderLayer( attention=create_attn_layer(), d_model=d_model, d_ff=d_ff, dropout=0.05, activation='gelu' ) for _ in range(e_layers) ] conv_layers = [ConvLayer(d_model) for _ in range(e_layers - 1)] encoder = Encoder( attn_layers=encoder_layers, conv_layers=conv_layers, norm_layer=nn.LayerNorm(d_model) ) batch_size, seq_len = 32, 96 x = torch.randn(batch_size, seq_len, d_model) enc_out, attentions = encoder(x, attn_mask=None) ``` -------------------------------- ### Execute Informer Model Training and Inference Source: https://github.com/zhouhaoyi/informer2020/blob/main/README.md Commands to initiate the training or prediction process for the Informer model. The first example demonstrates a specific run on the ETTm1 dataset, while the second provides a template for full parameter configuration. ```bash python -u main_informer.py --model informer --data ETTm1 --attn prob --freq t ``` ```bash python -u main_informer.py --model --data --root_path --data_path --features --target --freq --checkpoints --seq_len --label_len --pred_len --enc_in --dec_in --c_out --d_model --n_heads --e_layers --d_layers --s_layers --d_ff --factor --padding --distil --dropout --attn --embed --activation --output_attention --do_predict --mix --cols --itr --num_workers --train_epochs --batch_size --patience --des --learning_rate --loss --lradj --use_amp --inverse --use_gpu --gpu --use_multi_gpu --devices ``` -------------------------------- ### Decoder with Cross-Attention Source: https://context7.com/zhouhaoyi/informer2020/llms.txt This section details the decoder component of the Informer model, which utilizes both self-attention and cross-attention mechanisms for sequence generation. It includes a Python code example demonstrating its initialization and a forward pass. ```APIDOC ## Decoder with Cross-Attention ### Description The decoder processes the label sequence and attends to encoder output using both self-attention and cross-attention for generating predictions. ### Method N/A (This is a model component description, not an API endpoint) ### Endpoint N/A ### Parameters N/A ### Request Example ```python import torch import torch.nn as nn from models.decoder import Decoder, DecoderLayer from models.attn import ProbAttention, FullAttention, AttentionLayer d_model = 512 n_heads = 8 d_ff = 2048 d_layers = 1 # Decoder uses ProbAttention for self-attention, FullAttention for cross-attention decoder_layers = [ DecoderLayer( self_attention=AttentionLayer(ProbAttention(True, factor=5, attention_dropout=0.05), d_model, n_heads, mix=True), cross_attention=AttentionLayer(FullAttention(False, factor=5, attention_dropout=0.05), d_model, n_heads, mix=False), d_model=d_model, d_ff=d_ff, dropout=0.05, activation='gelu' ) for _ in range(d_layers) ] decoder = Decoder(layers=decoder_layers, norm_layer=nn.LayerNorm(d_model)) # Forward pass batch_size = 32 label_len, pred_len = 48, 24 dec_seq_len = label_len + pred_len # Decoder input and encoder output x = torch.randn(batch_size, dec_seq_len, d_model) # Decoder input cross = torch.randn(batch_size, 48, d_model) # Encoder output (after distillation) dec_out = decoder(x, cross, x_mask=None, cross_mask=None) print(f"Decoder output shape: {dec_out.shape}") # [32, 72, 512] ``` ### Response #### Success Response (200) N/A #### Response Example N/A ``` -------------------------------- ### Configure and Run Informer Experiment Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Demonstrates how to initialize experiment arguments using a dotdict, instantiate the Informer model, and execute training, testing, and prediction workflows. ```python args = dotdict({'data': 'ETTh1', 'seq_len': 96, 'label_len': 48, 'pred_len': 24, 'model': 'informer', 'train_epochs': 6, 'batch_size': 32, 'learning_rate': 0.0001, 'use_gpu': True}) exp = Exp_Informer(args) setting = f"{args.model}_{args.data}_ft{args.features}_sl{args.seq_len}_pl{args.pred_len}" model = exp.train(setting) exp.test(setting) exp.predict(setting, load=True) ``` -------------------------------- ### Initialize and Run Informer Model in Python Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Shows how to import and instantiate the Informer and InformerStack classes. It includes setting up input tensors and performing a forward pass to generate predictions. ```python import torch from models.model import Informer, InformerStack # Initialize Informer model model = Informer( enc_in=7, dec_in=7, c_out=7, seq_len=96, label_len=48, out_len=24, factor=5, d_model=512, n_heads=8, e_layers=2, d_layers=1, d_ff=2048, dropout=0.05, attn='prob', embed='timeF', freq='h', activation='gelu', output_attention=False, distil=True, mix=True, device=torch.device('cuda:0') ) # Forward pass batch_size = 32 x_enc = torch.randn(batch_size, 96, 7).cuda() x_mark_enc = torch.randn(batch_size, 96, 4).cuda() x_dec = torch.randn(batch_size, 72, 7).cuda() x_mark_dec = torch.randn(batch_size, 72, 4).cuda() # Get predictions output = model(x_enc, x_mark_enc, x_dec, x_mark_dec) print(f"Output shape: {output.shape}") # InformerStack variant model_stack = InformerStack( enc_in=7, dec_in=7, c_out=7, seq_len=96, label_len=48, out_len=24, e_layers=[3, 2, 1], d_layers=1, d_model=512, n_heads=8, d_ff=2048, attn='prob', distil=True, device=torch.device('cuda:0') ) ``` -------------------------------- ### Configure Multi-GPU Training Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Demonstrates how to enable multi-GPU training via command-line arguments or using PyTorch DataParallel in the model initialization script. ```bash python -u main_informer.py --model informer --data ETTh1 --use_gpu True --use_multi_gpu --devices '0,1,2,3' ``` ```python args_use_multi_gpu = True args_device_ids = [0, 1, 2, 3] model = Informer(...) if args_use_multi_gpu: model = nn.DataParallel(model, device_ids=args_device_ids) model = model.cuda() ``` -------------------------------- ### Implement ProbSparse and Full Attention Mechanisms Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Demonstrates how to initialize and use ProbSparse Attention for O(L log L) complexity compared to standard Full Attention. It includes wrapping these mechanisms into an AttentionLayer with projection heads. ```python import torch from models.attn import ProbAttention, FullAttention, AttentionLayer prob_attention = ProbAttention( mask_flag=True, factor=5, scale=None, attention_dropout=0.1, output_attention=False ) full_attention = FullAttention( mask_flag=True, factor=5, attention_dropout=0.1, output_attention=False ) d_model = 512 n_heads = 8 attention_layer = AttentionLayer( attention=prob_attention, d_model=d_model, n_heads=n_heads, d_keys=None, d_values=None, mix=False ) batch_size, seq_len = 32, 96 queries = torch.randn(batch_size, seq_len, d_model) keys = torch.randn(batch_size, seq_len, d_model) values = torch.randn(batch_size, seq_len, d_model) output, attn_weights = attention_layer(queries, keys, values, attn_mask=None) print(f"Attention output shape: {output.shape}") ``` -------------------------------- ### Train Informer Model via CLI Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Demonstrates how to execute training, evaluation, and prediction tasks using the main_informer.py script. It covers basic configurations, multivariate forecasting, univariate long-term forecasting, and enabling prediction mode. ```bash # Basic training on ETTh1 dataset with ProbSparse attention python -u main_informer.py --model informer --data ETTh1 --attn prob --freq h # Full multivariate training with custom parameters python -u main_informer.py \ --model informer \ --data ETTh1 \ --features M \ --seq_len 96 \ --label_len 48 \ --pred_len 24 \ --enc_in 7 \ --dec_in 7 \ --c_out 7 \ --d_model 512 \ --n_heads 8 \ --e_layers 2 \ --d_layers 1 \ --d_ff 2048 \ --factor 5 \ --dropout 0.05 \ --attn prob \ --embed timeF \ --activation gelu \ --train_epochs 6 \ --batch_size 32 \ --learning_rate 0.0001 \ --patience 3 \ --des 'Exp' \ --itr 5 # Univariate long-term forecasting (720 hours ahead) python -u main_informer.py \ --model informer \ --data ETTh1 \ --features S \ --seq_len 720 \ --label_len 336 \ --pred_len 720 \ --e_layers 2 \ --d_layers 1 \ --attn prob \ --des 'Exp' \ --itr 5 # Enable prediction on unseen future data python -u main_informer.py \ --model informer \ --data ETTh1 \ --features M \ --seq_len 96 \ --label_len 48 \ --pred_len 24 \ --attn prob \ --do_predict ``` -------------------------------- ### Implement Training Loop with Early Stopping and LR Adjustment Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Demonstrates how to integrate early stopping and dynamic learning rate scheduling into a PyTorch training loop. It assumes the existence of early_stopping and adjust_learning_rate utility functions. ```python model = torch.nn.Linear(10, 1) optimizer = torch.optim.Adam(model.parameters(), lr=0.0001) checkpoint_path = './checkpoints/experiment/' for epoch in range(100): val_loss = 0.05 early_stopping(val_loss, model, checkpoint_path) if early_stopping.early_stop: break class Args: learning_rate = 0.0001 lradj = 'type1' args = Args() for epoch in range(1, 20): adjust_learning_rate(optimizer, epoch, args) ``` -------------------------------- ### Execute Training Scripts Source: https://github.com/zhouhaoyi/informer2020/blob/main/README.md Iterates through all scripts located in the scripts directory and executes them using the project's makefile system. ```bash for file in `ls scripts`; do make run_module module="bash scripts/$script"; done ``` -------------------------------- ### Initialize Experiment Runner Source: https://context7.com/zhouhaoyi/informer2020/llms.txt This snippet initializes the Exp_Informer class, which serves as the main entry point for managing the training, validation, and testing lifecycle of the Informer model. ```python import argparse from exp.exp_informer import Exp_Informer from utils.tools import dotdict ``` -------------------------------- ### Load and Preprocess Time-Series Datasets Source: https://context7.com/zhouhaoyi/informer2020/llms.txt This snippet shows how to use the provided PyTorch Dataset classes to load ETT or custom time-series data. It covers configuration for multivariate/univariate features, normalization, and creating a DataLoader for training. ```python import torch from torch.utils.data import DataLoader from data.data_loader import Dataset_ETT_hour, Dataset_ETT_minute, Dataset_Custom train_dataset = Dataset_ETT_hour( root_path='./data/ETT/', data_path='ETTh1.csv', flag='train', size=[96, 48, 24], features='M', target='OT', scale=True, inverse=False, timeenc=1, freq='h' ) train_loader = DataLoader( train_dataset, batch_size=32, shuffle=True, num_workers=0, drop_last=True ) for batch_x, batch_y, batch_x_mark, batch_y_mark in train_loader: print(f"batch_x (encoder input): {batch_x.shape}") break predictions = torch.randn(32, 24, 7) original_scale = train_dataset.inverse_transform(predictions) ``` -------------------------------- ### Implement Early Stopping Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Configures an EarlyStopping object to monitor model performance during training and halt execution when validation metrics stop improving. ```python from utils.tools import EarlyStopping early_stopping = EarlyStopping(patience=3, verbose=True, delta=0) ``` -------------------------------- ### Implement Informer Decoder with Cross-Attention Source: https://context7.com/zhouhaoyi/informer2020/llms.txt This snippet demonstrates how to configure and instantiate the Informer decoder. It utilizes ProbAttention for self-attention and FullAttention for cross-attention mechanisms to process encoder outputs and generate sequence predictions. ```python import torch import torch.nn as nn from models.decoder import Decoder, DecoderLayer from models.attn import ProbAttention, FullAttention, AttentionLayer d_model = 512 n_heads = 8 d_ff = 2048 d_layers = 1 decoder_layers = [ DecoderLayer( self_attention=AttentionLayer( ProbAttention(True, factor=5, attention_dropout=0.05), d_model, n_heads, mix=True ), cross_attention=AttentionLayer( FullAttention(False, factor=5, attention_dropout=0.05), d_model, n_heads, mix=False ), d_model=d_model, d_ff=d_ff, dropout=0.05, activation='gelu' ) for _ in range(d_layers) ] decoder = Decoder( layers=decoder_layers, norm_layer=nn.LayerNorm(d_model) ) batch_size = 32 label_len, pred_len = 48, 24 dec_seq_len = label_len + pred_len x = torch.randn(batch_size, dec_seq_len, d_model) cross = torch.randn(batch_size, 48, d_model) dec_out = decoder(x, cross, x_mask=None, cross_mask=None) print(f"Decoder output shape: {dec_out.shape}") ``` -------------------------------- ### Train Informer Model Source: https://github.com/zhouhaoyi/informer2020/blob/main/README.md Command to initiate training of the Informer model on the ETTh1 dataset using ProbSparse attention. ```bash python -u main_informer.py --model informer --data ETTh1 --attn prob --freq h ``` -------------------------------- ### Dataset Loading Source: https://context7.com/zhouhaoyi/informer2020/llms.txt This section describes the PyTorch Dataset classes provided for loading and preprocessing time-series data. It covers automatic train/validation/test splitting, normalization, and handling different data frequencies (hourly, minutely, custom). ```APIDOC ## Dataset Loading ### Description PyTorch Dataset classes for loading and preprocessing time-series data with automatic train/val/test splitting and normalization. ### Method N/A (This is a utility description, not an API endpoint) ### Endpoint N/A ### Parameters N/A ### Request Example ```python import torch from torch.utils.data import DataLoader from data.data_loader import Dataset_ETT_hour, Dataset_ETT_minute, Dataset_Custom # Load hourly ETT dataset train_dataset = Dataset_ETT_hour( root_path='./data/ETT/', data_path='ETTh1.csv', flag='train', # 'train', 'val', or 'test' size=[96, 48, 24], # [seq_len, label_len, pred_len] features='M', # 'M': multivariate, 'S': univariate, 'MS': multi-to-single target='OT', # Target column for S/MS tasks scale=True, # Enable StandardScaler normalization inverse=False, # Inverse transform outputs timeenc=1, # 0: categorical, 1: continuous time features freq='h' # Time frequency ) # Create DataLoader train_loader = DataLoader( train_dataset, batch_size=32, shuffle=True, num_workers=0, drop_last=True ) # Iterate through batches for batch_x, batch_y, batch_x_mark, batch_y_mark in train_loader: print(f"batch_x (encoder input): {batch_x.shape}") # [32, 96, 7] print(f"batch_y (decoder target): {batch_y.shape}") # [32, 72, 7] print(f"batch_x_mark (time features): {batch_x_mark.shape}") # [32, 96, 4] print(f"batch_y_mark (time features): {batch_y_mark.shape}") # [32, 72, 4] break # Load minutely data minute_dataset = Dataset_ETT_minute( root_path='./data/ETT/', data_path='ETTm1.csv', flag='train', size=[96, 48, 24], features='M', timeenc=1, freq='t' # minutely ) # Custom dataset for your own data # CSV format: ['date', feature1, feature2, ..., target] custom_dataset = Dataset_Custom( root_path='./data/', data_path='your_data.csv', flag='train', size=[96, 48, 24], features='M', target='your_target_column', cols=['feature1', 'feature2', 'target'], # Optional: select specific columns timeenc=1, freq='h' ) # Inverse transform predictions back to original scale predictions = torch.randn(32, 24, 7) original_scale = train_dataset.inverse_transform(predictions) ``` ### Response #### Success Response (200) N/A #### Response Example N/A ``` -------------------------------- ### Normalize Time-Series Data with StandardScaler Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Shows how to fit a scaler on training data and transform it, as well as how to inverse transform predictions back to the original scale. ```python scaler = StandardScaler() train_data = np.random.randn(1000, 7) scaler.fit(train_data) normalized = scaler.transform(train_data) predictions = torch.randn(32, 24, 7) original_scale = scaler.inverse_transform(predictions) ``` -------------------------------- ### Configure Data Embedding Layers Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Shows how to combine value, positional, and temporal embeddings to create rich input representations. It covers the DataEmbedding wrapper and individual component usage. ```python import torch from models.embed import DataEmbedding, TokenEmbedding, PositionalEmbedding embedding = DataEmbedding( c_in=7, d_model=512, embed_type='timeF', freq='h', dropout=0.1 ) batch_size, seq_len = 32, 96 x = torch.randn(batch_size, seq_len, 7) x_mark = torch.randn(batch_size, seq_len, 4) embedded = embedding(x, x_mark) print(f"Embedded shape: {embedded.shape}") token_embed = TokenEmbedding(c_in=7, d_model=512) pos_embed = PositionalEmbedding(d_model=512, max_len=5000) token_out = token_embed(x) pos_out = pos_embed(x) ``` -------------------------------- ### Experiment Runner Source: https://context7.com/zhouhaoyi/informer2020/llms.txt This section introduces the `Exp_Informer` class, which serves as the central orchestrator for the entire Informer model pipeline, including training, validation, testing, and prediction phases. It also manages functionalities like early stopping and learning rate scheduling. ```APIDOC ## Experiment Runner ### Description The `Exp_Informer` class orchestrates the complete training, validation, testing, and prediction pipeline with early stopping and learning rate scheduling. ### Method N/A (This is a class description, not an API endpoint) ### Endpoint N/A ### Parameters N/A ### Request Example ```python import argparse from exp.exp_informer import Exp_Informer from utils.tools import dotdict # Example usage (conceptual): # parser = argparse.ArgumentParser(description='Informer Experiment Runner') # # Add arguments for configuration, data paths, hyperparameters, etc. # args = parser.parse_args() # # Convert args to dotdict for easier access # config = dotdict(vars(args)) # # Initialize the experiment runner # exp = Exp_Informer(config) # # Run the experiment (e.g., training) # exp.train() # # Evaluate the model # exp.test() ``` ### Response #### Success Response (200) N/A #### Response Example N/A ``` -------------------------------- ### Compute Time-Series Forecasting Metrics Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Provides utility functions to calculate standard regression metrics such as MAE, MSE, RMSE, MAPE, and MSPE for time-series predictions against ground truth data. ```python import numpy as np from utils.metrics import metric, MAE, MSE, RMSE, MAPE, MSPE pred = np.random.randn(1000, 24, 7) true = np.random.randn(1000, 24, 7) mae, mse, rmse, mape, mspe = metric(pred, true) mae_value = MAE(pred, true) ``` -------------------------------- ### Encoder with Distilling Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Constructs an encoder stack with optional distillation layers to reduce sequence length and highlight dominant features. ```APIDOC ## Encoder with Self-Attention Distilling ### Description Builds a deep encoder structure using multiple EncoderLayers and ConvLayer-based distillation. ### Parameters - **attn_layers** (list) - Required - List of EncoderLayer instances. - **conv_layers** (list) - Optional - List of ConvLayer instances for distillation. - **norm_layer** (nn.Module) - Required - Normalization layer. ### Request Example encoder = Encoder(attn_layers=encoder_layers, conv_layers=conv_layers, norm_layer=nn.LayerNorm(d_model)) ### Response - **enc_out** (Tensor) - The distilled output representation from the encoder. ``` -------------------------------- ### Extract Temporal Features Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Shows how to extract time-based features from datetime indices, supporting both continuous normalized encoding and categorical encoding for various time granularities. ```python import pandas as pd from utils.timefeatures import time_features dates = pd.DataFrame({'date': pd.date_range('2020-01-01', periods=100, freq='H')}) features = time_features(dates, timeenc=1, freq='h') features_cat = time_features(dates.copy(), timeenc=0, freq='h') ``` -------------------------------- ### ProbSparse Attention Mechanism Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Configures and executes the ProbSparse Attention mechanism, which reduces complexity to O(L log L) by selecting dominant queries. ```APIDOC ## ProbSparse Attention Mechanism ### Description Initializes and runs the ProbSparse Attention layer, designed for long sequence processing by sampling dominant keys. ### Method N/A (Class Initialization) ### Parameters - **mask_flag** (bool) - Required - Enable causal masking. - **factor** (int) - Required - Sampling factor for key selection. - **attention_dropout** (float) - Optional - Dropout rate. ### Request Example prob_attention = ProbAttention(mask_flag=True, factor=5, attention_dropout=0.1) ### Response - **output** (Tensor) - The attention-weighted representation of the input sequence. ``` -------------------------------- ### Data Embedding Module Source: https://context7.com/zhouhaoyi/informer2020/llms.txt Combines value, positional, and temporal embeddings to create a rich input representation for time-series forecasting. ```APIDOC ## Data Embedding ### Description Processes raw input features and time markers into high-dimensional embeddings. ### Parameters - **c_in** (int) - Required - Number of input features. - **d_model** (int) - Required - Embedding dimension. - **embed_type** (string) - Optional - Type of embedding ('timeF', 'fixed', 'learned'). - **freq** (string) - Optional - Time frequency code. ### Request Example embedding = DataEmbedding(c_in=7, d_model=512, embed_type='timeF', freq='h') ### Response - **embedded** (Tensor) - Combined embedding tensor of shape [batch_size, seq_len, d_model]. ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.