### Set up PyTorch Frame for Development Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/installation.rst This snippet outlines the steps to clone the PyTorch Frame repository and install it in editable mode with development dependencies. This setup is ideal for contributing to the project or making local modifications. ```bash git clone https://github.com/pyg-team/pytorch-frame.git cd pytorch-frame pip install -e .[dev] # Install with optional dependencies pip install -e .[dev,full] ``` -------------------------------- ### Install PyTorch Frame from GitHub Master Branch Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/installation.rst This snippet shows how to install the latest development version of PyTorch Frame directly from its GitHub master branch using pip. This is useful for accessing the most recent features and bug fixes. ```bash pip install git+https://github.com/pyg-team/pytorch-frame.git ``` -------------------------------- ### Install PyTorch Frame via PyPI Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/installation.rst This snippet demonstrates how to install PyTorch Frame using pip, including an option for full dependencies. It supports Python 3.9 to 3.13 on Linux, Windows, and macOS. ```bash pip install pytorch-frame # Install with optional dependencies pip install pytorch-frame[full] ``` -------------------------------- ### Initialize PyTorch Frame with Titanic Benchmark Dataset Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This Python snippet demonstrates how to load and inspect a pre-loaded benchmark dataset, specifically the Titanic dataset, within PyTorch Frame. It covers importing the dataset, initializing it from a specified root directory, checking its size and feature columns, materializing the dataset, and displaying the first few rows of the underlying pandas DataFrame. This is a foundational example for working with built-in datasets. ```python from torch_frame.datasets import Titanic dataset = Titanic(root='/tmp/titanic') len(dataset) dataset.feat_cols dataset.materialize() dataset.df.head(5) ``` -------------------------------- ### Install Python Dependencies for PyTorch Frame Benchmarking Source: https://github.com/pyg-team/pytorch-frame/blob/master/benchmark/README.md This snippet outlines the necessary Python packages to install before running performance benchmarks with PyTorch Frame. These dependencies include libraries for hyperparameter optimization (Optuna), metrics (torchmetrics), and various gradient boosting frameworks (XGBoost, CatBoost, LightGBM). ```bash pip install optuna pip install torchmetrics pip install xgboost pip install catboost pip install lightgbm ``` -------------------------------- ### Build PyTorch Frame Documentation Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/README.md Follow these steps to compile the PyTorch Frame documentation into viewable HTML files. This involves installing the Sphinx theme and executing the build commands within the documentation directory. ```Shell pip install git+https://github.com/pyg-team/pyg_sphinx_theme.git ``` ```Shell cd docs make html ``` -------------------------------- ### Load PyTorch Frame Data in Mini-Batches with DataLoader Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This example demonstrates how to utilize 'torch_frame.data.DataLoader' to efficiently load 'TensorFrame' objects in mini-batches, a common practice for training neural networks. It shows the instantiation of the DataLoader with batch size and shuffling options, and how to iterate through it to retrieve batched 'TensorFrame' objects for model input. ```python from torch_frame.data import DataLoader data_loader = DataLoader(tensor_frame, batch_size=32, shuffle=True) for batch in data_loader: batch # Expected output: # TensorFrame( # num_cols=7, # num_rows=32, # categorical (3): ['Pclass', 'Sex', 'Embarked'], # numerical (4): ['Age', 'SibSp', 'Parch', 'Fare'], # has_target=True, # device='cpu', # ) ``` -------------------------------- ### Initialize ExampleTransformer Model for PyTorch Frame Training Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This code snippet shows how to initialize the 'ExampleTransformer' model, preparing it for training on a specific device (GPU if available, otherwise CPU). It sets the model's hyperparameters such as 'channels', 'out_channels' (derived from dataset classes), 'num_layers', and 'num_heads', which are crucial for the model's architecture and performance. ```python import torch import torch.nn.functional as F device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') model = ExampleTransformer( channels=32, out_channels=dataset.num_classes, num_layers=2, num_heads=4, col_stats=dataset.col_stats, col_names_dict=dataset.col_names_dict, ) ``` -------------------------------- ### Install PyTorch Frame via pip for Python Source: https://github.com/pyg-team/pytorch-frame/blob/master/README.md This snippet demonstrates the standard method to install PyTorch Frame using pip. It is compatible with Python versions 3.9 through 3.13. This command installs the core library without additional dependencies. ```Python pip install pytorch-frame ``` -------------------------------- ### Install Transformers Library for Text Tokenization Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/handling_advanced_stypes/handle_text.rst This command installs the Hugging Face Transformers library, which is required for fine-tuning text models and performing text tokenization within PyTorch Frame. It's a necessary dependency for working with `stype.text_tokenized` columns. ```bash pip install transformers ``` -------------------------------- ### Install PEFT Library for Finetuning Text Models Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/handling_advanced_stypes/handle_text.rst This snippet provides the command to install the PEFT (Parameter-Efficient Finetuning) library. PEFT is a crucial dependency for applying efficient finetuning strategies like LoRA to large language models, which is demonstrated later for integrating text models into tabular learning workflows. ```bash pip install peft ``` -------------------------------- ### Prepare PyTorch Frame Datasets for Model Training Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This snippet demonstrates the essential steps for preparing tabular data for model training using PyTorch Frame. It covers loading a dataset (e.g., Yandex 'adult'), materializing it, shuffling, performing a train-test split, and finally creating 'DataLoader' instances for both training and testing sets, ensuring data is ready for batch processing. ```python from torch_frame.datasets import Yandex from torch_frame.data import DataLoader dataset = Yandex(root='/tmp/adult', name='adult') dataset.materialize() dataset.shuffle() train_dataset, test_dataset = dataset[:0.8], dataset[0.8:] train_loader = DataLoader(train_dataset.tensor_frame, batch_size=128, shuffle=True) test_loader = DataLoader(test_dataset.tensor_frame, batch_size=128) ``` -------------------------------- ### Materialize PyTorch Frame Dataset with Caching Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This snippet demonstrates how to materialize a PyTorch Frame Dataset into a TensorFrame. It shows how to enable caching for faster subsequent materializations and how to inspect the structure of the resulting TensorFrame, including its feature dictionaries and target tensor. ```python from torch_frame import stype # materialize the dataset dataset.materialize() # materialize the dataset with caching enabled dataset.materialize(path='/tmp/titanic/data.pt') # next materialization will load the cache dataset.materialize(path='/tmp/titanic/data.pt') tensor_frame = dataset.tensor_frame # Access keys of feature dictionary feat_dict_keys = tensor_frame.feat_dict.keys() # Expected output: dict_keys([, ]) # Access numerical features numerical_features = tensor_frame.feat_dict[stype.numerical] # Expected output: tensor([[22.0000, 1.0000, 0.0000, 7.2500], # [38.0000, 1.0000, 0. ``` -------------------------------- ### Train PyTorch Frame Model with Adam Optimizer Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This snippet demonstrates the training loop for a PyTorch Frame model. It includes parameters for model initialization (like num_heads, col_stats, col_names_dict), initializes an Adam optimizer, and iterates through epochs. For each batch, data is moved to the device, predictions are made, cross-entropy loss is calculated, and model parameters are updated via backpropagation. ```python num_heads=8, col_stats=train_dataset.col_stats, col_names_dict=train_dataset.tensor_frame.col_names_dict, ).to(device) optimizer = torch.optim.Adam(model.parameters()) for epoch in range(50): for tf in train_loader: tf = tf.to(device) pred = model(tf) loss = F.cross_entropy(pred, tf.y) optimizer.zero_grad() loss.backward() optimizer.step() ``` -------------------------------- ### Install Sentence-Transformers for PyTorch Frame Text Processing Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/handling_advanced_stypes/handle_text.rst This command installs the `sentence-transformers` Python package, a crucial dependency for utilizing pre-trained language models to generate text embeddings within PyTorch Frame. It ensures the package is updated to its latest version, enabling the functionality for `stype.text_embedded` columns. ```bash pip install -U sentence-transformers ``` -------------------------------- ### Analyze PyTorch Frame Encoder Benchmark Torch Profiler Output Source: https://github.com/pyg-team/pytorch-frame/blob/master/benchmark/encoder/README.md This example demonstrates the output from the Torch profiler, which provides a detailed table of operations sorted by their execution time. This output is crucial for identifying performance bottlenecks within PyTorch operations during the benchmark. ```text ----------------------------- ------------ ------------ ------------ ------------ ------------ ------------ Name Self CPU % Self CPU CPU total % CPU total CPU time avg # of Calls ----------------------------- ------------ ------------ ------------ ------------ ------------ ------------ aten::cat 47.49% 2.027s 48.05% 2.051s 1.025ms 2000 aten::nan_to_num 19.74% 842.584ms 39.35% 1.680s 419.945us 4000 aten::add 10.04% 428.549ms 10.04% 428.549ms 214.274us 2000 aten::index_select 6.28% 268.051ms 7.78% 331.959ms 165.980us 2000 aten::mul 4.96% 211.853ms 4.96% 211.853ms 211.853us 1000 aten::sub 1.36% 58.064ms 1.36% 58.064ms 58.064us 1000 aten::any 1.30% 55.612ms 1.41% 60.278ms 30.139us 2000 aten::div 1.22% 52.159ms 1.22% 52.159ms 52.159us 1000 ... ----------------------------- ------------ ------------ ------------ ------------ ------------ ------------ Self CPU time total: 4.268s ``` -------------------------------- ### Create Custom PyTorch Frame Dataset from Pandas DataFrame Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This Python example illustrates the process of constructing a custom PyTorch Frame Dataset from an existing pandas DataFrame. It requires defining a dictionary that maps each column name to its corresponding PyTorch Frame semantic type (stype) and specifying the target column. This functionality is crucial for integrating user-specific tabular data into the PyTorch Frame ecosystem for deep learning tasks. ```python import torch_frame from torch_frame.data import Dataset # Specify the stype of each column with a dictionary. col_to_stype = { "cat1": torch_frame.categorical, "cat2": torch_frame.categorical, "num1": torch_frame.numerical, "num2": torch_frame.numerical, "y": torch_frame.categorical, } # Set "y" as the target column. dataset = Dataset(df, col_to_stype=col_to_stype, target_col="y") ``` -------------------------------- ### Run PyTorch Frame Model Benchmarking with Hyperparameter Tuning Source: https://github.com/pyg-team/pytorch-frame/blob/master/benchmark/README.md This script executes a comprehensive benchmarking process for specified models on `DataFrameBenchmark` datasets. It allows configuration of model type, task type, dataset scale, and the number of Optuna trials for hyperparameter search, saving results to a specified path. It depends on the previously installed packages. ```bash # Specify the model from [TabNet, FTTransformer, ResNet, MLP, TabTransformer, # Trompt, ExcelFormer, FTTransformerBucket, XGBoost, CatBoost, LightGBM] model_type=TabNet # Specify the task type from [binary_classification, regression, # multiclass_classification] task_type=binary_classification # Specify the dataset scale from [small, medium, large] scale=small # Specify the dataset idx from [0, 1, ...] idx=0 # Specify the number of AutoML search trials num_trials=20 # Specify the path to save the results result_path=results.pt # Run hyper-parameter tuning and training of the specified model on a specified # dataset. python data_frame_benchmark.py --model_type $model_type\ --task_type $task_type\ --scale $scale\ --idx $idx\ --num_trials $num_trials\ --result_path $result_path ``` -------------------------------- ### Train a PyTorch Frame Tabular Model in Python Source: https://github.com/pyg-team/pytorch-frame/blob/master/README.md This snippet outlines the standard PyTorch training procedure for the `ExampleTransformer` model. It covers device setup (CPU/GPU), model instantiation, optimizer definition (Adam), and the main training loop. The loop iterates through epochs and batches, performing forward passes, calculating cross-entropy loss, and updating model parameters via backpropagation. ```python import torch import torch.nn.functional as F device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') model = ExampleTransformer( channels=32, out_channels=dataset.num_classes, num_layers=2, num_heads=8, col_stats=train_dataset.col_stats, col_names_dict=train_dataset.tensor_frame.col_names_dict, ).to(device) optimizer = torch.optim.Adam(model.parameters()) for epoch in range(50): for tf in train_loader: tf = tf.to(device) pred = model.forward(tf) loss = F.cross_entropy(pred, tf.y) optimizer.zero_grad() loss.backward() ``` -------------------------------- ### Evaluate PyTorch Frame Model Accuracy Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This snippet evaluates the trained PyTorch Frame model on the test dataset. It sets the model to evaluation mode (model.eval()), iterates through the test_loader, moves data to the device, and generates predictions. The accuracy is then computed by comparing the predicted classes with the true labels and printed to the console. ```python model.eval() correct = 0 for tf in test_loader: tf = tf.to(device) pred = model(tf) pred_class = pred.argmax(dim=-1) correct += (tf.y == pred_class).sum() acc = int(correct) / len(test_dataset) print(f'Accuracy: {acc:.4f}') # Accuracy: 0.8447 ``` -------------------------------- ### Access PyTorch Frame Dataset Column Statistics Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This snippet demonstrates how to inspect the statistical properties of columns within a PyTorch Frame Dataset. It shows how to retrieve the 'stype' for each column and specific statistics like count for categorical features or mean, standard deviation, and quantiles for numerical features, providing insights into the dataset's structure. ```python {'Survived': , 'Pclass': , 'Sex': , 'Age': , 'SibSp': , 'Parch': , 'Fare': , 'Embarked': } dataset.col_stats['Sex'] {: (['male', 'female'], [577, 314])} dataset.col_stats['Age'] {: 29.69911764705882, : 14.516321150817316, : [0.42, 20.125, 28.0, 38.0, 80.0]} ``` -------------------------------- ### Convert Pandas DataFrame to PyTorch Frame TensorFrame Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This code snippet illustrates the process of converting a standard 'pandas.DataFrame' into a 'torch_frame.data.TensorFrame' object. This conversion is essential for integrating existing tabular data with PyTorch Frame's specialized data structures, enabling efficient processing and model training within the framework. ```python new_tf = dataset.convert_to_tensor_frame(new_df) ``` -------------------------------- ### Define a Custom ExampleTransformer Model in PyTorch Frame Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This snippet provides the full Python class definition for 'ExampleTransformer', a custom neural network model designed for tabular data within PyTorch Frame. It showcases the use of 'StypeWiseFeatureEncoder' for feature encoding, 'EmbeddingEncoder' for categorical features, 'LinearEncoder' for numerical features, and 'TabTransformerConv' layers for processing, culminating in a linear decoder for output. ```python from typing import Any import torch_frame from torch_frame import TensorFrame, stype from torch_frame.data.stats import StatType from torch_frame.nn.conv import TabTransformerConv from torch_frame.nn.encoder import ( EmbeddingEncoder, LinearEncoder, StypeWiseFeatureEncoder, ) class ExampleTransformer(torch.nn.Module): def __init__( self, channels: int, out_channels: int, num_layers: int, num_heads: int, col_stats: dict[str, dict[StatType, Any]], col_names_dict: dict[torch_frame.stype, list[str]], ): super().__init__() self.encoder = StypeWiseFeatureEncoder( out_channels=channels, col_stats=col_stats, col_names_dict=col_names_dict, stype_encoder_dict={ stype.categorical: EmbeddingEncoder(), stype.numerical: LinearEncoder() }, ) self.tab_transformer_convs = torch.nn.ModuleList([ TabTransformerConv( channels=channels, num_heads=num_heads, ) for _ in range(num_layers) ]) self.decoder = torch.nn.Linear(channels, out_channels) def forward(self, tf: TensorFrame) -> torch.Tensor: x, _ = self.encoder(tf) for tab_transformer_conv in self.tab_transformer_convs: x = tab_transformer_conv(x) return self.decoder(x.mean(dim=1)) ``` -------------------------------- ### PyTorch Frame API: Semantic Types (stype) and TensorFrame Structure Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/get_started/introduction.rst This API documentation outlines the core data handling concepts in PyTorch Frame. It details the supported semantic types (stype.categorical, stype.numerical, stype.multicategorical, stype.text_embedded) and describes the structure of the torch_frame.data.TensorFrame object, which serves as the primary representation for tabular data. The TensorFrame includes col_names_dict for column organization by stype and feat_dict for storing feature tensors. ```APIDOC Class: torch_frame.stype Description: Represents the semantic type of a column in PyTorch Frame. Supported Types: - stype.categorical: Denotes categorical columns. - stype.numerical: Denotes numerical columns. - stype.multicategorical: Denotes multi-categorical columns. - stype.text_embedded: Denotes text columns that are pre-embedded. Class: torch_frame.data.TensorFrame Description: A table representation in PyTorch Frame, holding attributes for column names and feature tensors. Attributes: - col_names_dict (dict): A dictionary holding column names for each stype. - feat_dict (dict): A dictionary holding torch.Tensor of different stypes. ``` -------------------------------- ### Interpret PyTorch Frame Encoder Benchmark Latency Output Source: https://github.com/pyg-team/pytorch-frame/blob/master/benchmark/encoder/README.md This snippet shows an example of the latency output produced by the benchmark script. The latency value represents the single-run execution time of the encoder, providing a direct measure of performance. ```text Latency: 0.034277s ``` -------------------------------- ### Apply CatToNumTransform in PyTorch Frame Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/modules/transforms.rst This Python snippet demonstrates how to use `CatToNumTransform` to convert categorical features into numerical features within a `TensorFrame`. It illustrates the full transformation workflow, from fitting the transform on training data to applying it on test data, showcasing the resulting changes in column names and data types. This example requires the `torch_frame` library and the `Yandex` dataset. ```python from torch_frame.datasets import Yandex from torch_frame.transforms import CatToNumTransform from torch_frame import stype dataset = Yandex(root='/tmp/adult', name='adult') dataset.materialize() transform = CatToNumTransform() train_dataset = dataset.get_split('train') train_dataset.tensor_frame.col_names_dict[stype.categorical] >>> ['C_feature_0', 'C_feature_1', 'C_feature_2', 'C_feature_3', 'C_feature_4', 'C_feature_5', 'C_feature_6', 'C_feature_7'] test_dataset = dataset.get_split('test') transform.fit(train_dataset.tensor_frame, dataset.col_stats) transformed_col_stats = transform.transformed_stats transformed_col_stats.keys() >>> dict_keys(['C_feature_0_0', 'C_feature_1_0', 'C_feature_2_0', 'C_feature_3_0', 'C_feature_4_0', 'C_feature_5_0', 'C_feature_6_0', 'C_feature_7_0']) transformed_col_stats['C_feature_0_0'] >>> {: 0.6984029484029484, : 0.45895127199411595, : [0.0, 0.0, 1.0, 1.0, 1.0]} transform(test_dataset.tensor_frame) >>> TensorFrame( num_cols=14, num_rows=16281, numerical (14): ['N_feature_0', 'N_feature_1', 'N_feature_2', 'N_feature_3', 'N_feature_4', 'N_feature_5', 'C_feature_0_0', 'C_feature_1_0', 'C_feature_2_0', 'C_feature_3_0', 'C_feature_4_0', 'C_feature_5_0', 'C_feature_6_0', 'C_feature_7_0'], has_target=True, device=cpu, ) ``` -------------------------------- ### Handle Multicategorical Columns with Custom Delimiters in PyTorch Frame Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/handling_advanced_stypes/handle_heterogeneous_stypes.rst This example illustrates how to process pandas DataFrames containing multiple multicategorical columns, each potentially using a different delimiter. It shows the creation of a DataFrame with mixed multicategorical data and how to initialize a `torch_frame.data.Dataset` by specifying `stype.multicategorical` and custom delimiters via the `col_to_sep` argument. The output demonstrates the `col_stats` after dataset creation, confirming proper parsing. ```python categories = ['Category A', 'Category B', 'Category C', 'Category D'] multicategorical1 = [ random.sample(categories, k=random.randint(0, len(categories))) for _ in range(100) ] multicategorical2 = [ ','.join(random.sample(categories, k=random.randint(0, len(categories)))) for _ in range(100) ] multicategorical3 = [ '/'.join(random.sample(categories, k=random.randint(0, len(categories)))) for _ in range(100) ] # Create the DataFrame df = pd.DataFrame({ 'Multicategorical1': multicategorical1, 'Multicategorical2': multicategorical2, 'Multicategorical3': multicategorical3, }) dataset = Dataset( df, col_to_stype={ 'Multicategorical1': stype.multicategorical, 'Multicategorical2': stype.multicategorical, 'Multicategorical3': stype.multicategorical, }, col_to_sep={'Multicategorical2': ',', 'Multicategorical3': '/'}) dataset.col_stats # Expected output: # {'Multicategorical1': {: # (['Category B', 'Category D', 'Category A', 'Category C'], [61, 60, 56, 49])}, # 'Multicategorical2': {: # (['Category D', 'Category A', 'Category B', 'Category C'], [53, 52, 51, 46])}, # 'Multicategorical3': {: # (['Category D', 'Category B', 'Category C', 'Category A'], [52, 52, 51, 46])}} ``` -------------------------------- ### Specify Column-Specific Text Embedder Configurations in PyTorch Frame Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/handling_advanced_stypes/handle_text.rst This example illustrates how to provide distinct `TextEmbedderConfig` settings for individual text columns using a dictionary. It allows for flexible control, enabling different text embedders or batch sizes (e.g., 4 for 'text_col0', 8 for 'text_col1') based on column-specific requirements. This approach is beneficial when different text columns have varying characteristics or resource demands. ```python # Prepare text_embedder0 and text_embedder1 for text_col0 and text_col1, respectively. col_to_text_embedder_cfg = { "text_col0": TextEmbedderConfig(text_embedder=text_embedder0, batch_size=4), "text_col1": TextEmbedderConfig(text_embedder=text_embedder1, batch_size=8), } ``` -------------------------------- ### PyTorch Frame Encoder Benchmark Script Command-Line Arguments Source: https://github.com/pyg-team/pytorch-frame/blob/master/benchmark/encoder/README.md This section details the available command-line arguments for the `encoder_benchmark.py` script. Each argument controls specific aspects of the benchmark, such as dataset size, output channels, NaN inclusion, number of runs, warmup stages, and profiling options. ```APIDOC [ { "name": "--stype-kv", "type": "string (key-value pairs)", "description": "Specify the stype(s) and corresponding encoder(s) to run. Example: `categorical embedding numerical linear`." }, { "name": "--num-rows", "type": "integer", "default": 8192, "description": "The number of rows in the dataset." }, { "name": "--out-channels", "type": "integer", "default": 128, "description": "The number of output channels." }, { "name": "--with-nan", "type": "boolean (flag)", "description": "If specified, the dataset will include NaN values." }, { "name": "--runs", "type": "integer", "default": 1000, "description": "The number of runs for the benchmark." }, { "name": "--warmup-size", "type": "integer", "default": 200, "description": "The size of the warmup stage." }, { "name": "--torch-profile", "type": "boolean (flag)", "description": "If specified, torch profiling will be enabled." }, { "name": "--line-profile", "type": "boolean (flag)", "description": "If specified, line profiling will be enabled." }, { "name": "--line-profile-level", "type": "string", "default": "'encode_forward'", "description": "The level of line profiling." }, { "name": "--device", "type": "string", "default": "'cpu'", "description": "The device to run the benchmark on." } ] ``` -------------------------------- ### Prepare PyTorch Frame Tabular Data with DataLoader in Python Source: https://github.com/pyg-team/pytorch-frame/blob/master/README.md This snippet demonstrates how to prepare tabular data for training using PyTorch Frame. It shows how to instantiate a pre-defined dataset like `Yandex`, materialize it, and create a PyTorch-compatible `DataLoader` for efficient batch processing. The `DataLoader` handles batching and shuffling of `TensorFrame` objects. ```python from torch_frame.datasets import Yandex from torch_frame.data import DataLoader dataset = Yandex(root='/tmp/adult', name='adult') dataset.materialize() train_dataset = dataset[:0.8] train_loader = DataLoader(train_dataset.tensor_frame, batch_size=128, shuffle=True) ``` -------------------------------- ### API Documentation for PyTorch Frame GBDT Module Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/modules/gbdt.rst This snippet provides a structured API reference for the `torch_frame.gbdt` module. It outlines the module's purpose and indicates the types of components, such as classes, that it contains for Gradient Boosted Decision Tree implementations. ```APIDOC { "module": "torch_frame.gbdt", "summary": "Gradient Boosted Decision Trees module for PyTorch Frame.", "description": "The `torch_frame.gbdt` module provides core functionalities and implementations related to Gradient Boosted Decision Trees (GBDT) within the PyTorch Frame ecosystem. It is designed to integrate GBDT models seamlessly with tabular data processing in PyTorch. This module encapsulates various GBDT-related classes and utilities.", "contents": { "classes": [ "List of GBDT-related classes (e.g., specific GBDT model implementations, tree structures, or utility classes). Detailed documentation for each class, including methods, properties, and parameters, would be nested here." ] } } ``` -------------------------------- ### Load MultimodalTextBenchmark Dataset with Text Embedder Configuration Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/handling_advanced_stypes/handle_text.rst This snippet demonstrates the final step of loading a `MultimodalTextBenchmark` dataset, specifically 'wine_reviews', in PyTorch Frame. The `col_to_text_embedder_cfg` (defined previously) is implicitly used by the `Dataset` object to handle text columns according to the specified pre-encoding or tokenization strategies. This prepares the dataset for subsequent model training. ```python >>> import torch_frame >>> from torch_frame.datasets import MultimodalTextBenchmark >>> dataset = MultimodalTextBenchmark( ... root='/tmp/multimodal_text_benchmark/wine_reviews', ... name='wine_reviews', ``` -------------------------------- ### Run PyTorch Frame Encoder Benchmark with Specific Encoders Source: https://github.com/pyg-team/pytorch-frame/blob/master/benchmark/encoder/README.md This command executes the `encoder_benchmark.py` script, configuring it to use 'embedding' encoder for categorical columns and 'linear' encoder for numerical columns. This allows benchmarking specific encoder combinations on a generated dataset. ```bash python encoder_benchmark.py --stype-kv categorical embedding --stype-kv numerical linear ``` -------------------------------- ### Define a PyTorch Frame Tabular Transformer Model in Python Source: https://github.com/pyg-team/pytorch-frame/blob/master/README.md This snippet defines `ExampleTransformer`, a custom deep tabular model built with PyTorch Frame's modular architecture. It utilizes `StypeWiseFeatureEncoder` for input mapping, `TabTransformerConv` for iterative transformations, and a `Linear` decoder for final output. The model processes `TensorFrame` inputs and outputs a `Tensor`. ```python from torch import Tensor from torch.nn import Linear, Module, ModuleList from torch_frame import TensorFrame, stype from torch_frame.nn.conv import TabTransformerConv from torch_frame.nn.encoder import ( EmbeddingEncoder, LinearEncoder, StypeWiseFeatureEncoder, ) class ExampleTransformer(Module): def __init__( self, channels, out_channels, num_layers, num_heads, col_stats, col_names_dict, ): super().__init__() self.encoder = StypeWiseFeatureEncoder( out_channels=channels, col_stats=col_stats, col_names_dict=col_names_dict, stype_encoder_dict={ stype.categorical: EmbeddingEncoder(), stype.numerical: LinearEncoder() }, ) self.convs = ModuleList([ TabTransformerConv( channels=channels, num_heads=num_heads, ) for _ in range(num_layers) ]) self.decoder = Linear(channels, out_channels) def forward(self, tf: TensorFrame) -> Tensor: x, _ = self.encoder(tf) for conv in self.convs: x = conv(x) out = self.decoder(x.mean(dim=1)) return out ``` -------------------------------- ### Initialize PyTorch Frame Dataset with Text Tokenizer Configuration Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/handling_advanced_stypes/handle_text.rst This snippet demonstrates how to initialize a PyTorch Frame Dataset object, specifically configuring it to handle text columns using predefined text tokenizers. It shows how to pass a `col_to_text_tokenizer_cfg` dictionary to the Dataset constructor, specifying the `text_stype` for relevant columns. The output confirms the `stype` of the 'description' column. ```python import torch_frame from torch_frame.datasets import MultimodalTextBenchmark # Prepare text_tokenizer0 and text_tokenizer1 for text_col0 and text_col1, respectively. col_to_text_tokenizer_cfg = { "text_col0": TextTokenizerConfig(text_tokenizer=text_tokenizer0, batch_size=10_000), "text_col1": TextTokenizerConfig(text_tokenizer=text_tokenizer1, batch_size=20_000), } dataset = MultimodalTextBenchmark( root='/tmp/multimodal_text_benchmark/wine_reviews', name='wine_reviews', text_stype=torch_frame.text_tokenized, col_to_text_tokenizer_cfg=col_to_text_tokenizer_cfg, ) dataset.col_to_stype['description'] ``` -------------------------------- ### Benchmark Text Encoder Performance on Wine Reviews Dataset Source: https://github.com/pyg-team/pytorch-frame/blob/master/README.md This table displays the test accuracy of different pre-trained text encoders on the Wine Reviews dataset, which includes a text column. It compares various models from sources like Hugging Face, Cohere, OpenAI, and Voyage AI, showcasing their performance in a multimodal tabular context. ```APIDOC | Test Acc | Method | Model Name | Source | | :--------- | :------------ | :--------------------------------------------------------- | :----------- | | 0.7926 | Pre-trained | sentence-transformers/all-distilroberta-v1 (125M # params) | Hugging Face | | 0.7998 | Pre-trained | embed-english-v3.0 (dimension size: 1024) | Cohere | | 0.8102 | Pre-trained | text-embedding-ada-002 (dimension size: 1536) | OpenAI | | 0.8147 | Pre-trained | voyage-01 (dimension size: 1024) | Voyage AI | | 0.8203 | Pre-trained | intfloat/e5-mistral-7b-instruct (7B # params) | Hugging Face | ``` -------------------------------- ### torch_frame.data Module API Reference Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/modules/data.rst This API reference outlines the structure and main components of the `torch_frame.data` module, including its categories of data objects, statistical classes, data loaders, and general utility functions. It serves as a high-level overview of the module's capabilities for tabular data processing. ```APIDOC { "module": "torch_frame.data", "description": "The `torch_frame.data` module provides fundamental data structures, statistical tools, data loaders, and helper functions essential for managing and processing tabular data within the PyTorch Frame framework.", "sections": [ { "name": "Data Objects", "description": "Contains core classes for representing and manipulating tabular data, such as `Frame` objects. Specific classes are dynamically listed from `torch_frame.data.data_classes`.", "items_placeholder": "See `torch_frame.data.data_classes` for specific class definitions." }, { "name": "Stats", "description": "Includes classes designed for computing and storing various statistics of tabular datasets. Specific classes are dynamically listed from `torch_frame.data.stats_classes`.", "items_placeholder": "See `torch_frame.data.stats_classes` for specific class definitions." }, { "name": "Data Loaders", "description": "Provides classes for efficiently loading and batching tabular data for training and inference. Specific classes are dynamically listed from `torch_frame.data.loader_classes`.", "items_placeholder": "See `torch_frame.data.loader_classes` for specific class definitions." }, { "name": "Helper Functions", "description": "Offers utility functions that support various data-related operations within the module. Specific functions are dynamically listed from `torch_frame.data.helper_functions`.", "items_placeholder": "See `torch_frame.data.helper_functions` for specific function definitions." } ] } ``` -------------------------------- ### PyTorch Frame Datasets Module API Structure Source: https://github.com/pyg-team/pytorch-frame/blob/master/docs/source/modules/datasets.rst This documentation snippet outlines the API structure of the `torch_frame.datasets` module. It details how different categories of datasets (real-world, synthetic, and other) are organized and made accessible within the PyTorch Frame library, providing a high-level overview of the module's contents. ```APIDOC { "module": "torch_frame.datasets", "description": "The `torch_frame.datasets` module provides a comprehensive collection of datasets for use with the PyTorch Frame library. It categorizes datasets into real-world, synthetic, and other types, facilitating easy access and integration for tabular data tasks.", "contents": [ { "section_title": "Real-World Datasets", "description": "This section lists various real-world datasets available in `torch_frame`. These datasets are typically derived from actual data sources and are suitable for benchmarking and practical applications.", "classes_listed": "Dynamically populated from `torch_frame.datasets.real_world_datasets`" }, { "section_title": "Synthetic Datasets", "description": "This section includes synthetically generated datasets. These datasets are useful for testing algorithms under controlled conditions or for scenarios where real-world data is scarce.", "classes_listed": "Dynamically populated from `torch_frame.datasets.synthetic_datasets`" }, { "section_title": "Other Datasets", "description": "This section contains additional datasets that do not strictly fall into the 'real-world' or 'synthetic' categories. It serves as a general collection for diverse tabular data examples.", "classes_listed": "Dynamically populated from `torch_frame.datasets.other_datasets`" } ] } ``` -------------------------------- ### Examine PyTorch Frame Encoder Benchmark Line Profiler Output Source: https://github.com/pyg-team/pytorch-frame/blob/master/benchmark/encoder/README.md This snippet illustrates the output from the line profiler, which details the time spent on each line of a specified method. This granular view helps pinpoint exact lines of code contributing most to execution time, facilitating targeted optimization. ```python Total time: 1.03661 s File: {PF_BASE_PATH}/pytorch-frame/torch_frame/nn/encoder/stype_encoder.py Function: encode_forward at line 295 Line # Hits Time Per Hit % Time Line Contents ============================================================== 295 def encode_forward( 296 self, 297 feat: Tensor, 298 col_names: list[str] | None = None, 299 ) -> Tensor: 300 # TODO: Make this more efficient. 301 # Increment the index by one so that NaN index (-1) becomes 0 302 # (padding_idx) 303 # feat: [batch_size, num_cols] 304 1200 29867.4 24.9 2.9 feat = feat + 1 305 1200 944.3 0.8 0.1 xs = [] 306 3600 19345.7 5.4 1.9 for i, emb in enumerate(self.embs): 307 2400 466967.7 194.6 45.0 xs.append(emb(feat[:, i])) 308 # [batch_size, num_cols, hidden_channels] 309 1200 519044.9 432.5 50.1 x = torch.stack(xs, dim=1) 310 1200 440.8 0.4 0.0 return x ```