### Quick Start PEFT with TabTune

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/examples/peft-examples.md

A minimal example demonstrating how to initialize and train a TabularPipeline using the PEFT tuning strategy.

```python
from tabtune import TabularPipeline
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

# Load dataset
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Create PEFT pipeline
pipeline = TabularPipeline(
    model_name='TabICL',
    tuning_strategy='peft',
    tuning_params={
        'device': 'cuda',
        'epochs': 5,
        'learning_rate': 2e-4,
        'peft_config': {
            'r': 8,
            'lora_alpha': 16,
            'lora_dropout': 0.05
        }
    }
)

# Train (much faster and lighter than base-ft)
pipeline.fit(X_train, y_train)

# Evaluate
metrics = pipeline.evaluate(X_test, y_test)
print(f"Accuracy: {metrics['accuracy']:.4f}")
print(f"Model size: 1-2% of full model")
```

--------------------------------

### Clone Repository and Setup Virtual Environment

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/contributing/setup.md

Clone the TabTune repository, navigate to the directory, create a Python virtual environment, activate it, and install project dependencies.

```bash
git clone <your-fork>
cd TabTune_Internal
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -r requirements-mkdocs.txt
pip install -e .[dev]
```

--------------------------------

### Example TabularPipeline Usage

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/pipeline-overview.md

Demonstrates the setup, training, and evaluation of a TabularPipeline with specified model, strategy, and parameters. Requires importing TabularPipeline from the tabtune library.

```python
from tabtune import TabularPipeline

# Setup pipeline
pipeline = TabularPipeline(
    model_name='TabICL',
    tuning_strategy='peft',
    tuning_params={'device':'cuda','epochs':5,'learning_rate':2e-4},
    processor_params={'imputation_strategy':'median'},
    model_params={'n_estimators':16}
)

# Train
pipeline.fit(X_train, y_train)

# Predict & evaluate
metrics = pipeline.evaluate(X_test, y_test)
print(metrics)
```

--------------------------------

### Install TabTune

Source: https://github.com/lexsi-labs/tabtune/blob/main/README.md

Clone the repository, navigate to the directory, and install dependencies including the package itself.

```bash
git clone https://github.com/Lexsi-Labs/TabTune.git
cd TabTune
pip install -r requirements.txt
pip install -e .
```

--------------------------------

### Setup and Train Regression Pipeline

Source: https://github.com/lexsi-labs/tabtune/blob/main/README.md

Demonstrates setting up and training a TabTune pipeline for regression tasks using the California Housing dataset. Ensure TabTune and scikit-learn are installed.

```python
from tabtune import TabularPipeline
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split

X, y = fetch_california_housing(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

pipeline = TabularPipeline(
    model_name="OrionMSP",
    task_type="regression",
    tuning_strategy="inference",
    tuning_params={
        "epochs": 5,
        "learning_rate": 2e-5
    }
)

pipeline.fit(X_train, y_train)
metrics = pipeline.evaluate(X_test, y_test)

print(metrics)
```

--------------------------------

### Basic Workflow Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/tabicl.md

A complete end-to-end example of loading data, splitting, training, and evaluating.

```python
from tabtune import TabularPipeline
from sklearn.model_selection import train_test_split
import pandas as pd

# Load data
df = pd.read_csv('data.csv')
X = df.drop('target', axis=1)
y = df['target']

# Split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Create and train
pipeline = TabularPipeline(
    model_name='TabICL',
    tuning_strategy='base-ft',
    tuning_params={
        'device': 'cuda',
        'epochs': 5,
        'learning_rate': 2e-5,
        'n_episodes': 1000
    }
)

pipeline.fit(X_train, y_train)
metrics = pipeline.evaluate(X_test, y_test)

print(f"Accuracy: {metrics['accuracy']:.4f}")
print(f"F1 Score: {metrics['f1_score']:.4f}")
```

--------------------------------

### Quick Start: Tabular Pipeline Usage

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/index.md

Demonstrates loading a dataset, initializing, fitting, saving, loading, and evaluating a TabularPipeline. Ensure necessary libraries like pandas, scikit-learn, and openml are installed.

```python
import pandas as pd
from sklearn.model_selection import train_test_split
import openml
from tabtune import TabularPipeline

# Load dataset
dataset = openml.datasets.get_dataset(42178)
X, y, _, _ = dataset.get_data(target=dataset.default_target_attribute)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# Init and fit pipeline
pipeline = TabularPipeline(
    model_name="TabPFN",
    task_type="classification",
    tuning_strategy="base-ft",
    tuning_params={"device": "cpu"}
)
pipeline.fit(X_train, y_train)

# Save and load pipeline for prediction
pipeline.save("churn_pipeline.joblib")
loaded_pipeline = TabularPipeline.load("churn_pipeline.joblib")
predictions = loaded_pipeline.predict(X_test)
metrics = pipeline.evaluate(X_test, y_test)
print(metrics)
```

--------------------------------

### Run Unified API Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/examples/README.md

Navigates to the TabTune internal directory and executes the unified API example script.

```bash
cd TabTune_Internal
python examples/01_unified_api.py
```

--------------------------------

### Full Comparison Workflow Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/leaderboard.md

A comprehensive example demonstrating the complete workflow from data loading and splitting to adding various models with different strategies and running benchmarks.

```python
from tabtune import TabularLeaderboard
import pandas as pd
from sklearn.model_selection import train_test_split

# 1. Load data
df = pd.read_csv('dataset.csv')
X = df.drop('target', axis=1)
y = df['target']

# 2. Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# 3. Initialize leaderboard
leaderboard = TabularLeaderboard(X_train, X_test, y_train, y_test)

# 4. Add models - Inference Baseline
leaderboard.add_model('TabPFN', 'inference', name='TabPFN-Inference')

# 5. Add models - PEFT Strategies
for model in ['TabICL', 'OrionMSP', 'OrionBix', 'Mitra']:
    leaderboard.add_model(
        model,
        'peft',
        name=f'{model}-PEFT',
        tuning_params={
            'epochs': 3,
            'peft_config': {'r': 8, 'lora_alpha': 16}
        }
    )

# 6. Add models - Base Fine-Tuning (for comparison)
leaderboard.add_model(
    'TabICL',
    'base-ft',
    name='TabICL-BaseFT',
    tuning_params={'epochs': 5, 'learning_rate': 2e-5}
)

# 7. Run benchmarks
results = leaderboard.run(rank_by='accuracy', verbose=True)
```

--------------------------------

### Quick Start Regression with TabPFN

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/examples/regression.md

A 5-minute example demonstrating how to load a dataset, create, train, and predict using TabularPipeline with the TabPFN model for regression. Requires scikit-learn and tabtune.

```python
from tabtune import TabularPipeline
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split

# Load dataset
X, y = fetch_california_housing(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Create and train
pipeline = TabularPipeline(
    model_name='TabPFN',
    task_type='regression',
    tuning_strategy='inference'
)

pipeline.fit(X_train, y_train)

# Predict
predictions = pipeline.predict(X_test)
metrics = pipeline.evaluate(X_test, y_test)
```

--------------------------------

### Install Required Software Packages

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/advanced/multi-gpu.md

Install PyTorch and optional distributed training dependencies.

```bash
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install torch-distributed-rpc
pip install horovod  # Optional: for advanced distributed training
pip install pytorch-lightning  # Optional: simplified multi-GPU setup
```

--------------------------------

### Install TabTune from source

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/troubleshooting.md

Use this command to install the package in editable mode from the local directory.

```bash
cd TabTune
pip install -e .
```

--------------------------------

### Fine-Tuning Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/tabpfn.md

Example of how to fine-tune a TabPFN model using the TabTune library.

```APIDOC
## Fine-Tuning Example

```python
from tabtune import TabularPipeline

pipeline = TabularPipeline(
    model_name='TabPFN',
    tuning_strategy='base-ft',
    tuning_params={
        'device': 'cuda',
        'epochs': 5,
        'learning_rate': 2e-5,
        'batch_size': 256,
        'scheduler': 'cosine',
        'show_progress': True
    }
)

# Fine-tune on your data
pipeline.fit(X_train, y_train)

# Evaluate
metrics = pipeline.evaluate(X_test, y_test)
print(f"Accuracy: {metrics['accuracy']:.4f}")
```
```

--------------------------------

### Run All TabTune Examples Sequentially

Source: https://github.com/lexsi-labs/tabtune/blob/main/examples/README.md

This bash script iterates through all example files from 1 to 9 and executes them sequentially using Python. Ensure you are in the correct directory before running.

```bash
for i in {1..9}; do
    python example/0${i}_*.py
done
```

--------------------------------

### Example YAML Configuration

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/saving-loading.md

A sample structure for a saved pipeline configuration file.

```yaml
model_name: TabICL
task_type: classification
tuning_strategy: peft
tuning_params:
  device: cuda
  epochs: 5
  learning_rate: 2e-4
  peft_config:
    r: 8
    lora_alpha: 16
    lora_dropout: 0.05
processor_params:
  imputation_strategy: median
  categorical_encoding: onehot
  scaling_strategy: standard
model_params:
  n_estimators: 16
```

--------------------------------

### Verify FlashAttention-3 Installation

Source: https://github.com/lexsi-labs/tabtune/blob/main/tabtune/models/tabpfnv3/architectures/shared/fa3_setup.md

Import the flash_attn_func from flash_attn_interface to verify that the FlashAttention-3 backend has been successfully installed.

```python
from flash_attn_interface import flash_attn_func  # noqa: F401
```

--------------------------------

### Quick Start: 5-Minute Classification with TabPFN

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/examples/classification.md

Minimal code to load data, train a TabPFN model with inference tuning, and get predictions and accuracy.

```python
from tabtune import TabularPipeline
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

# Load dataset
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Create and train
pipeline = TabularPipeline(model_name='TabPFN', tuning_strategy='inference')
pipeline.fit(X_train, y_train)

# Predict
predictions = pipeline.predict(X_test)
metrics = pipeline.evaluate(X_test, y_test)

print(f"Accuracy: {metrics['accuracy']:.4f}")
```

--------------------------------

### Install PyTorch dependencies

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/troubleshooting.md

Install PyTorch for CPU or GPU support using the official index URL.

```bash
# For CPU only
pip install torch

# For GPU support (CUDA 11.8)
pip install torch --index-url https://download.pytorch.org/whl/cu118
```

--------------------------------

### Install Package with User Flag on Windows

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/getting-started/installation.md

If encountering permission denied errors on Windows during installation, try running the pip install command with the `--user` flag to install packages in the user's home directory.

```bash
# Solution: Run as administrator or use --user flag
pip install --user -e .
```

--------------------------------

### Install TabTune

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/saving-loading.md

Command to install the TabTune package via pip.

```bash
pip install tabtune
```

--------------------------------

### Install and Run Tests via CLI

Source: https://github.com/lexsi-labs/tabtune/blob/main/tests/README.md

Commands for installing development dependencies and executing test suites with different configurations.

```bash
# Install test dependencies
pip install -e ".[dev]"

# Run all tests
pytest

# Run fast tests only (skip slow tests)
pytest -m "not slow"

# Run with verbose output
pytest -v

# Run specific test file
pytest tests/test_tabular_pipeline.py

# Run fine-tuning tests only
pytest tests/test_finetuning.py -v

# Run fine-tuning tests for specific model
pytest tests/test_finetuning.py -k "TabICL" -v
```

--------------------------------

### Writing Unit Tests

Source: https://github.com/lexsi-labs/tabtune/blob/main/tests/README.md

Example of a fast unit test for component initialization.

```python
def test_pipeline_initialization():
    pipeline = TabularPipeline(model_name='TabPFN', tuning_strategy='inference')
    assert pipeline.model_name == 'TabPFN'
```

--------------------------------

### Set Up and Activate Virtual Environment

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/about/faq.md

Create and activate a Python virtual environment for TabTune. Install dependencies and the package in development mode.

```bash
python -m venv tabtune-env
source tabtune-env/bin/activate  # Linux/macOS
# tabtune-env\Scripts\activate   # Windows
pip install -r requirements.txt
pip install -e .
```

--------------------------------

### Install TabTune with GPU Support

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/about/faq.md

Commands to install PyTorch with CUDA support followed by the TabTune package.

```bash
# Install PyTorch with CUDA (check your CUDA version first with nvidia-smi)
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118

# Then install TabTune
pip install -r requirements.txt
pip install -e .
```

--------------------------------

### Clone TabTune Repository and Install Dependencies

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/getting-started/installation.md

Clone the TabTune repository from GitHub and install the required dependencies using pip. Navigate into the cloned directory before installing the package in editable mode.

```bash
git clone https://github.com/Lexsi-Labs/TabTune.git
pip install -r requirements.txt
cd TabTune
pip install -e .
```

--------------------------------

### Quick Baseline Model Setup

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/tabpfn.md

Establish a quick baseline performance using the TabPFN model with inference tuning strategy. Requires training data for fitting.

```python
from tabtune import TabularPipeline

# Establish baseline in seconds
pipeline = TabularPipeline(
    model_name='TabPFN',
    tuning_strategy='inference'
)
pipeline.fit(X_train, y_train)
baseline_score = pipeline.evaluate(X_test, y_test)
print(f"Baseline accuracy: {baseline_score['accuracy']:.4f}")
```

--------------------------------

### Install Package Dependencies

Source: https://github.com/lexsi-labs/tabtune/blob/main/tests/README.md

Commands to install the package in editable mode, including development dependencies.

```bash
pip install -e .

pip install -e ".[dev]"
```

--------------------------------

### Install Core ML and Data Handling Libraries

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/getting-started/installation.md

These are the essential packages automatically installed with TabTune for machine learning, data manipulation, and performance.

```bash
# Core ML libraries
torch>=2.0.0
numpy>=1.21.0
pandas>=1.3.0
scikit-learn>=1.0.0

# Data handling
openml>=0.12.0
datasets>=2.0.0

# PEFT support
peft>=0.4.0
accelerate>=0.20.0
transformers>=4.30.0

# Utilities
joblib>=1.0.0
tqdm>=4.60.0
```

--------------------------------

### Save Pipeline with Joblib

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/contexttab.md

Example of training and preparing a model for production deployment.

```python
import joblib
import os

os.environ['HF_TOKEN'] = 'your_token'

# Train
pipeline = TabularPipeline(
    model_name='ContextTab',
    tuning_strategy='base-ft',
    tuning_params={
        'epochs': 10,
        'learning_rate': 1e-4
    }
)

pipeline.fit(X_train, y_train)
metrics = pipeline.evaluate(X_test, y_test)
```

--------------------------------

### Launch Training with torchrun

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/advanced/multi-gpu.md

Use torchrun to manage distributed training processes for single or multi-node setups.

```bash
# Single-machine, 4 GPUs
torchrun --nproc_per_node=4 train_script.py

# Multi-machine (8 GPUs total)
torchrun \
  --nproc_per_node=4 \
  --nnodes=2 \
  --node_rank=0 \
  --master_addr=192.168.1.100 \
  --master_port=29500 \
  train_script.py
```

--------------------------------

### TabularPipeline Tuning Parameters Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/api/pipeline.md

Example of tuning parameters for TabularPipeline, including device, epochs, learning rate, and batch size. Supports PEFT configuration for parameter-efficient fine-tuning.

```python
tuning_params={
    "device": "cuda",
    "epochs": 5,
    "learning_rate": 2e-5,
    "batch_size": 8
}
```

--------------------------------

### Install TabTune in Development Mode

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/about/faq.md

Install TabTune using pip in development mode. Ensure you are in the TabTune_Internal directory.

```bash
cd TabTune_Internal
pip install -e .
```

--------------------------------

### TabularPipeline Model Parameters Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/api/pipeline.md

Example of model-specific parameters for TabularPipeline, such as n_estimators and softmax_temperature for the TabICL model.

```python
model_params={"n_estimators": 16, "softmax_temperature": 0.9}
```

--------------------------------

### Quick TabTune Installation Verification

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/getting-started/installation.md

Verify the TabTune installation by checking PyTorch version, CUDA availability, and attempting to import the TabularPipeline class.

```python
import torch
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA available: {torch.cuda.is_available()}")

# Test TabTune import
from tabtune import TabularPipeline
print("✅ TabTune successfully installed!")
```

--------------------------------

### Install FlashAttention-3 Wheel on Runtime Node

Source: https://github.com/lexsi-labs/tabtune/blob/main/tabtune/models/tabpfnv3/architectures/shared/fa3_setup.md

Install the previously built FlashAttention-3 wheel file on the target H100 node. This allows using the FA3 backend on the runtime environment.

```bash
pip install /tmp/fa3-wheel/flash_attn_3-*.whl
python -c "from flash_attn_interface import flash_attn_func; print('ok')"
```

--------------------------------

### Memory-Constrained Scenario Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/advanced/peft-lora.md

An example demonstrating how to check available GPU memory using PyTorch before proceeding with model training, useful for memory-constrained environments.

```python
from tabtune import TabularPipeline
import torch

# Check available GPU memory
print(f"Available GPU memory: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB")
```

--------------------------------

### TabularPipeline Example Usage

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/pipeline-overview.md

Demonstrates how to initialize, train, and evaluate a model using the TabularPipeline.

```APIDOC
## Example Usage

### Description
This example shows a typical workflow for setting up and using the `TabularPipeline`.

### Code
```python
from tabtune import TabularPipeline

# Setup pipeline
pipeline = TabularPipeline(
    model_name='TabICL',
    tuning_strategy='peft',
    tuning_params={'device':'cuda','epochs':5,'learning_rate':2e-4},
    processor_params={'imputation_strategy':'median'},
    model_params={'n_estimators':16}
)

# Train
pipeline.fit(X_train, y_train)

# Predict & evaluate
metrics = pipeline.evaluate(X_test, y_test)
print(metrics)
```
```

--------------------------------

### Hybrid Fine-Tuning with PEFT and Meta-Learning

Source: https://github.com/lexsi-labs/tabtune/blob/main/README.md

This example demonstrates initializing TabularPipeline for hybrid fine-tuning, combining meta-learning with PEFT parameters such as LoRA configuration, epochs, and learning rate.

```python
pipeline = TabularPipeline(
    model_name="TabICL",
    tuning_strategy="peft",
    tuning_params={
        'epochs': 20,
        'learning_rate': 1e-5,
        'finetune_mode': 'meta-learning',
        'peft_config': {
            'r': 16,
            'lora_alpha': 32,
            'lora_dropout': 0.1
        }
    }
)
```

--------------------------------

### Initialize PEFT Pipeline for Large Models

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/tabdpt.md

Setup for TabDPT using PEFT with specific rank adjustments.

```python
# PEFT works well with TabDPT's large architecture
pipeline = TabularPipeline(
    model_name='TabDPT',
    tuning_strategy='peft',
    tuning_params={
        'device': 'cuda',
        'epochs': 3,
        'learning_rate': 2e-4,
        'support_size': 512,  # Still large
        'peft_config': {'r': 16}  # Higher rank acceptable
    }
)
```

--------------------------------

### Automated Preprocessing Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/examples/README.md

This example highlights TabTune's DataProcessor for automatic model-specific preprocessing, adapting seamlessly for models like TabPFN, ContextTab, and ICL. It uses the Pima Indians Diabetes dataset.

```python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.datasets import fetch_openml

from tabtune import TabTune, DataProcessor

# Load dataset
X, y = fetch_openml(name='diabetes', version=1, return_X_y=True, as_frame=True)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize DataProcessor
processor = DataProcessor()

# Process data (automatically adapts to model requirements)
# For demonstration, we'll use a simple model that might benefit from preprocessing
# In a real scenario, you'd pass the model to TabTune and let it handle it

X_train_processed = processor.fit_transform(X_train, y_train)
X_test_processed = processor.transform(X_test)

print("Data processing complete.")
print(f"Processed training data shape: {X_train_processed.shape}")
print(f"Processed test data shape: {X_test_processed.shape}")

# You can then use these processed datasets with TabTune models
# tabtune = TabTune(model='TabPFN', processor=processor, random_state=42)
# tabtune.fit(X_train, y_train) # TabTune will use the fitted processor internally

```

--------------------------------

### Fine-Tune on Text-Heavy Dataset

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/contexttab.md

Example workflow for datasets containing text responses.

```python
from tabtune import TabularPipeline
import os

# Example: Customer survey data with text responses
os.environ['HF_TOKEN'] = 'your_token'

# X contains columns like:
# - age (numerical)
# - category (categorical)
# - feedback_text (text)
# - rating (numerical)

pipeline = TabularPipeline(
    model_name='ContextTab',
    tuning_strategy='base-ft',
    tuning_params={
        'device': 'cuda',
        'epochs': 10,
        'learning_rate': 1e-4,
        'warmup_steps': 200,
        'show_progress': True
    }
)

pipeline.fit(X_train, y_train)
metrics = pipeline.evaluate(X_test, y_test)

print(f"Accuracy: {metrics['accuracy']:.4f}")
```

--------------------------------

### Verify environment and version compatibility

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/troubleshooting.md

Check installed versions of PyTorch, CUDA, and scikit-learn, or create a new environment with the required Python version.

```python
import torch
print(f"PyTorch: {torch.__version__}")
print(f"CUDA: {torch.version.cuda}")
```

```bash
pip install --upgrade torch
```

```bash
pip install scikit-learn==1.7
```

```bash
# Using conda
conda create -n tabtune python=3.10
conda activate tabtune
```

--------------------------------

### Serve Documentation Locally

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/contributing/setup.md

Use MkDocs to serve the project documentation locally for development and preview.

```bash
mkdocs serve
```

--------------------------------

### Install FlashAttention-3 on Hopper Machine

Source: https://github.com/lexsi-labs/tabtune/blob/main/tabtune/models/tabpfnv3/architectures/shared/fa3_setup.md

Clone the FlashAttention repository, navigate to the hopper directory, and install the package in-place. This method is suitable for direct installation on a Hopper-class GPU machine.

```bash
git clone https://github.com/Dao-AILab/flash-attention.git
cd flash-attention/hopper
python setup.py install
```

--------------------------------

### Complete Tabtune Workflow Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/tabpfn.md

Demonstrates a full workflow using Tabtune, including data loading, splitting, establishing a zero-shot baseline, and fine-tuning the model. Suitable for small datasets.

```python
from tabtune import TabularPipeline, TabularLeaderboard
from sklearn.model_selection import train_test_split
import pandas as pd

# 1. Load data
df = pd.read_csv('small_dataset.csv')  # <10K rows ideal
X = df.drop('target', axis=1)
y = df['target']

# 2. Split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# 3. Strategy 1: Zero-shot baseline
print("=== Zero-Shot Baseline ===")
baseline = TabularPipeline(
    model_name='TabPFN',
    tuning_strategy='inference'
)
baseline.fit(X_train, y_train)
baseline_metrics = baseline.evaluate(X_test, y_test)
print(f"Baseline Accuracy: {baseline_metrics['accuracy']:.4f}")

# 4. Strategy 2: Fine-tuned
print("\n=== Fine-Tuned ===")
finetuned = TabularPipeline(
    model_name='TabPFN',
    tuning_strategy='base-ft',
    tuning_params={
        'device': 'cuda',
        'epochs': 5,
        'learning_rate': 2e-5,
        'show_progress': True
    }
)
finetuned.fit(X_train, y_train)
finetuned_metrics = finetuned.evaluate(X_test, y_test)
print(f"Fine-tuned Accuracy: {finetuned_metrics['accuracy']:.4f}")
```

--------------------------------

### Build Documentation with MkDocs

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/contributing/documentation.md

Use this command to build the project's documentation locally.

```bash
mkdocs build
```

--------------------------------

### Complete Workflow Example with TabularLeaderboard

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/leaderboard.md

Use this workflow to systematically benchmark different models on your tabular data. Ensure your data is loaded and split into training and testing sets before initialization. The `run` method benchmarks models and ranks them by accuracy.

```python
from tabtune import TabularLeaderboard
import pandas as pd
from sklearn.model_selection import train_test_split

# Step 1: Load and prepare data
print("Loading data...")
df = pd.read_csv('data.csv')
X = df.drop('target', axis=1)
y = df['target']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Step 2: Initialize leaderboard
print("Initializing leaderboard...")
lb = TabularLeaderboard(X_train, X_test, y_train, y_test)

# Step 3: Add baseline
print("Adding inference baseline...")
lb.add_model('TabPFN', 'inference')

# Step 4: Add PEFT models
print("Adding PEFT models...")
for model in ['TabICL', 'OrionMSP', 'OrionBix', 'TabDPT']:
    lb.add_model(
        model,
        'peft',
        tuning_params={'epochs': 3, 'peft_config': {'r': 8}}
    )

# Step 5: Run benchmarks
print("Running benchmarks...")
results = lb.run(rank_by='accuracy', verbose=True, n_jobs=-1)


```

--------------------------------

### Verify TabTune installation

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/troubleshooting.md

Check the installed version to confirm the package is correctly accessible in the Python environment.

```python
import tabtune
print(tabtune.__version__)
```

--------------------------------

### Run Basic Workflow

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/mitra.md

Demonstrates a complete end-to-end workflow including data loading, splitting, training, and evaluation.

```python
from tabtune import TabularPipeline
from sklearn.model_selection import train_test_split
import pandas as pd

# Load data
df = pd.read_csv('structured_data.csv')
X = df.drop('target', axis=1)
y = df['target']

# Split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Train with Mitra
pipeline = TabularPipeline(
    model_name='Mitra',
    tuning_strategy='base-ft',
    tuning_params={
        'device': 'cuda',
        'epochs': 3,
        'support_size': 128,
        'query_size': 128,
        'batch_size': 4,
        'learning_rate': 1e-5
    }
)

pipeline.fit(X_train, y_train)
metrics = pipeline.evaluate(X_test, y_test)

print(f"Accuracy: {metrics['accuracy']:.4f}")
print(f"F1 Score: {metrics['f1_score']:.4f}")
```

--------------------------------

### Basic Standard Pipeline

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/data-processing.md

Demonstrates initializing and using the DataProcessor with standard imputation, encoding, and scaling strategies. Fit on training data before transforming.

```python
from tabtune import DataProcessor

processor = DataProcessor(
    imputation_strategy='median',
    categorical_encoding='onehot',
    scaling_strategy='standard'
)

# Fit on training data
processor.fit(X_train, y_train)

# Transform training and test data
X_train_processed, y_train_processed = processor.transform(X_train, y_train)
X_test_processed = processor.transform(X_test)
```

--------------------------------

### Execute TabPFN Fine-Tuning Pipeline

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/tabpfn.md

Demonstrates initializing a TabularPipeline with fine-tuning parameters and executing the fit and evaluate methods.

```python
from tabtune import TabularPipeline

pipeline = TabularPipeline(
    model_name='TabPFN',
    tuning_strategy='base-ft',
    tuning_params={
        'device': 'cuda',
        'epochs': 5,
        'learning_rate': 2e-5,
        'batch_size': 256,
        'scheduler': 'cosine',
        'show_progress': True
    }
)

# Fine-tune on your data
pipeline.fit(X_train, y_train)

# Evaluate
metrics = pipeline.evaluate(X_test, y_test)
print(f"Accuracy: {metrics['accuracy']:.4f}")
```

--------------------------------

### Inference-Only Usage Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/tabpfn.md

Example demonstrating how to use TabPFN for zero-shot predictions and uncertainty estimation without fine-tuning.

```APIDOC
## Inference-Only Usage

### Zero-Shot Predictions

Use TabPFN's pre-trained weights for immediate predictions without training:

```python
from tabtune import TabularPipeline

# Create pipeline with inference strategy
pipeline = TabularPipeline(
    model_name='TabPFN',
    tuning_strategy='inference',
    model_params={
        'n_estimators': 16,
        'softmax_temperature': 0.9
    }
)

# No training needed - just preprocess and predict
pipeline.fit(X_train, y_train)  # Only does preprocessing
predictions = pipeline.predict(X_test)
uncertainty = pipeline.get_uncertainty(X_test)
```

### Uncertainty Estimation

(Uncertainty estimation is shown in the Zero-Shot Predictions example above by calling `pipeline.get_uncertainty(X_test)`)
```

--------------------------------

### Install PyTorch for specific CUDA versions

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/user-guide/troubleshooting.md

Install the correct PyTorch wheel matching the system's CUDA version.

```bash
# For CUDA 11.8
pip install torch --index-url https://download.pytorch.org/whl/cu118

# For CUDA 12.1
pip install torch --index-url https://download.pytorch.org/whl/cu121
```

--------------------------------

### Install Model-Specific Dependencies

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/getting-started/installation.md

Install additional packages required for specific TabTune models, such as ContextTab which needs HuggingFace Hub access, or for advanced preprocessing.

```bash
# For ContextTab (requires HuggingFace Hub access)
huggingface-hub>=0.15.0
sentence-transformers>=2.2.0

# For advanced preprocessing
category-encoders>=2.5.0
```

--------------------------------

### TabularPipeline Processor Parameters Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/api/pipeline.md

Example of processor parameters for TabularPipeline, specifying imputation and scaling strategies. Other options include categorical encoding and resampling.

```python
processor_params={
    "imputation_strategy": "median",
    "scaling_strategy": "standard"
}
```

--------------------------------

### Create and Activate Virtual Environment

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/getting-started/installation.md

Set up a virtual environment for TabTune using either venv or conda. Activate the environment to isolate project dependencies.

```bash
# Using venv
python -m venv tabtune-env
source tabtune-env/bin/activate  # Linux/macOS
# tabtune-env\Scripts\activate   # Windows

# Or using conda
conda create -n tabtune python=3.11
conda activate tabtune
```

--------------------------------

### Launch Distributed Training via torch.distributed.launch

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/advanced/multi-gpu.md

Use the legacy torch.distributed.launch module to spawn training processes across multiple GPUs.

```bash
# Single-machine, 4 GPUs
python -m torch.distributed.launch \
  --nproc_per_node=4 \
  train_script.py

# With additional args
python -m torch.distributed.launch \
  --nproc_per_node=4 \
  train_script.py \
  --learning_rate 2e-5 \
  --epochs 5
```

--------------------------------

### Unified API Demonstration

Source: https://github.com/lexsi-labs/tabtune/blob/main/examples/README.md

This example showcases TabTune's unified API, demonstrating identical .fit(), .predict(), and .evaluate() interfaces across different models without requiring model-specific API knowledge. It uses the German Credit dataset for binary classification.

```python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.datasets import fetch_openml
from sklearn.metrics import accuracy_score

from tabtune import TabTune

# Load dataset
# For demonstration, we use a small subset of the German Credit dataset
X, y = fetch_openml(name='diabetes', version=1, return_X_y=True, as_frame=True)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize TabTune with a model (e.g., RandomForestClassifier)
# You can replace RandomForestClassifier with other supported models
tabtune = TabTune(model='RandomForestClassifier', random_state=42)

# Fit the model
tabtune.fit(X_train, y_train)

# Predict
y_pred = tabtune.predict(X_test)

# Evaluate
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.4f}")

# Example of evaluating with a different metric
# evaluation_results = tabtune.evaluate(X_test, y_test, metrics=['precision', 'recall'])
# print(f"Evaluation Results: {evaluation_results}")

```

--------------------------------

### TabularPipeline Fit Method Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/api/pipeline.md

Example of fitting the TabularPipeline on training data. This process includes fitting the DataProcessor, applying preprocessing, and training the model if a tuning strategy other than 'inference' is used.

```python
pipeline = TabularPipeline(
    model_name="TabICL",
    tuning_strategy="base-ft"
)
pipeline.fit(X_train, y_train)
```

--------------------------------

### Configure PEFT Strategy

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/getting-started/quick-start.md

Demonstrates parameter-efficient fine-tuning using the LoRA strategy.

```python
# PEFT fine-tuning
peft_pipeline = TabularPipeline(
    model_name="TabPFN",
    task_type="classification",
    tuning_strategy="peft",
    tuning_params={
        "device": "cuda",
        "epochs": 3,
        "learning_rate": 2e-4,
        "peft_config": {"r": 8, "lora_alpha": 16, "lora_dropout": 0.05}
    }
)
peft_pipeline.fit(X_train, y_train)
metrics_peft = peft_pipeline.evaluate(X_test, y_test)
print("PEFT metrics:", metrics_peft)
```

--------------------------------

### TabularPipeline Predict Method Example

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/api/pipeline.md

Example of generating predictions using the fitted TabularPipeline on test data. The method automatically applies preprocessing and converts predictions back to the original label format.

```python
predictions = pipeline.predict(X_test)
print(f"Predictions shape: {predictions.shape}")
print(f"Unique classes: {np.unique(predictions)}")
```

--------------------------------

### Create Study with Pruning

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/advanced/hyperparameter-tuning.md

Demonstrates how to create an Optuna study with a median pruner for efficient hyperparameter optimization.

```python
study = optuna.create_study(
    direction='maximize',
    pruner=optuna.pruners.MedianPruner()
)

study.optimize(objective_with_pruning, n_trials=30)
```

--------------------------------

### Parallel Tuning with Ray Tune

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/advanced/hyperparameter-tuning.md

Shows how to set up and run parallel hyperparameter tuning using Ray Tune for a TabularPipeline model. Requires Ray to be initialized.

```python
import ray
from ray import tune

# Initialize Ray
ray.init()

def train_model(config):
    """Trainable function for Ray."""
    
    pipeline = TabularPipeline(
        model_name='TabICL',
        tuning_strategy='base-ft',
        tuning_params=config
    )
    
    pipeline.fit(X_train, y_train)
    metrics = pipeline.evaluate(X_val, y_val)
    
    return metrics

# Parallel tuning
results = tune.run(
    train_model,
    config={
        'learning_rate': tune.loguniform(1e-5, 1e-3),
        'epochs': tune.randint(1, 20),
        'batch_size': tune.choice([16, 32, 64])
    },
    num_samples=30,
    verbose=1
)

ray.shutdown()
```

--------------------------------

### TabularLeaderboard Usage

Source: https://github.com/lexsi-labs/tabtune/blob/main/README.md

Demonstrates how to initialize TabularLeaderboard, add models with different tuning strategies and parameters, and run the benchmark to display ranked results.

```APIDOC
## TabularLeaderboard Usage

This section shows how to use the `TabularLeaderboard` to compare multiple models and strategies on the same dataset.

### Initialization
Initialize the leaderboard with your data splits.
```python
from tabtune.TabularLeaderboard.leaderboard import TabularLeaderboard

leaderboard = TabularLeaderboard(X_train, X_test, y_train, y_test)
```

### Adding Models
Add model configurations to compare. You can specify `model_name`, `tuning_strategy`, `model_params`, and `tuning_params`.

```python
# Example with inference tuning strategy
leaderboard.add_model(
    model_name='TabICL',
    tuning_strategy='inference',
    model_params={'n_estimators': 16}
)

# Example with finetune tuning strategy and specific tuning parameters
leaderboard.add_model(
    model_name='TabICL',
    tuning_strategy='finetune',
    model_params={'n_estimators': 16},
    tuning_params={'epochs': 5, 'learning_rate': 1e-5, 'finetune_mode': 'meta-learning'}
)

# Example with another model and default tuning strategy
leaderboard.add_model(
    model_name='TabPFN',
    tuning_strategy='inference'
)
```

### Running the Benchmark
Run the benchmark and display ranked results.
```python
leaderboard.run()
```
```

--------------------------------

### Benchmark models with TabularLeaderboard

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/examples/benchmarking.md

Initializes a leaderboard with training and test sets, adds various models with specific tuning strategies, and executes the benchmark.

```python
from tabtune import TabularLeaderboard

leaderboard = TabularLeaderboard(X_train, X_test, y_train, y_test)

leaderboard.add_model(
    model_name='TabICL',
    tuning_strategy='inference',
    model_params={'n_estimators': 16}
)
leaderboard.add_model(
    model_name='TabICL',
    tuning_strategy='finetune',
    model_params={'n_estimators': 16},
    tuning_params={'epochs': 5, 'learning_rate': 1e-5}
)
leaderboard.add_model(
    model_name='TabPFN',
    tuning_strategy='inference'
)

leaderboard.run(rank_by='roc_auc_score')
```

--------------------------------

### Using Pytest Fixtures

Source: https://github.com/lexsi-labs/tabtune/blob/main/tests/README.md

Example of injecting fixtures defined in conftest.py into test functions.

```python
def test_example(minimal_data, random_seed):
    X_train, X_test, y_train, y_test = minimal_data
    # Use the data
    pass
```

--------------------------------

### Verify HuggingFace Setup

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/contexttab.md

Script to validate the HF_TOKEN environment variable and authenticate programmatically.

```python
from huggingface_hub import login
import os

# Check token
hf_token = os.getenv('HF_TOKEN')
if hf_token:
    login(hf_token)
    print("✅ Logged into Hugging Face Hub")
else:
    print("⚠️ HF_TOKEN not set - may fail for gated models")
```

--------------------------------

### Configure HuggingFace Hub Access

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/contexttab.md

Commands to install the CLI and authenticate for gated model access.

```bash
# Install HuggingFace CLI
pip install huggingface-hub

# Login with your token
huggingface-cli login

# Or set environment variable
export HF_TOKEN='your_token_here'
```

--------------------------------

### Memory-Constrained PEFT Workflow

Source: https://github.com/lexsi-labs/tabtune/blob/main/docs/models/tabicl.md

Example of using PEFT with reduced parameters to fit in limited memory.

```python
# Fit large model in limited memory with PEFT
pipeline = TabularPipeline(
    model_name='TabICL',
    tuning_strategy='peft',
    tuning_params={
        'device': 'cuda',
        'epochs': 5,
        'learning_rate': 2e-4,
        'support_size': 24,    # Reduced
        'query_size': 16,      # Reduced
        'batch_size': 4,       # Smaller batches
        'peft_config': {
            'r': 4,            # Lower rank
            'lora_alpha': 8,
            'lora_dropout': 0.1
        }
    }
)

pipeline.fit(X_train, y_train)
```