### Install tabdpt package

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/README.md

Install the TabDPT library from PyPI. Model weights are automatically downloaded on first use.

```bash
pip install tabdpt
```

--------------------------------

### Install Dependencies for Reproducing Results

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/README.md

Installs the specific dependency versions used in the TabDPT paper. Requires Python 3.11.

```bash
pip install .[reproduce-results]
```

--------------------------------

### Install python-dev on Ubuntu

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/README.md

Install the python-dev system package on Ubuntu, which is necessary for developing Python extensions.

```bash
sudo apt-get update
sudo apt-get install python-dev
```

--------------------------------

### Install TabDPT from source

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/README.md

Clone the repository and install the TabDPT library in editable mode. This method requires Python 3.10 or 3.11 and may need a C++ compiler and python-dev package.

```bash
git clone git@github.com:layer6ai-labs/TabDPT.git
cd TabDPT
pip install -e .
pip install --group dev
```

--------------------------------

### Install using uv package manager

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/README.md

If using the 'uv' package manager, synchronize dependencies by running 'uv sync'. Ensure you have Python 3.10 or 3.11 installed.

```bash
uv sync
```

--------------------------------

### Install g++ compiler on Ubuntu

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/README.md

Install the g++ C++ compiler on Ubuntu systems, which may be required for building certain dependencies.

```bash
sudo apt-get update
sudo apt-get install g++
```

--------------------------------

### TabDPTClassifier: Multi-class Classification Example

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Demonstrates how to use `TabDPTClassifier` for multi-class classification. It includes data loading, model instantiation with various parameters, fitting the model, and predicting class labels and probabilities. Ensure data is converted to float.

```python
from sklearn.datasets import load_digits
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import label_binarize
import numpy as np
from tabdpt import TabDPTClassifier

# Load data
X, y = load_digits(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.33, random_state=42
)

# Instantiate — weights are downloaded automatically from HuggingFace on first run
clf = TabDPTClassifier(
    normalizer="standard",      # StandardScaler preprocessing (matches training)
    inf_batch_size=512,          # inference batch size
    clip_sigma=4.0,              # outlier clipping threshold
    feature_reduction="pca",     # "pca" or "subsample" when n_features > model limit
    faiss_metric="l2",           # distance metric for retrieval
    device=None,                 # auto-select cuda if available, else cpu
    use_flash=True,              # FlashAttention (CUDA only)
    compile=True,                # torch.compile (CUDA only)
    verbose=True,                # show tqdm progress bar over ensembles
)

# Fit — stores and preprocesses training data; no gradient updates
clf.fit(X_train.astype(float), y_train.astype(float))

# Predict class labels (ensemble of 8, context window of 2048)
y_pred = clf.predict(
    X_test,
    n_ensembles=8,
    temperature=0.8,
    context_size=2048,
    permute_classes=True,  # randomize class order per ensemble member
    seed=42,
)
print("Accuracy:", accuracy_score(y_test, y_pred))
# Expected output: Accuracy: ~0.98

# Predict class probabilities
proba = clf.predict_proba(
    X_test,
    temperature=0.8,
    context_size=2048,
    seed=42,
)
print("Probabilities shape:", proba.shape)   # (n_test_samples, n_classes)
print("Sum per row:", proba.sum(axis=1)[:3]) # [1.0, 1.0, 1.0]
```

--------------------------------

### Configure Metrics and Run Evaluation Loop

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/notebooks/analysis.ipynb

Sets up a dictionary to determine if higher metric values are better and defines pairs of metrics and dataset suites to evaluate. It then iterates through these pairs, computes scores and confidence intervals using `get_scores_ci`, and stores the results.

```python
best_is_higher = {
    'auc':  True,
    'acc':  True,
    'corr': True,
    'r2':   True,

    'auc_rank': False,
    'acc_rank': False,
    'corr_rank': False,
    'r2_rank': False,
}

metric_suite_pairs = [
    ('auc',  'cc18'),  
    ('acc',  'cc18'),  
    ('corr', 'ctr'), 
    ('r2',   'ctr'),  
]
# add _rank to the metrics
# you can uncomment this to compute ranks instead of raw scores but you need more than one alg_name
# TODO - Currently doesn't work, need to fix
# metric_suite_pairs = [(m + "_rank", s) for m, s in metric_suite_pairs]

all_scores = {}
all_cis    = {}
algorithms = set()

for metric, suite in metric_suite_pairs:
    scores, ci = get_scores_ci(metric, suite, df) 
    scores = {k: float(v.squeeze()) for k, v in scores.items()}
    ci     = {k: tuple(v.squeeze()) for k, v in ci.items()}
    all_scores[(metric, suite)] = scores
    all_cis[(metric, suite)]    = ci
    algorithms.update(scores.keys())

algorithms = sorted(algorithms)
```

--------------------------------

### Download Model Weights for TabDPTEstimator

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Manually download model weights to pre-warm the cache or for offline deployments. The path to the cached file is returned.

```python
from tabdpt.estimator import TabDPTEstimator

# Pre-download weights (e.g., during Docker image build)
cached_path = TabDPTEstimator.download_weights()
print("Weights cached at:", cached_path)
# e.g., /home/user/.cache/huggingface/hub/models--Layer6--TabDPT/snapshots/.../tabdpt1_1.safetensors

# Pass the cached path to avoid network calls at inference time
from tabdpt import TabDPTClassifier
clf = TabDPTClassifier(model_weight_path=cached_path)
```

--------------------------------

### TabDPTEstimator.__init__

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

The base constructor for TabDPTClassifier and TabDPTRegressor. It handles downloading or loading model weights, parsing configuration, constructing the model, and setting up the normalizer.

```APIDOC
## `TabDPTEstimator.__init__` — Constructor and model loading

The shared base constructor for both `TabDPTClassifier` and `TabDPTRegressor`. On instantiation it downloads (or loads from a local path) the SafeTensor model weights, parses the embedded `OmegaConf` config, constructs the `TabDPTModel`, and sets up the chosen normalizer. Weights are cached by `huggingface_hub` after the first download.

```python
from tabdpt import TabDPTClassifier

# Option 1: Auto-download weights from HuggingFace (cached after first run)
clf = TabDPTClassifier()

# Option 2: Load weights from a local path (no network required)
clf_local = TabDPTClassifier(
    model_weight_path="/path/to/tabdpt1_1.safetensors",
    device="cuda",
    use_flash=True,
    compile=True,
)

# Option 3: CPU-only, no compilation, quantile normalization
clf_cpu = TabDPTClassifier(
    device="cpu",
    use_flash=False,
    compile=False,
    normalizer="quantile-normal",
    missing_indicators=True,   # adds binary columns for NaN positions
    clip_sigma=3.0,
    verbose=False,
)

# Manually move model to a different device after construction
clf.to("cuda:1")
```
```

--------------------------------

### TabDPTEstimator.download_weights

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

A static utility method to explicitly download model weights from the Layer6/TabDPT HuggingFace repository. It returns the path to the cached file. This method is called automatically during initialization if `model_weight_path` is not provided, but can also be called manually to pre-populate the cache.

```APIDOC
## TabDPTEstimator.download_weights

### Description
A static utility method that explicitly triggers a download of the model weights from the `Layer6/TabDPT` HuggingFace repository using `huggingface_hub.hf_hub_download`. The path to the cached file is returned. This is called automatically inside `__init__` when `model_weight_path` is not provided but can also be called manually to pre-warm the cache.

### Method
`TabDPTEstimator.download_weights()`

### Parameters
None

### Response
- **cached_path** (str) - The file path to the downloaded and cached model weights.

### Request Example
```python
from tabdpt.estimator import TabDPTEstimator

# Pre-download weights (e.g., during Docker image build)
cached_path = TabDPTEstimator.download_weights()
print("Weights cached at:", cached_path)
# e.g., /home/user/.cache/huggingface/hub/models--Layer6--TabDPT/snapshots/.../tabdpt1_1.safetensors

# Pass the cached path to avoid network calls at inference time
from tabdpt import TabDPTClassifier
clf = TabDPTClassifier(model_weight_path=cached_path)
```
```

--------------------------------

### TabDPTClassifier and TabDPTRegressor Constructor and Model Loading

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

The base constructor for TabDPT models downloads or loads weights, parses configuration, and sets up the normalizer. Weights are cached by huggingface_hub. You can specify the model path, device, and other options like flash attention and compilation.

```python
from tabdpt import TabDPTClassifier

# Option 1: Auto-download weights from HuggingFace (cached after first run)
clf = TabDPTClassifier()

# Option 2: Load weights from a local path (no network required)
clf_local = TabDPTClassifier(
    model_weight_path="/path/to/tabdpt1_1.safetensors",
    device="cuda",
    use_flash=True,
    compile=True,
)

# Option 3: CPU-only, no compilation, quantile normalization
clf_cpu = TabDPTClassifier(
    device="cpu",
    use_flash=False,
    compile=False,
    normalizer="quantile-normal",
    missing_indicators=True,   # adds binary columns for NaN positions
    clip_sigma=3.0,
    verbose=False,
)

# Manually move model to a different device after construction
clf.to("cuda:1")
```

--------------------------------

### Import Libraries for TabDPT Evaluation

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/notebooks/analysis.ipynb

Imports necessary libraries including numpy, pandas, and specific modules from rliable for analysis.

```python
import numpy as np
import pandas as pd
from rliable import library as rly
from rliable import metrics
```

--------------------------------

### Run Paper Evaluation Script

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/README.md

Executes the main evaluation script to generate results comparable to the paper. Run for each fold separately.

```bash
python paper_evaluation.py --fold 0
```

```bash
python paper_evaluation.py --fold 1
```

--------------------------------

### Display All Confidence Intervals

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/notebooks/analysis.ipynb

Prints the dictionary containing all computed confidence intervals for the evaluated metrics and suites.

```python
all_cis
```

--------------------------------

### Load and Prepare TabDPT Results Data

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/notebooks/analysis.ipynb

Loads results from multiple folds, concatenates them, and adds algorithm names. It also creates binary columns 'cc18' and 'ctr' based on the presence of non-NaN values in 'auc' and 'r2' respectively.

```python
# Get the csvs from TabDPT results on at least 2 folds
# This is important to compute confidence intervals with rliable
tabdpt_df0 = pd.read_csv('../results_fold0.csv')
tabdpt_df1 = pd.read_csv('../results_fold1.csv')

tabdpt_df0['fold'] = 0
tabdpt_df1['fold'] = 1
tabdpt_df = pd.concat([tabdpt_df0, tabdpt_df1], axis=0)

tabdpt_df['alg_name'] = 'TabDPT'

df = tabdpt_df  # can join several tables with same structure and different alg_name

# column cc18 is 1 when auc is not NaN, 0 otherwise
df['cc18'] = df['auc'].notna().astype(int)

# column ctr is 1 when r2 is not NaN, 0 otherwise
df['ctr'] = df['r2'].notna().astype(int)
```

--------------------------------

### Display All Scores

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/notebooks/analysis.ipynb

Prints the dictionary containing all computed scores for the evaluated metrics and suites.

```python
all_scores
```

--------------------------------

### TabDPTEstimator.fit for Preprocessing Training Data

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

The fit method preprocesses and stores training data, handling missing values with imputation and applying normalization. It also computes PCA projections if needed. This is a preprocessing step and does not involve gradient updates. Ensure missing_indicators is set if your data has NaNs.

```python
import numpy as np
from tabdpt import TabDPTClassifier

rng = np.random.default_rng(0)

# Simulate a dataset with missing values
X_train = rng.standard_normal((500, 20))
X_train[rng.random(X_train.shape) < 0.05] = np.nan   # inject 5% NaNs
y_train = rng.integers(0, 3, size=500).astype(float)

clf = TabDPTClassifier(
    missing_indicators=True,  # adds 20 binary indicator columns
    normalizer="robust",      # RobustScaler — better for outlier-heavy data
    feature_reduction="pca",
    verbose=False,
)

clf.fit(X_train, y_train)

print("Training samples stored:", clf.n_instances)  # 500
print("Features after indicators:", clf.X_train.shape[1])  # up to 40
print("Fitted:", clf.is_fitted_)  # True
```

--------------------------------

### Scikit-learn Pipeline and Cross-Validation Integration

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Demonstrates how TabDPTEstimator and TabDPTRegressor can be integrated into scikit-learn's Pipeline, cross_val_score, and GridSearchCV due to their inheritance from sklearn.base.BaseEstimator.

```APIDOC
## Scikit-learn Pipeline and Cross-Validation Integration

### Description
Because `TabDPTClassifier` and `TabDPTRegressor` inherit from `sklearn.base.BaseEstimator` and the respective mixin classes, they are fully compatible with scikit-learn's `Pipeline`, `cross_val_score`, and `GridSearchCV`.

### Usage Example
```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import FunctionTransformer
from tabdpt import TabDPTClassifier

X, y = load_iris(return_X_y=True)

# TabDPT handles its own normalization internally, but you can still wrap it
pipeline = Pipeline([
    # Cast to float64 to ensure numpy dtype compatibility
    ("cast", FunctionTransformer(lambda x: x.astype(np.float64))),
    ("clf", TabDPTClassifier(
        normalizer="standard",
        n_ensembles=4,          # NOTE: passed via predict, not fit
        verbose=False,
        compile=False,
        device="cpu",
    )),
])

# cross_val_score calls fit + predict automatically
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(
    pipeline, X, y.astype(float),
    cv=cv,
    scoring="accuracy",
)
print(f"CV Accuracy: {scores.mean():.4f} ± {scores.std():.4f}")
# Expected: ~0.97 ± 0.02
```
```

--------------------------------

### TabDPTEstimator.fit

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Preprocesses and stores training data, including mean imputation, applying the chosen normalizer, and computing PCA projection if necessary. This is a preprocessing step and does not involve gradient updates.

```APIDOC
## `TabDPTEstimator.fit` — Fit on training data

Preprocesses and stores training data. Applies mean imputation for missing values, the chosen normalizer scaler, and (if `n_features > model_max_features`) computes a PCA projection matrix. No gradient updates occur — this is purely a preprocessing and caching step. Must be called before `predict` or `predict_proba`.

```python
import numpy as np
from tabdpt import TabDPTClassifier

rng = np.random.default_rng(0)

# Simulate a dataset with missing values
X_train = rng.standard_normal((500, 20))
X_train[rng.random(X_train.shape) < 0.05] = np.nan   # inject 5% NaNs
y_train = rng.integers(0, 3, size=500).astype(float)

clf = TabDPTClassifier(
    missing_indicators=True,  # adds 20 binary indicator columns
    normalizer="robust",      # RobustScaler — better for outlier-heavy data
    feature_reduction="pca",
    verbose=False,
)

clf.fit(X_train, y_train)

print("Training samples stored:", clf.n_instances)  # 500
print("Features after indicators:", clf.X_train.shape[1])  # up to 40
print("Fitted:", clf.is_fitted_)  # True
```
```

--------------------------------

### TabDPTClassifier.ensemble_predict_proba: Ensemble Probability Estimation

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Shows how to use `ensemble_predict_proba` for more calibrated and accurate probability estimates by averaging logits from multiple forward passes. Recommended when `n_ensembles > 1`.

```python
from tabdpt import TabDPTClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

clf = TabDPTClassifier(device="cpu", compile=False, verbose=False)
clf.fit(X_train.astype(float), y_train.astype(float))

# Directly call ensemble_predict_proba for calibrated probabilities
proba = clf.ensemble_predict_proba(
    X_test,
    n_ensembles=4,
    temperature=0.8,
    context_size=2048,
    permute_classes=True,
    seed=0,
)
# Binary case: use the positive-class column
auc = roc_auc_score(y_test, proba[:, 1])
print(f"ROC-AUC: {auc:.4f}")  # Expected: ~0.999
```

--------------------------------

### TabDPTClassifier/Regressor Prediction with Ensembles and FAISS

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Generate predictions using the primary entry point. For `n_ensembles=1`, a single forward pass is performed. For `n_ensembles>1`, ensemble prediction is used for higher accuracy. FAISS k-NN retrieval is automatically used if training data exceeds `context_size`. Consider `temperature` and `permute_classes` for classification.

```python
import numpy as np
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from tabdpt import TabDPTClassifier

X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=7)

clf = TabDPTClassifier(verbose=False, device="cpu", compile=False)
clf.fit(X_train.astype(float), y_train.astype(float))

# Single pass (fast, less accurate)
y_single = clf.predict(X_test, n_ensembles=1, context_size=2048, seed=0)
print("Single-pass accuracy:", accuracy_score(y_test, y_single))

# Ensemble of 8 (slower, higher accuracy — recommended default)
y_ensemble = clf.predict(
    X_test,
    n_ensembles=8,
    temperature=0.8,
    context_size=2048,
    permute_classes=True,
    seed=0,
)
print("Ensemble accuracy:", accuracy_score(y_test, y_ensemble))
```

--------------------------------

### TabDPTRegressor for Continuous Value Regression

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Use TabDPTRegressor for regression tasks. It supports ensemble, FAISS-retrieval, and normalization options similar to the classifier. Ensure data is in float format before fitting.

```python
from sklearn.datasets import fetch_california_housing
from sklearn.metrics import r2_score, mean_absolute_error
from sklearn.model_selection import train_test_split
from tabdpt import TabDPTRegressor

X, y = fetch_california_housing(return_X_y=True)
# Limit to 4096 rows for speed (FAISS retrieval handles larger datasets)
X, y = X[:4096], y[:4096]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

reg = TabDPTRegressor(
    normalizer="standard",
    inf_batch_size=512,
    clip_sigma=4.0,
    feature_reduction="pca",
    faiss_metric="l2",
    verbose=True,
)

reg.fit(X_train.astype(float), y_train.astype(float))

y_pred = reg.predict(
    X_test,
    n_ensembles=2,       # fewer ensembles = faster; more = higher accuracy
    context_size=2048,
    seed=42,
)
print(f"R²:  {r2_score(y_test, y_pred):.4f}")         # Expected: ~0.83
print(f"MAE: {mean_absolute_error(y_test, y_pred):.4f}")
```

--------------------------------

### TabDPT Paper Citation

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/README.md

BibTeX entry for citing the TabDPT paper.

```bibtex
@inproceedings{
    ma2025tabdpt,
    title={Tab{DPT}: Scaling Tabular Foundation Models on Real Data},
    author={Junwei Ma and Valentin Thomas and Rasa Hosseinzadeh and Alex Labach and Hamidreza Kamkari and Jesse C. Cresswell and Keyvan Golestan and Guangwei Yu and Anthony L. Caterini and Maksims Volkovs},
    booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
    year={2025},
    url={https://openreview.net/forum?id=pIZxEOZCId}
}
```

--------------------------------

### TabDPTClassifier.ensemble_predict_proba

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Estimates probabilities by running predict_proba multiple times with different seeds and class permutations, then averaging the logits before softmax. This method provides better-calibrated and more accurate probabilities, especially when n_ensembles > 1.

```APIDOC
## TabDPTClassifier.ensemble_predict_proba

### Description
Runs `predict_proba` multiple times with different random seeds and class-order permutations, then averages the raw logits before applying softmax. This produces better-calibrated, more accurate probabilities than a single forward pass and is the recommended path when `n_ensembles > 1`.

### Method
`ensemble_predict_proba(X, n_ensembles=4, temperature=0.8, context_size=2048, permute_classes=True, seed=0)`

### Parameters
- **X** (numpy.ndarray) - Test data features.
- **n_ensembles** (int) - Number of ensemble members to use for averaging.
- **temperature** (float) - Softmax temperature for probability calibration.
- **context_size** (int) - Maximum number of training samples to consider per test point.
- **permute_classes** (bool) - Whether to randomize class order per ensemble member.
- **seed** (int) - Random seed for reproducibility.

### Response
#### Success Response (200)
- **proba** (numpy.ndarray) - Calibrated class probabilities (shape: n_test_samples, n_classes).

### Request Example
```python
from tabdpt import TabDPTClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

clf = TabDPTClassifier(device="cpu", compile=False, verbose=False)
clf.fit(X_train.astype(float), y_train.astype(float))

proba = clf.ensemble_predict_proba(
    X_test,
    n_ensembles=4,
    temperature=0.8,
    context_size=2048,
    permute_classes=True,
    seed=0,
)
auc = roc_auc_score(y_test, proba[:, 1])
print(f"ROC-AUC: {auc:.4f}")
```
```

--------------------------------

### TabDPTRegressor

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

A scikit-learn compatible estimator for regression tasks. It normalizes target values during inference and denormalizes predictions. Supports ensemble, FAISS-retrieval, and normalization options.

```APIDOC
## `TabDPTRegressor` — Continuous value regression

A scikit-learn `RegressorMixin`-compatible estimator for regression tasks. Internally normalizes target values during inference (mean/std of the training split of the context window), then denormalizes predictions. Supports the same ensemble, FAISS-retrieval, and normalization options as the classifier.

```python
from sklearn.datasets import fetch_california_housing
from sklearn.metrics import r2_score, mean_absolute_error
from sklearn.model_selection import train_test_split
from tabdpt import TabDPTRegressor

X, y = fetch_california_housing(return_X_y=True)
# Limit to 4096 rows for speed (FAISS retrieval handles larger datasets)
X, y = X[:4096], y[:4096]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

reg = TabDPTRegressor(
    normalizer="standard",
    inf_batch_size=512,
    clip_sigma=4.0,
    feature_reduction="pca",
    faiss_metric="l2",
    verbose=True,
)

reg.fit(X_train.astype(float), y_train.astype(float))

y_pred = reg.predict(
    X_test,
    n_ensembles=2,       # fewer ensembles = faster; more = higher accuracy
    context_size=2048,
    seed=42,
)
print(f"R²:  {r2_score(y_test, y_pred):.4f}")         # Expected: ~0.83
print(f"MAE: {mean_absolute_error(y_test, y_pred):.4f}")
```
```

--------------------------------

### Scikit-learn Pipeline with TabDPTClassifier

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Integrate TabDPTClassifier into scikit-learn pipelines for seamless use with cross-validation and hyperparameter tuning. Ensure input data is float64 for compatibility.

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import FunctionTransformer
from tabdpt import TabDPTClassifier

X, y = load_iris(return_X_y=True)

# TabDPT handles its own normalization internally, but you can still wrap it
pipeline = Pipeline([
    # Cast to float64 to ensure numpy dtype compatibility
    ("cast", FunctionTransformer(lambda x: x.astype(np.float64))),
    ("clf", TabDPTClassifier(
        normalizer="standard",
        n_ensembles=4,          # NOTE: passed via predict, not fit
        verbose=False,
        compile=False,
        device="cpu",
    )),
])

# cross_val_score calls fit + predict automatically
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(
    pipeline, X, y.astype(float),
    cv=cv,
    scoring="accuracy",
)
print(f"CV Accuracy: {scores.mean():.4f} ± {scores.std():.4f}")
# Expected: ~0.97 ± 0.02
```

--------------------------------

### TabDPTClassifier

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

A scikit-learn ClassifierMixin-compatible estimator for multi-class classification tasks. It handles missing values, offers normalization strategies, and uses FAISS k-NN retrieval for datasets larger than the context window.

```APIDOC
## TabDPTClassifier

### Description
A scikit-learn `ClassifierMixin`-compatible estimator that wraps the TabDPT transformer model for classification tasks. It supports datasets with any number of classes, handles missing values internally via mean imputation, optionally adds missing-value indicator columns, and offers multiple feature-normalization strategies. For datasets larger than `context_size`, it uses FAISS k-NN retrieval to select the most relevant training samples per test point.

### Method
`TabDPTClassifier(normalizer="standard", inf_batch_size=512, clip_sigma=4.0, feature_reduction="pca", faiss_metric="l2", device=None, use_flash=True, compile=True, verbose=True)`

### Parameters
- **normalizer** (str) - "standard" or "minmax" for preprocessing.
- **inf_batch_size** (int) - Batch size for inference.
- **clip_sigma** (float) - Outlier clipping threshold.
- **feature_reduction** (str) - "pca" or "subsample" when n_features > model limit.
- **faiss_metric** (str) - Distance metric for retrieval ("l2" or "inner_product").
- **device** (str or None) - "cuda" or "cpu", auto-selects if None.
- **use_flash** (bool) - Enables FlashAttention (CUDA only).
- **compile** (bool) - Enables torch.compile (CUDA only).
- **verbose** (bool) - Shows tqdm progress bar over ensembles.

### Method
`fit(X, y)`

### Parameters
- **X** (numpy.ndarray) - Training data features.
- **y** (numpy.ndarray) - Training data labels.

### Method
`predict(X, n_ensembles=8, temperature=0.8, context_size=2048, permute_classes=True, seed=42)`

### Description
Predicts class labels using an ensemble of models.

### Parameters
- **X** (numpy.ndarray) - Test data features.
- **n_ensembles** (int) - Number of ensemble members to use.
- **temperature** (float) - Softmax temperature for probability calibration.
- **context_size** (int) - Maximum number of training samples to consider per test point.
- **permute_classes** (bool) - Whether to randomize class order per ensemble member.
- **seed** (int) - Random seed for reproducibility.

### Method
`predict_proba(X, temperature=0.8, context_size=2048, seed=42)`

### Description
Predicts class probabilities for the test data.

### Parameters
- **X** (numpy.ndarray) - Test data features.
- **temperature** (float) - Softmax temperature for probability calibration.
- **context_size** (int) - Maximum number of training samples to consider per test point.
- **seed** (int) - Random seed for reproducibility.

### Response
#### Success Response (200)
- **y_pred** (numpy.ndarray) - Predicted class labels.
- **proba** (numpy.ndarray) - Predicted class probabilities (shape: n_test_samples, n_classes).

### Request Example
```python
from sklearn.datasets import load_digits
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
import numpy as np
from tabdpt import TabDPTClassifier

X, y = load_digits(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

clf = TabDPTClassifier(
    normalizer="standard",
    inf_batch_size=512,
    clip_sigma=4.0,
    feature_reduction="pca",
    faiss_metric="l2",
    device=None,
    use_flash=True,
    compile=True,
    verbose=True,
)

clf.fit(X_train.astype(float), y_train.astype(float))

y_pred = clf.predict(
    X_test,
    n_ensembles=8,
    temperature=0.8,
    context_size=2048,
    permute_classes=True,
    seed=42,
)
print("Accuracy:", accuracy_score(y_test, y_pred))

proba = clf.predict_proba(
    X_test,
    temperature=0.8,
    context_size=2048,
    seed=42,
)
print("Probabilities shape:", proba.shape)
print("Sum per row:", proba.sum(axis=1)[:3])
```
```

--------------------------------

### TabDPTClassifier.predict / TabDPTRegressor.predict

Source: https://context7.com/layer6ai-labs/tabdpt/llms.txt

Generates predictions. Can run a single forward pass or an ensemble of passes for improved accuracy. Automatically uses FAISS k-NN retrieval if training data exceeds context size.

```APIDOC
## `TabDPTClassifier.predict` / `TabDPTRegressor.predict` — Generate predictions

The primary prediction entry point. When `n_ensembles=1` it runs a single forward pass; when `n_ensembles>1` it internally calls `ensemble_predict_proba` (classifier) or `_ensemble_predict` (regressor) and returns the argmax or averaged prediction. Uses FAISS k-NN retrieval automatically when training data exceeds `context_size`.

```python
import numpy as np
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from tabdpt import TabDPTClassifier

X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=7)

clf = TabDPTClassifier(verbose=False, device="cpu", compile=False)
clf.fit(X_train.astype(float), y_train.astype(float))

# Single pass (fast, less accurate)
y_single = clf.predict(X_test, n_ensembles=1, context_size=2048, seed=0)
print("Single-pass accuracy:", accuracy_score(y_test, y_single))

# Ensemble of 8 (slower, higher accuracy — recommended default)
y_ensemble = clf.predict(
    X_test,
    n_ensembles=8,
    temperature=0.8,
    context_size=2048,
    permute_classes=True,
    seed=0,
)
print("Ensemble accuracy:", accuracy_score(y_test, y_ensemble))

```
```

--------------------------------

### Calculate Scores and Confidence Intervals

Source: https://github.com/layer6ai-labs/tabdpt/blob/main/notebooks/analysis.ipynb

Defines a function to compute scores and confidence intervals for a given metric and dataset suite. It handles data filtering, pivot table construction, NaN imputation with row means, and uses rliable for interval estimation.

```python
# Pivot table construction
def get_scores_ci(metric, suite, data):
    data_suite = data[data['cc18'] == 1] if suite == 'cc18' else data[data['ctr'] == 1]

    algorithm_metric_dict = {}
    for alg_name, group in data_suite.groupby('alg_name'):
        # Create a pivot table: rows are folds, columns are datasets, values are metric scores
        pivot_table = group.pivot(index='fold', columns='name', values=metric)
        scores =  pivot_table.values
        # if there are NaN values, replace them with row mean
        scores = np.where(np.isnan(scores), np.nanmean(scores, axis=1, keepdims=True), scores)
        algorithm_metric_dict[alg_name] = scores
    algorithms = list(algorithm_metric_dict.keys())

    # choose one of the following aggregate functions
    aggregate_func = lambda x: np.array([
        # metrics.aggregate_median(x),
        metrics.aggregate_iqm(x),
        # metrics.aggregate_mean(x),
        # metrics.aggregate_optimality_gap(x)
    ])
    aggregate_scores, aggregate_score_cis = rly.get_interval_estimates(
        algorithm_metric_dict, aggregate_func, reps=20000
    )

    aggregate_scores = {alg: aggregate_scores[alg] for alg in algorithms}
    aggregate_score_cis = {alg: aggregate_score_cis[alg] for alg in algorithms}
    return aggregate_scores, aggregate_score_cis
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.