### Install Transtab from Local Clone

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/install.md

Clone the Transtab repository locally and install it using pip. This method is useful for development or when direct pip installation fails.

```bash
git clone https://github.com/RyanWangZf/transtab.git
cd transtab
pip install .
```

--------------------------------

### Load Dataset and Start Training

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Placeholder comment indicating the next step is to load a dataset and begin supervised training.

```python
# load a dataset and start vanilla supervised training
```

--------------------------------

### Install Transtab from GitHub using Pip

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/install.md

Use this command to install the latest version of Transtab directly from its GitHub repository.

```bash
pip install git+https://github.com/RyanWangZf/transtab.git
```

--------------------------------

### Example Dataset Configuration

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/data_preparation.md

This dictionary defines the structure and parameters for a custom dataset, including column types, binary indicators, and data splits.

```python
EXAMPLE_DATACONFIG = {
    "example": { # dataset name
        "bin": ["bin1", "bin2"], # binary column names
        "cat": ["cat1", "cat2"], # categorical column names
        "num": ["num1", "num2"], # numerical column names
        "cols": ["bin1", "bin2", "cat1", "cat2", "num1", "num2"], # all column names
        "binary_indicator": ["1", "yes", "true", "positive", "t", "y"], # binary indicators in the binary columns, which will be converted to 1
        "data_split_idx": {
            "train":[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], # row indices for training set
            "val":[10, 11, 12, 13, 14, 15, 16, 17, 18, 19], # row indices for validation set
            "test":[20, 21, 22, 23, 24, 25, 26, 27, 28, 29], # row indices for test set
            }
        }
    }
```

--------------------------------

### Install OpenML Package

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Installs the OpenML package, which is required for loading datasets from the OpenML repository using Transtab.

```bash
pip install openml
```

--------------------------------

### Install Package from TestPyPI

Source: https://github.com/ryanwangzf/transtab/blob/main/pypi_build_commands.txt

Install a specific version of your package from the TestPyPI repository. This is useful for verifying that the package installs and functions correctly from the test index.

```bash
pip install --index-url https://test.pypi.org/simple/ transtab==0.0.2c
```

--------------------------------

### Install Rust Toolchain for Tokenizers Build

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/install.md

If you encounter build errors related to 'tokenizers' on MAC/Linux, install the Rust toolchain using this command. This is a prerequisite for building certain dependencies.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

--------------------------------

### Start Contrastive Pretraining Training

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/contrastive_learning.ipynb

Initiates the contrastive pretraining process with specified training arguments. The 'output_dir' parameter specifies where the model checkpoints will be saved.

```python
# start contrastive pretraining training
training_arguments = {
    'num_epoch':50,
    'batch_size':64,
    'lr':1e-4,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint'
    }

transtab.train(model, trainset, valset, collate_fn=collate_fn, **training_arguments)
```

--------------------------------

### Basic Usage of transtab for Supervised Learning

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/index.md

This snippet demonstrates the fundamental steps for using transtab, including loading data, building a classifier, training the model, and making predictions. Ensure you have the transtab library installed and the necessary dataset available.

```python
import transtab

# load dataset by specifying dataset name
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data('credit-g')

# build classifier
model = transtab.build_classifier(cat_cols, num_cols, bin_cols)

# start training
transtab.train(model, trainset, valset, **training_arguments)

# make predictions, df_x is a pd.DataFrame with shape (n, d)
# return the predictions ypred with shape (n, 1) if binary classification;
# (n, n_class) if multiclass classification.
ypred = transtab.predict(model, df_x)
```

--------------------------------

### Configure and Start Training

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/table_embedding.ipynb

Defines training arguments and initiates the contrastive pretraining process using Transtab's train function. Ensure trainset, valset, and collate_fn are properly defined before execution.

```python
training_arguments = {
    'num_epoch':50,
    'batch_size':64,
    'lr':1e-4,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint'
    }

transtab.train(model, trainset, valset, collate_fn=collate_fn, **training_arguments)
```

--------------------------------

### Configure and Start Transtab Model Training

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/contrastive_learning.ipynb

Define training parameters such as the number of epochs, evaluation metric, and output directory. Then, initiate the training process using the transtab.train function with the model, training set, and validation set.

```python
training_arguments = {
    'num_epoch':50,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint'
    }
transtab.train(model, trainset, valset, **training_arguments)
```

--------------------------------

### Training Progress Output

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/fast_train.ipynb

Example output showing the progress of model training, including epoch number, validation loss, training loss, learning rate, and time spent.

```text
Epoch:   0%|          | 0/50 [00:00<?, ?it/s]

```

--------------------------------

### Basic Usage of TransTab Classifier

Source: https://github.com/ryanwangzf/transtab/blob/main/README.md

Load a dataset, build a classifier, and train the model. Ensure PyTorch is installed and the correct version is followed.

```python
import transtab

# load dataset by specifying dataset name
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
     = transtab.load_data('credit-g')

# build classifier
model = transtab.build_classifier(cat_cols, num_cols, bin_cols)

# build regressor
# model = transtab.build_regressor(cat_cols, num_cols, bin_cols)

# start training
transtab.train(model, trainset, valset, **training_arguments)

# make predictions, df_x is a pd.DataFrame with shape (n, d)
# return the predictions ypred with shape (n, 1) if binary classification;
# (n, n_class) if multiclass classification.
ypred = transtab.predict(model, df_x)
```

--------------------------------

### Upload Package to PyPI

Source: https://github.com/ryanwangzf/transtab/blob/main/pypi_build_commands.txt

Upload your finalized package distribution files to the official PyPI repository. This makes your package publicly available for installation by users.

```bash
twine upload dist/*
```

--------------------------------

### Build and Use a Feature Extractor

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/transtab.build_extractor.md

Demonstrates how to build a feature extractor with specified categorical and numerical columns, create a sample DataFrame, and then use the extractor to get encoded outputs. The output is a dictionary containing various tensor representations of the input data.

```python
# build the feature extractor
extractor = transtab.build_extractor(categorical_columns=['gender'], numerical_columns=['age'])

# build a table for inputs
df = pd.DataFrame({'age':[1,2], 'gender':['male','female']})

# extract the outputs
outputs = extractor(df)

print(outputs)

'''
    {
    'x_num': tensor([[1.],[2.]], dtype=torch.float64),
    'num_col_input_ids': tensor([[2287]]),
    'x_cat_input_ids': tensor([[5907, 3287], [5907, 2931]]),
    'x_bin_input_ids': None,
    'num_att_mask': tensor([[1]]),
    'cat_att_mask': tensor([[1, 1], [1, 1]])
    }
'''
```

--------------------------------

### Initiate Transtab Model Training

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Calls the transtab.train function to start the model training process. It requires the model, training dataset, validation dataset, and the defined training arguments.

```python
transtab.train(model, trainset[1], valset[1], **training_arguments)
```

--------------------------------

### Train Contrastive Pretraining Model

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/example_pretrain.md

Starts the contrastive pretraining process. The collate_fn returned by build_contrastive_learner must be passed to the train function. Training arguments like number of epochs, batch size, learning rate, and output directory should be configured.

```python
# start contrastive pretraining training
training_arguments = {
    'num_epoch':50,
    'batch_size':64,
    'lr':1e-4,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint' # save the pretrained model
    }

# pass the collate function to the train function
transtab.train(model, trainset, valset, collate_fn=collate_fn, **training_arguments)
```

--------------------------------

### Start TransTab Regressor Training

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Initiates the training process for a TransTab regressor model with specified training arguments and datasets. Ensure the model, training set, and validation set are prepared before calling this function.

```python
training_arguments = {
    'num_epoch':50,
    'eval_metric':'mse',
    'eval_less_is_better':False,
    'output_dir':'./checkpoint',
    'batch_size':128,
    'lr':2e-4,
    }

transtab.train(model, trainset_reg[1], valset_reg[1], **training_arguments)
```

--------------------------------

### Upload Package to TestPyPI

Source: https://github.com/ryanwangzf/transtab/blob/main/pypi_build_commands.txt

Upload your package to the TestPyPI repository for testing purposes. This allows you to simulate the installation and distribution process without affecting the live PyPI.

```bash
python -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*
```

--------------------------------

### Set Random Seed and Change Directory

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/contrastive_learning.ipynb

Initializes the random seed for reproducibility and changes the current directory. Ensure the 'transtab' library is installed and accessible.

```python
import os
os.chdir('../')

import transtab

# set random seed
transtab.random_seed(42)
```

--------------------------------

### Training Output Log

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/table_embedding.ipynb

Example log output from a contrastive pretraining session, showing epoch number, validation loss, training loss, learning rate, and time spent per epoch.

```text
epoch: 0, test val_loss: 6.349929
epoch: 0, train loss: 72.9975, lr: 0.000100, spent: 1.3 secs
epoch: 1, test val_loss: 6.043663
epoch: 1, train loss: 62.8806, lr: 0.000100, spent: 2.2 secs
epoch: 2, test val_loss: 5.999826
epoch: 2, train loss: 61.3078, lr: 0.000100, spent: 3.0 secs
epoch: 3, test val_loss: 5.989734
epoch: 3, train loss: 61.0470, lr: 0.000100, spent: 3.9 secs
epoch: 4, test val_loss: 5.986117
epoch: 4, train loss: 60.9742, lr: 0.000100, spent: 4.8 secs
epoch: 5, test val_loss: 5.984314
epoch: 5, train loss: 60.9454, lr: 0.000100, spent: 5.8 secs
epoch: 6, test val_loss: 5.983197
epoch: 6, train loss: 60.9270, lr: 0.000100, spent: 6.7 secs
epoch: 7, test val_loss: 5.982450
epoch: 7, train loss: 60.9164, lr: 0.000100, spent: 7.6 secs
epoch: 8, test val_loss: 5.981885
epoch: 8, train loss: 60.9102, lr: 0.000100, spent: 8.5 secs
epoch: 9, test val_loss: 5.981443
epoch: 9, train loss: 60.9047, lr: 0.000100, spent: 9.5 secs
epoch: 10, test val_loss: 5.981087
epoch: 10, train loss: 60.9004, lr: 0.000100, spent: 10.3 secs
epoch: 11, test val_loss: 5.980795
epoch: 11, train loss: 60.8956, lr: 0.000100, spent: 11.3 secs
epoch: 12, test val_loss: 5.980557
epoch: 12, train loss: 60.8925, lr: 0.000100, spent: 12.3 secs
epoch: 13, test val_loss: 5.980357
epoch: 13, train loss: 60.8902, lr: 0.000100, spent: 13.3 secs
epoch: 14, test val_loss: 5.980191
epoch: 14, train loss: 60.8874, lr: 0.000100, spent: 14.5 secs
epoch: 15, test val_loss: 5.980050
epoch: 15, train loss: 60.8863, lr: 0.000100, spent: 15.5 secs
epoch: 16, test val_loss: 5.979930
epoch: 16, train loss: 60.8836, lr: 0.000100, spent: 16.4 secs
epoch: 17, test val_loss: 5.979825
epoch: 17, train loss: 60.8822, lr: 0.000100, spent: 17.3 secs
epoch: 18, test val_loss: 5.979736
epoch: 18, train loss: 60.8821, lr: 0.000100, spent: 18.2 secs
epoch: 19, test val_loss: 5.979657
epoch: 19, train loss: 60.8804, lr: 0.000100, spent: 19.2 secs
epoch: 20, test val_loss: 5.979586
epoch: 20, train loss: 60.8802, lr: 0.000100, spent: 20.3 secs
epoch: 21, test val_loss: 5.979523
epoch: 21, train loss: 60.8798, lr: 0.000100, spent: 21.3 secs
epoch: 22, test val_loss: 5.979466
epoch: 22, train loss: 60.8791, lr: 0.000100, spent: 22.2 secs
epoch: 23, test val_loss: 5.979416
epoch: 23, train loss: 60.8778, lr: 0.000100, spent: 23.2 secs
epoch: 24, test val_loss: 5.979372
epoch: 24, train loss: 60.8776, lr: 0.000100, spent: 24.2 secs
epoch: 25, test val_loss: 5.979331
epoch: 25, train loss: 60.8773, lr: 0.000100, spent: 25.1 secs
epoch: 26, test val_loss: 5.979294
epoch: 26, train loss: 60.8763, lr: 0.000100, spent: 26.0 secs
epoch: 27, test val_loss: 5.979260
epoch: 27, train loss: 60.8761, lr: 0.000100, spent: 27.0 secs
epoch: 28, test val_loss: 5.979229
epoch: 28, train loss: 60.8761, lr: 0.000100, spent: 27.9 secs
epoch: 29, test val_loss: 5.979202
epoch: 29, train loss: 60.8752, lr: 0.000100, spent: 28.9 secs
epoch: 30, test val_loss: 5.979175
epoch: 30, train loss: 60.8755, lr: 0.000100, spent: 29.8 secs
epoch: 31, test val_loss: 5.979153
epoch: 31, train loss: 60.8744, lr: 0.000100, spent: 30.8 secs
epoch: 32, test val_loss: 5.979130
epoch: 32, train loss: 60.8744, lr: 0.000100, spent: 31.6 secs
epoch: 33, test val_loss: 5.979110
epoch: 33, train loss: 60.8743, lr: 0.000100, spent: 32.4 secs
epoch: 34, test val_loss: 5.979090
epoch: 34, train loss: 60.8736, lr: 0.000100, spent: 33.4 secs
epoch: 35, test val_loss: 5.979072
epoch: 35, train loss: 60.8720, lr: 0.000100, spent: 34.3 secs
epoch: 36, test val_loss: 5.979054
epoch: 36, train loss: 60.8724, lr: 0.000100, spent: 35.2 secs
epoch: 37, test val_loss: 5.979037
epoch: 37, train loss: 60.8735, lr: 0.000100, spent: 36.2 secs
epoch: 38, test val_loss: 5.979021
epoch: 38, train loss: 60.8723, lr: 0.000100, spent: 36.9 secs
epoch: 39, test val_loss: 5.979005
epoch: 39, train loss: 60.8726, lr: 0.000100, spent: 37.8 secs
epoch: 40, test val_loss: 5.978991
epoch: 40, train loss: 60.8719, lr: 0.000100, spent: 38.5 secs
epoch: 41, test val_loss: 5.978974
epoch: 41, train loss: 60.8720, lr: 0.000100, spent: 39.3 secs
epoch: 42, test val_loss: 5.978961
epoch: 42, train loss: 60.8717, lr: 0.000100, spent: 40.1 secs
epoch: 43, test val_loss: 5.978946
epoch: 43, train loss: 60.8721, lr: 0.000100, spent: 40.9 secs
epoch: 44, test val_loss: 5.978931
epoch: 44, train loss: 60.8710, lr: 0.000100, spent: 41.8 secs
epoch: 45, test val_loss: 5.978916
epoch: 45, train loss: 60.8711, lr: 0.000100, spent: 42.7 secs
epoch: 46, test val_loss: 5.978899
epoch: 46, train loss: 60.8713, lr: 0.000100, spent: 43.6 secs
epoch: 47, test val_loss: 5.978884
```

--------------------------------

### Get Input Embeddings from DataFrame

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/table_embedding.ipynb

Use the encoder to get input embeddings from a DataFrame. This snippet shows how to obtain the shape and the first two rows of the output embeddings.

```python
df = trainset[0]
output = enc(df)
print(output.shape)
output[:2]
```

--------------------------------

### Load Multiple Datasets and Build Contrastive Learner

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/example_pretrain.md

Loads multiple datasets and builds a contrastive learner model. Set supervised=True for supervised VPCL. The num_partition and overlap_ratio parameters control column partitioning for sampling.

```python
import transtab

# load multiple datasets by passing a list of data names
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data(['credit-g','credit-approval'])

# build contrastive learner, set supervised=True for supervised VPCL
model, collate_fn = transtab.build_contrastive_learner(
    cat_cols, num_cols, bin_cols,
    supervised=True, # if take supervised CL
    num_partition=4, # num of column partitions for pos/neg sampling
    overlap_ratio=0.5, # specify the overlap ratio of column partitions during the CL
)
```

--------------------------------

### Load Data and Build Contrastive Learner

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/example_encode.md

Loads a dataset and builds a contrastive learner model. Ensure the dataset name is valid and column types are correctly specified.

```python
import transtab

# load a dataset and start vanilla supervised training
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data('credit-g')

# build transtab classifier model
model, collate_fn = transtab.build_contrastive_learner(cat_cols, num_cols, bin_cols)

# start training
training_arguments = {
    'num_epoch':50,
    'batch_size':64,
    'lr':1e-4,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint'
    }
transtab.train(model, trainset, valset, collate_fn=collate_fn, **training_arguments)
```

--------------------------------

### Initialize Transtab with Random Seed

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Imports the Transtab library and sets a random seed for reproducible results.

```python
import transtab

# set random seed
transtab.random_seed(42)
```

--------------------------------

### Load and Display Tabular Data

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/fast_train.ipynb

This snippet shows how to load a tabular dataset and display its first few rows and columns. It's useful for initial data exploration.

```python
import pandas as pd

df = pd.read_csv("examples/fast_train.ipynb")
df.head()
```

--------------------------------

### Finetuning with Pretrained Model

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

This is a comment indicating the intention to use a pretrained model for finetuning on a new dataset. It serves as a placeholder for subsequent code.

```python
# now let's use another data and try to leverage the pretrained model for finetuning

```

--------------------------------

### Load Custom Dataset with Transtab

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/data_preparation.md

Use this command to load a custom dataset from a specified directory after preprocessing.

```python
transtab.load_data('./data/dataset1')
```

--------------------------------

### Define Training Arguments for Transtab

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Sets up a dictionary of arguments for the Transtab training function. This includes parameters like the number of epochs, evaluation metric, output directory, batch size, and learning rate.

```python
training_arguments = {
    'num_epoch':50,
    'eval_metric':'auc',
    'eval_less_is_better':False,
    'output_dir':'./checkpoint',
    'batch_size':128,
    'lr':2e-4,
    }
```

--------------------------------

### Build Contrastive Learner for Supervised Training

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/table_embedding.ipynb

Builds a contrastive learner model for TransTab, configured for supervised training (VPCL). Specifies parameters for column partitioning and overlap ratio.

```python
# make a fast pre-train of TransTab contrastive learning model
# build contrastive learner, set supervised=True for supervised VPCL
model, collate_fn = transtab.build_contrastive_learner(
    cat_cols, num_cols, bin_cols, 
    supervised=True, # if take supervised CL
    num_partition=4, # num of column partitions for pos/neg sampling
    overlap_ratio=0.5, # specify the overlap ratio of column partitions during the CL
)

```

--------------------------------

### Suppress Warnings and Change Directory

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Initializes the environment by filtering warnings and changing the current working directory.

```python
import warnings
warnings.filterwarnings("ignore")
import os
os.chdir('../')
```

--------------------------------

### Load Data and Train Classifier with Transtab

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Loads a dataset, builds a classifier model, and initiates training. This snippet is suitable for supervised training tasks. Ensure the dataset names are valid and training arguments are configured.

```python
# load a dataset and start vanilla supervised training
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols = transtab.load_data(['credit-g', 'credit-approval'])

# build transtab classifier model
model = transtab.build_classifier(cat_cols, num_cols, bin_cols)

# start training
training_arguments = {
    'num_epoch':50,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint',
    'batch_size':128,
    'lr':1e-4,
    'weight_decay':1e-4,
    }
transtab.train(model, trainset[0], valset[0], **training_arguments)

# save model
model.save('./ckpt/pretrained')
```

--------------------------------

### Load Dataset for Training

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/table_embedding.ipynb

Loads the 'credit-g' dataset and splits it into training, validation, and testing sets. Identifies categorical, numerical, and binary columns.

```python
# load a dataset and start vanilla supervised training
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data('credit-g')
```

--------------------------------

### Build and Train TransTab Contrastive Learner for Pretraining

Source: https://github.com/ryanwangzf/transtab/blob/main/blog/README.md

Build a contrastive learner for pretraining, optionally in a supervised manner. Specifies column partitions and overlap ratio for contrastive learning.

```python
import transtab

# load multiple datasets by passing a list of data names
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data(['credit-g','credit-approval'])

# build contrastive learner, set supervised=True for supervised VPCL
model, collate_fn = transtab.build_contrastive_learner(
    cat_cols, num_cols, bin_cols,
    supervised=True, # if take supervised CL
    num_partition=4, # num of column partitions for pos/neg sampling
    overlap_ratio=0.5, # specify the overlap ratio of column partitions during the CL
)

# start contrastive pretraining training
training_arguments = {
    'num_epoch':50,
    'batch_size':64,
    'lr':1e-4,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint' # save the pretrained model
    }

# pass the collate function to the train function
transtab.train(model, trainset, valset, collate_fn=collate_fn, **training_arguments)
```

--------------------------------

### Build Classifier from Pretrained Checkpoint

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/example_pretrain.md

Loads a pretrained model from a specified checkpoint directory and then builds a classifier. The model's categorical, numerical, and binary column dictionaries are updated with the new dataset's column information.

```python
# load the pretrained model and finetune on a target dataset
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data('credit-approval')

# build transtab classifier model, and load from the pretrained dir
model = transtab.build_classifier(checkpoint='./checkpoint')

# update model's categorical/numerical/binary column dict
model.update({'cat':cat_cols,'num':num_cols,'bin':bin_cols})
```

--------------------------------

### Prepare Training Data for Regression

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Formats the training data into a list of tuples, where each tuple contains a feature set and a randomly generated target series. This is useful for setting up regression tasks.

```python
trainset_reg = [(trainset[0][0], pd.Series(np.random.randn(trainset[0][0].shape[0]))), (trainset[1][0], pd.Series(np.random.randn(trainset[1][0].shape[0])))]
```

--------------------------------

### Load Data with Custom Dataset Configuration

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/transtab.load_data.md

Load a dataset with a custom configuration, allowing specification of columns, categorical, binary, and numerical types. This is useful for manipulating the input table directly during loading.

```python
# customize dataset configuration
dataset_config = {
    'credit-g':{
        'columns':['a','b','c'], # specify the new columns for the table, should keep the same dimension as the original table.
        'cat':['a'], # specify all the categorical columns
        'bin':['b'], # specify all the binary columns
        'num':['c']} # specify all the numerical columns
        }

allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data('credit-g', dataset_config=dataset_config)
```

--------------------------------

### Check PyPI Package Integrity

Source: https://github.com/ryanwangzf/transtab/blob/main/pypi_build_commands.txt

Before uploading, run this command to check your distribution files for common issues. It helps ensure your package meets PyPI's requirements.

```bash
twine check dist/*
```

--------------------------------

### Feature Extractor Loading and Adaptation

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Logs the loading of the feature extractor and the adaptation of the model to a new number of output classes. This indicates that the model is being prepared for fine-tuning.

```text
2024-03-08 16:33:11.468 | INFO | transtab.modeling_transtab:load:222 - load feature extractor from ./ckpt/pretrained/extractor/extractor.json
2024-03-08 16:33:11.470 | INFO | transtab.modeling_transtab:_adapt_to_new_num_class:886 - Build a new classifier with num 2 classes outputs, need further finetune to work.
```

--------------------------------

### Build Source Distribution and Wheel

Source: https://github.com/ryanwangzf/transtab/blob/main/pypi_build_commands.txt

Use this command to create both a source distribution (sdist) and a wheel (bdist_wheel) for your Python package. These are the standard formats for distributing Python packages.

```bash
python setup.py sdist bdist_wheel
```

--------------------------------

### Model Loading Information

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Provides information about the model loading process, indicating missing or unexpected keys and the path from which the model was loaded. This is useful for debugging loading issues.

```text
2024-03-08 16:33:11.448 | INFO | transtab.modeling_transtab:load:787 - missing keys: []
2024-03-08 16:33:11.448 | INFO | transtab.modeling_transtab:load:788 - unexpected keys: []
2024-03-08 16:33:11.449 | INFO | transtab.modeling_transtab:load:789 - load model from ./ckpt/pretrained
```

--------------------------------

### Load Data and Build Classifier

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/example_transfer.md

Loads a dataset and builds a Transtab classifier model. Ensure the dataset name and column types are correctly specified.

```python
import transtab

# load a dataset and start vanilla supervised training
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data('credit-g')

# build transtab classifier model
model = transtab.build_classifier(cat_cols, num_cols, bin_cols)

# start training
training_arguments = {
    'num_epoch':50,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint'
    }
transtab.train(model, trainset, valset, **training_arguments)
```

--------------------------------

### Epochs 12 and 13 Training Metrics with Early Stopping

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Logs the training metrics for epochs 12 and 13, including test AUC, training loss, learning rate, and time spent. It also indicates the first counter for early stopping, suggesting the model might be plateauing.

```text
epoch: 12, test auc: 0.896104
EarlyStopping counter: 1 out of 5
epoch: 12, train loss: 1.8219, lr: 0.000200, spent: 2.3 secs
epoch: 13, test auc: 0.903896
epoch: 13, train loss: 1.7924, lr: 0.000200, spent: 2.4 secs
```

--------------------------------

### Load Pretrained Model for Classification

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/contrastive_learning.ipynb

Loads a dataset and initializes a TransTab classifier, loading weights from a specified checkpoint directory. The model's column dictionary is then updated.

```python
# load the pretrained model and finetune on a target dataset
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
     = transtab.load_data('credit-approval')

# build transtab classifier model, and load from the pretrained dir
model = transtab.build_classifier(checkpoint='./checkpoint')

# update model's categorical/numerical/binary column dict
model.update({'cat':cat_cols,'num':num_cols,'bin':bin_cols})
```

--------------------------------

### Contrastive Pretraining on Multiple Tables

Source: https://github.com/ryanwangzf/transtab/blob/main/README.md

Conduct contrastive pretraining on multiple distinct tables. Set supervised=True for supervised VPCL. The build_contrastive_learner function returns both the model and a collate_fn.

```python
# load from multiple tabular datasets
dataname_list = ['credit-g', 'credit-approval']
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
     = transtab.load_data(dataname_list)

# build contrastive learner, set supervised=True for supervised VPCL
model, collate_fn = transtab.build_contrastive_learner(
    cat_cols, num_cols, bin_cols, supervised=True)

# start contrastive pretraining training
transtab.train(model, trainset, valset, collate_fn=collate_fn, **training_arguments)
```

--------------------------------

### Load Data and Build Classifier

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/fast_train.md

Loads multiple datasets and builds a TransTab classifier model. Ensure datasets are compatible and specify categorical, numerical, and binary columns.

```python
import transtab

# load multiple datasets by passing a list of data names
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data(['credit-g','credit-approval'])

# build transtab classifier model
model = transtab.build_classifier(cat_cols, num_cols, bin_cols)

# specify training arguments, take validation loss for early stopping
training_arguments = {
    'num_epoch':5,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint'
    }
```

--------------------------------

### Training Completion and Model Saving

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/contrastive_learning.ipynb

Logs indicate the completion of training, saving of the best model checkpoint, and the total time taken for the training process.

```text
2022-08-31 10:56:45.227 | INFO     | transtab.trainer:train:132 - load best at last from ./checkpoint
2022-08-31 10:56:45.242 | INFO     | transtab.trainer:save_model:239 - saving model checkpoint to ./checkpoint
2022-08-31 10:56:45.379 | INFO     | transtab.trainer:train:137 - training complete, cost 27.2 secs.
```

--------------------------------

### Training Completion and Checkpoint Saving

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Logs messages indicating the loading of the best model checkpoint and the saving of a new checkpoint. It also confirms the completion of the training process and the total time taken.

```text
2022-10-05 08:35:04.023 | INFO     | transtab.trainer:train:136 - load best at last from ./checkpoint
2022-10-05 08:35:04.042 | INFO     | transtab.trainer:save_model:243 - saving model checkpoint to ./checkpoint
2022-10-05 08:35:04.167 | INFO     | transtab.trainer:train:141 - training complete, cost 13.1 secs.

```

--------------------------------

### Load Best Checkpoint and Predict

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/fast_train.md

Loads the best model checkpoint based on validation loss and makes predictions on test data. Ensure the 'output_dir' in training_arguments matches the path used for loading.

```python
model.load('./checkpoint')

x_test, y_test = testset[0]

ypred = transtab.predict(x_test)
```

--------------------------------

### Load and Prepare Data for Transtab

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/fast_train.ipynb

Loads multiple datasets by providing a list of data names. It also builds a classifier model and specifies training arguments, including early stopping based on validation loss.

```python
# load multiple datasets by passing a list of data names
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data(['credit-g','credit-approval'])

# build transtab classifier model
model = transtab.build_classifier(cat_cols, num_cols, bin_cols)

# specify training arguments, take validation loss for early stopping
training_arguments = {
    'num_epoch':50,
    'batch_size':128,
    'lr':1e-4,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint'
    }
```

--------------------------------

### Epochs 8 and 9 Training Metrics

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Logs the training metrics for epochs 8 and 9, including test AUC, training loss, learning rate, and time spent. This output continues the detailed monitoring of the model's training progress.

```text
epoch: 8, test auc: 0.889177
epoch: 8, train loss: 1.8919, lr: 0.000200, spent: 1.5 secs
epoch: 9, test auc: 0.890909
epoch: 9, train loss: 1.8794, lr: 0.000200, spent: 1.7 secs
```

--------------------------------

### Build and Train TransTab Regressor

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Builds a TransTab regressor model and initiates the training process with specified arguments. Ensure all column definitions (cat_cols, num_cols, bin_cols) and dataset splits (trainset_reg, valset_reg) are correctly provided.

```python
# build transtab classifier model
model = transtab.build_regressor(cat_cols, num_cols, bin_cols, device='cpu')

# start training
training_arguments = {
    'num_epoch':50,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint',
    'batch_size':128,
    'lr':1e-4,
    'weight_decay':1e-4,
    }
transtab.train(model, trainset_reg[0], valset_reg[0], **training_arguments)

# save model
model.save('./ckpt/pretrained')
```

--------------------------------

### Further Training Epoch and Metric Output

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Logs the training metrics for epochs 4 and 5, including test AUC, training loss, learning rate, and time spent. This output continues the detailed monitoring of the training process.

```text
epoch: 4, test auc: 0.872727
epoch: 4, train loss: 2.2617, lr: 0.000200, spent: 1.0 secs
epoch: 5, test auc: 0.879654
epoch: 5, train loss: 2.0867, lr: 0.000200, spent: 1.1 secs
```

--------------------------------

### Epochs 10 and 11 Training Metrics

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Presents the training metrics for epochs 10 and 11, detailing test AUC, training loss, learning rate, and time spent. This output is part of the continuous monitoring of the model's training progress.

```text
epoch: 10, test auc: 0.896970
epoch: 10, train loss: 1.8456, lr: 0.000200, spent: 2.0 secs
epoch: 11, test auc: 0.897835
epoch: 11, train loss: 1.8213, lr: 0.000200, spent: 2.2 secs
```

--------------------------------

### Load OpenML Dataset Information

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/contrastive_learning.ipynb

Displays information about a loaded OpenML dataset, including its index, name, dimensions, and the count of categorical, binary, and numerical features.

```log
openml data index: 29
load data from credit-approval
# data: 690, # feat: 15, # cate: 9,  # bin: 0, # numerical: 6, pos rate: 0.56
```

--------------------------------

### Load Multiple Datasets

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/transtab.load_data.md

Load multiple datasets by providing a list of local directory paths. This function returns all sets (all, train, validation, test) along with column type information.

```python
# specify the dataset dir
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data(['./data/credit-g','./data/credit-approval'])
```

--------------------------------

### Finalize Training and Save Model

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/contrastive_learning.ipynb

Logs indicating the completion of the training process, loading the best model checkpoint, and saving the final model to the specified output directory.

```log
2022-08-31 10:56:53.974 | INFO     | transtab.trainer:train:132 - load best at last from ./checkpoint
2022-08-31 10:56:54.000 | INFO     | transtab.trainer:save_model:239 - saving model checkpoint to ./checkpoint
2022-08-31 10:56:54.130 | INFO     | transtab.trainer:train:137 - training complete, cost 5.6 secs.
```

--------------------------------

### Training Epoch and Metric Output

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Logs the performance metrics for each training epoch, including the test AUC, training loss, learning rate, and time spent. This helps in monitoring model convergence and performance.

```text
epoch: 0, test auc: 0.282251
epoch: 0, train loss: 3.3862, lr: 0.000200, spent: 0.2 secs
epoch: 1, test auc: 0.865801
epoch: 1, train loss: 2.8794, lr: 0.000200, spent: 0.3 secs
```

--------------------------------

### Load Datasets with Transtab

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Loads multiple datasets from OpenML and separates them into training, validation, and testing sets. It also identifies categorical, numerical, and binary columns.

```python
# !pip install openml
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols = transtab.load_data(['credit-g', 'credit-approval'])
```

--------------------------------

### Load Pretrained Model and Update Columns

Source: https://github.com/ryanwangzf/transtab/blob/main/docs/source/example_transfer.md

Loads a previously saved Transtab model and updates its column definitions for finetuning on a new dataset. The model is loaded from the specified checkpoint directory.

```python
# now let's load another data and try to leverage the pretrained model for finetuning
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data('credit-approval')

# load the pretrained model
model.load('./checkpoint')

# update model's categorical/numerical/binary column dict
model.update({'cat':cat_cols,'num':num_cols,'bin':bin_cols})
```

--------------------------------

### Epochs 6 and 7 Training Metrics

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning.ipynb

Presents the training metrics for epochs 6 and 7, detailing test AUC, training loss, learning rate, and time spent. This output is part of the continuous monitoring of the model's training progress.

```text
epoch: 6, test auc: 0.880519
epoch: 6, train loss: 1.9774, lr: 0.000200, spent: 1.3 secs
epoch: 7, test auc: 0.883117
epoch: 7, train loss: 1.8739, lr: 0.000200, spent: 1.4 secs
```

--------------------------------

### Build and Train TransTab Classifier

Source: https://github.com/ryanwangzf/transtab/blob/main/blog/README.md

Load multiple datasets, build a TransTab classifier model, and train it using specified arguments. Uses validation loss for early stopping.

```python
import transtab

# load multiple datasets by passing a list of data names
allset, trainset, valset, testset, cat_cols, num_cols, bin_cols \
    = transtab.load_data(['credit-g','credit-approval'])

# build transtab classifier model
model = transtab.build_classifier(cat_cols, num_cols, bin_cols)

# specify training arguments, take validation loss for early stopping
training_arguments = {
    'num_epoch':5,
    'eval_metric':'val_loss',
    'eval_less_is_better':True,
    'output_dir':'./checkpoint'
    }

# start training
transtab.train(model, trainset, valset[0], **training_arguments)
```

--------------------------------

### Prepare Validation and Test Data for Regression

Source: https://github.com/ryanwangzf/transtab/blob/main/examples/transfer_learning_regressor.ipynb

Formats validation and test data into lists of tuples, each containing a feature set and a randomly generated target series. This prepares the data for regression model evaluation.

```python
valset_reg = [(valset[0][0], pd.Series(np.random.randn(valset[0][0].shape[0]))), (valset[1][0], pd.Series(np.random.randn(valset[1][0].shape[0])))]
testset_reg = [(testset[0][0], pd.Series(np.random.randn(testset[0][0].shape[0]))), (testset[1][0], pd.Series(np.random.randn(testset[1][0].shape[0])))]
```