Ensemble PyTorch

https://ensemble-pytorch.readthedocs.io/en/latest/
Admin
Ensemble PyTorch is a unified ensemble framework for PyTorch that provides easy-to-use APIs and high...
Tokens:34,701
Snippets:349
Trust Score:9.6
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install Ensemble-PyTorch

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

Install the stable version of Ensemble-PyTorch using pip. It has minimal dependencies and works well with PyTorch installed via Anaconda.

```bash
$ pip install torchensemble
```

--------------------------------

### Launch TensorBoard Logger

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

Start TensorBoard to visualize training and evaluation metrics. Ensure logs are directed to the 'logs/' directory.

```bash
tensorboard --logdir=logs/

```

--------------------------------

### Set Ensemble Optimizer

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

Configure the optimizer for training the ensemble model. This example uses the Adam optimizer with specified learning rate and weight decay.

```python
model.set_optimizer('Adam',             # parameter optimizer
                    lr=1e-3,            # learning rate of the optimizer
                    weight_decay=5e-4)  # weight decay of the optimizer

```

--------------------------------

### MNIST Classification with VotingClassifier

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

This example demonstrates training a VotingClassifier with 10 MLPs on the MNIST dataset. It includes model definition, data loading, training, and evaluation.

```python
import torch
import torch.nn as nn
from torch.nn import functional as F
from torchvision import datasets, transforms

from torchensemble import VotingClassifier
from torchensemble.utils.logging import set_logger

# Define Your Base Estimator
class MLP(nn.Module):

    def __init__(self):
        super(MLP, self).__init__()
        self.linear1 = nn.Linear(784, 128)
        self.linear2 = nn.Linear(128, 128)
        self.linear3 = nn.Linear(128, 10)

    def forward(self, data):
        data = data.view(data.size(0), -1)
        output = F.relu(self.linear1(data))
        output = F.relu(self.linear2(output))
        output = self.linear3(output)
        return output

# Load MNIST dataset
transform=transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
    ])

train = datasets.MNIST('../Dataset', train=True, download=True, transform=transform)
test = datasets.MNIST('../Dataset', train=False, transform=transform)
train_loader = torch.utils.data.DataLoader(train, batch_size=128, shuffle=True)
test_loader = torch.utils.data.DataLoader(test, batch_size=128, shuffle=True)

# Set the Logger
logger = set_logger('classification_mnist_mlp')

# Define the ensemble
model = VotingClassifier(
    estimator=MLP,
    n_estimators=10,
    cuda=True,
)

# Set the criterion
criterion = nn.CrossEntropyLoss()
model.set_criterion(criterion)

# Set the optimizer
model.set_optimizer('Adam', lr=1e-3, weight_decay=5e-4)

# Train and Evaluate
model.fit(
    train_loader,
    epochs=50,
    test_loader=test_loader,
)

```

--------------------------------

### Define a Multi-Layered Perceptron (MLP) Base Estimator

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

Define a custom neural network model inheriting from torch.nn.Module. Ensure it implements __init__ and forward methods. This example shows an MLP with specific layer sizes.

```python
import torch.nn as nn
from torch.nn import functional as F

class MLP(nn.Module):

    def __init__(self):
        super(MLP, self).__init__()
        self.linear1 = nn.Linear(784, 128)
        self.linear2 = nn.Linear(128, 128)
        self.linear3 = nn.Linear(128, 10)

    def forward(self, data):
        data = data.view(data.size(0), -1)  # flatten
        output = F.relu(self.linear1(data))
        output = F.relu(self.linear2(output))
        output = self.linear3(output)
        return output

```

--------------------------------

### predict

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Get predictions from the ensemble for given input data.

```APIDOC
## predict

### Description

Return the predictions of the ensemble given the testing data.

### Parameters

- **X** (tensor, numpy array) - A data batch in the form of tensor or numpy array.

### Returns

- **pred** - For classifiers, `n_outputs` is the number of distinct classes. For regressors, `n_output` is the number of target variables.

### Return type

tensor of shape (n_samples, n_outputs)
```

--------------------------------

### Set Up a Global Logger

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

Configure a global logger for tracking and printing intermediate logging information. Logs are saved to a text file and can optionally be visualized with TensorBoard.

```python
from torchensemble.utils.logging import set_logger

logger = set_logger('classification_mnist_mlp')

```

--------------------------------

### forward

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Performs a forward pass through the SoftGradientBoostingClassifier to get predictions.

```APIDOC
## forward

### Description
Performs a forward pass through the SoftGradientBoostingClassifier to get predictions.

### Method
`forward(*x)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **X** (tensor) - Required - An input batch of data, which should be a valid input data batch for base estimators in the ensemble.

### Request Example
```python
import torch

# Assuming 'classifier' is an instance of SoftGradientBoostingClassifier and is fitted
# Assuming 'sample_data' is a tensor of input data
sample_data = torch.randn(64, 784) # Example input tensor
predictions = classifier.forward(sample_data)
print(predictions.shape) # Output shape: torch.Size([64, n_classes])
```

### Response
#### Success Response (200)
- **proba** (tensor) - The predicted class distribution.

#### Response Example
```json
{
  "example": "tensor of shape (batch_size, n_classes)"
}
```
```

--------------------------------

### GradientBoostingClassifier Initialization

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the GradientBoostingClassifier with base estimators and ensemble parameters.

```APIDOC
## GradientBoostingClassifier

### Description
Implementation on the GradientBoostingClassifier.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **estimator** (torch.nn.Module) - Required - The class or object of your base estimator. If `class`, it should inherit from `torch.nn.Module`. If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.
* **n_estimators** (int) - Required - The number of base estimators in the ensemble.
* **estimator_args** (dict, default=None) - Optional - The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.
* **shrinkage_rate** (float, default=1) - Optional - The shrinkage rate used in gradient boosting.
* **cuda** (bool, default=True) - Optional - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU to train and evaluate the ensemble.
```

--------------------------------

### forward

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Performs a forward pass through the AdversarialTrainingClassifier to get predictions.

```APIDOC
## forward

### Description
Performs a forward pass through the AdversarialTrainingClassifier to get predictions.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **X** (tensor) - Required - An input batch of data, which should be a valid input data batch for base estimators in the ensemble.

### Request Example
None

### Response
#### Success Response (200)
* **proba** (tensor) - The predicted class distribution. Shape: (batch_size, n_classes)

#### Response Example
None
```

--------------------------------

### SoftGradientBoostingClassifier Initialization

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the SoftGradientBoostingClassifier with specified base estimators and ensemble parameters.

```APIDOC
## SoftGradientBoostingClassifier

### Description
Initializes the SoftGradientBoostingClassifier with specified base estimators and ensemble parameters.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **estimator** (torch.nn.Module) - Required - The class or object of your base estimator. If `class`, it should inherit from `torch.nn.Module`. If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.
- **n_estimators** (int) - Required - The number of base estimators in the ensemble.
- **estimator_args** (dict, default=None) - Optional - The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.
- **shrinkage_rate** (float, default=1) - Optional - The shrinkage rate used in gradient boosting.
- **cuda** (bool, default=True) - Optional - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU to train and evaluate the ensemble.
- **n_jobs** (int, default=None) - Optional - The number of workers for training the ensemble. This input argument is used for parallel ensemble methods such as `voting` and `bagging`. Setting it to an integer larger than `1` enables `n_jobs` base estimators to be trained simultaneously.

### Request Example
```python
from torch import nn
from ensemble_pytorch import SoftGradientBoostingClassifier

# Example usage with a simple MLP
class SimpleMLP(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super().__init__()
        self.layer_1 = nn.Linear(input_dim, hidden_dim)
        self.layer_2 = nn.Linear(hidden_dim, output_dim)

    def forward(self, x):
        x = torch.relu(self.layer_1(x))
        return self.layer_2(x)

# Instantiate the classifier
classifier = SoftGradientBoostingClassifier(
    estimator=SimpleMLP,
    n_estimators=10,
    estimator_args={'input_dim': 784, 'hidden_dim': 128, 'output_dim': 10},
    shrinkage_rate=0.1,
    cuda=True,
    n_jobs=2
)
```

### Response
None
```

--------------------------------

### forward

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Performs a forward pass through the ensemble to get predictions.

```APIDOC
## forward

### Description
Implementation on the data forwarding in SnapshotEnsembleClassifier.

### Parameters
- **X** (tensor) - An input batch of data, which should be a valid input data batch for base estimators in the ensemble.

### Returns
- **proba** (tensor of shape (batch_size, n_classes)) - The predicted class distribution.
```

--------------------------------

### FusionClassifier.forward

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Performs the data forwarding in FusionClassifier to get predictions.

```APIDOC
## forward

### Description
Implementation on the data forwarding in FusionClassifier.

### Parameters

#### `X` (tensor)
- Required - An input batch of data, which should be a valid input data batch for base estimators in the ensemble.

### Returns

- **proba** (tensor of shape (batch_size, n_classes)) - The predicted class distribution.
```

--------------------------------

### fit

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Trains the SoftGradientBoostingRegressor.

```APIDOC
## fit

### Description
Implementation on the training stage of SoftGradientBoostingRegressor.

### Parameters
- **train_loader** (torch.utils.data.DataLoader) - A data loader that contains the training data.
- **epochs** (int, default=100) - The number of training epochs per base estimator.
- **use_reduction_sum** (bool, default=True) - Whether to set `reduction="sum"` for the internal mean squared error used to fit each base estimator.
- **log_interval** (int, default=100) - The number of batches to wait before logging the training status.
- **test_loader** (torch.utils.data.DataLoader, default=None) - A data loader that contains the evaluating data. If `None`, no validation is conducted after each base estimator being trained. If not `None`, the ensemble will be evaluated on this dataloader after each base estimator being trained.
- **save_model** (bool, default=True) - Specify whether to save the model parameters. If test_loader is `None`, the ensemble containing `n_estimators` base estimators will be saved. If test_loader is not `None`, the ensemble with the best validation performance will be saved.
- **save_dir** (string, default=None) - Specify where to save the model parameters. If `None`, the model will be saved in the current directory. If not `None`, the model will be saved in the specified directory: `save_dir`.
```

--------------------------------

### forward

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Performs data forwarding in NeuralForestClassifier to get predicted class distributions.

```APIDOC
## forward

### Description
Performs data forwarding in NeuralForestClassifier to get predicted class distributions.

### Parameters
- **X** (tensor) - An input batch of data, which should be a valid input data batch for base estimators in the ensemble.

### Returns
- **proba** (tensor) - The predicted class distribution.
  - Type: tensor of shape (batch_size, n_classes)
```

--------------------------------

### fit()

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Training stage of the ensemble. This method initiates the training process for the ensemble model.

```APIDOC
## fit()

### Description
Training stage of the ensemble.

### Method
Not specified (assumed to be a method call within the library)

### Endpoint
Not applicable (SDK method)

### Parameters
None explicitly listed in the source.

### Request Example
```python
ensemble.fit()
```

### Response
None explicitly listed in the source.
```

--------------------------------

### Set Ensemble Criterion

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

Define and set the objective function for the ensemble model. This example uses CrossEntropyLoss for classification tasks.

```python
criterion = nn.CrossEntropyLoss()
model.set_criterion(criterion)

```

--------------------------------

### fit Method

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Trains the GradientBoostingRegressor model. It takes data loaders, training epochs, and optional parameters for logging, validation, early stopping, and model saving.

```APIDOC
## fit Method

### Description
Implementation on the training stage of GradientBoostingRegressor.

### Parameters
* **train_loader** (torch.utils.data.DataLoader) - A data loader that contains the training data.
* **epochs** (int, default=100) - The number of training epochs per base estimator.
* **use_reduction_sum** (bool, default=True) - Whether to set `reduction="sum"` for the internal mean squared error used to fit each base estimator.
* **log_interval** (int, default=100) - The number of batches to wait before logging the training status.
* **test_loader** (torch.utils.data.DataLoader, default=None) - A data loader that contains the evaluating data. If `None`, no validation is conducted after each base estimator being trained. If not `None`, the ensemble will be evaluated on this dataloader after each base estimator being trained.
* **early_stopping_rounds** (int, default=2) - Specify the number of tolerant rounds for early stopping. When the validation performance of the ensemble does not improve after adding the base estimator fitted in current iteration, the internal counter on early stopping will increase by one. When the value of the internal counter reaches `early_stopping_rounds`, the training stage will terminate instantly.
* **save_model** (bool, default=True) - Specify whether to save the model parameters. If test_loader is `None`, the ensemble containing `n_estimators` base estimators will be saved. If test_loader is not `None`, the ensemble with the best validation performance will be saved.
* **save_dir** (string, default=None) - Specify where to save the model parameters. If `None`, the model will be saved in the current directory. If not `None`, the model will be saved in the specified directory: `save_dir`.
```

--------------------------------

### Train and Evaluate a Voting Classifier Ensemble

Source: https://ensemble-pytorch.readthedocs.io/en/latest/index.html

Demonstrates how to set up, train, and evaluate a VotingClassifier using PyTorch. This includes loading data, defining the ensemble, setting the loss criterion, optimizer, and learning rate scheduler, followed by fitting the model and making predictions.

```python
from torchensemble import VotingClassifier  # voting is a classic ensemble strategy

# Load data
train_loader = DataLoader(...)
test_loader = DataLoader(...)

# Define the ensemble
ensemble = VotingClassifier(
    estimator=base_estimator,               # here is your deep learning model
    n_estimators=10,                        # number of base estimators
)
# Set the criterion
criterion = nn.CrossEntropyLoss()           # training objective
ensemble.set_criterion(criterion)

# Set the optimizer
ensemble.set_optimizer(
    "Adam",                                 # type of parameter optimizer
    lr=learning_rate,                       # learning rate of parameter optimizer
    weight_decay=weight_decay,              # weight decay of parameter optimizer
)

# Set the learning rate scheduler
ensemble.set_scheduler(
    "CosineAnnealingLR",                    # type of learning rate scheduler
    T_max=epochs,                           # additional arguments on the scheduler
)

# Train the ensemble
ensemble.fit(
    train_loader,
    epochs=epochs,                          # number of training epochs
)

# Evaluate the ensemble
acc = ensemble.predict(test_loader)         # testing accuracy

```

--------------------------------

### SoftGradientBoostingRegressor

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the SoftGradientBoostingRegressor with specified parameters.

```APIDOC
## SoftGradientBoostingRegressor

### Description
Implementation of the SoftGradientBoostingRegressor.

### Parameters
- **estimator** (torch.nn.Module) - The class or object of your base estimator. If `class`, it should inherit from `torch.nn.Module`. If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.
- **n_estimators** (int) - The number of base estimators in the ensemble.
- **estimator_args** (dict, default=None) - The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.
- **shrinkage_rate** (float, default=1) - The shrinkage rate used in gradient boosting.
- **cuda** (bool, default=True) - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU to train and evaluate the ensemble.
- **n_jobs** (int, default=None) - The number of workers for training the ensemble. This input argument is used for parallel ensemble methods such as `voting` and `bagging`. Setting it to an integer larger than `1` enables `n_jobs` base estimators to be trained simultaneously.
```

--------------------------------

### FastGeometricClassifier

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the FastGeometricClassifier with a base estimator, number of estimators, and configuration for training.

```APIDOC
## FastGeometricClassifier

### Description
Initializes the FastGeometricClassifier with a base estimator, number of estimators, and configuration for training.

### Parameters
#### Parameters
- **estimator** (torch.nn.Module) - The class or object of your base estimator. If `class`, it should inherit from `torch.nn.Module`. If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.
- **n_estimators** (int) - The number of base estimators in the ensemble.
- **estimator_args** (dict, default=None) - The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.
- **cuda** (bool, default=True) - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU to train and evaluate the ensemble.
```

--------------------------------

### fit Method

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Trains the GradientBoostingClassifier model.

```APIDOC
## fit

### Description
Implementation on the training stage of GradientBoostingClassifier.

### Method
POST (assumed, as it modifies state)

### Endpoint
`/GradientBoostingClassifier/fit` (assumed)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **train_loader** (torch.utils.data.DataLoader) - Required - A data loader that contains the training data.
* **epochs** (int, default=100) - Optional - The number of training epochs per base estimator.
* **use_reduction_sum** (bool, default=True) - Optional - Whether to set `reduction="sum"` for the internal mean squared error used to fit each base estimator.
* **log_interval** (int, default=100) - Optional - The number of batches to wait before logging the training status.
* **test_loader** (torch.utils.data.DataLoader, default=None) - Optional - A data loader that contains the evaluating data. If `None`, no validation is conducted after each base estimator being trained. If not `None`, the ensemble will be evaluated on this dataloader after each base estimator being trained.
* **early_stopping_rounds** (int, default=2) - Optional - Specify the number of tolerant rounds for early stopping. When the validation performance of the ensemble does not improve after adding the base estimator fitted in current iteration, the internal counter on early stopping will increase by one. When the value of the internal counter reaches `early_stopping_rounds`, the training stage will terminate instantly.
* **save_model** (bool, default=True) - Optional - Specify whether to save the model parameters. If test_loader is `None`, the ensemble containing `n_estimators` base estimators will be saved. If test_loader is not `None`, the ensemble with the best validation performance will be saved.
* **save_dir** (string, default=None) - Optional - Specify where to save the model parameters. If `None`, the model will be saved in the current directory. If not `None`, the model will be saved in the specified directory: `save_dir`.
```

--------------------------------

### evaluate Method

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Evaluates the performance of the GradientBoostingClassifier.

```APIDOC
## evaluate

### Description
Compute the classification accuracy of the ensemble given the testing dataloader and optionally the average cross-entropy loss.

### Method
POST (assumed, as it performs computation)

### Endpoint
`/GradientBoostingClassifier/evaluate` (assumed)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **test_loader** (torch.utils.data.DataLoader) - Required - A data loader that contains the testing data.
* **return_loss** (bool, default=False) - Optional - Whether to return the average cross-entropy loss.
```

--------------------------------

### forward Method

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Performs a forward pass through the GradientBoostingRegressor model to obtain predictions.

```APIDOC
## forward Method

### Description
Implementation on the data forwarding in GradientBoostingRegressor.

### Parameters
* **X** (tensor) - An input batch of data, which should be a valid input data batch for base estimators in the ensemble.

### Returns
* **pred** - The predicted values.

### Return type
tensor of shape (batch_size, n_outputs)
```

--------------------------------

### GradientBoostingClassifier

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Configuration options for the GradientBoostingClassifier.

```APIDOC
## GradientBoostingClassifier

### Parameters

- **return_loss** (bool, default=False) - Whether to return the average cross-entropy loss over all batches in the `test_loader`.

### Returns

- **accuracy** (float) - The classification accuracy of the fitted ensemble on `test_loader`.
- **loss** (float) - The average cross-entropy loss of the fitted ensemble on `test_loader`, only available when `return_loss` is True.
```

--------------------------------

### NeuralForestRegressor Class

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the NeuralForestRegressor with specified parameters.

```APIDOC
## NeuralForestRegressor

### Description
Implementation of the NeuralForestRegressor, a neural network-based ensemble regressor.

### Parameters
* **n_estimators** (int) - The number of neural trees in the ensemble.
* **depth** (int, default=5) - The depth of the neural tree. A tree with depth `d` has 2^d leaf nodes and 2^(d-1) internal nodes.
* **lamda** (float, default=1e-3) - The coefficient of the regularization term when training neural trees, as proposed in the paper: 'Distilling a neural network into a soft decision tree'.
* **cuda** (bool, default=True) - If True, use GPU for training and evaluation. If False, use CPU.
* **n_jobs** (int, default=None) - The number of workers for training the ensemble. Used for parallel ensemble methods like 'voting' and 'bagging'. Setting to an integer greater than 1 enables simultaneous training of `n_jobs` base estimators.
```

--------------------------------

### forward Method

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Performs a forward pass through the GradientBoostingClassifier model.

```APIDOC
## forward

### Description
Implementation on the data forwarding in GradientBoostingClassifier.

### Method
POST (assumed, as it performs computation)

### Endpoint
`/GradientBoostingClassifier/forward` (assumed)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **X** (tensor) - Required - An input batch of data, which should be a valid input data batch for base estimators in the ensemble.
```

--------------------------------

### VotingClassifier Initialization

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the VotingClassifier with base estimators, number of estimators, and configuration for GPU usage and parallel processing.

```APIDOC
## VotingClassifier Constructor

### Description
Initializes the VotingClassifier with base estimators, number of estimators, and configuration for GPU usage and parallel processing.

### Parameters

#### `estimator` (torch.nn.Module)
- The class or object of your base estimator.
- If `class`, it should inherit from `torch.nn.Module`.
- If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.

#### `n_estimators` (int)
- The number of base estimators in the ensemble.

#### `estimator_args` (dict, default=None)
- The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.

#### `cuda` (bool, default=True)
- If `True`, use GPU to train and evaluate the ensemble.
- If `False`, use CPU to train and evaluate the ensemble.

#### `n_jobs` (int, default=None)
- The number of workers for training the ensemble. This input argument is used for parallel ensemble methods such as `voting` and `bagging`. Setting it to an integer larger than `1` enables `n_jobs` base estimators to be trained simultaneously.
```

--------------------------------

### NeuralForestClassifier

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes a NeuralForestClassifier with specified ensemble parameters.

```APIDOC
## NeuralForestClassifier

### Description
Initializes a NeuralForestClassifier with specified ensemble parameters.

### Parameters
- **n_estimators** (int) - The number of neural trees in the ensemble.
- **depth** (int, default=5) - The depth of neural tree. A tree with depth `d` is with 2d leaf nodes and 2d−1 internal nodes.
- **lamda** (float, default=1e-3) - The coefficient of the regularization term when training neural trees.
- **cuda** (bool, default=True) - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU.
- **n_jobs** (int, default=None) - The number of workers for training the ensemble. Used for parallel ensemble methods.
```

--------------------------------

### fit

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Trains the SoftGradientBoostingClassifier using the provided training data.

```APIDOC
## fit

### Description
Trains the SoftGradientBoostingClassifier using the provided training data.

### Method
`fit(train_loader, epochs=100, use_reduction_sum=True, log_interval=100, test_loader=None, save_model=True, save_dir=None, on_epoch_end_cb=None)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **train_loader** (torch.utils.data.DataLoader) - Required - A data loader that contains the training data.
- **epochs** (int, default=100) - Optional - The number of training epochs per base estimator.
- **use_reduction_sum** (bool, default=True) - Optional - Whether to set `reduction="sum"` for the internal mean squared error used to fit each base estimator.
- **log_interval** (int, default=100) - Optional - The number of batches to wait before logging the training status.
- **test_loader** (torch.utils.data.DataLoader, default=None) - Optional - A data loader that contains the evaluating data. If `None`, no validation is conducted after each base estimator being trained. If not `None`, the ensemble will be evaluated on this dataloader after each base estimator being trained.
- **save_model** (bool, default=True) - Optional - Specify whether to save the model parameters. If test_loader is `None`, the ensemble containing `n_estimators` base estimators will be saved. If test_loader is not `None`, the ensemble with the best validation performance will be saved.
- **save_dir** (string, default=None) - Optional - Specify where to save the model parameters. If `None`, the model will be saved in the current directory. If not `None`, the model will be saved in the specified directory: `save_dir`.
- **on_epoch_end_cb** (callable, default=None) - Optional - A callback function to be executed at the end of each epoch.

### Request Example
```python
# Assuming 'classifier' is an instance of SoftGradientBoostingClassifier
# and 'train_loader' and 'test_loader' are DataLoader instances
classifier.fit(train_loader, epochs=50, test_loader=test_loader, save_dir='./models')
```

### Response
None
```

--------------------------------

### FusionClassifier Initialization

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the FusionClassifier with base estimators and ensemble configuration.

```APIDOC
## FusionClassifier

### Description
Initializes the FusionClassifier with base estimators and ensemble configuration.

### Parameters

#### `estimator` (torch.nn.Module)
- Required - The class or object of your base estimator. If `class`, it should inherit from `torch.nn.Module`. If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.

#### `n_estimators` (int)
- Required - The number of base estimators in the ensemble.

#### `estimator_args` (dict, default=None)
- Optional - The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.

#### `cuda` (bool, default=True)
- Optional - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU to train and evaluate the ensemble.

#### `n_jobs` (int, default=None)
- Optional - The number of workers for training the ensemble. This input argument is used for parallel ensemble methods such as `voting` and `bagging`. Setting it to an integer larger than `1` enables `n_jobs` base estimators to be trained simultaneously.
```

--------------------------------

### Initialize VotingClassifier Ensemble

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

Wrap a base estimator (e.g., MLP) with the VotingClassifier. Specify the number of estimators and whether to use CUDA for acceleration.

```python
from torchensemble import VotingClassifier

model = VotingClassifier(
    estimator=MLP,
    n_estimators=10,
    cuda=True,
)

```

--------------------------------

### evaluate

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Evaluates the SoftGradientBoostingRegressor on a test dataset.

```APIDOC
## evaluate

### Description
Compute the mean squared error (MSE) of the ensemble given the testing dataloader.

### Parameters
- **test_loader** (torch.utils.data.DataLoader) - A data loader that contains the testing data.

### Returns
- **mse** - The testing mean squared error of the fitted ensemble on `test_loader`.

### Return type
float
```

--------------------------------

### FastGeometricRegressor Class

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the FastGeometricRegressor with specified base estimators and training configurations.

```APIDOC
## FastGeometricRegressor Class

### Description
Initializes the FastGeometricRegressor with specified base estimators and training configurations.

### Parameters
- **estimator** (torch.nn.Module) - The class or object of your base estimator. If `class`, it should inherit from `torch.nn.Module`. If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.
- **n_estimators** (int) - The number of base estimators in the ensemble.
- **estimator_args** (dict, default=None) - The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.
- **cuda** (bool, default=True) - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU to train and evaluate the ensemble.
```

--------------------------------

### AdversarialTrainingClassifier

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the AdversarialTrainingClassifier with base estimator, ensemble size, and training configurations.

```APIDOC
## AdversarialTrainingClassifier

### Description
Initializes the AdversarialTrainingClassifier with base estimator, ensemble size, and training configurations.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **estimator** (torch.nn.Module) - Required - The class or object of your base estimator. If `class`, it should inherit from `torch.nn.Module`. If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.
* **n_estimators** (int) - Required - The number of base estimators in the ensemble.
* **estimator_args** (dict, default=None) - Optional - The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.
* **cuda** (bool, default=True) - Optional - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU to train and evaluate the ensemble.
* **n_jobs** (int, default=None) - Optional - The number of workers for training the ensemble. This input argument is used for parallel ensemble methods such as `voting` and `bagging`. Setting it to an integer larger than `1` enables `n_jobs` base estimators to be trained simultaneously.

### Request Example
None

### Response
None
```

--------------------------------

### set_scheduler()

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Set the learning rate scheduler for training the ensemble. This method configures the learning rate scheduler.

```APIDOC
## set_scheduler()

### Description
Set the learning rate scheduler for training the ensemble.

### Method
Not specified (assumed to be a method call within the library)

### Endpoint
Not applicable (SDK method)

### Parameters
None explicitly listed in the source.

### Request Example
```python
ensemble.set_scheduler()
```

### Response
None explicitly listed in the source.
```

--------------------------------

### set_scheduler Method

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Configures the learning rate scheduler for the GradientBoostingRegressor. Supports various scheduler types and their corresponding keyword arguments.

```APIDOC
## set_scheduler Method

### Description
Set the attributes on scheduler for GradientBoostingRegressor.

### Parameters
* **scheduler_name** (string) - The name of the scheduler, should be one of {`LambdaLR`, `MultiplicativeLR`, `StepLR`, `MultiStepLR`, `ExponentialLR`, `CosineAnnealingLR`, `ReduceLROnPlateau`}.
* **kwargs** (keyword arguments) - Keyword arguments on setting the scheduler. These keyword arguments will be directly passed to `torch.optim.lr_scheduler`.
```

--------------------------------

### fit

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Trains the SnapshotEnsembleClassifier on the provided training data.

```APIDOC
## fit

### Description
Implementation on the training stage of SnapshotEnsembleClassifier.

### Parameters
- **train_loader** (torch.utils.data.DataLoader) - A `DataLoader` container that contains the training data.
- **lr_clip** (list or tuple, default=None) - Specify the accepted range of learning rate. When the learning rate determined by the scheduler is out of this range, it will be clipped. The first element should be the lower bound of learning rate. The second element should be the upper bound of learning rate.
- **epochs** (int, default=100) - The number of training epochs.
- **log_interval** (int, default=100) - The number of batches to wait before logging the training status.
- **test_loader** (torch.utils.data.DataLoader, default=None) - A `DataLoader` container that contains the evaluating data. If `None`, no validation is conducted after each snapshot being generated. If not `None`, the ensemble will be evaluated on this dataloader after each snapshot being generated.
- **save_model** (bool, default=True) - Specify whether to save the model parameters. If test_loader is `None`, the ensemble with `n_estimators` base estimators will be saved. If test_loader is not `None`, the ensemble with the best validation performance will be saved.
- **save_dir** (string, default=None) - Specify where to save the model parameters. If `None`, the model will be saved in the current directory. If not `None`, the model will be saved in the specified directory: `save_dir`.
```

--------------------------------

### Load Saved Ensemble Model

Source: https://ensemble-pytorch.readthedocs.io/en/latest/quick_start.html

Use this snippet to load a previously saved ensemble model. Ensure `new_ensemble` is instantiated identically to the original model.

```python
from torchensemble.utils import io

io.load(new_ensemble, save_dir)  # reload
```

--------------------------------

### SoftGradientBoostingClassifier

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Configuration options for the SoftGradientBoostingClassifier.

```APIDOC
## SoftGradientBoostingClassifier

### Parameters

- **return_loss** (bool, default=False) - Whether to return the average cross-entropy loss over all batches in the `test_loader`.

### Returns

- **accuracy** (float) - The classification accuracy of the fitted ensemble on `test_loader`.
- **loss** (float) - The average cross-entropy loss of the fitted ensemble on `test_loader`, only available when `return_loss` is True.
```

--------------------------------

### set_scheduler

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Configures the learning rate scheduler for the FusionRegressor.

```APIDOC
## set_scheduler

### Description
Configures the learning rate scheduler for the FusionRegressor.

### Parameters
#### Parameters
- **scheduler_name** (string) - The name of the scheduler, should be one of {`LambdaLR`, `MultiplicativeLR`, `StepLR`, `MultiStepLR`, `ExponentialLR`, `CosineAnnealingLR`, `ReduceLROnPlateau`}.
- ****kwargs** (keyword arguments) - Keyword arguments on setting the scheduler. These keyword arguments will be directly passed to `torch.optim.lr_scheduler`.
```

--------------------------------

### fit

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Trains the FastGeometricRegressor ensemble using the provided training data and configuration.

```APIDOC
## fit

### Description
Implementation on the training stage of FastGeometricRegressor.

### Parameters
- **train_loader** (torch.utils.data.DataLoader) - A `DataLoader` container that contains the training data.
- **cycle** (int, default=4) - The number of cycles used to build each base estimator in the ensemble.
- **lr_1** (float, default=5e-2) - `alpha_1` in original paper used to adjust the learning rate, also serves as the initial learning rate of the internal optimizer.
- **lr_2** (float, default=1e-4) - `alpha_2` in original paper used to adjust the learning rate, also serves as the smallest learning rate of the internal optimizer.
- **epochs** (int, default=100) - The number of training epochs used to fit the dummy base estimator.
- **log_interval** (int, default=100) - The number of batches to wait before logging the training status.
- **test_loader** (torch.utils.data.DataLoader, default=None) - A `DataLoader` container that contains the evaluating data. If `None`, no validation is conducted after each real base estimator being generated. If not `None`, the ensemble will be evaluated on this dataloader after each base estimator being generated.
- **save_model** (bool, default=True) - Specify whether to save the model parameters. If test_loader is `None`, the ensemble fully trained will be saved. If test_loader is not `None`, the ensemble with the best validation performance will be saved.
- **save_dir** (string, default=None) - Specify where to save the model parameters. If `None`, the model will be saved in the current directory. If not `None`, the model will be saved in the specified directory: `save_dir`.
```

--------------------------------

### fit

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Trains the NeuralForestClassifier using the provided data loaders.

```APIDOC
## fit

### Description
Trains the NeuralForestClassifier using the provided data loaders.

### Parameters
- **train_loader** (torch.utils.data.DataLoader) - A `torch.utils.data.DataLoader` container that contains the training data.
- **epochs** (int, default=100) - The number of training epochs.
- **log_interval** (int, default=100) - The number of batches to wait before logging the training status.
- **test_loader** (torch.utils.data.DataLoader, default=None) - A `torch.utils.data.DataLoader` container that contains the evaluating data. If `None`, no validation is conducted. If not `None`, the ensemble will be evaluated after each epoch.
- **save_model** (bool, default=True) - Specify whether to save the model parameters. If `test_loader` is `None`, the fully trained ensemble is saved. If `test_loader` is not `None`, the ensemble with the best validation performance is saved.
- **save_dir** (string, default=None) - Specify where to save the model parameters. If `None`, the model is saved in the current directory. If not `None`, the model is saved in the specified directory.
```

--------------------------------

### set_optimizer Method

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Configures the optimizer for the GradientBoostingRegressor. Allows specifying the optimizer type and its associated keyword arguments.

```APIDOC
## set_optimizer Method

### Description
Set the attributes on optimizer for GradientBoostingRegressor.

### Parameters
* **optimizer_name** (string) - The name of the optimizer, should be one of {`Adadelta`, `Adagrad`, `Adam`, `AdamW`, `Adamax`, `ASGD`, `RMSprop`, `Rprop`, `SGD`}.
* **kwargs** (keyword arguments) - Keyword arguments on setting the optimizer, should be in the form: `lr=1e-3, weight_decay=5e-4, ...`. These keyword arguments will be directly passed to `torch.optim.Optimizer`.
```

--------------------------------

### set_optimizer()

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Set the parameter optimizer for training the ensemble. This method configures the optimizer used during the training phase.

```APIDOC
## set_optimizer()

### Description
Set the parameter optimizer for training the ensemble.

### Method
Not specified (assumed to be a method call within the library)

### Endpoint
Not applicable (SDK method)

### Parameters
None explicitly listed in the source.

### Request Example
```python
ensemble.set_optimizer()
```

### Response
None explicitly listed in the source.
```

--------------------------------

### forward()

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Data forward process of the ensemble. This method handles the forward pass of data through the ensemble.

```APIDOC
## forward()

### Description
Data forward process of the ensemble.

### Method
Not specified (assumed to be a method call within the library)

### Endpoint
Not applicable (SDK method)

### Parameters
None explicitly listed in the source.

### Request Example
```python
ensemble.forward()
```

### Response
None explicitly listed in the source.
```

--------------------------------

### fit

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Trains the AdversarialTrainingClassifier using the provided data loaders and training parameters.

```APIDOC
## fit

### Description
Trains the AdversarialTrainingClassifier using the provided data loaders and training parameters.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **train_loader** (torch.utils.data.DataLoader) - Required - A `torch.utils.data.DataLoader` container that contains the training data.
* **epochs** (int, default=100) - Optional - The number of training epochs.
* **epsilon** (float, default=0.01) - Optional - The step used to generate adversarial samples in the fast gradient sign method (FGSM), which should be in the range [0, 1].
* **log_interval** (int, default=100) - Optional - The number of batches to wait before logging the training status.
* **test_loader** (torch.utils.data.DataLoader, default=None) - Optional - A `torch.utils.data.DataLoader` container that contains the evaluating data. If `None`, no validation is conducted after each training epoch. If not `None`, the ensemble will be evaluated on this dataloader after each training epoch.
* **save_model** (bool, default=True) - Optional - Specify whether to save the model parameters. If test_loader is `None`, the ensemble fully trained will be saved. If test_loader is not `None`, the ensemble with the best validation performance will be saved.
* **save_dir** (string, default=None) - Optional - Specify where to save the model parameters. If `None`, the model will be saved in the current directory. If not `None`, the model will be saved in the specified directory: `save_dir`.

### Request Example
None

### Response
None
```

--------------------------------

### set_scheduler

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Configures the learning rate scheduler for the SoftGradientBoostingClassifier.

```APIDOC
## set_scheduler

### Description
Configures the learning rate scheduler for the SoftGradientBoostingClassifier.

### Method
`set_scheduler(scheduler_name, **kwargs)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **scheduler_name** (string) - Required - The name of the scheduler, should be one of {`LambdaLR`, `MultiplicativeLR`, `StepLR`, `MultiStepLR`, `ExponentialLR`, `CosineAnnealingLR`, `ReduceLROnPlateau`}.
- ****kwargs** (keyword arguments) - Required - Keyword arguments on setting the scheduler. These keyword arguments will be directly passed to `torch.optim.lr_scheduler`.

### Request Example
```python
# Assuming 'classifier' is an instance of SoftGradientBoostingClassifier
classifier.set_scheduler('StepLR', step_size=30, gamma=0.1)
```

### Response
None
```

--------------------------------

### SnapshotEnsembleClassifier

Source: https://ensemble-pytorch.readthedocs.io/en/latest/parameters.html

Initializes the SnapshotEnsembleClassifier with a base estimator and ensemble configuration.

```APIDOC
## SnapshotEnsembleClassifier

### Description
Initializes the SnapshotEnsembleClassifier with a base estimator and ensemble configuration.

### Parameters
- **estimator** (torch.nn.Module) - The class or object of your base estimator. If `class`, it should inherit from `torch.nn.Module`. If `object`, it should be instantiated from a class inherited from `torch.nn.Module`.
- **n_estimators** (int) - The number of base estimators in the ensemble.
- **estimator_args** (dict, default=None) - The dictionary of hyper-parameters used to instantiate base estimators. This parameter will have no effect if `estimator` is a base estimator object after instantiation.
- **cuda** (bool, default=True) - If `True`, use GPU to train and evaluate the ensemble. If `False`, use CPU to train and evaluate the ensemble.
```
Tokens:
Raw