PySCIPOpt-ML

https://pyscipopt-ml.readthedocs.io/en/latest
Admin
PySCIPOpt-ML is a Python interface that automatically formulates Machine Learning (ML) models into...
Tokens:40,384
Snippets:179
Trust Score:8.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Install PySCIPOpt-ML from source

Source: https://pyscipopt-ml.readthedocs.io/en/latest/basics

This command installs the PySCIPOpt-ML package directly from a local source directory. It is used after cloning the repository to set up the package for development or custom installations.

```bash
python -m pip install .
```

--------------------------------

### Install PySCIPOpt-ML using pip

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/basics.rst

Instructions to install the PySCIPOpt-ML package and its dependencies using pip, the Python package installer. This command is typically run within a virtual environment.

```console
(venv) pip install pyscipopt-ml
```

--------------------------------

### Install PySCIPOpt-ML from source

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/basics.rst

Instructions to install PySCIPOpt-ML directly from its source code. This method is used after cloning the package repository and navigating to its root directory.

```bash
python -m pip install .
```

--------------------------------

### Define, Train, and Embed PyTorch Neural Networks in SCIP

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/example_basic.rst

This code defines two sequential PyTorch neural networks (reg_1, reg_2), trains them using MSELoss and Adam optimizer with synthetic data, converts data to tensors, and then embeds these trained networks as constraints into a SCIP optimization model using add_predictor_constr from pyscipopt-ml.

```python
                nn.ReLU(),
                nn.Linear(layers_sizes[2], 1),
            )
            reg_2 = nn.Sequential(
                nn.Linear(n_inputs, layers_sizes[0]),
                nn.ReLU(),
                nn.Linear(layers_sizes[0], layers_sizes[1]),
                nn.ReLU(),
                nn.Linear(layers_sizes[1], layers_sizes[2]),
                nn.ReLU(),
                nn.Linear(layers_sizes[2], 1),
            )

            # Convert data into PyTorch tensors
            X_tensor = torch.tensor(X, dtype=torch.float32)
            y_1_tensor = torch.tensor(y_1, dtype=torch.float32)
            y_2_tensor = torch.tensor(y_2, dtype=torch.float32)

            # Create a DataLoader for handling batches
            dataset_1 = TensorDataset(X_tensor, y_1_tensor)
            dataset_2 = TensorDataset(X_tensor, y_2_tensor)
            batch_size = 32
            dataloader_1 = DataLoader(dataset_1, batch_size=batch_size, shuffle=True)
            dataloader_2 = DataLoader(dataset_2, batch_size=batch_size, shuffle=True)

            # Initialise the loss function and optimizer
            criterion = nn.MSELoss()
            optimizer_1 = optim.Adam(reg_1.parameters(), lr=0.001, weight_decay=0.0001)
            optimizer_2 = optim.Adam(reg_2.parameters(), lr=0.001, weight_decay=0.0001)

            # Training loop
            for epoch in range(200):
                for batch_X, batch_y in dataloader_1:
                    # Forward pass
                    outputs = reg_1(batch_X)

                    # Calculate loss
                    loss = criterion(outputs, batch_y.view(-1, 1))  # Assuming y is a 1D array

                    # Backward pass and optimization
                    optimizer_1.zero_grad()
                    loss.backward()
                    optimizer_1.step()
                for batch_X, batch_y in dataloader_2:
                    # Forward pass
                    outputs = reg_2(batch_X)

                    # Calculate loss
                    loss = criterion(outputs, batch_y.view(-1, 1))  # Assuming y is a 1D array

                    # Backward pass and optimization
                    optimizer_2.zero_grad()
                    loss.backward()
                    optimizer_2.step()

        # Now build the SCIP Model and embed the neural networks
        scip, input_vars, output_vars = build_basic_scip_model(n_inputs)
        mlp_cons_1 = add_predictor_constr(
            scip, reg_1, input_vars, output_vars[0], unique_naming_prefix="reg_1_"
        )
        mlp_cons_2 = add_predictor_constr(
            scip, reg_2, input_vars, output_vars[1], unique_naming_prefix="reg_2_"
        )

        return scip
```

--------------------------------

### Install PySCIPOpt-ML via pip

Source: https://pyscipopt-ml.readthedocs.io/en/latest/index

This command installs the PySCIPOpt-ML library from PyPI using pip. Python 3.8 or higher is required for successful installation.

```Shell
pip install pyscipopt-ml
```

--------------------------------

### Python: Initialize Basic SCIP Optimization Model

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/example_basic.rst

This Python function initializes a SCIP optimization model for embedding ML predictors. It creates input variables with tight bounds based on the training data range and defines output variables. The function then adds an equality constraint on the second output variable and sets the first output variable as the objective to be maximized, returning the configured SCIP model and its variables.

```python
def build_basic_scip_model(n_inputs):
    # Initialise a SCIP Model
    scip = Model()

    # Create the input variables
    input_vars = np.zeros((1, n_inputs), dtype=object)
    for i in range(n_inputs):
        # Tight bounds are important for MIP formulations of neural networks. They often drastically improve
        # performance. As our training data is in the range [-10, 10], we pass that as bounds [-10, 10].
        # These bounds will then propagate to other variables.
        input_vars[0][i] = scip.addVar(name=f"x_{i}", vtype="C", lb=-10, ub=10)

    # Create the output variables. (Note that these variables will be automatically constructed if not specified)
    output_vars = np.zeros((2, 1), dtype=object)
    for i in range(2):
        output_vars[i] = scip.addVar(name=f"y_{i}", vtype="C", lb=None, ub=None)

    # Now set additional constraints and set the objective
    scip.addCons(output_vars[1][0] == 10, name="fix_output_reg_2")
    scip.setObjective(output_vars[0][0])

    return scip, input_vars, output_vars
```

--------------------------------

### Install PySCIPOpt-ML using pip

Source: https://pyscipopt-ml.readthedocs.io/en/latest/basics

This command installs the PySCIPOpt-ML package via pip, typically within a virtual environment. It automatically includes dependencies like pyscipopt and numpy if they are not already installed.

```bash
(venv) pip install pyscipopt-ml
```

--------------------------------

### Install PySCIPOpt-ML using pip

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/index.rst

This snippet demonstrates how to install the PySCIPOpt-ML library from PyPI using pip. Python 3.8 or higher is required for compatibility.

```bash
pip install pyscipopt-ml
```

--------------------------------

### Python: Generate Random Quadratic Functions for Training Data

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/example_basic.rst

This Python function generates synthetic training data for two non-linear quadratic functions. It takes a random seed, number of input features, and number of samples as parameters. The function returns the input features (X) and two sets of corresponding output values (y_1, y_2) derived from the quadratic formulas, suitable for training machine learning models.

```python
def build_random_quadratic_functions(seed=42, n_inputs=5, n_samples=1000):
    # Set the random seed so the results don't change
    np.random.seed(seed)

    # Generate two random quadratic functions f(x) = x^{t}Qx + Ax + c
    quadratic_coefficients = np.round(
        np.random.uniform(0, 5, size=(2, n_inputs, n_inputs)), decimals=3
    )
    linear_coefficients = np.round(np.random.uniform(0, 1, size=(2, n_inputs)), decimals=3)
    constant = np.round(np.random.uniform(0, 1, size=(2,)), decimals=3)

    # Generate data from the quadratic function
    X = np.random.uniform(-10, 10, size=(n_samples, n_inputs))
    y_1 = np.zeros((n_samples,))
    y_2 = np.zeros((n_samples,))
    for i in range(n_samples):
        y_1[i] = (
            X[i] @ quadratic_coefficients[0] @ X[i].T
            + linear_coefficients[0] @ X[i].T
            + constant[0]
        )
        y_2[i] = (
            X[i] @ quadratic_coefficients[1] @ X[i].T
            + linear_coefficients[1] @ X[i].T
            + constant[1]
        )

    return X, y_1, y_2
```

--------------------------------

### Constructing a PySCIPOpt-ML Wine Blending Optimization Model

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/example_advanced.rst

This Python code defines the core logic for building a PySCIPOpt model that optimizes wine blending. It incorporates a pre-trained LightGBM regressor to predict wine quality, setting up decision variables for wine features, quality, and mixture proportions. The model includes constraints for blending ratios, vineyard capacity, budget, and a minimum quality threshold, culminating in an objective to maximize overall wine quality.

```python
        else:
            if gbdt_or_rf == "gbdt":
                reg = LGBMRegressor(
                    random_state=seed, n_estimators=n_estimators, max_depth=max_depth
                ).fit(X, quality)
            else:
                reg = LGBMRegressor(
                    random_state=seed,
                    n_estimators=n_estimators,
                    max_depth=max_depth,
                    boosting_type="rf",
                    bagging_freq=1,
                    bagging_fraction=0.5,
                ).fit(X, quality)

        # Create artificial data from some vineyards
        np.random.seed(seed)
        vineyard_order = np.arange(X.shape[0])
        np.random.shuffle(vineyard_order)
        vineyard_litre_limits = np.random.uniform(0.25, 0.35, n_vineyards)
        vineyard_costs = np.random.uniform(1, 2, n_vineyards)
        vineyard_features = []
        low_quality_vineyards_i = 0
        for i in vineyard_order:
            if low_quality_vineyards_i >= n_vineyards:
                break
            if quality[i] <= 5:
                low_quality_vineyards_i += 1
                vineyard_features.append(X[i])
        vineyard_features = np.array(vineyard_features)

        # Create the SCIP Model
        scip = Model()

        # Create variables deciding the features of each wine
        feature_vars = np.zeros((n_wines_to_produce, n_features), dtype=object)
        quality_vars = np.zeros((n_wines_to_produce, 1), dtype=object)
        wine_mixture_vars = np.zeros((n_wines_to_produce, n_vineyards), dtype=object)

        for i in range(n_wines_to_produce):
            quality_vars[i][0] = scip.addVar(vtype="C", lb=0, ub=10, name=f"quality_{i}")
            for j in range(n_features):
                max_val = np.max(X[:, j])
                min_val = np.min(X[:, j])
                lb = max(0, min_val - 0.1 * max_val)
                ub = 1.1 * max_val
                feature_vars[i][j] = scip.addVar(vtype="C", lb=lb, ub=ub, name=f"feature_{i}_{j}")
            for k in range(n_vineyards):
                wine_mixture_vars[i][k] = scip.addVar(
                    vtype="C", lb=0, ub=vineyard_litre_limits[k], name=f"mixture_{i}_{k}"
                )

        # Now create constraints on the wine blending
        for i in range(n_wines_to_produce):
            for j in range(n_features):
                scip.addCons(
                    feature_vars[i][j]
                    == quicksum(
                        wine_mixture_vars[i][k] * vineyard_features[k][j] for k in range(n_vineyards)
                    ),
                    name=f"mixture_cons_{i}_{j}",
                )
        for i in range(n_wines_to_produce):
            scip.addCons(
                quicksum(wine_mixture_vars[i][k] for k in range(n_vineyards)) == 1,
                name=f"wine_mix_{i}",
            )
        for k in range(n_vineyards):
            scip.addCons(
                quicksum(wine_mixture_vars[i][k] for i in range(n_wines_to_produce))
                <= vineyard_litre_limits[k],
                name=f"vineyard_limit_{k}",
            )

        # Add the budget constraint
        scip.addCons(
            quicksum(
                quicksum(wine_mixture_vars[i][k] * vineyard_costs[k] for k in range(n_vineyards))
                for i in range(n_wines_to_produce)
            )
            <= budget,
            name=f"budget_cons",
        )

        # Add the ML constraint. Add in a single batch!
        pred_cons = add_predictor_constr(
            scip, reg, feature_vars, quality_vars, unique_naming_prefix="wine_", epsilon=epsilon
        )

        # Add a constraint ensuring minimum wine quality on those produced
        for i in range(n_wines_to_produce):
            scip.addCons(quality_vars[i][0] >= min_wine_quality, name=f"min_quality_{i}")

        # Set the SCIP objective
        scip.setObjective(
            quicksum(quality_vars[i][0] for i in range(n_wines_to_produce)) / n_wines_to_produce,
            sense="maximize"
        )

        return scip
```

--------------------------------

### Python Function for Wine Blending Optimization with ML Integration

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/example_advanced.rst

This Python function, `build_and_optimise_wine_manufacturer`, sets up and solves the wine blending optimization problem. It trains an ML model (Random Forest or Gradient Boosting Regressor using scikit-learn, XGBoost, or LightGBM) to predict wine quality, then prepares the data for the optimization. The function takes parameters for seed, number of vineyards, wines to produce, minimum quality, ML framework, regressor type, estimators, max depth, and epsilon.

```python
def build_and_optimise_wine_manufacturer(
    seed=42,
    n_vineyards=35,
    n_wines_to_produce=5,
    min_wine_quality=4.25,
    sklearn_xgboost_lightgbm="sklearn",
    gbdt_or_rf="rf",
    n_estimators=3,
    max_depth=3,
    epsilon=0.0001,
):
    assert sklearn_xgboost_lightgbm in ("sklearn", "xgboost", "lightgbm")
    assert gbdt_or_rf in ("gbdt", "rf")

    # Path to red wine data
    data_dict = read_csv_to_dict("./tests/data/wineQualityReds.csv")

    features = [
        "fixed.acidity",
        "volatile.acidity",
        "citric.acid",
        "residual.sugar",
        "chlorides",
        "free.sulfur.dioxide",
        "total.sulfur.dioxide",
        "density",
        "pH",
        "sulphates",
        "alcohol",
    ]
    n_features = len(features)
    budget = 1.7 * n_wines_to_produce

    # Generate the actual input data arrays for the ML predictors
    X = []
    quality = np.array([float(x) for x in data_dict["quality"]]).reshape(
        -1,
    )
    for feature in features:
        X.append(np.array([float(x) for x in data_dict[feature]]))
    X = np.swapaxes(np.array(X), 0, 1)

    # Train the ML predictor
    if sklearn_xgboost_lightgbm == "sklearn":
        if gbdt_or_rf == "rf":
            reg = RandomForestRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
        else:
            reg = GradientBoostingRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
    elif sklearn_xgboost_lightgbm == "xgboost":
        if gbdt_or_rf == "gbdt":
            reg = XGBRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
        else:
            reg = XGBRFRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
```

--------------------------------

### Execute and Optimize SCIP Model with Embedded Neural Networks

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/example_basic.rst

This snippet demonstrates how to call the build_and_optimise_function_approximation_model function to obtain a SCIP model with embedded neural networks, optimize it, and then verify the embedding error against a threshold using np.max(mlp_cons.get_error()). This ensures the accuracy of the neural network's integration into the SCIP model.

```python
    # Get the SCIP Model with the embedded trained predictors
    scip = build_and_optimise_function_approximation_model()

    # Optimize the model
    scip.optimize()

    # We can check the "error" of the MIP embedding via the difference between SKLearn / Torch and SCIP output
    if np.max(mlp_cons_1.get_error()) > 10**-3:
        error = np.max(mlp_cons_1.get_error())
        raise AssertionError(f"Max error {error} exceeds threshold of {10 ** -3}")
    if np.max(mlp_cons_2.get_error()) > 10**-3:
        error = np.max(mlp_cons_2.get_error())
        raise AssertionError(f"Max error {error} exceeds threshold of {10 ** -3}")
```

--------------------------------

### Python: Train ML Regressors for Function Approximation

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/example_basic.rst

This Python function prepares and trains machine learning regressors for approximating non-linear functions. It first calls `build_random_quadratic_functions` to generate synthetic data. Depending on the `sklearn_or_torch` parameter, it trains either Scikit-Learn `MLPRegressor` instances or PyTorch `nn.Sequential` models with ReLU activations, using specified layer sizes. This function provides the trained ML models ready for integration into an optimization problem.

```python
def build_and_optimise_function_approximation_model(
seed=42, n_inputs=5, n_samples=1000, sklearn_or_torch="sklearn", layers_sizes=(20, 20, 10)
):
    assert len(layers_sizes) == 3

    X, y_1, y_2 = build_random_quadratic_functions(
        seed=seed, n_inputs=n_inputs, n_samples=n_samples
    )

    if sklearn_or_torch == "sklearn":
        reg_1 = MLPRegressor(
            random_state=seed,
            hidden_layer_sizes=(layers_sizes[0], layers_sizes[1], layers_sizes[2]),
        ).fit(X, y_1.reshape(-1))
        reg_2 = MLPRegressor(
            random_state=seed,
            hidden_layer_sizes=(layers_sizes[0], layers_sizes[1], layers_sizes[2]),
        ).fit(X, y_2.reshape(-1))
    else:
        torch.random.manual_seed(seed)
        reg_1 = nn.Sequential(
            nn.Linear(n_inputs, layers_sizes[0]),
            nn.ReLU(),
            nn.Linear(layers_sizes[0], layers_sizes[1]),
            nn.ReLU(),
            nn.Linear(layers_sizes[1], layers_sizes[2]),
```

--------------------------------

### Build and Optimize SCIP Model with XGBoost Random Forest Predictors

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/example_advanced.rst

This Python snippet demonstrates how to build and optimize a SCIP model using the `build_and_optimise_wine_manufacturer` function, embedding XGBoost Random Forest predictors. It configures the model with specific parameters like seed, number of vineyards, and wine quality, then optimizes it. An assertion checks the maximum error of the MIP embedding against a threshold, raising an error if exceeded.

```python
# Build the SCIP model with embedded ML predictors
scip = build_and_optimise_wine_manufacturer(
    seed=42,
    n_vineyards=35,
    n_wines_to_produce=5,
    min_wine_quality=4.25,
    sklearn_xgboost_lightgbm="xgboost",
    gbdt_or_rf="rf",
    n_estimators=3,
    max_depth=3,
    epsilon=0.0001
    )

# Optimise the SCIP model
scip.optimize()

# We can check the "error" of the MIP embedding via the difference between SKLearn and SCIP output
if np.max(pred_cons.get_error()) > 10**-3:
    error = np.max(pred_cons.get_error())
    raise AssertionError(f"Max error {error} exceeds threshold of {10 ** -3}")

return scip
```

--------------------------------

### Train Function Approximation Models (Scikit-Learn/PyTorch) for SCIP Integration (Python)

Source: https://pyscipopt-ml.readthedocs.io/en/latest/example_basic

This Python function trains machine learning models to approximate non-linear functions, supporting both Scikit-Learn's MLPRegressor and PyTorch's Sequential models. It handles data generation, model initialization, and the training loop setup, including loss functions and optimizers for PyTorch. The trained models are intended for subsequent embedding into a SCIP optimization model.

```python
def build_and_optimise_function_approximation_model(
seed=42, n_inputs=5, n_samples=1000, sklearn_or_torch="sklearn", layers_sizes=(20, 20, 10)
):
    assert len(layers_sizes) == 3

    X, y_1, y_2 = build_random_quadratic_functions(
        seed=seed, n_inputs=n_inputs, n_samples=n_samples
    )

    if sklearn_or_torch == "sklearn":
        reg_1 = MLPRegressor(
            random_state=seed,
            hidden_layer_sizes=(layers_sizes[0], layers_sizes[1], layers_sizes[2]),
        ).fit(X, y_1.reshape(-1))
        reg_2 = MLPRegressor(
            random_state=seed,
            hidden_layer_sizes=(layers_sizes[0], layers_sizes[1], layers_sizes[2]),
        ).fit(X, y_2.reshape(-1))
    else:
        torch.random.manual_seed(seed)
        reg_1 = nn.Sequential(
            nn.Linear(n_inputs, layers_sizes[0]),
            nn.ReLU(),
            nn.Linear(layers_sizes[0], layers_sizes[1]),
            nn.ReLU(),
            nn.Linear(layers_sizes[1], layers_sizes[2]),
            nn.ReLU(),
            nn.Linear(layers_sizes[2], 1),
        )
        reg_2 = nn.Sequential(
            nn.Linear(n_inputs, layers_sizes[0]),
            nn.ReLU(),
            nn.Linear(layers_sizes[0], layers_sizes[1]),
            nn.ReLU(),
            nn.Linear(layers_sizes[1], layers_sizes[2]),
            nn.ReLU(),
            nn.Linear(layers_sizes[2], 1),
        )

        # Convert data into PyTorch tensors
        X_tensor = torch.tensor(X, dtype=torch.float32)
        y_1_tensor = torch.tensor(y_1, dtype=torch.float32)
        y_2_tensor = torch.tensor(y_2, dtype=torch.float32)

        # Create a DataLoader for handling batches
        dataset_1 = TensorDataset(X_tensor, y_1_tensor)
        dataset_2 = TensorDataset(X_tensor, y_2_tensor)
        batch_size = 32
        dataloader_1 = DataLoader(dataset_1, batch_size=batch_size, shuffle=True)
        dataloader_2 = DataLoader(dataset_2, batch_size=batch_size, shuffle=True)

        # Initialise the loss function and optimizer
        criterion = nn.MSELoss()
        optimizer_1 = optim.Adam(reg_1.parameters(), lr=0.001, weight_decay=0.0001)
        optimizer_2 = optim.Adam(reg_2.parameters(), lr=0.001, weight_decay=0.0001)

        # Training loop
```

--------------------------------

### Example Decision Tree Split Conditions

Source: https://pyscipopt-ml.readthedocs.io/en/latest/formulations

Illustrative split conditions for an internal node of a decision tree, demonstrating how a feature (x_4) is split at a specific value (5). This example highlights potential ambiguities when the feature value is close to the split point.

```APIDOC
x_4 \leq 5\nx_4 \geq 5
```

--------------------------------

### Build Basic SCIP Optimization Model (Python)

Source: https://pyscipopt-ml.readthedocs.io/en/latest/example_basic

This Python function initializes a SCIP optimization model, defining input and output variables. It sets tight bounds for input variables based on the expected data range and adds a fixed constraint and an objective function. This model serves as a base for embedding trained machine learning predictors.

```python
def build_basic_scip_model(n_inputs):
    # Initialise a SCIP Model
    scip = Model()

    # Create the input variables
    input_vars = np.zeros((1, n_inputs), dtype=object)
    for i in range(n_inputs):
        # Tight bounds are important for MIP formulations of neural networks. They often drastically improve
        # performance. As our training data is in the range [-10, 10], we pass that as bounds [-10, 10].
        # These bounds will then propagate to other variables.
        input_vars[0][i] = scip.addVar(name=f"x_{i}", vtype="C", lb=-10, ub=10)

    # Create the output variables. (Note that these variables will be automatically constructed if not specified)
    output_vars = np.zeros((2, 1), dtype=object)
    for i in range(2):
        output_vars[i] = scip.addVar(name=f"y_{i}", vtype="C", lb=None, ub=None)

    # Now set additional constraints and set the objective
    scip.addCons(output_vars[1][0] == 10, name="fix_output_reg_2")
    scip.setObjective(output_vars[0][0])

    return scip, input_vars, output_vars
```

--------------------------------

### PySCIPOpt-ML API Reference Manual

Source: https://pyscipopt-ml.readthedocs.io/en/latest/api/MultiOutputConstr

Detailed reference for the PySCIPOpt-ML library's modules, classes, and functions, including various machine learning model constraints and helper utilities.

```APIDOC
API Reference manual:
  Generic Predictor Constraint
  Scikit-learn regression models:
    Linear Regression Constraint
    Partial Least Square Regression Constraint
    Logistic Regression Constraint
    Centroid Based Clustering Constraint
    Decision Tree Regressor Constraint
    Gradient Boosting Regressor Constraint
    Random Forest Constraint
    Pipeline Constraint
    Support Vector Constraint
    MLP Constraint
    MultiOutput Constraint:
      add_multi_output_regressor_constr()
      add_multi_output_classifier_constr()
      MultiOutputConstr
    Scikit-Learn Helper
  Pytorch Sequential Network Constraint
  Keras Model / Keras Sequential Network Constraint
  ONNX ModelProto Constraint
  XGBoost Constraint
  LightGBM Constraint
```

--------------------------------

### Decision Tree - Internal Node Split Example

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/formulations.rst

This snippet provides a concrete example of how decisions are formulated at an internal node of a decision tree. It illustrates the inequalities derived from a split based on a feature value, showing the conditions for traversing left or right branches.

```APIDOC
x_4 \leq 5

x_4 \geq 5
```

--------------------------------

### PySCIPOpt-ML API Reference

Source: https://pyscipopt-ml.readthedocs.io/en/latest/api/GradientBoostingConstr

Detailed API reference for PySCIPOpt-ML, outlining various machine learning model constraints and helper functions available for integration with SCIP.

```APIDOC
API Reference manual:
  Generic Predictor Constraint
  Scikit-learn regression models:
    Linear Regression Constraint
    Partial Least Square Regression Constraint
    Logistic Regression Constraint
    Centroid Based Clustering Constraint
    Decision Tree Regressor Constraint
    Gradient Boosting Regressor Constraint:
      add_gradient_boosting_regressor_constr()
      add_gradient_boosting_classifier_constr()
      GradientBoostingConstr
    Random Forest Constraint
    Pipeline Constraint
    Support Vector Constraint
    MLP Constraint
    MultiOutput Constraint
    Scikit-Learn Helper
  Pytorch Sequential Network Constraint
  Keras Model / Keras Sequential Network Constraint
  ONNX ModelProto Constraint
  XGBoost Constraint
  LightGBM Constraint
```

--------------------------------

### Python Function to Build and Optimize Wine Manufacturer Model

Source: https://pyscipopt-ml.readthedocs.io/en/latest/example_advanced

This comprehensive Python function, `build_and_optimise_wine_manufacturer`, demonstrates how to prepare wine quality data, train machine learning models (Random Forest, Gradient Boosting, XGBoost, LightGBM) based on user-specified frameworks, and then integrate the trained model with a SCIP optimization problem. It sets up variables for wine features, quality, and mixture, and defines constraints for blending, aiming to optimize wine production.

```python
def build_and_optimise_wine_manufacturer(
    seed=42,
    n_vineyards=35,
    n_wines_to_produce=5,
    min_wine_quality=4.25,
    sklearn_xgboost_lightgbm="sklearn",
    gbdt_or_rf="rf",
    n_estimators=3,
    max_depth=3,
    epsilon=0.0001,
):
    assert sklearn_xgboost_lightgbm in ("sklearn", "xgboost", "lightgbm")
    assert gbdt_or_rf in ("gbdt", "rf")

    # Path to red wine data
    data_dict = read_csv_to_dict("./tests/data/wineQualityReds.csv")

    features = [
        "fixed.acidity",
        "volatile.acidity",
        "citric.acid",
        "residual.sugar",
        "chlorides",
        "free.sulfur.dioxide",
        "total.sulfur.dioxide",
        "density",
        "pH",
        "sulphates",
        "alcohol",
    ]
    n_features = len(features)
    budget = 1.7 * n_wines_to_produce

    # Generate the actual input data arrays for the ML predictors
    X = []
    quality = np.array([float(x) for x in data_dict["quality"]]).reshape(
        -1,
    )
    for feature in features:
        X.append(np.array([float(x) for x in data_dict[feature]]))
    X = np.swapaxes(np.array(X), 0, 1)

    # Train the ML predictor
    if sklearn_xgboost_lightgbm == "sklearn":
        if gbdt_or_rf == "rf":
            reg = RandomForestRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
        else:
            reg = GradientBoostingRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
    elif sklearn_xgboost_lightgbm == "xgboost":
        if gbdt_or_rf == "gbdt":
            reg = XGBRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
        else:
            reg = XGBRFRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
    else:
        if gbdt_or_rf == "gbdt":
            reg = LGBMRegressor(
                random_state=seed, n_estimators=n_estimators, max_depth=max_depth
            ).fit(X, quality)
        else:
            reg = LGBMRegressor(
                random_state=seed,
                n_estimators=n_estimators,
                max_depth=max_depth,
                boosting_type="rf",
                bagging_freq=1,
                bagging_fraction=0.5,
            ).fit(X, quality)

    # Create artificial data from some vineyards
    np.random.seed(seed)
    vineyard_order = np.arange(X.shape[0])
    np.random.shuffle(vineyard_order)
    vineyard_litre_limits = np.random.uniform(0.25, 0.35, n_vineyards)
    vineyard_costs = np.random.uniform(1, 2, n_vineyards)
    vineyard_features = []
    low_quality_vineyards_i = 0
    for i in vineyard_order:
        if low_quality_vineyards_i >= n_vineyards:
            break
        if quality[i] <= 5:
            low_quality_vineyards_i += 1
            vineyard_features.append(X[i])
    vineyard_features = np.array(vineyard_features)

    # Create the SCIP Model
    scip = Model()

    # Create variables deciding the features of each wine
    feature_vars = np.zeros((n_wines_to_produce, n_features), dtype=object)
    quality_vars = np.zeros((n_wines_to_produce, 1), dtype=object)
    wine_mixture_vars = np.zeros((n_wines_to_produce, n_vineyards), dtype=object)

    for i in range(n_wines_to_produce):
        quality_vars[i][0] = scip.addVar(vtype="C", lb=0, ub=10, name=f"quality_{i}")
        for j in range(n_features):
            max_val = np.max(X[:, j])
            min_val = np.min(X[:, j])
            lb = max(0, min_val - 0.1 * max_val)
            ub = 1.1 * max_val
            feature_vars[i][j] = scip.addVar(vtype="C", lb=lb, ub=ub, name=f"feature_{i}_{j}")
        for k in range(n_vineyards):
            wine_mixture_vars[i][k] = scip.addVar(
                vtype="C", lb=0, ub=vineyard_litre_limits[k], name=f"mixture_{i}_{k}"
            )

    # Now create constraints on the wine blending
    for i in range(n_wines_to_produce):
        for j in range(n_features):
            scip.addCons(
                feature_vars[i][j]
                == quicksum(
                    wine_mixture_vars[i][k] * vineyard_features[k][j] for k in range(n_vineyards)
                ),
                name=f"mixture_cons_{i}_{j}",
            )
    for i in range(n_wines_to_produce):
        scip.addCons(
            quicksum(wine_mixture_vars[i][k] for k in range(n_vineyards)) == 1,
            name=f"wine_mix_{i}",
        )
    for k in range(n_vineyards):
        scip.addCons(
```

--------------------------------

### APIDOC: pyscipopt_ml.lightgbm.lgbgetter Class Reference

Source: https://pyscipopt-ml.readthedocs.io/en/latest/api/LightGBMConstr

Comprehensive API documentation for the `LGBgetter` class, including its constructor, inherited classes, properties, and methods with their parameters, return types, and potential exceptions.

```APIDOC
class pyscipopt_ml.lightgbm.lgbgetter.LGBgetter(predictor, input_vars, output_type='regular', **kwargs)
  Inherits from: AbstractPredictorConstr
  Description: Utility class for lightgbm models convertors.
    Implement some common functionalities: check predictor is fitted, output dimension, get error

  Properties:
    predictor
      Description: Lightgbm predictor embedded into SCIP model.

  Methods:
    extract_raw_data_and_create_tree_vars(epsilon=0.0)
      Description: Function for extracting information from lgb._Booster and creating additional modelling variables.
      Parameters:
        epsilon (float, optional): Small value used to impose strict inequalities for splitting nodes in MIP formulations.
      Returns:
        trees (list): A list of tree dictionaries similar in structure to that provided by SKlearn
        tree_vars (np.ndarray): A numpy array filled with variables that represent output of trees from the trained LightGBM model

    get_error(eps=None)
      Description: Returns error in SCIP’s solution with respect to the actual output of the trained predictor
      Parameters:
        eps (float or int or None, optional): The maximum allowed tolerance for a mismatch between the actual predictive model and SCIP. If the error is larger than eps an appropriate warning is printed
      Returns:
        error: The absolute values of the difference between SCIP’s solution and the trained ML model’s output given the input as defined by SCIP. The matrix is the same dimension as the output of the trained predictor. Using sklearn / pyscipopt, the absolute difference between model.predict(input) and scip.getVal(output).
      Return type: np.ndarray
      Raises:
        NoSolution: If SCIP has no solution (either was not optimized or is infeasible).
```

--------------------------------

### Train Neural Networks and Embed into SCIP Model

Source: https://pyscipopt-ml.readthedocs.io/en/latest/example_basic

This Python code snippet demonstrates a training loop for two distinct neural networks (reg_1, reg_2) using PyTorch-like operations. Following training, it shows how to construct a basic SCIP model and embed the trained neural networks as predictor constraints using add_predictor_constr from PySCIPOpt-ML, linking input and output variables.

```Python
        for epoch in range(200):
            for batch_X, batch_y in dataloader_1:
                # Forward pass
                outputs = reg_1(batch_X)

                # Calculate loss
                loss = criterion(outputs, batch_y.view(-1, 1))  # Assuming y is a 1D array

                # Backward pass and optimization
                optimizer_1.zero_grad()
                loss.backward()
                optimizer_1.step()
            for batch_X, batch_y in dataloader_2:
                # Forward pass
                outputs = reg_2(batch_X)

                # Calculate loss
                loss = criterion(outputs, batch_y.view(-1, 1))  # Assuming y is a 1D array

                # Backward pass and optimization
                optimizer_2.zero_grad()
                loss.backward()
                optimizer_2.step()

    # Now build the SCIP Model and embed the neural networks
    scip, input_vars, output_vars = build_basic_scip_model(n_inputs)
    mlp_cons_1 = add_predictor_constr(
        scip, reg_1, input_vars, output_vars[0], unique_naming_prefix="reg_1_"
    )
    mlp_cons_2 = add_predictor_constr(
        scip, reg_2, input_vars, output_vars[1], unique_naming_prefix="reg_2_"
    )

    return scip
```

--------------------------------

### Instantiate and Optimize Wine Manufacturing SCIP Model with XGBoost

Source: https://pyscipopt-ml.readthedocs.io/en/latest/example_advanced

This Python code demonstrates how to instantiate and optimize the wine manufacturing SCIP model, specifically configuring it to use XGBoost Random Forest predictors. It sets various parameters like vineyard count, number of wines, and quality requirements. After optimization, it includes an assertion to check the embedding error of the ML predictor, ensuring the SCIP output aligns with the ML framework's output within a given tolerance.

```Python
# Build the SCIP model with embedded ML predictors
scip = build_and_optimise_wine_manufacturer(
    seed=42,
    n_vineyards=35,
    n_wines_to_produce=5,
    min_wine_quality=4.25,
    sklearn_xgboost_lightgbm="xgboost",
    gbdt_or_rf="rf",
    n_estimators=3,
    max_depth=3,
    epsilon=0.0001
    )

# Optimise the SCIP model
scip.optimize()

# We can check the "error" of the MIP embedding via the difference between SKLearn and SCIP output
if np.max(pred_cons.get_error()) > 10**-3:
    error = np.max(pred_cons.get_error())
    raise AssertionError(f"Max error {error} exceeds threshold of {10 ** -3}")

return scip
```

--------------------------------

### PySCIPOpt-ML API Reference Manual Structure

Source: https://pyscipopt-ml.readthedocs.io/en/latest/api/SequentialConstr

This snippet outlines the hierarchical structure of the PySCIPOpt-ML API reference manual, detailing the main categories of constraints and models for which API documentation is available. It serves as a high-level map to the API.

```APIDOC
API Reference manual:
  Generic Predictor Constraint
  Scikit-learn regression models
  Pytorch Sequential Network Constraint
  Keras Model / Keras Sequential Network Constraint
  ONNX ModelProto Constraint
  XGBoost Constraint
  LightGBM Constraint
```

--------------------------------

### Define SCIP Model Constraints and Objective for Wine Production

Source: https://pyscipopt-ml.readthedocs.io/en/latest/example_advanced

This Python code snippet defines various constraints and the objective function for an SCIP optimization model in a wine manufacturing context. It includes vineyard limits, a budget constraint, a minimum wine quality constraint, and crucially, embeds a machine learning predictor using `add_predictor_constr`. The objective is to maximize the average wine quality.

```Python
quicksum(wine_mixture_vars[i][k] for i in range(n_wines_to_produce))
            <= vineyard_litre_limits[k],
            name=f"vineyard_limit_{k}",
        )

    # Add the budget constraint
    scip.addCons(
        quicksum(
            quicksum(wine_mixture_vars[i][k] * vineyard_costs[k] for k in range(n_vineyards))
            for i in range(n_wines_to_produce)
        )
        <= budget,
        name=f"budget_cons",
    )

    # Add the ML constraint. Add in a single batch!
    pred_cons = add_predictor_constr(
        scip, reg, feature_vars, quality_vars, unique_naming_prefix="wine_", epsilon=epsilon
    )

    # Add a constraint ensuring minimum wine quality on those produced
    for i in range(n_wines_to_produce):
        scip.addCons(quality_vars[i][0] >= min_wine_quality, name=f"min_quality_{i}")

    # Set the SCIP objective
    scip.setObjective(
        quicksum(quality_vars[i][0] for i in range(n_wines_to_produce)) / n_wines_to_produce,
        sense="maximize"
    )

    return scip
```

--------------------------------

### ReLU Activation Function (Big-M Formulation)

Source: https://pyscipopt-ml.readthedocs.io/en/latest/_sources/formulations.rst

This formulation presents the traditional Big-M approach for ReLU activation, which introduces binary variables and requires the user to provide appropriate input bounds. While empirically state-of-the-art, this method can fail if Big-M values become too large and is less numerically friendly than SOS1.

```APIDOC
y_j \geq 0

y_j \geq \sum_{i=1}^n \beta_{i,j} x_i + \beta_{0,j} \quad &\forall j

y_j \leq \sum_{i=1}^n \beta_{i,j} x_i + \beta_{0,j}  - (1 - z_j) L_j \quad &\forall j

y_j \leq z_j U_j \quad &\forall j
```

--------------------------------

### PySCIPOpt-ML API Reference Manual Structure

Source: https://pyscipopt-ml.readthedocs.io/en/latest/api/RandomForestConstr

This snippet provides a structured overview of the PySCIPOpt-ML API reference manual, listing available constraints for various machine learning models and helper functions. It outlines the hierarchical organization of the documentation, detailing classes and functions for integrating ML models into SCIP optimization problems.

```APIDOC
API Reference manual:
  Generic Predictor Constraint
  Scikit-learn regression models:
    Linear Regression Constraint
    PLSRegressionConstr
    Logistic Regression Constraint
    Centroid Based Clustering Constraint
    Decision Tree Regressor Constraint
    Gradient Boosting Regressor Constraint
    Random Forest Constraint:
      add_random_forest_regressor_constr()
      add_random_forest_classifier_constr()
      RandomForestConstr
    Pipeline Constraint
    Support Vector Constraint
    MLP Constraint
    MultiOutput Constraint
    Scikit-Learn Helper
  Pytorch Sequential Network Constraint
  Keras Model / Keras Sequential Network Constraint
  ONNX ModelProto Constraint
  XGBoost Constraint
  LightGBM Constraint
```

--------------------------------

### PySCIPOpt-ML API Reference Overview

Source: https://pyscipopt-ml.readthedocs.io/en/latest/api/AbstractPredictorConstr

This section outlines the key components of the PySCIPOpt-ML API, focusing on generic predictor constraints and integrations with various machine learning frameworks. It lists available functions, classes, methods, and properties for defining and incorporating ML models into optimization problems.

```APIDOC
Generic Predictor Constraint:
  add_predictor_constr()
  AbstractPredictorConstr:
    get_error()
    input
    input_values
    output
    output_values
    print_stats()
  Scikit-learn regression models
  Pytorch Sequential Network Constraint
  Keras Model / Keras Sequential Network Constraint
  ONNX ModelProto Constraint
  XGBoost Constraint
  LightGBM Constraint
```

--------------------------------

### LightGBM Constraint API Reference

Source: https://pyscipopt-ml.readthedocs.io/en/latest/api/SKGetter

API reference for the LightGBM Constraint, facilitating the use of LightGBM models as constraints within PySCIPOpt-ML. This allows for the incorporation of LightGBM's efficient gradient boosting capabilities into optimization problems.

```APIDOC
LightGBM Constraint:
  Description: Constraint for LightGBM models.
  Reference: LightGBMConstr.html
```
Tokens:
Raw