### Install bayesian-optimization via pip

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/index.md

Use this command to install the library using pip. Ensure you have pip installed.

```console
pip install bayesian-optimization
```

--------------------------------

### Setup standard optimizer

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/domain_reduction.ipynb

Initializes a standard `BayesianOptimization` object without any domain reduction transformer. This serves as a baseline for comparison.

```python
standard_optimizer = BayesianOptimization(
    f=ackley,
    pbounds=pbounds,
    verbose=0,
    random_state=1,
)
```

--------------------------------

### Install bayesian-optimization via Conda

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/index.md

Use this command to install the library using Conda from the conda-forge channel. Ensure you have Conda installed.

```console
conda install -c conda-forge bayesian-optimization
```

--------------------------------

### Setup optimizer with SDR

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/domain_reduction.ipynb

Initializes a `BayesianOptimization` object configured to use the `bounds_transformer` for sequential domain reduction. This optimizer will adapt its search space.

```python
mutating_optimizer = BayesianOptimization(
    f=ackley,
    pbounds=pbounds,
    verbose=0,
    random_state=1,
    bounds_transformer=bounds_transformer
)
```

--------------------------------

### Initialize Known Function Values

Source: https://github.com/bayesian-optimization/bayesianoptimization/wiki/Introduction

Use the `initialize` method to provide the optimizer with known function values and their corresponding coordinates before starting the optimization process.

```python
self.initialize({'target': [-1, -4], 'x': [0, 2], 'y': [0, 0]})
```

--------------------------------

### Run Bayesian Optimization Maximization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/visualization.ipynb

Starts the Bayesian Optimization process to find the maximum of the target function. `init_points` are random points to start, and `n_iter` is the number of optimization steps.

```python
optimizer.maximize(init_points=2, n_iter=0)
```

--------------------------------

### Instantiate Acquisition Function

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/advanced-tour.ipynb

Instantiate an acquisition function, such as UpperConfidenceBound. This guides the optimization process by determining the next point to probe.

```python
from bayes_opt import acquisition

acq = acquisition.UpperConfidenceBound(kappa=2.5)
```

--------------------------------

### Define parameter bounds

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/domain_reduction.ipynb

Sets the initial search space boundaries for the optimization problem. This example uses bounds of (-5, 5) for both 'x' and 'y' dimensions.

```python
pbounds = {'x': (-5, 5), 'y': (-5, 5)}
```

--------------------------------

### Suggest Next Point to Probe

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/advanced-tour.ipynb

Use the optimizer's suggest method to get the next set of parameters to evaluate. Initially, suggestions are random until points are registered.

```python
next_point_to_probe = optimizer.suggest()
print("Next point to probe is:", next_point_to_probe)
```

--------------------------------

### Get Parameter Keys for Probing

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/basic-tour.ipynb

Retrieve the keys of the parameter space to understand the expected order when providing probe parameters as an iterable.

```python
print(optimizer.space.keys)
```

--------------------------------

### Initialize and Register with TargetSpace

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/target_space.md

Demonstrates initializing a TargetSpace with a target function and parameter bounds, then registering a point and asserting the maximum value and parameters.

```python
>>> def target_func(p1, p2):
>>>     return p1 + p2
>>> pbounds = {"p1": (0, 1), "p2": (1, 100)}
>>> space = TargetSpace(target_func, pbounds, random_state=0)
>>> x = np.array([4, 5])
>>> y = target_func(x)
>>> space.register(x, y)
>>> assert self.max()["target"] == 9
>>> assert self.max()["params"] == {"p1": 1.0, "p2": 2.0}
```

--------------------------------

### Define and optimize a 1D integer-valued target function

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

Defines a 1D target function that uses rounded integer inputs and optimizes it using both a continuous and a discrete Bayesian Optimization setup. The discrete setup correctly models the function's integer nature.

```python
def target_function_1d(x):
    return np.sin(np.round(x)) - np.abs(np.round(x) / 5)

c_pbounds = {'x': (-10, 10)}
bo_cont = BayesianOptimization(target_function_1d, c_pbounds, verbose=0, random_state=1)

# one way of constructing an integer-valued parameter is to add a third element to the tuple
d_pbounds = {'x': (-10, 10, int)}
bo_disc = BayesianOptimization(target_function_1d, d_pbounds, verbose=0, random_state=1)

fig, axs = plt.subplots(2, 1, figsize=(10, 6), sharex=True, sharey=True)

bo_cont.maximize(init_points=2, n_iter=10)
bo_cont.acquisition_function._fit_gp(bo_cont._gp, bo_cont.space)

y_mean, y_std = bo_cont._gp.predict(np.linspace(-10, 10, 1000).reshape(-1, 1), return_std=True)
axs[0].set_title('Continuous')
axs[0].plot(np.linspace(-10, 10, 1000), target_function_1d(np.linspace(-10, 10, 1000)), 'k--', label='True function')
axs[0].plot(np.linspace(-10, 10, 1000), y_mean, label='Predicted mean')
axs[0].fill_between(np.linspace(-10, 10, 1000), y_mean - y_std, y_mean + y_std, alpha=0.3, label='Predicted std')
axs[0].plot(bo_cont.space.params, bo_cont.space.target, 'ro')

bo_disc.maximize(init_points=2, n_iter=10)
bo_disc.acquisition_function._fit_gp(bo_disc._gp, bo_disc.space)

y_mean, y_std = bo_disc._gp.predict(np.linspace(-10, 10, 1000).reshape(-1, 1), return_std=True)
axs[1].set_title('Discrete')
axs[1].plot(np.linspace(-10, 10, 1000), target_function_1d(np.linspace(-10, 10, 1000)), 'k--', label='True function')
axs[1].plot(np.linspace(-10, 10, 1000), y_mean, label='Predicted mean')
axs[1].fill_between(np.linspace(-10, 10, 1000), y_mean - y_std, y_mean + y_std, alpha=0.3, label='Predicted std')
axs[1].plot(bo_disc.space.params, bo_disc.space.target, 'ro')

for ax in axs:
    ax.grid(True)
fig.tight_layout()
```

--------------------------------

### Load and Continue Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/basic-tour.ipynb

Initialize a new optimizer and load a previously saved state, allowing optimization to resume from that point. Ensure bounds are correctly specified if they have changed.

```python
new_optimizer = BayesianOptimization(
    f=black_box_function,
    pbounds={"x": (-2, 3), "y": (-3, 3)},
    random_state=1,
    verbose=0
)

new_optimizer.load_state("./optimizer_state.json")

# Continue optimization
new_optimizer.maximize(
    init_points=0,
    n_iter=5
)
```

--------------------------------

### Probe Function

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/bayes_opt.md

Evaluate the function at the given points to guide the optimizer.

```APIDOC
## POST /bayesian-optimization/probe

### Description
Evaluate the function at the given points. This is useful to guide the optimizer.

### Method
POST

### Endpoint
/bayesian-optimization/probe

### Parameters
#### Request Body
- **params** (dict or iterable) - Required - The parameters where the optimizer will evaluate the function.
- **lazy** (boolean) - Optional - If True, the optimizer will evaluate the points when calling maximize(). Otherwise, it will evaluate them immediately. Defaults to True.

### Response
#### Success Response (200)
- None

#### Response Example
(No response body for this operation)
```

--------------------------------

### Visualize Initial Optimization Steps

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/visualization.ipynb

Plots the Gaussian Process and utility function after the initial random probes. This visualization shows the initial predictions and uncertainty of the model.

```python
plot_gp(optimizer, x, y)
```

--------------------------------

### Bayesian Optimization with Expected Improvement Per Unit Cost

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/acquisition_functions.ipynb

This snippet demonstrates initializing and running a Bayesian Optimization process using a custom acquisition function, ExpectedImprovementPerUnitCost. It also shows how to plot the Gaussian process and the acquisition function's cost.

```python
acquisition_function = ExpectedImprovementPerUnitCost(1e-4)
optimizer = BayesianOptimization(target, {'x': (-2, 10)}, acquisition_function=acquisition_function, random_state=SEED)
optimizer.maximize(init_points=3, n_iter=7)

fig, axs = plot_gp(optimizer, x, y)

ax2 = axs[1].twinx()
ax2.plot(x, acquisition_function.cost(x), label='Cost to access', color='orange')

axs[1].plot(x, -1 * super(acquisition.ExpectedImprovement, acquisition_function)._get_acq(gp=optimizer._gp)(x), 'k--', label='Expected Improvement')

lines, labels = axs[1].get_legend_handles_labels()
labels[0] = 'Expected Improvement per Unit Cost'
lines2, labels2 = ax2.get_legend_handles_labels()
axs[1].legend().remove()
ax2.legend(lines + lines2, labels + labels2, loc=2, bbox_to_anchor=(1.01, 1), borderaxespad=0.);
```

--------------------------------

### Initialize Bayesian Optimization with Custom Parameter

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

Instantiate `BayesianOptimization` using a custom parameter defined by `FixedPerimeterTriangleParameter` and a target function.

```python
param = FixedPerimeterTriangleParameter(
    name='sides',
    bounds=np.array([[0., 1.], [0., 1.], [0., 1.]]),
    perimeter=1.
)

pbounds = {'sides': param}
optimizer = BayesianOptimization(
    area_of_triangle,
    pbounds,
    random_state=1,
)

```

--------------------------------

### Implementing Custom Acquisition Function Parameters

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/acquisition_functions.ipynb

To enable saving and loading of custom acquisition functions, implement `get_acquisition_params` to return a dictionary of parameters and `set_acquisition_params` to load them. Include `_serialize_random_state` and `_deserialize_random_state` for perfect replicability.

```python
class CustomAcquisition:
    def __init__(self, param1, param2, random_state=None):
        self.param1 = param1
        self.param2 = param2
        self._random_state = self._serialize_random_state(random_state)

    def get_acquisition_params(self):
        return {
            'param1': self.param1,
            'param2': self.param2,
            'random_state': self._random_state
        }

    def set_acquisition_params(self, params):
        self.param1 = params['param1']
        self.param2 = params['param2']
        self._random_state = params['random_state']
        self._deserialize_random_state(self._random_state)

    def _serialize_random_state(self, random_state):
        # Implementation to serialize random state
        pass

    def _deserialize_random_state(self, serialized_state):
        # Implementation to deserialize random state
        pass
```

--------------------------------

### Initialize Sequential Domain Reduction Transformer

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/domain_reduction.ipynb

Creates an instance of the `SequentialDomainReductionTransformer` with a specified `minimum_window` of 0.5. This transformer will be used to dynamically adjust the search space during optimization.

```python
bounds_transformer = SequentialDomainReductionTransformer(minimum_window=0.5)
```

--------------------------------

### Initialize Bayesian Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/wiki/Introduction

Instantiate the BayesianOptimization object with a target function and parameter bounds. The `verbose` parameter controls the level of output during optimization.

```python
BayesianOptimization( f, pbounds, verbose=1 )
```

--------------------------------

### Optimization with Simple Inequality Constraint (y > x)

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/constraints.ipynb

Modify the objective function to return a very poor value (e.g., -10) when an inequality constraint (y <= x) is violated. This guides the optimizer away from invalid regions.

```python
def black_box_function_with_constraints(x, y):
    if y <= x:
        return -10
    else:
        return -x ** 2 - (y - 1) ** 2 + 1
    
optimizer = BayesianOptimization(
    f=black_box_function_with_constraints,
    pbounds=pbounds,
    random_state=0,
    verbose=0
)

optimizer.maximize(
    init_points=2,
    n_iter=100
)

print(f'the best solution with y>x is {optimizer.max}')
```

--------------------------------

### Instantiate Bayesian Optimizer

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/advanced-tour.ipynb

Instantiate the BayesianOptimization object. Set the acquisition function, parameter bounds, and other optimization parameters. The black-box function is not passed directly here when using the Suggest-Evaluate-Register paradigm.

```python
optimizer = BayesianOptimization(
    f=None,
    acquisition_function=acq,
    pbounds={'x': (-2, 2), 'y': (-3, 3)},
    verbose=2,
    random_state=1,
)
```

--------------------------------

### Optimizing SVM Hyperparameters with Bayesian Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

Tunes the hyperparameters of a Support Vector Machine (SVM) classifier using Bayesian Optimization. This example optimizes the kernel type and the log10 of the regularization parameter C.

```python
from sklearn.datasets import load_breast_cancer
from sklearn.svm import SVC
from sklearn.metrics import log_loss
from sklearn.model_selection import train_test_split
from bayes_opt import BayesianOptimization

data = load_breast_cancer()
X_train, y_train = data['data'], data['target']
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.2, random_state=1)
kernels = ['rbf', 'poly']

def f_target(kernel, log10_C):
    if kernel == 'poly2':
        kernel = 'poly'
        degree = 2
    elif kernel == 'poly3':
        kernel = 'poly'
        degree = 3
    elif kernel == 'rbf':
        degree = 3 # not used, equal to default

    C = 10**log10_C

    model = SVC(C=C, kernel=kernel, degree=degree, probability=True, random_state=1)
    model.fit(X_train, y_train)

    # Package looks for maximum, so we return -1 * log_loss
    loss = -1 * log_loss(y_val, model.predict_proba(X_val))
    return loss


params_svm ={
    'kernel': ['rbf', 'poly2', 'poly3'],
    'log10_C':(-1, +1),
}

optimizer = BayesianOptimization(
    f_target,
    params_svm,
    random_state=1,
    verbose=2
)

kernel = Matern(nu=2.5, length_scale=np.ones(optimizer.space.dim))
discrete_optimizer.set_gp_params(kernel=kernel)
optimizer.maximize(init_points=2, n_iter=8)
```

--------------------------------

### Initializing and Running Bayesian Optimization with Categorical Variables

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

Sets up a Bayesian Optimization problem with both continuous ('x1', 'x2') and categorical ('k') parameters. It then maximizes the objective function 'SPIRAL' over a specified number of iterations.

```python
pbounds = {'x1': (-1, 1), 'x2': (-1, 1), 'k': ('1', '2')}

categorical_optimizer = BayesianOptimization(
    f=SPIRAL,
    acquisition_function=acquisition.ExpectedImprovement(1e-2),
    pbounds=pbounds,
    verbose=2,
    random_state=1,
)
discrete_optimizer.set_gp_params(alpha=1e-3)

categorical_optimizer.maximize(
        init_points=2,
        n_iter=18,
    )
```

--------------------------------

### Defining Objective Functions for Categorical Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

Defines two objective functions, f1 and f2, and a SPIRAL function that selects between them based on a categorical input 'k'. This setup is used for Bayesian Optimization with mixed variable types.

```python
def f1(x1, x2):
    return -1*(x1 - np.sqrt(x1**2 + x2**2) * np.cos(np.sqrt(x1**2 + x2**2))**2 + 0.5 * np.sqrt(x1**2 + x2**2))

def f2(x1, x2):
    return -1*(x2 - np.sqrt(x1**2 + x2**2) * np.sin(np.sqrt(x1**2 + x2**2))**2 + 0.5 * np.sqrt(x1**2 + x2**2))

def SPIRAL(x1, x2, k):
    """cf Ladislav-Luksan
    """
    if k=='1':
        return f1(10 * x1, 10 * x2)
    elif k=='2':
        return f2(10 * x1, 10 * x2)
```

--------------------------------

### Initialize BayesianOptimization with ConstraintModel

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/constraints.ipynb

Create a BayesianOptimization model, passing the ConstraintModel instance as an additional keyword argument to handle constrained optimization.

```python

```

--------------------------------

### Prepare Data for Visualization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

Generates a grid of points within the parameter bounds and predicts function values using the Gaussian Process models from both optimizers. This is used for plotting.

```python
x = np.linspace(c_pbounds['x'][0], c_pbounds['x'][1], 1000)
y = np.linspace(c_pbounds['y'][0], c_pbounds['y'][1], 1000)

X, Y = np.meshgrid(x, y)

Z = discretized_function(X, Y)

params = [{'x': x_i, 'y': y_j} for y_j in y for x_i in x]
array_params = [continuous_optimizer._space.params_to_array(p) for p in params]
c_pred = continuous_optimizer._gp.predict(array_params).reshape(X.shape)
d_pred = discrete_optimizer._gp.predict(array_params).reshape(X.shape)

vmin = np.min([np.min(Z), np.min(c_pred), np.min(d_pred)])
vmax = np.max([np.max(Z), np.max(c_pred), np.max(d_pred)])

fig, axs = plt.subplots(1, 3)

axs[0].set_title('Actual function')
```

--------------------------------

### Initialize and run Bayesian Optimization with Greedy Acquisition

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/acquisition_functions.ipynb

Initializes the BayesianOptimization object with the target function, parameter space, and the custom GreedyAcquisition function. It then runs the optimization process for a specified number of initial points and iterations, followed by plotting the results.

```python
acquisition_function = GreedyAcquisition()
optimizer = BayesianOptimization(target, {'x': (-2, 10)}, acquisition_function=acquisition_function, random_state=SEED)
optimizer.maximize(init_points=3, n_iter=7)

plot_gp(optimizer, x, y);
```

--------------------------------

### Initialize Bayesian Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/README.md

Define the parameter bounds and initialize the BayesianOptimization object with the black-box function.

```python
pbounds = {'x': (2, 4), 'y': (-3, 3)}

optimizer = BayesianOptimization(
    f=black_box_function,
    pbounds=pbounds,
    random_state=1,
)
```

--------------------------------

### Initialize Bayesian Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/basic-tour.ipynb

Initialize the BayesianOptimization object with a black-box function and parameter bounds. Set verbose level for output during optimization.

```python
pbounds = {'x': (2, 4), 'y': (-3, 3)}
optimizer = BayesianOptimization(
    f=black_box_function,
    pbounds=pbounds,
    verbose=2, # verbose = 1 prints only when a maximum is observed, verbose = 0 is silent
    random_state=1,
)
```

--------------------------------

### Initialize Constrained Bayesian Optimizer

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/constraints.ipynb

Sets up the Bayesian Optimization process with defined parameter bounds and a constraint function. `verbose=0` ensures silent operation.

```python
pbounds = {'x': (0, 6), 'y': (0, 6)}

optimizer = BayesianOptimization(
    f=target_function,
    constraint=constraint,
    pbounds=pbounds,
    verbose=0, # verbose = 1 prints only when a maximum is observed, verbose = 0 is silent
    random_state=1,
)
```

--------------------------------

### Initialize Bayesian Optimizers

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

Initializes continuous and discrete Bayesian Optimization objects. Requires a function to optimize, an acquisition function, parameter bounds, and optional GP kernel parameters.

```python
continuous_optimizer = BayesianOptimization(
    f=discretized_function,
    acquisition_function=acquisition.ExpectedImprovement(xi=0.01, random_state=1),
    pbounds=c_pbounds,
    verbose=2,
    random_state=1,
)

continuous_optimizer.set_gp_params(kernel=Matern(nu=2.5, length_scale=np.ones(2)))
```

```python
discrete_optimizer = BayesianOptimization(
    f=discretized_function,
    acquisition_function=acquisition.ExpectedImprovement(xi=0.01, random_state=1),
    pbounds=d_pbounds,
    verbose=2,
    random_state=1,
)

discrete_optimizer.set_gp_params(kernel=Matern(nu=2.5, length_scale=np.ones(2)))
```

--------------------------------

### Run Bayesian Optimization with Thompson Sampling

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/acquisition_functions.ipynb

This snippet demonstrates how to initialize and run the BayesianOptimization class using the ThompsonSampling acquisition function. It also includes plotting the Gaussian Process and sampled acquisition functions.

```python
acquisition_function = ThompsonSampling()
optimizer = BayesianOptimization(target, {'x': (-2, 10)}, acquisition_function=acquisition_function, random_state=SEED)
optimizer.maximize(init_points=3, n_iter=7)

fig, axs = plot_gp(optimizer, x, y);

print("Adding GP samples to the plot... this can take up to several minutes.")

x_reduced = np.linspace(-2, 10, 100).reshape(-1, 1)
for i in range(99):
    y_sample = -1 * acquisition_function._get_acq(gp=optimizer._gp)(x_reduced)
    y_sample_argmax = (x_reduced[np.argmax(y_sample)], np.max(y_sample))
    axs[1].plot(x_reduced, y_sample, 'k', alpha=0.1, label='other samples from the GP' if i == 0 else None)
    axs[1].plot(*y_sample_argmax, 'r.', alpha=0.4, markersize=10, label='other argmaxes' if i == 0 else None)

axs[1].legend(loc=2, bbox_to_anchor=(1.01, 1), borderaxespad=0.);
```

--------------------------------

### Initialize and Run Bayesian Optimization with Multi-dimensional Constraint

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/constraints.ipynb

Initializes the Bayesian Optimization with a target function and a multi-dimensional nonlinear constraint. The optimizer is then executed. The library assumes conditional independence between multiple constraints.

```python
constraint = NonlinearConstraint(constraint_function_2_dim, constraint_lower, constraint_upper)
optimizer = BayesianOptimization(
    f=target_function,
    constraint=constraint,
    pbounds=pbounds,
    verbose=0, # verbose = 1 prints only when a maximum is observed, verbose = 0 is silent
    random_state=1,
)

optimizer.maximize(
    init_points=3,
    n_iter=10,
)
```

--------------------------------

### TargetSpace Initialization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/target_space.md

Initializes the TargetSpace with a target function, parameter bounds, and optional settings for random state and duplicate points.

```APIDOC
## TargetSpace Initialization

### Description
Initializes the TargetSpace with a target function, parameter bounds, and optional settings for random state and duplicate points.

### Parameters
- **target_func** - The function to be maximized.
- **pbounds** (dict) - Dictionary with parameter names as keys and a tuple with minimum and maximum values.
- **constraint** - Optional constraint function.
- **random_state** (int) - Optionally specify a seed for a random number generator.
- **allow_duplicate_points** (bool) - If True, the optimizer will allow duplicate points to be registered.
```

--------------------------------

### Initialize and Run Bayesian Optimization with Single Constraint

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/constraints.ipynb

Initializes the Bayesian Optimization with a target function and a single nonlinear constraint. The optimizer is then run for a specified number of initial points and iterations.

```python
constraint = NonlinearConstraint(constraint_function, constraint_lower, constraint_upper)

optimizer = BayesianOptimization(
    f=target_function,
    constraint=constraint,
    pbounds=pbounds,
    verbose=0, # verbose = 1 prints only when a maximum is observed, verbose = 0 is silent
    random_state=1,
)

optimizer.maximize(
    init_points=3,
    n_iter=10,
)
```

--------------------------------

### ExpectedImprovement Methods

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/acquisition/ExpectedImprovement.md

Methods for calculating acquisition values, managing parameters, and suggesting points.

```APIDOC
## ExpectedImprovement Methods

### base_acq

#### Description
Calculate the expected improvement.

#### Parameters
- **mean** (type) - Required - Mean of the predictive distribution.
- **std** (type) - Required - Standard deviation of the predictive distribution.

#### Returns
- np.ndarray - Acquisition function value.

#### Raises
- **ValueError** - If y_max is not set.

### decay_exploration

#### Description
Decay xi by a constant rate. Adjust exploration/exploitation trade-off by reducing xi.

#### Returns
- None

### get_acquisition_params

#### Description
Get the current acquisition function parameters.

#### Returns
- dict - Dictionary containing the current acquisition function parameters.

### set_acquisition_params

#### Description
Set the acquisition function parameters.

#### Parameters
- **params** (type) - Required - Dictionary containing the acquisition function parameters.

#### Returns
- None

### suggest

#### Description
Suggest a promising point to probe next.

#### Parameters
- **gp** (type) - Required - A fitted Gaussian Process.
- **target_space** (type) - Required - The target space to probe.
- **n_random** (int) - Optional - Number of random samples to use. Defaults to 10000.
- **n_smart** (int) - Optional - Number of starting points for the L-BFGS-B optimizer. Defaults to 10.
- **fit_gp** (bool) - Optional - Whether to fit the Gaussian Process to the target space. Set to False if the GP is already fitted. Defaults to True.
- **random_state** (type) - Optional - Random state to use for the optimization.

#### Returns
- np.ndarray - Suggested point to probe next.
```

--------------------------------

### Initialize BayesianOptimization object

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/README.md

Instantiate the BayesianOptimization object by providing the function to optimize and the bounds for its parameters. This is a constrained optimization technique.

```python
from bayes_opt import BayesianOptimization
```

--------------------------------

### Initialize BayesianOptimization Object

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/visualization.ipynb

Creates an instance of the BayesianOptimization class. It requires the target function, the bounds for the variables, and optionally an acquisition function and a random state for reproducibility.

```python
acquisition_function = acquisition.UpperConfidenceBound(kappa=5.)
optimizer = BayesianOptimization(target, {'x': (-2, 10)}, acquisition_function=acquisition_function, random_state=27)
```

--------------------------------

### Continue Optimization with New Bounds

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/basic-tour.ipynb

Resume the optimization process after adjusting bounds. Set `init_points` to 0 to only use Bayesian steps, continuing from the current state.

```python
optimizer.maximize(
    init_points=0,
    n_iter=5,
)
```

--------------------------------

### Explore Specific Points

Source: https://github.com/bayesian-optimization/bayesianoptimization/wiki/Introduction

The `explore` method allows you to specify a list of points that the optimizer must evaluate. These points are evaluated at the beginning of the optimization and can help reduce the overall time.

```python
self.explore({'x': [-1, 1, 2], 'y': [0, -3, 1]})
```

--------------------------------

### Import necessary libraries for Bayesian Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/constraints.ipynb

Import numpy, BayesianOptimization, matplotlib, and NonlinearConstraint for advanced constrained optimization tasks.

```python
import numpy as np
from bayes_opt import BayesianOptimization
import matplotlib.pyplot as plt
from scipy.optimize import NonlinearConstraint
```

--------------------------------

### Probe Optimization with Dictionary Parameters

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/basic-tour.ipynb

Specify points to probe using a dictionary of parameters. These points will be explored lazily on the next `maximize` call.

```python
optimizer.probe(
    params={"x": 0.5, "y": 0.7},
    lazy=True,
)
```

--------------------------------

### Maximize objective with standard optimizer

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/domain_reduction.ipynb

Runs the Bayesian optimization process using the standard configuration for a specified number of initial exploration points (`init_points`) and iterations (`n_iter`).

```python
standard_optimizer.maximize(
    init_points=2,
    n_iter=50,
)
```

--------------------------------

### Import necessary libraries for Bayesian Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/acquisition_functions.ipynb

Imports the required modules for Bayesian Optimization, including the main class, acquisition functions, and numerical/plotting utilities. Sets a random seed for reproducibility.

```python
from bayes_opt import BayesianOptimization
from bayes_opt import acquisition
import numpy as np

import matplotlib.pyplot as plt
from matplotlib import gridspec

SEED = 42
```

--------------------------------

### Standard Optimization Without Constraints

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/constraints.ipynb

Use this for basic optimization tasks where no constraints are present. Ensure the objective function and bounds are correctly defined.

```python
from bayes_opt import BayesianOptimization

def black_box_function_no_constraints(x, y):
    return -x ** 2 - (y - 1) ** 2 + 1

# Bounded region of parameter space
pbounds = {'x': (2, 4), 'y': (-3, 3)}

optimizer = BayesianOptimization(
    f=black_box_function_no_constraints,
    pbounds=pbounds,
    random_state=0,
    verbose=0
)

optimizer.maximize(
    init_points=2,
    n_iter=100
)

print(f'the best solution with no constraints is {optimizer.max}')
```

--------------------------------

### Import necessary libraries

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/exploitation_vs_exploration.ipynb

Imports the required libraries for Bayesian Optimization and plotting.

```python
import numpy as np
import matplotlib.pyplot as plt

from bayes_opt import BayesianOptimization
from bayes_opt import acquisition
```

--------------------------------

### Suggest Next Point

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/bayes_opt.md

Suggest a promising point to probe next based on the current optimization state.

```APIDOC
## GET /bayesian-optimization/suggest

### Description
Suggest a promising point to probe next.

### Method
GET

### Endpoint
/bayesian-optimization/suggest

### Response
#### Success Response (200)
- **suggestion** (dict) - A dictionary representing the suggested parameters.

#### Response Example
```json
{
  "param1": 0.2,
  "param2": 0.8
}
```
```

--------------------------------

### UpperConfidenceBound Initialization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/acquisition/UpperConfidenceBound.md

Parameters for initializing the UpperConfidenceBound class.

```APIDOC
### Parameters
#### Path Parameters
- **kappa** (float) - Optional - Governs the exploration/exploitation tradeoff. Lower prefers exploitation, higher prefers exploration. Defaults to 2.576.
- **exploration_decay** (float) - Optional - Decay rate for kappa. If None, no decay is applied.
- **exploration_decay_delay** (int) - Optional - Delay for decay. If None, decay is applied from the start.
- **random_state** (int) - Optional - Random state for reproducibility.
```

--------------------------------

### TargetSpace Methods

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/target_space.md

Methods for interacting with the TargetSpace, including parameter conversion, evaluation, sampling, and data registration.

```APIDOC
## TargetSpace Methods

### `params_to_array(params)`

**Description**: Convert a dictionary representation of parameters into an array version.

**Parameters**:
- **params** (dict) - A single point, with `len(params)` equal to the space's dimension.

**Returns**:
- `np.ndarray`: Representation of the parameters as an array.

### `probe(params)`

**Description**: Evaluate the target function on a point and register the result. If the point has been seen before and duplicate points are not allowed, a cached value is returned.

**Parameters**:
- **params** (list or np.ndarray) - A single point, with `len(params)` equal to the space's dimension.

**Returns**:
- `float | Tuple[float, float]`: The target function value, or a tuple containing the target function value and a constraint value.

**Example**:
```python
target_func = lambda p1, p2: p1 + p2
pbounds = {"p1": (0, 1), "p2": (1, 100)}
space = TargetSpace(target_func, pbounds)
space.probe([1, 5])
assert space.max()["target"] == 6
assert space.max()["params"] == {"p1": 1.0, "p2": 5.0}
```

### `random_sample(n_samples=0, random_state=None)`

**Description**: Sample random points from within the bounds of the space.

**Parameters**:
- **n_samples** (int, optional) - Number of samples to draw. If 0, a single sample is drawn. Defaults to 0.
- **random_state** (Any, optional) - The random state to use for sampling. Defaults to None.

**Returns**:
- `np.ndarray`: An array of sampled points. Shape is `(1, dim)` if `n_samples` is 0, otherwise `(n_samples, dim)`.

**Example**:
```python
target_func = lambda p1, p2: p1 + p2
pbounds = {"p1": (0, 1), "p2": (1, 100)}
space = TargetSpace(target_func, pbounds, random_state=0)
space.random_sample()
# Expected output: array([[ 0.54488318,   55.33253689]])
```

### `register(params, target, constraint_value=None)`

**Description**: Append a point and its target value to the known data. Runs in amortized constant time.

**Parameters**:
- **params** (list or np.ndarray) - A single point, with `len(params)` equal to the space's dimension.
- **target** (float) - The target function value.
- **constraint_value** (float, optional) - The constraint function value. Defaults to None.

**Raises**:
- `NotUniqueError`: If the point is not unique.

**Return type**: None

**Example**:
```python
target_func = lambda p1, p2: p1 + p2
pbounds = {"p1": (0, 1), "p2": (1, 100)}
space = TargetSpace(target_func, pbounds)
len(space)
# Expected output: 0
x = np.array([0, 0])
y = 1
space.register(x, y)
len(space)
# Expected output: 1
```

### `res()`

**Description**: Get all target values and constraint fulfillment for all parameters.

**Returns**:
- `list`: A list of dictionaries, where each dictionary contains 'target', 'params', and 'constraint' keys.

**Notes**: Does not report if points are within the bounds of the parameter space.

### `set_bounds(new_bounds)`

**Description**: Change the lower and upper search bounds of the parameters.

**Parameters**:
- **new_bounds** (dict) - A dictionary with parameter names as keys and their new bounds as values.

**Return type**: None

### `target` Property

**Description**: Get the target function values registered to this TargetSpace.

**Returns**:
- `np.ndarray`: An array of target function values.
```

--------------------------------

### Probe Optimization with Iterable Parameters

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/basic-tour.ipynb

Specify points to probe using an iterable of parameter values. Ensure the order matches the `pbounds` definition.

```python
optimizer.probe(
    params=[-0.3, 0.1],
    lazy=True,
)
```

--------------------------------

### Run Maximization with Zero Iterations

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/basic-tour.ipynb

Execute the maximization process with no initial points and no additional iterations, primarily to trigger the evaluation of probed points.

```python
optimizer.maximize(init_points=0, n_iter=0)
```

--------------------------------

### BayesianOptimization Class Initialization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/bayes_opt.md

Initializes the BayesianOptimization class to handle the optimization of a target function over a specific parameter space.

```APIDOC
## Initialize BayesianOptimization

### Description
Initializes the BayesianOptimization class to handle the optimization of a target function over a specific parameter space using Bayesian optimization.

### Method
Constructor

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **f** (function) - Required - The function to be maximized.
- **pbounds** (dict) - Required - Dictionary with parameter names as keys and a tuple with minimum and maximum values.
- **acquisition_function** (object) - Optional - The acquisition function to use for suggesting new points to evaluate. Defaults to UpperConfidenceBound for unconstrained problems and ExpectedImprovement for constrained problems.
- **constraint** (function) - Optional - A constraint function. The names of arguments of the constraint function and of `f` need to be the same.
- **random_state** (int or object) - Optional - Seed for random number generation. If None, an unseeded random state is generated.
- **verbose** (int) - Optional - The level of verbosity. Defaults to 2.
- **bounds_transformer** (object) - Optional - If provided, the transformation is applied to the bounds.
- **allow_duplicate_points** (bool) - Optional - If True, the optimizer will allow duplicate points to be registered. Defaults to False.

### Request Example
```json
{
  "f": "your_function",
  "pbounds": {"param1": (0, 1), "param2": (-5, 5)},
  "acquisition_function": null,
  "constraint": null,
  "random_state": null,
  "verbose": 2,
  "bounds_transformer": null,
  "allow_duplicate_points": false
}
```

### Response
#### Success Response (200)
None (Constructor does not return a value)

#### Response Example
None
```

--------------------------------

### Import necessary libraries for Bayesian Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

Imports the required libraries for Bayesian Optimization, including warnings, numpy, matplotlib, and specific modules from bayes_opt and sklearn.

```python
import warnings
import numpy as np
import matplotlib.pyplot as plt
from bayes_opt import BayesianOptimization
from bayes_opt import acquisition

from sklearn.gaussian_process.kernels import Matern

# suppress warnings about this being an experimental feature
warnings.filterwarnings(action="ignore")
```

--------------------------------

### View All Optimization Results

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/basic-tour.ipynb

Iterate through all recorded results, including parameters and target values for each optimization step, using the `res` property.

```python
for i, res in enumerate(optimizer.res):
    print("Iteration {}: \n\t".format(i, res))
```

--------------------------------

### Manual Optimization Loop

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/advanced-tour.ipynb

Manually implement the optimization loop by repeatedly suggesting points, evaluating the black-box function, and registering the results. This provides flexibility for logging, concurrent evaluation, or custom stopping criteria.

```python
for _ in range(5):
    next_point = optimizer.suggest()
    target = black_box_function(**next_point)
    optimizer.register(params=next_point, target=target)

    print(target, next_point)
print(optimizer.max)
```

--------------------------------

### Import necessary libraries

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/domain_reduction.ipynb

Imports required Python packages for Bayesian Optimization and plotting.

```python
import numpy as np
from bayes_opt import BayesianOptimization
from bayes_opt import SequentialDomainReductionTransformer
import matplotlib.pyplot as plt
```

--------------------------------

### Maximize objective with SDR

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/domain_reduction.ipynb

Runs the Bayesian optimization process for a specified number of initial exploration points (`init_points`) and iterations (`n_iter`), utilizing the sequential domain reduction.

```python
mutating_optimizer.maximize(
    init_points=2,
    n_iter=50,
)
```

--------------------------------

### BayesianOptimization State Management

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/bayes_opt.md

Provides methods for loading the state of a Bayesian Optimization process from a file.

```APIDOC
## Load BayesianOptimization State

### Description
Loads the optimizer state from a JSON file, allowing you to resume an optimization process.

### Method
POST

### Endpoint
`/load_state`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **path** (string) - Required - Path to the JSON file containing the optimizer state.

### Request Example
```json
{
  "path": "/path/to/your/optimizer_state.json"
}
```

### Response
#### Success Response (200)
None (This method modifies the optimizer state in place and does not return a value).

#### Response Example
None
```

--------------------------------

### Maximize Triangle Area with Sorted Parameters

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/parameter_types.ipynb

This code snippet demonstrates running Bayesian Optimization with a custom parameter class that sorts the triangle sides to exploit permutation symmetry. It initializes the optimizer with the custom parameter and then runs the maximization process.

```python
param = SortingFixedPerimeterTriangleParameter(
    name='sides',
    bounds=np.array([[0., 1.], [0., 1.], [0., 1.]]),
    perimeter=1.
)

pbounds = {'sides': param}
optimizer = BayesianOptimization(
    area_of_triangle,
    pbounds,
    random_state=1,
)

optimizer.logger.verbose = 2
optimizer.logger._default_cell_size = 15

optimizer.maximize(init_points=2, n_iter=8)
```

--------------------------------

### Bayesian Optimization with Upper Confidence Bound (Exploration)

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/exploitation_vs_exploration.ipynb

Performs Bayesian Optimization using the Upper Confidence Bound acquisition function with a high kappa value (10.0), encouraging exploration of uncertain regions.

```python
acquisition_function = acquisition.UpperConfidenceBound(kappa=10.)

bo = BayesianOptimization(
    f=f,
    acquisition_function=acquisition_function,
    pbounds={"x": (-2, 10)},
    verbose=0,
    random_state=987234,
)

bo.maximize(n_iter=10)

plot_bo(f, bo)
```

--------------------------------

### TargetSpace Methods

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/target_space.md

Provides access to various methods and properties of the TargetSpace class for managing and querying optimization data.

```APIDOC
## TargetSpace Methods and Properties

### array_to_params

#### Description
Convert an array representation of parameters into a dict version.

#### Parameters
- **x** - A single point, with len(x) == self.dim.

#### Returns
- dict - Representation of the parameters as dictionary.

### bounds (property)

#### Description
Get the bounds of this TargetSpace.

#### Returns
- np.ndarray

### calculate_bounds

#### Description
Calculate the float bounds of the parameter space.

#### Returns
- NDArray[Float]

### constraint (property)

#### Description
Get the constraint model.

#### Returns
- ConstraintModel

### constraint_values (property)

#### Description
Get the constraint values registered to this TargetSpace.

#### Returns
- np.ndarray

### continuous_dimensions (property)

#### Description
Get the continuous parameters.

#### Returns
- dict

### dim (property)

#### Description
Get the number of parameter names.

#### Returns
- int

### empty (property)

#### Description
Check if anything has been registered.

#### Returns
- bool

### kernel_transform

#### Description
Transform floating-point suggestions to values used in the kernel. Vectorized.

#### Parameters
- **value** - Value to transform.

#### Returns
- NDArray[Float]

### keys (property)

#### Description
Get the keys (or parameter names).

#### Returns
- list of str

### make_masks

#### Description
Create a dictionary of masks for the parameters. The mask can be used to select the corresponding parameters from an array.

#### Returns
- dict - A dictionary with the parameter names as keys and the corresponding mask as values.

### make_params

#### Description
Create a dictionary of parameters from a dictionary of bounds.

#### Parameters
- **pbounds** (dict) - A dictionary with the parameter names as keys and a tuple with minimum and maximum values.

#### Returns
- dict - A dictionary with the parameter names as keys and the corresponding parameter objects as values.

### mask (property)

#### Description
Return a boolean array of valid points. Points are valid if they satisfy both the constraint and boundary conditions.

#### Returns
- np.ndarray

### masks (property)

#### Description
Get the masks for the parameters.

#### Returns
- dict

### max

#### Description
Get maximum target value found and corresponding parameters. If there is a constraint present, the maximum value that fulfills the constraint within the parameter bounds is returned.

#### Returns
- dict - A dictionary with the keys ‘target’ and ‘params’. The value of ‘target’ is the maximum target value, and the value of ‘params’ is a dictionary with the parameter names as keys and the parameter values as values.

```

--------------------------------

### Random Sample Parameters

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/docsrc/reference/bayes_opt.md

Generate a random sample of parameters from the target space.

```APIDOC
## GET /bayesian-optimization/random-sample

### Description
Generate a random sample of parameters from the target space.

### Method
GET

### Endpoint
/bayesian-optimization/random-sample

### Parameters
#### Query Parameters
- **n** (integer) - Optional - Number of random samples to generate. Defaults to 1.

### Response
#### Success Response (200)
- **samples** (list of dict) - List of randomly sampled parameters.

#### Response Example
```json
[
  {
    "param1": 0.1,
    "param2": 0.9
  }
]
```
```

--------------------------------

### Define target and constraint functions

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/constraints.ipynb

Define the target function to maximize and the constraint function with its limit. Ensure constraint function parameter names match the target function.

```python
def target_function(x, y):
    # Gardner is looking for the minimum, but this packages looks for maxima, thus the sign switch
    return np.cos(2*x)*np.cos(y) + np.sin(x) 

def constraint_function(x, y):
    return np.cos(x) * np.cos(y) - np.sin(x) * np.sin(y)

constraint_limit = 0.5
```

--------------------------------

### Import Libraries for Bayesian Optimization

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/visualization.ipynb

Imports necessary libraries for Bayesian Optimization, including the main optimizer class, acquisition functions, numpy for numerical operations, and matplotlib for plotting.

```python
from bayes_opt import BayesianOptimization
from bayes_opt import acquisition
import numpy as np

import matplotlib.pyplot as plt
from matplotlib import gridspec
%matplotlib inline
```

--------------------------------

### Bayesian Optimization with Expected Improvement (Exploration)

Source: https://github.com/bayesian-optimization/bayesianoptimization/blob/master/examples/exploitation_vs_exploration.ipynb

Performs Bayesian Optimization using the Expected Improvement acquisition function with xi=0.1, balancing exploitation with exploration of potentially better regions.

```python
acquisition_function = acquisition.ExpectedImprovement(xi=0.1)

bo = BayesianOptimization(
    f=f,
    acquisition_function=acquisition_function,
    pbounds={"x": (-2, 10)},
    verbose=0,
    random_state=987234,
)

bo.maximize(n_iter=10)

plot_bo(f, bo)
```