### Run Examples using Git and Python

Source: https://hyperactive.readthedocs.io/en/latest/examples/general

Instructions on how to clone the Hyperactive repository from GitHub and run a provided example script using Python. This requires Git to be installed and configured.

```bash
# Clone the repository
git clone https://github.com/SimonBlanke/Hyperactive.git
cd Hyperactive/examples

# Run an example
python gfo/hill_climbing_example.py

```

--------------------------------

### Warm Starting Optimization with HillClimbing

Source: https://hyperactive.readthedocs.io/en/latest/examples/other

Accelerate convergence by starting optimization from known good points. This example uses the HillClimbing optimizer and requires a defined search_space and experiment object. The 'warm_start' parameter accepts a list of dictionaries representing initial parameter sets.

```python
from hyperactive.opt.gfo import HillClimbing

# Previous best parameters
warm_start_points = [
    {"n_estimators": 100, "max_depth": 10, "min_samples_split": 5},
]

optimizer = HillClimbing(
    search_space=search_space,
    n_iter=5,
    experiment=experiment,
    initialize={"warm_start": warm_start_points},
)
best_params = optimizer.solve()
```

--------------------------------

### Install Optuna Backend

Source: https://hyperactive.readthedocs.io/en/latest/installation

Installs the Optuna library, which provides backends for optimizers like TPE, CMA-ES, and NSGA-II. This can also be included via the 'all_extras' option during Hyperactive installation.

```bash
pip install optuna
```

--------------------------------

### Scikit-learn CV Experiment Setup (Python)

Source: https://hyperactive.readthedocs.io/en/latest/user_guide/experiments

This example shows how to set up a `SklearnCvExperiment` for tuning scikit-learn models. It includes importing necessary libraries, loading data, defining cross-validation strategy, and specifying the scoring metric. The experiment is then used with an optimizer.

```python
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import KFold
from sklearn.metrics import accuracy_score
from hyperactive.experiment.integrations import SklearnCvExperiment
from hyperactive.opt.gfo import HillClimbing

X, y = load_iris(return_X_y=True)

experiment = SklearnCvExperiment(
    estimator=RandomForestClassifier(random_state=42),
    X=X,
    y=y,
    cv=KFold(n_splits=5, shuffle=True, random_state=42),
    scoring=accuracy_score,  # Optional: defaults to estimator's score method
)

search_space = {
    "n_estimators": list(range(10, 200, 10)),
    "max_depth": list(range(1, 20)),
    "min_samples_split": list(range(2, 10)),
}

optimizer = HillClimbing(
    search_space=search_space,
    n_iter=5,
    experiment=experiment,
)
best_params = optimizer.solve()
```

--------------------------------

### Example: Using TPEOptimizer with Optuna

Source: https://hyperactive.readthedocs.io/en/latest/user_guide/optimizers/optuna

Demonstrates how to initialize and use the TPEOptimizer from Hyperactive's Optuna backend. This example assumes 'search_space' and 'objective' are pre-defined variables.

```python
from hyperactive.opt.optuna import TPEOptimizer

optimizer = TPEOptimizer(
    search_space=search_space,
    n_iter=5,
    experiment=objective,
)

```

--------------------------------

### Quick Example: Hyperactive HillClimbing Optimization

Source: https://hyperactive.readthedocs.io/en/latest/user_guide

Demonstrates a basic optimization task using Hyperactive's HillClimbing algorithm. It defines an objective function, a search space, and then uses the optimizer to find the best parameters. This example requires the 'hyperactive' library.

```python
from hyperactive.opt import HillClimbing

# 1. Define what to optimize
def objective(params):
    x, y = params["x"], params["y"]
    return -(x**2 + y**2)  # Minimize x² + y²

# 2. Define where to search
search_space = {
    "x": list(range(-10, 11)),
    "y": list(range(-10, 11)),
}

# 3. Choose how to optimize & run
optimizer = HillClimbing(search_space, n_iter=100, experiment=objective)
best = optimizer.solve()  # Returns {"x": 0, "y": 0}

```

--------------------------------

### Install Hyperactive with Optional Extras

Source: https://hyperactive.readthedocs.io/en/latest/installation

Installs Hyperactive with additional functionalities. Options include 'all_extras' for everything, 'sklearn-integration' (default), and 'sktime-integration' for time series data.

```bash
# Full installation with all extras (Optuna, PyTorch Lightning, etc.)
pip install hyperactive[all_extras]

# Scikit-learn integration (included by default)
pip install hyperactive[sklearn-integration]

# Sktime/skpro integration for time series
pip install hyperactive[sktime-integration]
```

--------------------------------

### Comparing Optimization Algorithms

Source: https://hyperactive.readthedocs.io/en/latest/examples/other

Compare the performance of different optimization strategies on the same problem. This example demonstrates how to instantiate and run multiple optimizers (HillClimbing, RandomSearch, BayesianOptimizer, ParticleSwarmOptimizer) and record their results. Ensure consistent use of 'random_state' for reproducible comparisons.

```python
from hyperactive.opt.gfo import (
    HillClimbing,
    RandomSearch,
    BayesianOptimizer,
    ParticleSwarmOptimizer,
)

optimizers = {
    "HillClimbing": HillClimbing,
    "RandomSearch": RandomSearch,
    "Bayesian": BayesianOptimizer,
    "ParticleSwarm": ParticleSwarmOptimizer,
}

results = {}
for name, OptClass in optimizers.items():
    optimizer = OptClass(
        search_space=search_space,
        n_iter=5,
        experiment=experiment,
        random_state=42,
    )
    best = optimizer.solve()
    score, _ = experiment.score(best)
    results[name] = {"params": best, "score": score}
    print(f"{name}: score={score:.4f}")
```

--------------------------------

### Clone and Run Hyperactive Tutorial Notebook

Source: https://hyperactive.readthedocs.io/en/latest/examples/interactive_tutorial

This code snippet demonstrates how to clone the Hyperactive tutorial repository, install its dependencies using pip, and launch the Jupyter Notebook for interactive learning. Ensure you have Git, pip, and Jupyter installed.

```bash
# Clone the tutorial repository
git clone https://github.com/SimonBlanke/hyperactive-tutorial.git
cd hyperactive-tutorial

# Install dependencies
pip install -r requirements.txt

# Launch Jupyter
jupyter notebook notebooks/hyperactive_tutorial.ipynb

```

--------------------------------

### Warm Starting Optimization with HillClimbing

Source: https://hyperactive.readthedocs.io/en/latest/user_guide/introduction

Demonstrates how to initialize an optimization process with warm starts using the HillClimbing optimizer. This allows the optimizer to begin from known good configurations, potentially speeding up convergence. It requires defining a search space, the number of iterations, an experiment function, and the warm start configurations.

```python
warm_start = [
    {"n_estimators": 100, "max_depth": 10},  # Start from known good point
]

optimizer = HillClimbing(
    search_space=search_space,
    n_iter=5,
    experiment=experiment,
    initialize={"warm_start": warm_start},
)

```

--------------------------------

### Install Hyperactive (Basic)

Source: https://hyperactive.readthedocs.io/en/latest/installation

Installs the core Hyperactive library and its essential dependencies using pip. This is suitable for most use cases, including integration with scikit-learn.

```bash
pip install hyperactive
```

--------------------------------

### Install Hyperactive for Development

Source: https://hyperactive.readthedocs.io/en/latest/installation

Installs Hyperactive from its source repository in development mode. This is useful for developers contributing to the project. It allows installing with test dependencies or all development dependencies.

```bash
# Clone the repository
git clone https://github.com/SimonBlanke/Hyperactive.git
cd Hyperactive

# Install in development mode with test dependencies
pip install -e ".[test]"

# Or install with all development dependencies
pip install -e ".[test,docs]"
```

--------------------------------

### Verify Hyperactive Installation

Source: https://hyperactive.readthedocs.io/en/latest/installation

A Python script to verify the Hyperactive installation. It imports the library, prints the version, and runs a quick optimization test using the HillClimbing optimizer.

```python
import hyperactive
print(f"Hyperactive version: {hyperactive.__version__}")

# Quick test
import numpy as np
from hyperactive.opt.gfo import HillClimbing


def objective(params):
    return -(params["x"] ** 2)


optimizer = HillClimbing(
    search_space={"x": np.arange(-5, 5, 0.1)},
    n_iter=5,
    experiment=objective,
)
best = optimizer.solve()
print(f"Test optimization successful: {best}")
```

--------------------------------

### Initialize Optimizer with Warm Starts in Python

Source: https://hyperactive.readthedocs.io/en/latest/user_guide/optimizers/configuration

Shows how to initialize a Hyperactive optimizer using pre-defined 'warm_start' points. This strategy is useful when you have known good starting configurations for the optimization process.

```python
# Start from known good points
optimizer = HillClimbing(
    search_space=search_space,
    n_iter=5,
    experiment=objective,
    initialize= {
        "warm_start": [
            {"param1": 10, "param2": 0.5},
            {"param1": 20, "param2": 0.3},
        ]
    },
)

```

--------------------------------

### Optimize Scikit-learn RandomForestClassifier with Hyperactive in Python

Source: https://hyperactive.readthedocs.io/en/latest/get_started

This example shows how to tune a Scikit-learn RandomForestClassifier using Hyperactive's SklearnCvExperiment and HillClimbing optimizer. It loads the Iris dataset, defines a search space for hyperparameters, and optimizes the model. Dependencies include scikit-learn, numpy, and hyperactive.

```python
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from hyperactive.experiment.integrations import SklearnCvExperiment
from hyperactive.opt.gfo import HillClimbing

# Load data
X, y = load_iris(return_X_y=True)

# Create an experiment that handles cross-validation
experiment = SklearnCvExperiment(
    estimator=RandomForestClassifier(random_state=42),
    X=X,
    y=y,
    cv=5,
)

# Define hyperparameter search space
search_space = {
    "n_estimators": list(range(10, 200, 10)),
    "max_depth": list(range(1, 20)),
    "min_samples_split": list(range(2, 10)),
}

# Optimize
optimizer = HillClimbing(
    search_space=search_space,
    n_iter=5,
    experiment=experiment,
)
best_params = optimizer.solve()

print(f"Best hyperparameters: {best_params}")

```

--------------------------------

### Initialize and Run QMCOptimizer with Scikit-learn Experiment

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.QMCOptimizer

Demonstrates basic usage of QMCOptimizer with a scikit-learn experiment. It sets up an SVC classifier, defines a parameter space, initializes the optimizer, and then runs the optimization process to find the best parameters.

```python
from hyperactive.experiment.integrations import SklearnCvExperiment
from hyperactive.opt.optuna import QMCOptimizer
from sklearn.datasets import load_iris
from sklearn.svm import SVC

X, y = load_iris(return_X_y=True)
sklearn_exp = SklearnCvExperiment(estimator=SVC(), X=X, y=y)
param_space = {
    "C": (0.01, 10),
    "gamma": (0.0001, 10),
}

optimizer = QMCOptimizer(
    param_space=param_space, n_trials=50, experiment=sklearn_exp
)
best_params = optimizer.solve()
```

--------------------------------

### Set up and Evaluate a PyTorch Experiment

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.experiment.integrations.TorchExperiment

This snippet demonstrates how to set up a PyTorch experiment using a custom data module and a simple lightning module. It then proceeds to evaluate the experiment with specified parameters, returning the validation result and associated metadata.

```python
>>> datamodule = RandomDataModule(batch_size=16)
>>> datamodule.setup()
>>>
>>> # Create Experiment
>>> experiment = TorchExperiment(
...     datamodule=datamodule,
...     lightning_module=SimpleLightningModule,
...     trainer_kwargs={'max_epochs': 3},
...     objective_metric="val_loss"
... )
>>>
>>> params = {"input_dim": 10, "hidden_dim": 16, "lr": 1e-3}
>>>
>>> val_result, metadata = experiment._evaluate(params)
```

--------------------------------

### Complete Example: Tuning a Random Forest Classifier

Source: https://hyperactive.readthedocs.io/en/latest/user_guide/introduction

A comprehensive example demonstrating how to tune a RandomForestClassifier using Hyperactive. It covers data loading, experiment definition, search space setup, optimizer selection, and running the optimization.

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from hyperactive.experiment.integrations import SklearnCvExperiment
from hyperactive.opt.gfo import BayesianOptimizer

# 1. Load your data
X, y = load_iris(return_X_y=True)

# 2. Define the experiment (what to optimize)
experiment = SklearnCvExperiment(
    estimator=RandomForestClassifier(),
    X=X, y=y, cv=5,
)

# 3. Define the search space (where to search)
search_space = {
    "n_estimators": list(range(10, 200, 10)),
    "max_depth": [3, 5, 10, 20, None],
    "min_samples_split": [2, 5, 10],
}

# 4. Choose an optimizer (how to search)
optimizer = BayesianOptimizer(
    search_space=search_space,
    n_iter=5,
    experiment=experiment,
    random_state=42,
)

# 5. Run and get the best parameters
best_params = optimizer.solve()
print(f"Best parameters: {best_params}")

```

--------------------------------

### Avoiding Overly Fine Initial Hyperparameter Search (Python)

Source: https://hyperactive.readthedocs.io/en/latest/user_guide/search_spaces

This snippet warns against starting hyperparameter optimization with an excessively fine search space. It contrasts a bad example of too many values for an initial search with a better example that uses a coarser range, emphasizing the benefit of refining later.

```python
# Bad for initial search
"lr": np.logspace(-4, -1, 100)

# Better: start coarse, refine later
"lr": np.logspace(-4, -1, 10)
```

--------------------------------

### DirectAlgorithm Initialization and Usage with Scikit-learn

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.DirectAlgorithm

Demonstrates how to initialize and use the DirectAlgorithm optimizer with a scikit-learn experiment. This involves defining an experiment (e.g., SVC with Iris dataset), configuring the search space and iterations for DirectAlgorithm, and then running the optimization process.

```python
from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from hyperactive.opt import DirectAlgorithm
import numpy as np

# 1. defining the experiment to optimize
X, y = load_iris(return_X_y=True)

sklearn_exp = SklearnCvExperiment(
    estimator=SVC(),
    X=X,
    y=y,
)

# 2. setting up the directAlgorithm optimizer
config = {
    "search_space": {
        "C": [0.01, 0.1, 1, 10],
        "gamma": [0.0001, 0.01, 0.1, 1, 10],
    },
    "n_iter": 100,
}
optimizer = DirectAlgorithm(experiment=sklearn_exp, **config)

# 3. running the optimization
best_params = optimizer.solve()

# Best parameters can also be accessed via
best_params = optimizer.best_params_
```

--------------------------------

### Minimize Objective Function in Python

Source: https://hyperactive.readthedocs.io/en/latest/faq/getting_started

Demonstrates how to configure Hyperactive to minimize an objective function by returning the negative of the metric. This is useful when the optimization goal is to reduce a value like error.

```python
def objective(params):
    error = compute_error(params)
    return -error  # Negate to minimize
```

--------------------------------

### PatternSearch Initialization and Usage Example

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.PatternSearch

Demonstrates how to initialize and use the PatternSearch optimizer with a scikit-learn experiment. This includes setting up the experiment, defining the search configuration, and running the optimization process to find the best parameters.

```python
from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from hyperactive.opt import PatternSearch
import numpy as np

# 1. defining the experiment to optimize:
X, y = load_iris(return_X_y=True)

sklearn_exp = SklearnCvExperiment(
    estimator=SVC(),
    X=X,
    y=y,
)

# 2. setting up the patternSearch optimizer:
config = {
    "search_space": {
        "C": [0.01, 0.1, 1, 10],
        "gamma": [0.0001, 0.01, 0.1, 1, 10],
    },
    "n_iter": 100,
}
optimizer = PatternSearch(experiment=sklearn_exp, **config)

# 3. running the optimization:
best_params = optimizer.solve()

# Best parameters can also be accessed via:
best_params = optimizer.best_params_
```

--------------------------------

### DownhillSimplexOptimizer Initialization and Usage

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.DownhillSimplexOptimizer

Demonstrates how to initialize and use the DownhillSimplexOptimizer with a scikit-learn experiment. This includes setting up the experiment, defining the search space and optimizer configuration, and running the optimization process.

```python
from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from hyperactive.opt import DownhillSimplexOptimizer
import numpy as np

# 1. defining the experiment to optimize:
X, y = load_iris(return_X_y=True)
sklearn_exp = SklearnCvExperiment(
    estimator=SVC(),
    X=X,
    y=y,
)

# 2. setting up the downhillSimplexOptimizer optimizer:
config = {
    "search_space": {
        "C": [0.01, 0.1, 1, 10],
        "gamma": [0.0001, 0.01, 0.1, 1, 10],
    },
    "n_iter": 100,
}
optimizer = DownhillSimplexOptimizer(experiment=sklearn_exp, **config)

# 3. running the optimization:
best_params = optimizer.solve()

# Best parameters can also be accessed via:
best_params = optimizer.best_params_
```

--------------------------------

### Use Global Search Algorithms for Optimization (Python)

Source: https://hyperactive.readthedocs.io/en/latest/troubleshooting/results

Addresses the issue of local search algorithms getting stuck by suggesting the use of global search algorithms. Examples include RandomSearch and BayesianOptimizer.

```python
from hyperactive.opt.gfo import RandomSearch, BayesianOptimizer

```

--------------------------------

### QMCOptimizer Class Initialization

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.QMCOptimizer

Illustrates the initialization of the QMCOptimizer class with various parameters. This includes defining the search space, number of trials, initialization methods, random state, early stopping criteria, maximum score, QMC type, scrambling, and an optional experiment object.

```python
optimizer = QMCOptimizer(
    param_space=param_space, 
    n_trials=100, 
    initialize=None, 
    random_state=None, 
    early_stopping=None, 
    max_score=None, 
    qmc_type='sobol', 
    scramble=True, 
    experiment=None
)
```

--------------------------------

### Initialize BayesianOptimizer in Python

Source: https://hyperactive.readthedocs.io/en/latest/get_started

This code snippet shows the initialization of a BayesianOptimizer from Hyperactive. It requires a search space, number of iterations, and an experiment object. This optimizer is suitable for expensive evaluations and smart exploration.

```python
from hyperactive.opt.gfo import BayesianOptimizer


```

```python
optimizer = BayesianOptimizer(
    search_space=search_space,
    n_iter=5,
    experiment=experiment,
)
best_params = optimizer.solve()


```

--------------------------------

### Migrating from Hyperactive v4 .run() to v5 .solve()

Source: https://hyperactive.readthedocs.io/en/latest/user_guide/migration

Highlights the renaming of the method used to start the optimization process. In v4, `hyper.run()` was used, while in v5, the equivalent method is `optimizer.solve()`. This change is part of the refactoring in v5.

```python
# Old
hyper.run()

# New
best_params = optimizer.solve()

```

--------------------------------

### Basic GPOptimizer Usage with SklearnCvExperiment

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.GPOptimizer

Demonstrates the basic usage of GPOptimizer with a scikit-learn cross-validation experiment. It shows how to define the parameter space, initialize the optimizer with an experiment and other parameters, and then run the optimization process to find the best parameters.

```python
from hyperactive.experiment.integrations import SklearnCvExperiment
from hyperactive.opt.optuna import GPOptimizer
from sklearn.datasets import load_iris
from sklearn.svm import SVC
X, y = load_iris(return_X_y=True)
sklearn_exp = SklearnCvExperiment(estimator=SVC(), X=X, y=y)
param_space = {
    "C": (0.01, 10),
    "gamma": (0.0001, 10),
}
optimizer = GPOptimizer(
    param_space=param_space, n_trials=50, experiment=sklearn_exp
)
best_params = optimizer.solve()
```

--------------------------------

### Tune Scikit-learn SVC with OptCV and Hyperactive in Python

Source: https://hyperactive.readthedocs.io/en/latest/get_started

This snippet demonstrates using Hyperactive's OptCV wrapper for a simpler Scikit-learn integration, similar to GridSearchCV. It tunes an SVC model using HillClimbing optimization. Dependencies include scikit-learn, and hyperactive.

```python
from sklearn.svm import SVC
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from hyperactive.integrations.sklearn import OptCV
from hyperactive.opt.gfo import HillClimbing

# Load and split data
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Define optimizer with search space
search_space = {"kernel": ["linear", "rbf"], "C": [0.1, 1, 10, 100]}
optimizer = HillClimbing(search_space=search_space, n_iter=5)

# Create tuned estimator (like GridSearchCV)
tuned_svc = OptCV(SVC(), optimizer)

# Fit and predict as usual
tuned_svc.fit(X_train, y_train)
y_pred = tuned_svc.predict(X_test)

# Access results
print(f"Best params: {tuned_svc.best_params_}")
print(f"Best estimator: {tuned_svc.best_estimator_}")

```

--------------------------------

### Optimize Custom Function with HillClimbing in Python

Source: https://hyperactive.readthedocs.io/en/latest/get_started

This snippet demonstrates how to optimize a custom objective function using the HillClimbing optimizer in Hyperactive. It defines an objective function, a search space, and then uses the optimizer to find the best parameters. Dependencies include numpy and hyperactive.opt.gfo.HillClimbing.

```python
import numpy as np
from hyperactive.opt.gfo import HillClimbing


# 1. Define your objective function
def objective(params):
    x = params["x"]
    y = params["y"]
    return -(x**2 + y**2)  # Hyperactive maximizes by default


# 2. Define the search space
search_space = {
    "x": np.arange(-5, 5, 0.1),
    "y": np.arange(-5, 5, 0.1),
}

# 3. Create an optimizer and solve
optimizer = HillClimbing(
    search_space=search_space,
    n_iter=5,
    experiment=objective,
)
best_params = optimizer.solve()

print(f"Best parameters: {best_params}")

```

--------------------------------

### Install Hyperactive with Pip

Source: https://hyperactive.readthedocs.io/en/latest/index

Installs the Hyperactive library using pip. Different commands are provided for base installation, installation with all extras, and installation with specific integrations for scikit-learn or sktime.

```bash
pip install hyperactive
```

```bash
pip install hyperactive[all_extras]
```

```bash
pip install hyperactive[sklearn-integration]
```

```bash
pip install hyperactive[sktime-integration]
```

--------------------------------

### Install Hyperactive using pip

Source: https://hyperactive.readthedocs.io/en/latest/troubleshooting/installation

Installs the Hyperactive library using pip. The 'all_extras' option installs all optional dependencies for full functionality.

```bash
pip install hyperactive

# Or with extras
pip install hyperactive[all_extras]
```

--------------------------------

### Instance Method: get_config

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.base.BaseExperiment

Retrieves the current configuration flags for the instance.

```APIDOC
## get_config

### Description
Get config flags for self. Configs are key-value pairs of `self`, typically used as transient flags for controlling behaviour. `get_config` returns dynamic configs, which override the default configs. Default configs are set in the class attribute `_config` of the class or its parent classes, and are overridden by dynamic configs set via `set_config`. Configs are retained under `clone` or `reset` calls.

### Method
Instance Method

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
None

### Response
#### Success Response (200)
- **config_dict** (dict) - A dictionary of config name: config value pairs.

#### Response Example
None
```

--------------------------------

### ForestOptimizer Initialization and Usage

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.ForestOptimizer

Demonstrates the basic setup and execution of ForestOptimizer for hyperparameter tuning with a scikit-learn experiment. It covers defining the experiment, configuring the optimizer with a search space and iteration count, and initiating the optimization process.

```python
from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from hyperactive.opt import ForestOptimizer
import numpy as np

X, y = load_iris(return_X_y=True)

sklearn_exp = SklearnCvExperiment(
    estimator=SVC(),
    X=X,
    y=y,
)

config = {
    "search_space": {
        "C": [0.01, 0.1, 1, 10],
        "gamma": [0.0001, 0.01, 0.1, 1, 10],
    },
    "n_iter": 100,
}

optimizer = ForestOptimizer(experiment=sklearn_exp, **config)
best_params = optimizer.solve()

# Best parameters can also be accessed via:
best_params = optimizer.best_params_
```

--------------------------------

### Run LipschitzOptimizer and Get Best Parameters

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.LipschitzOptimizer

Shows how to execute the optimization process using the `solve` method of the LipschitzOptimizer and how to access the best found parameters.

```python
# 3. Running the optimization
best_params = optimizer.solve()

# Best parameters can also be accessed via
best_params = optimizer.best_params_
```

--------------------------------

### Verify Hyperactive Installation

Source: https://hyperactive.readthedocs.io/en/latest/troubleshooting/installation

Verifies that Hyperactive is installed correctly by importing the library and printing its version. This command can be run directly in the Python interpreter.

```python
python -c "import hyperactive; print(hyperactive.__version__)"
```

--------------------------------

### Instance Methods for Evaluation and Configuration

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.experiment.integrations.SkproProbaRegExperiment

This section covers instance methods for evaluating parameters and retrieving configuration flags.

```APIDOC
## Instance Methods for Evaluation and Configuration

### Description
Provides instance methods for evaluating parameters and retrieving dynamic configuration settings.

### Methods

- `evaluate(params)`
  - **Description**: Evaluates the given parameters and returns the result along with metadata.
  - **Parameters**:
    - `params` (dict): A dictionary with string keys representing the parameters to evaluate.
  - **Returns**:
    - `float`: The evaluated value of the parameters.
    - `dict`: Additional metadata about the evaluation.

- `get_config()`
  - **Description**: Retrieves the current configuration flags for the instance, including dynamic overrides.
  - **Returns**:
    - `config_dict` (dict): A dictionary of configuration names and their values.
```

--------------------------------

### Quick Start: Simple Optimization with Hill Climbing

Source: https://hyperactive.readthedocs.io/en/latest/user_guide/introduction

Demonstrates the simplest way to perform optimization using the HillClimbing algorithm from Hyperactive. It defines a score function and then uses the optimizer to find the best parameter.

```python
from hyperactive.opt.gfo import HillClimbing

def score(p):
    return -(p["x"] ** 2)  # Find x that minimizes x²

opt = HillClimbing({"x": range(-10, 11)}, experiment=score)
best = opt.solve()  # {"x": 0}

```

--------------------------------

### Class Method: _create_test_instances_and_names

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.base.BaseExperiment

Creates a list of all test instances and their corresponding names.

```APIDOC
## _classmethod _create_test_instances_and_names

### Description
Create a list of all test instances and a list of names for them.

### Method
classmethod

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
None

### Response
#### Success Response (200)
- **objs** (list) - A list of instances of the class.
- **names** (list) - A list of strings, representing the names of the instances.

#### Response Example
None
```

--------------------------------

### Basic SimulatedAnnealing Usage with Scikit-learn

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.SimulatedAnnealing

Demonstrates a basic example of using SimulatedAnnealing to optimize hyperparameters for a scikit-learn SVC model. It includes setting up the experiment, configuring the optimizer, and running the optimization process.

```python
from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from hyperactive.opt import SimulatedAnnealing
import numpy as np

X, y = load_iris(return_X_y=True)

sklearn_exp = SklearnCvExperiment(
    estimator=SVC(),
    X=X,
    y=y,
)

config = {
    "search_space": {
        "C": [0.01, 0.1, 1, 10],
        "gamma": [0.0001, 0.01, 0.1, 1, 10],
    },
    "n_iter": 100,
}

optimizer = SimulatedAnnealing(experiment=sklearn_exp, **config)
best_params = optimizer.solve()

# Best parameters can also be accessed via:
best_params = optimizer.best_params_
```

--------------------------------

### Install Specific Older Version of Hyperactive

Source: https://hyperactive.readthedocs.io/en/latest/installation

Installs a specific older version of Hyperactive, such as v4.8.0, using pip. This is useful if you need to maintain compatibility with older projects or features.

```bash
pip install hyperactive==4.8.0
```

--------------------------------

### Install Hyperactive Optional Dependencies

Source: https://hyperactive.readthedocs.io/en/latest/troubleshooting/installation

Installs specific optional dependencies for Hyperactive using pip. This allows integration with libraries like scikit-learn or using Optuna as a backend.

```bash
# For scikit-learn integration
pip install hyperactive[sklearn-integration]

# For Optuna backend
pip install hyperactive[optuna]

# For all extras
pip install hyperactive[all_extras]
```

--------------------------------

### Initialize ParticleSwarmOptimizer with SklearnCvExperiment

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.ParticleSwarmOptimizer

Demonstrates how to initialize the ParticleSwarmOptimizer for a scikit-learn experiment. It involves defining the search space and passing it along with the experiment object to the optimizer.

```python
from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from hyperactive.opt import ParticleSwarmOptimizer
import numpy as np

X, y = load_iris(return_X_y=True)

sklearn_exp = SklearnCvExperiment(
    estimator=SVC(),
    X=X,
    y=y,
)

config = {
    "search_space": {
        "C": [0.01, 0.1, 1, 10],
        "gamma": [0.0001, 0.01, 0.1, 1, 10],
    },
    "n_iter": 100,
}

optimizer = ParticleSwarmOptimizer(experiment=sklearn_exp, **config)
```

--------------------------------

### Install All Extras for Hyperactive

Source: https://hyperactive.readthedocs.io/en/latest/examples/integrations

Installs Hyperactive with all available extra dependencies, including sktime/skpro for time series and PyTorch Lightning support.

```bash
pip install hyperactive[all_extras]
```

--------------------------------

### Create Test Instance (Python)

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.NSGAIIOptimizer

The `_create_test_instance` class method constructs an instance of the class using the first defined test parameter set. This is useful for testing purposes, allowing the creation of a default or specified test configuration.

```python
def _create_test_instance(parameter_set='default'):
    """Construct an instance of the class, using first test parameter set.

    Parameters
    ----------
    parameter_set : str, default=”default”
        Name of the set of test parameters to return, for use in tests. If no
        special parameters are defined for a value, will return “default” set.

    Returns
    -------
    instance : instance of the class with default parameters
    """
    return cls(**cls.get_test_params(parameter_set=parameter_set)[0])
```

--------------------------------

### Install Sktime Integration for Hyperactive

Source: https://hyperactive.readthedocs.io/en/latest/examples/integrations

Installs the Hyperactive library with the necessary dependencies for sktime integration, enabling time series forecasting and classification capabilities.

```bash
pip install hyperactive[sktime-integration]
```

--------------------------------

### Initialize LipschitzOptimizer with Scikit-learn Experiment

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.LipschitzOptimizer

Demonstrates how to initialize the LipschitzOptimizer with a scikit-learn experiment. This involves defining the experiment using SklearnCvExperiment and then passing it along with the search configuration to the LipschitzOptimizer.

```python
from hyperactive.opt import LipschitzOptimizer
from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.datasets import load_iris
from sklearn.svm import SVC
import numpy as np

# 1. Defining the experiment to optimize
X, y = load_iris(return_X_y=True)
sklearn_exp = SklearnCvExperiment(
    estimator=SVC(),
    X=X,
    y=y,
)

# 2. Setting up the lipschitzOptimizer optimizer
config = {
    "search_space": {
        "C": [0.01, 0.1, 1, 10],
        "gamma": [0.0001, 0.01, 0.1, 1, 10],
    },
    "n_iter": 100,
}
optimizer = LipschitzOptimizer(experiment=sklearn_exp, **config)
```

--------------------------------

### Example Commit Message Structure

Source: https://hyperactive.readthedocs.io/en/latest/get_involved/contributing

Provides an example of a well-structured commit message, including a concise subject line and a detailed body explaining the changes and their motivation.

```git
Add Bayesian optimizer warm start support

- Add warm_start parameter to BayesianOptimizer
- Update documentation with usage examples
- Add tests for warm start functionality
```

--------------------------------

### SimulatedAnnealing Constructor

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.SimulatedAnnealing

Initializes the SimulatedAnnealing optimizer with various configuration parameters.

```APIDOC
## SimulatedAnnealing Constructor

### Description
Initializes the SimulatedAnnealing optimizer with various configuration parameters.

### Method
`__init__`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **search_space** (dict[str, list]) - Required - The search space to explore. A dictionary with parameter names as keys and a numpy array as values.
- **initialize** (dict[str, int]) - Required - The method to generate initial positions. A dictionary with the following key literals and the corresponding value type: {“grid”: int, “vertices”: int, “random”: int, “warm_start”: list[dict]}
- **constraints** (list[callable]) - Optional - A list of constraints, where each constraint is a callable. The callable returns True or False dependend on the input parameters.
- **random_state** (None, int) - Optional - If None, create a new random state. If int, create a new random state seeded with the value.
- **rand_rest_p** (float) - Optional - The probability of a random iteration during the the search process.
- **epsilon** (float) - Optional - The step-size for the climbing.
- **distribution** (str) - Optional - The type of distribution to sample from.
- **n_neighbours** (int) - Optional - The number of neighbours to sample and evaluate before moving to the best of those neighbours.
- **annealing_rate** (float) - Optional - The rate at which the temperature is annealed.
- **start_temp** (float) - Optional - The initial temperature.
- **n_iter** (int) - Optional, default=100 - The number of iterations to run the optimizer.
- **verbose** (bool) - Optional, default=False - If True, print the progress of the optimization process.
- **experiment** (BaseExperiment) - Optional - The experiment to optimize parameters for. Optional, can be passed later via `set_params`.

### Request Example
```json
{
  "search_space": {
    "param1": [1, 2, 3],
    "param2": ["a", "b"]
  },
  "initialize": {
    "random": 10
  },
  "n_iter": 50,
  "verbose": true
}
```

### Response
#### Success Response (200)
This method initializes the object and does not return a value directly.

#### Response Example
None
```

--------------------------------

### QMCOptimizer Initialization

Source: https://hyperactive.readthedocs.io/en/latest/api_reference/auto_generated/hyperactive.opt.QMCOptimizer

Initializes the QMCOptimizer with a specified parameter space and optimization settings.

```APIDOC
## QMCOptimizer

### Description
Quasi-Monte Carlo optimizer.

### Method
`__init__`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **param_space** (dict[str, tuple or list or optuna distributions]) - Required - The search space to explore. Dictionary with parameter names as keys and either tuples/lists of (low, high) or optuna distribution objects as values.
- **n_trials** (int) - Optional, default=100 - Number of optimization trials.
- **initialize** (dict[str, int]) - Optional, default=None - The method to generate initial positions. A dictionary with the following key literals and the corresponding value type: {“grid”: int, “vertices”: int, “random”: int, “warm_start”: list[dict]}.
- **random_state** (None, int) - Optional, default=None - If None, create a new random state. If int, create a new random state seeded with the value.
- **early_stopping** (int) - Optional, default=None - Number of trials after which to stop if no improvement.
- **max_score** (float) - Optional, default=None - Maximum score threshold. Stop optimization when reached.
- **qmc_type** (str) - Optional, default='sobol' - Type of QMC sequence. Options: "sobol", "halton".
- **scramble** (bool) - Optional, default=True - Whether to scramble the QMC sequence.
- **experiment** (BaseExperiment) - Optional - The experiment to optimize parameters for. Optional, can be passed later via `set_params`.

### Request Example
```json
{
  "param_space": {
    "C": [0.01, 10],
    "gamma": [0.0001, 10]
  },
  "n_trials": 50,
  "experiment": "<SklearnCvExperiment object>"
}
```

### Response
#### Success Response (200)
- **QMCOptimizer object** - An instance of the QMCOptimizer class.

#### Response Example
```json
{
  "optimizer": "<QMCOptimizer object>"
}
```
```

--------------------------------

### Install Hyperactive in Development Mode

Source: https://hyperactive.readthedocs.io/en/latest/get_involved/contributing

Installs the Hyperactive package in editable mode, including dependencies for testing and documentation. This allows changes in the source code to be reflected immediately without reinstallation.

```bash
pip install -e ".[test,docs]"
```