### Initialize and Use QMCSampler

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/samplers/_qmc.html

Demonstrates how to create a QMCSampler and use it to optimize a simple quadratic function. This example shows basic setup and trial execution.

```python
import optuna


def objective(trial):
    x = trial.suggest_float("x", -1, 1)
    y = trial.suggest_int("y", -1, 1)
    return x**2 + y


sampler = optuna.samplers.QMCSampler()
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=8)
```

--------------------------------

### Install Optuna Dashboard

Source: https://optuna.readthedocs.io/en/stable/index.html

Install the Optuna Dashboard for real-time web visualization of optimization history and hyperparameter importance. This command-line installation is straightforward.

```bash
$ pip install optuna-dashboard

```

--------------------------------

### Install Optuna via pip

Source: https://optuna.readthedocs.io/en/stable/installation.html

Standard installation method for the latest stable release of Optuna.

```bash
$ pip install optuna
```

--------------------------------

### EMMREvaluator Initialization and Usage Example

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/terminator/improvement/emmr.html

This example demonstrates how to initialize EMMREvaluator and integrate it with Optuna's study and Terminator for early stopping. It shows the setup of samplers, study creation, trial suggestion, objective function evaluation, and the termination check.

```python
import optuna
from optuna.terminator import EMMREvaluator
from optuna.terminator import MedianErrorEvaluator
from optuna.terminator import Terminator

sampler = optuna.samplers.TPESampler(seed=0)
study = optuna.create_study(sampler=sampler, direction="minimize")
emmr_improvement_evaluator = EMMREvaluator()
median_error_evaluator = MedianErrorEvaluator(emmr_improvement_evaluator)
terminator = Terminator(
    improvement_evaluator=emmr_improvement_evaluator,
    error_evaluator=median_error_evaluator,
)


for i in range(1000):
    trial = study.ask()

    ys = [trial.suggest_float(f"x{i}", -10.0, 10.0) for i in range(5)]
    value = sum(ys[i] ** 2 for i in range(5))

    study.tell(trial, value)

    if terminator.should_terminate(study):
        # Terminated by Optuna Terminator!
        break
```

--------------------------------

### Example Output

Source: https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/001_first.html

This is an example of the output you might see after running the optimization, showing the found 'x' value and the corresponding function value.

```text
Found x: 1.9795173001678286, (x - 2)^2: 0.0004195409924148358
```

--------------------------------

### Install Visualization and Machine Learning Libraries

Source: https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/002_multi_objective.html

Installs necessary libraries for visualizing optimization results and calculating hyperparameter importance.

```bash
$ pip install plotly
$ pip install scikit-learn
$ pip install nbformat  # Required if you are running this tutorial in Jupyter Notebook.
```

--------------------------------

### WilcoxonPruner Example

Source: https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.WilcoxonPruner.html

Demonstrates how to use WilcoxonPruner with a study. This example requires `scipy` to be installed.

```python
import optuna
import numpy as np

def objective(trial):
    x = np.random.uniform(0, 10)
    y = -x ** 2 + np.random.normal(0, 1, 100)
    for step in range(100):
        trial.report(y[step], step)
    if trial.should_prune():
        raise optuna.exceptions.TrialPruned()
    return y.mean()

study = optuna.create_study(pruner=optuna.pruners.WilcoxonPruner(p_threshold=0.05, n_startup_steps=5))
study.optimize(objective, n_trials=20)

```

--------------------------------

### Install Plotly for Visualization

Source: https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/005_visualization.html

Install Plotly, the default visualization backend for Optuna. This is required for most visualization functions.

```bash
pip install plotly

# Required if you are running this tutorial in Jupyter Notebook.
pip install nbformat

```

--------------------------------

### Install Optuna development version

Source: https://optuna.readthedocs.io/en/stable/installation.html

Installs the latest development version directly from the master branch of the GitHub repository.

```bash
$ pip install git+https://github.com/optuna/optuna.git
```

--------------------------------

### Install Matplotlib for Visualization

Source: https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/005_visualization.html

Install Matplotlib as an alternative visualization backend for Optuna. Use this if you prefer Matplotlib over Plotly.

```bash
pip install matplotlib

```

--------------------------------

### Install OptunaHub

Source: https://optuna.readthedocs.io/en/stable/index.html

Install OptunaHub to access and publish Optuna features. This package manager command facilitates integration with the feature-sharing platform.

```bash
$ pip install optunahub

```

--------------------------------

### Install Visualization Libraries

Source: https://optuna.readthedocs.io/en/stable/_downloads/1f693b2e373774312e66b0c8f033b8a9/005_visualization.ipynb

Install Plotly for interactive visualizations or Matplotlib for static plots. Plotly is recommended for interactive features.

```console
pip install plotly

# Required if you are running this tutorial in Jupyter Notebook.
pip install nbformat
```

```console
pip install matplotlib
```

--------------------------------

### Install and Run Optuna Dashboard

Source: https://optuna.readthedocs.io/en/stable/_downloads/1f693b2e373774312e66b0c8f033b8a9/005_visualization.ipynb

Install the Optuna Dashboard for a web-based interface to view optimization history, hyperparameter importances, and relationships. This requires a persistent study, such as one stored in an RDB backend.

```console
pip install optuna-dashboard
optuna-dashboard sqlite:///example-study.db
```

--------------------------------

### Start GrpcStorageProxy Server

Source: https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html

This Python script starts a gRPC proxy server that uses an RDBStorage backend. This server acts as a central point for multiple clients to access the storage, distributing the load.

```python
from optuna.storages import run_grpc_proxy_server
from optuna.storages import get_storage

storage = get_storage("mysql+pymysql://username:password@127.0.0.1:3306/example")
run_grpc_proxy_server(storage, host="localhost", port=13000)

```

--------------------------------

### Initialize and use FileSystemArtifactStore

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_filesystem.html

Demonstrates setting up a local directory for artifact storage and uploading artifacts within an Optuna objective function.

```python
import os

import optuna
from optuna.artifacts import FileSystemArtifactStore
from optuna.artifacts import upload_artifact


base_path = "./artifacts"
os.makedirs(base_path, exist_ok=True)
artifact_store = FileSystemArtifactStore(base_path=base_path)


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)
    file_path = generate_example(...)
    upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )
    return ...
```

--------------------------------

### Initialize EMMREvaluator and Terminator

Source: https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.EMMREvaluator.html

This example demonstrates how to initialize EMMREvaluator, MedianErrorEvaluator, and Terminator for early stopping in Optuna studies. Ensure Optuna and its terminators are installed.

```python
import optuna
from optuna.terminator import EMMREvaluator
from optuna.terminator import MedianErrorEvaluator
from optuna.terminator import Terminator

sampler = optuna.samplers.TPESampler(seed=0)
study = optuna.create_study(sampler=sampler, direction="minimize")
emmr_improvement_evaluator = EMMREvaluator()
median_error_evaluator = MedianErrorEvaluator(emmr_improvement_evaluator)
terminator = Terminator(
    improvement_evaluator=emmr_improvement_evaluator,
    error_evaluator=median_error_evaluator,
)


for i in range(1000):
    trial = study.ask()

    ys = [trial.suggest_float(f"x{i}", -10.0, 10.0) for i in range(5)]
    value = sum(ys[i] ** 2 for i in range(5))

    study.tell(trial, value)

    if terminator.should_terminate(study):
        # Terminated by Optuna Terminator!
        break
```

--------------------------------

### Initialize and Use GCSArtifactStore

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_gcs.html

Demonstrates how to instantiate the GCSArtifactStore and use it within an Optuna objective function to upload artifacts.

```python
import optuna
from optuna.artifacts import GCSArtifactStore, upload_artifact


artifact_backend = GCSArtifactStore("my-bucket")


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)
    file_path = generate_example(...)
    upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )
    return ...
```

--------------------------------

### Optuna Artifact Upload and Download Example

Source: https://optuna.readthedocs.io/en/stable/_downloads/6e6a1710a8301512ef0074d3b66ee7fd/012_artifact_tutorial.ipynb

Demonstrates setting up a Boto3 artifact store, defining an objective function to upload artifacts, running an Optuna study, and downloading the artifact associated with the best trial. Assumes AWS credentials and S3 endpoint are set as environment variables.

```python
import os

import boto3
from botocore.config import Config
import optuna
from optuna.artifact import upload_artifact
from optuna.artifact import download_artifact
from optuna.artifact.boto3 import Boto3ArtifactStore


artifact_store = Boto3ArtifactStore(
    client=boto3.client(
        "s3",
        aws_access_key_id=os.environ[
            "AWS_ACCESS_KEY_ID"
        ],  # Assume that these environment variables are set up properly. The same applies below.
        aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
        endpoint_url=os.environ["S3_ENDPOINT"],
        config=Config(connect_timeout=30, read_timeout=30),
    ),
    bucket_name="example_bucket",
)


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)

    # Creating and writing an artifact.
    file_path = generate_example(...)  # This function returns some kind of file.
    artifact_id = upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )  # The return value is the artifact ID.
    trial.set_user_attr(
        "artifact_id", artifact_id
    )  # Save the ID in RDB so that it can be referenced later.

    return ...


study = optuna.create_study(
    study_name="test_study",
    storage="mysql://USER:PASS@localhost:3306/test",  # Set the appropriate URL.
)
study.optimize(objective, n_trials=100)

# Downloading artifacts associated with the best trial.
best_artifact_id = study.best_trial.user_attrs.get("artifact_id")
download_file_path = ...  # Set the path to save the downloaded artifact.
download_artifact(
    artifact_store=artifact_store, file_path=download_file_path, artifact_id=best_artifact_id
)
with open(download_file_path, "rb") as f:
    content = f.read().decode("utf-8")
print(content)

```

--------------------------------

### Create and manage a trial with ask and tell

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/study/study.html

Demonstrates the basic workflow of creating a trial using `study.ask()`, suggesting parameters, and completing the trial with `study.tell()`.

```python
import optuna


study = optuna.create_study()

trial = study.ask()

x = trial.suggest_float("x", -1, 1)

study.tell(trial, x**2)
```

--------------------------------

### Retrieving All Trials and Iterating

Source: https://optuna.readthedocs.io/en/stable/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb

Get a list of all trials conducted within an Optuna study. The example shows how to access the first two trials for inspection.

```python
study.trials
for trial in study.trials[:2]:  # Show first two trials
    print(trial)
```

--------------------------------

### Objective Function with Pruning Logic

Source: https://optuna.readthedocs.io/en/stable/_downloads/8a3b786e31e54819e53dfa737aebc72d/007_optuna_callback.ipynb

An example objective function that prunes all trials after the first 5 (trial.number starts at 0). This is used in conjunction with a callback to demonstrate stopping criteria.

```python
def objective(trial):
    if trial.number > 4:
        raise optuna.TrialPruned

    return trial.suggest_float("x", 0, 1)
```

--------------------------------

### Optuna Journal Storage Example

Source: https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/011_journal_storage.html

Demonstrates how to set up and use Journal Storage with a simple objective function. This is useful for distributed optimization over network file systems.

```python
import logging
import sys

import optuna


# Add stream handler of stdout to show the messages
optuna.logging.get_logger("optuna").addHandler(logging.StreamHandler(sys.stdout))
study_name = "example-study"  # Unique identifier of the study.
file_path = "./optuna_journal_storage.log"
storage = optuna.storages.JournalStorage(
    optuna.storages.journal.JournalFileBackend(file_path),  # NFS path for distributed optimization
)

study = optuna.create_study(study_name=study_name, storage=storage)


def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2


study.optimize(objective, n_trials=3)

```

--------------------------------

### Run Optuna optimization with enqueued trials and logging

Source: https://optuna.readthedocs.io/en/stable/_downloads/500d8afeaf8e808ed1a42064b3c9ea44/008_specify_params.ipynb

Starts the hyperparameter optimization process for a specified number of trials or a timeout. Includes setup for logging Optuna messages to standard output.

```python
import logging
import sys

# Add stream handler of stdout to show the messages to see Optuna works expectedly.
optuna.logging.get_logger("optuna").addHandler(logging.StreamHandler(sys.stdout))
study.optimize(objective, n_trials=100, timeout=600)
```

--------------------------------

### Set Up Optuna Study and Artifact Store

Source: https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html

This function initializes the Optuna study, sets user attributes for slab and molecule data, creates a temporary directory, and initializes the FileSystemArtifactStore.

```python
def main():
    study = create_study(
        study_name="test_study",
        storage="sqlite:///example.db",
        load_if_exists=True,
    )

    slab, E_slab = create_slab()
    study.set_user_attr("slab", atoms_to_json(slab))
    study.set_user_attr("E_slab", E_slab)

    mol, E_mol = create_mol()
    study.set_user_attr("mol", atoms_to_json(mol))
    study.set_user_attr("E_mol", E_mol)

    os.makedirs("./tmp", exist_ok=True)

    base_path = "./artifacts"
    os.makedirs(base_path, exist_ok=True)
    artifact_store = FileSystemArtifactStore(base_path=base_path)
    study.optimize(Objective(artifact_store), n_trials=3)
    print(
        f"Best trial is #{study.best_trial.number}
"
        f"    Its adsorption energy is {study.best_value}
"
        f"    Its adsorption position is
"
    )
```

--------------------------------

### Example Usage of PED-ANOVA Importance Evaluator

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/importance/_ped_anova/evaluator.html

Demonstrates how to create an Optuna study, optimize an objective function, and then use the PedAnovaImportanceEvaluator to get parameter importances. Ensure Optuna and the evaluator are imported.

```python
import optuna
from optuna.importance import PedAnovaImportanceEvaluator


def objective(trial):
    x1 = trial.suggest_float("x1", -10, 10)
    x2 = trial.suggest_float("x2", -10, 10)
    return x1 + x2 / 1000


study = optuna.create_study()
study.optimize(objective, n_trials=100)
evaluator = PedAnovaImportanceEvaluator()
importance = optuna.importance.get_param_importances(study, evaluator=evaluator)
```

--------------------------------

### Implement PatientPruner with MedianPruner

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/pruners/_patient.html

Example demonstrating how to use PatientPruner with MedianPruner for hyperparameter optimization. This setup prunes trials if the objective doesn't improve for a specified number of steps, using MedianPruner as the underlying pruning strategy.

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import SGDClassifier
from sklearn.model_selection import train_test_split

import optuna

X, y = load_iris(return_X_y=True)
X_train, X_valid, y_train, y_valid = train_test_split(X, y)
classes = np.unique(y)


def objective(trial):
    alpha = trial.suggest_float("alpha", 0.0, 1.0)
    clf = SGDClassifier(alpha=alpha)
    n_train_iter = 100

    for step in range(n_train_iter):
        clf.partial_fit(X_train, y_train, classes=classes)

        intermediate_value = clf.score(X_valid, y_valid)
        trial.report(intermediate_value, step)

        if trial.should_prune():
            raise optuna.TrialPruned()

    return clf.score(X_valid, y_valid)


study = optuna.create_study(
    direction="maximize",
    pruner=optuna.pruners.PatientPruner(optuna.pruners.MedianPruner(), patience=1),
)
study.optimize(objective, n_trials=20)
```

--------------------------------

### Optuna CLI: Create Study and Ask for Parameters

Source: https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/004_cli.html

Demonstrates using the Optuna CLI to create a new study and then ask for the first trial's parameters. This is part of the interactive Ask-and-Tell interface.

```bash
$ STUDY_NAME=`optuna create-study --storage sqlite:///example.db`
$ optuna ask --storage sqlite:///example.db --study-name $STUDY_NAME --sampler TPESampler \
     --search-space '{"x": {"name": "FloatDistribution", "attributes": {"step": null, "low": -10.0, "high": 10.0, "log": false}}}'


[I 2022-08-20 06:08:53,158] Asked trial 0 with parameters {'x': 2.512238141966016}.
{"number": 0, "params": {"x": 2.512238141966016}}

```

--------------------------------

### Create Study and Optimize Objective Function

Source: https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html

This snippet shows the basic workflow of creating an Optuna study with a MedianPruner and optimizing a defined objective function for a specified number of trials. It's useful for getting started with hyperparameter optimization.

```python
import logging
import sys
import optuna

def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return x ** 2

# Add stream handler of stdout to show the messages
optuna.logging.get_logger("optuna").addHandler(logging.StreamHandler(sys.stdout))

study = optuna.create_study(pruner=optuna.pruners.MedianPruner())
study.optimize(objective, n_trials=20)
```

--------------------------------

### Create and Load Study

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/study/study.html

Demonstrates how to create a new Optuna study with SQLite storage, optimize it with a simple objective function, and then load the study back from the same storage.

```python
import optuna


def objective(trial):
    x = trial.suggest_float("x", 0, 10)
    return x**2


study = optuna.create_study(storage="sqlite:///example.db", study_name="my_study")
study.optimize(objective, n_trials=3)

loaded_study = optuna.load_study(study_name="my_study", storage="sqlite:///example.db")
assert len(loaded_study.trials) == len(study.trials)
```

--------------------------------

### Enqueueing Trials with Specific Parameters in Optuna

Source: https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html

Demonstrates how to use `enqueue_trial` to pre-define parameters for future trials. This is useful for testing specific hyperparameter combinations or guiding the optimization process. The example shows enqueuing a trial with a specific value for 'x' and another with additional user attributes.

```python
import optuna

def objective(trial):
    x = trial.suggest_float("x", 0, 10)
    return x**2


study = optuna.create_study()
study.enqueue_trial({"x": 5})
study.enqueue_trial({"x": 0}, user_attrs={"memo": "optimal"})
study.optimize(objective, n_trials=2)

assert study.trials[0].params == {"x": 5}
assert study.trials[1].params == {"x": 0}
assert study.trials[1].user_attrs == {"memo": "optimal"}
```

--------------------------------

### Getting All Study Summaries

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/study/study.html

This example shows how to retrieve a list of summaries for all studies stored in a given storage. Each summary contains metadata about a study, such as its name and the number of trials. The `include_best_trial` flag can be set to `True` to include information about the best trial for each study, which may increase query time.

```python
import optuna


def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2


study = optuna.create_study(study_name="example-study", storage="sqlite:///example.db")
study.optimize(objective, n_trials=3)

study_summaries = optuna.study.get_all_study_summaries(storage="sqlite:///example.db")
assert len(study_summaries) == 1

study_summary = study_summaries[0]
assert study_summary.study_name == "example-study"
```

--------------------------------

### Access FixedTrial Start Time

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/trial/_fixed.html

Returns the datetime object when the trial started. This property is None if the trial has not started or if the start time is not recorded.

```python
@property
def datetime_start(self) -> datetime.datetime | None:
    return self._datetime_start
```

--------------------------------

### Suggest Various Parameter Types in Optuna

Source: https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html

Demonstrates how to suggest categorical, integer, and floating-point parameters using Optuna's trial object. Includes examples of log-scaled and discretized integer/float parameters.

```python
import optuna


def objective(trial):
    # Categorical parameter
    optimizer = trial.suggest_categorical("optimizer", ["MomentumSGD", "Adam"])

    # Integer parameter
    num_layers = trial.suggest_int("num_layers", 1, 3)

    # Integer parameter (log)
    num_channels = trial.suggest_int("num_channels", 32, 512, log=True)

    # Integer parameter (discretized)
    num_units = trial.suggest_int("num_units", 10, 100, step=5)

    # Floating point parameter
    dropout_rate = trial.suggest_float("dropout_rate", 0.0, 1.0)

    # Floating point parameter (log)
    learning_rate = trial.suggest_float("learning_rate", 1e-5, 1e-2, log=True)

    # Floating point parameter (discretized)
    drop_path_rate = trial.suggest_float("drop_path_rate", 0.0, 1.0, step=0.1)

```

--------------------------------

### Access Trial Start Datetime

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/trial/_trial.html

Retrieve the datetime when the trial started. Returns None if the trial has not started yet.

```python
return self._cached_frozen_trial.datetime_start
```

--------------------------------

### Initialize and Optimize with RandomSampler

Source: https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html

Demonstrates how to create a study with RandomSampler and optimize an objective function. Ensure Optuna is installed.

```python
import optuna
from optuna.samplers import RandomSampler


def objective(trial):
    x = trial.suggest_float("x", -5, 5)
    return x**2


study = optuna.create_study(sampler=RandomSampler())
study.optimize(objective, n_trials=10)

```

--------------------------------

### Install cmaes library

Source: https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html

Before using CmaEsSampler, ensure the 'cmaes' library is installed. This command installs the necessary package.

```bash
$ pip install cmaes

```

--------------------------------

### Optuna Ask and Tell with Define-and-Run API

Source: https://optuna.readthedocs.io/en/stable/_downloads/9e17bbf554dfa22e6c92f27c8c7df53d/009_ask_and_tell.ipynb

This example demonstrates the define-and-run API using Optuna's ask-and-tell interface. Pre-defined distributions are passed to study.ask(), and the returned trial object contains the sampled hyperparameters.

```python
study = optuna.create_study(direction="maximize")
n_trials = 10
for _ in range(n_trials):
    trial = study.ask(distributions)  # pass the pre-defined distributions.

    # two hyperparameters are already sampled from the pre-defined distributions
    C = trial.params["C"]
    solver = trial.params["solver"]

    clf = LogisticRegression(C=C, solver=solver)
    clf.fit(X_train, y_train)
    val_accuracy = clf.score(X_test, y_test)

    study.tell(trial, val_accuracy)
```

--------------------------------

### Install Pandas

Source: https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/001_rdb.html

Ensure you have Pandas installed to use the dataframe functionality.

```bash
$ pip install pandas

```

--------------------------------

### Upload and Download Artifacts with FileSystemArtifactStore

Source: https://optuna.readthedocs.io/en/stable/_downloads/6e6a1710a8301512ef0074d3b66ee7fd/012_artifact_tutorial.ipynb

This snippet demonstrates how to set up a local file system artifact store, upload an artifact generated within an objective function, save its ID as a trial attribute, and then download it after optimization. Ensure the `generate_example` function is defined and returns a file path.

```python
import os

import optuna
from optuna.artifacts import FileSystemArtifactStore
from optuna.artifacts import upload_artifact
from optuna.artifacts import download_artifact


base_path = "./artifacts"
os.makedirs(base_path, exist_ok=True)
artifact_store = FileSystemArtifactStore(base_path=base_path)


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)

    # Creating and writing an artifact.
    file_path = generate_example(...)  # This function returns some kind of file.
    artifact_id = upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )  # The return value is the artifact ID.
    trial.set_user_attr(
        "artifact_id", artifact_id
    )  # Save the ID in RDB so that it can be referenced later.

    return ...


study = optuna.create_study(study_name="test_study", storage="sqlite:///example.db")
study.optimize(objective, n_trials=100)

# Downloading artifacts associated with the best trial.
best_artifact_id = study.best_trial.user_attrs.get("artifact_id")
download_file_path = ...  # Set the path to save the downloaded artifact.
download_artifact(
    artifact_store=artifact_store, file_path=download_file_path, artifact_id=best_artifact_id
)
with open(download_file_path, "rb") as f:
    content = f.read().decode("utf-8")
print(content)
```

--------------------------------

### Initialize QMCSampler with Halton sequence and scrambling

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/samplers/_qmc.html

This example demonstrates initializing QMCSampler with the Halton sequence and enabling scrambling with a specific seed for reproducibility.

```python
sampler = optuna.samplers.QMCSampler(qmc_type="halton", scramble=True, seed=123)
```

--------------------------------

### Optuna Ask-and-Tell Interface: Using Predefined Distributions

Source: https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html

Illustrates how to pass previously defined distributions to the `ask()` method in Optuna's ask-and-tell interface. This allows for more control over the hyperparameter search space when creating trials.

```python
import optuna


study = optuna.create_study()

distributions = {

```

--------------------------------

### Optuna Ask and Tell with Hyperband Pruner

Source: https://optuna.readthedocs.io/en/stable/_downloads/9e17bbf554dfa22e6c92f27c8c7df53d/009_ask_and_tell.ipynb

This example demonstrates using the ask-and-tell interface with Optuna's HyperbandPruner. It shows how to report intermediate values and explicitly tell the study about pruned trials.

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import SGDClassifier
from sklearn.model_selection import train_test_split

import optuna


X, y = load_iris(return_X_y=True)
X_train, X_valid, y_train, y_valid = train_test_split(X, y)
classes = np.unique(y)
n_train_iter = 100

# define study with hyperband pruner.
study = optuna.create_study(
    direction="maximize",
    pruner=optuna.pruners.HyperbandPruner(
        min_resource=1, max_resource=n_train_iter, reduction_factor=3
    ),
)

for _ in range(20):
    trial = study.ask()

    alpha = trial.suggest_float("alpha", 0.0, 1.0)

    clf = SGDClassifier(alpha=alpha)
    pruned_trial = False

    for step in range(n_train_iter):
        clf.partial_fit(X_train, y_train, classes=classes)

        intermediate_value = clf.score(X_valid, y_valid)
        trial.report(intermediate_value, step)

        if trial.should_prune():
            pruned_trial = True
            break

    if pruned_trial:
        study.tell(trial, state=optuna.trial.TrialState.PRUNED)  # tell the pruned state
    else:
        score = clf.score(X_valid, y_valid)
        study.tell(trial, score)  # tell objective value
```

--------------------------------

### Install Optuna via conda

Source: https://optuna.readthedocs.io/en/stable/installation.html

Installation method using the conda package manager from the conda-forge channel.

```bash
$ conda install -c conda-forge optuna
```

--------------------------------

### Run gRPC Server with RDBStorage

Source: https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.run_grpc_proxy_server.html

This snippet demonstrates how to initialize an RDBStorage and then run a gRPC server to proxy requests to it. Ensure you have the necessary database driver (e.g., pymysql) installed.

```python
from optuna.storages import run_grpc_proxy_server
from optuna.storages import get_storage

storage = get_storage("mysql+pymysql://<user>:<pass>@<host>/<dbname>[?<options>]")
run_grpc_proxy_server(storage, host="localhost", port=13000)
```

--------------------------------

### Constrained Multi-Objective Acquisition Function Setup

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/samplers/_gp/sampler.html

Prepares arguments for acquisition functions in constrained multi-objective optimization. This involves calculating constraint values and determining feasibility for trials.

```python
else:
    constraint_vals, is_feasible = _get_constraint_vals_and_feasibility(
        study, completed_trials
    )
    constr_gpr_list, constr_threshold_list = self._get_constraints_acqf_args(
        constraint_vals, internal_search_space, normalized_params
    )
```

--------------------------------

### Initialize GrpcStorageProxy

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/storages/_grpc/client.html

Initializes the GrpcStorageProxy with host and port, setting up the gRPC channel and stub for communication with the storage server.

```python
storage = GrpcStorageProxy(host="localhost", port=13000)
study = optuna.create_study(storage=storage)
```

--------------------------------

### PercentilePruner Example Usage

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/pruners/_percentile.html

Demonstrates how to use PercentilePruner with SGDClassifier and Optuna. This example shows how to set up a study with the pruner and an objective function that reports intermediate values.

```Python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import SGDClassifier
from sklearn.model_selection import train_test_split

import optuna

X, y = load_iris(return_X_y=True)
X_train, X_valid, y_train, y_valid = train_test_split(X, y)
classes = np.unique(y)


def objective(trial):
    alpha = trial.suggest_float("alpha", 0.0, 1.0)
    clf = SGDClassifier(alpha=alpha)
    n_train_iter = 100

    for step in range(n_train_iter):
        clf.partial_fit(X_train, y_train, classes=classes)

        intermediate_value = clf.score(X_valid, y_valid)
        trial.report(intermediate_value, step)

        if trial.should_prune():
            raise optuna.TrialPruned()

    return clf.score(X_valid, y_valid)


study = optuna.create_study(
    direction="maximize",
    pruner=optuna.pruners.PercentilePruner(
        25.0, n_startup_trials=5, n_warmup_steps=30, interval_steps=10
    ),
)
study.optimize(objective, n_trials=20)
```

--------------------------------

### Import Optuna

Source: https://optuna.readthedocs.io/en/stable/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb

Import the Optuna library to begin using its functionalities.

```python
import optuna
```

--------------------------------

### Running Optuna Study with FileSystemArtifactStore

Source: https://optuna.readthedocs.io/en/stable/_downloads/6e6a1710a8301512ef0074d3b66ee7fd/012_artifact_tutorial.ipynb

Sets up and runs an Optuna study for optimizing adsorption structures. It configures an SQLite storage and a file system backend for artifacts, performs 100 trials, and saves the best structure as 'best_atoms.png'.

```python
def main():
    # Create a temporary directory for the artifact store
    temp_dir = tempfile.mkdtemp()
    artifact_store = FileSystemArtifactStore(temp_dir)

    # Create an Optuna study with SQLite storage and the artifact store
    study = create_study(storage=f"sqlite:///{os.path.join(temp_dir, "db.sqlite")}", 
                         artifact_store=artifact_store, 
                         load_if_exists=True)

    # Define the objective function
    objective = Objective(artifact_store)

    # Perform optimization
    study.optimize(objective, n_trials=100)

    # Display information for the best trial
    print("Best trial:")
    print(f"  Value: {study.best_value}")
    print(f"  Params: {study.best_params}")

    # Download the best artifact and save it as an image
    best_trial = study.best_trial
    download_artifact(best_trial, "atoms.json", "best_atoms.json")
    atoms = file_to_atoms("best_atoms.json")
    write("best_atoms.png", atoms)

if __name__ == "__main__":
    main()
```

--------------------------------

### Example Usage of SuccessiveHalvingPruner

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/pruners/_successive_halving.html

This example demonstrates how to use the SuccessiveHalvingPruner with SGDClassifier to minimize an objective function. It shows how to report intermediate values and prune trials when they should_prune().

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import SGDClassifier
from sklearn.model_selection import train_test_split

import optuna

X, y = load_iris(return_X_y=True)
X_train, X_valid, y_train, y_valid = train_test_split(X, y)
classes = np.unique(y)


def objective(trial):
    alpha = trial.suggest_float("alpha", 0.0, 1.0)
    clf = SGDClassifier(alpha=alpha)
    n_train_iter = 100

    for step in range(n_train_iter):
        clf.partial_fit(X_train, y_train, classes=classes)

        intermediate_value = clf.score(X_valid, y_valid)
        trial.report(intermediate_value, step)

        if trial.should_prune():
            raise optuna.TrialPruned()

    return clf.score(X_valid, y_valid)


study = optuna.create_study(
    direction="maximize", pruner=optuna.pruners.SuccessiveHalvingPruner()
)
study.optimize(objective, n_trials=20)
```

--------------------------------

### Running Optimization with a Custom Callback

Source: https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/007_optuna_callback.html

Demonstrates how to create a study, instantiate the custom callback, and run the optimization process with a specified number of trials and the custom callback.

```python
import logging
import sys

# Add stream handler of stdout to show the messages
optuna.logging.get_logger("optuna").addHandler(logging.StreamHandler(sys.stdout))

study_stop_cb = StopWhenTrialKeepBeingPrunedCallback(2)
study = optuna.create_study()
study.optimize(objective, n_trials=10, callbacks=[study_stop_cb])

```

--------------------------------

### Example Usage of BruteForceSampler

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/samplers/_brute_force.html

This example demonstrates how to use the BruteForceSampler to perform an exhaustive search over a defined objective function. It shows how to suggest categorical, integer, and float parameters.

```python
import optuna


def objective(trial):
    c = trial.suggest_categorical("c", ["float", "int"])
    if c == "float":
        return trial.suggest_float("x", 1, 3, step=0.5)
    elif c == "int":
        a = trial.suggest_int("a", 1, 3)
        b = trial.suggest_int("b", a, 3)
        return a + b


study = optuna.create_study(sampler=optuna.samplers.BruteForceSampler())
study.optimize(objective)
```

--------------------------------

### Optuna MedianPruner Example

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/pruners/_median.html

Example demonstrating how to use MedianPruner to minimize an objective function. It shows setting up SGDClassifier, reporting intermediate values, and handling pruning with `trial.should_prune()`.

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import SGDClassifier
from sklearn.model_selection import train_test_split

import optuna

X, y = load_iris(return_X_y=True)
X_train, X_valid, y_train, y_valid = train_test_split(X, y)
classes = np.unique(y)


def objective(trial):
    alpha = trial.suggest_float("alpha", 0.0, 1.0)
    clf = SGDClassifier(alpha=alpha)
    n_train_iter = 100

    for step in range(n_train_iter):
        clf.partial_fit(X_train, y_train, classes=classes)

        intermediate_value = clf.score(X_valid, y_valid)
        trial.report(intermediate_value, step)

        if trial.should_prune():
            raise optuna.TrialPruned()

    return clf.score(X_valid, y_valid)


study = optuna.create_study(
    direction="maximize",
    pruner=optuna.pruners.MedianPruner(
        n_startup_trials=5, n_warmup_steps=30, interval_steps=10
    ),
)
study.optimize(objective, n_trials=20)
```

--------------------------------

### Ask for Trial Parameters via CLI

Source: https://optuna.readthedocs.io/en/stable/_downloads/d1bddf5b39c9f90932947a22a3164d8d/004_cli.ipynb

Interactively ask Optuna for trial parameters using the 'ask' command. This is useful when the objective function cannot be directly defined in Python. Requires specifying storage, study name, sampler, and search space.

```bash
$ optuna ask --storage sqlite:///example.db --study-name $STUDY_NAME --sampler TPESampler     --search-space '{"x": {"name": "FloatDistribution", "attributes": {"step": null, "low": -10.0, "high": 10.0, "log": false}}}'


[I 2022-08-20 06:08:53,158] Asked trial 0 with parameters {'x': 2.512238141966016}.
{"number": 0, "params": {"x": 2.512238141966016}}
```

--------------------------------

### Setting up and using Journal Storage

Source: https://optuna.readthedocs.io/en/stable/_downloads/f72a41939bafec49754af4ac5e1dcdfc/011_journal_storage.ipynb

Use this snippet to initialize a JournalStorage with a file backend for distributed optimization. It demonstrates creating a study and running a simple optimization objective.

```python
import logging
import sys

import optuna


# Add stream handler of stdout to show the messages
optuna.logging.get_logger("optuna").addHandler(logging.StreamHandler(sys.stdout))
study_name = "example-study"  # Unique identifier of the study.
file_path = "./optuna_journal_storage.log"
storage = optuna.storages.JournalStorage(
    optuna.storages.journal.JournalFileBackend(file_path),  # NFS path for distributed optimization
)

study = optuna.create_study(study_name=study_name, storage=storage)


def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2


study.optimize(objective, n_trials=3)
```

--------------------------------

### TerminatorCallback Example

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/terminator/callback.html

Demonstrates how to use TerminatorCallback to stop an Optuna optimization. This example trains a RandomForestClassifier using cross-validation and terminates the study if the Terminator determines it's no longer beneficial.

```python
from sklearn.datasets import load_wine
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold

import optuna
from optuna.terminator import TerminatorCallback
from optuna.terminator import report_cross_validation_scores


def objective(trial):
    X, y = load_wine(return_X_y=True)

    clf = RandomForestClassifier(
        max_depth=trial.suggest_int("max_depth", 2, 32),
        min_samples_split=trial.suggest_float("min_samples_split", 0, 1),
        criterion=trial.suggest_categorical("criterion", ("gini", "entropy")),
    )

    scores = cross_val_score(clf, X, y, cv=KFold(n_splits=5, shuffle=True))
    report_cross_validation_scores(trial, scores)
    return scores.mean()


study = optuna.create_study(direction="maximize")
terminator = TerminatorCallback()
study.optimize(objective, n_trials=50, callbacks=[terminator])
```

--------------------------------

### Local Artifact Storage with SQLite and FileSystemArtifactStore

Source: https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html

This snippet shows how to set up a local artifact store using FileSystemArtifactStore and integrate it with an Optuna study using SQLite for trial persistence. It includes uploading artifacts within the objective function and downloading them after optimization.

```python
import os

import optuna
from optuna.artifacts import FileSystemArtifactStore
from optuna.artifacts import upload_artifact
from optuna.artifacts import download_artifact


base_path = "./artifacts"
os.makedirs(base_path, exist_ok=True)
artifact_store = FileSystemArtifactStore(base_path=base_path)


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)

    # Creating and writing an artifact.
    file_path = generate_example(...)  # This function returns some kind of file.
    artifact_id = upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )  # The return value is the artifact ID.
    trial.set_user_attr(
        "artifact_id", artifact_id
    )  # Save the ID in RDB so that it can be referenced later.

    return ...


study = optuna.create_study(study_name="test_study", storage="sqlite:///example.db")
study.optimize(objective, n_trials=100)

# Downloading artifacts associated with the best trial.
best_artifact_id = study.best_trial.user_attrs.get("artifact_id")
download_file_path = ...  # Set the path to save the downloaded artifact.
download_artifact(
    artifact_store=artifact_store, file_path=download_file_path, artifact_id=best_artifact_id
)
with open(download_file_path, "rb") as f:
    content = f.read().decode("utf-8")
print(content)

```

--------------------------------

### CmaEsSampler

Source: https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html

Initializes CMA-ES sampler.

CMA-ES is a population-based optimization algorithm.

Parameters
----------
n_startup_trials : int, default=5
    The number of initial random trials. CMA-ES is known to require a certain number of initial trials to adapt the covariance matrix. If this parameter is set to a value less than or equal to zero, the sampler will not perform any initial random trials and will start adapting the covariance matrix from the first trial. If this parameter is set to None, it will be set to 5. Note that CMA-ES requires at least one trial to start adaptation. If n_startup_trials is set to 0, the sampler will start adaptation from the first trial.

seed : int, optional
    Seed for random number generator. If None, the default random number generator is used.

consider_magic_number : bool, default=True
    Whether to consider the magic number for CMA-ES. The magic number is a heuristic that determines the step size of CMA-ES. If this parameter is set to True, the magic number will be used. If this parameter is set to False, the magic number will not be used. This parameter is useful for controlling the exploration-exploitation trade-off in CMA-ES. For example, if you want to explore more, you can set this parameter to False.

independent_trials : bool, default=False
    Whether to consider independent trials. If this parameter is set to True, each trial will be considered independent. If this parameter is set to False, trials will be considered dependent. This parameter is useful for controlling the exploration-exploitation trade-off in CMA-ES. For example, if you want to explore more, you can set this parameter to True.

kwargs : dict
    Additional keyword arguments to pass to the CMA-ES algorithm.

```APIDOC
## CmaEsSampler

Initializes CMA-ES sampler.

CMA-ES is a population-based optimization algorithm.

### Parameters

*   **n_startup_trials** (int, default=5) - The number of initial random trials. CMA-ES is known to require a certain number of initial trials to adapt the covariance matrix. If this parameter is set to a value less than or equal to zero, the sampler will not perform any initial random trials and will start adapting the covariance matrix from the first trial. If this parameter is set to None, it will be set to 5. Note that CMA-ES requires at least one trial to start adaptation. If n_startup_trials is set to 0, the sampler will start adaptation from the first trial.
*   **seed** (int, optional) - Seed for random number generator. If None, the default random number generator is used.
*   **consider_magic_number** (bool, default=True) - Whether to consider the magic number for CMA-ES. The magic number is a heuristic that determines the step size of CMA-ES. If this parameter is set to True, the magic number will be used. If this parameter is set to False, the magic number will not be used. This parameter is useful for controlling the exploration-exploitation trade-off in CMA-ES. For example, if you want to explore more, you can set this parameter to False.
*   **independent_trials** (bool, default=False) - Whether to consider independent trials. If this parameter is set to True, each trial will be considered independent. If this parameter is set to False, trials will be considered dependent. This parameter is useful for controlling the exploration-exploitation trade-off in CMA-ES. For example, if you want to explore more, you can set this parameter to True.
*   **kwargs** (dict) - Additional keyword arguments to pass to the CMA-ES algorithm.
```

--------------------------------

### Retrieve and download all artifacts for a trial

Source: https://optuna.readthedocs.io/en/stable/reference/artifacts.html

Demonstrates how to fetch artifact metadata for a trial and download each associated file using an artifact store.

```python
import os

import optuna


# Get the storage that contains the study of interest.
storage = optuna.storages.get_storage(storage=...)

# Instantiate the artifact store used for the study.
# Optuna does not provide the API that stores the used artifact store information, so
# please manage the information in the user side.
artifact_store = ...

# Load study that contains the artifacts of interest.
study = optuna.load_study(study_name=..., storage=storage)

# Fetch the best trial.
best_trial = study.best_trial

# Fetch all the artifact meta connected to the best trial.
artifact_metas = optuna.artifacts.get_all_artifact_meta(best_trial, storage=storage)

download_dir_path = "./best_trial_artifacts/"
os.makedirs(download_dir_path, exist_ok=True)

for artifact_meta in artifact_metas:
    download_file_path = os.path.join(download_dir_path, artifact_meta.filename)
    # Download the artifacts to ``download_file_path``.
    optuna.artifacts.download_artifact(
        artifact_store=artifact_store,
        artifact_id=artifact_meta.artifact_id,
        file_path=download_file_path,
    )
```

--------------------------------

### Initialize Boto3ArtifactStore

Source: https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_boto3.html

Instantiate Boto3ArtifactStore with a bucket name. Optionally provide a Boto3 client and a flag to avoid buffer copying during uploads.

```python
artifact_store = Boto3ArtifactStore("my-bucket")
```