KerasTuner

https://keras.io/keras_tuner/
Admin
KerasTuner is an easy-to-use, scalable hyperparameter optimization framework with built-in Bayesian...
Tokens:48,753
Snippets:400
Trust Score:10
Update:5 months ago
Show doc for...
Tokens:
Raw
### Example of best hyperparameters

Source: https://keras.io/keras_tuner/guides/custom_tuner

This output shows an example of the dictionary containing the best hyperparameters found by the tuner, including values for 'units', 'batch_size', and 'learning_rate'.

```text
{'units': 128, 'batch_size': 32, 'learning_rate': 0.0034272591820215972}
```

--------------------------------

### Install KerasTuner

Source: https://keras.io/keras_tuner/guides/custom_tuner

Install the KerasTuner library quietly.

```python
!pip install keras-tuner -q
```

--------------------------------

### Run Trial: Black-box Optimization

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Example of `run_trial` used as a black-box optimizer for a simple function.

```python
def run_trial(self, trial, *args, **kwargs):
    hp = trial.hyperparameters
    x = hp.Float("x", -2.0, 2.0)
    y = x * x + 2 * x + 1
    return y
```

--------------------------------

### Import Libraries and Prepare Data

Source: https://keras.io/keras_tuner/guides/custom_tuner

Imports necessary libraries and generates random data for training and validation. This setup is for demonstration purposes.

```python
import keras_tuner
import tensorflow as tf
import keras
import numpy as np


x_train = np.random.rand(1000, 28, 28, 1)
y_train = np.random.randint(0, 10, (1000, 1))
x_val = np.random.rand(1000, 28, 28, 1)
y_val = np.random.randint(0, 10, (1000, 1))
```

--------------------------------

### Install KerasTuner

Source: https://keras.io/keras_tuner/guides/distributed_tuning

Install KerasTuner quietly using pip.

```bash
!pip install keras-tuner -q

```

--------------------------------

### Run Trial: Build and Fit Model

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Example of `run_trial` used to build and fit a Keras model using `self.hypermodel`.

```python
def run_trial(self, trial, *args, **kwargs):
    hp = trial.hyperparameters
    model = self.hypermodel.build(hp)
    return self.hypermodel.fit(hp, model, *args, **kwargs)
```

--------------------------------

### Example of model summary

Source: https://keras.io/keras_tuner/guides/custom_tuner

This displays the architecture summary of the best model found by the tuner, detailing the layers, output shapes, and parameter counts.

```text
Model: "functional_1"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape              ┃    Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ input_layer (InputLayer)        │ (None, 28, 28, 1)         │          0 │
├─────────────────────────────────┼───────────────────────────┼────────────┤
│ flatten (Flatten)               │ (None, 784)               │          0 │
├─────────────────────────────────┼───────────────────────────┼────────────┤
│ dense (Dense)                   │ (None, 128)               │    100,480 │
├─────────────────────────────────┼───────────────────────────┼────────────┤
│ dense_1 (Dense)                 │ (None, 10)                │      1,290 │
└─────────────────────────────────┴───────────────────────────┴────────────┘


 Total params: 101,770 (397.54 KB) 


 Trainable params: 101,770 (397.54 KB) 


 Non-trainable params: 0 (0.00 B)
```

--------------------------------

### KerasTuner Trial Summary Example

Source: https://keras.io/keras_tuner/guides/failed_trials

This is an example of a successful trial summary, showing the hyperparameters used and the resulting score. It helps in understanding the configuration of successful models.

```text
Trial 0002 summary
Hyperparameters:
units_1: 10
units_2: 30
Score: 0.08265472948551178
```

--------------------------------

### Instantiate and Use Custom HyperModel

Source: https://keras.io/keras_tuner/getting_started

Instantiate your custom HyperModel and use its build and fit methods to create and train a model. This example demonstrates building a model and calling fit with dummy data.

```python
hp = keras_tuner.HyperParameters()
hypermodel = MyHyperModel()
model = hypermodel.build(hp)
hypermodel.fit(hp, model, np.random.rand(100, 28, 28), np.random.rand(100, 10))

```

--------------------------------

### Example Hyperparameter Choices for GridSearch

Source: https://keras.io/keras_tuner/api/tuners/grid

Demonstrates how to define hyperparameters using hp.Choice for GridSearch. This tuner will explore all combinations of the specified values.

```python
optimizer = hp.Choice("model_name", values=["sgd", "adam"])
learning_rate = hp.Choice("learning_rate", values=[0.01, 0.1])

```

--------------------------------

### Initialize RandomSearch Tuner

Source: https://keras.io/keras_tuner/getting_started

Initialize the RandomSearch tuner with the hypermodel, objective, and search parameters. Set overwrite=True to start a new search.

```python
tuner = keras_tuner.RandomSearch(
    hypermodel=build_model,
    objective="val_accuracy",
    max_trials=3,
    executions_per_trial=2,
    overwrite=True,
    directory="my_dir",
    project_name="helloworld",
)
```

--------------------------------

### Install KerasTuner

Source: https://keras.io/keras_tuner

Install the latest release of KerasTuner using pip. This command upgrades the package if it's already installed.

```bash
pip install keras-tuner --upgrade

```

--------------------------------

### Start hyperparameter search

Source: https://keras.io/keras_tuner/guides/custom_tuner

Initiate the hyperparameter search process by calling `tuner.search()`. Pass the training and validation data, along with any other arguments required by your `MyHyperModel.fit()` method.

```python
tuner.search(x=x_train, y=y_train, validation_data=(x_val, y_val))
```

--------------------------------

### Trial Completion Output

Source: https://keras.io/keras_tuner/guides/failed_trials

Example output showing a completed trial with its validation loss.

```text
Trial 7 Complete [00h 00m 01s]
val_loss: 0.14219732582569122


```

--------------------------------

### Tuner.run_trial with HyperbandOracle

Source: https://keras.io/keras_tuner/api/oracles/hyperband

Example of how a custom Tuner class might implement the run_trial method to work with the HyperbandOracle. It handles loading previous trials and managing epochs based on special hyperparameters set by the oracle.

```python
def run_trial(self, trial, *args, **kwargs):
    hp = trial.hyperparameters
    if "tuner/trial_id" in hp:
        past_trial = self.oracle.get_trial(hp['tuner/trial_id'])
        model = self.load_model(past_trial)
    else:
        model = self.hypermodel.build(hp)

    initial_epoch = hp['tuner/initial_epoch']
    last_epoch = hp['tuner/epochs']

    for epoch in range(initial_epoch, last_epoch):
        self.on_epoch_begin(...)
        for step in range(...):
            # Run model training step here.
        self.on_epoch_end(...)


```

--------------------------------

### Instantiate HyperImageAugment with RandAugment

Source: https://keras.io/keras_tuner/api/hypermodels/hyper_image_augment

Example of instantiating HyperImageAugment to use RandAugment, defining search ranges for translation and enabling 'augment_layers' to search between 0 and 3.

```python
hm_aug = HyperImageAugment(input_shape=(32, 32, 3),
                           translate_x=0.5,
                           translate_y=[0.2, 0.4]
                           contrast=None)

```

--------------------------------

### SklearnTuner Example Usage

Source: https://keras.io/keras_tuner/api/tuners/sklearn

Demonstrates how to use SklearnTuner for hyperparameter tuning with Scikit-learn models. It requires defining a build_model function, configuring the tuner with an Oracle and cross-validation, and then performing the search.

```python
import keras_tuner
from sklearn import ensemble
from sklearn import datasets
from sklearn import linear_model
from sklearn import metrics
from sklearn import model_selection

def build_model(hp):
  model_type = hp.Choice('model_type', ['random_forest', 'ridge'])
  if model_type == 'random_forest':
    model = ensemble.RandomForestClassifier(
        n_estimators=hp.Int('n_estimators', 10, 50, step=10),
        max_depth=hp.Int('max_depth', 3, 10))
  else:
    model = linear_model.RidgeClassifier(
        alpha=hp.Float('alpha', 1e-3, 1, sampling='log'))
  return model

tuner = keras_tuner.tuners.SklearnTuner(
    oracle=keras_tuner.oracles.BayesianOptimizationOracle(
        objective=keras_tuner.Objective('score', 'max'),
        max_trials=10),
    hypermodel=build_model,
    scoring=metrics.make_scorer(metrics.accuracy_score),
    cv=model_selection.StratifiedKFold(5),
    directory='.',
    project_name='my_project')

X, y = datasets.load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = model_selection.train_test_split(
    X, y, test_size=0.2)

tuner.search(X_train, y_train)

best_model = tuner.get_best_models(num_models=1)[0]
```

--------------------------------

### Tune Hyperparameters Used in Build and Fit

Source: https://keras.io/keras_tuner/getting_started

This example demonstrates how to define a hyperparameter in `build` and access it in `fit` using `hp.get()`. It's useful when a hyperparameter affects both model architecture and preprocessing steps, such as image size.

```python
class MyHyperModel(keras_tuner.HyperModel):
    def build(self, hp):
        image_size = hp.Int("image_size", 10, 28)
        inputs = keras.Input(shape=(image_size, image_size))
        outputs = layers.Flatten()(inputs)
        outputs = layers.Dense(
            units=hp.Int("units", min_value=32, max_value=512, step=32),
            activation="relu",
        )(outputs)
        outputs = layers.Dense(10, activation="softmax")(outputs)
        model = keras.Model(inputs, outputs)
        model.compile(
            optimizer="adam",
            loss="categorical_crossentropy",
            metrics=["accuracy"],
        )
        return model

    def fit(self, hp, model, x, y, validation_data=None, **kwargs):
        if hp.Boolean("normalize"):
            x = layers.Normalization()(x)
        image_size = hp.get("image_size")
        cropped_x = x[:, :image_size, :image_size, :]
        if validation_data:
            x_val, y_val = validation_data
            cropped_x_val = x_val[:, :image_size, :image_size, :]
            validation_data = (cropped_x_val, y_val)
        return model.fit(
            cropped_x,
            y,
            # Tune whether to shuffle the data in each epoch.
            shuffle=hp.Boolean("shuffle"),
            validation_data=validation_data,
            **kwargs,
        )


tuner = keras_tuner.RandomSearch(
    MyHyperModel(),
    objective="val_accuracy",
    max_trials=3,
    overwrite=True,
    directory="my_dir",
    project_name="tune_hypermodel",
)

tuner.search(x_train, y_train, epochs=2, validation_data=(x_val, y_val))
```

--------------------------------

### KerasTuner Another Failed Trial Error Example

Source: https://keras.io/keras_tuner/guides/failed_trials

This example demonstrates another `FailedTrialError` with a different number of parameters, reinforcing the common issue of models exceeding size limits with certain hyperparameter combinations.

```text
Trial 0008 summary
Hyperparameters:
units_1: 30
units_2: 30
Traceback (most recent call last):
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/base_tuner.py", line 273, in _try_run_and_update_trial
    self._run_and_update_trial(trial, *fit_args, **fit_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/base_tuner.py", line 238, in _run_and_update_trial
    results = self.run_trial(trial, *fit_args, **fit_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 314, in run_trial
    obj_value = self._build_and_fit_model(trial, *args, **copied_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 232, in _build_and_fit_model
    model = self._try_build(hp)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 164, in _try_build
    model = self._build_hypermodel(hp)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 155, in _build_hypermodel
    model = self.hypermodel.build(hp)
  File "/tmp/ipykernel_21713/2463037569.py", line 20, in build_model
    raise keras_tuner.errors.FailedTrialError(
keras_tuner.src.errors.FailedTrialError: Model too large! It contains 1591 params.
```

--------------------------------

### Start Hyperparameter Search

Source: https://keras.io/keras_tuner/getting_started

Initiate the hyperparameter search process. Pass training data, epochs, and validation data to the search method. Arguments are passed to model.fit() for each execution.

```python
tuner.search(x_train, y_train, epochs=2, validation_data=(x_val, y_val))
```

--------------------------------

### Best Validation Loss and Total Time

Source: https://keras.io/keras_tuner/guides/failed_trials

Example output displaying the best validation loss found so far and the total elapsed time for the search.

```text
Best val_loss So Far: 0.09755773097276688
Total elapsed time: 00h 00m 04s


```

--------------------------------

### Start Hyperparameter Search

Source: https://keras.io/keras_tuner

Begin the hyperparameter search using the tuner's search method. Provide training data, number of epochs, and validation data. Retrieve the best performing model after the search is complete.

```python
tuner.search(x_train, y_train, epochs=5, validation_data=(x_val, y_val))
best_model = tuner.get_best_models()[0]

```

--------------------------------

### Start Hyperparameter Search with TensorBoard Callback

Source: https://keras.io/keras_tuner/guides/visualize_tuning

Initiate the hyperparameter search using the tuner's search method, specifying training data, validation split, epochs, and TensorBoard callback for logging.

```python
tuner.search(
    x_train,
    y_train,
    validation_split=0.2,
    epochs=2,
    # Use the TensorBoard callback.
    # The logs will be write to "/tmp/tb_logs".
    callbacks=[keras.callbacks.TensorBoard("/tmp/tb_logs")],
)
```

--------------------------------

### Get Best Models

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Retrieve the best trained model instances from the tuning process. The models are loaded with weights from their best epoch.

```python
Tuner.get_best_models(num_models=1)
```

--------------------------------

### Trial Completion Output

Source: https://keras.io/keras_tuner/getting_started

Example output showing a single trial's completion, including the elapsed time and the value of the validation mean absolute error.

```text
Trial 3 Complete [00h 00m 01s]
val_mean_absolute_error: 0.39589792490005493
```

--------------------------------

### Instantiate HyperImageAugment with Specific Augmentations

Source: https://keras.io/keras_tuner/api/hypermodels/hyper_image_augment

Example of instantiating HyperImageAugment with a fixed input shape and specific search ranges for rotation and translation. 'translate_y' and 'contrast' are disabled.

```python
hm_aug = HyperImageAugment(input_shape=(32, 32, 3),
                           augment_layers=0,
                           rotate=[0.2, 0.3],
                           translate_x=0.1,
                           translate_y=None,
                           contrast=None)

```

--------------------------------

### Define Custom HyperModel with Multiple Metrics

Source: https://keras.io/keras_tuner/getting_started

Implement a custom HyperModel that returns a dictionary of metrics. Use one of the keys as the objective for tuning. This example maximizes the negative Mean Absolute Error (MAE), which is equivalent to minimizing MAE.

```python
class HyperRegressor(keras_tuner.HyperModel):
    def build(self, hp):
        model = keras.Sequential(
            [
                layers.Dense(units=hp.Int("units", 32, 128, 32), activation="relu"),
                layers.Dense(units=1),
            ]
        )
        model.compile(
            optimizer="adam",
            loss="mean_squared_error",
        )
        return model

    def fit(self, hp, model, x, y, validation_data, **kwargs):
        model.fit(x, y, **kwargs)
        x_val, y_val = validation_data
        y_pred = model.predict(x_val)
        # Return a dictionary of metrics for KerasTuner to track.
        return {
            "metric_a": -np.mean(np.abs(y_pred - y_val)),
            "metric_b": np.mean(np.square(y_pred - y_val)),
        }

tuner = keras_tuner.RandomSearch(
    hypermodel=HyperRegressor(),
    # Objective is one of the keys.
    # Maximize the negative MAE, equivalent to minimize MAE.
    objective=keras_tuner.Objective("metric_a", "max"),
    max_trials=3,
    overwrite=True,
    directory="my_dir",
    project_name="custom_eval_dict",
)
tuner.search(
    x=np.random.rand(100, 10),
    y=np.random.rand(100, 1),
    validation_data=(np.random.rand(20, 10), np.random.rand(20, 1)),
)

tuner.results_summary()
```

--------------------------------

### Tuner.on_batch_begin

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Callback called at the beginning of a batch during the search process.

```APIDOC
## `on_batch_begin` method

### Description
Called at the beginning of a batch.

### Arguments
  * **trial** : A `Trial` instance.
  * **model** : A Keras `Model`.
  * **batch** : The current batch number within the current epoch.
  * **logs** : Additional metrics.
```

--------------------------------

### Callback: On Batch Begin

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Callback method called at the beginning of a batch during tuning.

```python
Tuner.on_batch_begin(trial, model, batch, logs)
```

--------------------------------

### Callback: On Epoch Begin

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Callback method called at the beginning of an epoch during tuning.

```python
Tuner.on_epoch_begin(trial, model, epoch, logs=None)
```

--------------------------------

### KerasTuner Failed Trial Error Example

Source: https://keras.io/keras_tuner/guides/failed_trials

This example shows a `FailedTrialError` traceback, indicating that a trial failed because the model built with the given hyperparameters was too large. This helps in debugging and adjusting hyperparameter search spaces.

```text
Trial 0007 summary
Hyperparameters:
units_1: 30
units_2: 20
Traceback (most recent call last):
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/base_tuner.py", line 273, in _try_run_and_update_trial
    self._run_and_update_trial(trial, *fit_args, **fit_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/base_tuner.py", line 238, in _run_and_update_trial
    results = self.run_trial(trial, *fit_args, **fit_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 314, in run_trial
    obj_value = self._build_and_fit_model(trial, *args, **copied_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 232, in _build_and_fit_model
    model = self._try_build(hp)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 164, in _try_build
    model = self._build_hypermodel(hp)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 155, in _build_hypermodel
    model = self.hypermodel.build(hp)
  File "/tmp/ipykernel_21713/2463037569.py", line 20, in build_model
    raise keras_tuner.errors.FailedTrialError(
keras_tuner.src.errors.FailedTrialError: Model too large! It contains 1271 params.
```

--------------------------------

### KerasTuner Yet Another Failed Trial Error Example

Source: https://keras.io/keras_tuner/guides/failed_trials

This example illustrates a `FailedTrialError` occurring during the model building phase, specifically when attempting to build a model with a large number of units. It highlights the importance of constraining hyperparameter ranges to prevent excessively large models.

```text
Trial 0009 summary
Hyperparameters:
units_1: 40
units_2: 10
Traceback (most recent call last):
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/base_tuner.py", line 273, in _try_run_and_update_trial
    self._run_and_update_trial(trial, *fit_args, **fit_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/base_tuner.py", line 238, in _run_and_update_trial
    results = self.run_trial(trial, *fit_args, **fit_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 314, in run_trial
    obj_value = self._build_and_fit_model(trial, *args, **copied_kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 232, in _build_and_fit_model
    model = self._try_build(hp)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 164, in _try_build
    model = self._build_hypermodel(hp)
  File "/home/codespace/.local/lib/python3.10/site-packages/keras_tuner/src/engine/tuner.py", line 155, in _build_hypermodel
    model = self.hypermodel.build(hp)
  File "/tmp/ipykernel_21713/2463037569.py", line 20, in build_model
    raise keras_tuner.errors.FailedTrialError(

```

--------------------------------

### Initialize Tuner Class

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Instantiate the base Tuner class with various configuration options. This is the foundation for creating custom tuners.

```python
keras_tuner.Tuner(
    oracle,
    hypermodel=None,
    max_model_size=None,
    optimizer=None,
    loss=None,
    metrics=None,
    distribution_strategy=None,
    directory=None,
    project_name=None,
    logger=None,
    tuner_id=None,
    overwrite=False,
    executions_per_trial=1,
    **kwargs
)
```

--------------------------------

### Import necessary libraries

Source: https://keras.io/keras_tuner/guides/failed_trials

Import Keras, Keras layers, KerasTuner, and NumPy for setting up the hyperparameter tuning environment.

```python
import keras
from keras import layers
import keras_tuner
import numpy as np

```

--------------------------------

### Get Best Trials

Source: https://keras.io/keras_tuner/api/oracles/base_oracle

Retrieves the best performing trials from the Oracle, specifying the number of trials to return.

```python
Oracle.get_best_trials(num_trials=1)

```

--------------------------------

### Get Tuner State

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Retrieve the current state of the Tuner object, which is useful for saving and resuming tuning processes.

```python
Tuner.get_state()
```

--------------------------------

### Tuner.on_epoch_begin

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Callback called at the beginning of an epoch during the search process.

```APIDOC
## `on_epoch_begin` method

### Description
Called at the beginning of an epoch.

### Arguments
  * **trial** : A `Trial` instance.
  * **model** : A Keras `Model`.
  * **epoch** : The current epoch number.
  * **logs** : Additional metrics.
```

--------------------------------

### Get Best Hyperparameters

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Retrieve the best hyperparameters found during the tuning process. This can be used to reinstantiate the best model.

```python
Tuner.get_best_hyperparameters(num_trials=1)
```

```python
best_hp = tuner.get_best_hyperparameters()[0]
model = tuner.hypermodel.build(best_hp)
```

--------------------------------

### Get Oracle State

Source: https://keras.io/keras_tuner/api/oracles/base_oracle

Retrieves the current state of the Oracle object, typically used for saving the tuning progress.

```python
Oracle.get_state()

```

--------------------------------

### Get Hyperparameter Value

Source: https://keras.io/keras_tuner/api/hyperparameters

Retrieves the current value of a specified hyperparameter. Ensure the hyperparameter has been defined and is accessible within the current scope.

```python
HyperParameters.get(name)
```

--------------------------------

### Initialize RandomSearch Tuner

Source: https://keras.io/keras_tuner/guides/visualize_tuning

Initialize the RandomSearch tuner with a model-building function, number of trials, overwrite setting, objective metric, and a directory for results.

```python
tuner = keras_tuner.RandomSearch(
    build_model,
    max_trials=10,
    # Do not resume the previous search in the same directory.
    overwrite=True,
    objective="val_accuracy",
    # Set a directory to store the intermediate results.
    directory="/tmp/tb",
)
```

--------------------------------

### Initialize Oracle Class

Source: https://keras.io/keras_tuner/api/oracles/base_oracle

Instantiates the base Oracle class with various configuration options for hyperparameter tuning.

```python
keras_tuner.Oracle(
    objective=None,
    max_trials=None,
    hyperparameters=None,
    allow_new_entries=True,
    tune_new_entries=True,
    seed=None,
    max_retries_per_trial=0,
    max_consecutive_failed_trials=3,
)

```

--------------------------------

### Hyperband Class Constructor

Source: https://keras.io/keras_tuner/api/tuners/hyperband

Initializes the Hyperband Tuner. Configure parameters like max_epochs, factor, and hyperband_iterations to control the search strategy. It's recommended to use early stopping during training.

```python
keras_tuner.Hyperband(
    hypermodel=None,
    objective=None,
    max_epochs=100,
    factor=3,
    hyperband_iterations=1,
    seed=None,
    hyperparameters=None,
    tune_new_entries=True,
    allow_new_entries=True,
    max_retries_per_trial=0,
    max_consecutive_failed_trials=3,
    **kwargs
)

```

--------------------------------

### Instantiate and Print an Integer Hyperparameter

Source: https://keras.io/keras_tuner/getting_started

Shows how to directly instantiate and retrieve a value for an integer hyperparameter using `hp.Int`. This is useful for understanding hyperparameter behavior or for debugging.

```python
hp = keras_tuner.HyperParameters()
print(hp.Int("units", min_value=32, max_value=512, step=32))
```

```text
32

```

--------------------------------

### BayesianOptimization Class Constructor

Source: https://keras.io/keras_tuner/api/tuners/bayesian

Initializes the BayesianOptimization tuner with various parameters to control the search process. Key arguments include the hypermodel, objective function, maximum number of trials, and parameters for exploration-exploitation balance.

```python
keras_tuner.BayesianOptimization(
    hypermodel=None,
    objective=None,
    max_trials=10,
    num_initial_points=None,
    alpha=0.0001,
    beta=2.6,
    seed=None,
    hyperparameters=None,
    tune_new_entries=True,
    allow_new_entries=True,
    max_retries_per_trial=0,
    max_consecutive_failed_trials=3,
    **kwargs
)

```

--------------------------------

### Define Integer Hyperparameter for Dense Layer Units

Source: https://keras.io/keras_tuner/getting_started

Use `hp.Int` to define a hyperparameter for the number of units in a Dense layer. This example tunes units between 32 and 512 with a step of 32.

```python
import keras
from keras import layers


def build_model(hp):
    model = keras.Sequential()
    model.add(layers.Flatten())
    model.add(
        layers.Dense(
            # Define the hyperparameter.
            units=hp.Int("units", min_value=32, max_value=512, step=32),
            activation="relu",
        )
    )
    model.add(layers.Dense(10, activation="softmax"))
    model.compile(
        optimizer="adam",
        loss="categorical_crossentropy",
        metrics=["accuracy"],
    )
    return model

```

--------------------------------

### Configure Chief Service Environment Variables

Source: https://keras.io/keras_tuner/guides/distributed_tuning

Set environment variables for the chief service in a distributed KerasTuner setup. This configures the tuner ID, IP address, and port for the chief process.

```bash
export KERASTUNER_TUNER_ID="chief"
export KERASTUNER_ORACLE_IP="127.0.0.1"
export KERASTUNER_ORACLE_PORT="8000"
python run_tuning.py

```

--------------------------------

### Import KerasTuner and Keras

Source: https://keras.io/keras_tuner

Import the necessary libraries, KerasTuner for hyperparameter tuning and Keras for model building.

```python
import keras_tuner
import keras

```

--------------------------------

### BayesianOptimizationOracle Class

Source: https://keras.io/keras_tuner/api/oracles/bayesian

Initializes a BayesianOptimizationOracle. This oracle uses Bayesian optimization with an underlying Gaussian process model and the Upper Confidence Bound (UCB) acquisition function to guide the hyperparameter search.

```APIDOC
## BayesianOptimizationOracle

### Description
Initializes a BayesianOptimizationOracle. This oracle uses Bayesian optimization with an underlying Gaussian process model and the Upper Confidence Bound (UCB) acquisition function to guide the hyperparameter search.

### Class Signature
```python
keras_tuner.oracles.BayesianOptimizationOracle(
    objective=None,
    max_trials=10,
    num_initial_points=None,
    alpha=0.0001,
    beta=2.6,
    seed=None,
    hyperparameters=None,
    allow_new_entries=True,
    tune_new_entries=True,
    max_retries_per_trial=0,
    max_consecutive_failed_trials=3,
)
```

### Arguments
* **objective** : A string, `keras_tuner.Objective` instance, or a list of `keras_tuner.Objective`s and strings. If a string, the direction of the optimization (min or max) will be inferred. If a list of `keras_tuner.Objective`, we will minimize the sum of all the objectives to minimize subtracting the sum of all the objectives to maximize. The `objective` argument is optional when `Tuner.run_trial()` or `HyperModel.fit()` returns a single float as the objective to minimize.
* **max_trials** : Integer, the total number of trials (model configurations) to test at most. Note that the oracle may interrupt the search before `max_trial` models have been tested if the search space has been exhausted. Defaults to 10.
* **num_initial_points** : Optional number of randomly generated samples as initial training data for Bayesian optimization. If left unspecified, a value of 3 times the dimensionality of the hyperparameter space is used.
* **alpha** : Float, the value added to the diagonal of the kernel matrix during fitting. It represents the expected amount of noise in the observed performances in Bayesian optimization. Defaults to 1e-4.
* **beta** : Float, the balancing factor of exploration and exploitation. The larger it is, the more explorative it is. Defaults to 2.6.
* **seed** : Optional integer, the random seed.
* **hyperparameters** : Optional `HyperParameters` instance. Can be used to override (or register in advance) hyperparameters in the search space.
* **tune_new_entries** : Boolean, whether hyperparameter entries that are requested by the hypermodel but that were not specified in `hyperparameters` should be added to the search space, or not. If not, then the default value for these parameters will be used. Defaults to True.
* **allow_new_entries** : Boolean, whether the hypermodel is allowed to request hyperparameter entries not listed in `hyperparameters`. Defaults to True.
* **max_retries_per_trial** : Integer. Defaults to 0. The maximum number of times to retry a `Trial` if the trial crashed or the results are invalid.
* **max_consecutive_failed_trials** : Integer. Defaults to 3. The maximum number of consecutive failed `Trial`s. When this number is reached, the search will be stopped. A `Trial` is marked as failed when none of the retries succeeded.
```

--------------------------------

### Test Model Building with HyperParameters

Source: https://keras.io/keras_tuner/guides/visualize_tuning

Test the `build_model` function by initializing `HyperParameters` and building a model. This verifies that the model can be constructed with specified hyperparameter values.

```python
# Initialize the `HyperParameters` and set the values.
hp = keras_tuner.HyperParameters()
hp.values["model_type"] = "cnn"
# Build the model using the `HyperParameters`.
model = build_model(hp)
# Test if the model runs with our data.
model(x_train[:100])
# Print a summary of the model.
model.summary()

```

--------------------------------

### Initialize tuner with retry and consecutive failure limits

Source: https://keras.io/keras_tuner/guides/failed_trials

Configure a GridSearch tuner with `max_retries_per_trial` and `max_consecutive_failed_trials` to manage trial failures. The tuner will attempt to retry failed trials and will only terminate the search after a specified number of consecutive failures.

```python
tuner = keras_tuner.GridSearch(
    hypermodel=build_model,
    objective="val_loss",
    overwrite=True,
    max_retries_per_trial=3,
    max_consecutive_failed_trials=8,
)

# Use random data to train the model.
tuner.search(
    x=np.random.rand(100, 20),
    y=np.random.rand(100, 1),
    validation_data=(
        np.random.rand(100, 20),
        np.random.rand(100, 1),
    ),
    epochs=10,
)

```

--------------------------------

### Load and Display TensorBoard in Colab

Source: https://keras.io/keras_tuner/guides/visualize_tuning

If running in Google Colab, use these magic commands to load the TensorBoard extension and display the logs within the notebook.

```python
%load_ext tensorboard
```

```python
%tensorboard --logdir /tmp/tb_logs
```

--------------------------------

### Initialize HyperResNet hypermodel

Source: https://keras.io/keras_tuner/getting_started

Instantiate a pre-made tunable hypermodel for computer vision tasks. HyperResNet comes pre-compiled with `loss='categorical_crossentropy'` and `metrics=['accuracy']`. Configure the tuner with the hypermodel, objective, and other parameters.

```python
from keras_tuner.applications import HyperResNet

hypermodel = HyperResNet(input_shape=(28, 28, 1), classes=10)

tuner = keras_tuner.RandomSearch(
    hypermodel,
    objective="val_accuracy",
    max_trials=2,
    overwrite=True,
    directory="my_dir",
    project_name="built_in_hypermodel",
)

```

--------------------------------

### Get and Summarize Best Models

Source: https://keras.io/keras_tuner/getting_started

Retrieves the top N best models from a Keras Tuner search and prints a summary of the best performing model. Ensure the tuner object is already defined and has completed a search.

```python
models = tuner.get_best_models(num_models=2)
best_model = models[0]
best_model.summary()
```

--------------------------------

### HyperbandOracle Constructor

Source: https://keras.io/keras_tuner/api/oracles/hyperband

Initializes the HyperbandOracle with configuration parameters for the Hyperband tuning algorithm. Key parameters include objective, max_epochs, factor, and hyperband_iterations.

```python
keras_tuner.oracles.HyperbandOracle(
    objective=None,
    max_epochs=100,
    factor=3,
    hyperband_iterations=1,
    seed=None,
    hyperparameters=None,
    allow_new_entries=True,
    tune_new_entries=True,
    max_retries_per_trial=0,
    max_consecutive_failed_trials=3,
)

```

--------------------------------

### Configure Worker Service Environment Variables

Source: https://keras.io/keras_tuner/guides/distributed_tuning

Set environment variables for worker services in a distributed KerasTuner setup. Each worker needs a unique ID, and they must connect to the chief's IP address and port.

```bash
export KERASTUNER_TUNER_ID="tuner0"
export KERASTUNER_ORACLE_IP="127.0.0.1"
export KERASTUNER_ORACLE_PORT="8000"
python run_tuning.py

```

--------------------------------

### Configure and Run RandomSearch Tuner

Source: https://keras.io/keras_tuner/getting_started

Initializes a Keras Tuner RandomSearch tuner, specifying the hypermodel, objective (val_mean_absolute_error with minimization), number of trials, and output directory. It then initiates the search process using random data and prints the results summary.

```python
tuner = keras_tuner.RandomSearch(
    hypermodel=build_regressor,
    # The objective name and direction.
    # Name is the f"val_{snake_case_metric_class_name}".
    objective=keras_tuner.Objective("val_mean_absolute_error", direction="min"),
    max_trials=3,
    overwrite=True,
    directory="my_dir",
    project_name="built_in_metrics",
)

tuner.search(
    x=np.random.rand(100, 10),
    y=np.random.rand(100, 1),
    validation_data=(np.random.rand(20, 10), np.random.rand(20, 1)),
)

tuner.results_summary()
```

--------------------------------

### Build Regressor Model with Built-in Metric

Source: https://keras.io/keras_tuner/getting_started

Defines a Keras Sequential model for regression and compiles it with 'adam' optimizer, 'mean_squared_error' loss, and MeanAbsoluteError as a metric. This setup is used to demonstrate using built-in metrics as tuning objectives.

```python
def build_regressor(hp):
    model = keras.Sequential(
        [
            layers.Dense(units=hp.Int("units", 32, 128, 32), activation="relu"),
            layers.Dense(units=1),
        ]
    )
    model.compile(
        optimizer="adam",
        loss="mean_squared_error",
        # Objective is one of the metrics.
        metrics=[keras.metrics.MeanAbsoluteError()],
    )
    return model
```

--------------------------------

### Configure and Run Distributed Hyperparameter Search

Source: https://keras.io/keras_tuner/guides/distributed_tuning

Initializes the Hyperband tuner with a mirrored distribution strategy for distributed tuning. It then loads and preprocesses the MNIST dataset and initiates the search process.

```python
tuner = keras_tuner.Hyperband(
    hypermodel=build_model,
    objective="val_accuracy",
    max_epochs=2,
    factor=3,
    hyperband_iterations=1,
    distribution_strategy=tf.distribute.MirroredStrategy(),
    directory="results_dir",
    project_name="mnist",
    overwrite=True,
)

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

# Reshape the images to have the channel dimension.
x_train = (x_train.reshape(x_train.shape + (1,)) / 255.0)[:1000]
y_train = y_train.astype(np.int64)[:1000]
x_test = (x_test.reshape(x_test.shape + (1,)) / 255.0)[:100]
y_test = y_test.astype(np.int64)[:100]

tuner.search(
    x_train,
    y_train,
    steps_per_epoch=600,
    validation_data=(x_test, y_test),
    validation_steps=100,
    callbacks=[keras.callbacks.EarlyStopping("val_accuracy")],
)
```

--------------------------------

### GridSearch Class Constructor

Source: https://keras.io/keras_tuner/api/tuners/grid

Initializes the GridSearch tuner. It requires a hypermodel and an objective to tune. Optional arguments include max_trials, seed, and parameters for handling new or failed trials.

```python
keras_tuner.GridSearch(
    hypermodel=None,
    objective=None,
    max_trials=None,
    seed=None,
    hyperparameters=None,
    tune_new_entries=True,
    allow_new_entries=True,
    max_retries_per_trial=0,
    max_consecutive_failed_trials=3,
    **kwargs
)

```

--------------------------------

### `build` method

Source: https://keras.io/keras_tuner/api/hypermodels/base_hypermodel

Builds a model. This method should be overridden by users to define their model architecture.

```APIDOC
## `build` method

### Description
Builds a model. This method should be overridden by users to define their model architecture.

### Arguments
* **hp** : A `HyperParameters` instance.

### Returns
A model instance.
```

--------------------------------

### create_trial

Source: https://keras.io/keras_tuner/api/oracles/base_oracle

Creates a new trial for hyperparameter tuning.

```APIDOC
## create_trial()

### Description
Creates a new trial for hyperparameter tuning. This method is typically called by the Tuner to get a new set of hyperparameters to evaluate.
```

--------------------------------

### Initialize HyperParameters

Source: https://keras.io/keras_tuner/api/hyperparameters

Instantiate the HyperParameters container. This object can be passed to HyperModel.build(hp) to construct a model.

```python
keras_tuner.HyperParameters()
```

--------------------------------

### Instantiate Objective

Source: https://keras.io/keras_tuner/api/tuners/objective

Instantiate the Objective class with a name and direction. The direction should be 'min' or 'max'.

```python
keras_tuner.Objective(name, direction)
```

--------------------------------

### Load Model from Trial

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Loads a Model from a given trial. For models reporting intermediate results, rely on `trial.best_step`.

```python
Tuner.load_model(trial)
```

--------------------------------

### Define Dynamic and Conditional Hyperparameters

Source: https://keras.io/keras_tuner/getting_started

Illustrates how to define hyperparameters dynamically, such as the number of layers, and how to create conditional hyperparameters that are only used when certain conditions are met (e.g., `units_3` used when `num_layers` > 3).

```python
def build_model(hp):
    model = keras.Sequential()
    model.add(layers.Flatten())
    # Tune the number of layers.
    for i in range(hp.Int("num_layers", 1, 3)):
        model.add(
            layers.Dense(

```

--------------------------------

### Build and Retrain a Model with HyperModel

Source: https://keras.io/keras_tuner/getting_started

Use the HyperModel to build and retrain the best model found during hyperparameter tuning. This allows for direct control over the model training process with the optimal configuration.

```python
hypermodel = MyHyperModel()
best_hp = tuner.get_best_hyperparameters()[0]
model = hypermodel.build(best_hp)
hypermodel.fit(best_hp, model, x_all, y_all, epochs=1)
```

--------------------------------

### Custom training loop logic

Source: https://keras.io/keras_tuner/guides/custom_tuner

This code demonstrates the core logic within a custom training loop, including iterating through training and validation data, running training and validation steps, and calling callbacks after each epoch. The `on_epoch_end` callback is crucial for logging metrics that the tuner will use.

```python
# Iterate the training data to run the training step.
for images, labels in train_ds:
    run_train_step(images, labels)

# Iterate the validation data to run the validation step.
for images, labels in validation_data:
    run_val_step(images, labels)

# Calling the callbacks after epoch.
epoch_loss = float(epoch_loss_metric.result().numpy())
for callback in callbacks:
    # The "my_metric" is the objective passed to the tuner.
    callback.on_epoch_end(epoch, logs={"my_metric": epoch_loss})
epoch_loss_metric.reset_state()

print(f"Epoch loss: {epoch_loss}")
best_epoch_loss = min(best_epoch_loss, epoch_loss)

# Return the evaluation metric value.
return best_epoch_loss
```

--------------------------------

### Instantiate HyperParameters

Source: https://keras.io/keras_tuner/getting_started

Create an instance of KerasTuner's HyperParameters class to define the search space for your model.

```python
build_model(keras_tuner.HyperParameters())
```

--------------------------------

### Perform Hyperparameter Search

Source: https://keras.io/keras_tuner/api/tuners/base_tuner

Performs a search for the best hyperparameter configurations, passing arguments to `run_trial`.

```python
Tuner.search(*fit_args, **fit_kwargs)
```

--------------------------------

### Create Trial

Source: https://keras.io/keras_tuner/api/oracles/base_oracle

Calls the create_trial method of the Oracle class to generate a new trial for hyperparameter tuning.

```python
keras_tuner.Oracle.create_trial()

```
Tokens:
Raw