### Setup Development Environment Source: https://github.com/keras-team/keras-tuner/blob/master/CONTRIBUTING.md Installs project dependencies and sets up a pre-commit hook for linting. Ensure you have pip installed and upgraded. ```shell pip install --upgrade pip pip install -e ".[tensorflow-cpu,tests]" echo "sh shell/lint.sh" > .git/hooks/pre-commit chmod a+x .git/hooks/pre-commit ``` -------------------------------- ### Install KerasTuner Source: https://github.com/keras-team/keras-tuner/blob/master/README.md Install the latest release of KerasTuner using pip. Ensure you have Python 3.8+ and TensorFlow 2.0+ installed. ```bash pip install keras-tuner ``` -------------------------------- ### Distributed Tuning Setup for KerasTuner Source: https://context7.com/keras-team/keras-tuner/llms.txt KerasTuner supports distributed tuning via environment variables. The chief process starts the Oracle gRPC server, and worker processes proxy trial requests through it. No code changes are required, only environment variables differ. ```python # chief.py (run first on the chief machine) import os os.environ["KERASTUNER_TUNER_ID"] = "chief" # No KERASTUNER_ORACLE_IP set → this instance acts as the Oracle server import keras_tuner, keras, numpy as np def build_model(hp): model = keras.Sequential([ keras.layers.Input(shape=(10,)), keras.layers.Dense(hp.Int("units", 32, 256, step=32), activation="relu"), keras.layers.Dense(1), ]) model.compile(optimizer="adam", loss="mse") return model tuner = keras_tuner.RandomSearch( build_model, objective="val_loss", max_trials=20, directory="dist_dir", project_name="dist_demo", ) x = np.random.rand(1000, 10) y = np.random.rand(1000, 1) # Chief blocks here, serving trial requests to workers via gRPC tuner.search(x, y, epochs=10, validation_split=0.2) # ------------------------------------------------------------------- # worker.py (run on each worker machine, pointing to the chief) # ------------------------------------------------------------------- # import os # os.environ["KERASTUNER_TUNER_ID"] = "tuner1" # os.environ["KERASTUNER_ORACLE_IP"] = "" # os.environ["KERASTUNER_ORACLE_PORT"] = "8000" # ... same tuner definition and tuner.search() call ... # Workers automatically proxy Oracle calls to the chief via gRPC ``` -------------------------------- ### BayesianOptimization for Guided Search Source: https://context7.com/keras-team/keras-tuner/llms.txt Employ BayesianOptimization for a guided hyperparameter search that balances exploration and exploitation using a Gaussian Process surrogate model. Configure parameters like num_initial_points, alpha, and beta for the acquisition function. ```python import keras_tuner import keras import numpy as np def build_model(hp): n_layers = hp.Int("n_layers", 1, 3) model = keras.Sequential([keras.layers.Input(shape=(20,))]) for i in range(n_layers): model.add(keras.layers.Dense( hp.Int(f"units_{i}", 16, 128, step=16), activation="relu", )) model.add(keras.layers.Dense(1)) model.compile( optimizer=keras.optimizers.SGD( learning_rate=hp.Float("lr", 1e-3, 0.5, sampling="log"), momentum=hp.Float("momentum", 0.0, 0.99, step=0.1), ), loss="mse", ) return model tuner = keras_tuner.BayesianOptimization( build_model, objective="val_loss", max_trials=20, num_initial_points=5, # random warm-up trials before GP kicks in alpha=1e-4, # noise level in GP kernel beta=2.6, # exploration vs. exploitation balance seed=1, directory="bayes_dir", project_name="bayes_demo", overwrite=True, ) x = np.random.rand(400, 20) y = np.random.rand(400, 1) tuner.search(x, y, epochs=15, validation_split=0.2, verbose=0) tuner.results_summary(num_trials=3) # Example output: # Results summary # Showing 3 best trials # ...Trial 07 | val_loss: 0.0821 | units_0: 96 | lr: 0.021 ... ``` -------------------------------- ### Search for Best Model Source: https://github.com/keras-team/keras-tuner/blob/master/README.md Start the hyperparameter search using the `search` method. Provide training and validation data, along with the number of epochs. Retrieve the best model after the search is complete. ```python tuner.search(x_train, y_train, epochs=5, validation_data=(x_val, y_val)) best_model = tuner.get_best_models()[0] ``` -------------------------------- ### Run Hyperparameter Search with Tuner.search Source: https://context7.com/keras-team/keras-tuner/llms.txt Use `tuner.search()` to execute hyperparameter trials. Arguments are forwarded to `model.fit()`. Set `overwrite=False` to resume a previous search. ```python import keras_tuner import keras import numpy as np def build_model(hp): model = keras.Sequential([ keras.layers.Input(shape=(15,)), keras.layers.Dense(hp.Int("units", 32, 128, step=32), activation="relu"), keras.layers.Dense(1), ]) model.compile( optimizer="adam", loss="mse", metrics=["mae"] ) return model tuner = keras_tuner.BayesianOptimization( build_model, objective="val_mae", max_trials=8, executions_per_trial=2, # train each config twice; average the results directory="search_dir", project_name="resume_demo", overwrite=True, ) x = np.random.rand(600, 15) y = np.random.rand(600, 1) callbacks = [ keras.callbacks.EarlyStopping(monitor="val_mae", patience=5), keras.callbacks.TensorBoard(log_dir="./logs"), ] tuner.search( x, y, epochs=50, validation_split=0.2, callbacks=callbacks, verbose=1, ) # Resume from the same directory (overwrite=False is the default) tuner2 = keras_tuner.BayesianOptimization( build_model, objective="val_mae", max_trials=8, executions_per_trial=2, directory="search_dir", project_name="resume_demo", # overwrite=False ← default: reloads existing results ) print("Remaining trials:", tuner2.remaining_trials) ``` -------------------------------- ### Retrieve Best Hyperparameters and Models Source: https://context7.com/keras-team/keras-tuner/llms.txt Use `get_best_hyperparameters()` and `get_best_models()` to retrieve search results. For production, retrain on the full dataset using the best hyperparameters. ```python import keras_tuner import keras import numpy as np def build_model(hp): model = keras.Sequential([ keras.layers.Input(shape=(10,)), keras.layers.Dense(hp.Int("units", 32, 128, step=32), activation="relu"), keras.layers.Dense(1), ]) model.compile(optimizer="adam", loss="mse") return model tuner = keras_tuner.RandomSearch( build_model, objective="val_loss", max_trials=5, directory="results_dir", project_name="results_demo", overwrite=True, ) x = np.random.rand(400, 10) y = np.random.rand(400, 1) tuner.search(x, y, epochs=10, validation_split=0.2, verbose=0) # --- Get best hyperparameters --- best_hps = tuner.get_best_hyperparameters(num_trials=3) for i, hp in enumerate(best_hps): print(f"Rank {i+1}: units={hp.get('units')}") # Rank 1: units=96 # Rank 2: units=64 # Rank 3: units=128 # --- Get best trained models (use for inference) --- best_model = tuner.get_best_models(num_models=1)[0] preds = best_model.predict(x[:5]) print("Predictions:", preds.flatten()) # --- Best practice: retrain on full data with best HPs --- best_hp = best_hps[0] final_model = tuner.hypermodel.build(best_hp) final_model.fit(x, y, epochs=20, verbose=0) ``` -------------------------------- ### Import KerasTuner and TensorFlow Source: https://github.com/keras-team/keras-tuner/blob/master/README.md Import the necessary libraries for using KerasTuner and TensorFlow. ```python import keras_tuner from tensorflow import keras ``` -------------------------------- ### Rebuild Protocol Buffers Source: https://github.com/keras-team/keras-tuner/blob/master/CONTRIBUTING.md Rebuilds generated Python files from .proto definitions. This is necessary after modifying any .proto files. ```shell pip install grpcio-tools python -m grpc_tools.protoc --python_out=. --grpc_python_out=. --proto_path=. keras_tuner/protos/keras_tuner.proto python -m grpc_tools.protoc --python_out=. --grpc_python_out=. --proto_path=. keras_tuner/protos/service.proto ``` -------------------------------- ### RandomSearch for Baseline Hyperparameter Tuning Source: https://context7.com/keras-team/keras-tuner/llms.txt Use RandomSearch to quickly find a baseline hyperparameter configuration. It samples configurations uniformly at random. Specify the model-building function, objective, maximum trials, and a directory for storing results. ```python import keras_tuner import keras import numpy as np def build_model(hp): model = keras.Sequential([ keras.layers.Input(shape=(30,)), keras.layers.Dense(hp.Int("units", 32, 256, step=32), activation="relu"), keras.layers.Dense(1), ]) model.compile( optimizer=keras.optimizers.Adam( hp.Float("learning_rate", 1e-4, 1e-2, sampling="log") ), loss="mse", ) return model tuner = keras_tuner.RandomSearch( build_model, objective="val_loss", max_trials=10, seed=42, directory="random_search", project_name="demo", overwrite=True, ) x = np.random.rand(500, 30) y = np.random.rand(500, 1) tuner.search(x, y, epochs=10, validation_split=0.2, verbose=0) # Retrieve the best 2 models (loaded with best-epoch weights) best_models = tuner.get_best_models(num_models=2) print(best_models[0].summary()) # Retrieve the best hyperparameter set and retrain from scratch best_hp = tuner.get_best_hyperparameters(1)[0] print("Best units:", best_hp.get("units")) print("Best LR: ", best_hp.get("learning_rate")) # Print a human-readable summary of all trials tuner.results_summary(num_trials=5) ``` -------------------------------- ### GridSearch: Exhaustive Search Over Discrete Spaces Source: https://context7.com/keras-team/keras-tuner/llms.txt Use GridSearch for small, fully discrete search spaces to ensure every combination is tested. Continuous hp.Float() without a step will be sampled at 10 evenly-spaced points. Ensure to cap trials with `max_trials` to avoid unbounded runs on large spaces. ```python import keras_tuner import keras import numpy as np def build_model(hp): model = keras.Sequential([ keras.layers.Input(shape=(10,)), keras.layers.Dense( hp.Choice("units", [32, 64, 128]), activation=hp.Choice("activation", ["relu", "tanh"]), ), keras.layers.Dense(1), ]) model.compile( optimizer=hp.Choice("optimizer", ["adam", "sgd"]), loss="mse", ) return model # Total combinations: 3 × 2 × 2 = 12 trials tuner = keras_tuner.GridSearch( build_model, objective="val_loss", max_trials=12, # cap to avoid unbounded runs on large spaces directory="grid_dir", project_name="grid_demo", overwrite=True, ) x = np.random.rand(300, 10) y = np.random.rand(300, 1) tuner.search(x, y, epochs=5, validation_split=0.2, verbose=0) tuner.search_space_summary() # Output: # Search space summary # Default search space size: 3 # units (Choice) {'values': [32, 64, 128], ...} # activation (Choice) {'values': ['relu', 'tanh'], ...} # optimizer (Choice) {'values': ['adam', 'sgd'], ...} ``` -------------------------------- ### Initialize RandomSearch Tuner Source: https://github.com/keras-team/keras-tuner/blob/master/README.md Initialize a KerasTuner `RandomSearch` object. Specify the model building function, the objective to optimize (e.g., 'val_loss'), and the maximum number of trials. ```python tuner = keras_tuner.RandomSearch( build_model, objective='val_loss', max_trials=5) ``` -------------------------------- ### Implement Custom Oracle with KerasTuner Source: https://context7.com/keras-team/keras-tuner/llms.txt Subclass keras_tuner.Oracle and implement populate_space() to define a custom search algorithm. Use @keras_tuner.synchronized for thread safety in parallel tuning. The end_trial method can be overridden for custom post-trial logic. ```python import keras_tuner import random class MyRandomOracle(keras_tuner.Oracle): """A minimal custom oracle that picks hyperparameter values uniformly.""" def populate_space(self, trial_id): # Use the built-in helper to generate a unique random combination values = self._random_values() if values is None: # Search space exhausted return {"status": "STOPPED", "values": None} return {"status": "RUNNING", "values": values} @keras_tuner.synchronized def end_trial(self, trial): super().end_trial(trial) # Custom post-trial logic (e.g. logging to external system) if trial.score is not None: print(f"Trial {trial.trial_id} finished | score={trial.score:.4f}") import keras import numpy as np def build_model(hp): model = keras.Sequential([ keras.layers.Input(shape=(5,)), keras.layers.Dense(hp.Choice("units", [16, 32, 64]), activation="relu"), keras.layers.Dense(1), ]) model.compile(optimizer="adam", loss="mse") return model tuner = keras_tuner.Tuner( oracle=MyRandomOracle(objective="val_loss", max_trials=5, seed=99), hypermodel=build_model, directory="custom_oracle_dir", project_name="oracle_demo", overwrite=True, ) x = np.random.rand(200, 5) y = np.random.rand(200, 1) tuner.search(x, y, epochs=5, validation_split=0.2, verbose=0) # Trial 00 finished | score=0.1023 # Trial 01 finished | score=0.0981 # ... ``` -------------------------------- ### Implement Custom Tuner by Subclassing Tuner Source: https://context7.com/keras-team/keras-tuner/llms.txt Override `run_trial()` in a `keras_tuner.Tuner` subclass for full control over the training loop, custom loss functions, or non-Keras objectives. Returning a float or dict reports metrics to the Oracle. ```python import keras_tuner import keras import numpy as np class MyTuner(keras_tuner.Tuner): def run_trial(self, trial, x, y, **kwargs): hp = trial.hyperparameters model = self.hypermodel.build(hp) batch_size = hp.Int("batch_size", 16, 128, step=16) history = model.fit( x, y, epochs=10, batch_size=batch_size, validation_split=0.2, verbose=0, ) # Return the best validation loss across epochs val_loss = min(history.history["val_loss"]) return {"val_loss": val_loss} def build_model(hp): model = keras.Sequential([ keras.layers.Input(shape=(8,)), keras.layers.Dense(hp.Int("units", 16, 64, step=16), activation="relu"), keras.layers.Dense(1), ]) model.compile(optimizer="adam", loss="mse") return model tuner = MyTuner( oracle=keras_tuner.oracles.RandomSearchOracle( objective="val_loss", max_trials=6, seed=7, ), hypermodel=build_model, directory="custom_tuner_dir", project_name="custom_demo", overwrite=True, ) x = np.random.rand(300, 8) y = np.random.rand(300, 1) tuner.search(x, y) print("Best HP:", tuner.get_best_hyperparameters(1)[0].values) # Best HP: {'units': 48, 'batch_size': 32} ``` -------------------------------- ### Hyperband for Resource-Efficient Search Source: https://context7.com/keras-team/keras-tuner/llms.txt Utilize Hyperband for efficient hyperparameter tuning, especially when epoch budget is limited. It adaptively allocates more epochs to promising configurations and early-stops weak ones. Configure max_epochs, factor for halving, and hyperband_iterations. ```python import keras_tuner import keras import numpy as np def build_model(hp): model = keras.Sequential([ keras.layers.Input(shape=(50,)), keras.layers.Dense( hp.Int("units", 32, 512, step=32), activation="relu" ), keras.layers.Dropout(hp.Float("dropout", 0.0, 0.5, step=0.1)), keras.layers.Dense(10, activation="softmax"), ]) model.compile( optimizer=keras.optimizers.Adam( hp.Float("lr", 1e-4, 1e-1, sampling="log") ), loss="sparse_categorical_crossentropy", metrics=["accuracy"], ) return model tuner = keras_tuner.Hyperband( build_model, objective="val_accuracy", # maximized automatically max_epochs=50, # max epochs per model factor=3, # halving factor hyperband_iterations=2, # how many full Hyperband sweeps seed=0, directory="hyperband_dir", project_name="hyperband_demo", overwrite=True, ) x = np.random.rand(1000, 50) y = np.random.randint(0, 10, size=(1000,)) early_stop = keras.callbacks.EarlyStopping(monitor="val_accuracy", patience=3) tuner.search(x, y, validation_split=0.2, callbacks=[early_stop], verbose=0) best_hp = tuner.get_best_hyperparameters(1)[0] print("Best config:", best_hp.values) # Retrain final model on full dataset final_model = tuner.hypermodel.build(best_hp) final_model.fit(x, y, epochs=best_hp.get("tuner/epochs", 10)) ``` -------------------------------- ### Define Model Search Space with HyperParameters Source: https://context7.com/keras-team/keras-tuner/llms.txt Use `hp.Int`, `hp.Float`, `hp.Choice`, and `hp.Boolean` to define tunable parameters within a model-building function. Conditional scopes allow parameters to be active only when a parent parameter meets specific criteria. The `hp.values` attribute shows the default values. ```python import keras_tuner import keras # --- hp.Choice: pick one value from a discrete list --- # --- hp.Int: integer range (inclusive on both ends) --- # --- hp.Float: floating-point range --- # --- hp.Boolean: True / False --- # --- hp.Fixed: constant, never tuned --- def build_model(hp): # Choose number of layers num_layers = hp.Int("num_layers", min_value=1, max_value=4, default=2) # Choose activation function activation = hp.Choice("activation", ["relu", "tanh", "swish"]) # Tune dropout only when depth > 1 use_dropout = hp.Boolean("use_dropout", default=False) model = keras.Sequential() model.add(keras.layers.Input(shape=(20,))) for _ in range(num_layers): units = hp.Int("units", min_value=32, max_value=512, step=32) model.add(keras.layers.Dense(units, activation=activation)) if use_dropout: with hp.conditional_scope("use_dropout", [True]): rate = hp.Float("dropout_rate", 0.1, 0.5, step=0.1) model.add(keras.layers.Dropout(rate or 0.0)) # Log-scale learning rate: samples 1e-4, 1e-3, 1e-2, 1e-1 lr = hp.Float("learning_rate", 1e-4, 1e-1, sampling="log") model.add(keras.layers.Dense(1)) model.compile( optimizer=keras.optimizers.Adam(learning_rate=lr), loss="mse", metrics=["mae"], ) return model # Inspect the space without running a full search hp = keras_tuner.HyperParameters() build_model(hp) print(hp.values) # Expected output (default values): # {'num_layers': 2, 'activation': 'relu', 'use_dropout': False, # 'units': 32, 'learning_rate': 0.0001} ``` -------------------------------- ### SklearnTuner: Hyperparameter Search for Scikit-learn Models Source: https://context7.com/keras-team/keras-tuner/llms.txt Integrate KerasTuner's search algorithms with scikit-learn models using cross-validated evaluation. This tuner works with pipelines, custom scorers, and group-based CV splitters. Ensure your `build_model` function returns a scikit-learn compatible estimator. ```python import keras_tuner from sklearn import ensemble, linear_model, metrics, model_selection, datasets import numpy as np def build_model(hp): model_type = hp.Choice("model_type", ["random_forest", "ridge"]) if model_type == "random_forest": return ensemble.RandomForestClassifier( n_estimators=hp.Int("n_estimators", 10, 100, step=10), max_depth=hp.Int("max_depth", 2, 10), random_state=0, ) else: return linear_model.RidgeClassifier( alpha=hp.Float("alpha", 1e-3, 10.0, sampling="log"), ) tuner = keras_tuner.SklearnTuner( oracle=keras_tuner.oracles.BayesianOptimizationOracle( objective=keras_tuner.Objective("score", direction="max"), max_trials=15, seed=42, ), hypermodel=build_model, scoring=metrics.make_scorer(metrics.accuracy_score), cv=model_selection.StratifiedKFold(n_splits=5), directory="sklearn_dir", project_name="sklearn_demo", overwrite=True, ) X, y = datasets.load_iris(return_X_y=True) X_train, X_test, y_train, y_test = model_selection.train_test_split( X, y, test_size=0.2, random_state=0 ) tuner.search(X_train, y_train) best_model = tuner.get_best_models(num_models=1)[0] print("Test accuracy:", best_model.score(X_test, y_test)) # Expected output: Test accuracy: ~0.97 ``` -------------------------------- ### Define Model Building Function with Hyperparameters Source: https://github.com/keras-team/keras-tuner/blob/master/README.md Create a function that builds a Keras model. Use the `hp` argument to define hyperparameters, such as the number of units and activation functions for dense layers. ```python def build_model(hp): model = keras.Sequential() model.add(keras.layers.Dense( hp.Choice('units', [8, 16, 32]), activation='relu')) model.add(keras.layers.Dense(1, activation='relu')) model.compile(loss='mse') return model ``` -------------------------------- ### Create Reusable Search Space with HyperModel Source: https://context7.com/keras-team/keras-tuner/llms.txt Subclass `keras_tuner.HyperModel` to encapsulate a search space and custom training logic. Override `build(hp)` to define the model architecture and `fit(hp, model, ...)` to customize the training process. This allows for more complex and reusable hyperparameter tuning configurations. ```python import keras_tuner import keras import numpy as np class MyHyperModel(keras_tuner.HyperModel): def build(self, hp): model = keras.Sequential([ keras.layers.Input(shape=(10,)), keras.layers.Dense( hp.Int("units", 16, 128, step=16), activation=hp.Choice("activation", ["relu", "elu"]), ), keras.layers.Dense(1), ]) model.compile( optimizer=keras.optimizers.Adam( hp.Float("lr", 1e-4, 1e-2, sampling="log") ), loss="mse", ) return model def fit(self, hp, model, *args, **kwargs): # Tune the number of epochs as a hyperparameter return model.fit( *args, epochs=hp.Int("epochs", 5, 30, step=5), **kwargs, ) # Use the class with any tuner tuner = keras_tuner.RandomSearch( MyHyperModel(), objective="val_loss", max_trials=5, directory="my_dir", project_name="hypermodel_demo", overwrite=True, ) x_train = np.random.rand(200, 10) y_train = np.random.rand(200, 1) x_val = np.random.rand(50, 10) y_val = np.random.rand(50, 1) tuner.search(x_train, y_train, validation_data=(x_val, y_val), verbose=0) best_hp = tuner.get_best_hyperparameters(1)[0] print(best_hp.values) ``` -------------------------------- ### Objective: Controlling Optimization Direction Source: https://context7.com/keras-team/keras-tuner/llms.txt Use `keras_tuner.Objective` to specify the metric to optimize and its direction (minimize or maximize). For custom metrics or multi-objective optimization, explicitly define `Objective` instances. When a string is passed to `objective=`, KerasTuner infers direction from known metric names. ```python import keras_tuner # Single objective — explicit direction obj = keras_tuner.Objective("val_f1_score", direction="max") print(obj) # Objective(name="val_f1_score", direction="max") # Multi-objective: minimize val_loss AND maximize val_accuracy multi_obj = [ keras_tuner.Objective("val_loss", direction="min"), keras_tuner.Objective("val_accuracy", direction="max"), ] tuner = keras_tuner.RandomSearch( hypermodel=lambda hp: None, # placeholder objective=multi_obj, max_trials=10, directory="multi_obj_dir", project_name="multi_obj", overwrite=True, ) # The oracle minimizes: sum(min objectives) - sum(max objectives) print(tuner.oracle.objective) # MultiObjective(name="multi_objective", direction="min"): # [Objective(name="val_loss", direction="min"), # Objective(name="val_accuracy", direction="max")] ``` -------------------------------- ### KerasTuner BibTeX Citation Source: https://github.com/keras-team/keras-tuner/blob/master/README.md BibTeX entry for citing KerasTuner in academic publications. Includes title, authors, year, and URL. ```bibtex @misc{omalley2019kerastuner, title = {KerasTuner}, author = {O'Malley, Tom and Bursztein, Elie and Long, James and Chollet, Franois and Jin, Haifeng and Invernizzi, Luca and others}, year = 2019, howpublished = {\url{https://github.com/keras-team/keras-tuner}} } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.