### Automatic Tuning Example Setup Source: https://mlr3tuning.mlr-org.com/reference/AutoTuner.html Sets up a task and performs a data split for training and an external set, commonly used before initiating automatic tuning. ```R # Automatic Tuning # split to train and external set task = tsk("penguins") split = partition(task, ratio = 0.8) ``` -------------------------------- ### TunerBatchDesignPoints Example Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_design_points.html Example demonstrating how to use the `TunerBatchDesignPoints` to tune hyperparameters for a learner on a given task. ```APIDOC ## Examples __``` # Hyperparameter Optimization # load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1), minsplit = to_tune(2, 128), minbucket = to_tune(1, 64) ) # create design design = mlr3misc::rowwise_table( ~cp, ~minsplit, ~minbucket, 0.1, 2, 64, 0.01, 64, 32, 0.001, 128, 1 ) # run hyperparameter tuning on the Palmer Penguins data set instance = tune( tuner = tnr("design_points", design = design), task = tsk("penguins"), learner = learner, resampling = rsmp("holdout"), measure = msr("classif.ce") ) # best performing hyperparameter configuration instance$result #> cp minbucket minsplit learner_param_vals x_domain classif.ce #> #> 1: 0.01 32 64 0.07826087 ``` ``` -------------------------------- ### Example Usage of Grid Search Tuner Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_grid_search.html Example demonstrating how to use the grid search tuner for hyperparameter optimization. ```APIDOC ## Examples __``` # Hyperparameter Optimization # load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) # run hyperparameter tuning on the Palmer Penguins data set instance = tune( tuner = tnr("grid_search"), task = tsk("penguins"), learner = learner, resampling = rsmp("holdout"), measure = msr("classif.ce"), term_evals = 10 ) # best performing hyperparameter configuration instance$result #> cp learner_param_vals x_domain classif.ce #> #> 1: -2.302585 0.05217391 ``` ``` -------------------------------- ### Install mlr3tuning Development Version from GitHub Source: https://mlr3tuning.mlr-org.com/index.html Installs the development version of the mlr3tuning package directly from its GitHub repository using the 'pak' package manager. ```r # install.packages("pak") pak::pak("mlr-org/mlr3tuning") ``` -------------------------------- ### Example Usage of TunerBatchGenSA Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_gensa.html An example demonstrating how to use the TunerBatchGenSA for hyperparameter optimization with a classification learner. ```APIDOC ## Examples ```R # example only runs if GenSA is available if (mlr3misc::require_namespaces("GenSA", quietly = TRUE)) { # Hyperparameter Optimization # load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE), minsplit = to_tune(p_dbl(2, 128, trafo = as.integer)), minbucket = to_tune(p_dbl(1, 64, trafo = as.integer)) ) } ``` ``` -------------------------------- ### Example: Hyperparameter Optimization with ti() Source: https://mlr3tuning.mlr-org.com/reference/ti.html Demonstrates a full hyperparameter optimization workflow using `ti()`. This includes setting up a task, learner with a tunable parameter, the tuning instance, an optimization algorithm, running the optimization, and applying the best hyperparameters back to the learner. ```R # Hyperparameter optimization on the Palmer Penguins data set task = tsk("penguins") # Load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) # Construct tuning instance instance = ti( task = task, learner = learner, resampling = rsmp("cv", folds = 3), measures = msr("classif.ce"), terminator = trm("evals", n_evals = 4) ) # Choose optimization algorithm tuner = tnr("random_search", batch_size = 2) # Run tuning tuner$optimize(instance) #> cp learner_param_vals x_domain classif.ce #> #> 1: -7.66021 0.06389525 # Set optimal hyperparameter configuration to learner learner$param_set$values = instance$result_learner_param_vals # Train the learner on the full data set learner$train(task) ``` -------------------------------- ### Hyperparameter Optimization Example Source: https://mlr3tuning.mlr-org.com/reference/TuningInstanceBatchSingleCrit.html Demonstrates a full hyperparameter optimization workflow using TuningInstanceBatchSingleCrit. This includes setting up the task, learner with a tunable parameter, tuning instance, tuner, running the optimization, and applying the best found parameters back to the learner. ```R # Hyperparameter optimization on the Palmer Penguins data set task = tsk("penguins") # Load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) # Construct tuning instance instance = ti( task = task, learner = learner, resampling = rsmp("cv", folds = 3), measures = msr("classif.ce"), terminator = trm("evals", n_evals = 4) ) # Choose optimization algorithm tuner = tnr("random_search", batch_size = 2) # Run tuning tuner$optimize(instance) #> cp learner_param_vals x_domain classif.ce #> #> 1: -3.036646 0.06392067 # Set optimal hyperparameter configuration to learner learner$param_set$values = instance$result_learner_param_vals # Train the learner on the full data set learner$train(task) ``` -------------------------------- ### Example: Tune rpart Learner on Penguins Data Source: https://mlr3tuning.mlr-org.com/reference/ti_async.html This example demonstrates hyperparameter optimization for an rpart learner on the penguins dataset. It shows how to define the task, learner with a tunable parameter (`cp`), resampling strategy, performance measure, and terminator. The tuning instance is then created using `ti()`, an optimization algorithm (`random_search`) is chosen, and the tuning is executed. Finally, the optimal hyperparameters are applied to the learner. ```R # Hyperparameter optimization on the Palmer Penguins data set task = tsk("penguins") # Load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) # Construct tuning instance instance = ti( task = task, learner = learner, resampling = rsmp("cv", folds = 3), measures = msr("classif.ce"), terminator = trm("evals", n_evals = 4) ) # Choose optimization algorithm tuner = tnr("random_search", batch_size = 2) # Run tuning tuner$optimize(instance) #> cp learner_param_vals x_domain classif.ce #> #> 1: -9.003 0.06987033 # Set optimal hyperparameter configuration to learner learner$param_set$values = instance$result_learner_param_vals # Train the learner on the full data set learner$train(task) ``` -------------------------------- ### Example: Nested Resampling with AutoTuner Source: https://mlr3tuning.mlr-org.com/reference/extract_inner_tuning_archives.html This example demonstrates how to set up a nested resampling scenario using `auto_tuner` and `resample`. Ensure `store_models = TRUE` is used in `resample` to enable archive extraction. The `cp` hyperparameter is tuned using random search. ```R learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE)) # create auto tuner at = auto_tuner( tuner = tnr("random_search"), learner = learner, resampling = rsmp ("holdout"), measure = msr("classif.ce"), term_evals = 4) resampling_outer = rsmp("cv", folds = 2) rr = resample(tsk("iris"), at, resampling_outer, store_models = TRUE) # extract inner archives extract_inner_tuning_archives(rr) ``` -------------------------------- ### Hyperparameter Optimization Example Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_gensa.html Example of using the GenSA tuner for hyperparameter optimization. This code requires the GenSA package to be available and demonstrates setting up a learner with a tuning space. ```R # example only runs if GenSA is available if (mlr3misc::require_namespaces("GenSA", quietly = TRUE)) { # Hyperparameter Optimization # load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE), minsplit = to_tune(p_dbl(2, 128, trafo = as.integer)), minbucket = to_tune(p_dbl(1, 64, trafo = as.integer)) ) } ``` -------------------------------- ### Example: Nested Resampling with Inner Tuning Results Extraction Source: https://mlr3tuning.mlr-org.com/reference/extract_inner_tuning_results.html Demonstrates a complete nested resampling workflow, including setting up a learner with tunable parameters, creating an auto-tuner, performing nested resampling, and finally extracting the inner tuning results. This example uses the iris dataset and a simple cross-validation for outer resampling. ```R learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE)) # create auto tuner at = auto_tuner( tuner = tnr("random_search"), learner = learner, resampling = rsmp ("holdout"), measure = msr("classif.ce"), term_evals = 4) resampling_outer = rsmp("cv", folds = 2) rr = resample(tsk("iris"), at, resampling_outer, store_models = TRUE) # extract inner results extract_inner_tuning_results(rr) ``` -------------------------------- ### Create a CallbackBatchTuning Instance Source: https://mlr3tuning.mlr-org.com/reference/callback_batch_tuning.html Creates a `CallbackBatchTuning` instance with a specified ID and an `on_optimization_end` stage. This example demonstrates saving the tuning archive to a file when the optimization concludes. ```R callback_batch_tuning("mlr3tuning.backup", on_optimization_end = function(callback, context) { saveRDS(context$instance$archive, "archive.rds") } ) #> #> * Active Stages: on_optimization_end ``` -------------------------------- ### Example of Internal Hyperparameter Tuning Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_internal.html This example demonstrates how to use the TunerBatchInternal for hyperparameter tuning on the pima indians diabetes dataset using an xgboost learner. ```APIDOC ## Examples ```R # example only runs if mlr3learners and xgboost are available if (mlr3misc::require_namespaces(c("mlr3learners", "xgboost"), quietly = TRUE)) { library(mlr3learners) # Retrieve task task = tsk("pima") # Load learner and set search space learner = lrn("classif.xgboost", nrounds = to_tune(upper = 1000, internal = TRUE), early_stopping_rounds = 10, validate = "test", eval_metric = "merror" ) # Internal hyperparameter tuning on the pima indians diabetes data set instance = tune( tnr("internal"), tsk("iris"), learner, rsmp("cv", folds = 3), msr("internal_valid_score", minimize = TRUE, select = "merror") ) # best performing hyperparameter configuration instance$result_learner_param_vals instance$result_learner_param_vals$internal_tuned_values } #> NULL ``` ``` -------------------------------- ### Internal Hyperparameter Tuning Example Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_internal.html This example demonstrates internal hyperparameter tuning for an XGBoost learner on the pima dataset using cross-validation. It shows how to set up the task, learner with tunable parameters, and the tuning instance. The best performing hyperparameter configuration is then accessed from the instance's results. ```R if (mlr3misc::require_namespaces(c("mlr3learners", "xgboost"), quietly = TRUE)) { library(mlr3learners) # Retrieve task task = tsk("pima") # Load learner and set search space learner = lrn("classif.xgboost", nrounds = to_tune(upper = 1000, internal = TRUE), early_stopping_rounds = 10, validate = "test", eval_metric = "merror" ) # Internal hyperparameter tuning on the pima indians diabetes data set instance = tune( tnr("internal"), tsk("iris"), learner, rsmp("cv", folds = 3), msr("internal_valid_score", minimize = TRUE, select = "merror") ) # best performing hyperparameter configuration instance$result_learner_param_vals instance$result_learner_param_vals$internal_tuned_values } ``` -------------------------------- ### CMA-ES Hyperparameter Optimization Example Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_cmaes.html Demonstrates a full hyperparameter optimization workflow using the CMA-ES tuner. This includes setting up a learner with tunable parameters, defining a tuning instance with a task, resampling strategy, and performance measure, and then executing the tuning process. It also shows how to access the results and fit a final model. ```R if (mlr3misc::require_namespaces("adagio", quietly = TRUE)) { # Hyperparameter Optimization # load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE), minsplit = to_tune(p_dbl(2, 128, trafo = as.integer)), minbucket = to_tune(p_dbl(1, 64, trafo = as.integer)) ) # run hyperparameter tuning on the Palmer Penguins data set instance = tune( tuner = tnr("cmaes"), task = tsk("penguins"), learner = learner, resampling = rsmp("holdout"), measure = msr("classif.ce"), term_evals = 10) # best performing hyperparameter configuration instance$result # all evaluated hyperparameter configuration as.data.table(instance$archive) # fit final model on complete data set learner$param_set$values = instance$result_learner_param_vals learner$train(tsk("penguins")) } ``` -------------------------------- ### Optimize Tuning Instance with Tuner Source: https://mlr3tuning.mlr-org.com/index.html Starts the hyperparameter optimization process by passing the tuning instance to the selected tuner. The tuner will search for the best hyperparameters. ```r tuner$optimize(instance) ``` -------------------------------- ### Run Hyperparameter Tuning with Design Points Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_design_points.html Execute hyperparameter tuning using the TunerBatchDesignPoints with a specified learner, task, resampling strategy, and performance measure. The example demonstrates tuning a `classif.rpart` learner on the penguins dataset. ```R # load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1), minsplit = to_tune(2, 128), minbucket = to_tune(1, 64) ) # create design design = mlr3misc::rowwise_table( ~cp, ~minsplit, ~minbucket, 0.1, 2, 64, 0.01, 64, 32, 0.001, 128, 1 ) # run hyperparameter tuning on the Palmer Penguins data set instance = tune( tuner = tnr("design_points", design = design), task = tsk("penguins"), learner = learner, resampling = rsmp("holdout"), measure = msr("classif.ce") ) # best performing hyperparameter configuration instance$result #> cp minbucket minsplit learner_param_vals x_domain classif.ce #> #> 1: 0.01 32 64 0.07826087 ``` -------------------------------- ### Grid Search Hyperparameter Optimization Example Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_grid_search.html Demonstrates how to perform hyperparameter tuning using the grid search tuner. Load a learner, define its search space, and then use the `tune` function with the grid search tuner, a task, resampling strategy, and performance measure. The `term_evals` argument controls the number of hyperparameter combinations to evaluate. ```R learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) instance = tune( tuner = tnr("grid_search"), task = tsk("penguins"), learner = learner, resampling = rsmp("holdout"), measure = msr("classif.ce"), term_evals = 10 ) # best performing hyperparameter configuration instance$result #> #> #> #> #> cp learner_param_vals x_domain classif.ce #> #> 1: -2.302585 0.05217391 ``` -------------------------------- ### mlr_tuners$get("random_search") Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners.html Retrieves a specific tuner object from the dictionary by its ID. This example shows how to get the 'random_search' tuner. ```APIDOC ## mlr_tuners$get(id) ### Description Retrieves a specific tuner object from the dictionary by its ID. ### Method Dictionary method ### Parameters #### Path Parameters None #### Query Parameters - **id** (character(1)) - The unique identifier of the tuner to retrieve. ### Request Example ```R mlr_tuners$get("random_search") ``` ### Response #### Success Response (200) - **Tuner** - The requested tuner object. #### Response Example ```R #> #> ── : Random Search ───────────────────────────────────── #> • Parameters: batch_size=1 #> • Parameter classes: , , , and #> • Properties: dependencies, single-crit, and multi-crit #> • Packages: mlr3tuning and bbotk ``` ``` -------------------------------- ### Basic AutoTuner Usage Source: https://mlr3tuning.mlr-org.com/reference/AutoTuner.html Demonstrates how to set up and use the AutoTuner for hyperparameter tuning. This involves defining a learner with tunable parameters, creating an AutoTuner instance with a specific tuner, resampling method, and performance measure, and then training the tuner on a task. Finally, it shows how to predict with the tuned model and access the tuning results. ```R learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) # create auto tuner at = auto_tuner( tuner = tnr("random_search"), learner = learner, resampling = rsmp ("holdout"), measure = msr("classif.ce"), term_evals = 4) # tune hyperparameters and fit final model at$train(task, row_ids = split$train) # predict with final model at$predict(task, row_ids = split$test) # show tuning result at$tuning_result # model slot contains trained learner and tuning instance at$model # shortcut trained learner at$learner # shortcut tuning instance at$tuning_instance ``` -------------------------------- ### Display Tuning Instance Summary Source: https://mlr3tuning.mlr-org.com/index.html Prints a summary of the created tuning instance, showing its state, objective, search space (including tunable parameters like 'cost' and 'gamma' with their ranges), and terminator. ```r ## ## ── ───────────────────────────────────────────────────────────────── ## • State: Not optimized ## • Objective: ## • Search Space: ## id class lower upper nlevels ## 1: cost ParamDbl -11.51293 11.51293 Inf ## 2: gamma ParamDbl -11.51293 11.51293 Inf ## • Terminator: ``` -------------------------------- ### Create TunerBatchFromOptimizerBatch Instance Source: https://mlr3tuning.mlr-org.com/reference/TunerBatchFromOptimizerBatch.html Use the new() method to create an instance of TunerBatchFromOptimizerBatch. Requires a bbotk::Optimizer object and optionally a manual string. ```r TunerBatchFromOptimizerBatch$new(optimizer, man = NA_character_) ``` -------------------------------- ### Example Usage of CMA-ES Tuner Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_cmaes.html An example demonstrating how to use the CMA-ES tuner for hyperparameter optimization on the Palmer Penguins dataset. ```APIDOC ## Examples __``` # example only runs if adagio is available if (mlr3misc::require_namespaces("adagio", quietly = TRUE)) { # Hyperparameter Optimization # load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE), minsplit = to_tune(p_dbl(2, 128, trafo = as.integer)), minbucket = to_tune(p_dbl(1, 64, trafo = as.integer)) ) # run hyperparameter tuning on the Palmer Penguins data set instance = tune( tuner = tnr("cmaes"), task = tsk("penguins"), learner = learner, resampling = rsmp("holdout"), measure = msr("classif.ce"), term_evals = 10) # best performing hyperparameter configuration instance$result # all evaluated hyperparameter configuration as.data.table(instance$archive) # fit final model on complete data set learner$param_set$values = instance$result_learner_param_vals learner$train(tsk("penguins")) } ``` ``` -------------------------------- ### Install mlr3tuning from CRAN Source: https://mlr3tuning.mlr-org.com/index.html Installs the latest stable release of the mlr3tuning package from the Comprehensive R Archive Network (CRAN). ```r install.packages("mlr3tuning") ``` -------------------------------- ### Create ObjectiveTuning Instance Source: https://mlr3tuning.mlr-org.com/reference/ObjectiveTuning.html Constructs a new ObjectiveTuning instance. Requires a task, learner, resampling strategy, and performance measures. Optional arguments control benchmark result storage, model storage, value checking, callbacks, and the internal search space. ```R ObjectiveTuning$new( task, learner, resampling, measures, store_benchmark_result = TRUE, store_models = FALSE, check_values = FALSE, callbacks = NULL, internal_search_space = NULL ) ``` -------------------------------- ### ObjectiveTuningBatch$new() Source: https://mlr3tuning.mlr-org.com/reference/ObjectiveTuningBatch.html Creates a new instance of the ObjectiveTuningBatch class. ```APIDOC ## ObjectiveTuningBatch$new() ### Description Creates a new instance of this R6 class. ### Usage ```R ObjectiveTuningBatch$new( task, learner, resampling, measures, store_benchmark_result = TRUE, store_models = FALSE, check_values = FALSE, archive = NULL, callbacks = NULL, internal_search_space = NULL ) ``` ### Arguments * `task` (mlr3::Task) - Task to operate on. * `learner` (mlr3::Learner) - Learner to tune. * `resampling` (mlr3::Resampling) - Resampling that is used to evaluate the performance of the hyperparameter configurations. Uninstantiated resamplings are instantiated during construction so that all configurations are evaluated on the same data splits. Already instantiated resamplings are kept unchanged. Specialized Tuner change the resampling e.g. to evaluate a hyperparameter configuration on different data splits. This field, however, always returns the resampling passed in construction. * `measures` (list of mlr3::Measure) - Measures to optimize. * `store_benchmark_result` (logical(1)) - If `TRUE` (default), store resample result of evaluated hyperparameter configurations in archive as mlr3::BenchmarkResult. * `store_models` (logical(1)) - If `TRUE`, fitted models are stored in the benchmark result (`archive$benchmark_result`). If `store_benchmark_result = FALSE`, models are only stored temporarily and not accessible after the tuning. This combination is needed for measures that require a model. * `check_values` (logical(1)) - If `TRUE`, hyperparameter values are checked before evaluation and performance scores after. If `FALSE` (default), values are unchecked but computational overhead is reduced. * `archive` (ArchiveBatchTuning) - Reference to archive of TuningInstanceBatchSingleCrit | TuningInstanceBatchMultiCrit. If `NULL` (default), benchmark result and models cannot be stored. * `callbacks` (list of mlr3misc::Callback) - List of callbacks. * `internal_search_space` (paradox::ParamSet or NULL) - The internal search space. ``` -------------------------------- ### TunerBatchFromOptimizerBatch$new() Source: https://mlr3tuning.mlr-org.com/reference/TunerBatchFromOptimizerBatch.html Creates a new instance of the TunerBatchFromOptimizerBatch class. ```APIDOC ## Method `new()` Creates a new instance of this R6 class. ### Usage ``` TunerBatchFromOptimizerBatch$new(optimizer, man = NA_character_) ``` ### Arguments `optimizer` bbotk::Optimizer Optimizer that is called. `man` (`character(1)`) String in the format `[pkg]::[topic]` pointing to a manual page for this object. The referenced help package can be opened via method `$help()`. ``` -------------------------------- ### as.data.table.ArchiveTuning Source: https://mlr3tuning.mlr-org.com/reference/ArchiveBatchTuning.html Returns a tabular view of all evaluated hyperparameter configurations, joined with benchmark results. ```APIDOC ## as.data.table.ArchiveTuning ### Description Returns a tabular view of all evaluated hyperparameter configurations. ### Usage ```R as.data.table.ArchiveTuning(x, unnest = "x_domain", exclude_columns = "uhash", measures = NULL) ``` ### Arguments * `x` (ArchiveTuning) The ArchiveTuning object. * `unnest` (character()) Transforms list columns to separate columns. Set to `NULL` if no column should be unnested. * `exclude_columns` (character()) Exclude columns from table. Set to `NULL` if no column should be excluded. * `measures` (List of mlr3::Measure) Score hyperparameter configurations on additional measures. ``` -------------------------------- ### Create and Run Random Search Tuning Instance Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_random_search.html Demonstrates how to set up and execute a hyperparameter tuning process using the random search tuner. It involves defining a learner with tunable parameters, a task, resampling strategy, and a performance measure. ```R learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) # run hyperparameter tuning on the Palmer Penguins data set instance = tune( tuner = tnr("random_search"), task = tsk("penguins"), learner = learner, resampling = rsmp("holdout"), measure = msr("classif.ce"), term_evals = 10 ) # best performing hyperparameter configuration instance$result #> cp learner_param_vals x_domain classif.ce #> #> 1: -2.675162 0.07826087 ``` -------------------------------- ### Get Log-Likelihood Source: https://mlr3tuning.mlr-org.com/reference/AutoTuner.html Returns the log-likelihood value of the final tuned model. ```R AutoTuner$loglik() ``` -------------------------------- ### AutoTuner$oob_error() Source: https://mlr3tuning.mlr-org.com/reference/AutoTuner.html Gets the out-of-bag error estimate from the final tuned model. ```APIDOC ## AutoTuner$oob_error() ### Description The out-of-bag error of the final model. ### Usage ```R AutoTuner$oob_error() ``` ### Returns `numeric(1)`. ``` -------------------------------- ### Initialize Backup Callback Source: https://mlr3tuning.mlr-org.com/reference/mlr3tuning.backup.html Instantiate the mlr3tuning.backup callback, specifying the path where results will be saved. This callback is active during optimization. ```R clbk("mlr3tuning.backup", path = "backup.rds") #> : Backup Benchmark Result Callback #> * Active Stages: on_optimizer_after_eval, on_optimization_begin ``` -------------------------------- ### Create and Run a Multi-Objective Tuning Instance Source: https://mlr3tuning.mlr-org.com/reference/TuningInstanceBatchMultiCrit.html Demonstrates how to set up a multi-objective hyperparameter optimization task using the Palmer Penguins dataset. It includes defining the task, learner with a tunable parameter, resampling strategy, performance measures, and a terminator. The tuning is then executed using a random search tuner with a specified batch size, and the results are inspected. ```R task = tsk("penguins") # Load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) # Construct tuning instance instance = ti( task = task, learner = learner, resampling = rsmp("cv", folds = 3), measures = msrs(c("classif.ce", "time_train")), terminator = trm("evals", n_evals = 4) ) # Choose optimization algorithm tuner = tnr("random_search", batch_size = 2) # Run tuning tuner$optimize(instance) #> cp learner_param_vals x_domain classif.ce time_train #> #> 1: -3.08083 0.09583016 0.003333333 # Optimal hyperparameter configurations instance$result #> cp learner_param_vals x_domain classif.ce time_train #> #> 1: -3.08083 0.09583016 0.003333333 # Inspect all evaluated configurations as.data.table(instance$archive) #> cp classif.ce time_train runtime_learners timestamp #> #> 1: -3.259804 0.09583016 0.004000000 0.022 2026-03-17 07:30:06 #> 2: -3.759791 0.09583016 0.003666667 0.020 2026-03-17 07:30:06 #> 3: -2.565382 0.09583016 0.008666667 0.036 2026-03-17 07:30:06 #> 4: -3.080830 0.09583016 0.003333333 0.019 2026-03-17 07:30:06 #> warnings errors x_domain batch_nr resample_result #> #> 1: 0 0 1 #> 2: 0 0 1 #> 3: 0 0 2 #> 4: 0 0 2 ``` -------------------------------- ### Get Importance Scores Source: https://mlr3tuning.mlr-org.com/reference/AutoTuner.html Returns the importance scores computed from the final tuned model. ```R AutoTuner$importance() ``` -------------------------------- ### Create TunerBatchNLoptr Instance Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_nloptr.html Demonstrates the creation of a new instance of the `TunerBatchNLoptr` class. This is a basic R6 class constructor call. ```R TunerBatchNLoptr$new() ``` -------------------------------- ### Get Selected Features Source: https://mlr3tuning.mlr-org.com/reference/AutoTuner.html Retrieves the character vector of features selected by the final tuned model. ```R AutoTuner$selected_features() ``` -------------------------------- ### Tuner Class Help Method Source: https://mlr3tuning.mlr-org.com/reference/Tuner.html Opens the help page associated with the tuner object, referenced by the `$man` field. This allows users to access detailed documentation. ```R Tuner$help() ``` -------------------------------- ### Get Out-of-Bag Error Source: https://mlr3tuning.mlr-org.com/reference/AutoTuner.html Returns the out-of-bag error as a single numeric value for the final tuned model. ```R AutoTuner$oob_error() ``` -------------------------------- ### ti() function for Tuning Instance Construction Source: https://mlr3tuning.mlr-org.com/reference/ti.html Constructs a `TuningInstanceBatchSingleCrit` or `TuningInstanceBatchMultiCrit` object, which defines the setup for hyperparameter optimization. ```APIDOC ## ti() ### Description Function to construct a TuningInstanceBatchSingleCrit or TuningInstanceBatchMultiCrit. ### Usage ```R ti( task, learner, resampling, measures = NULL, terminator, search_space = NULL, store_benchmark_result = TRUE, store_models = FALSE, check_values = FALSE, callbacks = NULL ) ``` ### Arguments * **task** (`mlr3::Task`): Task to operate on. * **learner** (`mlr3::Learner`): Learner to tune. * **resampling** (`mlr3::Resampling`): Resampling that is used to evaluate the performance of the hyperparameter configurations. Uninstantiated resamplings are instantiated during construction so that all configurations are evaluated on the same data splits. Already instantiated resamplings are kept unchanged. Specialized Tuner change the resampling e.g. to evaluate a hyperparameter configuration on different data splits. This field, however, always returns the resampling passed in construction. * **measures** (`mlr3::Measure` or list of `mlr3::Measure`): A single measure creates a `TuningInstanceBatchSingleCrit` and multiple measures a `TuningInstanceBatchMultiCrit`. If `NULL`, default measure is used. * **terminator** (`bbotk::Terminator`): Stop criterion of the tuning process. * **search_space** (`paradox::ParamSet`): Hyperparameter search space. If `NULL` (default), the search space is constructed from the `paradox::TuneToken` of the learner's parameter set (`learner$param_set`). When using `to_tune()` tokens, dependencies for hierarchical search spaces are automatically handled. * **store_benchmark_result** (`logical(1)`): If `TRUE` (default), store resample result of evaluated hyperparameter configurations in archive as `mlr3::BenchmarkResult`. * **store_models** (`logical(1)`): If `TRUE`, fitted models are stored in the benchmark result (`archive$benchmark_result`). If `store_benchmark_result = FALSE`, models are only stored temporarily and not accessible after the tuning. This combination is needed for measures that require a model. * **check_values** (`logical(1)`): If `TRUE`, hyperparameter values are checked before evaluation and performance scores after. If `FALSE` (default), values are unchecked but computational overhead is reduced. * **callbacks** (list of `mlr3misc::Callback`): List of callbacks. ### Examples ```R # Hyperparameter optimization on the Palmer Penguins data set task = tsk("penguins") # Load learner and set search space learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE) ) # Construct tuning instance instance = ti( task = task, learner = learner, resampling = rsmp("cv", folds = 3), measures = msr("classif.ce"), terminator = trm("evals", n_evals = 4) ) # Choose optimization algorithm tuner = tnr("random_search", batch_size = 2) # Run tuning tuner$optimize(instance) #> cp learner_param_vals x_domain classif.ce #> #> 1: -7.66021 0.06389525 # Set optimal hyperparameter configuration to learner learner$param_set$values = instance$result_learner_param_vals # Train the learner on the full data set learner$train(task) ``` ``` -------------------------------- ### Tuner Help Method Source: https://mlr3tuning.mlr-org.com/reference/Tuner.html Opens the help page associated with the Tuner object, referenced by its '$man' field. ```APIDOC ## Method `help()` Opens the corresponding help page referenced by field `$man`. ### Usage __``` Tuner$help() ``` ``` -------------------------------- ### tnr("random_search") Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners.html A sugar function to retrieve a tuner object by its ID. This is a convenient alternative to using `mlr_tuners$get()`. ```APIDOC ## tnr(id, ..., .key) ### Description A sugar function to retrieve a tuner object by its ID. This is a convenient alternative to using `mlr_tuners$get()`. ### Method Sugar function ### Parameters #### Path Parameters None #### Query Parameters - **id** (character(1)) - The unique identifier of the tuner to retrieve. - **...** - Additional arguments passed to the tuner constructor. - **.key** (character(1)) - Internal parameter, not typically used by users. ### Request Example ```R tnr("random_search") ``` ### Response #### Success Response (200) - **Tuner** - The requested tuner object. #### Response Example ```R #> #> ── : Random Search ───────────────────────────────────── #> • Parameters: batch_size=1 #> • Parameter classes: , , , and #> • Properties: dependencies, single-crit, and multi-crit #> • Packages: mlr3tuning and bbotk ``` ``` -------------------------------- ### Control Parameters Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_cmaes.html The CMA-ES tuner has a `start_values` control parameter that determines whether to create random start values or base them on the center of the search space. ```APIDOC ## Control Parameters `start_values` `character(1)` Create `random` start values or based on `center` of search space? In the latter case, it is the center of the parameters before a trafo is applied. For the meaning of the control parameters, see `adagio::pureCMAES()`. Note that we have removed all control parameters which refer to the termination of the algorithm and where our terminators allow to obtain the same behavior. ``` -------------------------------- ### Create TuningInstanceBatchMultiCrit Source: https://mlr3tuning.mlr-org.com/reference/TuningInstanceBatchMultiCrit.html Use the `TuningInstanceBatchMultiCrit$new()` constructor to create a new instance. It requires a task, learner, resampling strategy, measures, and a terminator. Optional arguments include search_space, store_benchmark_result, store_models, check_values, and callbacks. ```R TuningInstanceBatchMultiCrit$new( task, learner, resampling, measures, terminator, search_space = NULL, store_benchmark_result = TRUE, store_models = FALSE, check_values = FALSE, callbacks = NULL ) ``` -------------------------------- ### Instantiate TunerAsyncFromOptimizerAsync Source: https://mlr3tuning.mlr-org.com/reference/TunerAsyncFromOptimizerAsync.html Creates a new instance of the TunerAsyncFromOptimizerAsync class. Requires a bbotk::Optimizer object and optionally accepts a manual string. ```R TunerAsyncFromOptimizerAsync$new(optimizer, man = NA_character_) ``` -------------------------------- ### Retrieve Learner Parameter Values Source: https://mlr3tuning.mlr-org.com/reference/ArchiveAsyncTuning.html Use this method to get the parameter values for a specific evaluation by its index or unique hash. Ensure that 'i' and 'uhash' are not provided simultaneously. ```R ArchiveAsyncTuning$learner_param_vals(i = NULL, uhash = NULL) ``` -------------------------------- ### Extract Base Learner Source: https://mlr3tuning.mlr-org.com/reference/AutoTuner.html Retrieves the base learner from potentially nested learner objects, such as those used in mlr3pipelines. Set `recursive = 0` to get the tuned learner directly. ```R AutoTuner$base_learner(recursive = Inf) ``` -------------------------------- ### Get a specific tuner object Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners.html Retrieves a specific tuner object from the dictionary using its ID. This shows the details of the 'random_search' tuner, including its parameters, properties, and associated packages. ```R mlr_tuners$get("random_search") #> #> ── : Random Search ───────────────────────────────────── #> • Parameters: batch_size=1 #> • Parameter classes: , , , and #> • Properties: dependencies, single-crit, and multi-crit #> • Packages: mlr3tuning and bbotk ``` -------------------------------- ### as.data.table.ArchiveTuning Source: https://mlr3tuning.mlr-org.com/reference/ArchiveAsyncTuning.html Returns a tabular view of all evaluated hyperparameter configurations. ```APIDOC ## as.data.table.ArchiveTuning() ### Description Returns a tabular view of all evaluated hyperparameter configurations. ### Usage ``` as.data.table.ArchiveTuning(x, unnest = "x_domain", exclude_columns = "uhash", measures = NULL) ``` ### Arguments * `x` (ArchiveTuning): The archive object. * `unnest` (character()): Transforms list columns to separate columns. Set to `NULL` if no column should be unnested. * `exclude_columns` (character()): Exclude columns from table. Set to `NULL` if no column should be excluded. * `measures` (List of mlr3::Measure): Score hyperparameter configurations on additional measures. ``` -------------------------------- ### learner() Source: https://mlr3tuning.mlr-org.com/reference/ArchiveBatchTuning.html Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash `uhash`. `i` and `uhash` are mutually exclusive. Learner does not contain a model. Use `$learners()` to get learners with models. ```APIDOC ## Method learner() ### Description Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash `uhash`. `i` and `uhash` are mutually exclusive. Learner does not contain a model. Use `$learners()` to get learners with models. ### Usage ``` ArchiveBatchTuning$learner(i = NULL, uhash = NULL) ``` ### Arguments * `i` (integer(1)) - The iteration value to filter for. * `uhash` (logical(1)) - The `uhash` value to filter for. ``` -------------------------------- ### ArchiveAsyncTuningFrozen$learner() Source: https://mlr3tuning.mlr-org.com/reference/ArchiveAsyncTuningFrozen.html Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash `uhash`. `i` and `uhash` are mutually exclusive. Learner does not contain a model. Use `$learners()` to get learners with models. ```APIDOC ## Method `learner()` Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash `uhash`. `i` and `uhash` are mutually exclusive. Learner does not contain a model. Use `$learners()` to get learners with models. ### Usage __``` ArchiveAsyncTuningFrozen$learner(i = NULL, uhash = NULL) ``` ### Arguments `i` (`integer(1)`) The iteration value to filter for. `uhash` (`logical(1)`) The `uhash` value to filter for. ``` -------------------------------- ### Create and Configure CallbackBatchTuning Source: https://mlr3tuning.mlr-org.com/reference/CallbackBatchTuning.html Creates a CallbackBatchTuning instance named 'mlr3tuning.backup'. This callback is configured to save the tuning archive to a file named 'archive.rds' after the optimization process ends. ```R callback_batch_tuning("mlr3tuning.backup", on_optimization_end = function(callback, context) { saveRDS(context$instance$archive, "archive.rds") } ) #> #> #* Active Stages: on_optimization_end ``` -------------------------------- ### TunerBatchIrace$new() Source: https://mlr3tuning.mlr-org.com/reference/mlr_tuners_irace.html Creates a new instance of the TunerBatchIrace class. ```APIDOC ## Method `new()` ### Description Creates a new instance of this R6 class. ### Usage ```r TunerBatchIrace$new() ``` ``` -------------------------------- ### Tuning Instance Construction Source: https://mlr3tuning.mlr-org.com/reference/index.html Syntactic sugar functions for constructing tuning instances. ```APIDOC ## ti() ### Description Syntactic sugar for Tuning Instance Construction. ### Function Signature ti(...) ### Parameters * ...: Arguments passed to the respective tuning instance constructor. ``` ```APIDOC ## ti_async() ### Description Syntactic sugar for Asynchronous Tuning Instance Construction. ### Function Signature ti_async(...) ### Parameters * ...: Arguments passed to the respective asynchronous tuning instance constructor. ```