AggregateBayesianOptimization

Aggregate Bayesian optimization results

Since R2024b

Description

An AggregateBayesianOptimization object contains the aggregate results of multiple hyperparameter optimization problems that have the same optimization settings but different constraint bounds. The object is an optional output argument of any fitting function, such as fitcdiscr, that accepts the HyperparameterOptimizationOptions name-value argument.

Creation

Create an AggregateBayesianOptimization object by calling a fitting function in the list below with the OptimizeHyperparameters and HyperParameterOptimizationOptions name-value arguments and specifying to return hyperparameter optimization results. You must set ConstraintBounds and ConstraintType in the structure or HyperparameterOptimizationOptions object that you pass to the HyperParameterOptimizationOptions name-value argument of the fitting function.

• Classification fitting functions: fitcdiscr, fitcecoc, fitcensemble, fitcgam, fitckernel, fitcknn, fitclinear, fitcnb, fitcnet, fitcsvm, fitctree, fitcauto

• Regression fitting functions: fitrensemble, fitrgam, fitrgp, fitrkernel, fitrlinear, fitrnet, fitrsvm, fitrtree, fitrauto

For example, [Mdl,hpoResults]=fitcecoc(X,Y,OptimizeHyperparameters="all",HyperparameterOptimizationOptions=hpoObject) fits a multiclass ECOC model using the predictors X and the class labels Y, optimizes all eligible model hyperparameters using the options and constraint bounds in the HyperparameterOptimizationOptions object hpoObject, and returns the trained model Mdl and the AggregateBayesianOptimization object hpoResults.

Properties

Problem Definition Properties

Constraint — Constraint function

"kfoldLoss" | "CompactModelSize" | "LearnerForCoderSize"

This property is read-only.

Constraint function, specified as a character vector. The values of Constraint and Objectivedepends on the values of the ConstraintTarget and ConstraintType properties.

Value of ConstraintTarget	Value of ConstraintType	Value of Constraint	Value of Objective
"matlab"	"size"	"CompactModelSize"	"kfoldLoss"
"matlab"	"loss"	"kfoldLoss"	"CompactModelSize"
"coder"	"size"	"LearnerForCoderSize"	"kfoldLoss"
"coder"	"loss"	"kfoldLoss"	"LearnerForCoderSize"

Data Types: string

Objective — Objective function

"kfoldLoss" | "CompactModelSize" | "LearnerForCoderSize"

This property is read-only.

Objective function, specified as a character vector. The values of Objective and Constraint depend on the values of the ConstraintTarget and ConstraintType properties.

Value of ConstraintTarget	Value of ConstraintType	Value of Constraint	Value of Objective
"matlab"	"size"	"CompactModelSize"	"kfoldLoss"
"matlab"	"loss"	"kfoldLoss"	"CompactModelSize"
"coder"	"size"	"LearnerForCoderSize"	"kfoldLoss"
"coder"	"loss"	"kfoldLoss"	"LearnerForCoderSize"

Data Types: string

ConstraintBounds — Constraint bounds

N-by-2 numeric matrix

This property is read-only.

Constraint bounds, specified as an N-by-2 numeric matrix, where N is the number of optimization problems. The columns of ConstraintBounds contain the values of the lower and upper bounds on ConstraintType.

Data Types: single | double

ConstraintTarget — Constraint target

"matlab" | "coder"

This property is read-only.

Constraint target for the optimization problems, specified as "matlab" or "coder".

Data Types: char | string

ConstraintType — Constraint type

"size" | "loss"

This property is read-only.

Constraint type for the optimization problems, specified as "size" or "loss".

Data Types: char | string

Solution Properties

MinObjective — Minimum observed value of objective function

N-by-1 numeric vector

This property is read-only.

Minimum observed value of the objective function Objective for each optimization problem, specified as an N-by-1 numeric vector, where N is the number of rows in ConstraintBounds.

Data Types: single | double

LearnerAtMinObjective — Attained model type

N-by-1 string array

This property is read-only.

Attained model type for each optimization problem, specified as an N-by-1 string array, where N is the number of rows in ConstraintBounds.

Data Types: string

ConstraintAtMinObjective — Attained constraint value

N-by-1 numeric vector

This property is read-only.

Attained constraint value for each optimization problem, specified as an N-by-1 numeric vector, where N is the number of rows in ConstraintBounds. Each row of ConstraintAtMinObjective contains the value of Constraint at the minimum observed value of the objective function (Objective), for the corresponding constraint bounds in ConstraintBounds.

Data Types: single | double

ConstraintBoundsAreSatisfied — Constraint bounds indicator

N-by-1 logical array

This property is read-only.

Constraint bounds indicator for each optimization problem, specified as an N-by-1 logical array, where N is the number of rows in ConstraintBounds. An element in ConstraintBoundsAreSatisfied has a value of 1 if the final model satisfies the corresponding constraint bounds in ConstraintBounds, and 0 otherwise.

Data Types: table

Feasible — Indicator of model feasibility

N-by-1 logical array

This property is read-only.

Indicator of model feasibility for each optimization problem, specified as an N-by-1 logical array, where N is the number of rows in ConstraintBounds. An element in Feasible has a value of 1 if the final model is feasible (satisfies constraints), and 0 if it is infeasible. If you create an AggregateBayesianOptimization object by calling a fit function with the hyperparameter optimization option Optimizer="bayesopt", then an optimization problem can have a ConstraintBoundsAreSatisfied value of 1 and a Feasible value of 0. This result can occur if the final attained point satisfies the constraint bounds in ConstraintBounds but lies outside the optimization confidence bounds (see bestPoint).

Data Types: logical

VariableDescriptions — Hyperparameters used in optimization problem

N-by-1 cell array of optimizableVariable objects

This property is read-only.

Hyperparameters used in each optimization problem, specified as an N-by-1 cell array of optimizableVariable objects. If you create an AggregateBayesianOptimization object using fitcauto or fitrauto, then the size of each optimizableVariable object is 1-by-K, where K depends on the model type and hyperparameter optimization options. Otherwise, the size of each optimizableVariable object is K-by-1.

Data Types: cell array

HyperparameterOptimizationResults — Hyperparameter optimization results

N-by-1 cell array of BayesianOptimization objects | N-by-1 cell array of tables

This property is read-only.

Hyperparameter optimization results for each optimization problem, specified as an N-by-1 cell array, where N is the number of rows in ConstraintBounds. If the structure or HyperParameterOptimizationOptions object that you pass to a fitting function has the specification Optimizer="bayesopt" (the default), then the elements of HyperParameterOptimizationResults are BayesianOptimization objects. Otherwise, the elements are tables of hyperparameters.

If you create an AggregateBayesianOptimization object by calling a fit function with the hyperparameter optimization options SaveIntermediateResults=true and a ConstraintBounds value that is not [], then an additional iteration is stored in the BayesianOptimization objects (or tables). During this iteration, which is not shown in the command-line output of the fit function, the software evaluates the constraint and objective functions, but does not perform any optimization or model training.

Data Types: cell array

BayesianOptimizationResults — Optimization results

N-by-1 cell array of BayesianOptimization objects

This property is read-only.

Optimization results for each problem, specified as an N-by-1 cell array of BayesianOptimization objects, where N is the number of rows in ConstraintBounds. If the HyperParameterOptimizationOptions object that you pass to a fitting function has the specification Optimizer="bayesopt" (the default), then the contents of BayesianOptimizationResults and HyperparameterOptimizationResults are the same.

If you create an AggregateBayesianOptimization object by calling a fit function with the hyperparameter optimization options SaveIntermediateResults=true and a ConstraintBounds value that is not [], then an additional iteration is stored in the BayesianOptimization objects. During this iteration (which is not displayed in the command-line output of the fit function), the software evaluates the constraint and objective functions, but does not perform any optimization or model training.

Data Types: cell array

Object Functions

plot	Plot aggregated hyperparameter optimization results
resume	Resume hyperparameter optimization problems
summary	Summary table for AggregateBayesianOptimization object

Examples

Hyperparameter Optimization with Multiple Constraint Bounds

Repeat the hyperparameter optimization of a trained classification tree model several times using the same optimization settings, but with different constraint bounds each time.

Load the ionosphere data set.

load ionosphere.mat

Create a HyperparameterOptimizationOptions object that contains the settings for three hyperparameter optimization problems. For each problem, specify to use the compact size of the trained model as the constraint, and use the default settings for the other optimization options. Specify the constraint bounds as 0 to 5000 bytes for the first problem, 0 to 20000 bytes for the second problem, and 0 to 50000 bytes for the third problem.

rng(0,"twister"); % For reproducibility
hpoOptions = hyperparameterOptimizationOptions(ConstraintType="size", ...
    ConstraintBounds=[5000; 20000; 50000])

hpoOptions = 
  HyperparameterOptimizationOptions with properties:

                  Objective: "kfoldLoss"
                 Constraint: "CompactModelSize"
    AcquisitionFunctionName: "expected-improvement-per-second-plus"
           ConstraintBounds: [3x2 double]
           ConstraintTarget: "matlab"
             ConstraintType: "size"
                      KFold: 5
                    MaxTime: Inf
                  Optimizer: "bayesopt"
                Repartition: 0
    SaveIntermediateResults: 0
                  ShowPlots: 1
                UseParallel: 0
                    Verbose: 1

hpoOptions is a HyperparameterOptimizationOptions object that contains hyperparameter optimization options for the classification tree fitting function.

Modify the default property values in hpoOptions to suppress the display of plots and command-line output.

hpoOptions.ShowPlots=false;
hpoOptions.Verbose=0;

Train a classification tree model using the fitctree function, and optimize the model hyperparameters for each optimization problem subject to the constraints in hpoOptions. Additionally return the results object hpoResults.

[Mdl,hpoResults]=fitctree(X,Y,OptimizeHyperparameters="auto", ...
    HyperparameterOptimizationOptions=hpoOptions)

Mdl=3×1 cell array
    {1x1 ClassificationTree}
    {1x1 ClassificationTree}
    {1x1 ClassificationTree}

hpoResults = 
  AggregateBayesianOptimization with properties:

                            Objective: "kfoldLoss"
                           Constraint: "CompactModelSize"
          BayesianOptimizationResults: {3x1 cell}
             ConstraintAtMinObjective: [3x1 double]
                     ConstraintBounds: [3x2 double]
         ConstraintBoundsAreSatisfied: [3x1 logical]
                     ConstraintTarget: "matlab"
                       ConstraintType: "size"
                             Feasible: [3x1 logical]
    HyperparameterOptimizationResults: {3x1 cell}
                LearnerAtMinObjective: [3x1 string]
                         MinObjective: [3x1 double]
                 VariableDescriptions: {3x1 cell}

Mdl is a cell array that contains the trained classification model object for each optimization problem. hpoResults is an AggregateBayesianOptimization object that contains the options and results for each optimization problem.

Version History

Introduced in R2024b