MATLAB Answers

Change the range of potential hyperparameters when optimizing hyperparameter choice using fitrensemble

6 views (last 30 days)
Adam Richardson
Adam Richardson on 20 May 2020
Commented: Alan Weiss on 21 May 2020
Hi,
I am experimenting with randon search and grid search optimizers when optimizing hyperparameters for a boosted trees regression model, using the fitrensemble function.
To speed up the process, and also avoid overfitting, I want to specify non-default ranges to explore for the hyperparamters - however after reading documentation, I still don't understand how to do this.
For example, in the code below:
Mdl_ls = fitrensemble(X,Y,'Learners','tree', ...
'OptimizeHyperparameters',{'NumLearningCycles','LearnRate','MaxNumSplits'},...
'HyperparameterOptimizationOptions',struct(...
'Optimizer','randomsearch', 'MaxObjectiveEvaluations', 500, 'ShowPlots', false, 'Verbose', 1 ));
How would I go about changing the search range for the LearnRate to between 0.001 and 0.2 - rather than the defaul range of [1e-3,1].

  0 Comments

Sign in to comment.

Accepted Answer

Alan Weiss
Alan Weiss on 20 May 2020
Looking in the documentation for fitrensemble I find this:
----
Set nondefault parameters by passing a vector of optimizableVariable objects that have nondefault values. For example,
load carsmall
params = hyperparameters('fitrensemble',[Horsepower,Weight],MPG,'Tree');
params(4).Range = [1,20];
Pass params as the value of OptimizeHyperparameters.
----
Is that clear to you? For more information, see https://www.mathworks.com/help/stats/hyperparameters.html#bvdtepr-1
Alan Weiss
MATLAB mathematical toolbox documentation

  2 Comments

Adam Richardson
Adam Richardson on 21 May 2020
Thanks!
Therefore, for my question - the approach is (setting range for three hyperparameters and optimizing these):
params = hyperparameters('fitrensemble',X,Y,'Tree');
params(2).Range = [500, 1000] %NumLearningCycles
params(3).Range = [0.001, 0.2] %LearnRate
params(5).Range = [5, 15] %MaxNumSplits
params(1).Optimize = false;
params(2).Optimize = true;
params(3).Optimize = true;
params(4).Optimize = false;
params(5).Optimize = true;
params(6).Optimize = false;
Mdl_ls = fitrensemble(X,Y,'Learners','tree', ...
'OptimizeHyperparameters',params,...
'HyperparameterOptimizationOptions',struct(...
'Optimizer','randomsearch', 'MaxObjectiveEvaluations', 500, 'ShowPlots', false, 'Verbose', 1, 'Holdout', 2/3 ));
Alan Weiss
Alan Weiss on 21 May 2020
Good job! Thanks for the follow-up showing your solution.
Alan Weiss
MATLAB mathematical toolbox documentation

Sign in to comment.

More Answers (0)


Translated by