MATLAB Answers

Change the range of potential hyperparameters when optimizing hyperparameter choice using fitrensemble

6 views (last 30 days)
Adam Richardson
Adam Richardson on 20 May 2020
Commented: Alan Weiss on 21 May 2020
I am experimenting with randon search and grid search optimizers when optimizing hyperparameters for a boosted trees regression model, using the fitrensemble function.
To speed up the process, and also avoid overfitting, I want to specify non-default ranges to explore for the hyperparamters - however after reading documentation, I still don't understand how to do this.
For example, in the code below:
Mdl_ls = fitrensemble(X,Y,'Learners','tree', ...
'Optimizer','randomsearch', 'MaxObjectiveEvaluations', 500, 'ShowPlots', false, 'Verbose', 1 ));
How would I go about changing the search range for the LearnRate to between 0.001 and 0.2 - rather than the defaul range of [1e-3,1].


Sign in to comment.

Accepted Answer

Alan Weiss
Alan Weiss on 20 May 2020
Looking in the documentation for fitrensemble I find this:
Set nondefault parameters by passing a vector of optimizableVariable objects that have nondefault values. For example,
load carsmall
params = hyperparameters('fitrensemble',[Horsepower,Weight],MPG,'Tree');
params(4).Range = [1,20];
Pass params as the value of OptimizeHyperparameters.
Is that clear to you? For more information, see
Alan Weiss
MATLAB mathematical toolbox documentation


Adam Richardson
Adam Richardson on 21 May 2020
Therefore, for my question - the approach is (setting range for three hyperparameters and optimizing these):
params = hyperparameters('fitrensemble',X,Y,'Tree');
params(2).Range = [500, 1000] %NumLearningCycles
params(3).Range = [0.001, 0.2] %LearnRate
params(5).Range = [5, 15] %MaxNumSplits
params(1).Optimize = false;
params(2).Optimize = true;
params(3).Optimize = true;
params(4).Optimize = false;
params(5).Optimize = true;
params(6).Optimize = false;
Mdl_ls = fitrensemble(X,Y,'Learners','tree', ...
'Optimizer','randomsearch', 'MaxObjectiveEvaluations', 500, 'ShowPlots', false, 'Verbose', 1, 'Holdout', 2/3 ));
Alan Weiss
Alan Weiss on 21 May 2020
Good job! Thanks for the follow-up showing your solution.
Alan Weiss
MATLAB mathematical toolbox documentation

Sign in to comment.

More Answers (0)

Translated by