change learning rate trainlm

How to change learning rate in trainlm? matlab allows changing mu and Gradient

tnx

[Phuong Huynh]

I also wonder about this problem. I think trainlm function (optimizer) auto choose LR for us (http://matlab.izmiran.ru/help/toolbox/nnet/trainlm.html). Because other function allows us to change LR (for example :traingdx) when we change parameter net.trainParam.lr

Answer 1

[Krishna]

Hello,

From what I understand you want to change the learning rate of ‘trainlm’ optimiser. If you go through the documentation of ‘trainlm’ function you can see that there is no way to set a learning rate of this optimiser, but you can control the learning dynamics via mu, the adaptive parameter that adjusts the learning process in Levenberg-Marquardt as it acts similarly to the learning rate. A smaller mu allows smaller steps during learning, which can lead to finer adjustments but slower convergence. A larger mu takes bigger steps, which may speed up convergence but risks overshooting.

Set the hyper-paramter ‘net.trainParam.mu_dec’ to specify how much to decrease ‘mu’ when the error decreases after an iteration. Also set parameter ‘net.trainParam.mu_inc’ how much to increase ‘mu’ when the error increase after an iteration.

Please go through the following documentation to learn more regarding the hyper-parameters of Levenberg-Marquardt algorithm and also the mathematical update function being used in the following algorithm,

https://in.mathworks.com/help/deeplearning/ref/trainlm.html

Also please go through the following documentation to learn more about how to ask question on MATLAB answer and get a fast response,

https://in.mathworks.com/matlabcentral/answers/6200-tutorial-how-to-ask-a-question-on-answers-and-get-a-fast-answer

Hope this helps.