How to make convergence criteria for Levenberg-Marquardt algorithm

How to make convergence criteria for Levenberg-Marquardt algorithm, please give practical hint for matlab implementatio

Answer 1

[Krishna]

Hello,

From the question I understand you want to know the convergence criteria of Levenberg-Marquardt algorithm.

• The first criteria is if when the maximum number of epochs is reached then the training is stopped irrespective if the convergence is achieved or not.
• The second criteria is if the parameters changes are below a threshold then convergence is achieved.
• The third criteria is minimum gradient value is reached. You can adjust this by changing the hyper-parameter ‘net.trainlm.min_grad.’
• The fourth criteria is if the performance goal is reached (training loss value). You can adjust this by changing the hyper-parameter ‘net.trainlm.goal.’

These are the 4 criterions by which you can check the convergence of trainlm algorithm.

Please go through the following documentation to learn more regarding the hyper-parameters of Levenberg-Marquardt algorithm and also the mathematical update function being used in the following algorithm,

https://in.mathworks.com/help/deeplearning/ref/trainlm.html

Also please go through the following documentation to learn more about how to ask question on MATLAB answer and get a fast response,

https://in.mathworks.com/matlabcentral/answers/6200-tutorial-how-to-ask-a-question-on-answers-and-get-a-fast-answer

Hope this helps.

[Walter Roberson]

If the maximum number of epochs is reached, then you do not have convergence.

[Krishna]

Yes, have corrected the answer, if the maximum epochs is reached then the training is stopped irrespective if the convergence is achieved or not.