MATLAB Answers

Auto differentiation vs finite differences in optimization toolbox

2 views (last 30 days)
acelaya0
acelaya0 on 9 Jul 2021
Answered: Alan Weiss on 11 Jul 2021
Is there a situation where finite differences is faster than automatic differentiation when using the "solve" function call in the optimization toolbox?
I'm using the optimization toolbox to solve an optimization problem with a complex loss funcation and relatively few optimization variables. I'm noticing a substantial speed up when changing the value of "ObjectiveDerivative" from "auto" to "finite-differences."
Any clarification would be greatly appreciated!

Answers (1)

Alan Weiss
Alan Weiss on 11 Jul 2021
Yes, finite differences can be faster than AD. Typically, this occurs in situations like yours where the function or functions are complicated , and the resulting AD expressions are even more complex.
That said, sometimes you can help the solver by setting up your problem in a way that enables solve to operate efficiently. See Create Efficient Optimization Problems and, to a lesser extent for your problem, Separate Optimization Model from Data.
Alan Weiss
MATLAB mathematical toolbox documentation

Products


Release

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by