fminunc

Find minimum of unconstrained multivariable function

Syntax

x = fminunc(fun,x0)

x = fminunc(fun,x0,options)

x = fminunc(problem)

[x,fval] = fminunc(___)

[x,fval,exitflag,output] = fminunc(___)

[x,fval,exitflag,output,grad,hessian] = fminunc(___)

Description

Nonlinear programming solver.

Finds the minimum of a problem specified by

$$\underset{x}{\min }f(x)$$

where f(x) is a function that returns a scalar.

x is a vector or a matrix; see Matrix Arguments.

x = fminunc(fun,x0) starts at the point x0 and attempts to find a local minimum x of the function described in fun. The point x0 can be a scalar, vector, or matrix.

Note

Passing Extra Parameters explains how to pass extra parameters to the objective function and nonlinear constraint functions, if necessary.

fminunc is for nonlinear problems without constraints. If your problem has constraints, generally use fmincon. See Optimization Decision Table.

x = fminunc(fun,x0,options) minimizes fun with the optimization options specified in options. Use optimoptions to set these options.

x = fminunc(problem) finds the minimum for problem, a structure described in problem.

[x,fval] = fminunc(___), for any syntax, returns the value of the objective function fun at the solution x.

[x,fval,exitflag,output] = fminunc(___) additionally returns a value exitflag that describes the exit condition of fminunc, and a structure output with information about the optimization process.

[x,fval,exitflag,output,grad,hessian] = fminunc(___) additionally returns:

• grad — Gradient of fun at the solution x.

• hessian — Hessian of fun at the solution x. See fminunc Hessian.

Examples

Minimize a Polynomial

Minimize the function $$f(x)=3x_1^2+2x_1{x}_2+x_2^2-4x_1+5x_2$$.

To do so, write an anonymous function fun that calculates the objective.

fun = @(x)3*x(1)^2 + 2*x(1)*x(2) + x(2)^2 - 4*x(1) + 5*x(2);

Call fminunc to find a minimum of fun near [1,1].

x0 = [1,1];
[x,fval] = fminunc(fun,x0)

Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.

x = 1×2

    2.2500   -4.7500

fval = 
-16.3750

Supply Gradient

fminunc can be faster and more reliable when you provide derivatives.

Write an objective function that returns the gradient as well as the function value. Use the conditionalized form described in Including Gradients and Hessians. The objective function is Rosenbrock's function,

$$f(x)=100{\left(x_2-x_1^2\right)}^2+(1-x_1{)}^2,$$

which has gradient

$$\nabla f(x)=\left[\begin{array}{c}-400\left(x_2-x_1^2\right)x_1-2\left(1-x_1\right)\\ 200\left(x_2-x_1^2\right)\end{array}\right]$$.

The code for the objective function with gradient appears at the end of this example.

Create options to use the objective function’s gradient. Also, set the algorithm to 'trust-region'.

options = optimoptions('fminunc','Algorithm','trust-region','SpecifyObjectiveGradient',true);

Set the initial point to [-1,2]. Then call fminunc.

x0 = [-1,2];
fun = @rosenbrockwithgrad;
x = fminunc(fun,x0,options)

Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.

x = 1×2

    1.0000    1.0000

The following code creates the rosenbrockwithgrad function, which includes the gradient as the second output.

function [f,g] = rosenbrockwithgrad(x)
% Calculate objective f
f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2;

if nargout > 1 % gradient required
    g = [-400*(x(2)-x(1)^2)*x(1) - 2*(1-x(1));
        200*(x(2)-x(1)^2)];
end
end

Use Problem Structure

Solve the same problem as in Supply Gradient using a problem structure instead of separate arguments.

Write an objective function that returns the gradient as well as the function value. Use the conditionalized form described in Including Gradients and Hessians. The objective function is Rosenbrock's function,

$$f(x)=100{\left(x_2-x_1^2\right)}^2+(1-x_1{)}^2$$,

which has gradient

$$\nabla f(x)=\left[\begin{array}{c}-400\left(x_2-x_1^2\right)x_1-2\left(1-x_1\right)\\ 200\left(x_2-x_1^2\right)\end{array}\right]$$.

The code for the objective function with gradient appears at the end of this example.

Create options to use the objective function’s gradient. Also, set the algorithm to 'trust-region'.

options = optimoptions('fminunc','Algorithm','trust-region','SpecifyObjectiveGradient',true);

Create a problem structure including the initial point x0 = [-1,2]. For the required fields in this structure, see problem.

problem.options = options;
problem.x0 = [-1,2];
problem.objective = @rosenbrockwithgrad;
problem.solver = 'fminunc';

Solve the problem.

x = fminunc(problem)

Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.

x = 1×2

    1.0000    1.0000

The following code creates the rosenbrockwithgrad function, which includes the gradient as the second output.

function [f,g] = rosenbrockwithgrad(x)
% Calculate objective f
f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2;

if nargout > 1 % gradient required
    g = [-400*(x(2)-x(1)^2)*x(1)-2*(1-x(1));
        200*(x(2)-x(1)^2)];
end
end

Obtain Optimal Objective Function Value

Find both the location of the minimum of a nonlinear function and the value of the function at that minimum. The objective function is

$$f(x)=x(1)e^{-{\Vert x\Vert }_2^2}+{\Vert x\Vert }_2^2/20$$.

fun = @(x)x(1)*exp(-(x(1)^2 + x(2)^2)) + (x(1)^2 + x(2)^2)/20;

Find the location and objective function value of the minimizer starting at x0 = [1,2].

x0 = [1,2];
[x,fval] = fminunc(fun,x0)

Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.

x = 1×2

   -0.6691    0.0000

fval = 
-0.4052

Examine the Solution Process

Choose fminunc options and outputs to examine the solution process.

Set options to obtain iterative display and use the 'quasi-newton' algorithm.

options = optimoptions(@fminunc,'Display','iter','Algorithm','quasi-newton');

The objective function is

$$f(x)=x(1)e^{-{\Vert x\Vert }_2^2}+{\Vert x\Vert }_2^2/20.$$

fun = @(x)x(1)*exp(-(x(1)^2 + x(2)^2)) + (x(1)^2 + x(2)^2)/20;

Start the minimization at x0 = [1,2], and obtain outputs that enable you to examine the solution quality and process.

x0 = [1,2];
[x,fval,exitflag,output] = fminunc(fun,x0,options)

                                                        First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           3         0.256738                         0.173
     1           6         0.222149              1          0.131  
     2           9          0.15717              1          0.158  
     3          18        -0.227902       0.438133          0.386  
     4          21        -0.299271              1           0.46  
     5          30        -0.404028       0.102071         0.0458  
     6          33        -0.404868              1         0.0296  
     7          36        -0.405236              1        0.00119  
     8          39        -0.405237              1       0.000252  
     9          42        -0.405237              1       7.97e-07  

Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.

x = 1×2

   -0.6691    0.0000

fval = 
-0.4052

exitflag = 
1

output = struct with fields:
       iterations: 9
        funcCount: 42
         stepsize: 2.9343e-04
     lssteplength: 1
    firstorderopt: 7.9721e-07
        algorithm: 'quasi-newton'
          message: 'Local minimum found....'

• The exit flag 1 shows that the solution is a local optimum.

• The output structure shows the number of iterations, number of function evaluations, and other information.

• The iterative display also shows the number of iterations and function evaluations.

Use "lbfgs" Hessian Approximation for Large Problem

When your problem has a large number of variables, the default value of the HessianApproximation can cause fminunc to use a large amount of memory and run slowly. To use less memory and run faster, specify HessianApproximation="lbfgs".

For example, if you attempt to minimize the multirosenbrock function (listed below) with 1e5 variables using the default parameters, fminunc issues an error.

N = 1e5;
x0 = -2*ones(N,1);
x0(2:2:N) = 2;
[x,fval] = fminunc(@multirosenbrock,x0)

Error using eye
Requested 100000x100000 (74.5GB) array exceeds maximum array size preference (63.9GB). This might cause MATLAB to become
unresponsive.

Error in optim.internal.fminunc.AbstractDenseHessianApproximation (line 21)
        this.Value = eye(nVars);

Error in optim.internal.fminunc.BFGSHessianApproximation (line 14)
        this = [e-mail]oximation(nVars);

Error in fminusub (line 73)
    HessApprox = optim.internal.fminunc.BFGSHessianApproximation(sizes.nVar);

Error in fminunc (line 488)
   [x,FVAL,GRAD,HESSIAN,EXITFLAG,OUTPUT] = fminusub(funfcn,x, ...

To solve this problem, set the HessianApproximation option to "lbfgs". To speed the solution, set options to use the supplied gradient.

N = 1e5;
x0 = -2*ones(N,1);
x0(2:2:N) = 2;
options = optimoptions("fminunc",HessianApproximation="lbfgs",...
    SpecifyObjectiveGradient=true);
[x,fval] = fminunc(@multirosenbrock,x0,options);

Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.

The theoretical solution is x(i) = 1 for all i. Check the accuracy of the returned solution.

max(abs(x-1))

ans =

   1.3795e-04

This code creates the multirosenbrock function.

function [f,g] = multirosenbrock(x)
% Get the problem size
n = length(x);  
if n == 0, error('Input vector, x, is empty.'); end
if mod(n,2) ~= 0
   error('Input vector, x ,must have an even number of components.');
end
% Evaluate the vector function
odds  = 1:2:n;
evens = 2:2:n;
F = zeros(n,1);
F(odds,1)  = 1-x(odds);
F(evens,1) = 10.*(x(evens)-x(odds).^2); 
f = sum(F.^2);
if nargout >= 2 % Calculate gradient
    g = zeros(n,1);
    g(evens) = 200*(x(evens)-x(odds).^2);
    g(odds) = -2*(1 - x(odds)) - 400*(x(evens)-x(odds).^2).*x(odds);
end
end

Input Arguments

fun — Function to minimize

function handle | function name

Function to minimize, specified as a function handle or function name. fun is a function that accepts a vector or array x and returns a real scalar f, the objective function evaluated at x.

fminunc passes x to your objective function in the shape of the x0 argument. For example, if x0 is a 5-by-3 array, then fminunc passes x to fun as a 5-by-3 array.

Specify fun as a function handle for a file:

x = fminunc(@myfun,x0)

where myfun is a MATLAB^® function such as

function f = myfun(x)
f = ...            % Compute function value at x

You can also specify fun as a function handle for an anonymous function:

x = fminunc(@(x)norm(x)^2,x0);

If you can compute the gradient of fun and the SpecifyObjectiveGradient option is set to true, as set by

options = optimoptions('fminunc','SpecifyObjectiveGradient',true)

then fun must return the gradient vector g(x) in the second output argument.

If you can also compute the Hessian matrix and the HessianFcn option is set to 'objective' via options = optimoptions('fminunc','HessianFcn','objective') and the Algorithm option is set to 'trust-region', fun must return the Hessian value H(x), a symmetric matrix, in a third output argument. fun can give a sparse Hessian. See Hessian for fminunc trust-region or fmincon trust-region-reflective algorithms for details.

The trust-region algorithm allows you to supply a Hessian multiply function. This function gives the result of a Hessian-times-vector product without computing the Hessian directly. This can save memory. See Hessian Multiply Function.

Example: fun = @(x)sin(x(1))*cos(x(2))

Data Types: char | function_handle | string

x0 — Initial point

real vector | real array

Initial point, specified as a real vector or real array. Solvers use the number of elements in x0 and the size of x0 to determine the number and size of variables that fun accepts.

Example: x0 = [1,2,3,4]

Data Types: double

options — Optimization options

output of optimoptions | structure such as optimset returns

Optimization options, specified as the output of optimoptions or a structure such as optimset returns.

Some options apply to all algorithms, and others are relevant for particular algorithms. See Optimization Options Reference for detailed information.

Some options are absent from the optimoptions display. These options appear in italics in the following table. For details, see View Optimization Options.

All Algorithms
Algorithm	Choose the fminunc algorithm. Choices are 'quasi-newton' (default) or 'trust-region'. The 'trust-region' algorithm requires you to provide the gradient (see the description of fun), or else fminunc uses the 'quasi-newton' algorithm. For information on choosing the algorithm, see Choosing the Algorithm.
CheckGradients	Compare user-supplied derivatives (gradient of objective) to finite-differencing derivatives. Choices are false (default) or true. For optimset, the name is DerivativeCheck and the values are 'on' or 'off'. See Current and Legacy Option Names. The CheckGradients option will be removed in a future release. To check derivatives, use the checkGradients function.
Diagnostics	Display diagnostic information about the function to be minimized or solved. Choices are 'off' (default) or 'on'.
DiffMaxChange	Maximum change in variables for finite-difference gradients (a positive scalar). The default is Inf.
DiffMinChange	Minimum change in variables for finite-difference gradients (a positive scalar). The default is 0.
Display	Level of display (see Iterative Display): 'off' or 'none' displays no output. 'iter' displays output at each iteration, and gives the default exit message. 'iter-detailed' displays output at each iteration, and gives the technical exit message. 'notify' displays output only if the function does not converge, and gives the default exit message. 'notify-detailed' displays output only if the function does not converge, and gives the technical exit message. 'final' (default) displays only the final output, and gives the default exit message. 'final-detailed' displays only the final output, and gives the technical exit message.
FiniteDifferenceStepSize	Scalar or vector step size factor for finite differences. When you set FiniteDifferenceStepSize to a vector v, the forward finite differences delta are delta = v.*sign′(x).*max(abs(x),TypicalX); where sign′(x) = sign(x) except sign′(0) = 1. Central finite differences are delta = v.*max(abs(x),TypicalX); A scalar FiniteDifferenceStepSize expands to a vector. The default is sqrt(eps) for forward finite differences, and eps^(1/3) for central finite differences. The trust-region algorithm uses FiniteDifferenceStepSize only when CheckGradients is set to true. For optimset, the name is FinDiffRelStep. See Current and Legacy Option Names.
FiniteDifferenceType	Finite differences, used to estimate gradients, are either 'forward' (the default), or 'central' (centered). 'central' takes twice as many function evaluations, but should be more accurate. The trust-region algorithm uses FiniteDifferenceType only when CheckGradients is set to true. For optimset, the name is FinDiffType. See Current and Legacy Option Names.
FunValCheck	Check whether objective function values are valid. The default setting, 'off', does not perform a check. The 'on' setting displays an error when the objective function returns a value that is complex, Inf, or NaN.
MaxFunctionEvaluations	Maximum number of function evaluations allowed, a nonnegative integer. The default value is 100*numberOfVariables. See Tolerances and Stopping Criteria and Iterations and Function Counts. For optimset, the name is MaxFunEvals. See Current and Legacy Option Names.
MaxIterations	Maximum number of iterations allowed, a nonnegative integer. The default value is 400. See Tolerances and Stopping Criteria and Iterations and Function Counts. For optimset, the name is MaxIter. See Current and Legacy Option Names.
OptimalityTolerance	Termination tolerance on the first-order optimality (a nonnegative scalar). The default is 1e-6. See First-Order Optimality Measure. For optimset, the name is TolFun. See Current and Legacy Option Names.
OutputFcn	Specify one or more user-defined functions that an optimization function calls at each iteration. Pass a function handle or a cell array of function handles. The default is none ([]). See Output Function and Plot Function Syntax.
PlotFcn	Plots various measures of progress while the algorithm executes; select from predefined plots or write your own. Pass a built-in plot function name, a function handle, or a cell array of built-in plot function names or function handles. For custom plot functions, pass function handles. The default is none ([]): 'optimplotx' plots the current point. 'optimplotfunccount' plots the function count. 'optimplotfval' plots the function value. 'optimplotstepsize' plots the step size. 'optimplotfirstorderopt' plots the first-order optimality measure. Custom plot functions use the same syntax as output functions. See Output Functions for Optimization Toolbox and Output Function and Plot Function Syntax. For optimset, the name is PlotFcns. See Current and Legacy Option Names.
SpecifyObjectiveGradient	Gradient for the objective function defined by the user. See the description of fun to see how to define the gradient in fun. Set to true to have fminunc use a user-defined gradient of the objective function. The default false causes fminunc to estimate gradients using finite differences. You must provide the gradient, and set SpecifyObjectiveGradient to true, to use the trust-region algorithm. This option is not required for the quasi-Newton algorithm. For optimset, the name is GradObj and the values are 'on' or 'off'. See Current and Legacy Option Names.
StepTolerance	Termination tolerance on x, a nonnegative scalar. The default value is 1e-6. See Tolerances and Stopping Criteria. For optimset, the name is TolX. See Current and Legacy Option Names.
TypicalX	Typical x values. The number of elements in TypicalX is equal to the number of elements in x0, the starting point. The default value is ones(numberofvariables,1). fminunc uses TypicalX for scaling finite differences for gradient estimation. The trust-region algorithm uses TypicalX only for the CheckGradients option.
trust-region Algorithm
FunctionTolerance	Termination tolerance on the function value, a nonnegative scalar. The default is 1e-6. See Tolerances and Stopping Criteria. For optimset, the name is TolFun. See Current and Legacy Option Names.
HessianFcn	If set to [] (default), fminunc approximates the Hessian using finite differences. If set to 'objective', fminunc uses a user-defined Hessian for the objective function. The Hessian is the third output of the objective function (see fun). For optimset, the name is HessFcn. See Current and Legacy Option Names.
HessianMultiplyFcn	Hessian multiply function, specified as a function handle. For large-scale structured problems, this function computes the Hessian matrix product H*Y without actually forming H. The function is of the form W = hmfun(Hinfo,Y) where Hinfo contains the matrix used to compute H*Y. The first argument is the same as the third argument returned by the objective function fun, for example [f,g,Hinfo] = fun(x) Y is a matrix that has the same number of rows as there are dimensions in the problem. The matrix W = H*Y, although H is not formed explicitly. fminunc uses Hinfo to compute the preconditioner. For information on how to supply values for any additional parameters hmfun needs, see Passing Extra Parameters. Note To use the HessianMultiplyFcn option, HessianFcn must be set to []. For an example, see Minimization with Dense Structured Hessian, Linear Equalities. For optimset, the name is HessMult. See Current and Legacy Option Names.
HessPattern	Sparsity pattern of the Hessian for finite differencing. Set HessPattern(i,j) = 1 when you can have ∂2fun/∂x(i)∂x(j) ≠ 0. Otherwise, set HessPattern(i,j) = 0. Use HessPattern when it is inconvenient to compute the Hessian matrix H in fun, but you can determine (say, by inspection) when the ith component of the gradient of fun depends on x(j). fminunc can approximate H via sparse finite differences (of the gradient) if you provide the sparsity structure of H as the value for HessPattern. In other words, provide the locations of the nonzeros. When the structure is unknown, do not set HessPattern. The default behavior is as if HessPattern is a dense matrix of ones. Then fminunc computes a full finite-difference approximation in each iteration. This computation can be expensive for large problems, so it is usually better to determine the sparsity structure.
MaxPCGIter	Maximum number of preconditioned conjugate gradient (PCG) iterations, a positive scalar. The default is max(1,floor(numberOfVariables/2)). For more information, see Trust Region Algorithm.
PrecondBandWidth	Upper bandwidth of preconditioner for PCG, a nonnegative integer. By default, fminunc uses diagonal preconditioning (upper bandwidth of 0). For some problems, increasing the bandwidth reduces the number of PCG iterations. Setting PrecondBandWidth to Inf uses a direct factorization (Cholesky) rather than the conjugate gradients (CG). The direct factorization is computationally more expensive than CG, but produces a better quality step towards the solution.
SubproblemAlgorithm	Determines how the iteration step is calculated. The default, 'cg', takes a faster but less accurate step than 'factorization'. See fminunc trust-region Algorithm.
TolPCG	Termination tolerance on the PCG iteration, a positive scalar. The default is 0.1.
quasi-newton Algorithm
HessianApproximation	Specifies how fminunc calculates the Hessian. The choices are: "bfgs" (default) "lbfgs" {"lbfgs",Positive Integer} The choice "lbfgs" is the same as {"lbfgs",10}, meaning the default "lbfgs" memory value is 10. Use "lbfgs" for problems with many variables. See Solve Nonlinear Problem with Many Variables. For optimset, the option name is HessUpdate and the values are "bfgs", "lbfgs", {"lbfgs",Positive Integer}, "dfp", and "steepdesc". See Current and Legacy Option Names. Note Usually, the "dfp" and "steepdesc" values do not work well. They are available for educational purposes; see Banana Function Minimization.
ObjectiveLimit	A tolerance (stopping criterion) that is a scalar. If the objective function value at an iteration is less than or equal to ObjectiveLimit, the iterations halt because the problem is presumably unbounded. The default value is -1e20.
UseParallel	When true, fminunc estimates gradients in parallel. Disable by setting to the default, false. trust-region requires a gradient in the objective, so UseParallel does not apply. See Parallel Computing.

Example: options = optimoptions('fminunc','SpecifyObjectiveGradient',true)

problem — Problem structure

structure

Problem structure, specified as a structure with the following fields:

Field Name	Entry
objective	Objective function
x0	Initial point for x
solver	'fminunc'
options	Options created with optimoptions

Data Types: struct

Output Arguments

x — Solution

real vector | real array

Solution, returned as a real vector or real array. The size of x is the same as the size of x0. Typically, x is a local solution to the problem when exitflag is positive. For information on the quality of the solution, see When the Solver Succeeds.

fval — Objective function value at solution

real number

Objective function value at the solution, returned as a real number. Generally, fval = fun(x).

exitflag — Reason fminunc stopped

integer

Reason fminunc stopped, returned as an integer.

1	Magnitude of gradient is smaller than the OptimalityTolerance tolerance.
2	Change in x was smaller than the StepTolerance tolerance.
3	Change in the objective function value was less than the FunctionTolerance tolerance.
5	Predicted decrease in the objective function was less than the FunctionTolerance tolerance.
0	Number of iterations exceeded MaxIterations or number of function evaluations exceeded MaxFunctionEvaluations.
-1	Algorithm was terminated by the output function.
-3	Objective function at current iteration went below ObjectiveLimit.

output — Information about the optimization process

structure

Information about the optimization process, returned as a structure with fields:

iterations	Number of iterations taken
funcCount	Number of function evaluations
firstorderopt	Measure of first-order optimality
algorithm	Optimization algorithm used
cgiterations	Total number of PCG iterations ('trust-region' algorithm only)
lssteplength	Size of line search step relative to search direction ('quasi-newton' algorithm only)
stepsize	Final displacement in x
message	Exit message

grad — Gradient at the solution

real vector

Gradient at the solution, returned as a real vector. grad gives the gradient of fun at the point x(:).

hessian — Approximate Hessian

real matrix

Approximate Hessian, returned as a real matrix. For the meaning of hessian, see Hessian Output.

If the HessianApproximation option is "lbfgs" or {"lbfgs" n} then the returned hessian is [].

Data Types: double

More About

Enhanced Exit Messages

The next few items list the possible enhanced exit messages from fminunc. Enhanced exit messages give a link for more information as the first sentence of the message.

Local Minimum Found

The solver located a point that seems to be a local minimum, since the First-Order Optimality Measure is less than the OptimalityTolerance tolerance.

For suggestions on how to proceed, see When the Solver Succeeds.

Initial Point is a Local Minimum

The initial point seems to be a local minimum because the First-Order Optimality Measure is less than the OptimalityTolerance tolerance.

For suggestions on how to proceed, see Final Point Equals Initial Point.

Local Minimum Possible

The solver may have reached a local minimum, but cannot be certain because the First-Order Optimality Measure is not less than the OptimalityTolerance tolerance.

For suggestions on how to proceed, see Local Minimum Possible.

Solver Stopped Prematurely

The solver stopped because it reached a limit on the number of iterations or function evaluations before it minimized the objective to the requested tolerance.

For suggestions on how to proceed, see Too Many Iterations or Function Evaluations.

Problem Appears Unbounded

The solver reached a feasible point whose objective function value was less than or equal to the ObjectiveLimit tolerance. The problem is unbounded, or poorly scaled, or the ObjectiveLimit option is too high.

For suggestions on how to proceed, see Problem Unbounded.

Conflict

The 'trust-region' algorithm requires that you provide a gradient in the objective function and set the SpecifyObjectiveGradient option to true.

To proceed, do one of the following:

• Set the Algorithm option to 'quasi-newton'.

• Ensure that your objective function provides a gradient and set the SpecifyObjectiveGradient option to true. See Including Gradients and Hessians.

Definitions for Exit Messages

The next few items contain definitions for terms in the fminunc exit messages.

tolerance

Generally, a tolerance is a threshold which, if crossed, stops the iterations of a solver. For more information on tolerances, see Tolerances and Stopping Criteria.

local minimum

A local minimum of a function is a point where the function value is smaller than at nearby points, but possibly greater than at a distant point.

A global minimum is a point where the function value is smaller than at all other feasible points.

Solvers try to find a local minimum. The result can be a global minimum. For more information, see Local vs. Global Optima.

First-Order Optimality Measure

The first-order optimality measure is the maximum of the absolute value of the components of the gradient vector (also known as the infinity norm of the gradient). This should be zero at a minimizing point.

For more information, see First-Order Optimality Measure.

Size Of the Gradient

The size of the gradient is the maximum of the absolute value of the components of the gradient vector (also known as the infinity norm). This should be zero at a minimizing point.

For more information, see First-Order Optimality Measure.

OptimalityTolerance

The tolerance called OptimalityTolerance relates to the first-order optimality measure. Iterations end when the first-order optimality measure is less than OptimalityTolerance.

The first-order optimality measure of an unconstrained problem is the maximum of the absolute value of the components of the gradient vector (also known as the infinity norm of the gradient). This should be zero at a minimizing point.

For more information, see First-Order Optimality Measure.

FunctionTolerance

FunctionTolerance is a tolerance for the size of the latest change in objective function value.

StepTolerance

StepTolerance is a tolerance for the size of the last step, meaning the size of the change in location where the objective function was evaluated.

Norm of Current Step

The norm of the current step is the size of the change in location where the objective function was evaluated.

For more information, see Tolerances and Stopping Criteria.

Objective Limit

The solver reached a feasible point whose objective function value was less than or equal to the ObjectiveLimit tolerance. The problem is unbounded, or poorly scaled, or the ObjectiveLimit option is too high.

For suggestions on how to proceed, see Problem Unbounded.

Line Search Interval

The line search interval is the line segment along the search direction in which the solver attempts to minimize the objective function. If this interval is too small, the solver exits. The solver calculates the search direction and the size of this interval according to various algorithms described in Unconstrained Nonlinear Optimization Algorithms.

Output or Plot Function

An output function (or plot function) is evaluated once per iteration of a solver. It can report many optimization quantities during the course of a solver's progress, and can halt the solver.

For more information, see Output Functions for Optimization Toolbox or Plot Functions.

MaxIterations

MaxIterations is a tolerance on the number of iterations the solver performs. When the solver has taken MaxIterations iterations, the iterations end.

For more information, see Iterations and Function Counts or Tolerances and Stopping Criteria.

MaxFunctionEvaluations

MaxFunctionEvaluations is a tolerance on the number of points where the solver evaluates the objective and/or constraint functions. When the solver has evaluated functions at MaxFunctionEvaluations points, the iterations end.

For more information, see Iterations and Function Counts or Tolerances and Stopping Criteria.

Search Direction

The search direction is the vector from the current point along which the solver looks for an improvement. Optimization Toolbox™ solvers compute search directions via various algorithms, described in Unconstrained Nonlinear Optimization Algorithms.

Algorithms

Quasi-Newton Algorithm

By default, the quasi-newton algorithm uses the BFGS Quasi-Newton method with a cubic line search procedure. This quasi-Newton method uses the BFGS ([1],[5],[8], and [9]) formula for updating the approximation of the Hessian matrix. You can also specify the low-memory BFGS algorithm ("lbfgs") as the HessianApproximation option. While not recommended, you can specify the DFP ([4],[6], and [7]) formula, which approximates the inverse Hessian matrix, by setting the option to 'dfp'. You can specify a steepest descent method by setting the option to 'steepdesc', although this setting is usually inefficient. See fminunc quasi-newton Algorithm.

Trust Region Algorithm

The trust-region algorithm requires that you supply the gradient in fun and set SpecifyObjectiveGradient to true using optimoptions. This algorithm is a subspace trust-region method and is based on the interior-reflective Newton method described in [2] and [3]. Each iteration involves the approximate solution of a large linear system using the method of preconditioned conjugate gradients (PCG). See fminunc trust-region Algorithm, Trust-Region Methods for Nonlinear Minimization and Preconditioned Conjugate Gradient Method.

Alternative Functionality

App

The Optimize Live Editor task provides a visual interface for fminunc.

References

[1] Broyden, C. G. “The Convergence of a Class of Double-Rank Minimization Algorithms.” Journal Inst. Math. Applic., Vol. 6, 1970, pp. 76–90.

[2] Coleman, T. F. and Y. Li. “An Interior, Trust Region Approach for Nonlinear Minimization Subject to Bounds.” SIAM Journal on Optimization, Vol. 6, 1996, pp. 418–445.

[3] Coleman, T. F. and Y. Li. “On the Convergence of Reflective Newton Methods for Large-Scale Nonlinear Minimization Subject to Bounds.” Mathematical Programming, Vol. 67, Number 2, 1994, pp. 189–224.

[4] Davidon, W. C. “Variable Metric Method for Minimization.” A.E.C. Research and Development Report, ANL-5990, 1959.

[5] Fletcher, R. “A New Approach to Variable Metric Algorithms.” Computer Journal, Vol. 13, 1970, pp. 317–322.

[6] Fletcher, R. “Practical Methods of Optimization.” Vol. 1, Unconstrained Optimization, John Wiley and Sons, 1980.

[7] Fletcher, R. and M. J. D. Powell. “A Rapidly Convergent Descent Method for Minimization.” Computer Journal, Vol. 6, 1963, pp. 163–168.

[8] Goldfarb, D. “A Family of Variable Metric Updates Derived by Variational Means.” Mathematics of Computing, Vol. 24, 1970, pp. 23–26.

[9] Shanno, D. F. “Conditioning of Quasi-Newton Methods for Function Minimization.” Mathematics of Computing, Vol. 24, 1970, pp. 647–656.

Extended Capabilities

Automatic Parallel Support

Accelerate code by automatically running computation in parallel using Parallel Computing Toolbox™.

To run in parallel, set the 'UseParallel' option to true.

options = optimoptions('solvername','UseParallel',true)

For more information, see Using Parallel Computing in Optimization Toolbox.

Version History

Introduced before R2006a

R2023b: CheckGradients option will be removed

The CheckGradients option will be removed in a future release. To check the first derivatives of objective functions or nonlinear constraint functions, use the checkGradients function.