Error using the function fmincon

function [LLF, LL, H  ] = garchlike(data)

p=1;
q=1;

   if isempty(q)
      m=p;
   else
       m  =  max(p,q);   
   end

%this are the parameters

psi=[0.01;0.01];

gamma=[0;0.02;0.02;0];

lambda=[0;0.03;0.03;0];
A=[1;-0.04;-0.03;1];

stdestimate = std(data,1);   

data   =  [stdestimate ; data];

T =  size(data,1);

B=(reshape(A,2,2))^-1;

%I am squaring each element of the inverse of A

B2=((reshape(A,2,2))^-1).^2;

%getting the Bl matrix(squared interactions)

Bl=[B2(1,:);B(1,:).*B(2,:);B2(2,:)];

garchc=Bl*psi;

garcha=Bl*(reshape(gamma,2,2))*reshape(A.^2,2,2);

garchb= Bl*(reshape(lambda,2,2))*reshape(A.^2,2,2);

H(1,1).day=[0.001; 0; 0; 0.002; ];
%convert to matrix

H(1,1).day=ivech(H(1,1).day);

for t = (m + 1):T

H(1,t).day= ivech(garchc + garcha*(diag(H(1,t-1).day))+ garchb*(data((t-1),:).^2)');

end
% Getting the loglikelihood

LLF=zeros(1,T);

%the loklikelihood

for t = (m + 2):T

LLF(t)= log(det(H(1,t).day))+ data(t,:)/(H(1,t).day)*data(t,:)';

end
t=(m+2):T;

LLF=-LLF(t);

LL=sum(LLF);

end

I have this function from up and trying to minimize using fmincon:

[parameters,LLF,EXITFLAG, OUTPUT,HESSIAN] =  fmincon('garchlike',    [psi;gamma;lambda;A],sumA ,sumB ,[] , [] , LB , UB,[],options, data, p , q);

I don't know why is not working and gives me an error : "Error using garchlike Error using garchlike Too many input arguments. Error in fmincon (line 631) initVals.f = feval(funfcn{3},X,varargin{:}); Caused by: Failure in initial user-supplied objective function evaluation. FMINCON cannot continue."

I appreciate any help you can give me/tips..

Answer 1

[Walter Roberson]

Your call to fmincon() has three variables after your options structure. That is not any documented syntax for fmincon(). In practice, for backwards compatibility, it will attempt to pass those extra variables as additional arguments to the function being called. That would result in 4 inputs, but your function only accepts one input.

It appears to me that you should not be passing data, p, q into fmincon()

[Laura Pop]

I've tryed not to put data,p,q in the fmincon. I got the same error in fmincon (line 631).

Should be a problem because I have the Ht in a structure format and not cells?

Or should maybe because data is a matrix 800x3?

[Walter Roberson]

Try

[parameters,LLF,EXITFLAG, OUTPUT,HESSIAN] =  fmincon(@garchlike,    [psi;gamma;lambda;A],sumA ,sumB ,[] , [] , LB , UB,[],options);

and for debugging purposes, modify garchlike a little:

function [LLF, LL, H  ] = garchlike(data, varargin)
   persistent warned
   if nargin > 1 && isempty(warned)
     warn(sprintf('garchlike called with %d extra arguments', nargin - 1));
     warned = true;
   end

Note: each time you retest and want the potential for notifications re-armed, execute

clear garchlike

[Laura Pop]

@Walter Roberson I tryed what you told me but still nothing:( Gives me the same error in fmincon at line 631: initVals.f = feval(funfcn{3},X,varargin{:});

I just cannot fiogure it out why is not working,for univariate case is going fine(when I have one variable):(

Answer 2

[Matt J]

I'm just guessing, but this might be what you're trying to do

   x0=[psi;gamma;lambda;A];

   fmincon(@(x,data,p,q) garchlike,  x0  ,sumA ,sumB ,[] , [] , LB , UB,[],options);

You should test that the modified function below works before you try feeding it to fmincon. Note that it now takes 4 input arguments.

 function [LLF, LL, H  ] = garchlike(x,data,p,q)

   %this are the parameters
    psi=x(1)
    gamma=x(2);
    lambda=x(3);
    A=x(4);

       if isempty(q)
          m=p;
       else
           m  =  max(p,q);   
       end

    stdestimate = std(data,1);   
    data   =  [stdestimate ; data];
    T =  size(data,1);
    B=(reshape(A,2,2))^-1;
    %I am squaring each element of the inverse of A
    B2=((reshape(A,2,2))^-1).^2;
    %getting the Bl matrix(squared interactions)
    Bl=[B2(1,:);B(1,:).*B(2,:);B2(2,:)];
    garchc=Bl*psi;
    garcha=Bl*(reshape(gamma,2,2))*reshape(A.^2,2,2);
    garchb= Bl*(reshape(lambda,2,2))*reshape(A.^2,2,2);
    H(1,1).day=[0.001; 0; 0; 0.002; ];
    %convert to matrix
    H(1,1).day=ivech(H(1,1).day);
    for t = (m + 1):T
    H(1,t).day= ivech(garchc + garcha*(diag(H(1,t-1).day))+ garchb*(data((t-1),:).^2)');
    end
    % Getting the loglikelihood
    LLF=zeros(1,T);
    %the loklikelihood
    for t = (m + 2):T
    LLF(t)= log(det(H(1,t).day))+ data(t,:)/(H(1,t).day)*data(t,:)';
    end
    t=(m+2):T;
    LLF=-LLF(t);
    LL=sum(LLF);
    end

[Laura Pop]

I just eliminated the p,q from the function and left only m=1.

gives me the same error at line 631 in fmincon. Also that there are not enough input arguments, error @(x,data)garchlike,erorr at: psi=x(1:2);

 gamma=x(3:6);

 lambda=x(7:10);

A=x(11:14);

[Laura Pop]

I used the debugger and check the line 631 which is:

 initVals.f = feval(funfcn{3},X,varargin{:});

The initVals.f(is 14x1,the initial values I have),funfcn(is 1x5 cell,'fun' 'fmincon' garchlike [] []),X(14x1),varargin(empty).Also tells me that the fmincon must have at least 4 inputs,don't I have 4?

[Matt J]

gives me the same error at line 631 in fmincon.

As I told you, you shouldn't be running fmincon yet. You should be running garchlike and making sure it works first.

Also tells me that the fmincon must have at least 4 inputs,don't I have 4?

We can't know. You haven't show us how you're now running fmincon, now that p and q have been eliminated.

[Walter Roberson]

Change

@(data,X) garchlike

to

@(X) garchlike(data, X)

[Laura Pop]

I tryed and gives me the following error that I need to stop because is running for more then 30 secunds:

The error at the end after I am terminating:

"Operation terminated by user during garchlike1 (line 47)

In @(X)garchlike(data,X)

In C:\Program Files\MATLAB\R2012b\toolbox\shared\optimlib\finDiffEvalAndChkErr.p>finDiffEvalAndChkErr (line 26)

In C:\Program Files\MATLAB\R2012b\toolbox\shared\optimlib\finitedifferences.p>finitedifferences (line 128)

In nlconst (line 303)
                    [gf,gnc(:,nonlIneqs_idx),gnc(:,nonlEqs_idx),numEvals] = ...

In fmincon (line 837)
    [X,FVAL,LAMBDA,EXITFLAG,OUTPUT,GRAD,HESSIAN]=..."

And before appers for many times the following error:

"Warning: Matrix is singular, close to singular or badly scaled. Results may be inaccurate. RCOND = NaN. 

> In garchlike at 47

    In @(X)garchlike(data,X)

    In C:\Program Files\MATLAB\R2012b\toolbox\shared\optimlib\finDiffEvalAndChkErr.p>finDiffEvalAndChkErr at 26

    In C:\Program Files\MATLAB\R2012b\toolbox\shared\optimlib\finitedifferences.p>finitedifferences at 128

    In nlconst at 303

    In fmincon at 837 "

[Walter Roberson]

This is not necessarily incorrect behavior. If your matrix is close to singular, then your conditions are so sensitive to round-off error that the answer may well be useless.

[Laura Pop]

Thank you very much to both of you Matt J and Walter Roberson for your comments. So as I understood the problem should be in the initial values.I tryed using also fminunc and change the initial values but didn't had succes.

I noticed that when I use likelihhod with - sign give me this error:

" User objective function returned complex; trying a new point... Solver stopped prematurely."

I don't get why a complex since if I am running gives me -1.2927e-4 the likelihood.

ANd when I change the likelihood to be with sign + I have the error from the top.

Do you have some ideas about what else should I try in order to fix it?

[Walter Roberson]

As a sub-expression you have

log(det(H(1,t).day))

if the determinant is negative, the log() of it would be complex. Is it certain that none of the determinants are negative ?

[Laura Pop]

Is gaving me the following error:

Error using optimfcnchk/checkfun (line 320)

User function '@(XX)multi_garchlikelihood_marinela(XX,data)' returned a complex value when evaluated;

   FMINUNC cannot continue.

Error in C:\Program Files\MATLAB\R2012b\toolbox\optim\optim\private\lineSearch.p>sectioningPhase (line 296)

Error in C:\Program Files\MATLAB\R2012b\toolbox\optim\optim\private\lineSearch.p>lineSearch (line 73)

Error in fminusub (line 205)
    [alpha,f,grad,exitflagLnSrch,funcCountLnSrch,faultTolStruct] = ...

Error in fminunc (line 416)
   [x,FVAL,GRAD,HESSIAN,EXITFLAG,OUTPUT] = fminusub(funfcn,x, ...

I don't get why is telling me(If I am understanding well) that when the objective function(multi_garch_likelihood) is evaluated the result is complex?? When I am running is working very well and is givin me a scalar value.

[Laura Pop]

Yes,I am sure because I checked already the function garchlikelihood step by step.:( I am trying different things and every time get some different error..I tryed also to relax the tolerance,change the initial values..

If I am going with likelihood sign"-" tells me that the objective is a comnplex value. If I am going with likelihood sign "+" tells me that that the inverse of H(1,t).day is badly scaled..

[Walter Roberson]

Put in a conditional breakpoint after LLF is assigned to:

dbstop at LINENUMBER if ~isreal(LLF)

you can also click at right-click at the place you would normally left-click to set a breakpoint: that will allow you to enter the appropriate conditional expression.

[Matt J]

In addition to Walter's last remark, you should really modify this final for-loop as follows

    for t = (m + 2):T

     LLF(t)= log(det(H(1,t).day))+ data(t,:)/(H(1,t).day)*data(t,:)';

     if ~isreal(LLF(t)) | ~isfinite(LLF(t))

      LL=Inf; return

     end
    end

You should also be using the sqp algorithm

options=optimoptions(options,'Algorithm','sqp');

which can recover from steps into regions where the loglikelihood is not finite.

I don't get why is telling me(If I am understanding well) that when the objective function(multi_garch_likelihood) is evaluated the result is complex?? When I am running is working very well and is givin me a scalar value.

Because you have only been testing "safe" values of the parameters. If all psi, gamma, and lambda are chosen zero, for example, you will not get a finite real value for the loglikelihood. If you aren't using the sqp algorithm, you are not doing anything to prevent fmincon from exploring those parameter choices.

[Laura Pop]

I tryed using also 'sqp' in the format both of you said,but is not able to find me a local minimum,is just telling me that is possible. What should I change in order to actually find?

I think I figure it out what was my problem,the inverse of H(1,t).day is really close to zero and this I think create problem to Matlab when I calculate the loglikelihood so gave me a complex value.

Untill now I used the TolX and TolFun set to 1e-6. I noticed that if I change the tolerance to 1e-2 to both and using fminunc (without specifing:if ~isreal(LLF(t)) | ~isfinite(LLF(t)) LL=Inf; return) then is able to find me a minimum after 2 iterations.

However since I haven't used before this function I don't know if is the right minimum since Matlab finds it after 2 iterations?

[Matt J]

I tryed using also 'sqp' in the format both of you said,but is not able to find me a local minimum,is just telling me that is possible. What should I change in order to actually find?

If it's telling you "Local Minimum Possible", you can't automatically assume that it failed. FMINCON will never return an exit message "Local minimum definitely found". You can find out more information about the exit condition by calling fmincon with 4 output arguments

 [x,fval,exitflag,output] = fmincon(...)

and/or by turning the 'Display' option to 'iter'.

Until now I used the TolX and TolFun set to 1e-6. I noticed that if I change the tolerance to 1e-2 to both and using... then is able to find me a minimum after 2 iterations.

I think what you really saw is a message that the TolX or TolFun stopping criterion was met. I don't think you're claiming to see evidence of an actual minimum.

It is easy to make fmincon hit the TolX/TolFun stopping criteria early if you make their values really large, as you have done. If you make them Inf, you should see fmincon stop in 1 iteration and return your initial guess as the solution. Therefore, setting TolX and TolFun to arbitrary large values is usually a bad thing to do.

However since I haven't used before this function I don't know if is the right minimum since Matlab finds it after 2 iterations?

You should always be testing your code on simulated problem data where you know the true solution in advance.

[Laura Pop]

I alreday called the function with all the outputs.And the problem is with the tolerance functions.Solutions are either to relax or change the initials values.. I kind of tryed all sort of things..

I agree with testing the code on something I know the results..I've been doing this for the case where I have the constant only.(so the resut of MLE is just the variance of the data).. I am not assuming that failed when saying that 'local minimum possible',but just don't know what to do anymore.

If is usually good to set up small values for TolX and Tolfun,then that means that the problem is the initial values that I give. Is not able to find me(not even with sqp)the minimum. The inverse of H(1,t).day is really close to zero and this I think create problem to Matlab when I calculate the loglikelihood so gave me a complex value(also when I am using sqp).

[Matt J]

I alreday called the function with all the outputs.And the problem is with the tolerance functions.

What evidence of that did you see in the fmincon output data?

I kind of tryed all sort of things..

You shouldn't be trying all sorts of random things. If there is a good reason to change TolX and TolFun to a particular value then you should do so, but from what I can tell, you do not know what these parameters mean. You are just picking values for them by random experimentation.

[Laura Pop]

You are right,I have been struggling with this problem since 1 month maybe;I am just a beginner and I am aware is not an excuse..

I just want to make the program work properly.

The problem is then with the values that I give.(if the tolerance function should work with large values).

The likelihood LL returns complex values,implies that inverse of H(1,t).day is badly scaled(error which I get),implies that the values I give are not good.

I think maybe I should think at a way to give the values in order to have the(det(H(1,t).day) positive. How should I impose these restrictions?

If this will always be positive then I won't get a complex value and maybe I'll be able to find a minimum..

[Matt J]

The likelihood LL returns complex values,implies that inverse of H(1,t).day is badly scaled(error which I get),implies that the values I give are not good.

I think we already solved that problem: by setting LL=Inf when the determinant goes negative and using the 'sqp' algorithm. You gave no evidence that that did not work.

[Laura Pop]

True I have been seting like that but still I get the following error when using the options:

options=optimset('algorithm','sqp','FinDiffType','central','Display','iter','TolX',1e-6,'TolFun',1e-6,'Diagnostics','on','MaxIter',1000,'MaxFunEvals',12000,'LargeScale','off','FunValCheck','on','PlotFcns',{@optimplotx,@optimplotfval,@optimplotfirstorderopt});

Warning: Matrix is singular to working precision.

> In multi_garchlikelihood_adriana at 35

In @(XX)multi_garchlikelihood_adriana(XX,data)

In optimfcnchk>checkfun at 316

In C:\Program Files\MATLAB\R2012b\toolbox\optim\optim\private\evalObjAndConstr.p>evalObjAndConstr at 135
In C:\Program Files\MATLAB\R2012b\toolbox\optim\optim\sqpLineSearch.p>sqpLineSearch at 287
In fmincon at 906

Warning: Matrix is singular, close to singular or badly scaled. Results may be inaccurate. RCOND = NaN.

> In multi_garchlikelihood_adriana at 67

In @(XX)multi_garchlikelihood_adriana(XX,data)
In optimfcnchk>checkfun at 316
In C:\Program Files\MATLAB\R2012b\toolbox\optim\optim\private\evalObjAndConstr.p>evalObjAndConstr at 135
In C:\Program Files\MATLAB\R2012b\toolbox\optim\optim\sqpLineSearch.p>sqpLineSearch at 287
In fmincon at 906 
Error using optimfcnchk/checkfun (line 322)
User function '@(XX)multi_garchlikelihood_adriana(XX,data)' returned Inf or -Inf when evaluated;
FMINCON cannot continue.

Error in C:\Program Files\MATLAB\R2012b\toolbox\optim\optim\private\evalObjAndConstr.p>evalObjAndConstr (line 135)

Error in C:\Program Files\MATLAB\R2012b\toolbox\optim\optim\sqpLineSearch.p>sqpLineSearch (line 287)

Error in fmincon (line 906)

    [X,FVAL,EXITFLAG,OUTPUT,LAMBDA,GRAD,HESSIAN] = sqpLineSearch(funfcn,X,full(A),full(B),full(Aeq),full(Beq), ...

[Laura Pop]

Is telling me that the inverse of A is badly scaled then the LLF will be badly scaled...

[Laura Pop]

If I am using the options without "'FunValCheck','on'" then I get:

End diagnostic information
                                                        Norm of First-order
Iter F-count            f(x) Feasibility  Steplength        step  optimality
  0      61   -1.922432e+04   0.000e+00                           1.342e+06
User objective function returned Inf; trying a new point...
  1     171   -2.059890e+04   2.442e-23   2.569e-08   4.231e-03   2.013e+06
User objective function returned Inf; trying a new point...
  2     299   -2.150776e+04   2.442e-23   4.184e-11   6.815e-04   3.934e+06
User objective function returned Inf; trying a new point...
  3     451   -2.162888e+04   2.442e-23   9.733e-14   3.764e-05   4.828e+06
User objective function returned Inf; trying a new point...
  4     551   -2.162888e+04   2.442e-23   3.234e-16   8.961e-07   4.828e+06

Local minimum possible. Constraints satisfied.

fmincon stopped because the size of the current step is less than the selected value of the step size tolerance and constraints are satisfied to within the default value of the constraint tolerance.

Optimization stopped because the relative changes in all elements of x are less than options.TolX = 1.000000e-06, and the relative maximum constraint violation, 2.441629e-23, is less than options.TolCon = 1.000000e-06.

Optimization Metric Options

max(abs(delta_x./x)) = 7.18e-07 TolX = 1e-06 (selected)

relative max(constraint violation) = 2.44e-23 TolCon = 1e-06 (default)

The exitlag is 2(change in X less then tolerance).

[Walter Roberson]

Then your optimization works.

MATLAB cannot know what the "true" minimum value is, so it cannot issue a response such as "Minimum found!". MATLAB can just say that given the conditions you gave, the location it found fit the termination conditions you gave.

[Laura Pop]

Okay,so the optimization is working. But the estimated values that I get are not right.(doesn't finds different values from the one that I gave,is estimating the values only where I have zero).

I also get big differences in values if either I use LL=sum(LLF) or LL=-sum(LLF). What should I do then?

Is it okay to apply fmincon if I don't have any constraints,no bonds?(just by setting the LB=-inf and UB=inf)

[Matt J]

the estimated values that I get are not right.(doesn't finds different values from the one that I gave,is estimating the values only where I have zero).

What happens when you run fmincon using the known solution as your initial point? If the algorithm doesn't stop immediately, then you have coded your objective function or your data simulation incorrectly. If your known solution is a minimum of the objective you've provided, the algorithm should recognize this immediately once derivatives are calculated.

I also get big differences in values if either I use LL=sum(LLF) or LL=-sum(LLF). What should I do then?

There is no reason the minimum of sum(LLF) and -sum(LLF) should be the same. They are very different functions.

Is it okay to apply fmincon if I don't have any constraints,no bonds?(just by setting the LB=-inf and UB=inf)

If fmincon gives no errors when you do so, why would you be worried?

[Laura Pop]

As I said is working for a simple case where I have only the constant(in my model) returns what I am expected to do.

Still don't know what to do when I have "local minimum possible" but not local minimum found.Walter is saying is working,so is good.

Normally then I should just accept the values that are estimated as beeing right?(even if some of them are just the same as the initial one that I give)

[Matt J]

If the first order optimality measure at the point where fmincon stops is small and the exitflag is good, then you can probably be confident that fmincon fulfilled its part of the bargain.

The result can still be bad, however, if you have coded something incorrectly in your objective function or constraints. fmincon can't do anything about that. Obviously also, the initial guess is important. If you initialize at a point where the function is locally flat, the algorithm will see it as a local min, and won't move.

[Laura Pop]

I belive I figure it out the problem!

I noticed that if I change order of the psi values,psi=XX(4:6) and put the values of lambda=diag(XX(1:3)). IS still estimating only the values of psi and the values of lambda remain the same or as I said close to the values I give..:((

I tryed also writting the psi as psi=[XX(1);XX(2);XX(3)]and is working..

Some ideas about why doesn't estimate lambda(which is a 3x3 matrix,I am interesting only on the diagonal values)? Or how could I writte in order to estimate it?

[Matt J]

Compare the magnitudes of

   ivech(garchc + garchb*(data((t-1),:).^2)');

with the magnitudes of

   ivech(garchc);

[Laura Pop]

I already compared the magnitude: ivech(garchc + garchb*(data((t-1),:).^2)'); LL =

    -1.478566945911877e+06

   ivech(garchc);

LL =

    -1.410525122414626e+06

And if I am runnig the ivech(garchc + garchb*(data((t-1),:).^2)') with the values that I found in the first part is telling me that the minimum was found and yes gives me the same values of psi as before(but no changes for lambda). If I am running with the values that I give find me the values of psi only..,still no changes for lambda.Focus on find the minimum only for psi:(

[Matt J]

Attach "data" so that we can run run your code.

[Matt J]

I actually cannot run this because I don't have ivech, nor do I know what it does.

[Matt J]

You recently changed LL=-sum(LLF) to LL=sum(LLF). I'm confused about which form of the code is now the official one.

Either way, the objective function has no upper or lower bound. FMINCON is just pushing it to -Inf by making the H(t).day matrices go to zero or to infinity, depending on the sign of LL. The reason your lambda barely change is that the quickest path to -Inf is to make psi either very large or very small, without modifying XX(5:6).

There is probably some mistake in your model, or your coding of it, because a loglikelihood should always have an upper bound.

[Laura Pop]

Is the last one I posted.I realized about the mistake so the right one is with LL=sum(LLF). I don't have any constraints so that why I just put -inf and +inf.However if I am using also fminunc I get the same results and doesn't want to estimate:(

[Matt J]

Yes, but we now know why the results are bad. As I explained in my last comment, your objective function is incorrect: it shouldn't be unbounded, but it is.

[Matt J]

There is probably some mistake in your model, or your coding of it, because a loglikelihood should always have an upper bound.

I guess that's not true, come to think of it. As a function of the parameters, a loglikelihood can surely be unbounded.

However, it shows that you need constraints...

[Laura Pop]

I tryed to set up the LB to zero and UB to 2 and is giving me some results,is estimating the paramters to lambda. I have general constraints for the parameters so psi>0 and lambda>=0 so if I am setting the LB,UB like this should this be okay? I noticed is going well for Arch(1) but not for Garch..:(

[Matt J]

It's your model and only you know what "okay" means, but since you aren't getting uniformly good results, I would guess there's a problem. I have the vague impression that constraints relating psi to lambda might be needed. Currently, the objective function seems very sensitive to psi, but much less so to lambda. Constraints relating the two could help with that. However, I don't know what would make sense for those additional constraints. That's a modelling question that only you could answer.