FMINCON: Interior-Point algo -> Is there a way to pass extra parameters to the hessian function or allow the objective function to return 3 output variables (function value, gradient and hessian) when there are no nonlinear constraints?

Hi,

I am trying to solve a nonlinear constrained optimization problem using fmincon.

Here is the breakdown ...

• Objective is nonlinear but has an analytic gradient and hessian.
• Objective must be fed a data structure containing various fields. Its value, as well as its gradient and hessian, are computed based on the provided data and the vector of parameters being optimized.
• Objective is relatively complicated with lots of nested functions and potentially lots of data.
• Linear equalities, inequalities and\or bounds may be a part of the mix depending on circumstances (eliminates the possiblity for using trust-region algorithm).
• No nonlinear constraints - implies that the hessian needs no adjustment via the lagrangian.

So what's the issue?

In order to take advantage of the fact that I can compute an analytic gradient and hessian, I need to rely on the interior-point algorithm or the trust-region algorithm. However, as stated above, the trust-region algorithm is not going to be ideal in most circumstances since I will most likely have some combination of linear constraints.

This brings me to the interior-point algorithm ...

It seems as though I can't take advantage of the fact that my objective function outputs 1, 2 or 3 variables (it's conditionalized) such that [f, g, h] = myfun(x, DATA) where 'x' is the vector of decision variables (parameters) being optimized and 'DATA' contains the necessary variables to compute the objective, gradient and hessian as needed.

If I use the HessFcn option and provide a function handle, it requires a syntax of hessianfcn(x, lambda) where lambda holds the lagrangian information - this is not a suitable syntax since I cannot compute my hessian without 'DATA'. Furthermore, note that there are no nonlinear constraints so 'lambda' is basically empty and has no effect on the hessian. Also, even if my hessian could be computed solely using 'x', this approach would not be ideal - computing the hessian requires computing the objective function and gradient so it would basically double the computing effort at each iteration. If I could simply obtain my 3 output variables, I would not have to recalculate the objective and gradient.

So to sum it all up ...

In using the 'interior-point' algorithm with fmincon, how do I go about bypassing the need for hessianfcn(x, lambda) and simply allow my objective to output 3 variables?

This is a special case that, as far as I know, has not been accounted for - when there are no nonlinear constraints, 'interior-point' should be able to accept 3 output variables from the provided objective function because no lagrangian-related adjustments are necessary! Please, correct me if I am wrong.

At the very least, if that is not doable, I would settle for passing extra parameters to hessianfcn.

Any thoughts on this?

I would appreciate any feedback on this.

Thanks!

[Walter Roberson]

http://www.mathworks.com/help/matlab/math/parameterizing-functions.html

[Richard Shefteshy]

Hi Walter,

Thanks for the quick reply.

I'm not sure how this helps the problem of bypassing the syntax of hessfcn(x, lambda) though.

Unfortunately, I'm having a hard time wrapping my head around this.

How would you go about defining HessFcn in the options? There's no way of simply unlocking the possibility of outputting 3 variables from my objective function - it's already setup that way (if I remove the linear constraints, I use trust-region and the hessian goes through).

[Richard Shefteshy]

Do I go something like ....

HessFcn = @(x,lambda)myhessfcn(x, lambda, DATA)

where [h] = myhessfcn(x, lambda, DATA) ?

[Walter Roberson]

'Hessianfcn', @(x,lambda) MyHessianFcn(x, lambda, Data)

Answer 1

[Matt J]

Also, even if my hessian could be computed solely using 'x', this approach would not be ideal - computing the hessian requires computing the objective function and gradient so it would basically double the computing effort at each iteration.

To address this issue, you can adapt the technique described here,

https://www.mathworks.com/help/optim/ug/objective-and-nonlinear-constraints-in-the-same-function.html

The link talks about using nested functions and externally scoped variables to share values between the objective and constraint functions, but the same technique applies if you want to share between the objective function code and the HessianFcn code (or any other code).

For you, this would probably look like the following:

function x=optimizationMain(DATA,x0,A,b,Ae,beq,lb,ub)


  state.x=[]; state.f=[];state.g=[];
   
  opts=optimoptions('fmincon','SpecifyObjectiveGradient',true,...
                              'HessianFcn',@myHessianFcn);
  
   x=fmincon(@objectiveWrapper,x0,A,b,Ae,beq,lb,ub,opts); %run optimization

    function varargout=objectiveWrapper(x)
        
        if ~isequal(x,state.x)
         [varargout{1:nargout}]=myfun(x,DATA);
          if nargout>1, 
             state.x=x; state.f=varargout{1}; state.g=varargout{2}; 
          end
        else
           varargout={state.f,state.g};
        end         

        
    end

    function H=MyHessianFcn(x, ~)
        
        if ~isequal(x,state.x)
            state.x=x;
            [state.f,state.g]=myfun(x,DATA);
        end

        H=.... some function of state and DATA
      
       
    end


end

[Richard Shefteshy]

Hi Matt,

Thanks for the reply and for the solution!

I just did a quick implementation - it works and produces the analytic hessian that I am looking for.

I have not messed around much with nested functions like that - in my description, what I meant by nested functions is that my objective calls another function which calls another function etc... in order to compute the objective, gradient and hessian. For example, my likelihood function being optimized is a combination of multiple instances of a probability function which, in turn, depends on a link function which, in turn, depends on a score function.

Anyways, your solution works though I made some minor adjustments for my needs. Instead of DATA, I pass myfun as a function handle to which I pass DATA as an extra parameter. myfun also returns the 3 outputs [f, g, h] = myfun(...) so I set objectiveWrapper to return [f,g,h] and store them in 'state'. Then, MyHessisnFcn simply returns H = state.h though it can easily be adapted to handle nonlinear constraints (lambda).

Again, thanks for the help - I really appreciate it.