Sign of lambda.eqlin

I am trying to understand the sign of the lambdas for my equality constraints - and can't seem to find how this is encoded in the documentation.

My assumption is that Aeq*x = beq is encoded either:

transpose(lambda) * (Aeq * x - b)

OR

transpose(lambda) * (b - Aeq * x)

Can someone help me confirm? Or show me a way I could check?

Thank you!!

[Walter Roberson]

Does it matter? Those details are handled internally, in code that you have no access to

Answer 1

[William Rose]

@Aidan,

[edit: fix typos; code not affected]

[Edit 2: Fix errors in code which @David Goodmanson pointed out - thank you. Conclusion is not changed.]

It seems that lambda is used as follows:

I say this because I used fmincon to minimize a constrained function, and then I evaluated the derivatives of the Lagrangian defined both ways:

and

The derivatives of the Lagrangian with respect to x1, x2, ..., and λ should equal zero at the minimum. They do for Lplus, but not for Lminus.

See script below. In this script,

f(x)=x1^2+x2^2

i.e. f(x)=paraboloid with a minimum at x1=x2=0. The constraint is

x1+x2=1

i.e. a line with slope -45 degrees which goes through x=(1,0) and x=(0,1).

% Define constants
x0=[2,2];       % initial guess
Aeq=[1,1];      % equality constraint
beq=1;          % equality constraint
% Next: minimize function subject to constraint
[x,fval,~,~,lambda,grad,hessn] = fmincon(@myfun,x0,[],[],Aeq,beq);

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

L=lambda.eqlin;          % lambda at solution point
% Display results of minimization
fprintf('x1=%.3f, x2=%.3f, f(x)=%.3f, λ=%.3f.\n',x,fval,L)

x1=0.500, x2=0.500, f(x)=0.500, λ=-1.000.

% Display derivatives with respect to x1, x2, λ
fprintf('d(LaGr+)/dx1=%.3f\n',(LaGrPlus(x+[.001,0],L,Aeq,beq)-LaGrPlus(x,L,Aeq,beq))/.001)

d(LaGr+)/dx1=0.001

fprintf('d(LaGr+)/dx2=%.3f\n',(LaGrPlus(x+[0,.001],L,Aeq,beq)-LaGrPlus(x,L,Aeq,beq))/.001)

d(LaGr+)/dx2=0.001

fprintf('d(LaGr+)/dλ=%.3f\n',(LaGrPlus(x,L+.001,Aeq,beq)-LaGrPlus(x,L,Aeq,beq))/.001)

d(LaGr+)/dλ=0.000

fprintf('d(LaGr-)/dx1=%.3f\n',(LaGrMinus(x+[.001,0],L,Aeq,beq)-LaGrMinus(x,L,Aeq,beq))/.001)

d(LaGr-)/dx1=2.001

fprintf('d(LaGr-)/dx2=%.3f\n',(LaGrMinus(x+[0,.001],L,Aeq,beq)-LaGrMinus(x,L,Aeq,beq))/.001)

d(LaGr-)/dx2=2.001

fprintf('d(LaGr-)/dλ=%.3f\n',(LaGrMinus(x,L+.001,Aeq,beq)-LaGrMinus(x,L,Aeq,beq))/.001)

d(LaGr-)/dλ=0.000

function f=myfun(x)
f=norm(x)^2;
end
function y=LaGrPlus(x,lambda,Aeq,beq)
y=myfun(x)+lambda*(Aeq*x'-beq);
end
function y=LaGrMinus(x,lambda,Aeq,beq)
y=myfun(x)-lambda*(Aeq*x'-beq);
end

[Matt J]

Also, this equation in the documentation,

makes it pretty clear that linear inequalities are λ-weighted according to,

+lambda*(Aineq*x-bineq)

so it would stand to reason that equalities would be weighted similarly.

[David Goodmanson]

Hi William,

in your last three fprintf lines you have, e.g.

(LaGrMinus(x+[.001,0],L,Aeq,beq)-LaGrPlus(x,L,Aeq,beq))/.001)
instead of
(LaGrMinus(x+[.001,0],L,Aeq,beq)-LaGrMinus(x,L,Aeq,beq))/.001)
                                       ^

[William Rose]

@David Goodmanson, thank you for pointing out the error in my code. I have fixed it above now. The conclusion is not altered.

Answer 2

[David Goodmanson]

Hi Aiden,

William's example is

f(x) = x^2+y^2    Aeq=[1,1]     beq = 1
L(x,lambda) = f(x) + lambda*(Aeq*x'-beq)       % convention is plus sign on second term.

Using the the ancient technique of pencil and paper I came up with lambda = -1, which is what fmincon produces.

So (Aeq*x-beq) appears to be the correct form.

It's consistent with what Matt suggested and WIlliam has.