Which condition must be satisfied so that three real numbers are all positive or negative? (fmincon related)

Hello all,

let me go straight to the question: I have three values, which are the eigenvalues of a 3 x 3 matrix (calculated symbolically, so these are pretty large expressions) and I can't find a numerical relation between the three to ensure they are all positive or negative. Once I have this expression I intend to use as a nonlinear constrain to an optimisation problem.

Answer 1

[John D'Errico]

For real vector X (of any length)

all(X<0) || all(X>0)

Having said that, this is NOT a valid constraint for fmincon. It will result in a search space that is split into two discontiguous sub-domains.

[Jorge Mira Pérez]

So, is there an expression that would work with fmincon?

[John D'Errico]

Nope. Not as such.

The first problem is all(X>0) is not a continuous relation. But that can be fixed, as below.

Pose this as two distinct subproblems. Thus solve the problem once with a constraint min(X)>=0. Then solve it again with the constraint max(X)<=0. Pick the better of the two results.

Note that I wrote inequality constraints of the form <= and >=. fmincon cannot handle strict inequalities. In fact, fmincon will typically allow some of the constraints to be violated by TolCon.

Best MIGHT be to formulate each problem as having three constraints.

X(1) >= 0
X(2) >= 0
X(3) >= 0

The problem with min and max is they are not differentiable. Since X here is a vector apparently of eigenvalues, I'd still be a bit worried about convergence issues and differentiability of the constraints, because eig does not enforce the order the eigenvalues come out in. That could create serious problems of discontinuity of the constraints, so worse than a simple non-differentiability from just using min and max.

So min and max (as I wrote the constraints above) may actually work better and more stably for fmincon. So best probably is to solve the problem twice, once with min(X)>=0, once with max(x)<=0.

[Jorge Mira Pérez]

Dear John, thanks a lot for elaborating your answer this much! I truly appreciate it.

I thought the same as you, and will try soon this approach as it seems the only feasible way.

But something is pretty interesting arised yesterday. As you know, I have three complicated nonlinear constrains, so, when I apply these to the model, the goodness of the fit is worse than if I don't apply them (the solution is similar, but has different RMSE). Do you think fmicon is messing with the solution because of the nonlinear constrains?

[John D'Errico]

If you constrain a system, then IF the constraints are active, the solver cannot find as good a solution as it would have had there been no constraints. So your goodness of fit must generally decrease by some amount IF the constraints were active.

How can this surprise you in the very least amount? Effectively, you just told fmincon, "DON'T GO THERE!", even though fmincon sniffled, replying "But I want to go there. If you want the best solution in terms of the objective being smaller, I need to go there." Highly anthropomorphized of course.

There are circumstances where this is not the case of course, if there are multiple solutions, or if the constraint is active, but the solution found would have been at the same spot even without a constraint. (That would be a rare case.) fmincon does not assure you that it will always find the globally optimal solution. And when you introduce constraints into the problem, it is no longer the same problem.

So is fmincon "messing" with the solution? Sigh, no, YOU are messing with the solution. You were the one who changed the problem by introducing constraints. If you disallow fmincon from going into certain regions of the search space, preventing solutions from being found that would have otherwise been allowed, then the result will often be less "good". At the same time, if that part of the search space creates non-meaningful solutions, then the bad solution was already not "good".