To find if my optimisation function is convex or non convex?

I have to optimise paramters "(theta)" of a model of a railway network "F". This railway network model is given to me in the form of a MATLAB-p file, so it is a black box. I have been given two data sets from the model, input u and output y. The output of my railway network model "F" is say y_cap.

Then y_cap = F(u,theta). 

I plan to take my optimisation function so as to minimise square of error between y and y_cap that is (y-y_cap)^2 and tune the parameters theta. 

That is, objective function = (y-y_cap)^2.

Is my objective function convex or non-convex? How to comment on its convexity? 

Answer 1

[Matt J]

If you know y_cap to be a linear function of theta, then it is convex. Beyond that, or some other structural information about y_cap(theta), you cannot know.

Answer 2

[Walter Roberson]

Perhaps you should be using Black Box Modeling

However, it seems clear that any one output of the railway network is going to depend upon all of the inputs (or at least more than one input). You therefore have a function which is mapping R^(a x b) to R^(22 x 8) for some matrix of size a x b, possibly 22 x 8 itself. And according to the definitions of convex that I can find, convex functions might be over R^(a x b) but their domain has to be R = R^1 -- they have to result in a single value, not a matrix of values. Therefore your function cannot be convex.

If, on the off chance (that seems unlikely considering our knowledge that trains cannot be arbitrarily assigned to tracks as if they were the only train in service) that the situation is equivalent to

for J = 1 : 22
  for K = 1 : 8
    y_cap(J,K) = F(u(J,K), theta);
  end
end

then it would make sense to talk about F(scalar,theta) being convex or not.

And the answer to that for a black box is that you can sometimes numerically disprove that something is convex, but you cannot prove that it is convex. Consider

   y = x^2 - ((x-3.8241)/100)^(-500)

This has the base being the convex x^2 you would experimentally determine it to be convex, unless you just happened to test between 3.8421-1/100 and 3.8421+1/100 where it would be effectively -infinity, and so is not actually convex. But you cannot know that unless you have access to the analytic form or get lucky on the testing.

[HARISH SATYAVADA]

@walter If i solve the optimisation function using lsqnonlin and end up with mutiple minima for multiple starting points then my function is not convex correct?

[Matt J]

It is correct if you have evidence that these minima are not global minima, e.g., if they have different resnorms. However, a convex function can have multiple global minima. For example, the convex function f(x)=constant is globally minimized by all x.

[Walter Roberson]

A convex function can be flat over a segment, and so can have a minima in multiple locations.