Why did Matlab performance massively decrease for basic arithmetic operations from Matlab R2014b to 2015 and 2017 version?

I'm supposed to program a tool to run on R2015 but only have R2014b and R2017a available on the computer I am using. I thought using R2014b would be safer bet compatibility wise. So far so good. Everything seemed to be working fine. The problem is the code needs ~4s when running in R2014b but several minutes in R2015 and R2017 (using the same computer). The part of the code that takes a lot longer in the R2017 version (and I assume it is the same part in R2015) is calculating some of the variables of a system of linear equations with ~30 variables in total. To make it faster I didn't want to use a numeric solver but found the symbolic solutions. Problematic: the solution for 1 variable is >2 Million figures long even when using variable names with only one character. In a first step to make the code more manageable I found repeating patterns and replaced them. So the code looks something like:

Replace(:,1)= -D.*R.^2.*U.*n.*o.*p.*v+D.*S.^2.*U.*n.*o.*p.*w+R.^2.*U.*W.*n.*o.*p.*v %...
Replace(:,2)= M.*R.*W.*d.*n.*o.*p-.*M.*R.*n.*o.*p.*w+D.*M.*S.*n.*o.*p.*v %...
Result= Replace(:,1) .*V.*d.*n.*o-Replace(:,1).*O.^2.*S.*d.*o.*p.*w+ Replace(:,2) %...

A-Z & a-z being column vectors with double precision (double is necessary since some of the intermittend solutions get >3.4*10^38).

Does anyone have an idea why the code is running so much faster in R2014 and how I can achieve the same speed in R2015? If need be I’ll try to export the portion as C-code but I’m not too sure if that will resolve the issue(and it would be preferred to only use “direct” Matlab code).

Thanks for your help.

[John D'Errico]

No, exporting as C will almost never make things significantly faster.

And these are not basic arithmetic calculations. These seem to be symbolic computations that you are describing.

Or did you do the computation symbolically in advance, and then use all doubles in there? This won't gain you any speed, compared to solving a linear system with only 30 variables, just using backslash. I assume that you did not try to write your own gaussian elimination code? That is a really bad idea, if that is how you wanted to solve the linear system. I've even seen people think they need to use Cramer's rule and determinants to solve linear systems of equations.

And using a symbolic solution won't gain you anything in terms of the numerical computations. They will still be subject to numerical problems if you had them with backslash.

So it is not at all clear what you are doing here. It is not obvious why you thought that linear systems of equations with 30 variables would be slow to solve.

[Desoxyribose]

First of all thanks for your input.

With converting to C-I was simply hoping to maintain the R2014 characteristics since the speed there is fine (although if we can find an even better solution in this discussion that would be even better).

I'll try to clarify:

I did the symbolic computation in advance. Solved symbolically for the variables that I need. (One of) The resulting formula is the one with ~2M figures. I found the symbolic solution by using the "linsolve"-function for the first 26 variables (all 30 wasn't possible since it would take indefinately). After that to get to the end result I "manually" inserted the "semisolutions" and isolated one of the remaining "unsolved" variables in the 4 equations that were left. Then inserted one of them (the shortest) thus annihilating one of the remaining ones.

I do believe that my symbolic solution even though that it is that immensly long is the shortest (multiplied out) solution with no douplications. If you do know of a working simplification method though I'd happily try it out.

Yes all the inputs are doubles.

Could you elaborate on "backslash"? English sin't my mother tongue I only know backslash as "\".

When running the code

Result= Replace(:,1) .*V.*d.*n.*o-Replace(:,1).*O.^2.*S.*d.*o.*p.*w+ Replace(:,2) .*U.*W.*n.*o.*p.*v+ Replace(:,2) .*U.*E.*b.*o.*p.*v

would simply become: Result = [17.846;...].*[5;...].*[3;...].*[2;...]-[17.846;...].*[7;...].*[3;...].*[8;...]

[Desoxyribose]

Ah also about why I thought it would be slow to solve regularly:

I tried it with simply giving the numeric values to the linsolve function. It works but is quite a lot slower in R2014. The break even point (for the pc I am using) is when I let it calculate 600 values (so A-Z &a-z are 600x1 matrices). The program is supposed to do 10-40000 variants at once though which is why I used the "pre solving symbolically once, easy arithmetic operations in the "end product" approach.

I'm also simply intrigued by the situation since a newer version of matlab seems to perform a lot poorer in basic operations. That's weird. So I assume it rather has to do something with "loading the long formula" into the cpu (yes I know that's not how it works) than the "+-*/" itself.

[John D'Errico]

R2017 is not orders of magnitude slower when doing floating point computations than R2014. But what exactly you are doing is not clear, or what is causing the slowdown. It is also not clear what you mean by 10-40000 variants. I assume that means you are solving 10-40000 linear systems. I think the issue here is you are describing what you did far too vaguely for us to know what is wrong, but also for us to know if you really understand how to solve such systems efficiently.

The massive symbolic computations that you have done are often the source of significant numerical problems. What is mathematically correct need not be even close to being numerically any good at all. Your comment, for example, of some intermediate computations being on the order of 1e38 points out that the computations you do may be creating numerical garbage, perhaps without you realizing it.

I would suggest learning about the profile tools, to see where the time is being taken.

[Desoxyribose]

Thanks again.

Please excuse my inaccuracies. You are correct when assuming that I do not know how to solve a system of equations "numerically good". By finding the symbolic solution I found the mathematically correct solution which surely contains quite a few unnecessary/irrelevant term.

Yes with 10-40000 variations I meant that the column vectors that get inserted into the formula have a size of up to 40000x1. Which would be equivalent of solving the system of ~30 variables up to 40000 times with scalar inputs.

By using the profiler (and simply advancing code line by line) I was able to track the problem down to the lines that I provided in my original question, but thank you for the suggestion.

I will try the workaround with C-Code. Should that also fail I'll try to "numerically improve" my solution. Do you have any suggestions on that? How would you try to quickly solve such an system of equations? The target time is ~0.02s per set of data (so roughly 15min for the whole lot of 40k variations).

I know it is stupid to ask such a general question but unfortunately I can't provide the original code.

[Christine Tobler]

The problem with using a symbolic solution to a linear system is not so much that unnecessary terms are used: You risk losing a lot of precision in the intermediate steps of the calculation. So a formula that is mathematically accurate may well be numerically unsafe.

Have you tried solving the linear system directly in MATLAB? As John has said, I would expect that to be much faster (and more accurate) than evaluating the very long symbolic solution. To solve a linear system A*x=b, if matrix A and vector b are available as numeric (not symbolic) variables, call

x = A \ b;

If you have thousands of such systems, the best way is probably to just wrap a for-loop around this statement, but if A stays the same and only b changes, you can replace b1, b2, b3 with a matrix B = [b1 b2 b3], and solve all in one go using A \ B.

About the difference in performance from R2014 to R2015: In R2015b, the old just-in-time compiler (JIT) was replaced with the new execution engine. This has made many cases faster, but it's possible that the JIT was faster for your case.

[Walter Roberson]

"but it's possible that the JIT was faster for your case."

Interesting... Christine, is the implication that currently MATLAB uses something that is no longer called just-in-time compiler (JIT) ? If so then is the new term "executation engine" ?

"JIT" at least gave us a hint as to how the execution was being handled; is it no longer "just in time" compiling, or is it still "just in time" technology but a different name ?

[Desoxyribose]

Thanks for the suggestions, Christine. The accuracy is not an issue (simulating the system gives almost similar results). But you are definitely right accuracy has to be reduced due to all the operations.

To be honest I did not try to solve it directly via A\b (now I know what John meant with backslash) even though it is of course the "cleanest". Reason for that was that I had a predecessor software that used a similar approach as I did. When that initially didn't work I used linsolve with numeric values which should do roughly the same if I understand the documentation correctly (maths isn't my favourite subject to talk about). That was faster for a few hundred data sets but when the column vectors were long enough to really shine it was slower. I'll have to loop since A is also changing.

I'll definitely take a look at it in the next 48-72h. I have a strong feeling that that could be the solution for my problem. I'll leave the question open until then in case I have some follow up questions. I hope that is okay.

The compiler change could be the explanation for the initial problem but I guess there is nothing I could do about that then.

[Philip Borghesani]

Walter,

The new Execution engine is still a JIT. The official description describes it quite well. Internally we refer to the old engine as the JIT and the new system as the Execution Engine or by an internal acronym. Occasionally these naming systems leak out or cause confusion.

I will guess that in this case the code is being executed with an eval statement or as a script, not in a function, and that is a significant factor in the performance difference.

It is also possible that the performance difference is due to compilation time not execution time. What is the time difference on the second or 100th call to the code?

[Walter Roberson]

Thanks for the clarification, Philip.

[Desoxyribose]

Philip, the code is in a function and does not contain an eval().

Anyways I shamefully have to admit: The first suggestions from John (which I didn't understand at that time) and then explained by Christine of simply using "backslash" did proof to be faster by a magnitude than my code even in R2014. When I tried it before my "symbolic solve" approach I must have had an (or some) errors) in the original equations which is why I stubbornly proclaimed it to be slower (it was some while ago so I don't remember everything 100%).

Anyways: Thank you everybody. You helped me find a great solution to my problem and I have learned several things along the way (one of it being that a predecessor doesn't always have to be right/taken the best approach). Since only Walters comments is listed as "Answer" I'll accept his answer since I think the hint with the compiler change from 2014 to 2015 might be useful for others to see.

Answer 1

[Walter Roberson]

"In a first step to make the code more manageable I found repeating patterns and replaced them."

If you use matlabFunction() with the 'file' option naming a file, then by default 'Optimize' is true for that case, and MATLAB will spend time looking for repeated computations and making appropriate temporary variables.

The resulting code is not typically all that easy to read, but it does tend to be about as efficient as you could get. (Well, other than the fact that it does not always optimize calculations of .^ to higher integer powers as best possible.)

The drawback of this is that it can take a fair bit of time. I have seen it take over an hour for longer expressions. For a calculation such as you describe it would not surprise me if it would take over an hour.

The revised calculation is equivalent algebraically, but not numerically. Numerically,

A + B*C + D

is not the same as

 A + D + B*C

However, when you are using symbolic expressions, the order that it uses internally is based upon some arbitrary sorting rules that are difficult to work out, so the order is already numerically suspect even if you do not turn on optimization.