Matlab cast before calculation rule

I have been tracing some floating point precision issue in my design. Found out, mixing single and double types are sometime confusing. Is there are general rule for cast before math or there is certain property function can set global rule.

I found out 2 scenarios in my design:

The final results are all double,

1. [double vector] = [single vector] + [double vector], the double is cast to single before the math is performed (simulink fcn function)

2. [single vector] = [single vector] + [double vector];

[double_vector] = [single vector];

the double is not cast to single before the math is performed (matlab code)

Example:

 A = 192;
 B = A - 7.165700000000015;  % Original B value is single type not representable, the math seems to give the B value correctly. eps(B) = 2.842170943040401e-14
 C = A - B 
 ans =
    7.165700000000015

 double(single(A) - B) % this is neither single - single nor double - double.
 ans = 
    7.165699958801270 % my matlab code gets this

 double(single(A) - single(B))
 ans = 
    7.165695190429688 % my simulin fcn code gives this

Please comment,

especially if I need fcn code to behavior exactly as matlab code how to prevent B from been cast to single before doing the math?

[James Tursa]

Instead of listing pseudo-code as above, please post actual m-code. The rules can vary depending on exactly what you are doing.

Answer 1

[James Tursa]

I will caveat everything below with the caution that the JIT can sometimes optimize things differently than you might expect. E.g., it is my understanding that single class constants in parsed m-files can be stored as double class constants under some circumstances. Since I don't know the JIT rules (they are unpublished and change from release to release) I will not attempt to describe them. But they can result in answers at the command prompt level being different from the same code in an m-file.

   A = 192;
   B = A - 7.165700000000015;  % Original B value is single type ...

In the above, the original type of B is irrelevant. Since this is an assignment into B, and not indexed on the left-hand-side, B will simply be replaced with whatever the result of the right-hand-side is. Note that the result would have been different if B started out as a single class variable and you had done this (indexed on the left-hand-side which forces a type conversion in the assignment):

   B(1) = A - 7.165700000000015;

-----------------------------------------------------------------------

   C = A - B

The above is just a simple double - double calculation, since both A and B are double.

-----------------------------------------------------------------------

   double(single(A) - B) % this is neither single - single nor double - double.
   ans = 
      7.165699958801270 % my matlab code gets this

The above is calculated as follows:

double(single(double(single(A)) - B))

That is, the single - double calculation is actually done in double (or perhaps 80-bit) behind the scenes by first converting the single operand to double and doing the subtraction. Then that intermediate result is converted back to single. Then you explicitly turn that result back into a double for the final answer.

-------------------------------------------------------------------------

CAUTION: The above results may only be valid for scalar operations. I have noticed that for vector or matrix operations the JIT choice for converting what to what can differ from the scalar case. So beware of generalizations here.

You might find this FEX submission helpful when doing these types of investigations:

http://www.mathworks.com/matlabcentral/fileexchange/22239-num2strexact-exact-version-of-num2str

[legendbb]

Thank you very much for your insightful explanation.