Matrix multiplication gives different result than manual dot products with each column

N = 1000;

B = 10;

C = 2;

x = randn(N,B);

b = randn(B,C);

matMult = x * b;

dotProd = [x*b(:,1), x*b(:,2)];

isequal(matMult, dotProd) %==0

mean(matMult(:) - dotProd(:)) % approaches 0

Does anyone know why these two expressions don't give the same answer?

[Stephen23]

"I was looking for to understand the computer science behind the differences."

https://www.mathworks.com/matlabcentral/answers/57444-faq-why-is-0-3-0-2-0-1-not-equal-to-zero

https://www.mathworks.com/matlabcentral/answers/316889-why-two-equal-numbers-are-not-equal

http://www.mathworks.com/matlabcentral/answers/102419-how-do-i-determine-if-the-error-in-my-answer-is-the-result-of-round-off-error-or-a-bug

http://www.mathworks.com/matlabcentral/answers/140656-about-numerical-difficulty-of-equality-comparison-operation-between-two-double-real-numbers-in-matlab

This is highly recommended reading:

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

http://faculty.tarleton.edu/agapie/documents/cs_343_arch/papers/1991_Goldberg_FloatingPoint.pdf

[Andrew]

Thank you!

Answer 1

[Bruno Luong]

"Does anyone know why these two expressions don't give the same answer? "

Why should they? The algorithm can perform diffrent sum orders, different accumulation scheme, different thread numbers. One should not expect they to be exactly equal.

[Andrew]

From a person with thorough knowledge of linear algebra but minimal computer science, one would expect them to be identical because the two expressions (matMult & dotProd) request the same mathematical operation. Your list of algorithmic differences is what I was looking for to understand the computer science behind the differences.

[Bruno Luong]

When one works with numerical calculation on computer, one should be awared about the imperfect of finite precision aritmetics (round off error, non associativity, non commutivity, ...).

The "*" MATLAB operation is performed by Blas and Lapack library and can be differently implemented by platform and version. They are not documented by TMW because they reserved the freedom to change it (and they did serveral times in the pass).

The proof is that your code returns identical result on Linux machine R2022b (see @KSSV answer), but not on Windows (for example my laptop or your computer). It can also change with HW such as the number of cores of the CPU.

Bottomline is that use should not numerical calculation of the same mathematical expression returns the exact same answer if they call different function calls to achieve the same expression.

Answer 2

[KSSV]

As you are comparing floatting point number, you should not use isequal. You should proceed like shown below:

N = 1000;
B = 10;
C = 2;
x = randn(N,B);
b = randn(B,C);
matMult = x * b;
dotProd = [x*b(:,1), x*b(:,2)];

tol = 10^-5 ; % can be changed 
all(abs(matMult-dotProd)<tol,'all')

ans = logical
   1