Meaningful absolute difference for error calculation, absolute forward error.

Looking at a function f(x) and reference function ref_f(x) considered to be true (golden sample, the reference)

As definition the absolute difference is abs(f(x) - ref_f(x)) or absolute forward error.

(The forward error is a measure of the difference between the approximation and the true value of a function.)

(The forward error of the algorithm is the difference between the result and the solution.)

To be meaningful the resulted value should be finite, and plottable.

• If absolute difference is nan or inf the result should be real max

Also there are some issues with IEEE754 limitations NAN and Inf values that should considered:

• If both functions are equal to NAN than, there is no difference between functions, so the error might be zero.
• If both functions are equal to Inf than, there is no difference between functions, so the error might be zero, but if inf has different signs the error is max.
• nextafter() and nexttoward() float numbers should generate the correct difference (eps)
• rounding issues
• guard digits
• cancellation

Is there a MATALAB implementation (build in) for this function ?

like fae, abs_err, absDifErr ..

https://en.wikipedia.org/wiki/Approximation_error

https://en.wikipedia.org/wiki/Absolute_difference

https://en.wikipedia.org/wiki/Numerical_stability

Math 515 Fall, 2012 Brief Notes on Conditioning, Stability and Finite Precision Arithmetic

www.mathcs.emory.edu/.../Conditioning_and_FLOPS.pdf

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

https://www.boost.org/doc/libs/1_39_0/libs/math/doc/sf_and_dist/html/math_toolkit/toolkit/internals2/error_test.html

http://www.cplusplus.com/reference/cmath/nextafter/

Tried an implementation like (is this the fastest way?) :

function [absErr] = absDifErr_single(V1, V2)

    if and( isfinite(V1),  isfinite(V2))
        absErr = abs(V1 - V2);
        
        if ~isfinite(absErr)
            absErr = realmax('single');
        end
    else
    % special zone    
        if or( isfinite(V1),  isfinite(V2)) 
            % at least one is finte other nan/inf
            absErr = realmax('single');
        else
            if isinf(V1) % V1 inf
                if isinf(V2) % V1 inf V2 inf
                    if sign(V1)==sign(V2) % inf inf same sign
                        absErr = 0;
                    else  % inf inf diferent sign
                        absErr = realmax('single');
                    end
                else % inf nan
                    absErr = realmax('single');
                end
            else % nan ( nan or inf)   
                if isinf(V2) % nan inf
                    absErr = realmax('single');
                else % nan nan
                     absErr = 0;
                end                    
            end            
        end
    end
end

Tried a test on it:

close all
clear all
tic

% nan nan
disp(['abs(+nan - +nan) '  num2str(abs(+nan - +nan)) ', Absolute difference error ' num2str(absDifErr_single(+nan, +nan)) ])
disp(['abs(-nan - -nan) '  num2str(abs(-nan - -nan)) ', Absolute difference error ' num2str(absDifErr_single(-nan, -nan)) ])
disp(['abs(+nan - -nan) '  num2str(abs(+nan - -nan)) ', Absolute difference error ' num2str(absDifErr_single(+nan, -nan)) ])
disp(['abs(-nan - +nan) '  num2str(abs(-nan - +nan)) ', Absolute difference error ' num2str(absDifErr_single(-nan, +nan)) ])
% inf inf
disp(['abs(+inf - +inf) '  num2str(abs(+inf - +inf)) ', Absolute difference error ' num2str(absDifErr_single(+inf, +inf)) ])
disp(['abs(-inf - -inf) '  num2str(abs(-inf - -inf)) ', Absolute difference error ' num2str(absDifErr_single(-inf, -inf)) ])
disp(['abs(+inf - -inf) '  num2str(abs(+inf - -inf)) ', Absolute difference error ' num2str(absDifErr_single(+inf, -inf)) ])
disp(['abs(-inf - +inf) '  num2str(abs(-inf - +inf)) ', Absolute difference error ' num2str(absDifErr_single(-inf, +inf)) ])

% inf nan
disp(['abs(+inf - +nan) '  num2str(abs(+inf - +nan)) ', Absolute difference error ' num2str(absDifErr_single(+inf, +nan)) ])
disp(['abs(-inf - -nan) '  num2str(abs(-inf - -nan)) ', Absolute difference error ' num2str(absDifErr_single(-inf, -nan)) ])
disp(['abs(+inf - -nan) '  num2str(abs(+inf - -nan)) ', Absolute difference error ' num2str(absDifErr_single(+inf, -nan)) ])
disp(['abs(-inf - +nan) '  num2str(abs(-inf - +nan)) ', Absolute difference error ' num2str(absDifErr_single(-inf, +nan)) ])
% nan inf
disp(['abs(+nan - +inf) '  num2str(abs(+nan - +inf)) ', Absolute difference error ' num2str(absDifErr_single(+nan, +inf)) ])
disp(['abs(-nan - -inf) '  num2str(abs(-nan - -inf)) ', Absolute difference error ' num2str(absDifErr_single(-nan, -inf)) ])
disp(['abs(+nan - -inf) '  num2str(abs(+nan - -inf)) ', Absolute difference error ' num2str(absDifErr_single(+nan, -inf)) ])
disp(['abs(-nan - +inf) '  num2str(abs(-nan - +inf)) ', Absolute difference error ' num2str(absDifErr_single(-nan, +inf)) ])

% nan normals
disp(['abs(+nan - +5) '  num2str(abs(+nan - +5)) ', Absolute difference error ' num2str(absDifErr_single(+nan, +5)) ])
disp(['abs(-nan - -5) '  num2str(abs(-nan - -5)) ', Absolute difference error ' num2str(absDifErr_single(-nan, -5)) ])
disp(['abs(+nan - -5) '  num2str(abs(+nan - -5)) ', Absolute difference error ' num2str(absDifErr_single(+nan, -5)) ])
disp(['abs(-nan - +5) '  num2str(abs(-nan - +5)) ', Absolute difference error ' num2str(absDifErr_single(-nan, +5)) ])
% normals nan
disp(['abs(+5 - +nan) '  num2str(abs(+5 - +nan)) ', Absolute difference error ' num2str(absDifErr_single(+5, +nan)) ])
disp(['abs(-5 - -nan) '  num2str(abs(-5 - -nan)) ', Absolute difference error ' num2str(absDifErr_single(-5, -nan)) ])
disp(['abs(+5 - -nan) '  num2str(abs(+5 - -nan)) ', Absolute difference error ' num2str(absDifErr_single(+5, -nan)) ])
disp(['abs(-5 - +nan) '  num2str(abs(-5 - +nan)) ', Absolute difference error ' num2str(absDifErr_single(-5, +nan)) ])

% inf normals
disp(['abs(+inf - +5) '  num2str(abs(+inf - +5)) ', Absolute difference error ' num2str(absDifErr_single(+inf, +5)) ])
disp(['abs(-inf - -5) '  num2str(abs(-inf - -5)) ', Absolute difference error ' num2str(absDifErr_single(-inf, -5)) ])
disp(['abs(+inf - -5) '  num2str(abs(+inf - -5)) ', Absolute difference error ' num2str(absDifErr_single(+inf, -5)) ])
disp(['abs(-inf - +5) '  num2str(abs(-inf - +5)) ', Absolute difference error ' num2str(absDifErr_single(-inf, +5)) ])
% normals inf
disp(['abs(+5 - +inf) '  num2str(abs(+5 - +inf)) ', Absolute difference error ' num2str(absDifErr_single(+5, +inf)) ])
disp(['abs(-5 - -inf) '  num2str(abs(-5 - -inf)) ', Absolute difference error ' num2str(absDifErr_single(-5, -inf)) ])
disp(['abs(+5 - -inf) '  num2str(abs(+5 - -inf)) ', Absolute difference error ' num2str(absDifErr_single(+5, -inf)) ])
disp(['abs(-5 - +inf) '  num2str(abs(-5 - +inf)) ', Absolute difference error ' num2str(absDifErr_single(-5, +inf)) ])

% normals normals equal
disp(['abs(+5 - +5) '  num2str(abs(+5 - +5)) ', Absolute difference error ' num2str(absDifErr_single(+5, +5)) ])
disp(['abs(-5 - -5) '  num2str(abs(-5 - -5)) ', Absolute difference error ' num2str(absDifErr_single(-5, -5)) ])
disp(['abs(+5 - -5) '  num2str(abs(+5 - -5)) ', Absolute difference error ' num2str(absDifErr_single(+5, -5)) ])
disp(['abs(-5 - +5) '  num2str(abs(-5 - +5)) ', Absolute difference error ' num2str(absDifErr_single(-5, +5)) ])

% normals next normals
a = typecast(uint32(hex2dec('40a00000')),'single'); % 5
b = typecast(uint32(hex2dec('40a00001')),'single'); % next 5 -> 5.000000476837158203125
disp(['abs(+5 - +5next) '  num2str(abs(+a - +b)) ', Absolute difference error ' num2str(absDifErr_single(+a, +b)) ])
disp(['abs(-5 - -5next) '  num2str(abs(-a - -b)) ', Absolute difference error ' num2str(absDifErr_single(-a, -b)) ])
disp(['abs(+5 - -5next) '  num2str(abs(+a - -b)) ', Absolute difference error ' num2str(absDifErr_single(+a, -b)) ])
disp(['abs(-5 - +5next) '  num2str(abs(-a - +b)) ', Absolute difference error ' num2str(absDifErr_single(-a, +b)) ])

% normals after normals
a = typecast(uint32(hex2dec('40a00000')),'single'); % 5
b = typecast(uint32(hex2dec('409fffff')),'single'); % after 5 -> 4.999999523162841796875
disp(['abs(+5 - +5after) '  num2str(abs(+a - +b)) ', Absolute difference error ' num2str(absDifErr_single(+a, +b)) ])
disp(['abs(-5 - -5after) '  num2str(abs(-a - -b)) ', Absolute difference error ' num2str(absDifErr_single(-a, -b)) ])
disp(['abs(+5 - -5after) '  num2str(abs(+a - -b)) ', Absolute difference error ' num2str(absDifErr_single(+a, -b)) ])
disp(['abs(-5 - +5after) '  num2str(abs(-a - +b)) ', Absolute difference error ' num2str(absDifErr_single(-a, +b)) ])

% normals normals unequal
disp(['abs(+5 - +6) '  num2str(abs(+5 - +6)) ', Absolute difference error ' num2str(absDifErr_single(+5, +6)) ])
disp(['abs(-5 - -6) '  num2str(abs(-5 - -6)) ', Absolute difference error ' num2str(absDifErr_single(-5, -6)) ])
disp(['abs(+5 - -6) '  num2str(abs(+5 - -6)) ', Absolute difference error ' num2str(absDifErr_single(+5, -6)) ])
disp(['abs(-5 - +6) '  num2str(abs(-5 - +6)) ', Absolute difference error ' num2str(absDifErr_single(-5, +6)) ])

% realmax('single')
a = realmax('single');
disp(['abs(+rM - +rM) '  num2str(abs(+a - +a)) ', Absolute difference error ' num2str(absDifErr_single(+a, +a)) ])
disp(['abs(-rM - -rM) '  num2str(abs(-a - -a)) ', Absolute difference error ' num2str(absDifErr_single(-a, -a)) ])
disp(['abs(+rM - -rM) '  num2str(abs(+a - -a)) ', Absolute difference error ' num2str(absDifErr_single(+a, -a)) ])
disp(['abs(-rM - +rM) '  num2str(abs(-a - +a)) ', Absolute difference error ' num2str(absDifErr_single(-a, +a)) ])

% realmin('single')
a = realmin('single');
disp(['abs(+rm - +rm) '  num2str(abs(+a - +a)) ', Absolute difference error ' num2str(absDifErr_single(+a, +a)) ])
disp(['abs(-rm - -rm) '  num2str(abs(-a - -a)) ', Absolute difference error ' num2str(absDifErr_single(-a, -a)) ])
disp(['abs(+rm - -rm) '  num2str(abs(+a - -a)) ', Absolute difference error ' num2str(absDifErr_single(+a, -a)) ])
disp(['abs(-rm - +rm) '  num2str(abs(-a - +a)) ', Absolute difference error ' num2str(absDifErr_single(-a, +a)) ])

% realmax realmin
a = realmax('single');
b = realmin('single');
disp(['abs(+rM - +rm) '  num2str(abs(+a - +b)) ', Absolute difference error ' num2str(absDifErr_single(+a, +b)) ])
disp(['abs(-rM - -rm) '  num2str(abs(-a - -b)) ', Absolute difference error ' num2str(absDifErr_single(-a, -b)) ])
disp(['abs(+rM - -rm) '  num2str(abs(+a - -b)) ', Absolute difference error ' num2str(absDifErr_single(+a, -b)) ])
disp(['abs(-rM - +rm) '  num2str(abs(-a - +b)) ', Absolute difference error ' num2str(absDifErr_single(-a, +b)) ])

% realmin realmax
b = realmax('single');
a = realmin('single');
disp(['abs(+rm - +rM) '  num2str(abs(+a - +b)) ', Absolute difference error ' num2str(absDifErr_single(+a, +b)) ])
disp(['abs(-rm - -rM) '  num2str(abs(-a - -b)) ', Absolute difference error ' num2str(absDifErr_single(-a, -b)) ])
disp(['abs(+rm - -rM) '  num2str(abs(+a - -b)) ', Absolute difference error ' num2str(absDifErr_single(+a, -b)) ])
disp(['abs(-rm - +rM) '  num2str(abs(-a - +b)) ', Absolute difference error ' num2str(absDifErr_single(-a, +b)) ])

% zero
a = single(0);
disp(['abs(+0 - +0) '  num2str(abs(+a - +a)) ', Absolute difference error ' num2str(absDifErr_single(+a, +a)) ])
disp(['abs(-0 - -0) '  num2str(abs(-a - -a)) ', Absolute difference error ' num2str(absDifErr_single(-a, -a)) ])
disp(['abs(+0 - -0) '  num2str(abs(+a - -a)) ', Absolute difference error ' num2str(absDifErr_single(+a, -a)) ])
disp(['abs(-0 - +0) '  num2str(abs(-a - +a)) ', Absolute difference error ' num2str(absDifErr_single(-a, +a)) ])

% nan zero
a = single(0);
disp(['abs(+nan - +0) '  num2str(abs(+nan - +a)) ', Absolute difference error ' num2str(absDifErr_single(+nan, +a)) ])
disp(['abs(-nan - -0) '  num2str(abs(-nan - -a)) ', Absolute difference error ' num2str(absDifErr_single(-nan, -a)) ])
disp(['abs(+nan - -0) '  num2str(abs(+nan - -a)) ', Absolute difference error ' num2str(absDifErr_single(+nan, -a)) ])
disp(['abs(-nan - +0) '  num2str(abs(-nan - +a)) ', Absolute difference error ' num2str(absDifErr_single(-nan, +a)) ])

% inf zero
a = single(0);
disp(['abs(+inf - +0) '  num2str(abs(+inf - +a)) ', Absolute difference error ' num2str(absDifErr_single(+inf, +a)) ])
disp(['abs(-inf - -0) '  num2str(abs(-inf - -a)) ', Absolute difference error ' num2str(absDifErr_single(-inf, -a)) ])
disp(['abs(+inf - -0) '  num2str(abs(+inf - -a)) ', Absolute difference error ' num2str(absDifErr_single(+inf, -a)) ])
disp(['abs(-inf - +0) '  num2str(abs(-inf - +a)) ', Absolute difference error ' num2str(absDifErr_single(-inf, +a)) ])

toc

Outputs:

abs(+nan - +nan) NaN, Absolute difference error 0

abs(-nan - -nan) NaN, Absolute difference error 0

abs(+nan - -nan) NaN, Absolute difference error 0

abs(-nan - +nan) NaN, Absolute difference error 0

abs(+inf - +inf) NaN, Absolute difference error 0

abs(-inf - -inf) NaN, Absolute difference error 0

abs(+inf - -inf) Inf, Absolute difference error 3.402823466385289e+38

abs(-inf - +inf) Inf, Absolute difference error 3.402823466385289e+38

abs(+inf - +nan) NaN, Absolute difference error 3.402823466385289e+38

abs(-inf - -nan) NaN, Absolute difference error 3.402823466385289e+38

abs(+inf - -nan) NaN, Absolute difference error 3.402823466385289e+38

abs(-inf - +nan) NaN, Absolute difference error 3.402823466385289e+38

abs(+nan - +inf) NaN, Absolute difference error 3.402823466385289e+38

abs(-nan - -inf) NaN, Absolute difference error 3.402823466385289e+38

abs(+nan - -inf) NaN, Absolute difference error 3.402823466385289e+38

abs(-nan - +inf) NaN, Absolute difference error 3.402823466385289e+38

abs(+nan - +5) NaN, Absolute difference error 3.402823466385289e+38

abs(-nan - -5) NaN, Absolute difference error 3.402823466385289e+38

abs(+nan - -5) NaN, Absolute difference error 3.402823466385289e+38

abs(-nan - +5) NaN, Absolute difference error 3.402823466385289e+38

abs(+5 - +nan) NaN, Absolute difference error 3.402823466385289e+38

abs(-5 - -nan) NaN, Absolute difference error 3.402823466385289e+38

abs(+5 - -nan) NaN, Absolute difference error 3.402823466385289e+38

abs(-5 - +nan) NaN, Absolute difference error 3.402823466385289e+38

abs(+inf - +5) Inf, Absolute difference error 3.402823466385289e+38

abs(-inf - -5) Inf, Absolute difference error 3.402823466385289e+38

abs(+inf - -5) Inf, Absolute difference error 3.402823466385289e+38

abs(-inf - +5) Inf, Absolute difference error 3.402823466385289e+38

abs(+5 - +inf) Inf, Absolute difference error 3.402823466385289e+38

abs(-5 - -inf) Inf, Absolute difference error 3.402823466385289e+38

abs(+5 - -inf) Inf, Absolute difference error 3.402823466385289e+38

abs(-5 - +inf) Inf, Absolute difference error 3.402823466385289e+38

abs(+5 - +5) 0, Absolute difference error 0

abs(-5 - -5) 0, Absolute difference error 0

abs(+5 - -5) 10, Absolute difference error 10

abs(-5 - +5) 10, Absolute difference error 10

abs(+5 - +5next) 4.7684e-07, Absolute difference error 4.7684e-07

abs(-5 - -5next) 4.7684e-07, Absolute difference error 4.7684e-07

abs(+5 - -5next) 10, Absolute difference error 10

abs(-5 - +5next) 10, Absolute difference error 10

abs(+5 - +5after) 4.7684e-07, Absolute difference error 4.7684e-07

abs(-5 - -5after) 4.7684e-07, Absolute difference error 4.7684e-07

abs(+5 - -5after) 10, Absolute difference error 10

abs(-5 - +5after) 10, Absolute difference error 10

abs(+5 - +6) 1, Absolute difference error 1

abs(-5 - -6) 1, Absolute difference error 1

abs(+5 - -6) 11, Absolute difference error 11

abs(-5 - +6) 11, Absolute difference error 11

abs(+rM - +rM) 0, Absolute difference error 0

abs(-rM - -rM) 0, Absolute difference error 0

abs(+rM - -rM) Inf, Absolute difference error 3.402823466385289e+38

abs(-rM - +rM) Inf, Absolute difference error 3.402823466385289e+38

abs(+rm - +rm) 0, Absolute difference error 0

abs(-rm - -rm) 0, Absolute difference error 0

abs(+rm - -rm) 2.351e-38, Absolute difference error 2.351e-38

abs(-rm - +rm) 2.351e-38, Absolute difference error 2.351e-38

abs(+rM - +rm) 3.402823466385289e+38, Absolute difference error 3.402823466385289e+38

abs(-rM - -rm) 3.402823466385289e+38, Absolute difference error 3.402823466385289e+38

abs(+rM - -rm) 3.402823466385289e+38, Absolute difference error 3.402823466385289e+38

abs(-rM - +rm) 3.402823466385289e+38, Absolute difference error 3.402823466385289e+38

abs(+rm - +rM) 3.402823466385289e+38, Absolute difference error 3.402823466385289e+38

abs(-rm - -rM) 3.402823466385289e+38, Absolute difference error 3.402823466385289e+38

abs(+rm - -rM) 3.402823466385289e+38, Absolute difference error 3.402823466385289e+38

abs(-rm - +rM) 3.402823466385289e+38, Absolute difference error 3.402823466385289e+38

abs(+0 - +0) 0, Absolute difference error 0

abs(-0 - -0) 0, Absolute difference error 0

abs(+0 - -0) 0, Absolute difference error 0

abs(-0 - +0) 0, Absolute difference error 0

abs(+nan - +0) NaN, Absolute difference error 3.402823466385289e+38

abs(-nan - -0) NaN, Absolute difference error 3.402823466385289e+38

abs(+nan - -0) NaN, Absolute difference error 3.402823466385289e+38

abs(-nan - +0) NaN, Absolute difference error 3.402823466385289e+38

abs(+inf - +0) Inf, Absolute difference error 3.402823466385289e+38

abs(-inf - -0) Inf, Absolute difference error 3.402823466385289e+38

abs(+inf - -0) Inf, Absolute difference error 3.402823466385289e+38

abs(-inf - +0) Inf, Absolute difference error 3.402823466385289e+38

Elapsed time is 0.088918 seconds.

These values seems strange 0x40a00000 vs 0x40a00001

abs(+5 - +5next) 4.7684e-07, Absolute difference error 4.7684e-07

abs(-5 - -5next) 4.7684e-07, Absolute difference error 4.7684e-07

abs(+5 - -5next) 10, Absolute difference error 10

abs(-5 - +5next) 10, Absolute difference error 10

abs(+5 - +5after) 4.7684e-07, Absolute difference error 4.7684e-07

abs(-5 - -5after) 4.7684e-07, Absolute difference error 4.7684e-07

abs(+5 - -5after) 10, Absolute difference error 10

abs(-5 - +5after) 10, Absolute difference error 10

Symmetrical form: max( abs(f(x) - ref_f(x)) , abs(ref_f(x) - f(x)) ) is better, (is commutative) ?

I wonder that for special data math (like ARM cores) that do not support subnormals and/or

INF/NAN are treated like big numbers how this absDifErr can be parametrized ?

[Firan Lucian]

Some special cases are also treated here:

https://bitbashing.io/comparing-floats.html

// We first check if the values are NaN.
// If this is the case, they're inherently unequal;
// return the maximum distance between the two.

if (isnan(a) || isnan(b)) return max;

// If one's infinite, and they're not equal,
// return the max distance between the two.

if (isinf(a) || isinf(b)) return max;

[Firan Lucian]

other variant

function [absErr] = abs_err_SP(f_x, ref_x)
% MATLAB R2018b
% calculates absolute error for SP real value
% output is bounded to 0 realmax
% if both values identical -> error is zero ( inf==inf, -inf==-inf, nan==nan)
% if one not finite and other finite -> error is max

    % real == real, inf == inf, -inf==-inf
    if f_x == ref_x
         absErr = 0;
         return;
    % nan == nan
    elseif and(isnan(f_x), isnan(ref_x))
         absErr = 0;
         return;
    % at least one is finite, other nan/inf
    elseif or( and(isfinite(f_x), ~isfinite(ref_x)), and(~isfinite(f_x), isfinite(ref_x)))
        absErr = realmax('single');
        return;
    % one is nan the other is inf
    elseif or( and(isnan(f_x), isinf(ref_x)), and(isinf(f_x), isnan(ref_x)))
        absErr = realmax('single');
        return;
    %both different inf: -INF == INF, INF == -INF
    elseif and(isinf(f_x), isinf(ref_x))
        absErr = realmax('single');
        return;
    else
        absErr = abs(f_x - ref_x); 
        if ~isfinite(absErr)
            absErr = realmax('single');
        end        
    end      

end