Bug in rounding functions? Who can explain? :)
1 Ansicht (letzte 30 Tage)
Ältere Kommentare anzeigen
Pavel
am 11 Nov. 2015
Bearbeitet: Stephen23
am 11 Nov. 2015
>> format long
>> 4.1*808e6
ans =
3.312800000000000e+09
>> fix(4.1*808e6)
ans =
3.312799999000000e+09
>> fix(4*808e6 + .1*808e6)
ans =
3.312800000000000e+09
0 Kommentare
Akzeptierte Antwort
Walter Roberson
am 11 Nov. 2015
>> 4.1*808e6 - 3312800000
ans =
-4.76837158203125e-07
That is, due to floating point roundoff, 4.1*808e6 falls just short of an integer. fix() is the same as floor() for positive numbers so fix() removes the fraction after 3312799999 that is not quite 1.0, exactly like how floor(1.99999999) is 1.0 not 2.0
Weitere Antworten (0)
Siehe auch
Kategorien
Mehr zu Logical finden Sie in Help Center und File Exchange
Produkte
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!