matlab digit precision is not correct?
Ältere Kommentare anzeigen
Quoting from:
By default, MATLAB® uses 16 digits of precision.
But why when I write 900000000+2 (8 zeros after 9) it returns 900000002 but writing 900000000+2 (9 zeros after 9)returns 9.0000e+09 isn't this an 8 digit precision?
Antworten (1)
James Tursa
am 7 Apr. 2020
Bearbeitet: James Tursa
am 7 Apr. 2020
That is just a display artifact. The entire number is still there in memory. Do this to change the display format
format longg
and then repeat your test.
1 Kommentar
Ahmad Agbaria
am 8 Apr. 2020
Kategorien
Mehr zu Logical finden Sie in Hilfe-Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!