matlab digit precision is not correct?

Quoting from:
By default, MATLAB® uses 16 digits of precision.
But why when I write 900000000+2 (8 zeros after 9) it returns 900000002 but writing 900000000+2 (9 zeros after 9)returns 9.0000e+09 isn't this an 8 digit precision?

Antworten (1)

James Tursa
James Tursa am 7 Apr. 2020
Bearbeitet: James Tursa am 7 Apr. 2020

0 Stimmen

That is just a display artifact. The entire number is still there in memory. Do this to change the display format
format longg
and then repeat your test.

Kategorien

Mehr zu Get Started with MATLAB finden Sie in Hilfe-Center und File Exchange

Gefragt:

am 7 Apr. 2020

Kommentiert:

am 8 Apr. 2020

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by