rounding down using floor
2 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
so im using:
floor(10^decimal*(total/AVERAGE_COUNT))/10^decimal;
where:
decimal = 4, AVERAGE_COUNT = 9, total = 8.9289
this entry gives me 0.9920, however the answer should be 0.9921
furthermore, when I just do
total/AVERAGE_COUNT;
it gives me 0.9921 which is correct but i need to use the floor function for the rest of my data.
how do i get around this
3 Kommentare
per isakson
am 1 Sep. 2020
On my R2018b, Win10 this script
%%
format long
decimal = 4; AVERAGE_COUNT = 9; total = 8.9289;
x = floor(10^decimal*(total/AVERAGE_COUNT))/10^decimal
outputs
x =
0.992100000000000
>>
Antworten (1)
Dana
am 1 Sep. 2020
I'm on R2020a on Windows 10 and I also get 0.9921.
Is it possible that total does not equal exactly 8.9289? For example, if total actually equals 8.92889 (which would usually display as 8.9289 in the command window), then you'll end up with 0.9920.
0 Kommentare
Siehe auch
Kategorien
Mehr zu Holidays / Seasons finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!