Casio FX991EX

[Albert Chan]

Quote:I'm not sure the calculator really has an internal 15-digit precision answer, as simple calculations
such as 1e-12+1-1 = 1e-12 versus 1e-13+1-1 = 0 suggests it only has 13 digits.

That was Casio trying to be "smart", by flushing close-to-zero to zero

http://people.ku.edu/~wzhuang/CAMseminar...ecrets.pdf

I was unable to figure out what the algorithm is:

Casio fx-3650p:
1e10 + 0.94 - 1e10 ==> 0
1e10 + 0.95 - 1e10 ==> 1

Casio fx-260 Solar
1e10 + 0.79 - 1e10 ==> 0
1e10 + 0.80 - 1e10 ==> 0.8
1e10 + 0.89 - 1e10 ==> 0.8
1e10 + 0.90 - 1e10 ==> 0.9

I really hate this feature. (above look like a bug to me)

With it on, 1e-13+1-1 = 0, can fx-991EX still claim 15-digits precision ?