PROBLEM WITH PI π

[bxparks]

The first time I used a TI graphing calculator was a few months ago, and I typed the following the TI-84+SE:

Code:


1 / 3 ENTER # shows .3333333333, 10 digits
* 3 ENTER # shows 1
- 1 ENTER # shows 0

I was shocked, because getting a "0" is impossible for a finite precision calculator (using base-2 or base-10) so I thought the calculator was broken. The TI calculator, I learned later, uses a BCD floating point representation that holds 14 significant digits. It is apparently doing some internal rounding and error truncation that I don't fully understand, and it actually makes me slightly *less* trusting of the results shown on these calculators.