Possible bug on HP50G with very large numbers

[Wes Loewer]

(11-02-2019 04:11 PM)Gerald H Wrote:  For input
234070
your programme gives an incorrect answer.

The program gives 5.36934575514 instead of 5.36934575513, which I think is as good as a UserRPL can get using real numbers. The correct rounding would require sysRPL's extended reals (or a more sophisticated program).

This example is an interesting one. The answer to 16 decimal places is 5.369345755134671 which when rounded to 12 significant digits is 5.36934575513. The program calculates -0.630654244865 + 6 which would have been 5.369345755135 but is rounded to 5.36934575514. It's the classic case where having fewer significant digits happens to give a better answer than having more.

Code:

5.36934575513     12 significant digits
5.369345755135    13 significant digits, rounds to 5.36934575514
5.3693457551347   14 significant digits, rounds to 5.36934575513 again