Different trig algorithms in CAS and Home?

[Dieter]

(01-04-2018 08:23 AM)chromos Wrote:  BTW is there reason for such 'low' 12 digit precision in HP Prime? Even my very old TI-58C with 4kHz CPU and 480 bytes RAM computes to 13 digits.

The TI58 and 59 work with 13-digit precision, but this does not mean you get 13-digit accuracy. Try sin(45°), cos(45°) and sqrt(0,5). Even in simple calculations the last digit usually is truncated instead of correctly rounded: try 2/3, which yields 0,6666666666666 instead of ...67.

The Prime (like other 12-digit HP calculators) internally works with 15 (!) digits. The result is finally rounded to 12 digits and then displayed. This way the returned value usually is exact in all digits. OK, even this way there are cases where the last digit may be slightly off. Avoiding this would require an even higher internal precision.

Dieter