Question for Parisse about CAS floating point

[Joe Horn]

(04-06-2015 01:44 PM)compsystems Wrote:  the nspire and ti68k calculators work with 12 digits, I think this should be the lower limit to improve accuracy, you say?

hp-prime calculator

approx(1519/99) => 15.3434343434 10 digits =(

That's merely what's shown in the display. The internal value in CAS is exactly:
15.343434343434267930206260643899440765380859375
Count the underlined digits.

(04-06-2015 01:44 PM)compsystems Wrote:  nspire and ti68k calculators

approx(1519/99) => 15.343434343434 12 digits =)

Notice that the Prime's value is identical to 12 decimal places, and in fact its result is closer to the actual value of 1519/99 than the competitions'. So, Prime wins that round. =)

(04-06-2015 01:44 PM)compsystems Wrote:  ////////////////

hp-prime calculator

approx(1222222/99000) => 12.3456767677 10 digits =(

That's merely what's shown in the display. The internal value in CAS is exactly:
12.3456767676767498187473393045365810394287109375
Count the underlined digits.

(04-06-2015 01:44 PM)compsystems Wrote:  nspire and ti68k calculators

approx(1222222/99000) => 12.345676767677 12 digits =)

Notice that this time Prime's internal value is even more accurate than the competitions' results... 13 accurately rounded decimal places, not just 12. =)

Random thought: Saying that Prime's accuracy in Home view is less than TI's, is like saying that a race car in low gear is slower than a motorcycle. It's true, but pointless. Yes, Home's results are limited to 12 BCD digits, but CAS results are slightly wider at 48 bits. If you want to compare Prime's accuracy with other machines, be fair and use Prime's CAS, rather than falsely implying that Prime is less accurate than other machines.

Prime's internal values shown above were obtained by a version of my "hex" program, rewritten to output the exact decimal values. I'll post it in the Prime Software Gallery later this evening.

Edit: The "dec" program is HERE.