HP Prime Miscalculating

[Wes Loewer]

(11-01-2015 02:05 PM)Vtile Wrote:  If so why the UI is not designed to solve the issues related to engine, like that it can not solve the most simplistic elementary school level calculus correctly, makes me wonder.

Well, that's the nature of any numeric calculation which uses finite precision. Since the values are not stored exactly, you cannot expect to get exact results. The fact that ((4.0/3.0-1.0)*3.0-1.0) does not result in zero does not mean that the software is faulty. The "incorrect" result is simply an artifact of the way numbers are stored.

Here are a few different results:
Binary Storage:
Prime-CAS: -1.42108547152e-14 = -(2^-46)
MS Excel: -2.22045E-16 = -(2^-52)
Maxima (50 decimals): -2.67276...e-51 = -(2^-168)

Decimal Storage:
Prime-Home: -1.0e-11
HP-50g: : -1.0e-11
TI-Npsire: -1.0e-13

(11-01-2015 02:05 PM)Vtile Wrote:  SIN(Pi/2) should be precisely 0 and the answer...

You probably meant COS(Pi/2)=0 or SIN(Pi) = 0. (SIN(Pi/2) = 1) This is a good example how how some calculators "lie" when they give you what you expect, not what the calculated result really is.

On the ti calculators, the numeric value of pi is 3.1415926535898. The actual result of sin(3.1415926535898) is about -6.76e-15, but when the calculator sees this special value, it returns the value 0 because that's what students expect/want to see.

In other words, sometimes calculators attempt to hide the artifacts of finite precision. That's probably good for students, but it does drive numerical purists crazy.

A simpler example is 1.0/3.0*3.0 . The exact result is 1 of course, but a finite precision calculation will result in something like 0.999999999 .