why is my hp prime giving me the wrong answer

[guest2112]

imgur link here https://imgur.com/gallery/GPQizqS (file to large here). this has caused me to sit here for an hour trying to see what i did wrong for this problem then i ask my friend and her calculator gave her a different answer which was the correct answer as well as an online rref calculator, you can see for urself typing it into an RREF calculator u get a different answer than my calculator.
all my inputs for the
matrix are fractions with denominator or 3. now when i do ref of my matrix multiplied by 3 (to get rid of the fractions) it spits out the correct answer. does the hp prime have a problem with fractions? Also just figured out if i use cas mode it spits the right answer but in non-cas mode it gives the wrong answer.
correct : row 1: 1,-1,1/2 row 2: 0,0,0 row 3: 0,0,0
my answer: 1,-1,0 row 2: 0,0,1 row 3: 0,0,0

[Albert Chan]

HOME> 2/3 - 1/3*2

0.000000000001      ; = 0.666666666667 - 0.333333333333*2

Other calculators may simply fudge this to zero. (Try it!)
HOME is keeping the tiny difference, thus rref gives different result.

[john gustaf stebbins]

Guest2112, doing this in the CAS gives the result you are looking for. The CAS system does not reduce the intermediate results to floating point, so the approximations Albert highlighted are not an issue.

[Steve Simpkin]

(11-09-2024 01:41 PM)Albert Chan Wrote:  HOME> 2/3 - 1/3*2

0.000000000001      ; = 0.666666666667 - 0.333333333333*2

Other calculators may simply fudge this to zero. (Try it!)
HOME is keeping the tiny difference, thus rref gives different result.

Albert,
Most modern TI and Casio models that I tried appear to return an answer of zero for 2/3 - 1/3*2. Oddly enough the TI-89 returns a value of 1.E-14 when in Approx mode.

[Albert Chan]

TI BA II Plus: 2/3 - 1/3*2 --> 1E-13

Perhaps it is a 'business' calculator, thus no fudging numbers?

[Allen]

because ε

[C.Ret]

(11-09-2024 11:29 PM)Allen Wrote:  Because ε

Exactly Allen,
… and to all these definitions given in your reference, we must not forget to also add this one which is specific to the HP Prime! Fortuitously, the latter explains the different behavior of this calculator between its HOME and CAS modes.

[jthole]

(11-09-2024 11:29 PM)Allen Wrote:  because ε

Here's a simple program (not for the Prime, but easy to adapt) to find an approximation of the machine precision:

# ------------------------------------------------------------------------------
# HEWLETT·PACKARD 15C Simulator program
# Created with version 4.6.00
# ------------------------------------------------------------------------------
# ------------------------------------------------------------------------------

000 { }
001 { 42 21 0 } f LBL 0
002 { 1 } 1
003 { 44 0 } STO 0
004 { 42 21 .0 } f LBL .0
005 { 48 } .
006 { 5 } 5
007 { 45 20 0 } RCL × 0
008 { 44 0 } STO 0
009 { 1 } 1
010 { 40 } +
011 { 43 36 } g LSTΧ
012 { 43 30 6 } g TEST x≠y
013 { 22 .0 } GTO .0
014 { 45 0 } RCL 0
015 { 43 32 } g RTN

# ------------------------------------------------------------------------------

[Albert Chan]

Hi, C.Ret

FYI, epsilon in HP setup is not machine epsilon, which is fixed for fixed precision.
Below ± b^(-p) can be used to get whatever definition of machine epsilon needed.

Home> 1.0 - 1/3 - 1/3 - 1/3      → 0.000000000001 = 1e-12
Home> 1e-12 + 1 - 1               → 0.
Home> 1e-12 - 1 + 1               → 0.000000000001

Cas> 1.0 - 1/3 - 1/3 - 1/3         → 3.5527136788e−15 = 2^-48
Cas> 2.0^-48 + 1 - 1               → 0.
Cas> 2.0^-48 - 1 + 1               → 3.5527136788e−15 = 2^-48

lua> 1.0 - 1/3 - 1/3 - 1/3         → 1.1102230246251565e-16 = 2^-53
lua> 2^-53 + 1 - 1                  → 0.
lua> 2^-53 - 1 + 1                  → 1.1102230246251565e-16

[C.Ret]

Thanks CHAN for these details.

Indeed, with all these epsilons, it's enough to get a little lost. And I am a bit.

What is the use of this epsilon parameter I found in the second page of the CAS settings menu?
Is it for the solver?

[Albert Chan]

(11-10-2024 02:47 PM)C.Ret Wrote:  What is the use of this epsilon parameter I found in the second page of the CAS settings menu?

It is a global variable, and can be use for anything, by anyone.
What comes to mind is solver, linsolve, rref, exact (althought it does not honor setting if ε=0) ...

HP Prime User Guide is no help. What functions use ε? Compared ε to what?

Quote:Any number smaller than the value specified for epsilon will be shown as zero.

[Albert Chan]

(11-10-2024 02:17 PM)Albert Chan Wrote:  Cas> 1.0 - 1/3 - 1/3 - 1/3         → 3.5527136788e−15 = 2^-48
Cas> 2.0^-48 + 1 - 1               → 0.
Cas> 2.0^-48 - 1 + 1               → 3.5527136788e−15 = 2^-48

HP Prime Cas side float round by chopping, instead of to-nearest.
Above is just really lucky, thus the need for confirmation steps!

We got nothing if HP Prime Cas chop off 4 bits of IEEE double, instead of 5

lua> x = +0x1.5555555555550p-2 -- ieee double 1/3, chopped off 4 bits
lua> dtoh(1-x)
+0x1.5555555555558p-1
lua> dtoh(+0x1.5555555555550p-1 - x) -- (1-x) chopped off 4 bits, then subtract x
+0x1.5555555555550p-2
lua> dtoh(+0x1.5555555555550p-2 - x) -- (1-x-x) chopped off 4 bits, then subtract x
+0x0.0000000000000p+0

It would be nice if HP Prime Cas just use IEEE double. (same for XCas!)

---

With b = 3k ± 1, round-to-nearest, halfway-to-even, x = '1/3' ALWAYS have ±1/3 ULP error.
'1 - 1/3 - 1/3 - 1/3 = ± b^-p' trick will then always work.

Example, base 17, precision of 3 digits
Note: under-estimated error = exact - approx

1/3 = (17³/3) / 17³ = (1638-1/3) / 17³
x = '1/3' = 1638/17³, error(x) = -1/3 ULP of 1

1-x = (4913-1638) / 17³ = 3275 / 17³
1-x-x = (3275-1638) / 17³ = 1637 / 17³
1-x-x-x = (1637-1638) / 17³ = -1 / 17³

error(1-x-x-x) = (1/3 + 1/3 + 1/3) ULP of 1 = 1 ULP of 1

For base 2, we have to worry about 2^n boundary, but idea is the same.
Because 2<3, 1/3 need to be normalized. Again, assumed precision = 3

1/3 = ½ * (2/3) = ½ * (2/3*2³) / 2³ = ½ * (5+1/3)/2³
x = '1/3' = 2.5/2³, error(x) = +1/3 ULP of ½

1-x = (8-2.5) / 2³ = 5.5 / 2³, 5.5 (= 0b101.1) rounded up to 6 (halfway-to-even)
1-x-x = (6-2.5) / 2³ = 3.5 / 2³
1-x-x-x = (3.5-2.5) / 2³ = 1 / 2³

error(1-x-x-x) = (-1/3 - 1 - 1/3 - 1/3) ULP of ½ = -2 ULP of ½ = -1 ULP of 1

[C.Ret]

(11-10-2024 06:07 PM)Albert Chan Wrote:  

(11-10-2024 02:47 PM)C.Ret Wrote:  What is the use of this epsilon parameter I found in the second page of the CAS settings menu?

It is a global variable, and can be use for anything, by anyone.
What comes to mind is solver, linsolve, rref, exact (althought it does not honor setting if ε=0) ...

You should have added RREF to your list, because thanks to your explanation, I just discovered that guest2112 could have had the right answer directly in the HOME mode by simply adjusting the value of epsilon in the second page of the CAS setup menu!

Indeed, the screenshots below show how with the default value of 1.E-12, the RREF instruction fails to reduce M1.

But, by replacing this value with a more realistic one (such as 1.E-05 in my capture), we may obtain immediately the correct result:


Note that I use the "Floating 3" numeric format in order to have a more readable capture where the numbers are displayed with a limited number of decimals. Of course, the M1 matrix does contain internally the values ​​in full precision.

In CAS mode, curiously, a system message tells me that a lower value 1e-6 replaces my setting. Obviously, CAS mode allows you to get rid of the problem of precision or rounding errors by entering the matrix M1 in an exact form.
Nevertheless, the proposed precision of 1e-6 allows to obtain a correct numerical reduction of M1. Infact, correct numeric result appears at \( 1.1197\textbf{E-12} \cong 2^{-39.7} \).

Every day I discover new aspects of this mischievous HP Prime.

Too bad there was no system message to warn guest2112 that RREF was not finding a correct solution and was struggling with an unsuitable epsilon for his matrix!