+- HP Forums (https://www.hpmuseum.org/forum)
+-- Forum: HP Calculators (and very old HP Computers) (/forum-3.html)
+--- Forum: HP Prime (/forum-5.html)
+--- Thread: XCAS float precision (/thread-11525.html)
XCAS float precision - Albert Chan - 10-04-2018 02:52 PM
I finally join the club, and downloaded XCas (for Windows 32-bits)
If XCas float default is IEEE double precision, I should get this:
Python: 3.141592653589794 - 3.141592653589793 => 8.88e-16
This is above in hex-float, where ULP = 2^-51 ~ 4.44e-16:
0x1.921fb54442d1aP1 - 0x1.921fb54442d18P1 = 2 ULP ~ 8.88e-16
Instead, I get something unexpected, suggesting XCas uses 54-bits float (or more)
(Also, returned float is not normalized, with a zero before decimal point)
XCAS: 3.141592653589794 - 3.141592653589793 => 0.111e-14 (2.5 ULP ?)
Just to confirm above with 54-bits float math (now, ULP = 2^-52)
0x1.921fb54442d1a0P+1 - 0x1.921fb54442d178P+1 = 5 ULP ~ 1.11E-15 (match above)
RE: XCAS float precision - parisse - 10-04-2018 07:22 PM
Xcas is using longfloats (via MPFR) for representation of floats having more than 14 digits.
RE: XCAS float precision - Albert Chan - 10-04-2018 09:36 PM
(10-04-2018 07:22 PM)parisse Wrote: Xcas is using longfloats (via MPFR) for representation of floats having more than 14 digits.
Thanks you. It now make sense.
pibf := 3.141592653589793 // bigfloat
pi53 := 3.141592653 + 5.89793-10 // machine float
evalf(pibf, 20) => 3.1415926535897928940 // big float confirmed
evalf(pi53, 20) => 3.14159265359 // pi53 is 0.5 ULP bigger than pibf
It seems when big float mixed with machine float, big-float is "demoted".
evalf(pibf + 0.0, 20) => 3.14159265359
pibf - pi53 //==> 0.0
pibf == pi53 //==> true, big float demoted before comparison.
RE: XCAS float precision - parisse - 10-05-2018 06:04 PM
Indeed. Mixing floats and long floats will convert long float to float.