XCAS float precision
|
10-04-2018, 02:52 PM
(This post was last modified: 10-04-2018 03:17 PM by Albert Chan.)
Post: #1
|
|||
|
|||
XCAS float precision
I finally join the club, and downloaded XCas (for Windows 32-bits)
If XCas float default is IEEE double precision, I should get this: Python: 3.141592653589794 - 3.141592653589793 => 8.88e-16 This is above in hex-float, where ULP = 2^-51 ~ 4.44e-16: 0x1.921fb54442d1aP1 - 0x1.921fb54442d18P1 = 2 ULP ~ 8.88e-16 Instead, I get something unexpected, suggesting XCas uses 54-bits float (or more) (Also, returned float is not normalized, with a zero before decimal point) XCAS: 3.141592653589794 - 3.141592653589793 => 0.111e-14 (2.5 ULP ?) Just to confirm above with 54-bits float math (now, ULP = 2^-52) 0x1.921fb54442d1a0P+1 - 0x1.921fb54442d178P+1 = 5 ULP ~ 1.11E-15 (match above) |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
XCAS float precision - Albert Chan - 10-04-2018 02:52 PM
RE: XCAS float precision - parisse - 10-04-2018, 07:22 PM
RE: XCAS float precision - Albert Chan - 10-04-2018, 09:36 PM
RE: XCAS float precision - parisse - 10-05-2018, 06:04 PM
|
User(s) browsing this thread: 1 Guest(s)