Machine epsilon of the HP Prime
|
09-20-2022, 11:24 AM
Post: #7
|
|||
|
|||
RE: Machine epsilon of the HP Prime
For HP Prime, formal definition (maximum relative error) and variant definition (ulp of 1.) are the same.
CAS round number to 53 bits double (round-to-nearest), then chop to 48 bits, u = maximum relative error = (1 ULP) / (2^47 ULP) = 2^-47 1 = 2^47 ULP, unaffected by rounding-mode. This probably made variant defintion, ε = ulp of 1., more popular. |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
Machine epsilon of the HP Prime - cdeaglejr - 09-17-2022, 04:28 PM
RE: Machine epsilon of the HP Prime - rprosperi - 09-17-2022, 05:28 PM
RE: Machine epsilon of the HP Prime - EdS2 - 09-19-2022, 01:05 PM
RE: Machine epsilon of the HP Prime - EdS2 - 09-18-2022, 08:11 AM
RE: Machine epsilon of the HP Prime - Albert Chan - 09-18-2022, 12:20 PM
RE: Machine epsilon of the HP Prime - Albert Chan - 09-20-2022 11:24 AM
RE: Machine epsilon of the HP Prime - EdS2 - 09-19-2022, 06:31 AM
|
User(s) browsing this thread: 1 Guest(s)