Post Reply 
Machine epsilon of the HP Prime
09-20-2022, 11:24 AM
Post: #7
RE: Machine epsilon of the HP Prime
For HP Prime, formal definition (maximum relative error) and variant definition (ulp of 1.) are the same.
CAS round number to 53 bits double (round-to-nearest), then chop to 48 bits,

u = maximum relative error = (1 ULP) / (2^47 ULP) = 2^-47

1 = 2^47 ULP, unaffected by rounding-mode.
This probably made variant defintion, ε = ulp of 1., more popular.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Machine epsilon of the HP Prime - EdS2 - 09-19-2022, 01:05 PM
RE: Machine epsilon of the HP Prime - EdS2 - 09-18-2022, 08:11 AM
RE: Machine epsilon of the HP Prime - Albert Chan - 09-20-2022 11:24 AM
RE: Machine epsilon of the HP Prime - EdS2 - 09-19-2022, 06:31 AM



User(s) browsing this thread: 1 Guest(s)