H->HMS conversion HP-15C vs. HP42S vs HP67
|
09-12-2018, 08:20 PM
(This post was last modified: 09-13-2018 03:51 AM by Albert Chan.)
Post: #54
|
|||
|
|||
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
(09-12-2018 02:08 PM)Albert Chan Wrote: Correctly implemented strtod guaranteed maximum error 0.5 ULP I made an error adding up the ULPs. What should be added is relative errors, not ULPs Let binary mantissa of X = m1, of scaled Y = 100*X = m2 So, 2^52 <= m1 < 2^53, and same for m2 (units in ULP) Max relative (under-estimated) error = 0.5/m1 + 0.5/m2 Max absolute error (ULP) = Max relative error * m2 = 0.5 + 0.5 m2/m1 = 0.5 + 0.5 * max(m2/m1) = 0.5 + 0.5 * (2^53-1) / 2^52 < 1.5 ULP Technically +1 ULP still work, but it is cutting it very close. (If Y < 0.5 ULP below an integer, it will round-up to integer) 100.00000000000001*X patch is safer: Let m3 = binary mantissa of 100.0 = 0.78125 * 2^53 Max relative (under-estimated) error = 0.5/m1 + 0.5/m2 - 1/m3 Max absolute error (ULP) = Max relative error * m2 = 0.5 + 0.5 m2/m1 - m2/m3 = 0.5 + m2 * (1/(2 m1) - 1/m3) Since (2 m1) > m3, the factor is negative, we need to minimize both terms Max absolute (under-estimated) error = 0.5 + 2^52 * (1/2^53 - 1/(0.78125*2^53)) = 0.36 ULP < 0.5 ULP In other words, DD.MM * 100.00000000000001 >= DDMM (in binary float) |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: