H->HMS conversion HP-15C vs. HP42S vs HP67
|
09-13-2018, 04:47 AM
Post: #55
|
|||
|
|||
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
Above error bound can be tighten a bit, since we know scaling factor ~ 100.0
m1 and m2 are related by the factor k = m3/2^53 = 0.78125 If m1/2^53 >= 0.64, m2/m1 = k, otherwise m2/m1 = 2*k With factor 100.0: Max absolute error = 0.5 + 0.5 m2/m1 = 0.5 + k ~ 1.28 ULP +1 ULP fix reduce this to k - 0.5 ~ 0.28 ULP With factor 100.00000000000001: Max absolute error = 0.5 + 0.5 m2/m1 - min(m2/m3) If m1/2^53 >= 0.64, error = 0.5 + 0.5 k - 0.5/k ~ 0.25 ULP Otherwise, error = 0.5 + 0.5 (2 k) - k/k = k - 0.5 ~ 0.28 ULP Both approach have the same lower bound ! |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 19 Guest(s)