H->HMS conversion HP-15C vs. HP42S vs HP67

[Albert Chan]

(09-09-2018 06:13 AM)Albert Chan Wrote:  Y = 100*X
Y = nextafter(Y, Y + Y)
DMS->DEG = (2.5*Y - int(Y) - 60*int(Y/100)) / 90

Max error = decimal to binary error + scaling error = 0.5 + 0.5 = 1.0 ULP
--> 1 ULP is enough to get to safety, away from "40 seconds bug"

Above can be simplified and maybe more accurate by using non-integer factors

Y = 100.00000000000001 * X
DMS->DEG = (2.5*Y - int(Y) - 60*int(Y/100)) * 0.011111111111111108

first factor (with 15 zeroes) = 100.000 ... = 1 ULP above 100.0, to fix "40 seconds" bug
second factor (with 15 ones) = 0.01111... = 2 ULP below 1/90.0, to compensate over-estimation

For angle=183°38' , above produce error of +0.73 ULP (against exact 183 + 38/60)
For 1000000000°4', above produce error of +0.47 ULP ... not too bad.