H->HMS conversion HP-15C vs. HP42S vs HP67

[Albert Chan]

(09-10-2018 07:31 AM)Pekis Wrote:  (patched): DEG->DMS: (90*X+INT(X*120/2)+100*INT(X*2/2))/250
(unchanged): DMS->DEG: (250*X-INT(100*X)-60*INT(X))/90

What do you think of it ?

I propose patch DMS->DEG, to fix the "40 seconds" bug

DMS->DEG: (250*X-INT(100.00000000000001*X)-60*INT(X))/90

formula is basically unchanged, until X in "40 seconds bug" region.

In other words, X is at most 2 ULP away from "buggy" 100 seconds.
The patch remove the 100 seconds, round-up an extra minute instead.
(Not precisely 1 ULP fix with nextafter(), but good enough)

Your patch DEG->DMS have no effect for binary float, *2 and /2 is exact.

For calculator that used BCD, *2 /2 trick only work if X is "filled" with digits.
But, not always. On HP-12C, X=9.999999995, DEG->DMS = 9.5960 (exact)

Since 9°59'60" = 10°, "60 seconds bug" is not serious (easily fixed in the head)
At worse, this is a display bug, wrongly assumed number is decimal (see post 37).