H->HMS conversion HP-15C vs. HP42S vs HP67

[Albert Chan]

(09-07-2018 07:07 AM)Pekis Wrote:  And with my funny one-liners:
DEG->DMS: (90*X+INT(60*X)+100*INT(X))/250
DMS->DEG: (250*X-INT(100*X)-60*INT(X))/90

Hi, Pekis

Above work (mostly) in Excel, because Excel does some behind-the-scene rounding.

Example: if angle = 2° 1' , X = 2.01 (IEEE double of 2.0099999...)
DMS->DEG formula:

Python: (502.499999... - 200 - 120) / 90 = 2.0277777... (wrong!)
Excel : (502.5 - 201 - 120) / 90 = 2.0166666... (all terms rounded implicitly)

I would rather do explicit scaling / rounding, before conversion (see post #17):

X = 2.0099999 ... --> Y = 20100.00 (scaled and rounded to hundredth of second)

DMS->DEG: (Y - 40*int(Y/100) - 2400*int(Y/10000)) / 3600 = (20100 - 8040 - 4800) / 3600 ~ 2.0166666...