Post Reply 
H->HMS conversion HP-15C vs. HP42S vs HP67
09-09-2018, 04:21 AM
Post: #31
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
Just think of a great idea: fast, simple, accurate DMS->DEG

The whole point of scaling / rounding is to avoid the "40 seconds" bug.
All we need is to give up 1 ULP binary float accuracy, and we are safe.
(correct strtod conversion guaranteed max error <= 0.5 ULP)

Using this thread original example, X = 183.3799999 ...

X = nextafter(X, X+X) = 183.38000000000002387 ...

Now, safe to apply Pekis DMS->DEG formula, and we are done Smile

(09-07-2018 07:07 AM)Pekis Wrote:  And with my funny one-liners:
DEG->DMS: (90*X+INT(60*X)+100*INT(X))/250
DMS->DEG: (250*X-INT(100*X)-60*INT(X))/90
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-09-2018 04:21 AM



User(s) browsing this thread: 18 Guest(s)