H->HMS conversion HP-15C vs. HP42S vs HP67
|
09-09-2018, 04:21 AM
Post: #31
|
|||
|
|||
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
Just think of a great idea: fast, simple, accurate DMS->DEG
The whole point of scaling / rounding is to avoid the "40 seconds" bug. All we need is to give up 1 ULP binary float accuracy, and we are safe. (correct strtod conversion guaranteed max error <= 0.5 ULP) Using this thread original example, X = 183.3799999 ... X = nextafter(X, X+X) = 183.38000000000002387 ... Now, safe to apply Pekis DMS->DEG formula, and we are done (09-07-2018 07:07 AM)Pekis Wrote: And with my funny one-liners: |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 18 Guest(s)