H->HMS conversion HP-15C vs. HP42S vs HP67
|
09-09-2018, 01:54 PM
(This post was last modified: 09-09-2018 01:59 PM by Thomas Okken.)
Post: #39
|
|||
|
|||
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
The problem with using nextafter() is that the closest decimal equivalent isn't always going to be higher. In other words, you're always rounding up, but sometimes 0.3999 really is 0.3999, not a mangled 0.4000.
How about this: instead of multiplying by 10^12, I'll multiply by whichever power of 10 makes the integral part of the number have 12 digits (or 15, or whatever choice is safe), then round to the nearest integer, then remove what was originally the integer part, and finally normalize the scale of the remainder. That should preserve the nice properties of the current Free42 Binary implementation, while improving its handling of binary round-off as much as possible (without doing the equivalent of a full-blown dtoa(), anyway). |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 16 Guest(s)