Post Reply 
H->HMS conversion HP-15C vs. HP42S vs HP67
09-09-2018, 01:54 PM (This post was last modified: 09-09-2018 01:59 PM by Thomas Okken.)
Post: #39
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
The problem with using nextafter() is that the closest decimal equivalent isn't always going to be higher. In other words, you're always rounding up, but sometimes 0.3999 really is 0.3999, not a mangled 0.4000.

How about this: instead of multiplying by 10^12, I'll multiply by whichever power of 10 makes the integral part of the number have 12 digits (or 15, or whatever choice is safe), then round to the nearest integer, then remove what was originally the integer part, and finally normalize the scale of the remainder. That should preserve the nice properties of the current Free42 Binary implementation, while improving its handling of binary round-off as much as possible (without doing the equivalent of a full-blown dtoa(), anyway).
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Thomas Okken - 09-09-2018 01:54 PM



User(s) browsing this thread: