Post Reply 
H->HMS conversion HP-15C vs. HP42S vs HP67
09-07-2018, 02:40 PM
Post: #22
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
(09-07-2018 09:43 AM)Werner Wrote:  
(09-06-2018 05:31 PM)Dieter Wrote:  it looks like the usual binary roundoff problem.

Yet Free42 binary (of which I must be the only user?) gets it right ;-)

The approach I chose for ->HR for the binary version is robust but not very pretty: multiply x by 1e12, round to nearest integer, and then do everything using integer math. Plus, there is a special case in for |x| < 0.01, where the conversion reduces to a simple division by 0.36.

The idea was to make sure that it will never get anything wrong that the HP-42S gets right, but the simplistic approach I chose does mean a bit of precision is lost. If someone has a better algorithm lying around for IEEE-754 double precision binary, I would be interested. Smile
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Thomas Okken - 09-07-2018 02:40 PM



User(s) browsing this thread: 11 Guest(s)