Post Reply 
H->HMS conversion HP-15C vs. HP42S vs HP67
09-10-2018, 04:28 PM
Post: #46
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
(09-10-2018 01:31 PM)Pekis Wrote:  for simple and usual calculations (in BCD), it removes the annoying "60 seconds bug".
For example, DEG->DMS(DMS->DEG(0.08)) goes back to 0.08, and not 0.0760.

I mis-understood your patch, thinking it cover all cases ...
If simple cases round-trip are what you are after, *2/2 can be removed, to this:

(patched): DEG->DMS: (90*X+INT(X*120/2)+100*INT(X))/250

(09-10-2018 02:10 PM)Werner Wrote:  T := H.MS + FP(H.MS*100)/150;
HR := T + FP(T)/1.5;

that I posted earlier has no problems with the billion degrees 4 minutes problem,
and doesn't have to resort to introducing errors of a few ulps.

Without error analysis (ULPs), you cannot get this right.
There are too many cases above formula can fail: 29', 57', 58', 1°13', 1°14', 1°15', 1°16', ...

Example: 57 minutes should convert to 57/60 = 0.95 degrees

T = 0.56999 ... + 0.00666 ... = 0.57666 ... (I assumed FP meant fractional part)
HR = 0.57666 ... + 0.38444 ... = 0.96111 ... (again, off by 40 seconds)
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-10-2018 04:28 PM



User(s) browsing this thread: 1 Guest(s)