Post Reply 
H->HMS conversion HP-15C vs. HP42S vs HP67
09-18-2018, 04:56 PM
Post: #62
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
(09-13-2018 01:46 PM)Albert Chan Wrote:  Before the fix, we may see 60 seconds, sometimes 60 minutes, but it is accurate.
A little cleaning-up, and we are good ...

After the fix, potentially you get into fixing phantom bug, similar to +25 seconds patch:
Just an example, on HP-12C, try X = 1000.999999:

Pekis DEG->DMS formula: (90*X + INT(60*X) + 100*INT(X)) / 250 ...

After some thinking, even the original formula is not safe ...

Example, angle = 9.20 degrees:
X = 9.19999 99999 99999 3 (degree), DEG->DMS(X) = 9.11999 99999 99999 2

Unless DD.MMSS allowed round-off, to 9.12, above just hit a 40 seconds bug.
This is a proposed fix:

Code:
def dms_to_deg(x):
    y = (100 + 1e-14) * x              # fix 40 seconds bug
    y = 2.5*y - int(y) - 60*int(y/100)
    return y * (1/90 - 3e-18)          # compensate above fix

def deg_to_dms(x):
    y = (3600 + 9e-13) * x             # y > 3600 D + 60 M + S
    z = 100 * int(y/3600) + int(y/60)  # z = 160 D + M (exact)
    y += 40 * z                        # y ~ 10000 D + 100 M + S
    return y * (1/10000 + 2e-20)       # avoid round-down 0.5 ULP

New deg_to_dms(X) = 9.12000 00000 00001
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-18-2018 04:56 PM



User(s) browsing this thread: 21 Guest(s)