H->HMS conversion HP-15C vs. HP42S vs HP67
09-18-2018, 04:56 PM
Post: #62
 Albert Chan Senior Member Posts: 2,494 Joined: Jul 2018
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
(09-13-2018 01:46 PM)Albert Chan Wrote:  Before the fix, we may see 60 seconds, sometimes 60 minutes, but it is accurate.
A little cleaning-up, and we are good ...

After the fix, potentially you get into fixing phantom bug, similar to +25 seconds patch:
Just an example, on HP-12C, try X = 1000.999999:

Pekis DEG->DMS formula: (90*X + INT(60*X) + 100*INT(X)) / 250 ...

After some thinking, even the original formula is not safe ...

Example, angle = 9.20 degrees:
X = 9.19999 99999 99999 3 (degree), DEG->DMS(X) = 9.11999 99999 99999 2

Unless DD.MMSS allowed round-off, to 9.12, above just hit a 40 seconds bug.
This is a proposed fix:

Code:
def dms_to_deg(x):     y = (100 + 1e-14) * x              # fix 40 seconds bug     y = 2.5*y - int(y) - 60*int(y/100)     return y * (1/90 - 3e-18)          # compensate above fix def deg_to_dms(x):     y = (3600 + 9e-13) * x             # y > 3600 D + 60 M + S     z = 100 * int(y/3600) + int(y/60)  # z = 160 D + M (exact)     y += 40 * z                        # y ~ 10000 D + 100 M + S     return y * (1/10000 + 2e-20)       # avoid round-down 0.5 ULP

New deg_to_dms(X) = 9.12000 00000 00001
 « Next Oldest | Next Newest »

 Messages In This Thread H->HMS conversion HP-15C vs. HP42S vs HP67 - Siegfried - 09-06-2018, 11:16 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - BartDB - 09-06-2018, 11:30 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Massimo Gnerucci - 09-06-2018, 11:35 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Robert VM - 09-06-2018, 12:58 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Paul Dale - 09-06-2018, 11:32 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - ijabbott - 09-06-2018, 06:51 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Thomas Okken - 09-06-2018, 10:37 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Siegfried - 09-07-2018, 09:42 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Didier Lachieze - 09-06-2018, 12:05 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Gene - 09-06-2018, 01:07 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Paul Dale - 09-07-2018, 03:40 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - rprosperi - 09-06-2018, 01:21 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - burkhard - 09-06-2018, 01:42 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-06-2018, 02:14 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Dieter - 09-06-2018, 05:01 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - burkhard - 09-06-2018, 05:14 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Dieter - 09-06-2018, 05:31 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-06-2018, 07:36 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Werner - 09-07-2018, 09:43 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Thomas Okken - 09-07-2018, 02:40 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-08-2018, 12:59 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Zaphod - 09-06-2018, 05:58 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - burkhard - 09-06-2018, 06:02 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Pekis - 09-07-2018, 07:07 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-07-2018, 03:43 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-08-2018, 06:03 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-09-2018, 04:21 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Joe Horn - 09-09-2018, 05:10 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Carsen - 09-09-2018, 05:29 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Werner - 09-09-2018, 07:50 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - teenix - 09-09-2018, 08:59 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - AndiGer - 09-15-2018, 09:54 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - AndiGer - 09-07-2018, 04:57 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Gene - 09-07-2018, 08:49 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Thomas Okken - 09-08-2018, 02:35 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-08-2018, 03:45 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-09-2018, 06:13 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-09-2018, 11:39 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Werner - 09-09-2018, 08:51 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Thomas Okken - 09-09-2018, 01:54 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-09-2018, 03:25 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Pekis - 09-10-2018, 07:31 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-10-2018, 12:46 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Pekis - 09-10-2018, 01:31 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-10-2018, 04:28 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Werner - 09-12-2018, 07:14 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-12-2018, 02:08 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-12-2018, 08:20 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Werner - 09-14-2018, 12:46 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Joe Horn - 09-10-2018, 03:32 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-10-2018, 06:18 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Joe Horn - 09-10-2018, 06:53 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Dieter - 09-10-2018, 07:13 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Joe Horn - 09-10-2018, 07:06 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - DaveBr - 09-20-2018, 08:52 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Werner - 09-12-2018, 07:06 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Werner - 09-10-2018, 02:10 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-13-2018, 04:47 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Pekis - 09-13-2018, 08:47 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-13-2018, 01:46 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-18-2018 04:56 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Pekis - 09-13-2018, 03:11 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-15-2018, 02:47 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Craig Bladow - 09-21-2018, 03:06 AM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-21-2018, 12:16 PM RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Ken S - 09-21-2018, 06:08 AM

User(s) browsing this thread: 2 Guest(s)