Post Reply 
H->HMS conversion HP-15C vs. HP42S vs HP67
09-12-2018, 07:06 AM
Post: #51
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
(09-10-2018 12:46 PM)Albert Chan Wrote:  I propose patch DMS->DEG, to fix the "40 seconds" bug

DMS->DEG: (250*X-INT(100.00000000000001*X)-60*INT(X))/90

Why not

DMS->DEG: (250*X-INT(100*X+0.25)-60*INT(X))/90

- either X is 'well behaved' and it makes no difference
- or SS=0 and H.MM is not exactly representable (the 40-seconds), and then it rounds up. Independent of the precision used.

Cheers, Werner

41CV†,42S,48GX,49G,DM42,DM41X,17BII,15CE,DM15L,12C,16CE
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Werner - 09-12-2018 07:06 AM



User(s) browsing this thread: 2 Guest(s)