Post Reply 
HMS-> & ->HMS Prog. for 12C
01-12-2018, 12:36 PM
Post: #10
RE: HMS-> & ->HMS Prog. for 12C
Thanks for the answer.

DEG->DMS(8/60) would be evaluated as 0.08 (expected result) on an infinite precision machine with my formula

DEG->DMS(x)=(90*x+INT(60*x)+100*INT(x))/250
( DMS->DEG(x)=(250*x-INT(100*x)-60*INT(x))/90 )

Look at the result on a limited precision machine as the argument tends to 8/60:
0.13 ---> 0.0748
0.133 ---> 0.07588
0.1333 ---> 0.075988
0.13333 ---> 0.0759988
0.133333 ---> 0.07599988
0.1333333 ---> 0.075999988
...
"8/60" ---> 0.076

Is there something wrong seeing 0.075999999999.... naturally rounded to 0.076 by the machine ?

What do you think of it ?
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
HMS-> & ->HMS Prog. for 12C - Carsen - 01-10-2018, 05:18 AM
RE: HMS-> & ->HMS Prog. for 12C - Carsen - 01-10-2018, 05:22 AM
RE: HMS-> & ->HMS Prog. for 12C - Gamo - 01-10-2018, 06:49 AM
RE: HMS-> & ->HMS Prog. for 12C - Dieter - 01-10-2018, 10:37 AM
RE: HMS-> & ->HMS Prog. for 12C - Carsen - 01-11-2018, 04:46 AM
RE: HMS-> & ->HMS Prog. for 12C - Pekis - 01-11-2018, 06:08 AM
RE: HMS-> & ->HMS Prog. for 12C - Dieter - 01-11-2018, 07:03 PM
RE: HMS-> & ->HMS Prog. for 12C - Pekis - 01-11-2018, 09:22 PM
RE: HMS-> & ->HMS Prog. for 12C - Dieter - 01-12-2018, 09:13 AM
RE: HMS-> & ->HMS Prog. for 12C - Carsen - 01-13-2018, 01:26 AM
RE: HMS-> & ->HMS Prog. for 12C - Pekis - 01-12-2018 12:36 PM
RE: HMS-> & ->HMS Prog. for 12C - Dieter - 01-12-2018, 12:50 PM



User(s) browsing this thread: 1 Guest(s)