HMS-> & ->HMS Prog. for 12C
|
01-12-2018, 12:36 PM
Post: #10
|
|||
|
|||
RE: HMS-> & ->HMS Prog. for 12C
Thanks for the answer.
DEG->DMS(8/60) would be evaluated as 0.08 (expected result) on an infinite precision machine with my formula DEG->DMS(x)=(90*x+INT(60*x)+100*INT(x))/250 ( DMS->DEG(x)=(250*x-INT(100*x)-60*INT(x))/90 ) Look at the result on a limited precision machine as the argument tends to 8/60: 0.13 ---> 0.0748 0.133 ---> 0.07588 0.1333 ---> 0.075988 0.13333 ---> 0.0759988 0.133333 ---> 0.07599988 0.1333333 ---> 0.075999988 ... "8/60" ---> 0.076 Is there something wrong seeing 0.075999999999.... naturally rounded to 0.076 by the machine ? What do you think of it ? |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
HMS-> & ->HMS Prog. for 12C - Carsen - 01-10-2018, 05:18 AM
RE: HMS-> & ->HMS Prog. for 12C - Carsen - 01-10-2018, 05:22 AM
RE: HMS-> & ->HMS Prog. for 12C - Gamo - 01-10-2018, 06:49 AM
RE: HMS-> & ->HMS Prog. for 12C - Dieter - 01-10-2018, 10:37 AM
RE: HMS-> & ->HMS Prog. for 12C - Carsen - 01-11-2018, 04:46 AM
RE: HMS-> & ->HMS Prog. for 12C - Pekis - 01-11-2018, 06:08 AM
RE: HMS-> & ->HMS Prog. for 12C - Dieter - 01-11-2018, 07:03 PM
RE: HMS-> & ->HMS Prog. for 12C - Pekis - 01-11-2018, 09:22 PM
RE: HMS-> & ->HMS Prog. for 12C - Dieter - 01-12-2018, 09:13 AM
RE: HMS-> & ->HMS Prog. for 12C - Carsen - 01-13-2018, 01:26 AM
RE: HMS-> & ->HMS Prog. for 12C - Pekis - 01-12-2018 12:36 PM
RE: HMS-> & ->HMS Prog. for 12C - Dieter - 01-12-2018, 12:50 PM
|
User(s) browsing this thread: 8 Guest(s)