FORTRAN floating point accuracy problems
|
04-11-2016, 06:19 AM
Post: #38
|
|||
|
|||
RE: FORTRAN floating point accuracy problems
Hello,
Perhaps you should try these formulas: For x>0: DEG->DMS(x)=(90*x+INT(60*x)+100*INT(x))/250 It's mathematically exact, but special care is needed in evaluating INT(60*x) on a calculator. => It seems better to round 60*x to last internal digit before applying INT. DMS->DEG(x)=(250*x-INT(100*x)-60*INT(x))/90 See you. |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)