Post Reply 
FORTRAN floating point accuracy problems
04-11-2016, 06:19 AM
Post: #38
RE: FORTRAN floating point accuracy problems
Hello,

Perhaps you should try these formulas:

For x>0:
DEG->DMS(x)=(90*x+INT(60*x)+100*INT(x))/250

It's mathematically exact, but special care is needed in evaluating INT(60*x) on a calculator.
=> It seems better to round 60*x to last internal digit before applying INT.

DMS->DEG(x)=(250*x-INT(100*x)-60*INT(x))/90

See you.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: FORTRAN floating point accuracy problems - Pekis - 04-11-2016 06:19 AM



User(s) browsing this thread: 1 Guest(s)