Post Reply 
FORTRAN floating point accuracy problems
03-27-2016, 11:13 PM (This post was last modified: 03-27-2016 11:13 PM by Dieter.)
Post: #4
RE: FORTRAN floating point accuracy problems
(03-27-2016 10:47 PM)Paul Dale Wrote:  For this kind of problem, I find first dealing with the sign and then getting into an integer domain as fast as possible and then applying corrections there is usually the best approach.

OK, but how do you prevent the roundoff problems due to binary arithmetics? Consider the example with 33,3000. Isolating the minutes leads to 29,99999... instead of 30, which then is interpreted as 29 minutes and 99,99999... seconds, i.e. 30 minutes and 39,9999... seconds. So some kind of rounding has to be applied.

Dieter
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: FORTRAN floating point accuracy problems - Dieter - 03-27-2016 11:13 PM



User(s) browsing this thread: 1 Guest(s)