FORTRAN floating point accuracy problems
|
03-27-2016, 11:13 PM
(This post was last modified: 03-27-2016 11:13 PM by Dieter.)
Post: #4
|
|||
|
|||
RE: FORTRAN floating point accuracy problems
(03-27-2016 10:47 PM)Paul Dale Wrote: For this kind of problem, I find first dealing with the sign and then getting into an integer domain as fast as possible and then applying corrections there is usually the best approach. OK, but how do you prevent the roundoff problems due to binary arithmetics? Consider the example with 33,3000. Isolating the minutes leads to 29,99999... instead of 30, which then is interpreted as 29 minutes and 99,99999... seconds, i.e. 30 minutes and 39,9999... seconds. So some kind of rounding has to be applied. Dieter |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 6 Guest(s)