FORTRAN floating point accuracy problems
|
03-28-2016, 03:48 AM
Post: #5
|
|||
|
|||
RE: FORTRAN floating point accuracy problems
The most recent time I did this was using decimal arithmetic and it took several goes.
Yes, rounding will be required. At least it is easy enough to detect the errant case -- when the seconds value is greater than sixty there is possibly a problem. Figuring out where and how to round is another matter entirely -- this is getting close to fudging the answer. - Pauli |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 4 Guest(s)