FORTRAN floating point accuracy problems
|
04-07-2016, 01:36 PM
Post: #36
|
|||
|
|||
RE: FORTRAN floating point accuracy problems
I was waiting till you came up with a problematic case ;-)
Actually, however, depending on how you look at it: - this is a case of wrong input (99 for seconds) - the result is correct (try it on Free42 Decimal, for instance) But I agree that you can't see it as it is rounded in the display.. BTW with x = 1e13/(5e13 + 1), on Free42 Binary, INT(100*x) delivers 19 and not 20 as you stated, and there are no negatives during the computation? Cheers, Werner 41CV†,42S,48GX,49G,DM42,DM41X,17BII,15CE,DM15L,12C,16CE |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 3 Guest(s)