FORTRAN floating point accuracy problems
|
03-31-2016, 02:45 PM
(This post was last modified: 03-31-2016 02:55 PM by HP67.)
Post: #20
|
|||
|
|||
RE: FORTRAN floating point accuracy problems
Thanks very much.
Mostly I'm using this to convert decimal time back and forth. .001 seconds should be plenty of resolution. I'll be happy with a reliable hh.mmsst for now. I think a rounding function and a double precision truncation function would be more generally useful but I'll study what you came up with. I can anticipate a few problems but fortunately I am almost always wrong about math so maybe this time it will work in my favor. FORTRAN IV doesn't have any integer division except for integer types and doesn't have 8 byte integers. When we get back to dividing again won't we go through the same or similar conversions/promotions as before? So I am expecting problems. Again, I'm probably wrong. I'll play around with your suggestion. Eventually I would like to come up with an 8 byte truncation function for reals. The rounding function is very handy also. In understanding your declarations, Excel VBA long means 8 byte int? No comparable type in FORTRAN IV. Will a 4 byte int work? I think you explained that it will, so I don't understand why you chose long in your code. It ain't OVER 'till it's 2 PICK |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 3 Guest(s)