FORTRAN floating point accuracy problems
|
03-27-2016, 03:02 PM
Post: #1
|
|||
|
|||
FORTRAN floating point accuracy problems
I'm getting some unacceptable results from rounding in old FORTRAN. I know there are some very talented mathematicians on the board here so I wonder if you guys would be willing to help me. To say I'm bad at math would be an understatement! I have no idea where things are going wrong.
I'm trying to accomplish something very simple in a language which has very little in the way of helpful functions to do what I want. Given a floating point input value for degrees, minutes, seconds in the form dd.mmsst... I want to produce the value in decimal degrees. This is no problem of math even for me. The problem is extracting the digits with only a truncation function and nothing else helpful in the repetoire as far as I can see. Here is the piece of code: Code:
and here are the results of some tests: Code:
You can see in test case #3 above there is a big problem extracting the minutes. The code winds up with 29 instead of 30 because of the truncation function. I find this kind of problem is coming up a lot in various calculations I'm doing and I don't know how to code around it. Can anybody please explain how I can rearrange the computations or do something differently to avoid this problem? Thanks. It ain't OVER 'till it's 2 PICK |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 3 Guest(s)