FORTRAN floating point accuracy problems

[Dieter]

(05-04-2016 11:05 AM)HP67 Wrote:  Outstanding! Thanks, Dieter!

Fine. There is just one issue: if the input is very small, i.e. dms is a fraction of a second, or maybe even something below 1E–16, the input may be rounded down to zero. This can be avoided by replacing the respective lines by this:

Code:

If dms >= 1 Then
   k = int(log10(dms))
Else
   k = int(log10(dms)) - 1
End If

If there was a FLOOR function the test was not required at all: k=FLOOR(log10(dms).
Maybe you can write one in Assembler. ;-)

Now add a few lines that handle dms<0 (there is a DSIGN function in Fortran IV) or dms=0 (simply return zero), and you're done.

Edit: this could be a solution.

Code:

Function dms2dd(dms)
Const safedigits = 16
adms = Abs(dms)

If adms >= 1 Then
   k = Int(log10(adms))
Else
   If adms = 0 Then
      k = 0
   Else
      k = Int(log10(adms)) - 1
   End If
End If

n = safedigits - 3 - k

dd = Int(adms)
mmss = Round(100 * (adms - dd), n)
mm = Int(mmss)
ss = (mmss - mm) * 100

dms2dd = dsign(dd + mm / 60 + ss / 3600, dms)

End Function

If I get it right, dsign transfers the sign of the second argument to the first one, i.e. dsgn(x, y) = abs(x) * sign(y), which is quite handy here.

Dieter