Post Reply 
FORTRAN floating point accuracy problems
04-03-2016, 05:21 PM (This post was last modified: 04-03-2016 05:22 PM by Dieter.)
Post: #27
RE: FORTRAN floating point accuracy problems
(04-03-2016 05:17 PM)HP67 Wrote:  Again, I am only carrying 6 decimal places of expected results although I am displaying 7.

This means that "differences" in the 7th place actually are not present. ;-)
Maybe you can update the "expect" values to seven decimals.

(04-03-2016 05:17 PM)HP67 Wrote:  Thanks, I'll try your suggestions. But first I'm going to try to write an assembler subroutine for DINT. Then I can use your rounding routine which would really be great.

Hmmm... I would love to know whether the suggested rounding fixes the issue. Maybe you can add these little changes and report the results... ?-)

Dieter
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: FORTRAN floating point accuracy problems - Dieter - 04-03-2016 05:21 PM



User(s) browsing this thread: 1 Guest(s)