Post Reply 
Accuracy and the power function
02-20-2014, 05:17 PM
Post: #12
RE: Accuracy and the power function
(02-19-2014 01:07 PM)Werner Wrote:  Truncation (and keeping track of lost non-zero digits with the Sticky Bit) in the 15-digit routines is not only common practice, it's a necessity if you want to be able to produce a correctly rounded 12-digit result for simple calculations.

Thank you for this example.
Here's another one:

5.02000000124e15 + 5009 (exact result: 5.020000001245009e15) would be rounded to 15 digits as 5.02000000124501e15.
After rounding this to 12 digits (towards odd or even) the result is 5.02000000125e15.
Which is the correct 12-digit value.

5.02000000124e15 + 5009 (exact result: 5.020000001245009e15) would be truncated after 15 digits to 5.02000000124500e15.
After rounding this to 12 digits (towards even) the result is 5.02000000124e15.
Which is 1 ULP low.

So, as usual, it depends. ;-)

Dieter
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
Accuracy and the power function - Dieter - 02-16-2014, 02:43 PM
RE: Accuracy and the power function - Dieter - 02-20-2014 05:17 PM



User(s) browsing this thread: 1 Guest(s)