Post Reply 
Intel underestimates error bounds by 1.3 quintillion
10-15-2014, 09:31 PM
Post: #1
Intel underestimates error bounds by 1.3 quintillion
Not a new discovery, but nicely written up and probably new to many here.

http://randomascii.wordpress.com/2014/10...intillion/

"The worst-case error for the fsin instruction for small inputs is actually about 1.37 quintillion units in the last place, leaving fewer than four bits correct. For huge inputs it can be much worse..."
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
Intel underestimates error bounds by 1.3 quintillion - BruceH - 10-15-2014 09:31 PM



User(s) browsing this thread: 1 Guest(s)