@Thomas Klemm -> CORDIC Article
|
06-04-2014, 05:57 PM
Post: #23
|
|||
|
|||
RE: @Thomas Klemm -> CORDIC Article
(06-03-2014 02:02 PM)pito Wrote: I've flashed my CM3 with the old stuff I've found - when running below code with 34digits I get (I've added your result for comparison): Does anyone know of a table of correct values for this? newRPL gave me (for 34 digits): 8.999999999999999999999999999999979 I ran a loop from 9 to 900 digits, and the difference is always only the last 2 digits, being the worst difference 85 ULP. But how do we know right from wrong? Claudio |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 4 Guest(s)