Free42 possible accuracy flaw
|
03-23-2022, 08:05 AM
Post: #10
|
|||
|
|||
RE: Free42 possible accuracy flaw
0.1^2=9.999999999999999999999999999999996e-3
Estimate for exactness: take the number of significant digits, that is, the mantissa length minus the number of trailing zeroes, and multiply that by the exponent; if <= 34, use squaring? The estimate doesn't have to be sharp: multiply the mantissa by such a power of 10 that you get the smallest possible integer, then perform the repeated-squaring algorithm, and if the result is < 10^35, fix the exponent and return it, and otherwise, let bid128_pow() do its thing. |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)