Free42 possible accuracy flaw
|
03-23-2022, 07:57 AM
(This post was last modified: 03-23-2022 08:56 AM by J-F Garnier.)
Post: #9
|
|||
|
|||
RE: Free42 possible accuracy flaw
History may help here.
In classic HP BCD machines, there are these special cases to ensure "nice" results for powers of ten: - LOG10: check for exact power of 10, - 10^X: check for integer, - Y^X: check for y-mantissa=1 and x=integer. (03-22-2022 10:08 PM)Thomas Okken Wrote: 10^2 = 99.99999999999999999999999999999999 Strangely, the Intel lib power function seems to have more problems with small numbers than with large numbers. 100 0.5 y^x ---> 9.999999999999999999999999999999999 0.01 0.5 y^x --> 9.999999999999999999999999999999998e-2 1E6000 0.5 y^x -> 1.e3000 (exact) It seems counter-intuitive for me. [added:] This is the opposite of HP machines, we start to get slightly inaccurate y^x for large numbers e.g. 1E480 0.5 y^x --> 9.99999999999E239 J-F |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 2 Guest(s)