A quick precision test
|
06-05-2014, 01:18 AM
(This post was last modified: 06-05-2014 01:19 AM by Claudio L..)
Post: #39
|
|||
|
|||
RE: A quick precision test
(06-04-2014 10:09 PM)Paul Dale Wrote:(06-04-2014 02:17 PM)Claudio L. Wrote: The power function is guaranteed to give you an accurate result with 34 good digits, for any 2 parameters (in this case, your guess that it's less precise is incorrect). I don't know that much, but the documentation of mpdecimal says the error is 1 ULP, so we use 9 extra digits and then we round, so the 'bad rounding' case would have to be in the form 999999995 in order for the 1 ULP error to give a badly rounded result. Notice that I said 'accurate digits' not 'correctly rounded'. The 1 ULP case may still happen, with a chance of 1 in 10^9. (06-04-2014 10:09 PM)Paul Dale Wrote:Quote:I don't know how decNumber does it in particular, but in general powers with small integer exponents are handled by a sequence of multiplications, while powers of real numbers or large numbers are usually done through logarithms: I agree, but "close" as in the last digit may be off by 1 due to rounding and only in some corner cases (like 1 in 10^9 chance), on most numbers you should get the right value. I'd love to test some corner cases to see how the algorithms behave (like that case that needs 1600 digits, do you have any links to good material on this subject?). I did some tests of my own but I don't know what the corner cases are. I'd like to have some literature where they list cases that I can use to stress the algorithms and see the results. Claudio |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 7 Guest(s)