HP41: accuracy of 13-digit routines

[Ángel Martin]

(09-02-2015 06:07 PM)Dieter Wrote:  So the accuracy level depends on the function, and sometimes even on its argument. Since the four basic arithmetic functions as well as x² and √x deliver correctly rounded results they must be virtually exact internally, i.e. in all 13 internal digits, maybe the last one truncated. This would yield a max. error of 0,501 ULP.

The logs, 10^x and the trig functions (in DEG/GRAD) and their inverses may exhibit a slightly larger error, yet "significantly less" than 1 ULP. Maybe we can translate this to 0,6 ULP or 1 unit in the 11th place, or a relative error within 1E–11, maybe 0,7 ULP (rel. error < 2E–11). So there are 11 exact or nearly-exact digits.

And finally the error may spill into the last of the 10 digits presented to the user, where the 10th digit may be off by 2 or even 3 units.

Sure, I meant that the accuracy will always be correct to the 10th decimal digit but I didn't imply that all algorithms would be equally effective in the digits beyond that. Definitely the transcendental functions are more complicated to implement, but not only those show weak points.

I don't know if there were changes made to the 15C MCODE, but I can tell you that on the 41 the square root at least is far from being that accurate. If you do a chained calculation in MCODE with [SQR-13] followed by a 13-digit multiplication you not always get the same original argument to the 13th. digit.... and there's NOT such a thing as X^2 as individual function either, it uses the same multiplication code.

(09-02-2015 06:07 PM)Dieter Wrote:  What does this mean for a Gamma implementation in 13-digit MCode? The functions used in the Lanczos algorithm are mostly +, –, x and / which are (nearly) exact. On the other hand the logs may only have 11 valid digits. So this is what can be expected: about 11 exact digits, and a final error similar to that of the internal log or trig functions.

the X! on the 15C is the gamma function, right? So there you also have it to test for the accuracy to the 10th decimal digit. As far as GAMMA in the SandMath I checked the accuracy for the natural arguments were ok (see the table in its manual) but didn't do any further checks for non-integer arguments. BTW there's also exponentials in the Lanczos formula, as well as sine factors in the reflection formula.

Cheers,
'AM