[WP34s] Regularized incomplete Beta function

[Paul Dale]

(05-08-2014 10:58 PM)Dieter Wrote:  The user may enter dof values that lead to problems and strange results. At the moment I am looking at the Chi² quantile. For Chi² = n the CDF should be near 0,5 (slightly larger). Try this with n=2000 and you get 0,5042... (correct). Then try n=3000 and the 34s returns zero (!).

This is a bug, not a gradual loss of accuracy

The series expansion for the incomplete gamma function (as opposed to the continued fraction variation) isn't converging in 500 iterations for x around the degrees of freedom. For the n = 2000 case it is taking 440 iterations to converge. For n = 3000, it wants 532 iterations. For n = 250000, 4591 iterations are required.

Perhaps I got the threshold for switching between the continued fraction and the series expansion wrong.

This might also explain the gradual loss of digits you are seeing at time. That would match the behaviour of the continued fraction path which attempts to return something based on the best guess. The series code just returns zero when its iteration count expires.


Pauli