[WP34s] Regularized incomplete Beta function
|
05-05-2014, 09:49 PM
Post: #30
|
|||
|
|||
RE: [WP34s] Regularized incomplete Beta function
(05-05-2014 08:30 PM)Dieter Wrote: However, if the proposed routine runs in DP mode I could not find a case where the result had less than 30 correct digits. The XROM code runs in double precision regardless of the user's setting. This is good for rounding back to single precision. Changing the convergence check will pretty much force an extra iteration to detect the convergence. Thus it will cause a slow down. So an increase to the maximum to 6 seems reasonable "just to be sure". No, I don't have an example that increases accuracy. I'm leaning towards the minimum of thirty digits being sufficient in DP, since we don't guarantee accuracy there. - Pauli |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 3 Guest(s)