WP34s: Domain Error in RMDR and MOD

[Dieter]

I just noticed a strange behaviour of the RMDR and MOD functions. For large arguments they may return a "Domain Error". Some further investigation showed that these functions seem to work for y < 1E820 · x, and as soon as y gets larger the error occurs:

Code:

[DBLON]

[EEX] 820 [ENTER] 360 [x]   =>   3,6 E+822
360 [RMDR]                  =>   Domain Error
    [MOD]                   =>   Domain Error

[x<>y] [ENTER] [ULP] [-]    =>   3,599999999999999999999999999999999 E+822
360 [RMDR]                  =>   80

Since the arguments are out of range in single precision mode, all this happens only after switching to double precision.
Any idea what's going on here?

Dieter

[Paul Dale]

The decNumber mod function isn't efficient and requires enough digits to represent the source numbers and the result (i.e. 2*384 + 16). I increase the number of carried digits for these operations to a little above this so that single precision never fails. I didn't go as far as supporting double precision -- there isn't sufficient RAM to do this directly.

Making double precision inputs work across the entire range will require a different algorithm and since we don't guarantee any results in double precision I didn't bother


- Pauli

[Dieter]

(10-21-2015 09:39 PM)Paul Dale Wrote:  The decNumber mod function isn't efficient and requires enough digits to represent the source numbers and the result (i.e. 2*384 + 16). I increase the number of carried digits for these operations to a little above this so that single precision never fails.

Wait... does this mean that RMDR and MOD internally use 400+ digits?

(10-21-2015 09:39 PM)Paul Dale Wrote:  Making double precision inputs work across the entire range will require a different algorithm and since we don't guarantee any results in double precision I didn't bother

OK. Maybe a table with such restrictions would be helpful. I am also thinking of the distributions, Gamma and Beta with extreme cases of their parameters.

Dieter

[Paul Dale]

Mod uses 450 digits internally.
Modulo reduction by two pi for the trig functions uses 820.

So in double precision, neither is adequate to support the entire range.


Pauli