Post Reply 
WP34s: Domain Error in RMDR and MOD
10-22-2015, 01:00 PM
Post: #3
RE: WP34s: Domain Error in RMDR and MOD
(10-21-2015 09:39 PM)Paul Dale Wrote:  The decNumber mod function isn't efficient and requires enough digits to represent the source numbers and the result (i.e. 2*384 + 16). I increase the number of carried digits for these operations to a little above this so that single precision never fails.

Wait... does this mean that RMDR and MOD internally use 400+ digits?

(10-21-2015 09:39 PM)Paul Dale Wrote:  Making double precision inputs work across the entire range will require a different algorithm and since we don't guarantee any results in double precision I didn't bother Smile

OK. Maybe a table with such restrictions would be helpful. I am also thinking of the distributions, Gamma and Beta with extreme cases of their parameters.

Dieter
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: WP34s: Domain Error in RMDR and MOD - Dieter - 10-22-2015 01:00 PM



User(s) browsing this thread: 2 Guest(s)