WP34s: Domain Error in RMDR and MOD
|
10-22-2015, 01:00 PM
Post: #3
|
|||
|
|||
RE: WP34s: Domain Error in RMDR and MOD
(10-21-2015 09:39 PM)Paul Dale Wrote: The decNumber mod function isn't efficient and requires enough digits to represent the source numbers and the result (i.e. 2*384 + 16). I increase the number of carried digits for these operations to a little above this so that single precision never fails. Wait... does this mean that RMDR and MOD internally use 400+ digits? (10-21-2015 09:39 PM)Paul Dale Wrote: Making double precision inputs work across the entire range will require a different algorithm and since we don't guarantee any results in double precision I didn't bother OK. Maybe a table with such restrictions would be helpful. I am also thinking of the distributions, Gamma and Beta with extreme cases of their parameters. Dieter |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
WP34s: Domain Error in RMDR and MOD - Dieter - 10-21-2015, 12:13 PM
RE: WP34s: Domain Error in RMDR and MOD - Paul Dale - 10-21-2015, 09:39 PM
RE: WP34s: Domain Error in RMDR and MOD - Dieter - 10-22-2015 01:00 PM
RE: WP34s: Domain Error in RMDR and MOD - Paul Dale - 10-22-2015, 09:57 PM
|
User(s) browsing this thread: 2 Guest(s)