newRPL - build 1255 released! [updated to 1299]
08-16-2019, 01:22 PM
Post: #564
 Claudio L. Senior Member Posts: 1,880 Joined: Dec 2013
RE: newRPL - build 1255 released! [updated to 1282]
(08-15-2019 10:49 PM)Claudio L. Wrote:  So I'm adding 7 zeros after the number, and forcing the system to approximate *that* number.
Clearly, I just need to determine how many "fractional" digits are in the actual argument and use that as tolerance

Huh! I have to admit, I didn't anticipate that problem, but it does make sense!

Also, it might make more sense to use 5*10^(-prec-1) rather than 10^(-prec). Adjusted for the integer digits, of course!

Good suggestion, thanks. I now understand the algorithm better, if the tolerance is too tight all you get is:
aaa.bbb ->Q ----> 'aaabbb/1000' simplified, which is what the old ->Q did, and not the smallest "coolest" fraction as PDQ is supposed to produce.

Once it works nicely, I'll release an update for you to test and bug-check on corner cases.
 « Next Oldest | Next Newest »