Post Reply 
I->R and R->I without (rounding) errors
08-04-2021, 07:42 AM
Post: #7
RE: I->R and R->I without (rounding) errors
(08-04-2021 07:18 AM)BINUBALL Wrote:  
(08-04-2021 04:47 AM)cahlucas Wrote:  Dear Mr. Horn,
Here is an example from your own creations (B->I and I->B):
2 41 ^ --> 2199023255552
2199023255552 0. + --> 2.19902325555E12 (or I->R)
2.19902325555E12 R->I --> 2199023255550
This should actually return the original number, but the last 2 have become a 0.
Dear cahlucas,
Unfortunately real on 50g has only 12 digit precision. So after you convert integer to real, last digit is lost. There is no way to get digit beyond 12 digits on real.

Binuball is correct. It is not the fault of I->R that you see roundoff errors when converting 2^41 to a real. The roundoff is caused by the limitations of reals themselves in RPL. They cannot have more than 12 mantissa digits, no matter what. No amount of System RPL or anything else can change that fact. If you need more than 12 digits of accuracy, then you MUST use some other object type than reals.

<0|ΙΈ|0>
-Joe-
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: I->R and R->I without (rounding) errors - Joe Horn - 08-04-2021 07:42 AM



User(s) browsing this thread: 3 Guest(s)