Base converter with fractions?

02212017, 07:30 PM
(This post was last modified: 02212017 07:31 PM by Han.)
Post: #21




RE: Base converter with fractions?
(02212017 05:59 PM)Jan_D Wrote:(02032017 01:51 PM)Jan_D Wrote: I slightly modified your program, not because it is not good enough but to make it better readable. Be careful with using nonCAS variables for CAS results. The CAS can handle integers of order larger than \( 10^{15} \) whereas we lose significant digits using nonCAS variables. So for "small" cases, your modifications would work just fine. However, for "larger" numbers (where after conversion into base 10, we would exceed the internal limit on the size of an integer), you would likely get the wrong result due to the conversion of an exact value from the CAS side into an approximate value in the nonCAS side. Graph 3D  QPI  SolveSys 

02222017, 06:03 PM
Post: #22




RE: Base converter with fractions?
(02212017 07:30 PM)Han Wrote: Be careful with using nonCAS variables for CAS results. The CAS can handle integers of order larger than \( 10^{15} \) whereas we lose significant digits using nonCAS variables. So for "small" cases, your modifications would work just fine. However, for "larger" numbers (where after conversion into base 10, we would exceed the internal limit on the size of an integer), you would likely get the wrong result due to the conversion of an exact value from the CAS side into an approximate value in the nonCAS side. You are completely right Han. I did some testing, but not enough. I examined the content of tmp(2) and found out that it contained fractions, like 31/32 instead of numeric values. Therefore I thought it did exact calculations, but you are right that this goes wrong for large integers, numerators or denominators. 

« Next Oldest  Next Newest »

User(s) browsing this thread: