newRPL: [UPDATED April 27-2017] Firmware for testing available for download
|
06-18-2016, 06:10 PM
Post: #303
|
|||
|
|||
RE: newRPL: [UPDATED May-06-16] Firmware for testing available for download
(06-18-2016 03:17 PM)Guenter Schink Wrote: Weird! Actually, the result I got is '(100000001/1000000000)^1000000000'. What happens is you stored an exact value into X, therefore EVAL will try to compute only an exact result. This means the fraction 1+1/1e9 results in 100000001/100000000, then it does the power, which means multiplying this fraction by itself 1e9 is of the order of 2^29, so it does at least 29 fraction multiplications, each time simplifying the fraction and increasing the precision as needed to keep the result exact. After a while, it reaches a point where the maximum system precision of 2000 digits is not enough to produce an exact answer, so it gives up and presents the result in exact form. Now if you store 1e9. (see the trailing dot there?) the system will know you are interested in approximated numbers, then it will instantly present you with the same answer as ->NUM, at the current system precision. |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 7 Guest(s)