(11C) (15C) Very fast binary converter + fast modulo

03172018, 09:17 PM
(This post was last modified: 03172018 09:20 PM by Dieter.)
Post: #5




RE: (11C) Very fast binary converter + fast modulo
(03172018 06:06 PM)michaelzinn Wrote: > try 25 mod 7 to get 3,999999997 instead of 4. The basic problem is that it is impossible to exactly represent all fractions with a limited number of digits. 1/7 = 0,142857142857...., so multiplying this by 7 again with finite precision will never yield an integer result. Rounding helps here and there, but not in any case. In the 25 mod 7 case this happens with 10digit precision: 25/7 => 3,571428571 FRAC => 0,5714285710 x 7 => 3,9999999997 It gets even worse with 25000 mod 7: 25000/7 => 3571,428571 FRAC => 0,4285710000 // now only six decimals are left ! x 7 => 2,999997000 (03172018 06:06 PM)michaelzinn Wrote: Can anyone recommend a more accurate emulator? Sorry, no idea. But you seem to use a simulator instead of an an emulator (which would reproduce the real thing exactly, running the original firmware). If, for instance, the calculations in your simulator are performend in standard 15digit binary arithmetics, roundoff errors like the one above are covered since only 10 digits are visible in the display. What do you get if you see the result "4" with the modulo routine and then subtract 4 from this? It is 0 or something different? And what is the result if you calculate 10/3 (=3,333333333) and then double the result? Is it 6,666666666 or 6,666666667 ? (03172018 06:06 PM)michaelzinn Wrote: Heh, I ignored LastX and how stack lifting works because I didn't understand it quickly. I guess I should go back to reading the manual. From your blog post I understand that all this is new to you and something like ..."vintage computing", while most of the members here have grown up with all this, and RPN is nothing that one even has to think about – it feels natural and intuitive. ;) (03172018 06:06 PM)michaelzinn Wrote: Also, how exactly are numbers stored in registers? The app I'm using might be inaccurate here, do real HP calculators use decimals with a floating point? Or is it binary? No, it's decimal arithmetics (BCD). The internal registers consist of 7 bytes = 56 bits with one nybble (4 bits) for each digit in mantissa (10) and exponent (2), and two more for their respective signs. Since the midseventies most HPs use extended internal precision with 3 more guard digits for their calculations. This significantly improves the accuracy of the more complex functions since these are calculated with 13digit internal precision before the result is rounded back to 10 digits and returned to the user. Dieter 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)