HP Prime Miscalculating

10242015, 12:48 PM
Post: #12




RE: HP Prime Miscalculating
(10232015 05:41 AM)Joe Horn Wrote: Long answer: It's because Home uses BCD (which can represent 3.6 and 3.24 exactly) and CAS uses binary floating point (which can't). When you type 3.6 in Home, it's EXACTLY 3.6, but when you type 3.6 in CAS, it actually generates this value: So the prime uses a 48 bits mantissa to work with binary floating point, and not 52 like in IEEE 754 double precision. How will be a number represented? I mean in double precision you have 1 bit for the sign, 11 for the exponent and 52 for the mantissa. What about the Prime? If it works with 64 bits and the mantissa is truncated from 52 to 48 what are the 4 "missing" bits used for? And how is a BCD number represented? I would say 48 bits for the mantissa as we have 12 digits (which is the same as in CAS, so it seems consistent). And the other bits? Thank you for the answer Reto 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)