Post Reply 
More then 12 digit...
12-18-2017, 05:07 AM (This post was last modified: 12-18-2017 05:09 AM by Joe Horn.)
Post: #8
RE: More then 12 digit...
My "dec" program for the Prime does NOT perform extended-precision math. It merely outputs ALL the digits of CAS floating-point numbers, which are stored in binary form internally (not 12-digit BCD like traditional HP calculators, and Prime's Home view) and are therefore never fully displayed to the user (contrary to HP's tradition of "What You See Is What You Have"). So the output of the "dec" program is "exact" only in the sense that it lets you see the exact decimal representation of any CAS floating point number. That's dec's only purpose: to let the user see exactly what number they actually have. It does NOT increase Prime's math accuracy in any way.

Example: In Home, PI is stored exactly as the BCD value 3.14159265359 with each digit occupying one BCD position in the 12-digit mantissa. In CAS, however, all floating point numbers are stored internally as a 48-bit binary number with a binary exponent. Here's CAS's pi exactly represented in hex:

1.921FB54442Dp+1
... which means "1.921FB54442D in hex, times 2^1".

And here is the exactly equivalent number expressed in decimal form:
3.1415926535897824578569270670413970947265625

Notice that the above is the same as the ACTUAL value of pi (the irrational mathematical constant) only for the first 14 digits (underlined above), after which all the digits are WRONG.

Fun example: dec(.1) returns 9.99999999999994315658113919198513031005859375E-2 because in CAS what LOOKS like 0.1 is ACTUALLY that huge mess internally (or 1.999999999998p-4 if you prefer hex). Why? Because that's the largest number less than or equal to 0.1 which can be represented in 48 bits.

If Prime's CAS had the LongFloat library implemented, the user would be able to set DIGITS:=45 and then obtain the CORRECT value of pi to all 45 places. Maybe some day!

<0|ɸ|0>
-Joe-
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
More then 12 digit... - salvomic - 12-16-2017, 01:41 PM
RE: More then 12 digit... - Joe Horn - 12-17-2017, 05:20 AM
RE: More then 12 digit... - salvomic - 12-17-2017, 07:29 AM
RE: More then 12 digit... - Archanus - 12-17-2017, 04:50 PM
RE: More then 12 digit... - salvomic - 12-17-2017, 05:12 PM
RE: More then 12 digit... - Archanus - 12-17-2017, 05:38 PM
RE: More then 12 digit... - salvomic - 12-17-2017, 05:41 PM
RE: More then 12 digit... - Joe Horn - 12-18-2017 05:07 AM
RE: More then 12 digit... - salvomic - 12-18-2017, 09:15 AM
RE: More then 12 digit... - Stevetuc - 12-20-2017, 08:17 AM
RE: More then 12 digit... - pier4r - 12-19-2017, 06:17 AM
RE: More then 12 digit... - salvomic - 12-19-2017, 11:44 AM
RE: More then 12 digit... - GregLewis - 12-19-2017, 01:53 PM
RE: More then 12 digit... - Tim Wessman - 12-20-2017, 01:42 AM
RE: More then 12 digit... - grsbanks - 12-20-2017, 08:33 AM
RE: More then 12 digit... - salvomic - 12-20-2017, 08:40 AM
RE: More then 12 digit... - Claudio L. - 12-21-2017, 12:01 AM
RE: More then 12 digit... - pier4r - 12-21-2017, 11:54 AM
RE: More then 12 digit... - FabVir - 03-15-2022, 05:26 PM
RE: More then 12 digit... - parisse - 03-20-2022, 12:01 PM
RE: More then 12 digit... - FabVir - 03-24-2022, 10:02 PM
RE: More then 12 digit... - Thomas_Sch - 03-30-2022, 11:40 AM



User(s) browsing this thread: 1 Guest(s)