Post Reply 
Question for Parisse about CAS floating point
04-10-2015, 07:58 PM
Post: #7
RE: Question for Parisse about CAS floating point
(04-07-2015 02:44 AM)Joe Horn Wrote:  
(04-06-2015 01:44 PM)compsystems Wrote:  the nspire and ti68k calculators work with 12 digits, I think this should be the lower limit to improve accuracy, you say?

hp-prime calculator

approx(1519/99) => 15.3434343434 10 digits =(

That's merely what's shown in the display. The internal value in CAS is exactly:
15.343434343434267930206260643899440765380859375
Count the underlined digits.

My understanding of the CAS semantics for its is inexact numeric values is that they are not exactly equal to any rational or real number. That said, interpreting the CAS value as an IEEE 754 double I seem to get
15.3434343434342963519156910479068756103515625

In an Advanced Graphing app plot, the lower and upper bounds used for 1519/99 (when the fraction is worked with in an approximate manner) are
15.343434343434342537193515454418957233428955078125
and
15.3434343434343443135503548546694219112396240234375,
respectively.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Question for Parisse about CAS floating point - jte - 04-10-2015 07:58 PM



User(s) browsing this thread: 1 Guest(s)