Post Reply 
Accuracy?
06-18-2021, 09:51 PM
Post: #8
RE: Accuracy?
(06-18-2021 05:01 PM)HansB Wrote:  HP Prime does the same
1/3+1/3+1/3 -> .999999999999 (12 digits)
(unless you use CAS view).

So does the Prime work internally to 15 digits, but rounds every intermediate and the end result to 12 digits? Why?

Prime Home uses the same floating-point format (12-digit BCD, 15 digits internally) as all Saturn-based HP calculators since the 71B. Prime CAS uses the more common 64 bit binary floating point format, but integers and rational numbers are exact, like the 50g exact mode.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
Accuracy? - HansB - 06-18-2021, 09:44 AM
RE: Accuracy? - Albert Chan - 06-18-2021, 03:26 PM
RE: Accuracy? - Valentin Albillo - 06-18-2021, 04:04 PM
RE: Accuracy? - HansB - 06-18-2021, 05:01 PM
RE: Accuracy? - rampo - 06-18-2021, 05:16 PM
RE: Accuracy? - John Keith - 06-18-2021 09:51 PM
RE: Accuracy? - Steve Simpkin - 06-18-2021, 06:40 PM
RE: Accuracy? - rprosperi - 06-18-2021, 10:02 PM
RE: Accuracy? - Valentin Albillo - 06-19-2021, 02:47 AM
RE: Accuracy? - rampo - 06-18-2021, 05:13 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 03:07 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 05:46 PM



User(s) browsing this thread: 4 Guest(s)