Post Reply 
Accuracy?
06-18-2021, 09:44 AM (This post was last modified: 06-18-2021 10:13 AM by HansB.)
Post: #1
Accuracy?
Why does my Prime G2 shows different results with equal inputs? I like to understand how accurate to how many digits Prime calculates, and how many digits will be displayed. It appears I cannot trust the last digit displayed.
For instance (all textbook input, standard number format, results grouped here for better readability):
Code:
sqrt(3/4)      -> 0.866 025 403 784    (cos(30))
sqrt(3)/2      -> 0.866 025 403 785
phone calculator: 0.866 025 403 784 438 6...

sqrt(3/7)       -> 0.654 653 670 708
sqrt(3)/sqrt(7) -> 0.654 653 670 71
phone calculator:  0.654 653 670 707 977 1...

cubicroot(3/8) -> 0.721 124 785 154
cubicroot(3)/2 -> 0.721 124 785 155
phone calculator: 0.721 124 785 153 704 1...
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
Accuracy? - HansB - 06-18-2021 09:44 AM
RE: Accuracy? - Albert Chan - 06-18-2021, 03:26 PM
RE: Accuracy? - Valentin Albillo - 06-18-2021, 04:04 PM
RE: Accuracy? - HansB - 06-18-2021, 05:01 PM
RE: Accuracy? - rampo - 06-18-2021, 05:16 PM
RE: Accuracy? - John Keith - 06-18-2021, 09:51 PM
RE: Accuracy? - Steve Simpkin - 06-18-2021, 06:40 PM
RE: Accuracy? - rprosperi - 06-18-2021, 10:02 PM
RE: Accuracy? - Valentin Albillo - 06-19-2021, 02:47 AM
RE: Accuracy? - rampo - 06-18-2021, 05:13 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 03:07 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 05:46 PM



User(s) browsing this thread: 1 Guest(s)