Post Reply 
Approximate pi to 24 digits via keyboard
01-31-2015, 06:17 PM
Post: #8
RE: Approximate pi to 24 digits via keyboard
(01-31-2015 11:25 AM)Gilles Wrote:  After some test I think there is a rounding process with the 12th (no displayed) digit :
[ image ]
It looks curious but explains why sqrt(7)*sqrt(7) returns exactly 7 on my old Casio602P

Hmmm... calculate a result with 12 digits, and if this result matches an exact 10-digit result within +7/–8 ULP, simply return this rounded 10-digit result. Sounds like a typical example for cosmetic rounding. ;-)

Dieter
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Approximate pi to 24 digits via keyboard - Dieter - 01-31-2015 06:17 PM



User(s) browsing this thread: 5 Guest(s)