Post Reply 
"Why do calculators get this wrong?" (YouTube)
07-20-2020, 12:25 PM
Post: #5
RE: "Why do calculators get this wrong?" (YouTube)
(07-19-2020 03:24 PM)EdS2 Wrote:  ... I see a commenter has found that the denominator needs to be a factor of 25200 for the search to work.

Perhaps Casio is not going for "best" rational approximation, but simply multiply by 25200/pi

lua> k = 25200/pi       -- ≈ 8021.409131831525
lua> 11^6/13 * k
1093108890.9997375
lua> 11^6/17 * k
835906798.9997994

Rounded to 13 digits, first number is integer, but not the second

→ 11^6/13 ≈ 1093108891/25200 * pi = 156158413/3600 * pi
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: "Why do calculators get this wrong?" (YouTube) - Albert Chan - 07-20-2020 12:25 PM



User(s) browsing this thread: 1 Guest(s)