"Why do calculators get this wrong?" (YouTube)

[EdS2]

(07-20-2020 02:37 PM)EdS2 Wrote:  Thanks Albert - 13 digits might well be the solution to the problem.

I should have said more: it's the idea that multiplying by 3600 and testing for an integer result is a likely approach, combined with the observation that the two cases produce results with different near-integer sizes, such that one case has more digits beyond the point. So the closer match of the /17 case no longer looks closer, because the /13 case lost one digit of precision.

Very interesting.