Post Reply 
Division vs DIV - Valentin's findings
04-02-2023, 05:31 PM
Post: #3
RE: Division vs DIV - Valentin's findings
Mmm, but... if what we had was a rounding from a 13th digit, wouldn't we see it about half of the time? We see it rarely, and I think we don't believe there is a 13th digit, so there's something about the mechanics of division here which is, I think, a bit surprising.

(These are decimal calculations... division proceeds by shift and subtract? Is there an estimating of the next digit?)
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Division vs DIV - Valentin's findings - EdS2 - 04-02-2023 05:31 PM



User(s) browsing this thread: 1 Guest(s)