Post Reply 
Discrepancy between HP 50g and DM42
03-13-2021, 01:51 PM (This post was last modified: 03-13-2021 01:52 PM by PukkaJ.)
Post: #8
RE: Discrepancy between HP 50g and DM42
Just for clarification: 5.67E-8 has 2 decimals in the mantissa, but doesn’t that internally become 0.0000000567 so you really end up with 10 decimals (so you can round the result to a higher precision then when you use say 5.67, all depending on the other parameters involved of course)?
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Discrepancy between HP 50g and DM42 - PukkaJ - 03-13-2021 01:51 PM



User(s) browsing this thread: 1 Guest(s)