HP calcs are really not that accurate..
|
12-01-2017, 08:44 PM
Post: #5
|
|||
|
|||
RE: HP calcs are really not that accurate..
(12-01-2017 07:49 PM)DA74254 Wrote: I put 2 on the stack and sqrt it 5 times, then squared it back. All my 5 HP's returned 1.99999999979 instead of 2.0. That goes for the Android Pro version as well. Make that 78 times and you will get 1.99999999963 on Free42. Your 12-digit HPs have only 3 guard digits and intermediate results are rounded to 12 digits. Free42 has 22 guard digits, so to say, and it does not round intermediate results. The HP rounding philosophy is better explained in these old threads started by Rodger Rosenbaum; What should you get? What should we get, Part 2 |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 5 Guest(s)