sin(x)^2+cos(x)^2 = 1 on different calcs
|
09-26-2018, 06:27 PM
Post: #17
|
|||
|
|||
RE: sin(x)^2+cos(x)^2 = 1 on different calcs
(09-26-2018 05:27 AM)cyrille de brébisson Wrote: =100-99.99-0.01 -> 5.1157E-15 The difference is not small, if think in ULPs (above case, 1 ULP = 2^-59): 100 - float(99.99) = 0x147ae147ae2000 ULP float(0.01) = 0x147ae147ae147b ULP Difference = 0x2000 - 0x147b = 2949 ULP Quote:However, I have NO clue what might be going on with I am using *old* MS Excel 95. It does not have set-to-zero "feature". So, both expressions return the same -5.6E-17 |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 3 Guest(s)