Accuracy of Free42 and DM42
|
12-21-2017, 08:30 AM
Post: #10
|
|||
|
|||
RE: Accuracy of Free42 and DM42
Found it -- the special case for 45° is coded like
if (x == 45) result = sqrt(0.5) In the decimal case, this sqrt() call needs to be resolved as the Phloat version, not the standard C library double version. Apparently the desired thing happens in most builds, but not in the Windows build, and of course the standard library sqrt() only gives 16 digits of precision. Do the latest Windows builds also do this? Now that I'm thinking about this, it's kind of odd that this would ever do the right thing. Instead of sqrt(0.5) it should be sqrt(phloat(0.5)). |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 5 Guest(s)