A quick precision test

[Paul Dale]

(06-05-2014 09:25 PM)Claudio L. Wrote:  With this, I can get transcendentals correct to the last digit that will be displayed to the user on all the test cases I tried (except the ones where you lose precision, like your COS(near PI) case, but that comes from the argument, not the algorithm).

The COS example I gave can be calculated correctly. The argument isn't the issue -- it is represented exactly in the floating point number system. It really is the algorithm causing the loss. Witness that there are several libraries available that handle the equivalent bad case properly for binary arithmetic GNU's libc, MPFR are just two.


- Pauli