A quick precision test

[Claudio L.]

(06-05-2014 10:20 PM)Paul Dale Wrote:  

(06-05-2014 09:25 PM)Claudio L. Wrote:  With this, I can get transcendentals correct to the last digit that will be displayed to the user on all the test cases I tried (except the ones where you lose precision, like your COS(near PI) case, but that comes from the argument, not the algorithm).

The COS example I gave can be calculated correctly. The argument isn't the issue -- it is represented exactly in the floating point number system. It really is the algorithm causing the loss. Witness that there are several libraries available that handle the equivalent bad case properly for binary arithmetic GNU's libc, MPFR are just two.


- Pauli

In my case it is not the algorithm, it's the argument. CORDIC does not converge for angles > 1 radian, so I compute that cos() as sin(PI/2-alpha). Here's where I loose the digits: My PI constant only has 2016 digits, so if alpha = PI/2-1e-35, my substraction PI/2-(PI/2-1e-35) gives me 1e-35 with only 2016-35 accurate digits. That's where I'm losing my 35 digits. I could get them back only by storing PI/2 with twice the system precision, so PI/2-1ULP would give me (2016*2-2016)=2016 good digits.

Claudio