A quick precision test

[Claudio L.]

(06-05-2014 12:06 PM)Paul Dale Wrote:  As an example (which isn't related to rounding), what does COS(1.57079632679489661923132169163975144) equal? Is it 2.09858469968755291048739331660132323e-36 or something different? I cheated here and downloaded your (rather nice BTW) newRPL demo and it gets the last thirteen digits incorrect. TAN of the same value loses fourteen digits and this answer is a huge number rather than something very tiny.

My original 34S test case was COS(1.570796326794896619231321691639751) where newRPL gets the last ten digits incorrect -- the 34S gets more digits incorrect here. With more memory available, I'd fix this.


- Pauli

2 things: the first demo came out with a bug that returns the TAN of PI/2-Arg instead of TAN(arg) (among several other bugs I've already fixed), I'll publish a new demo this weekend, now with DOLIST [end of shameless advertising].
I did your test:
COS(1.57079632679489661923132169163975144)

and I got
2.098584699687552910487472296153908

What makes you think those digits are incorrect? I did the same with preccalc (from sourceforge, very nice calc for windows), and it gives me the exact same value.
I've also increased precision to many more digits (keeping the argument at 36 digits they way you have it) and still the same result.

newRPL does lose digits, of course, it's part of the implementation, but in this case, your angle is in the order of 10^-35 difference with PI/2, so I'm losing 35 digits, but you can only see them at maximum precision (2016 digits).
I for instance ran it at 2016-35=1981 digits precision and still gave me identical results as preccalc, up to the very last digit.
If I run it at 2016 digits, I can clearly see the last 35 digits are garbage.

I've also checked with Wolfram Alpha, and... exact same number! (I checked up to 36 digits only, the others are left to the reader to compare...)

hint: Alpha won't give you more than the number of digits you input, so it will show only 5 digits unless you add lots of trailing zeros to the input argument.

I also tried your COS() case with 34 digits and once more, it agrees with preccalc and Alpha on all 34 digits.

I think you are taking for granted some results as "correct" that are perhaps not too good?

Claudio