Inaccuracy of TAN near 75° in rad mode
06-30-2015, 08:45 PM (This post was last modified: 06-30-2015 08:53 PM by Dieter.)
Post: #20
 Dieter Senior Member Posts: 2,397 Joined: Dec 2013
RE: [WP 34s] Innacuracy of TAN near 75° in rad mode
(06-30-2015 08:11 PM)Marcio Wrote:  In this particular case, the 73 degrees, say it was rounded to 9 digits instead?

?!? - sorry, I don't understand what you mean here. But the 73° case shows the roundoff problem very nicely:

So the 12-digit result is almost halfway between 1,27409035395 and ...96.
tan(1,27409035395) = 3,27085261842
tan(1,27409035396) = 3,27085261853

The latter is what you get on a 12-digit calculator where 73° "equals" 1,27409035396.

The calculator should return tan(1,27409035395586...) = 3,27085261848
But all it can calculate is tan(1,27409035395) = 3,27085261842    or    tan(1,27409035396) = 3,27085261853

The error in the result is the error in the argument (4,14E–12) times the tan derivative at this point (approx. 11,7), which yields 4,8E–11 or 5 units in the 12th digit – as you can see here: ...1848 vs. ...1853. Changing the argument by the smallest possible delta (1 ULP) makes the tangent change by 11–12 ULP (cf. ...1842 vs. ...1853).

Dieter
 « Next Oldest | Next Newest »