Post Reply 
Inaccuracy of TAN near 75° in rad mode
06-30-2015, 08:45 PM (This post was last modified: 06-30-2015 08:53 PM by Dieter.)
Post: #20
RE: [WP 34s] Innacuracy of TAN near 75° in rad mode
(06-30-2015 08:11 PM)Marcio Wrote:  In this particular case, the 73 degrees, say it was rounded to 9 digits instead?

?!? - sorry, I don't understand what you mean here. But the 73° case shows the roundoff problem very nicely:

73° = 1,27409035395586... rad
So the 12-digit result is almost halfway between 1,27409035395 and ...96.
tan(1,27409035395) = 3,27085261842
tan(1,27409035396) = 3,27085261853

The latter is what you get on a 12-digit calculator where 73° "equals" 1,27409035396.

The calculator should return tan(1,27409035395586...) = 3,27085261848
But all it can calculate is tan(1,27409035395) = 3,27085261842    or    tan(1,27409035396) = 3,27085261853

The error in the result is the error in the argument (4,14E–12) times the tan derivative at this point (approx. 11,7), which yields 4,8E–11 or 5 units in the 12th digit – as you can see here: ...1848 vs. ...1853. Changing the argument by the smallest possible delta (1 ULP) makes the tangent change by 11–12 ULP (cf. ...1842 vs. ...1853).

Dieter
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: [WP 34s] Innacuracy of TAN near 75° in rad mode - Dieter - 06-30-2015 08:45 PM



User(s) browsing this thread: 1 Guest(s)