Inaccuracy of TAN near 75° in rad mode

06302015, 08:45 PM
(This post was last modified: 06302015 08:53 PM by Dieter.)
Post: #20




RE: [WP 34s] Innacuracy of TAN near 75° in rad mode
(06302015 08:11 PM)Marcio Wrote: In this particular case, the 73 degrees, say it was rounded to 9 digits instead? ?!?  sorry, I don't understand what you mean here. But the 73° case shows the roundoff problem very nicely: 73° = 1,27409035395586... rad So the 12digit result is almost halfway between 1,27409035395 and ...96. tan(1,27409035395) = 3,27085261842 tan(1,27409035396) = 3,27085261853 The latter is what you get on a 12digit calculator where 73° "equals" 1,27409035396. The calculator should return tan(1,27409035395586...) = 3,27085261848 But all it can calculate is tan(1,27409035395) = 3,27085261842 or tan(1,27409035396) = 3,27085261853 The error in the result is the error in the argument (4,14E–12) times the tan derivative at this point (approx. 11,7), which yields 4,8E–11 or 5 units in the 12th digit – as you can see here: ...1848 vs. ...1853. Changing the argument by the smallest possible delta (1 ULP) makes the tangent change by 11–12 ULP (cf. ...1842 vs. ...1853). Dieter 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)