[WP34S] DEG and RAD  diffs

06072014, 01:46 AM
(This post was last modified: 06072014 01:49 AM by Claudio L..)
Post: #30




RE: [WP34S] DEG and RAD  diffs
(06062014 10:53 PM)Paul Dale Wrote:(06062014 07:53 PM)Claudio L. Wrote: Also, when x is close to zero, doing sqrt(1+x^2) is really bad for precision. The x^2 "spreads" your useful digits throughout your exponent range, then the square root compresses them back, and you lost about half of them at the end (see what I meant with "small angles are tougher on precision loss"). You'r right, that term won't lose precision. I looked at my code, and here's where I was losing digits: // COMPUTE ACOS(X) = ATAN2(SQRT(1X^2),X) when x~1 (ACOS(X)~0) I couldn't get all the digits right (this is when the sqrt() brings your digits back). I should double check my posts, but I don't always have my devel machine with me to look at the code. Claudio 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)