Pi/2 radians different from 90 deg?

[Joe Horn]

(12-13-2021 07:35 PM)celltx Wrote:  Thank you, Joe.

Actually, not really 12 digits. To get a similar result in degrees, one needs to calculate tan(90+3e-8). So, it's more like 9-10 digit accuracy.

The reason that final results are not always accurate to 12 digits is that intermediate results are rounded to 12 digits, and the roundoff error can accumulate. Each intermediate step of every algebraic calculation in Home view is calculated internally to 15 truncated digits, then that result is rounded to 12 significant digits. For example, tan(pi/2) is calculated internally like this:

Step 1: pi
--> 3.14159265358979 (15 truncated digits)
--> gets rounded to exactly 3.14159265359

Step 2: take THAT number (which is not pi!) and divide THAT by 2
--> 1.570796326795 (exactly)
--> gets rounded to exactly 1.5707963268
Notice that roundoff error has already accumulated: pi/2 is closer to 1.57079632679

Step 3: take the tangent of THAT result (which is not pi/2)
--> -195948537905.977 (15 truncated digits)
--> gets rounded to exactly -195948537906