PROBLEM WITH PI π

[bxparks]

(12-29-2023 02:39 PM)Albert Chan Wrote:  float(1/3)*3 = 1 ± ULP(1/3) = 1 ± ULP(1)/4 --> rounded back to 1
[...]
float(1/3)*3 = 1 - ULP(1/3) = 1 - ULP(1)/10 --> rounded below 1

Interesting. I'm not familiar with ULP analysis. I know that it stands for "unit in last place", but how does the second equality happen? The "ULP(1/3) = ULP(1)/4" and the "ULP(1/3) = ULP(1)/10".

My reasoning was that 1/3 = 0.0101_0101 (base-2, to 8 bits in this example). Multiply that by 11 (base-2) to get 0.1111_1111 (base-2). Subtract 1 to get -0.0000_0001 (base-2), which cannot equal 0.

(12-29-2023 02:39 PM)Albert Chan Wrote:  TI is getting 0 because of its flush-to-zero "feature".

That's a good explanation for what's happening, thank you.