CAS: Keyed in 1 SHIFT x 90 but did not get (0,1)

[akmarul]

Hello all. I keyed in 1 angle 90 degree in CAS but did not get (0,1). But i got (1.25e-14,1). Is this a bug. I have latest FW on my G2. Thanks.

[toml_12953]

(12-29-2022 02:05 PM)akmarul Wrote:  Hello all. I keyed in 1 angle 90 degree in CAS but did not get (0,1). But i got (1.25e-14,1). Is this a bug. I have latest FW on my G2. Thanks.

This is a result of floating point rounding. Most calculators and computers have it. It's not a bug, just a limitation on numeric representation. 1.25e-14 is so close to 0 that in real life you'd never notice the difference. Some calculators try to give you the answer you expect rather than what's actually calculated and so introduce errors of their own.

0.0000000000000125

is pretty small!

[akmarul]

Thanks Tom L. I don't see the same rounding in HOME (non-CAS). If I key in 1 angle 90degree, I will get (0,1). Is this intended? Thanks.

[akmarul]

(12-29-2022 03:02 PM)toml_12953 Wrote:  

(12-29-2022 02:05 PM)akmarul Wrote:  Hello all. I keyed in 1 angle 90 degree in CAS but did not get (0,1). But i got (1.25e-14,1). Is this a bug. I have latest FW on my G2. Thanks.

This is a result of floating point rounding. Most calculators and computers have it. It's not a bug, just a limitation on numeric representation. 1.25e-14 is so close to 0 that in real life you'd never notice the difference. Some calculators try to give you the answer you expect rather than what's actually calculated and so introduce errors of their own.

0.0000000000000125

is pretty small!

Thanks Tom. In HOME, I don't see the same issue. How is HOME different on this? Thanks.