+- HP Forums (https://www.hpmuseum.org/forum)
+-- Forum: HP Calculators (and very old HP Computers) (/forum-3.html)
+--- Forum: HP Prime (/forum-5.html)
+--- Thread: CAS: Keyed in 1 SHIFT x 90 but did not get (0,1) (/thread-19355.html)
CAS: Keyed in 1 SHIFT x 90 but did not get (0,1) - akmarul - 12-29-2022 02:05 PM
Hello all. I keyed in 1 angle 90 degree in CAS but did not get (0,1). But i got (1.25e-14,1). Is this a bug. I have latest FW on my G2. Thanks.
RE: CAS: Keyed in 1 SHIFT x 90 but did not get (0,1) - toml_12953 - 12-29-2022 03:02 PM
(12-29-2022 02:05 PM)akmarul Wrote: Hello all. I keyed in 1 angle 90 degree in CAS but did not get (0,1). But i got (1.25e-14,1). Is this a bug. I have latest FW on my G2. Thanks.
This is a result of floating point rounding. Most calculators and computers have it. It's not a bug, just a limitation on numeric representation. 1.25e-14 is so close to 0 that in real life you'd never notice the difference. Some calculators try to give you the answer you expect rather than what's actually calculated and so introduce errors of their own.
0.0000000000000125
is pretty small!
RE: CAS: Keyed in 1 SHIFT x 90 but did not get (0,1) - akmarul - 12-29-2022 11:15 PM
Thanks Tom L. I don't see the same rounding in HOME (non-CAS). If I key in 1 angle 90degree, I will get (0,1). Is this intended? Thanks.
RE: CAS: Keyed in 1 SHIFT x 90 but did not get (0,1) - akmarul - 12-30-2022 07:03 AM
(12-29-2022 03:02 PM)toml_12953 Wrote:(12-29-2022 02:05 PM)akmarul Wrote: Hello all. I keyed in 1 angle 90 degree in CAS but did not get (0,1). But i got (1.25e-14,1). Is this a bug. I have latest FW on my G2. Thanks.
This is a result of floating point rounding. Most calculators and computers have it. It's not a bug, just a limitation on numeric representation. 1.25e-14 is so close to 0 that in real life you'd never notice the difference. Some calculators try to give you the answer you expect rather than what's actually calculated and so introduce errors of their own.
0.0000000000000125
is pretty small!
Thanks Tom. In HOME, I don't see the same issue. How is HOME different on this? Thanks.