Calculation of determinant on HP-15C

[Valentin Albillo]

(03-12-2021 08:26 AM)Werner Wrote:  Certainly on the 48G and up, when flag -54 is clear (as I believe it is, by default), then an extra check is performed before the determinant is calculated: to see whether all matrix elements are integer - for then, the result must be integer, too, and it is *changed* to the nearest integer

This was discussed at length in several threads in the old MoHPC forum decades ago, and I remember that someone actually produced a (small) matrix where using that "cheating" flag resulted in the integer determinant being computed with an error of one full unit, so the cheating didn't pay that time as you got a much worse (a full unit) incorrect result by using it than without.

The general conclusion was that tweaking the results to make them conform to (integer) expectations (and to try and mimic greater accuracy that actually wasn't there) was a shoddy work which might embarrassingly backfire. Examples where this was done for other non-HP calculators were also discussed as well.

Regards.
V.