DB48X: HP48-like RPL implementation for DM42

[c3d]

(10-02-2024 06:27 PM)grbrum Wrote:  Still whatever was wrong with my state file, it is still interesting that this error only happened with 1100 / 2.2

It actually happens if you manually enter 500.0 or 500. on the command line, i.e. the problem is with the value 500.

I found and fixed the problem. You correctly attributed it to the NoTrailingDecimal setting. The elimination of the trailing . was triggered, which caused it to, internally, write 500, suppressing the . because there were no non-zero digits left to display, then write 0000 to respect the FIX 4 setting.

If you select FIX 2, then you get 50000, and so on.

Note that the internal representation for variable-size decimals uses a dense binary-decimal encoding, where 10 bits store three decimal digits (values 0-999, the values 1000-1023 being "lost"). As a result, the problem does not occur for 50.0 or 5.0, you need at least 3 digits for it to actually trigger.

Quote:I also tried 1000/2.2 ; 900/2.2; 1200/2.2; 1100/2 and all return expected results.
only 1100/2.2 gave a weird display result.

It's because all these other values have trailing decimals, except the 1100/2 calculation, which was done using only integers. If you had typed 1100/2. or 1100./2, i.e. with a decimal separator on either side, you get the same problem with 5500000 being shown.

Quote:by means of elimination, the error is on the NoTrailingDecimal line.

You are entirely correct.

Thanks a lot for the report, this will be fixed in the next release.