The Rogue HP-12c

[dm319]

Ok Albert has solved it. We think it is likely BCD.

I had been working on the principal that some 16 digit decimals are not representable by 64 bit binary, but I was struggling to find them. Albert pointed out to me that numbers just under 1e16 often shared binary representations. Here are representations from 1e16 down to 9999999999999990:

Code:

[1] 00 80 e0 37 79 c3 41 43
[1] 00 80 e0 37 79 c3 41 43
[1] ff 7f e0 37 79 c3 41 43
[1] fe 7f e0 37 79 c3 41 43
[1] fe 7f e0 37 79 c3 41 43
[1] fe 7f e0 37 79 c3 41 43
[1] fd 7f e0 37 79 c3 41 43
[1] fc 7f e0 37 79 c3 41 43
[1] fc 7f e0 37 79 c3 41 43
[1] fc 7f e0 37 79 c3 41 43
[1] fb 7f e0 37 79 c3 41 43

I think the representation is big-endian in this case, but you can see several values have the same encoding. Albert tells me that numbers between 2^53 and 1e16 must be an even integer! So:

1e16
3
-

should yield an odd number if BCD and even if binary. I was subtracting 1e16 afterwards to look at what number had been calculated, but sadly the 12c was returning 0. In fact it did it for 1e15 and 1e14 also, which suggests it has some funny rounding algorithm.

To prevent the 'flush-to-zero', Albert suggested:

1e16
999999
-
999999e10
-

which should have yielded:
9'999'000'001

but instead we got:
9'999'000'010

This is likely because

Code:

10'000'000'000'000'000
               999'999 -

is truncated to 16 digits:

Code:

10'000'000'000'000'00_
               999'99_ -

Using an example where we weren't pushing into 17 digits:

999999e10
999999
-
999998e10
-

Then we get:
9'999'000'001 rather than 9'999'000'000 as expected with binary.

Very much channelling Albert here - but it doesn't look like binary due to the above solution, and it looks like BCD as the truncation happened when aligning decimal digits - yielding the unexpected 10.

So there we have it. Looks like BCD, but doing some weird rounding stuff also.

EDIT: Type 2^52 -> 2^53. And clarification of why it is BCD.