3421A CAL RAM

[djd328]

(04-25-2019 08:44 PM)Dave Frederickson Wrote:  

(04-25-2019 05:24 AM)djd328 Wrote:  Did you investigate further?

No I did not. Once I was able to save, restore, and reset the CAL RAM data I concluded my investigation.

Regarding the 9's, like your YYYYYYY value for the 3v range, I would think that would be a negative offset from zero.

Dave

Ok understood, it's an interesting problem, but yes I've got my constants and can reload them... the only reason to care is if I want to manually tune them (and I do have a reason to do that, but that's for another time).

9s... Yes, that seems plausible, but then 999 = 100110011001is a lot of sign bits! One parity bit and one sign bit? I think it's pretty likely that those are -ve numbers, but it's not straight sign/mag or 2's compliment. Nibble cells seem also encoded, with error detect correct code probably not limited to the 4 nibbles at the end of each row.