WIP: 16C firmware hack for more memory

[brouhaha]

Back in 2005 when I came up with the first microcode patch for 15C extended memory (which was incomplete, and expanded upon by Hrast Programmer and J-F Garnier), I also wanted to expand the memory of the 16C. At the time, the 16C microcode to translate a user register number to a hardware register number and nibble offset seemed really complicated, and I couldn't figure out how the code dealt with the user registers staring at hardware 0xe0. I gave up on that, and haven't touched it until recently. Having spent time recently digging into the microcode of all Voyager models, I thought maybe it was time to try 16C expanded memory again.

I don't have it completely sorted out, but I did figure out how the 0xe0 RAM base works in both the register addressing, and in limiting the size of a program. I'm able to make that 0xc0, which allows for 427 program steps, or 854 registers (if word size is <= 4 bits). In principle I should be able to expand it even further. but I haven't tried yet.

What works is:
1. STO/RCL direct (0-F, .0-.F) work correctly, and the hardware address is computed correctly.
2. STO/RCL(i) work fine up to some limit that is computed incorrectly, Beyond that they give Error 3. For register numbers below the incorect limit, the STO/RCL (i) works correctly, using the correctly computed hardware address.
3. Can enter, SST, and BST through programs up to 427 steps
4. Label search seems to work correctly throughout the expanded program space.

What doesn't work is:
1. f MEM display of the count of available registers is very wrong
2. The limit for STO/RCL (i) is wrong
3. GTO . nnn doesn't work for 204 and above
4. The display of program line numbers above 210 (not 203!) wrap around, such that 211 is 99, etc., and wrap around again each time the number hits 210.

If I get these issues fixed, I'll try even more memory. Displaying program steps beyond 999 can't easily be made to work, so I'll probably put in a hard limit of 999.

STO/RCL (i) similarly can't easily be made to support register numbers beyond 999, and I'm not going to attempt to do anything beyond that. I'll probably clip the limit of registers displayed by f MEM to 999, even if there is in principle more memory that could be available.

The 16C with the standard-sized Voyager R2D2 ROM (6K words) has only 8 words left over, of which there are no more than 3 consecutive. This patch will require a larger ROM size. That's not a problem in simulation, but I don't know that extra-large ROM would work with HP firmware in an HP ARM 12C (either AT91SAM7L or ATSAM4C). It _might_ fit in the HP 15C LE, because it looks like their image includes the gap between the two ROMs. I've heard that other models don't really work in the HP firmware, but I haven't confirmed that myself. I'm only testing in my own Nonpareil.

Disclaimer: I'm not promising that I'll complete this work or publish it at any particular time. I'm mentioning it for amusement purposes only.

[dmh]

This sounds very interesting, good luck!

I've been wondering about synthetic programming on the other Voyagers that also exhibit the rotate X functionality like on the 15C.

Any chance you've looked or seen anything that might provide insight as to whether this is possible and how (eg alpha display)?

Thanks, dmh

[brouhaha]

(01-22-2023 02:53 AM)brouhaha Wrote:  What doesn't work is:
[...]
3. GTO . nnn doesn't work for 204 and above
4. The display of program line numbers above 210 (not 203!) wrap around, such that 211 is 99, etc., and wrap around again each time the number hits 210.

I've just gotten those fixed.

For GTO ., the 16C firmware has a compare for >= 204. I just had to NOP that out. It still compares to the actual end of program, so I don't think the compare against 204 was actually necessary.

For line number display, the 16C firmware does the conversion from the internal program pointer to the BCD step number in three pieces, for bytes within register, unit registers, and multiples of 16 registers. The code only handled one increment of 16 registers, because there were only 29 total in a real 16C. Patching that to loop fixed it, at least as far as step 427, the last step if the RAM base address is lowered to 0xc0 instead of the standard 0xe0. I think it should be good to 999, though. As with the 15C, line 1000 will probably not be shown correctly.

If I don't patch the firmware to limit program size to 999 steps, how would people feel about steps 1000 to 1nnn being displayed as negative 000 through negative nnn? For that matter, would that be acceptable on the 15C?

[brouhaha]

(01-22-2023 09:08 AM)dmh Wrote:  I've been wondering about synthetic programming on the other Voyagers that also exhibit the rotate X functionality like on the 15C.

Any chance you've looked or seen anything that might provide insight as to whether this is possible and how (eg alpha display)?

The Voyager ON-key14 (y^x, D, PMT) combination causes a hard reset of the Nut processor resulting in an abrupt jump of 22 bit cycles in the Nut. Lesser known is that it does that to all of the Nut's internal 56-bit data registers, and also some other internal non-56-bit registers including the status register. That's why under some conditions, performing that shift operation turns on or off complex mode, WITHOUT allocating or deallocating the complex stack. There are probably other side effects that haven't been characterized.

The main reason the shift is useful for 15C synthetics is that it allows creation of invalid matrix descriptors that point to other places in RAM, such as the pool.

Since none of the other Voyager models have any similar pointer mechanisms, I don't think synthetic programming on the other models is practical with or without the use of ON-D.

The work I'm doing in on modifying the firmware, which is difficult on the real Voyager hardware, but perhaps not too difficult on the ARM-based models from both HP and SwissMicros. That potentially allows us to do anything the hardware is capable of. On the SwissMicros calculators, in principle we could hack the calculator firmware to allow individual pixel control of the LCD, and use of the beeper and real time clock. While this is technically possible, it's not currently on my (near-infinite) to-do list.

I do want to figure out how the Voyager firmware is stored inside the SwissMicros firmware, so that I can test my hacks to the Voyager firmware on them. I haven't yet done anything about that. From what I hear, in early versions of SwissMicros firmware that was easier than in newer versions, but I haven't confirmed that myself.

Alternatively, I could just write my own Nut simulation code etc. to run on the SwissMicros hardware, without need for their proprietary firmware.

[brouhaha]

(01-22-2023 02:53 AM)brouhaha Wrote:  What doesn't work is:
1. f MEM display of the count of available registers is very wrong
2. The limit for STO/RCL (i) is wrong

I've got the f MEM display working, provided there aren't more than 999 registers. I'll have to figure out how to limit it.

I was mistaken about the limit for STO/RCL (i); it seems to be computed correctly.

I still need to try increasing the memory even further.

[Stevetuc]

(01-22-2023 07:00 PM)brouhaha Wrote:  [...]
I think it should be good to 999, though. As with the 15C, line 1000 will probably not be shown correctly.

If I don't patch the firmware to limit program size to 999 steps, how would people feel about steps 1000 to 1nnn being displayed as negative 000 through negative nnn? For that matter, would that be acceptable on the 15C?

999 steps are plenty. However negative display for steps above 999 is a good solution and perhaps better than limiting the steps.

[rprosperi]

(01-22-2023 07:00 PM)brouhaha Wrote:  If I don't patch the firmware to limit program size to 999 steps, how would people feel about steps 1000 to 1nnn being displayed as negative 000 through negative nnn? For that matter, would that be acceptable on the 15C?

That's a clever idea, and IMHO is a great solution. Though feasibly confusing to a new user, anyone writing programs of that size, on either 15C or 16C, will certainly grasp the meaning and make using the editor for large programs clear. NICE!

[J-F Garnier]

(01-23-2023 01:36 PM)rprosperi Wrote:  

(01-22-2023 07:00 PM)brouhaha Wrote:  If I don't patch the firmware to limit program size to 999 steps, how would people feel about steps 1000 to 1nnn being displayed as negative 000 through negative nnn? For that matter, would that be acceptable on the 15C?

That's a clever idea, and IMHO is a great solution. Though feasibly confusing to a new user, anyone writing programs of that size, on either 15C or 16C, will certainly grasp the meaning and make using the editor for large programs clear. NICE!

But it doesn't solve the GTO. (GTO EEX on the 15C) that is still limited to 999, unless Eric finds a clever solution for that too.
Has the 16C the same kind of strange display as the 15C for step 1000 ?
For me, 999 steps are fine.

J-F

[brouhaha]

(01-23-2023 04:47 PM)J-F Garnier Wrote:  But it doesn't solve the GTO. (GTO EEX on the 15C) that is still limited to 999, unless Eric finds a clever solution for that too.

I was thinking about GTO . CHS nnn, which would be somewhat consistent with the negative step number display.

Quote:Has the 16C the same kind of strange display as the 15C for step 1000 ?

It does not. I put in a trivial 1659 step program, the maximum possible without really major changes, and aside from not being able to GTO . past 999, everything seemed to work fine. Insertion or deletion of steps at the beginning of the program is very slow due to having to rotate and move 237 hardware registers, but I'm still running Nonpareil at normal Voyager speed. There's no reason I can't speed it up, like the ARM-based 12C.

The loop that does insertion with register rotation and move has an astonishing amount of stuff in it, beyond what I'd have expected. I don't yet understand why, as I think the inner loop could actually be much simpler. I suppose that with limited memory the performance was good enough that optimization of the loop wasn't thought necessary. The step deletion inner loop is somewhat more streamlined.

Quote:For me, 999 steps are fine.

Hacking in the negative line number display and GTO . CHS would be a fair bit of work, so I'm leaning toward just putting in the 999 step limit, which looks very easy.

Having only 16 labels makes huge numbers of steps have somewhat questionable utility anyhow. I have thoughts for fixing that, but jt would be a much bigger project.

[rprosperi]

(01-24-2023 03:28 AM)brouhaha Wrote:  Hacking in the negative line number display and GTO . CHS would be a fair bit of work, so I'm leaning toward just putting in the 999 step limit, which looks very easy.

I'd do the 999 for now and leave the others for later, if and when (likely not) folks find a real justified need for more. 999 steps is likely good enough for 99.9% of the potential users. But the idea of using negative is still clever elegant.

[brouhaha]

OK, I got the 999 program step limit implemented.

f CLEAR REG wasn't clearing all the data registers. because I'd overlooked the constant it uses of (register RAM address base - 1), so I've fixed that.

I've just discovered that the code that takes a register number (possibly from I) and computes the hardware address and shift position sometimes skips over one nibble, That never happens within the first 203 bytres of registers. With 4 bit registers, and 237 hardware registers for user data & program, and a program taking up two hardware regisres, there are 235 hardware registers available for use as data registers. When the word size is 1 to 4 bits, that should be 3290 4-bit registers (0 through 3289). f MEM shows "290", which is correct aside from being left-truncated to three digits. However, I actually only get 3286 registers (0 through 3285). Four are missing, and they are due to skipped nibbles in RAM at hardware addresses 0x40, 0x71, 0xa2, and 0xd3.

With word size 8, there are also four skipped nibbles, in the same hardware addresses but not the same digit positions, resulting in two 8-bit registers missing.

With word size 12, there's only one missing register, and the skipped nibbles are in the same hardware addresses,

With word size 56, there's only one missing register, and the skipped nibbles are in 0x41, 0x72, 0xa3, and 0xd4, one word higher.

So the skipped nibbles have a definite pattern that should help me identify why the microcode is doing that.

I'm somewhat surprised that the computation works as well as it does, and that it actually supports over 3000 registers. In FLOAT mode, f MEM shows 235 registers, but the indirect addressing in FLOAT can't handle more than 0 through 99.

I need to hack f MEM to be able to show 4 digits of register count. There's enough room in the display if I shift the "r-" one space left.

Question: should I hack the f MEM code to show r-099 in FLOAT mode, or let it show more than 100 even though you can't access those past 99? Limirig it to 99 is probably not too difficult. Certainly not as difficult as hacking the indirect code to allow more than 99 in FLOAT mode. Though maybe I should do that.

[brouhaha]

(01-24-2023 08:00 AM)brouhaha Wrote:  I need to hack f MEM to be able to show 4 digits of register count. There's enough room in the display if I shift the "r-" one space left.

I've got the four digit display of the available number of registers working. It was not as hard as I thought. The code was already successfully getting a four-digit count, it just was only putting three digits into the display.

   

The HP microcode that computes the register count is iterative, for the number of registers. It takes the number of hardware 56-bit registers available, multiplies by 14 to get the number of 4-bit nibbles available, then repeatedly subtracts the number of nibbles in the word size. The number of subtractions before underflow is the user register count.

With the stock 203 bytes of RAM, there can't be more than 406 registers (with no program and WSIZE of 4 or less), and 406 iterations doesn't take too long. With 3318 registers, running at normal 16C speed, it takes almost ten seconds. I should probably rewrite it to do a long division.

[brouhaha]

I just wrote an integer long division subroutine for a patch to the HP-16C firmware. My subroutine is elegant and fast, beautiful to behold. It's perfect in ever way, with the one minor exception that it doesn't actually do what I needed.

Somehow as I was writing it, I forgot that while the inputs were binary integers, the output needs to be decimal (BCD). So I tried to fix that in a simple way, but when e.g. a subtract in digit position 2 occurs, I have to add 256 (BCD), not 100.

Back to the drawing board.

[brouhaha]

Woo-hoo! The new long division code is working! Now f MEM is essentially instantaneous in all cases. The original 16C code, even without memory expansion, had a noticeable delay for small word sizes unless a program was using up much of the RAM.

I know writing a routine like this would be trivial for some of the expert Nut microcode programmers that have been writing new modules for the 41C, but despite having written the first open-source microcode-level simulators for HP calculators, and spending thousands of hours studying HP's microcode from various models, I haven't actually written all that much of it. For me, this seems like an accomplishment.

In the original 16C code that did the division by repeated subtraction with no shift, the inner loop was six words. I replaced five of those six words with a gosub to the new patch, and three nops. The patch routine is 37 words.

The 16C ROM was really packed in tight, so two of the patches I've had to make only fit in an expanded ROM space. That's a total of 42 extra words so far. It shouldn't be a problem in simulators if they've reserved enough room for the 15C ROM (second R2D2 chip with ROM from 0x2000 to 0x37ff).

[brouhaha]

Indirect addressing was working to over 3300 in integer modes (HEX, DEC, OCT, BIN), but the code for FLOAT mode was limiting it to 99. I just got a patch working to allow up to 999, though with the current memory map, the maximum 56-bit register range is 0 to 237. Actually only to 236 at the moment due to the register address computation bug that I still need to fix.

Hopefully no one needs to store more than 237 floating-point values (or 56-bit integer values) in an HP-16C. :-)

So far I've patched 26 words in the base ROM, and added 57 words of expansion ROM.

Next on my TODO list for the 16C: track down the skipped nybble bug in the address computation. At the moment, I only know where that computation is done, but I don't at all understand how it does it.

This 16C hacking has been fairly fun. The previous attempt at 16C hacking, back in 2004 or 2005, was frustrating and nonproductive.

I did work back then on a hardware hack to do the ROM patching and RAM expansion for a real 15C. I need to dust that design off and bring it up to date, and try hacking the real machines (original Nut versions, not ARM).

But before that, I really should get back to 12C hacking, because while the 16C is of more use to me personally, the 12C improvements are probably useful to a wider audience, and I should be able to program them into a current production 12C (ARM-based).

In principle, I think I could do the memory expansion for the 11C fairly easily, but I don't think I'll bother, since the 15C is pretty much a superset of the 11C. Also, it wouldn't be hard to increase the available program steps of the 10C from 79 to 99, but I think that's even less worthwhile..

[EdS2]

This is great - following with interest!

[brouhaha]

The occasional skipped nibble problem in register addressing turns out to be quite a bit more complicated than I expected. The problem occurs in a section of code that takes a 12-bit binary nybble offset from the base of RAM, and divides it by 14 (because there are 14 nybbles in each hardware RAM register). The quotient is the hardware register RAM address offset (still relative to the base of RAM), and the remainder is the starting digit position within that hardware register. Since the divisor is a constant 14, this is done by multiplying by a fixed point approximation of the reciprocal of 14. There's some additional trickiness with efficiently getting a remainder of 0 to 13 out of the reciprocal multiplication, instead of a negative power of two, and I still don't really understand how they're doing that. It involves using 4688 as the approximation of 65536/14, rather than 4681, but there's more stuff going on after that which I haven't yet grasped.

The precision of the reciprocal was apparently chosen such that the division by 14 gets the correct quotient and remainder for input values from 0 to 405, as required by the HP-16C with 203 bytes of memory. In fact, ti works correctly up to 682. Unfortunately it winds up off by progressively larger amounts beyond that, gaining approximately a nibble for every increase of 682 of the input.

Because of the extreme cleverness of HP's code, there's no obvious way to extend it to handling a larger input range. It's not a matter of just adding a few more digits to the reciprocal, or more loop iterations. (They aren't even using a loop.) I think the best solution is just to replace the reciprocal multiplication by a more conventional long division routine. Luckily I just wrote one of those recently to solve a different problem with expanding the 16C memory. I can't use exactly the same routine, as the requirements are different, but I can use somewhat similar code. In fact, the new code is six words shorter than HP's ultra-clever code. It might take a few instructions longer for some input values. The HP code takes a fixed 39 cycles, which is not data dependent. My new code has data dependent timing, with a minimum of 30 cycles. I have not yet determined typical or worst case timings. Given how much other overhead there is in executing a RCL or STO instruction, the overall percentage slowdown will be very small.

Overall for this project, I expected to have to hack a whole lot of moderately tricky routines to make memory expansion work, but it turned out that a some things already worked, several things needed only trivial patches, the division for f MEM needed to be extended, and this latest find on the register addressing is the only thing that has been extremely tricky.

[brouhaha]

I just finished testing the new division routine for the register addressing, and it works fine for a moderately large number of test cases. I think now the hacked 16C code handles both program and register memory just fine.

The next thing I'm going to do is change it so that there's no conversion of data memory to program memory as the program grows. There will always be room for 999 program steps, and over 1750 bytes of data memory. This part will likely NOT be compatible with existing Nut simulation on some platforms, such as the ARM-based HP-12C and HP-15C LE, and the SwissMicros Voyager calculators, without hacking the ARM code.

Rather than hack HP's ARM code or SwissMicro's ARM code, I'm planning to write my own. Definitely for the ATSAM4LC2A used in the latest 12C. If that goes well, maybe also for the LPC1115 in the SwissMicros Voyagers.

I've got a few more features I'd like to add to the 16C Nut code. There aren't any vacant key locations, so it will be tricky to come up with ways for the user to access the additional features from the keyboard.

[Garry Lancaster]

I'm finding all this fascinating!

My most-used calculator is my DM16L, and 203 steps is nowhere near enough for all the programs I'd like to have loaded at once, so the idea of being able to get this expanded to 999 or more at some point is very appealing.

If you're looking for ideas for further functionality to add , one thing I long wished for was the availability of basic trig and exponential functions (some of the programs I normally have loaded are for this purpose, adapted from Valentin Albillo and Luiz C Viera's versions).

If the existing code from other Voyager models could be successfully adapted, it would seem like these could potentially be added as alternative f-Shifted functions on the top row and/or right column of the keyboard when in float mode, as all of the current functions in those positions are only active in integer modes.

[brouhaha]

(01-28-2023 10:38 AM)Garry Lancaster Wrote:  these could potentially be added as alternative f-Shifted functions on the top row and/or right column of the keyboard when in float mode, as all of the current functions in those positions are only active in integer modes.

That's a great idea!