[34S] Proposal for Entry RPN mode with dynamic stack

[matthiaspaul]

Based on discussions with friends and my own experience as (mostly) RPL user, I'd like to suggest two possible new features for the WP 34S (or derivatives and successors) to make it more attractive to a broader user base.

I understand that traditional users of HP RPN calculators appreciate and sometimes depend on the special mechanics of the 4-level automatic stack (conditional duplication of X value into Y on ENTER, and duplication of T value into Z when dropping a stack level).
Nevertheless, Hewlett-Packard abandoned these features when they introduced RPL with its "unlimited stack" decades ago, and they even dropped these features in all recent RPN calculators (except for models which emulate traditional RPN calculators). Also, many third-party RPN solutions do not implement them as well, so there are quite many RPL/RPN users, who are puzzled when they find out, that, on the WP34S (which implements "Classical RPN"),

3 ENTER 4 ENTER +

does not produce the same result (8 versus 7) as

3 ENTER 4 +

I would therefore like to propose to add an "Entry RPN" mode to the WP 34S, which should be active, when the previously unused flag 50 is set. If flag 50 is cleared, the calculator should continue to work exactly as before in "Classical RPN" mode. We could also add an "RPNMODE" command with arguments 1 ("Classical RPN") and 2 ("Entry RPN") to ease mode switches. This should be very easy to implement and at almost no costs in regard to code size.

While the special behaviour of the ENTER key can cause confusion and wrong results if used inappropriately, and it therefore should become configurable by this new command, I don't think the duplication of the T register into Z can cause any harm, so, there's no need to disable this as well (at least not in this "minimal" implementation).

In addition to this, there's another feature which could be bound to the same flag 50 as well: Dynamic stack.

While the limited memory capabilities of the WP 34S hardware do not allow for a really "unlimited" stack, I think, the idea of having a larger stack does not face resistance even among traditional RPN users who have learnt to master almost all problems with a 4-level stack only - at least, if it could be had "for free" and does not get into their way. The WP 34S even implements an optional 8-level stack, however, even this might be unsufficient in some situations. (For example, I would love to use the WP 34S as a "scratchpad" for bit manipulation and optimization tasks in integer mode, which requires that a "stream" of parallel bit data would have to be kept on the stack.)

My idea of a dynamic stack is to let the 4- or 8-level base stack grow into currently unused R registers from top to bottom, if two conditions are met: a special dynamic stack mode is enabled, and the register to be occupied for the stack does not hold a non-zero value. All we'd need for this to work would be a pointer indicating the current "top of stack" inside the register memory area. It does not conflict with memory allocated for programs or dynamically allocated memory for statistical data.
This dynamic stack allocation would continue to work for as long as we don't run out of registers (that is reach R00 -- letter-named registers will always remain available), or if the register to be occupied would hold a value other than 0, indicating that it is in use already and we mustn't loose its contents. In this case, the dynamic stack cannot grow any larger for as long as this register value isn't changed to 0 first.
Instead, the earliest stored value inside the dynamic stack area would be dropped, thereby making room for the new value to be stored on the other end of the dynamic stack. A second circular pointer inside this dynamically allocated stack area would be used to indirectly address the last stored value. When the size of the buffer changes later on, the buffer would have to be normalized again. This would happen, when the "blocking" register holding a non-zero value would be changed to 0 and a new value be stored on the stack, thereby growing the dynamic stack's size by one. It would also happen as soon as the dynamic stack shrinks. This double-pointer-indirection would reduce necessary memory copying to a minimum.
When the stack shrinks, the contents of the last stored value in the dynamic stack would be dropped back into the T register, the register set to 0 and deallocated from stack use by (normalizing the buffer and) adjusting the stack pointer accordingly.
I'm undecided if the traditional T register duplication should happen again as soon as the dynamic stack is fully deallocated, of if it would be better to insert 0 then. I'm tending towards leaving it as it is, so that the dynamic stack is transparent to users of "Classical RPN". Mind, that, in contrast to the unlimited stack in RPL, in this dynamic RPN stack implementation, a minimum of 4 or 8 stack levels would always remain available for compatibility.

Once a register is dynamically allocated to hold stack rather than register data, it no longer exists as actual R register. Assuming, the dynamic stack would have allocated R99 above, the next stack-up operation(s) would attempt to dynamically allocate R98, R97, R96, respectively, and consequently those registers would no longer be available. However, recalling these (no longer existing) registers would still be possible, because we know that their value was 0 before it was allocated as dynamic stack. When attempting to store values in these no longer existing registers, the dynamic stack would automatically make room and shrink downto the size necessary to make the memory available as register again. If the dynamic stack content is more important than the register values, the user can always use lower registers.

In order to address the elements on the stack, we could extend the addressing methods for [c]STO, [c]RCL and [c]x<> by a "R/S 1 .. 100" stack-relative addressing scheme. 1 would be stack level 1, that is X. In 4-level stack mode, RCL R/S 5 would recall the first value on the dynamically allocated stack (beyond T), RCL R/S 6 the second value, and so on. Addressing unavailable stack levels would result in an error.

Regarding stack manipulation commands, I think, they should continue to work as before, except for [c]R^ and [c]Rv, which either should work on the whole dynamically stack, or better new [c]ROLL^ and [c]ROLLv commands would have to be added in order to provide the facility to work on the whole stack instead of the base stack ([c]R^/v aka ROT) only.

Of course, there are race conditions and side effects:

A user (or program) could attempt to store a value in a register, which is temporarily allocated for stack use. There are no restrictions in doing so except for that storing values in such non-existing registers will shrink the dynamic stack and thereby loose old stack data.
From the stack's perspective, a specific stack size (above the 4- or 8-level base stack) cannot be guaranteed under all conditions. If the stack cannot grow any larger, the contents at the top of the stack will get lost - as it would have happened much earlier already, if the stack couldn't grow at all. This isn't a problem in itself, but user cooperation is required if s/he or a program depends on a particular stack depth (beyond the base stack).
These problems could be worked around either by increasing the memory allocated for registers via REGS (if there is free program memory available) or at least remedied by making it a rule to use up registers from the bottom to the top (instead of using f.e. R90 before R10).

If these features or effects are undesired, it is always possible to switch back to RPNMODE 1 for "Classical RPN" with old ENTER key behaviour and no dynamic stack.

Comments?

Greetings,

Matthias

[Bit]

(02-15-2015 10:28 PM)matthiaspaul Wrote:  In addition to this, there's another feature which could be bound to the same flag 50 as well: Dynamic stack.

Which "flag 50" are you referring to? Flag 50 in UState controls Y register display and is free only if Y register display is either completely disabled or permanently enabled.

(02-15-2015 10:28 PM)matthiaspaul Wrote:  If these features or effects are undesired, it is always possible to switch back to RPNMODE 1 for "Classical RPN" with old ENTER key behaviour and no dynamic stack.

Both features could be useful IMO. However, why not make them independently configurable? Someone who's used to the RPN ENTER might want to use an 'infinite' stack.

[Bit]

What should happen if a program was written in one mode and then started in the other?

Without some extra rules, some programs would work correctly, others would be obviously broken, yet others would be broken in ways that may be very hard to notice. The user could be made responsible for dealing with the first two cases... but probably not with the third.

[matthiaspaul]

(02-15-2015 10:52 PM)Bit Wrote:  

(02-15-2015 10:28 PM)matthiaspaul Wrote:  In addition to this, there's another feature which could be bound to the same flag 50 as well: Dynamic stack.

Which "flag 50" are you referring to? Flag 50 in UState controls Y register display and is free only if Y register display is either completely disabled or permanently enabled.

Okay, so it has been occupied recently, unfortunately (but for a good purpose!

My choice of flag 50 was based on revision 3.1 of the manual as newer versions are only available as printed manual, AFAIK.

Quote:

(02-15-2015 10:28 PM)matthiaspaul Wrote:  If these features or effects are undesired, it is always possible to switch back to RPNMODE 1 for "Classical RPN" with old ENTER key behaviour and no dynamic stack.

Both features could be useful IMO. However, why not make them independently configurable? Someone who's used to the RPN ENTER might want to use an 'infinite' stack.

Sure, but I was living under the impression that we are limited to 64 flags for some reasons. If we can extend UState beyond that, things would certainly become easier (also in other areas).

Greetings,

Matthias

[Thomas Klemm]

(02-15-2015 10:28 PM)matthiaspaul Wrote:  so there are quite many RPL/RPN users, who are puzzled when they find out that, on the WP34S (which implements "Classical RPN"),

3 ENTER 4 ENTER +

does not produce the same result (8 versus 7) as

3 ENTER 4 +

Are there seriously RPL-users that hit that 2nd ENTER key though it isn't needed?
Why would you do that?

Cheers
Thomas

[Bit]

(02-15-2015 11:07 PM)matthiaspaul Wrote:  

(02-15-2015 10:52 PM)Bit Wrote:  Which "flag 50" are you referring to? Flag 50 in UState controls Y register display and is free only if Y register display is either completely disabled or permanently enabled.

Okay, so it has been occupied recently, unfortunately (but for a good purpose!

My choice of flag 50 was based on revision 3.1 of the manual as newer versions are only available as printed manual, AFAIK.

It's best to check the source code, such things might even have changed since version 3.3 was released.

It's unfortunate that newer versions of the manual aren't available as PDFs, and preferably in the original editable format as well. That's pretty much the only component of this project that isn't open-source. Walter would know the reason, perhaps there's some contractual obligation with the publisher.

I have the spiral bound printed manual and while it's nice and well made, most of the time I use the PDF because it's just so much faster to navigate: you can search and click on links.

(02-15-2015 11:07 PM)matthiaspaul Wrote:  

(02-15-2015 10:52 PM)Bit Wrote:  Both features could be useful IMO. However, why not make them independently configurable? Someone who's used to the RPN ENTER might want to use an 'infinite' stack.

Sure, but I was living under the impression that we are limited to 64 flags for some reasons. If we can extend UState beyond that, things would certainly become easier (also in other areas).

As far as I can see, you'd need to sacrifice something in order to increase the number of internal flags. Some of the user flags would be an obvious choice as there are way more of them than most people would ever need.

[Paul Dale]

(02-15-2015 11:38 PM)Bit Wrote:  

(02-15-2015 11:07 PM)matthiaspaul Wrote:  Sure, but I was living under the impression that we are limited to 64 flags for some reasons. If we can extend UState beyond that, things would certainly become easier (also in other areas).

As far as I can see, you'd need to sacrifice something in order to increase the number of internal flags. Some of the user flags would be an obvious choice as there are way more of them than most people would ever need.

Yes, we're limited to 64 bits of system flags -- this is the size of a long long integer. Extending this will steal memory from something else. Pretty much everything is a trade off at this point.

We've deliberately avoided using (the numbered) user flags for system things. They are the user's and not ours.


A dynamic stack could exist in in the program/stack space with some shuffling instead of consuming registers directly. A user can always increase the free space here by reducing the number of registers.


Personally, I'd like to lose the weird stack lift behaviour in favour of something more consistent...


Pauli

[MarkHaysHarris777]

(02-15-2015 10:28 PM)matthiaspaul Wrote:  Based on discussions with friends and my own experience as (mostly) RPL user, I'd like to suggest two possible new features for the WP 34S (or derivatives and successors) to make it more attractive to a broader user base.
. . .
Matthias

its not a bad idea, Matthias. (I'm not advocating for *any* change, nor do I have an agenda) I would simplify the request, to allowing *other* modes of operation (forget the 50 flag). But, if I understand the state of affairs, the firmware is maxed out. Anyway these are the modes I would like to see- sometime.

1) Classic RPN; just as it is now
2) RPN, with input buffer; x does NOT copy into y on ENTER; auto T drop disabled
3) RPL, unlimited stack; as you suggest using regs 00-(?)

I also would like to see a constant function that would disable auto drop... y holds the constant 'value' rather than T (but that's me, classic RPN works fine)

Matthias, I think your idea is well thought-out. Thanks for the read.

-------------------------------------------------------------------------------
Sometimes I square a number by hitting [x] after ENTER... sometimes I double a value by hitting [+] after ENTER. There are other tricks too; well documented. These are not missed on new generation users who are comfortable with RPL (if Hewlett had thought of RPL, and if he had enough memory in the first place, we wouldn't be having this talk). The idea is maybe more of an 'on the drawing board' idea for 43s?

PS These are the RPN 'style' configuration options for RealCalc on Droid:
1) Unlimited stack, no buffer
2) Unlimited stack, entry buffer
3) Four-level XYZT stack




cheers,
marcus

[Bit]

(02-16-2015 01:00 AM)Paul Dale Wrote:  We've deliberately avoided using (the numbered) user flags for system things. They are the user's and not ours.

That's a policy I wholeheartedly agree with. But we could reduce the number of flags visible or accessible to users.

[matthiaspaul]

(02-15-2015 11:01 PM)Bit Wrote:  

(02-15-2015 10:28 PM)matthiaspaul Wrote:  

What should happen if a program was written in one mode and then started in the other?

The user would have to take care of this, possibly by enforcing the correct runtime environment in the init portion of the program. A program may generally depend on quite a number of mode settings (f.e. number of available registers, stack-size 4 vs. 8, float vs. integer, etc.), so this would be no exception.

Quote:Without some extra rules, some programs would work correctly, others would be obviously broken, yet others would be broken in ways that may be very hard to notice. The user could be made responsible for dealing with the first two cases... but probably not with the third.

A valid point, but for as long as a program does not use stack-wise relative addressing of higher than base-level stack levels (via the proposed R/S addressing extension), register usage should be completely transparent for programs within the limits of what was allocated via REGS, originally. Recalling a value from an allocated R- but meanwhile re-purposed stack register will return 0 (the original value it had before being re-purposed). Attempting to store a value in such a register will make the R register available again immediately and shrink the dynamic stack accordingly. (The alternative solution would be to throw a hard error to make the user aware of the condition, but it might also confuse people, when an originally allocated register would happen to be no longer available out of a sudden. It would also be possible to subdivide the register area into two static areas, one for actual registers and the other for the stack, however, it would give up some flexibility and might be more cumbersome to set up for the user.)
So, for as long as a program does not depend on a particular stack depth beyond the 4- or 8-level base stack (which, in contrast to RPL, is always guaranteed to exist in this dynamic RPN stack model), the program shouldn't notice a difference, and programs depending on it should know of the preconditions to be met. If higher-numbered registers are not used, it can be assumed that the stack can grow into them without problems (this is basically the same as sub-dividing this area into registers and stack statically, just easier to set up and more flexible). So, in many (if not most) cases, the user can take care of the necessary conditions. Still, you are making a valid point, since the stack depth needed to solve a problem may not always be known beforehand.

Well, the stack depth available isn't known in RPL as well, the difference is that it can be assumed to be "deep enough" in virtually all cases, and also much larger than the possible 0..100 extra levels in this proposed dynamic RPN stack mode -- and if not, you'd get an error message instead of stack contents being silently dropped.

If not a hard error message, at least some visual clue might be desirable to indicate a soft stack overflow. Such states could also be made software-detectable by some means.

Greetings,

Matthias

[matthiaspaul]

(02-16-2015 01:00 AM)Paul Dale Wrote:  

(02-15-2015 11:38 PM)Bit Wrote:  As far as I can see, you'd need to sacrifice something in order to increase the number of internal flags. Some of the user flags would be an obvious choice as there are way more of them than most people would ever need.

Yes, we're limited to 64 bits of system flags -- this is the size of a long long integer. Extending this will steal memory from something else. Pretty much everything is a trade off at this point.

We've deliberately avoided using (the numbered) user flags for system things. They are the user's and not ours.

Having a look at the UState flags (still based on 3.1), the LCD contrast setting and the FAST mode flag were possible candidates to be repurposed. While I can see some limited uses for them, this is hardware-related information which might be possible to store elsewhere. Ideally, the system's power management and thereby a need to switch between different speeds, if necessary, should be completely transparent for a user. (From a user's perspective, it should always run at maximum speed, while from the hardware's point of view, this may not always be possible whilst maintaining system integrity with an already drained battery. I have some ideas how to possibly improve at least some aspects of this as well, but it would be off-topic in this thread -- later.)

Quote:Personally, I'd like to lose the weird stack lift behaviour in favour of something more consistent...

I'm undecided on this. Being mostly an RPL user, I don't personally depend on it. It even has ruined a few calculations before I learned that the WP 34S implements it the classical way, and I still sometimes happen to hit the ENTER key too often switching between calculators.
However, I also understand that this particular behaviour is part of the "magic" of HP's way of RPN to more traditional users, and I would not want to take that away.
So, ideally, to make this calculator as user-friendly as possible to any type of RPN/RPL user, new and old, a RPN mode switch seems like a good solution for me, in particular since at least this part of my proposal could be had at almost no costs code-wise.

Greetings,

Matthias

[Paul Dale]

(02-16-2015 01:58 AM)matthiaspaul Wrote:  Having a look at the UState flags (still based on 3.1), the LCD contrast setting and the FAST mode flag were possible candidates to be repurposed. While I can see some limited uses for them...

The more important question is do you want these saved and restored using the STOM/RCLM commands? We're limited to 64 bits here since that is how long our registers are. Adding new status bits is possible but something will have to go.

Personally, FAST is one I'd want to keep.

The LCD contrast, on the other hand, isn't something likely to be altered by a program and need restoring. In fact I don't think programs can change this currently.

So that means moving these four bits elsewhere -- what do we lose by doing so?


I suspect this is all moot unless someone volunteers to code the proposal


- Pauli

[matthiaspaul]

(02-16-2015 01:02 AM)MarkHaysHarris777 Wrote:  2) RPN, with input buffer; x does NOT copy into y on ENTER; auto T drop disabled

Well, not completely. ENTER ENTER should still duplicate into Y (as it does in RPL).

Quote:Sometimes I square a number by hitting [x] after ENTER... sometimes I double a value by hitting [+] after ENTER.

This sort-of still works in RPL (and Entry RPN), but you'd have to press ENTER twice in succession.

Greetings,

Matthias

[matthiaspaul]

(02-16-2015 02:51 AM)Paul Dale Wrote:  

(02-16-2015 01:58 AM)matthiaspaul Wrote:  Having a look at the UState flags (still based on 3.1), the LCD contrast setting and the FAST mode flag were possible candidates to be repurposed. While I can see some limited uses for them...

The more important question is do you want these saved and restored using the STOM/RCLM commands?

Given that this mode has possible side effects on program execution as Bit pointed out correctly, it would be highly desirable to make it part of UState, so that it can be easily retrieved, changed, saved and restored in programs.

Quote:Personally, FAST is one I'd want to keep.

Yes, for as long as it may be necessary to monitor and change the speed (f.e. when printing with batteries low), this is important information. It may also be useful when optimizing programs for speed.

Quote:The LCD contrast, on the other hand, isn't something likely to be altered by a program and need restoring. In fact I don't think programs can change this currently.

So that means moving these four bits elsewhere -- what do we lose by doing so?

If it would be important information for a user or program, we could always implement dedicated commands like LCD and LCD? to set and retrieve the current setting.

Quote:I suspect this is all moot unless someone volunteers to code the proposal

Well, I have a clear mind model of how to possibly implement it (based on the abstract description of the calculator's memory organization in the 3.1 manual), but I am not yet familiar with the current implementation on any detail level (I only had a few quick looks at the sources so far). Setting up a development environment for this is on my task list, as I have many other ideas for improvement as well (subject to other threads as time allows). However, although I am excited about this project, right now I am still swamped with other tasks (and this may continue to hold true for some while). So, for me, it could be a mid-term thing only. If someone would have fun and time to go for it earlier, ... ;-)

Greetings,

Matthias

[Nigel (UK)]

(02-16-2015 01:18 AM)Bit Wrote:  

(02-16-2015 01:00 AM)Paul Dale Wrote:  We've deliberately avoided using (the numbered) user flags for system things. They are the user's and not ours.

That's a policy I wholeheartedly agree with. But we could reduce the number of flags visible or accessible to users.

Along with Jeff O I'm currently working on a "Complex Lock" mode for the WP34S (nearly finished now!). The flags I've used for this have been high-numbered user flags. To stop a user changing these flags unintentionally I've written code to throw an error if write access to these flags is attempted.

Is this a useful approach? Could this ever be part of the standard build?

Nigel (UK)

[Paul Dale]

(02-16-2015 09:13 AM)Nigel (UK) Wrote:  Is this a useful approach? Could this ever be part of the standard build?

Without seeing the end result, I can't tell.

We do have a lot of conditional options in features.h which have been contributed. So including your code conditionally is distinctly possible.

Enabling it as part of the standard build generally requires a higher standard -- the UI has to make sense and be consistent and it has to address a shortcoming or provide a generally useful feature. Walter usually has the last word on new features.


- Pauli

[Bit]

(02-16-2015 02:56 AM)matthiaspaul Wrote:  

(02-16-2015 01:02 AM)MarkHaysHarris777 Wrote:  2) RPN, with input buffer; x does NOT copy into y on ENTER; auto T drop disabled

Well, not completely. ENTER ENTER should still duplicate into Y (as it does in RPL).

Quote:Sometimes I square a number by hitting [x] after ENTER... sometimes I double a value by hitting [+] after ENTER.

This sort-of still works in RPL (and Entry RPN), but you'd have to press ENTER twice in succession.

The behavior of ENTER is another thing that could break programs. Programs would need to be able to save, change and restore that setting, too.

By the way, what would be the advantage of this on a calculator like the 34S? Is it simply that some people are used to it and why not make it easier for them (not a bad goal IMO), or do you think it's a superior UI? If it's the latter, then could you please explain why?

[Thomas Radtke]

(02-15-2015 10:28 PM)matthiaspaul Wrote:  Nevertheless, Hewlett-Packard abandoned these features when they introduced RPL with its "unlimited stack" decades ago, and they even dropped these features in all recent RPN calculators (except for models which emulate traditional RPN calculators).

The 35s still comes without entry line different from x.

The different initial RPN implementations probably just reflected that RAM was expensive. I guess it paid to blow up the ROM code instead to introduce all sorts of tricks to make this work for the user (while being inconsistent over different models).

The first model with a fixed stack and an entry line was the 20b. This was the end of an era, and it came not too soon. Thanks, HP.

[matthiaspaul]

(02-16-2015 05:18 PM)Bit Wrote:  The behavior of ENTER is another thing that could break programs. Programs would need to be able to save, change and restore that setting, too.

Yes. So, depending on if this would be coupled to the same flag as the dynamic stack, we'd need one or two UState flags.

I'd prefer the two flags version, but we should be conservative in allocating flags, as we might need a few more flags for other features in the future. In some cases, "unfolding" state information into separate flags (like Nigel's and your's significant numbers feature) might help to decrease the size of the code necessary to evaluate such flags. I don't propose to change this now, but we might need to keep that in mind as an area for possible future optimization.

Quote:By the way, what would be the advantage of this on a calculator like the 34S? Is it simply that some people are used to it and why not make it easier for them (not a bad goal IMO), or do you think it's a superior UI? If it's the latter, then could you please explain why?

I guess, it is a mix of all of this. :-) Making it more easy and reliably to use (without retraining) to a wider audience is certainly one of the goals. Desired flexibility not only as a sign of excellence in a product, but also to have more people stress-testing the implementation and possibly contributing to this and follow-up projects.
Both methods have advantages and disadvantages, therefore it is difficult to declare one as better as the other. Classical RPN is slightly more keystroke efficient and powerful in a 4-level stack model, but it needs to be explained and mastered, whereas Entry RPN is slightly more intuitive and straight-forward and therefore easier to learn for new users who may expect an ENTER key to behave the same way as on other devices without surprises and implicit sideeffects. Also, Entry RPN resembles RPL in this respect, which makes it more reliably to use for users frequently switching between RPL and RPN calculators.
Thinking about it, I guess, the reason, why Entry RPN is more intuitive, is also down to how the postfix concept is taught in school or university (if at all). You first enter the arguments, then the operation. I have seen people failing to understand that you must press ENTER to input the first number, but, at least in Classical RPN, are not allowed to press ENTER to input the second number as well. It sounds arbitrary and illogical at first, until the reasons for why it was implemented this way are explained as well. It lacks the "symmetry", and recognizing inherited symmetries often makes it easier to learn something new (it's a bit about successful communication consisting of new information as well as redundancy).

Greetings,

Matthias

[Bit]

Matthias, if it's not too much effort, could you point me to a detailed, strictly precise description of exactly how the 'Entry RPN' mode should work including all corner cases? Something that'd pass as a formal definition?

I'm just curious about what it'd really take to implement it. Perhaps I'm being overly paranoid and this particular feature is actually quite simple, but it has been my experience that a reasonable explanation that covers 99.99% of cases and is basically good enough in practice, versus getting it 100.00% right, are often two very different things.