Threaded Mode | Linear Mode

spiff72 · 09-16-2024, 03:27 PM

I am trying to figure out if this is a bug, or operator error...

If I have a program stored in a variable, and recall it to the stack, I find that the editor is acting strangely. As a simple example, take the following program for converting a number on the stack from mm to in:
<< 25.4 * >>

I wanted to include actual units in this, so I attempted to edit it by recalling it to the stack, pressing the right arrow button to enter edit mode, and then placed the cursor after the first <<. If I try to enter a 1 and then append units of mm to this by using the 1st shift UNIT menu and picking length, next page, mm ... instead of appending the _mm to the 1 I just entered, the cursor jumps to the end of the 25.4 and appends it there instead. I tried this in the sim initially, and then on my hardware DM42, and the same thing happens.

If I enter the program from scratch at the command line, I am able to enter the program as I intended.

I'm assuming that this could be done with some of the units functions, but my end program looked like this:
<< 1_mm * 25.4_mm/in / >>

Why does the cursor jump to the 25.4 in this scenario?

c3d · (This post was last modified: 09-16-2024 07:21 PM by c3d.)

(09-16-2024 03:27 PM)spiff72 Wrote: I am trying to figure out if this is a bug, or operator error...

If I have a program stored in a variable, and recall it to the stack, I find that the editor is acting strangely. As a simple example, take the following program for converting a number on the stack from mm to in:
<< 25.4 * >>

I wanted to include actual units in this, so I attempted to edit it by recalling it to the stack, pressing the right arrow button to enter edit mode, and then placed the cursor after the first <<. If I try to enter a 1 and then append units of mm to this by using the 1st shift UNIT menu and picking length, next page, mm ... instead of appending the _mm to the 1 I just entered, the cursor jumps to the end of the 25.4 and appends it there instead. I tried this in the sim initially, and then on my hardware DM42, and the same thing happens.

If I enter the program from scratch at the command line, I am able to enter the program as I intended.

I'm assuming that this could be done with some of the units functions, but my end program looked like this:
<< 1_mm * 25.4_mm/in / >>

Why does the cursor jump to the 25.4 in this scenario?

This is clearly a bug. The editor is trying to make sure the unit is added at the end of the number and not in the middle. In that scenario, that leads it to incorrectly jump to the next number. Because the computation was just slightly completely entirely wrong.

This is bug #1192, and it should be fixed on the dev branch.

spiff72 · 09-16-2024, 08:01 PM

(09-16-2024 07:20 PM)c3d Wrote:
(09-16-2024 03:27 PM)spiff72 Wrote: I am trying to figure out if this is a bug, or operator error...

If I have a program stored in a variable, and recall it to the stack, I find that the editor is acting strangely. As a simple example, take the following program for converting a number on the stack from mm to in:
<< 25.4 * >>

I wanted to include actual units in this, so I attempted to edit it by recalling it to the stack, pressing the right arrow button to enter edit mode, and then placed the cursor after the first <<. If I try to enter a 1 and then append units of mm to this by using the 1st shift UNIT menu and picking length, next page, mm ... instead of appending the _mm to the 1 I just entered, the cursor jumps to the end of the 25.4 and appends it there instead. I tried this in the sim initially, and then on my hardware DM42, and the same thing happens.

If I enter the program from scratch at the command line, I am able to enter the program as I intended.

I'm assuming that this could be done with some of the units functions, but my end program looked like this:
<< 1_mm * 25.4_mm/in / >>

Why does the cursor jump to the 25.4 in this scenario?

This is clearly a bug. The editor is trying to make sure the unit is added at the end of the number and not in the middle. In that scenario, that leads it to incorrectly jump to the next number. Because the computation was just slightly completely entirely wrong.

This is bug #1192, and it should be fixed on the dev branch.

Thanks!

M0R33z · 09-27-2024, 01:53 PM

Is it possible to program the DB48x in SystemRPL ? It just crossed my mind while I'm sitting here at work :-)

c3d · (This post was last modified: 09-27-2024 06:48 PM by c3d.)

(09-27-2024 01:53 PM)M0R33z Wrote: Is it possible to program the DB48x in SystemRPL ? It just crossed my mind while I'm sitting here at work :-)

I have some bad news followed by some good news.

The bad news: no. SystemRPL is a way to directly invoke entry points in the ROM of a given model. Since DB48X is a completely new implementation of RPL, there is no way to provide meaningfully compatible SystemRPL without having users download an HP ROM.

More annoyingly, going that route would have precluded many of the innovations that make DB48X interesting, such as ultra-dense object format, variable-precision decimals, true fractions, phasors (complex numbers in polar form), exact angle conversions, and many more. All these require a complete "rethinking" of low-level details such as the internal representation of RPL objects. Some of this is explained in my FOSDEM 2023 presentation.

Just to give an very simple example, SystemRPL makes relatively heavy use of "binary integer" values. #03FF9 will represent binary integer 1. This uses 2.5 bytes. A "normal" binary integer uses 5 bytes. The binary integer values are 20 bits, which matches the A (address, 5 nibbles) field of Saturn processor registers. All this is "nonsensical" on a 32-bit CPU like the ARM chips in the DM32/DM42. To leverage years of development, HP implemented a full Saturn emulator on their ARM chips just to be able to reuse code that was essentially 4-bit oriented. This is how you end up with 2.5 bytes for object sizes.

DB48X takes the approach that integers are first-class citizen, so while there are true binary integers, you can use them directly in computations. The encoding for 19 is (in current versions) 1F 13, it's only two bytes. You can use the BYTES command to see what the encoding looks like. This makes all the knowledge about SystemRPL names such as NINETEEN a bit irrelevant. You add these binary integers with normal +, not some special #+. This means that you pay the extra cost of type checking, but then this also means that 1 1+3i + gives you 2+3i, and that this is also coded using binary integers, 11 1F 02 1F 03. So now you have a complex number in 5 bytes, the same space that HP's SystemRPL would use to encode a regular binary integer.

In short, SystemRPL is much less necessary with DB48X.

This leads me to the good news.

The good news: SystemRPL was initially a way to code the HP ROM in a dense way. DB48X is fully open source, and written in quasi-normal C++. I say "quasi-normal" because it requires a bit of discipline to enable garbage collection and dense object representation in C++. In any case, if you need to write something "system level" that you would like to add to DB48X, you can. If you want that code to invoke any internal routine in DB48X, you can. If you want to code something in ARM assembly language, you can (I have not needed that so far, but it's definitely possible). If you want to have the compiler sweat a little to turn a C++ expression into an equivalent RPL program, you can. In short, you can do the kind of system-level programming that you could not even dream of on HP calculators.

SystemRPL as used by non-HP programmers (i.e. those not using it to actually write the ROMs) was also a way to leverage a vast catalog of preexisting small utilities. Here too, I have some good news: DB48X has a library system that, in the very next release, will become even more useful, by allowing you to have individual library entries in their own source file. Let me illustrate how that can replace SystemRPL.

Say that you need #4+PICK. The RPL code for that is "4 + PICK". First, let me remind you that the actual encoding of that code in today's DB48X is 1F043E34. So that's 4 bytes, which means that the saving compared to the 2.5 bytes is not that great. But let's imagine you really want that new word. Now, add this entry to your library.csv:

Code:

"SysRPL"

        "Add4Pick",     "« 4 + Pick »"

Now, your Library menu has a "SysRPL" submenu, which features a new "word" called Add4Pick. The exact encoding of that object will depend on your library, but on my machine today, it encodes as 0D 19. So now that's two bytes, a 20% reduction in memory usage relative to classical RPL.

Ah, but you might object that SystemRPL was also a way to cross-build for HP calculators using HP tools. By the way, you probably do not know that, but in the 1990s, I had created an alternative to the official HP tools called HPDS (I did not even know at the time that the HP tools existed). This is what I used for example to create my Lemmings game. End of the digression.

To that objection about cross-building, I would answer that you don't even need such cross-building tools with DB48X. A nice thing about the library approach I just mentioned is that it's "live": connect your DM32/DM42, edit the library.csv file, and you immediately update the programs in the library. If you write your programs using the emulator, you can have your favorite text editor, Emacs, on the side, edit your library objects, and test them in the simulator without even restarting it.

In the current release, what you could put in the library was limited, because you had to have small-enough objects to fit in a CSV file. That was a bit inconvenient. On the dev branch, and in the release I plan to release this week-end, this problem is fixed by letting you put the objects in their own file. You do that by having the definition of the object begin with =. If there is only an = sign in the definition, then the name of the library entry is also the name of the file in the library subdirectory.

I have converted the various demos that shipped with previous releases to this format. For example, the program illustrating the benefits of the tail recursion optimization in DB48X using the Collatz conjecture looks like this:

Code:

«

    @ --------------------------------------------------------------------

    @

    @ Collatz conjecture check - Check tail recursion in a test

    @

    @ --------------------------------------------------------------------

    @ Interesting values to start with:

    @ - Long stopping time: 1161, 2223, 2463, 2919, 3711, 6171

    @ - High trajectory value: 138367, 159487, 270271, 665215, 704511

    @ See https://en.wikipedia.org/wiki/Collatz_conjecture

    DUP "      " + 1 DISP MEM 2 DISP

    IF DUP 1 ≠ THEN

        IF DUP 2 MOD THEN

            3 * 1 +

        ELSE

            2 /

        END

        ⓁCollatzConjecture

    END

»

The library configuration file shows how this is made available to the user via the Library menu:

Code:

"Mathematics"

    "CollatzBenchmark",    "="

    "CollatzConjecture",    "="

        "CountPrimes",          "="

        "TriangleEquations",    "="

M0R33z · 09-28-2024, 05:19 AM

I have some bad news followed by some good news.

Many thanks for the excellent explanation. I think I have at least a rough understanding. I had only programmed in SystemRPL because of the speed. That's why it crossed my mind.

Many thanks for your efforts.

nickapos · (This post was last modified: 09-28-2024 06:21 AM by nickapos.)

Hi Christophe,
I have a question about the files under config folder. Do,you have any documentation or video anywhere on how we can use them to extend the available units, equations or constants.
The reason why I am asking is that I did not even know of their existence until recently when I had to go through the folder to share one of my state files and if I understand this correctly, it is a very elegant way of extending the functionality of the calculator.

Thank you very much

c3d · 09-28-2024, 10:08 AM

(09-28-2024 05:19 AM)M0R33z Wrote: I have some bad news followed by some good news.

Many thanks for the excellent explanation. I think I have at least a rough understanding. I had only programmed in SystemRPL because of the speed. That's why it crossed my mind.

Many thanks for your efforts.

I have tried to get good performance through other means, mostly carefully optimized code validated with various performance measurements.

This performance noticed by users. Our own @spiff72 wrote a post stating I report SPEED, which I appreciated very much.

Here are a couple of examples:

A DM42 today runs the NQueens benchmark in roughly 1.2s on USB power, 4.5s on battery. The HP50G runs in 2.3s using SYSRPL, and 22.6s in UserRPL, and the HP50G has a faster CPU.
Thanks to a technique optimizing tail recursion without consuming memory, DB48X can run a benchmark checking the Collatz conjecture on a large number in 15.3s, where the HP50G requires 397s in UserRPL. This is a benchmark you cannot run on earlier HP calculators because it requires support for arbitrary-precision integers.
On a floating-point function summation loop, the DM32 and DM42 are in the "100K loops" range, and run in 1768s using the default 24-digit precision. This is slower than the stock DM42 firmware at 261s, because the DB48x variable-precision support takes its chunk of meat here. On the other hand, if I select 16-digits precision and enable hardware acceleration, then I get 78s, and with 7 digits I get 59s. That puts me in range with a Numworks with Micropython or TI NSpire CXII. And then there is the iPhone app, which does 1M loops in 9s on iPhone 12.

Finally, an overrated part of "performance" is "how many keys do I need to do this or that". This is also an aspect of DB48X that I am really paying attention to.

c3d · 09-28-2024, 10:18 AM

(09-28-2024 06:20 AM)nickapos Wrote: Hi Christophe,
I have a question about the files under config folder. Do,you have any documentation or video anywhere on how we can use them to extend the available units, equations or constants.
The reason why I am asking is that I did not even know of their existence until recently when I had to go through the folder to share one of my state files and if I understand this correctly, it is a very elegant way of extending the functionality of the calculator.

Thank you very much

I do not have a specific place where those are documented. This is distributed throughout the documentation, including in the Quickstart Guide. As for videos, I know that it's shown in various demos, although there is no demo specifically on how to edit these files.

Up until recently, the configuration files contained the entire set of constants, units, equations or characters. This would give you an example of what the file should look like. I hope that this is sufficient as a tutorial.

Starting with v0.7.12, these files turned into extensions, with the bulk of the corresponding items being built-in. That was the original intent, putting everything in the files was only to stress test the file code, and impacts performance notably for unit operations, because file access is slower than memory access.

c3d · 09-28-2024, 10:35 AM

(09-28-2024 05:19 AM)M0R33z Wrote: I have some bad news followed by some good news.

Many thanks for the excellent explanation. I think I have at least a rough understanding. I had only programmed in SystemRPL because of the speed. That's why it crossed my mind.

Many thanks for your efforts.

About performance, I forgot to mention memory usage. In this thread, I compared DB48X to old HP calculators like the HP41. The bottom line is that, whereas HP's implementation of RPL was using much more memory than older calculators, DB48X is in the same ballpark as an HP41 for similar programs.

John Keith · 09-28-2024, 11:41 AM

(09-27-2024 06:39 PM)c3d Wrote: For example, the program illustrating the benefits of the tail recursion optimization in DB48X using the Collatz conjecture looks like this:

Code:

« @ -------------------------------------------------------------------- @ @ Collatz conjecture check - Check tail recursion in a test @ @ -------------------------------------------------------------------- @ Interesting values to start with: @ - Long stopping time: 1161, 2223, 2463, 2919, 3711, 6171 @ - High trajectory value: 138367, 159487, 270271, 665215, 704511 @ See https://en.wikipedia.org/wiki/Collatz_conjecture DUP " " + 1 DISP MEM 2 DISP IF DUP 1 ≠ THEN IF DUP 2 MOD THEN 3 * 1 + ELSE 2 / END ⓁCollatzConjecture END »

Tail recursion optimization is very interesting to me. Non-trivial recursive programs tend to be slow in RPL. One question: what is the circle-L Ⓛ symbol?

c3d · 09-28-2024, 12:27 PM

(09-28-2024 11:41 AM)John Keith Wrote:
(09-27-2024 06:39 PM)c3d Wrote: For example, the program illustrating the benefits of the tail recursion optimization in DB48X using the Collatz conjecture looks like this:

Code:

« @ -------------------------------------------------------------------- @ @ Collatz conjecture check - Check tail recursion in a test @ @ -------------------------------------------------------------------- @ Interesting values to start with: @ - Long stopping time: 1161, 2223, 2463, 2919, 3711, 6171 @ - High trajectory value: 138367, 159487, 270271, 665215, 704511 @ See https://en.wikipedia.org/wiki/Collatz_conjecture DUP " " + 1 DISP MEM 2 DISP IF DUP 1 ≠ THEN IF DUP 2 MOD THEN 3 * 1 + ELSE 2 / END ⓁCollatzConjecture END »

Tail recursion optimization is very interesting to me. Non-trivial recursive programs tend to be slow in RPL.

Part of the problem is that they accumulate return stacks, this is what the tail recursion optimization addresses. Another part is addressing speed for symbols, and that's where library items and local symbols are faster than regular globals.

Quote: One question: what is the circle-L Ⓛ symbol?

It indicates the name of a library entry (as opposed to a regular global). On the screen, it shows as a white-on-black [lib], but only while editing.

Internally, here is why this is interesting for performance.

When you have a global name like ABC, you have an object that takes 5 bytes (1 byte for the type ID, 1 byte for length, 3 bytes for ABC), and needs to be looked up on evaluation. So that is a little slow. I have considered various optimizations, but so far, nothing I could think of would reasonably preserve expected RPL semantics (like you can call another program that changes the content of that variable, etc).

When you have a local ABC name, as in « → ABC DEF « ABC DEF * DEF ^ » », it is really encoded as "LocalVariable 0", and DEF as "LocalVariable 1". Local variables are stored directly as pointers above the stack, so accessing local variable 0 is exactly as cheap as accessing a stack item. There is no lookup needed, so it's pretty fast. And each occurence of ABC in the inner program is only 2 bytes, one encoding the type ID and one encoding the index (0, 1, ...). So it's quite dense as well. If you compare to traditional RPN, we are pretty darn close to "RCL 25".

When you have a library entry, it's the same thing. While it shows on the command line as [lib]FooBar (with the Unicode Ⓛ being shown as [lib]), it's encoded internally as something like XLIB 27, where 27 is the index in the library. There are other prefixes like Ⓒ for constants and Ⓔ for equations, that follow the same principle. What shows up as "pi", for instance, is constant 3 on my system (the index depends on your constants.csv file), and is encoded as 0B 03.

In the current state of 'dev', a library item is looked up and loaded up from the file and parsed every time you use it, so it's really slow. But I'm working on having an optimization similar to what happens for local variables, so that you have an array of pointers for your various library items, so that when you use XLIB 12, it simply fetches xlibs[12-1], if not found loads and compiles it, and subsequent uses are super fast. I suspect that this week-end's release could have this optimization already, because it's not very complicated to do.

This means I could even give a DB48X meaning to ATTACH and DETACH. ATTACH would fetch and (re-)compile a particular submenu in your library (nlibrary would be the index of the submenu to pre-load from the library.csv file). DETACH would purge the pre-computed pointers so that you re-fetch them from disk on next use.

Anyway, it's not done yet, but that's the plan.

c3d · 09-29-2024, 08:59 PM

(09-16-2024 03:27 PM)spiff72 Wrote: I am trying to figure out if this is a bug, or operator error...

(...)

Why does the cursor jump to the 25.4 in this scenario?

This should be fixed in 0.8.0, which I just published. Thanks for reporting that.

c3d · (This post was last modified: 09-29-2024 09:08 PM by c3d.)

Release v0.8.0 of DB48X was just published.

This release adds symbolic integration, differentiation and equation solving, delivers an extensive equation library, and improves the library feature to enable large library objects to be defined in separate files and optimize execution speed for library items.
It also improves the rendering of complex equations and fixes a number of crashes or user-interface problems.

Special thanks to Jean Wilson for the major contributions to the equations library. Let me just quote his numbers (I did not check, but I assume it's accurate):

590 equations, which is 87% more than the 315 on the HP50g.
625 variables, which is 58% more than the 396 in the HP50g.

The quasi-completion of the equations library alone is sufficient to justify bumping up the version number to 0.8.0. Symbolic integration, differentiation and equation solving are a nice bonus.

To be clear, those features are now implemented, but not necessarily fully tested nor complete. We need hands to test the equations against the HP50g advanced reference manual and otherwise validate them. Symbolic integration patterns are there, but some useful rules are not implemented yet. ISOL is relatively limited compared to the HP50g version. But it's a good start.

M0R33z · 09-30-2024, 06:58 PM

@c3d

I am currently testing equations with collect and expand.
Example: 60*x+x*192*x

-> Simplify returns: 60*x+192*x*x
-> Expand and Collect returns the same result: 60*x+x*192*x

I would have expected Collect returns: 60*x+192*x^2
On the HP48GX I get exactly this result

grbrum · 09-30-2024, 09:01 PM

@c3d

when typing a variable name, or letter, or even in Symbolic mode "x", i see myself holding shift until ABC comes up, typing the letter and then holding shift again.
I wonder if the DM42 has a hardware limitation that does not allow 2 buttons to be pressed simultaneously.
would it be possible to hold shift, press X (or other letter), then release shift? this would make entry a lot smoother.

another possible idea, if in entry mode, one of the keys could represent X directly,
if I remember right, I think Erable used to do that (was it the DEL key? - I do not remember exactly)

thanks

c3d · 09-30-2024, 11:20 PM

(09-30-2024 06:58 PM)M0R33z Wrote: @c3d

I am currently testing equations with collect and expand.
Example: 60*x+x*192*x

-> Simplify returns: 60*x+192*x*x
-> Expand and Collect returns the same result: 60*x+x*192*x

I would have expected Collect returns: 60*x+192*x^2
On the HP48GX I get exactly this result

The way expand and collect work today is not satisfactory at all. I have implemented a part of what is needed to improve it (polynomial objects), but there is quite a bit of additional work that remains to be done, notably implementing polynomial factorization correctly.

ISOL, differentiation and integration are similarly limited compared to its HP counterpart. There is no integration by parts, no resolution of general polynomials, etc.

c3d · 09-30-2024, 11:25 PM

(09-30-2024 09:01 PM)grbrum Wrote: @c3d

when typing a variable name, or letter, or even in Symbolic mode "x", i see myself holding shift until ABC comes up, typing the letter and then holding shift again.
I wonder if the DM42 has a hardware limitation that does not allow 2 buttons to be pressed simultaneously.
would it be possible to hold shift, press X (or other letter), then release shift? this would make entry a lot smoother.

This already exists, but with the arrow keys that lets you select either uppercase or lowercase. It's called transient alpha mode.

Quote:another possible idea, if in entry mode, one of the keys could represent X directly,
if I remember right, I think Erable used to do that (was it the DEL key? - I do not remember exactly)

thanks

Not enough keys for that unfortunately (remember that we have one entire rows of keys missing compared to the HP48). I don't see what key I would be willing to sacrifice to a dedicated X key. When I implement user-defined key assignments, you will be able to do that, though.

M0R33z · 10-01-2024, 03:53 AM

(09-30-2024 11:20 PM)c3d Wrote:
(09-30-2024 06:58 PM)M0R33z Wrote: @c3d

I am currently testing equations with collect and expand.
Example: 60*x+x*192*x

-> Simplify returns: 60*x+192*x*x
-> Expand and Collect returns the same result: 60*x+x*192*x

I would have expected Collect returns: 60*x+192*x^2
On the HP48GX I get exactly this result

The way expand and collect work today is not satisfactory at all. I have implemented a part of what is needed to improve it (polynomial objects), but there is quite a bit of additional work that remains to be done, notably implementing polynomial factorization correctly.

ISOL, differentiation and integration are similarly limited compared to its HP counterpart. There is no integration by parts, no resolution of general polynomials, etc.

Thank you very much for the clarification.

tdh79 · 10-01-2024, 10:49 AM

(07-10-2023 12:07 AM)Helix Wrote:
(07-08-2023 11:00 PM)c3d Wrote: I gave a talk about this project at FOSDEM this year. You can see what it looks like in that video, as well as the design philosophy behind it.

As I find RPL superior to RPN, this project could justify the purchase of a DM42.
And I like very much the interface!

I have some basic questions:
- how to quickly access the variables menu? I don’t remember seeing it in the video.
- will comments be possible inside programs, apart from the inelegant "text" DROP ?
- will list processing be available, like in the HP48 G/GX or the HP50G?
- I know it’s very premature, but if variable floating-point precision is implemented some day, will it replace the fixed floating-point precision library? I don’t like the idea of scarifying speed to gain useless precision.

RPL different from RPN? I thought one was a programming language and the other a method for inputting data?