DB48X: HP48-like RPL implementation for DM42

[c3d]

(07-15-2024 12:29 PM)spiff72 Wrote:  I'm curious about something regarding these .csv files:

Does the calc actually read from the FAT disk during operation, or does it access them once (at startup) to read them in, and then close the FAT disk?

Yes, it reads the file every time. Loading them in memory would take too much space, at least on the DM42. Plus quick experiments showed that reading from disk or reading data from read-only memory happens at roughly the same speed (both are from flash memory anyway).

[c3d]

(07-18-2024 04:03 AM)spiff72 Wrote:  I just wanted to chime in (for once) without reporting an issue...

Instead, I am reporting SPEED!

I recently heard of the R47 calculator (variant of the C47 with two shift keys), and made an overlay and keylabel set for it. I thought I would give it a try on my DM42 before applying any overlays or labels, and I was immediately struck by the sluggishness of the calculator (on battery power). Just repeatedly pressing the square root of a number over and over has a very noticeable delay/lag. If you do it fast enough and often enough, you can stop and then wait several seconds for the calculator to catch up.

I still remember commenting to c3d a while back when I felt that DB48x was feeling sluggish to me (when dividing 2 numbers with a decimal point (and lots of digits)), and in the next release there was a significant and noticeable difference in speed. (If I recall correctly it was a screen rendering or graphical adjustment.)

There's no way for me to go back to the other projects right now (C47) - they just "feel" too slow to me after using DB48x for many months (maybe a year?). I feel like only the stock DM42 firmware is faster from my experience.

Thanks a lot, I appreciate the compliment. However, I'm curious about the DM42 firmware being faster. Are there specific operations where this is particularly noticeable and where I should spend time optimizing if I can?

[spiff72]

(07-18-2024 08:56 AM)c3d Wrote:  Thanks a lot, I appreciate the compliment. However, I'm curious about the DM42 firmware being faster. Are there specific operations where this is particularly noticeable and where I should spend time optimizing if I can?

You're welcome! Honestly, I was shooting from the hip with that statement about the DM42 firmware being faster. I haven't used it in a long time, but it felt like program editing was much faster there than on the C47 platform. I haven't done any heavy programming with DB48x that needed navigation around big programs.

I really don't have any complaints regarding speed on DB48x. I am curious about the DM32 hardware, though, and how that performs. My biggest complaint about the DM42 hardware is the very limited FAT filesystem size, but I don't think the DM32 improves on that at all.

[rprosperi]

(07-18-2024 01:01 PM)spiff72 Wrote:  

(07-18-2024 08:56 AM)c3d Wrote:  Thanks a lot, I appreciate the compliment. However, I'm curious about the DM42 firmware being faster. Are there specific operations where this is particularly noticeable and where I should spend time optimizing if I can?

You're welcome! Honestly, I was shooting from the hip with that statement about the DM42 firmware being faster. I haven't used it in a long time, but it felt like program editing was much faster there than on the C47 platform. I haven't done any heavy programming with DB48x that needed navigation around big programs.

I really don't have any complaints regarding speed on DB48x. I am curious about the DM32 hardware, though, and how that performs. My biggest complaint about the DM42 hardware is the very limited FAT filesystem size, but I don't think the DM32 improves on that at all.

The C47 project is still in active development and the code has not been optimized and also contains 'extra' stuff to accomodate debugging. Once the functionality is complete, the team is planning to remove the extra bloat and then fine-tune for better performance. Speed increase factors from 2-3 and most likelu more are expected.

You're right, the DM32 FAT is also 6MB, a limit of the hardware used.

[spiff72]

(07-18-2024 04:23 PM)rprosperi Wrote:  The C47 project is still in active development and the code has not been optimized and also contains 'extra' stuff to accomodate debugging. Once the functionality is complete, the team is planning to remove the extra bloat and then fine-tune for better performance. Speed increase factors from 2-3 and most likelu more are expected.

You're right, the DM32 FAT is also 6MB, a limit of the hardware used.

I am excited to hear that - I look forward to trying it again once the performance is improved! Until that point, the sluggishness is a deal-breaker for me. The R47 option adds an interesting option with the two shift keys that might pry me away from DB48x on a trial basis once the speed improves.

[spiff72]

Interactive Stack question...

Maybe this is expected behavior, but if I want to edit the first level of the stack (let's say I had an extraneous digit on the first level and I want to go back and remove that digit), I hit the left arrow key, then "Edit" from the function menu (F1). From there, I was expecting I could just backspace (the "Drop" key above the division symbol) the extra digit out and hit enter to exit the interactive mode.

Instead, when I do this, the drop key does it's "normal" function of dropping the bottom level on the stack.

How would I actually perform the edits after pressing the edit function key in interactive stack mode?

[c3d]

(07-18-2024 06:29 PM)spiff72 Wrote:  Interactive Stack question...

Maybe this is expected behavior, but if I want to edit the first level of the stack (let's say I had an extraneous digit on the first level and I want to go back and remove that digit), I hit the left arrow key, then "Edit" from the function menu (F1). From there, I was expecting I could just backspace (the "Drop" key above the division symbol) the extra digit out and hit enter to exit the interactive mode.

Instead, when I do this, the drop key does it's "normal" function of dropping the bottom level on the stack.

How would I actually perform the edits after pressing the edit function key in interactive stack mode?

I think that I messed up the Edit feature by trying to mix what the HPs call Edit and Echo. I need to rethink that. It all came from not liking that Echo does not show what is being echoed.

[spiff72]

(07-18-2024 07:17 PM)c3d Wrote:  

(07-18-2024 06:29 PM)spiff72 Wrote:  Interactive Stack question...

Maybe this is expected behavior, but if I want to edit the first level of the stack (let's say I had an extraneous digit on the first level and I want to go back and remove that digit), I hit the left arrow key, then "Edit" from the function menu (F1). From there, I was expecting I could just backspace (the "Drop" key above the division symbol) the extra digit out and hit enter to exit the interactive mode.

Instead, when I do this, the drop key does it's "normal" function of dropping the bottom level on the stack.

How would I actually perform the edits after pressing the edit function key in interactive stack mode?

I think that I messed up the Edit feature by trying to mix what the HPs call Edit and Echo. I need to rethink that. It all came from not liking that Echo does not show what is being echoed.

OK - so it's not just me misunderstanding the feature then...

Thanks!

[c3d]

(07-18-2024 08:24 PM)spiff72 Wrote:  

(07-18-2024 07:17 PM)c3d Wrote:  I think that I messed up the Edit feature by trying to mix what the HPs call Edit and Echo. I need to rethink that. It all came from not liking that Echo does not show what is being echoed.

OK - so it's not just me misunderstanding the feature then...

Thanks!

No, it's more like me misimplementing it by merging "Echo" and "Edit". I'll try to fix it for the next release, but I have to figure out a design that also allows the result of "Echo" to show up in the command line, because I believe that is an improvement over HP's implementation.

[c3d]

Pre-release v0.7.11 is available for download.

This release is a refinement minor release. The primary focus is the interactive stack, which now lets you edit items, sort either according to memory representation or by value, display information about objects, and jump directly to a given stack level using digits.

The simple random number generator implemented in 0.7.10 was replaced with an additive congruential random number generator (ACORN), which can be configured in number of bits and number of iterations. A side effect is that there is now regression testing for single-variable statistics.

The history feature was also improved by automatically enabling the EditMenu when selecting history, and then having the (unshifted) word left and word right commands automatically cycle through history if used at beginning or end of the editing buffer.

Features

help: Add ability to display BMP images in help files
images: Convert help images to BMP
ui: Add Edit feature to interactive stack
ui: Add history menu entries to EditMenu
ui: Accept UNDO while in interactive stack mode
ui: Have word previous/next cycle through history
ui: Accept digits to select stack level in interactive stack
random: ACORN random number generator

Bug fixes

editor: Fix spacing after number followed by - sign
ui: Do not set the editing field from interactive stack
ui: Replace interactive stack "Edit" with "Echo"
ui: Block user input while using interactive stack
ui: Do not draw menu markers when displaying interactive stack
runtime: Avoid crash running above allocated memory in move_globals

Enhancements

ui: Reorganize code handling interactive stack keys
help: Adjust help area to new height for menus

[c3d]

(06-27-2024 04:06 AM)jeanwilson Wrote:  Is there a simple RPL function or program that would output the actual precision which is actually set in the display menu? I am currently working on developing a pseudo-random generator (PRNG based on linear multiplicative congruence algorithms) for the db48x-db50x RPL environment that can both run as quickly as possible while still taking advantage of adjustable calculation precision in order to maximize the repetition period. For the purposes of statistical tests, I am constructing the cumulative distribution function from samples. Such precision output would be useful because the parameters of the PRNG algorithm will explicitly depend on the number of digits chosen in the Display menu of the calculator.

I believe that I answered the question about how to recall the precision setting earlier:

'Precision' RCL

With 0.7.11, the `Rand` and `Random` commands now use an ACORN generator generator. It does not directly adjust with the precision, but gives a configurable number of random bits, set by the RandomGeneratorBits setting.

If you are interested in random number generation, I'd really like your feedback about how this is implemented now. It looks OK to me, but what do I know ;-)

Quote:Concerning, the state saving function, I observed that I am not able to save the calculator state under a new name (useful if a want to communicate the precise configuration of a given problem). Doing Setup>State>Save state, leads to <New Files>; I then correctly enter a new Filename, but the calculator then prompts with - [R/S] to Confirm. And there I am stuck because the [R/S] key is inaccessible since the DM32 key template is gone. I tried any key (including [=]) with no success. One indirect way to circumvent this is to create copies of given states in the computer and upload them afterwards into the calculator. And then save my new states under these pre-existing named ones.

This one proved more tricky and it's still not working. SwissMicros answered my questions, but I'm apparently still doing something wrong, so at the moment, this is still broken on DM32.

[jeanwilson]

Thank you Christophe for your follow-up concerning the problem of saving states from the DM32 calculator (db50x). As I wrote, this minor problem remains easy to get around by downloading previously named state files.

Furthermore, congratulations on successfully completing an elegant implementation of the ACORN random number generation algorithm (ACORN is a family of pseudo-random number generators, invented by R. S. Wikramaratna). This fairly recent (2010s) and very effective technology had interested me since a few months, but I had not managed to figure out how to initialize the process (either I did not find the right documentation or maybe the implementation is really so simple that it seems to be beyond my understanding !).

The alternatives that I anticipated before were technologies dating from the early 1990s (see ref 1, 2 & 3) and turned out to be significantly less advanced and much simpler than what you implemented (they were strictly linear multiplicative congruence methods, which I do not believe that they now satisfied all of the more advanced statistical tests of the TestU01 v1.2.3 suite). What you have achieved is therefore particularly remarkable in terms of speed of execution and I thank you for it. What I have been able to verify so far is that your method satisfies the Kolmogorov-Smirnov criterion with flying colors (cor coef: 99.998%).

I was able to verify this by calculating the CDF distribution directly on the DM32 calculator (so in the db50x v7.11 environment, on a relatively small sampling of 100,000 draws). It would have been preferable to sample from a larger number of random draws using the much greater execution speed of the simulator but it appears that this is impossible for me at the moment because I only have access to the firmware version v.7.10. Indeed, since you upgraded to the most recent versions 7.10 and 7.11, I noticed that (on a Win10 platform and the Fedora Remix environment) the simulator was built each time with the penultimate version number (so recently, 7.10 rather than the most recent version 7.11). I tried to directly address the most recent version in the GitHub repository during the git clone operation but without success because I cannot find the right syntax. Do you have any suggestions ?

A few quick questions.
1. I believe that you are now using the Bignum calculation library instead of the original IEEE 754-2008 128-bit library. Is this indeed the case (as I understand it, to free up more flash memory)? Does the loss of execution speed become significant in this case? Is it proportional to the chosen precision (or to the number of bits required)?
2. Is it then possible with the type of library used to consider a direct implementation of interval arithmetic (similar to what is available in the Longfloat 3.1b package for the HP 50g calculator)?

(1) VAN ES, A. J., GILL, R. D., &0 VAN PUTTEN, C. (1983). Random number generators for a pocket calculator. Statistica Neerlandica, 37,95-102.
(2) PATRICK, O. (1993) A theoretical and empirical comparison of mainframe, microcomputer, and pocket calculator pseudorandom number generators. Behavior Research Methods. Instruments. & Computers. 25 (3). 384-395
(3) KNUTH, D.E. (1981), The art of computer programming, Vol. 11, Seminumerical algorithms (2nd edition),Addison-Wesley, Reading, Massachusetts.

EDIT
Correction to my request/problem regarding the emulator:
I just rebuilt the emulator and this time the most recent version (v7.11) was produced! So false alarm. And taking advantage of its speed I launched a simulation of a billion random draws, just to check if the period is larger than this number. And it appears to be verified after 26 hours of calculation on the db48x emulator! A lame comparison with the Prime G2 emulator which has lower precision (15 digits rather than 24) using its own random number generator (algorithm unknown) shows also a period of at least a billion obtained in less than 52 minutes.

Two other quick questions:
3. I have not transferred the firmware from the db48x to my DM42 calculator. But I was wondering if the execution speed is actually faster for the db50x (on the DM32 calculator) given that the processor has a clock frequency twice as fast? If not, is it due to the different calculator operating systems (DMCP)?
4. Is it possible one day to try adapting the db48x firmware on the HP Prime G2, which would definitely make it an incredible beast as the fastest and most interesting variable precision RPL calculator one could dream of ... (the six F1 to F6 buttons being AlphaApps, AlphaSymb, AlphaLeftArrow, AlphaRightArrow, AlphaHelp and AlphaEsc)? Let's numerically fantasize !

[c3d]

Pre-release 0.7.12 of DB48X is available for download.

Video demo:

[c3d]

(07-24-2024 03:45 PM)jeanwilson Wrote:  Thank you Christophe for your follow-up concerning the problem of saving states from the DM32 calculator (db50x). As I wrote, this minor problem remains easy to get around by downloading previously named state files.

If you need to create a file locally, you can with something like this:

Code:

42 "/state/mystate.48s" STO

And then you have a file named "mystate.48s" that you can overwrite.

Quote:Furthermore, congratulations on successfully completing an elegant implementation of the ACORN random number generation algorithm (ACORN is a family of pseudo-random number generators, invented by R. S. Wikramaratna). This fairly recent (2010s) and very effective technology had interested me since a few months, but I had not managed to figure out how to initialize the process (either I did not find the right documentation or maybe the implementation is really so simple that it seems to be beyond my understanding !).

The alternatives that I anticipated before were technologies dating from the early 1990s (see ref 1, 2 & 3) and turned out to be significantly less advanced and much simpler than what you implemented (they were strictly linear multiplicative congruence methods, which I do not believe that they now satisfied all of the more advanced statistical tests of the TestU01 v1.2.3 suite). What you have achieved is therefore particularly remarkable in terms of speed of execution and I thank you for it. What I have been able to verify so far is that your method satisfies the Kolmogorov-Smirnov criterion with flying colors (cor coef: 99.998%).

Thanks a lot for the verification and for reporting it!

Quote:I was able to verify this by calculating the CDF distribution directly on the DM32 calculator (so in the db50x v7.11 environment, on a relatively small sampling of 100,000 draws). It would have been preferable to sample from a larger number of random draws using the much greater execution speed of the simulator but it appears that this is impossible for me at the moment because I only have access to the firmware version v.7.10. Indeed, since you upgraded to the most recent versions 7.10 and 7.11, I noticed that (on a Win10 platform and the Fedora Remix environment) the simulator was built each time with the penultimate version number (so recently, 7.10 rather than the most recent version 7.11). I tried to directly address the most recent version in the GitHub repository during the git clone operation but without success because I cannot find the right syntax. Do you have any suggestions ?

To me, you should just need a "git pull stable" to get the latest stable release. If that does not work, please share with me the instructions you are using.

Quote:A few quick questions.
1. I believe that you are now using the Bignum calculation library instead of the original IEEE 754-2008 128-bit library.

To be precise, it's "my" bignum calculation library. It was coded from scratch to accommodate the particular needs of DB48X in terms of speed and size. For example, precision is adjusted dynamically, so a constant like 1.23 in your program always takes 5 bytes only.

Quote: Is this indeed the case (as I understand it, to free up more flash memory)? Does the loss of execution speed become significant in this case? Is it proportional to the chosen precision (or to the number of bits required)?

This is documented in the "performance" section of the documentation.

Quote:2. Is it then possible with the type of library used to consider a direct implementation of interval arithmetic (similar to what is available in the Longfloat 3.1b package for the HP 50g calculator)?

Ah, interval arithmetic, one of my long dreams. Well, yes, it can be considered. It's not super high on my todo list at the moment, though.

Quote:EDIT
Correction to my request/problem regarding the emulator:
I just rebuilt the emulator and this time the most recent version (v7.11) was produced! So false alarm.

Good news!

Quote: And taking advantage of its speed I launched a simulation of a billion random draws, just to check if the period is larger than this number. And it appears to be verified after 26 hours of calculation on the db48x emulator! A lame comparison with the Prime G2 emulator which has lower precision (15 digits rather than 24) using its own random number generator (algorithm unknown) shows also a period of at least a billion obtained in less than 52 minutes.

The PrimeG2 uses hardware-accelerated floating-point. Generally, if you want to get similar performance out of your simulator, you need to select hardware acceleration and a precision of either 7 digits (FP32, i.e. C 'float') or 16 digits (FP64, i.e. C 'double').

However, that won't help your specific case, because the random number generator is only producing "decimal" objects, not "hwdouble" or "hwfloat". I only connect to hardware-accelerated functions when they exist, and there is no built-in ACORN generator in the current math library.

Quote:Two other quick questions:
3. I have not transferred the firmware from the db48x to my DM42 calculator. But I was wondering if the execution speed is actually faster for the db50x (on the DM32 calculator) given that the processor has a clock frequency twice as fast? If not, is it due to the different calculator operating systems (DMCP)?

The DM32 is faster on battery, but the firmware at the moment does not boost the CPU clock when connected on USB, so the DM42 is faster on USB. The performance link I gave above also has DM32/DM42 comparisons if you want actual numbers.

Quote:4. Is it possible one day to try adapting the db48x firmware on the HP Prime G2, which would definitely make it an incredible beast as the fastest and most interesting variable precision RPL calculator one could dream of ... (the six F1 to F6 buttons being AlphaApps, AlphaSymb, AlphaLeftArrow, AlphaRightArrow, AlphaHelp and AlphaEsc)? Let's numerically fantasize !

On a Prime G1, this might be doable. On a G2, I think that nobody really knows how to create custom firmware. Maybe I'm wrong.

[spiff72]

I believe that Christophe is correct about G1 vs G2 Prime hardware.

I don't think anyone has been able to do custom firmware on the G2, but the G1 does have a working newRPL firmware available. I bought a G1 off eBay just for this purpose, but I don't really use it for newRPL because it gets used too infrequently, and inevitably the battery drains completely if I forget to charge it. I believe that there is a known battery drain issue with the newRPL firmware.

[c3d]

Release v0.7.13 is out for testing.

The primary focus for this release is the solver, with the completion of the first section in the equation library, Columns and Beams.

This release also introduces an interesting RPL extension, for loops on arrays and lists. For example, try this:

Code:

{ 1 2 3 "ABC" } for i i 2 * 1 + next

This is also the first release where I only ship the .tar.gz files.

[spiff72]

(08-05-2024 07:31 AM)c3d Wrote:  Release v0.7.13 is out for testing.

The primary focus for this release is the solver, with the completion of the first section in the equation library, Columns and Beams.

This release also introduces an interesting RPL extension, for loops on arrays and lists. For example, try this:

Code:

{ 1 2 3 "ABC" } for i i 2 * 1 + next

This is also the first release where I only ship the .tar.gz files.

Can you clarify the installation process with the new .tgz files? I see that there are several folders in there (state, config, and help). Should those be copied over to the calc when copying the qspi and pgm files? My assumption is "YES" for the help files, and state files are optional. I typically keep my units.csv file because i have customized it, so just overwrite the remaining .csv files?

EDIT:
In the interest of keeping space available on the FAT drive, can the db50x.md file be omitted on a DB48x?

EDIT2:
Can the shifts.bmp file in the first level of the help folder be omitted?

[mahi]

I'm not Christophe, but I think I can answer your questions.

(08-06-2024 12:51 PM)spiff72 Wrote:  Can you clarify the installation process with the new .tgz files? I see that there are several folders in there (state, config, and help). Should those be copied over to the calc when copying the qspi and pgm files? My assumption is "YES" for the help files, and state files are optional. I typically keep my units.csv file because i have customized it, so just overwrite the remaining .csv files?

That's right. Normal procedure would be copying the entire contents of the appropriate *.tgz file to the FAT drive of the calculator, overwriting existing files. However, if you made changes to the csv files (like you did), you should skip those files. Also the state files can be omitted safely.

Quote:In the interest of keeping space available on the FAT drive, can the db50x.md file be omitted on a DB48x?

Yes, you can. In fact, I think it's a bug the file is there (it has been like that since 0.4.4 which was around the time DB48X was split into a DM42 and DM32 version).

It doesn't harm, but the db50x.md file is not used by DB48X and similarly the db48x.md file is not used by DB50X. Both help files are nearly identical. Most differences deal with the different keyboards between the DM42 and DM32.

Quote:Can the shifts.bmp file in the first level of the help folder be omitted?

It has no effect on normal operation of the calculator, but it is used in chapter "Keyboard interaction" of the built-in help. If you omit it, you'll get the "Path not found" error when you scroll through that chapter. I'd advise against omitting help images (if you use the help), as it may cause strange behavior of the help viewer (see #1081).

@c3d: I notice that there are a lot of dot-underscore-files in the archive of exactly 163 bytes which appear to be something from MacOS (eg ._db48x.md). I think these should be stripped from the release archives.

[spiff72]

I ended up leaving the shift.bmp in place and essentially did everything you suggested (including pulling out the files with the . prefix).

It just seemed odd that the shift.bmp file was not in the "img" subfolder under "help" (like all the other images are).

[manolinpinguin]

New here.

I got a DM32 and installed DB50x.
It is a huge upgrade from stock firmware.

I ran into an issue and I am hoping someone here can help me. When I try to use help I get a "Hel error: No help for ...". I also noticed that none of the .bmp images used for the Equation Library are being displayed. Any ideas?

I have gone through the process of going back to the Swissmicros firmware and reinstall the DB50x and nothing. I have copied the "help" folder with all the files and subfolder (I have tried with and without the DB48x.md).

Thanks.