Emulator Source Load and Save

[BarryMead]

You may have noticed that the tools for assembling/disassembling and managing WP-34S source programs are very crude compared to
the way Torsten does it in his HP-15C simulator.

When I tried to explain the concept of being able to instantly load a program from a vast library of user supported WP-34S source programs
the core development team shrugged it off, saying you can already do everything you want with the tools we provided.

Well I beg to differ. If you want to load a program into the wp34s you have to go through the following list of complex steps.

Step One. Run the emulator and issue these commands: P.FCN END, f CLP , and press Y for Yes then (Exit the emulator) (This creates a wp34s.dat with an empty RAM program area)
Step Two. Copy the empty program wp34s.dat file from where your emulator saves it to the wp34s/trunk/tools directory
Step Three. Compile your source from the tools directory with: perl wp34s_lib.pl sourcefile.wp34s -ilib wp34s.dat -f -state (This adds the program to the empty wp34s.dat file)
Step Four. Copy the updated wp34s.dat file from the wp34s/trunk/tools directory back to the directory where the emulator needs it.
Step Five. Run the Emulator (Now the newly compiled program is in RAM)
Step Six. Transfer the RAM to your calculator over the serial interface (P.FCN SENDP) on Emulator and (P.FCN RECV) on calculator.

This is a clumsy and complex set of operations just to load a source program into your calculator.
And to make matters worse, each version of the emulator (Bit's) or (Complex) has different wp34s.op file to use with the compiler.
So keeping all of this stuff straight can be quite difficult.

I proposed that the Emulator have a "Built-In Source Level Program Loader (Compiler)" which could call the underlying primitives
as needed to handle the compile. This would make it much more likely that users would share their programs with others.

The emulator also needs a capability to Disassemble the programs in it's RAM and save them to a source file.

This procedure currently looks like this:

Step One. Transfer the program in RAM to the emulator (over the serial cable) with (P.FCN SENDP) on the calculator and (P.FCN RECV) on the emulator.
Step Two. Exit the Emulator and copy the wp34s.dat file from where the emulator uses it to the wp34s/trunk/tools directory.
Step Three. Disassemble the program in the tools directory with: perl wp34s_asm.pl -dis wp34s.dat -s 0 -ns -dis_alias -o sourcefile.wp34s

So there are complex steps going in both directions to SAVE a program from the calculator to a source file, and to LOAD a program from
a source file into the calculator. I believe that if these operations were made as simple as they are in Torsten's HP-15C Simulator then more people
would share the programs they develop, and the community would benefit as a whole.

Oh well, If I were a little more familiar with Qt programming, I would modify Pascal's emulator code myself to add these Source Load/Save features.

There are only so many tasks that one person can tackle.

If more than one of us were to request these features and phrase them in a way that others could appreciate the benefits, then perhaps there is
a chance that more developers would show an interest in adding this capability the the WP-34S emulator, or Qt emulator.

[Marcus von Cube]

"Du rennst offene Türen ein!"

Which roughly translates to "You're banging against an open door".

I'm thinking of this for some time now but it's not easy to implement if we just want to call Neil's Perl based tools from the emulator. We'll need a couple of configuration options to achieve this. (Where is Perl installed? Where are the tools? etc.) And we have to add menu options and dialogues to the emulator. Since "my environment" is the classic emulator (the only one I can modify and compile myself) it's just too much work to enhance the GUI to support it. I'm not a GUI guy, I'm afraid.

Doing it all in C seems easier but we lose the features of the preprocessor...

[BarryMead]

(11-25-2015 06:13 PM)Marcus von Cube Wrote:  "Du rennst offene Türen ein!"

Which roughly translates to "You're banging against an open door".

I'm thinking of this for some time now but it's not easy to implement if we just want to call Neil's Perl based tools from the emulator. We'll need a couple of configuration options to achieve this. (Where is Perl installed? Where are the tools? etc.) And we have to add menu options and dialogues to the emulator. Since "my environment" is the classic emulator (the only one I can modify and compile myself) it's just too much work to enhance the GUI to support it. I'm not a GUI guy, I'm afraid.

Doing it all in C seems easier but we lose the features of the preprocessor...

The bare bones emulator would be hard pressed to add this feature, I agree. I wish I was more familiar with Qt programming. If I were, I would take on the task myself. As you no-doubt can tell from my earlier posting, I already KNOW how to load/save source files to/from my emulator or calculator. I think for the average user it would be rather difficult for them to learn all of these steps, so they just don't bother and their WP-34S programming contributions remain UNSHARED and lost to the community.

[walter b]

Barry, if you don't mind I'll include your explanations in a future update of TFM. Thanks for posting.

d:-)

[fhub]

(11-25-2015 06:13 PM)Marcus von Cube Wrote:  I'm thinking of this for some time now but it's not easy to implement if we just want to call Neil's Perl based tools from the emulator. We'll need a couple of configuration options to achieve this. (Where is Perl installed? Where are the tools? etc.) And we have to add menu options and dialogues to the emulator. Since "my environment" is the classic emulator (the only one I can modify and compile myself) it's just too much work to enhance the GUI to support it. I'm not a GUI guy, I'm afraid.

Doing it all in C seems easier but we lose the features of the preprocessor...

Marcus, I've made a small package for exactly these 4 purposes:
compiling sourcefiles to RAM (or LIB) and listing all programs in RAM (or LIB) to a sourcefile.
It contains 4 batch-files (AddRAM/AddLIB where you just have to enter a list of sourcefiles as parameter, and ListRAM/ListLIB without any parameters) and Neil's 3 tools (asm, lib and pp) in compiled form (i.e. 3 EXEs, each less than 1 MB), so no Perl is needed at all.

It's very comfortable to use (I'm using it all the time), so if this small package would be interesting to you (or maybe even help you in integrating these features into the emulator), then just let me know and I'll upload it (it's just a 2MB ZIP-file).

Franz

[BarryMead]

(11-25-2015 06:37 PM)fhub Wrote:  It's very comfortable to use (I'm using it all the time), so if this small package would be interesting to you (or maybe even help you in integrating these features into the emulator), then just let me know and I'll upload it (it's just a 2MB ZIP-file).

Franz

I want a copy! Perhaps if you have enough attachment space left on your account your could post the zip file as an attachment right here in this forum. What do you think?

[BarryMead]

(11-25-2015 06:29 PM)walter b Wrote:  Barry, if you don't mind I'll include your explanations in a future update of TFM. Thanks for posting.

d:-)

I don't mind. I think you can tell that I think sharing is a good thing!

[fhub]

(11-25-2015 07:00 PM)BarryMead Wrote:  I want a copy! Perhaps if you have enough attachment space left on your account
your could post the zip file as an attachment right here in this forum. What do you think?

Ok, I try it with an attachment (I don't know if a 2MB file is possible) ...

No, it allows only 1 MB as attachment, so here's an external link:
http://rghost.net/6nWBRWcVY (new version!)

Either put all unzipped files into a separate folder, or just put them into the emulator directory (in this case you don't need to copy wp34s.dat and/or wp34s-lib.dat between the 2 folders).

To add one or more programs to the RAM type:
AddRAM prog1 [prog2] ...

Remarks:
1) if the files have the extension .wp34s, then you can omit this extension
2) if the file wp34s.dat does NOT exist (e.g. if you've deleted it before), then a new one is created automatically

To add one or more programs to the LIBrary type:
AddLib prog1 [prog2] ...

(same remarks as above, only for the file wp34s-lib.dat)

And to list all RAM or LIB programs just enter:
ListRAM [file]
or
ListLIB [file]

Remarks:
without parameter [file] the created list- or sourcefiles are wp34s.lst resp. wp34s-lib.lst (else the name 'file' is used for the source listing)

Edit: Modified programs so that also program names like e.g. LBL A (or local labels like e.g. LBL 00) are handled correctly (when compiling/decompiling).

Franz

[BarryMead]

(11-25-2015 06:13 PM)Marcus von Cube Wrote:  If we just want to call Neil's Perl based tools from the emulator. We'll need a couple of configuration options to achieve this. (Where is Perl installed? Where are the tools? etc.) And we have to add menu options and dialogues to the emulator. Since "my environment" is the classic emulator (the only one I can modify and compile myself) it's just too much work to enhance the GUI to support it. I'm not a GUI guy, I'm afraid.

The Windows exe files are already provided, and Linux or Mac users could install the Perl Packager (On Linux systems this command does it: sudo apt-get install libpar-packer-perl). Using this perl packager program "pp" one could compile native binary executable versions of: wp34s_asm.pl, wp34s_conv.pl, wp34s_lib.pl, and wp34s_pp.pl.
Compiling these perl scripts into native binary executables is easy once the Perl Packager is installed (pp -o wp34s_asm wp34s_asm.pl ... etc.). As long as they are installed where the PATH environment variable can find them, then the calling of these primitives would be identical across the various platforms from the Qt Emulator's point of view.

system("wp34s_lib sourcefile.wp34s -ilib wp34s.dat -f -state");

for example.

Any thoughts?

[rprosperi]

Replying to no one in particular, or to everyone:

I fully agree with Barry's comments about the value to the community of enhancing the emulator's capabilities to easily upload/download, and therefore more easily share 34S applications.

Just as Jake & Jackie's Barcode listings of all published 41C programs and Torsten's Simulator allow simple upload/download of 15C programs to a DM-15, this would make it trivial for users to share programs, without requiring tedious re-keying, which is rarely perfect the first (or second, or even third) time for anything but trivial programs. The added complexity of the 34S instruction set, and the required large number of keystrokes makes this capability even more valuable. The "complexity" is of course due to the vast power of the enhanced features in the 34S, so this is not criticism, just acknowledging that the extra power comes with a price (as it always does).

The ultimate result is that most of the significant programs that have been written, debugged and shared (btw, thank you authors) are probably rarely used, due to the tedious task of reentering and the myriad steps Barry described to download, or to re-key on the machine itself, again a frustrating process for long programs.

My recent acquisition of a DM-15C and Torsten's 15C Simulator have dramatically increased the number of programs I've entered to try out, since the "loading" process is very quick allowing more time to explore the problems the program solves. For moderate to complex programs, the time it takes to enter a program typically far exceeds the time then spent using it; after all, I think we want to use the programs more than enter them.

I imagine (just guessing, but feels likely) that the biggest benefit is that more programs would actually be shared, if it's quick and easy to do so. Probably, many of the best 34S programs written by users to date have never been seen by others, simply because it's a pain to do so.

I have none of the programming skills required to do this, so I really shouldn't be complaining about it not being done; rather I'm offering an opinion from a 34S user that doesn't really use it as often as I'd like for the reasons noted.

I love the 34S, I just would like to able to use it, and love it, more.

[fhub]

(11-25-2015 08:21 PM)rprosperi Wrote:  I imagine (just guessing, but feels likely) that the biggest benefit is that more programs would actually be shared, if it's quick and easy to do so. Probably, many of the best 34S programs written by users to date have never been seen by others, simply because it's a pain to do so.

Well, try my WP34sTools package above - maybe you'll change your mind of "it's a pain to do so".

Franz

[Dieter]

(11-25-2015 06:13 PM)Marcus von Cube Wrote:  I'm thinking of this for some time now but it's not easy to implement if we just want to call Neil's Perl based tools from the emulator.

Why not start with something very simple that should be easy to do: allow the emulator to load a specific .dat file other than wp34s.dat. We have been discussing this some time ago and it seems to be a good thing that adds a lot of usabilty. This way .dat files could be shared, and even loaded into the emulator by simple drag and drop. Once the program is in the emulator is can be easily transferred to "real hardware" using the SENDP command.

A corresponding "save as" command would allow saving different calculator state files. I have seen how immensly useful this is while working on the statistical distributions package for the HP35s emulator (which is a pain to use in other aspects, but that's a different story).

What do you think?

Dieter

[BarryMead]

(11-25-2015 09:15 PM)Dieter Wrote:  Why not start with something very simple that should be easy to do: allow the emulator to load a specific .dat file other than wp34s.dat. We have been discussing this some time ago and it seems to be a good thing that adds a lot of usabilty. This way .dat files could be shared, and even loaded into the emulator by simple drag and drop. Once the program is in the emulator is can be easily transferred to "real hardware" using the SENDP command.

A corresponding "save as" command would allow saving different calculator state files. I have seen how immensly useful this is while working on the statistical distributions package for the HP35s emulator (which is a pain to use in other aspects, but that's a different story).

Dieter

We don't have to reach the goal line in one big step. Dieter has an excellent point.

[BarryMead]

I related quirk about the assembler/disassembler is that there are programs that you can key into your calculator/emulator and disassemble, but that you cannot re-assemble without errors.

For instance if you create a program in your memory that uses the Hot Key labels A, B, C, or D, and save (Disassemble) the program to a source file. This file CANNOT be reassembled. I presume this error was done to force library programmers to use Quoted labels 'A' instead of A. If you use 'A' your program will compile, but 'A' is not the same as HotKey A.

Anyway to get around this compiler error, I just changed my source files to use quoted labels ('A' instead of A), then manually corrected the programs in the calculator/emulator back to Hot Key A once the program was assembled and loaded.

This is one more frustrating hurdle making it less easy for people to write and share their programs.

[Paul Dale]

(11-25-2015 10:19 PM)BarryMead Wrote:  For instance if you create a program in your memory that uses the Hot Key labels A, B, C, or D, and save (Disassemble) the program to a source file. This file CANNOT be reassembled. I presume this error was done to force library programmers to use Quoted labels 'A' instead of A. If you use 'A' your program will compile, but 'A' is not the same as HotKey A.

This seems more like a bug in the assembler. it wasn't intentional and it definitely wasn't to force quoted labels.


Pauli

[Marcus von Cube]

(11-25-2015 09:15 PM)Dieter Wrote:  Why not start with something very simple that should be easy to do: allow the emulator to load a specific .dat file other than wp34s.dat. ...

A corresponding "save as" command would allow saving different calculator state files...
What do you think?

Du kannst Gedanken lesen.:-)

This is one of the things on my todo list for the classic emulator for some time. I just didn't find the time so far.

[Marcus von Cube]

(11-25-2015 07:21 PM)fhub Wrote:  here's an external link:
http://rghost.net/8cQ7458H6

Chrome on my Macbook refuses to load the file. It is considered harmful.

[fhub]

(11-25-2015 11:08 PM)Marcus von Cube Wrote:  Chrome on my Macbook refuses to load the file. It is considered harmful.

I'd rather say your Chrome is harmful ...

Franz

[fhub]

(11-25-2015 10:19 PM)BarryMead Wrote:  I related quirk about the assembler/disassembler is that there are programs that you can key into your calculator/emulator and disassemble, but that you cannot re-assemble without errors.

For instance if you create a program in your memory that uses the Hot Key labels A, B, C, or D, and save (Disassemble) the program to a source file. This file CANNOT be reassembled. I presume this error was done to force library programmers to use Quoted labels 'A' instead of A. If you use 'A' your program will compile, but 'A' is not the same as HotKey A.

Hi Barry,

I've updated the link for my WP34sTools.zip in post #8 - it now also handles LBL A (or B,C,D) correctly when disassembling and reassambling, just a small change in the Perl script wp34s_lib was necessary.
(it even works if the program starts only with a local label like e.g. LBL 00)

Franz

[BarryMead]

(12-06-2015 12:20 PM)fhub Wrote:  I've updated the link for my WP34sTools.zip in post #8 - it now also handles LBL A (or B,C,D) correctly when disassembling and reassambling, just a small change in the Perl script wp34s_lib was necessary.
(it even works if the program starts only with a local label like e.g. LBL 00)

Franz

Franz:

Would you please also include the corrected PERL script wp34s_lib.pl? We in the Linux (and Mac) worlds don't have as much use for the .exe only versions. After validation that no library compiles get broken, it would be nice to see this correction included in the SVN repository. That's one less hurdle to open sharing of wp34s programs.

Thanks everyone, Barry