New transfer tool for Linux: HPex

[Liam Hays]

Hello! I'd like to introduce something I've created. But before that, I want to note that for those of us using Linux and HP RPL calculators, we don't have a lot of options. HPTalx, the GUI option, is outdated, with a strange interface and no XModem support. Kermit's prompt interface is flexible (and works with XModem) but requires a lot of configuration to be compatible with the HP 48. Kermit is also CLI only.

So, out of necessity, I have created a new software tool: HPex (pronounced h-pecks). It's a fully-featured Linux application for transferring objects to and from HP RPL calculators. HPex has complete graphical and command line interfaces, Kermit and XModem support, automatic serial port detection, and best of all: a threaded, message-based architecture. It's written in Python 3 with wxPython and has few dependencies. I've attached a screenshot of the main window, to showcase the GUI.

After much work, HPex is now ready for its first stable release---version 1.0.0. I won't elaborate much more here, so check it out at https://github.com/liamhays/hpex. To download, click "Releases" on the right side of that page and download "Source code" for version 1.0.0. Follow the installation and usage instructions on the URL I linked.

Try HPex out, and if you find any bugs, problems, points of confusion, or anything else, please say something! I've never released a big project like this before, so I'm excited to see how it goes. Also, please note that HPex is currently unstable for use with the HP 48gII/49 series/50g (which I collectively refer to as the "Meta Kernel calculators"). Until I can get one of these calculators, I recommend only using HPex with the HP 48 series.

[rprosperi]

Neat, thanks!

Would you expect this would also work under Windows, for example under the Anaconda/Python environment? COM ports are of course different, etc. but nearly anything today would be preferred over Conn-4x, it's ancient?

[Liam Hays]

(03-21-2022 05:20 PM)rprosperi Wrote:  Neat, thanks!

Would you expect this would also work under Windows, for example under the Anaconda/Python environment? COM ports are of course different, etc. but nearly anything today would be preferred over Conn-4x, it's ancient?

It's worth trying. Some of the features (particularly serial port detection) might fail or crash. I actually assumed that there were already better tools for Windows...but I didn't do any research. Good job, me! (sarcasm)

If there's a big demand for Windows support, I'd be happy to add it. The problem, though, is that setting up a functional Python environment on Windows is generally a lot more of a hassle than setting one up on Linux. One way around this is to use a program like PyInstaller, which can package up all of a Python program's dependencies, but then the final installer executable is quite large.

[rprosperi]

(03-22-2022 04:01 PM)Liam Hays Wrote:  The problem, though, is that setting up a functional Python environment on Windows is generally a lot more of a hassle than setting one up on Linux. One way around this is to use a program like PyInstaller, which can package up all of a Python program's dependencies, but then the final installer executable is quite large.

Another program that is Python based (pyILPer) is widely used on Windows by lots of members here and the recommended platform to install is Anaconda (or full Conda) and it's dead simple to install and reliable.

I think the reason there is no other modern Windows tool is that Conn4x still works, though for Win7+ you need to disable Driver Signing Enforcement Check at boot in order to install the drivers.

[SammysHP]

Thank you for this neat program! Without further testing I've put it on my todo list for Arch Linux AUR packaging. Just because I like to support HP related software and because it is so nicely described and documented.

[drbarnack]

Thanks for making this fantastic tool! I have it running on my Raspberry Pi, and I've finally been able to transfer libraries to my HP48.

I followed these instructions for installing wxPython on the Raspberry Pi if anyone else takes that route.

[SammysHP]

Btw. I started to package this tool for Arch Linux AUR, but I'm not completely sure how to install python packages if they don't provide a setuptools setup.py or PEP 517 configuration. Maybe you can help me by adding such install definitions.

[Liam Hays]

All the packages I use do have a setup.py. wxPython is just really slow to compile---but it's already in the Arch main package repos. (I run Arch).

What package(s) doesn't have setup.py?

[SammysHP]

Your own package.

The goal is to have all files in /usr/lib/python*/site-packages/hpex which can be done with setup.py install or some PEP 517 magic plus some tools (never tried it myself).

[Liam Hays]

Oh. Whoops.

I actually never considered a simple setup.py. I was thinking about making an AppImage, but that would require bundling all the dependencies into one file (resulting in a large executable) and would probably require a setuptools script anyway.

I'll throw a setup.py on the todo list. If you put HPex on the AUR, would I be expected to maintain the AUR package?

[SammysHP]

(03-31-2022 10:35 PM)Liam Hays Wrote:  I'll throw a setup.py on the todo list. If you put HPex on the AUR, would I be expected to maintain the AUR package?

Great! The PKGBUILD is already finished and I will push it once the setup.py is available and after I have tested the package.

If you like I can add you as a maintainer, but otherwise you don't need to worry about maintaining it. I'll push hpex-git first which always builds the latest version from the git. Specific release packages make sense when there are enough users (which I somehow doubt considering the rather small HP calculator community, especially only the overlap wihh the Arch community).

[Liam Hays]

Thank you! Just a warning, it might be a little bit (a week or two) until I get a working setup.py, I'm quite busy right now.

If I don't need to be added to the AUR package, I would prefer not to be. I also intend to extend HPex to Windows at some point, since that is clearly a much larger HP user set than Linux.

[floppy]

(04-01-2022 10:35 PM)Liam Hays Wrote:  Thank you! Just a warning, it might be a little bit (a week or two) until I get a working setup.py, I'm quite busy right now.

If I don't need to be added to the AUR package, I would prefer not to be. I also intend to extend HPex to Windows at some point, since that is clearly a much larger HP user set than Linux.

First, thanks for your program and contribution.
Windows? dont forget the raspbian users, too (raspberry PI with linux raspbian).
An example of a raspberry PI setup for an HP41 I achieved till now is in the attachment.
in short:
raspberry PI4
raspbian 32bits on it
"wine" and its binairies 32bits emulated with "box86"
ilper (for windows) or/and V41 started with "wine" (I can start ilper on my linux64bits with wine, too, and therefore programs can travel easypeasy between the ARM, PC, HP41, V41 on ARM/PC)

[SammysHP]

(04-01-2022 10:35 PM)Liam Hays Wrote:  Thank you! Just a warning, it might be a little bit (a week or two) until I get a working setup.py, I'm quite busy right now.

If I don't need to be added to the AUR package, I would prefer not to be. I also intend to extend HPex to Windows at some point, since that is clearly a much larger HP user set than Linux.

Sorry for the late response, but I hadn't noticed that you've already added the setup.py! Now that it's there I've updated my PKGBUILD and pushed it to the AUR: https://aur.archlinux.org/packages/hpex-git

(Looks like I also became maintainer for python-xmodem because it wasn't in the AUR yet.)

[Liam Hays]

Thank you. I need to mention, though, that I'm not spending too much time on HPex right now. I'll probably come back, but frankly I realized that the userbase for HPex is (sadly) almost pointlessly small. I think I'll have more time over the summer to finish the changes I've started so far. I have already completed plenty of work on integrating support for the XModem server as well as running on Windows, and I'd rather not leave that abandoned.

[ramon_ea1gth]

I'm a Linux user and I wished a modern replacement of hptalx, so this looks great! Some time ago, I compiled hptalx from source, but the behaviour was a bit quirky sometimes.

So I just prepared my Linux Mint (Ubuntu-based) to run HPex. From the packet manager, I installed python-wxgtk4.0 to have ready wxPython.

And with pip, in addition to the other requirements (xmodem, pypubsub, ptyprocess and pyserial), I had to install dataclasses, not indicated in the version of the instructions I have. However, I have to confess that I am with Python 3.6.9, so maybe that's the problem. Anyway, the GUI has launched successfully, so now, let's see what happens with my HP48 GX and with my HP 50g! (the HP 50g is the one most I use).

Thanks for your program!

[Edit]:

I am testing HPex with my HP 50g. For some reason, when XModem is checked, the Connect label is not enabled. So I'm testing Kermit mode: when connecting, the HP 50g screen shows that about 10 packets are sent to the computer, so Kermit is working. However, the HPex connection is not achieved, the calculator files are not displayed and the terminal shows this output:

Code:


$python3 hpex.py
finished or failed, err: End Of File (EOF). Exception style platform.
kermit_done in HPex
first connect
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/wx/core.py", line 3190, in <lambda>
    lambda event: event.callable(*event.args, **event.kw) )
  File "/home/ramros/.local/lib/python3.6/site-packages/pubsub/core/publisher.py", line 216, in sendMessage
    topicObj.publish(**msgData)
  File "/home/ramros/.local/lib/python3.6/site-packages/pubsub/core/topicobj.py", line 452, in publish
    self.__sendMessage(msgData, topicObj, msgDataSubset)
  File "/home/ramros/.local/lib/python3.6/site-packages/pubsub/core/topicobj.py", line 482, in __sendMessage
    listener(data, self, allData)
  File "/home/ramros/.local/lib/python3.6/site-packages/pubsub/core/listener.py", line 237, in __call__
    cb(**kwargs)
  File "/home/ramros/hpex/hpex-1.0.0/hpex_gui.py", line 1009, in kermit_done
    self.process_kermit_data(out)
  File "/home/ramros/hpex/hpex-1.0.0/hpex_gui.py", line 868, in process_kermit_data
    crc=KermitProcessTools.checksum_to_hexstr(i[3])))
  File "/home/ramros/hpex/hpex-1.0.0/helpers.py", line 92, in checksum_to_hexstr
    return '#' + str(hex(int(checksum))).replace('0x', '').upper() + 'h'
ValueError: invalid literal for int() with base 10: '13036,'

Let's see if I find something else.

Best regards!

[Thomas Klemm]

(05-16-2022 07:52 PM)ramon_ea1gth Wrote:  However, I have to confess that I am with Python 3.6.9, so maybe that's the problem. Anyway, the GUI has launched successfully, so now, let's see what happens with my HP48 GX and with my HP 50g! (the HP 50g is the one most I use).

From the README:

Quote:Also important: While HPex is currently stable when used with the HP 48 series, I don't currently own any of the Meta Kernel calculators (the 49G, 49g+, 50g, and 48gII). As a result, consider HPex unstable and untested for use with these models.

Software Requirements
Python 3.9 or better

(05-16-2022 07:52 PM)ramon_ea1gth Wrote:  I had to install dataclasses, not indicated in the version of the instructions I have.

They came only with Python 3.7.
But it appears that they were back-ported to 3.6, so you could install them as a package.

Code:

  File "/home/ramros/hpex/hpex-1.0.0/hpex_gui.py", line 868, in process_kermit_data
    crc=KermitProcessTools.checksum_to_hexstr(i[3])))
  File "/home/ramros/hpex/hpex-1.0.0/helpers.py", line 92, in checksum_to_hexstr
    return '#' + str(hex(int(checksum))).replace('0x', '').upper() + 'h'
ValueError: invalid literal for int() with base 10: '13036,'

From the code in process_kermit_data it looks as if the last element crc contains an additional comma at the end: '13036,'
This leads to the error since it can not be translated to an int.

Code:

        kermit_vars = [row.split() for row in lines]
        
        for i in kermit_vars:
            # each element of each row is the name, size, type, and
            # crc in that order
            
            # undo the thing we did in kermit_done()
            t = i[2]
            if t == 'RealNumber':
                t = 'Real Number'
            elif t == 'GlobalName':
                t = 'Global Name'
                
            self.hpvars.append(
                KermitVariable(name=i[0], size=i[1], vtype=t,
                               crc=KermitProcessTools.checksum_to_hexstr(i[3])))

In order to analyse the problem you could insert a print statement in line 858 of /home/ramros/hpex/hpex-1.0.0/hpex_gui.py:

Code:

            # each element of each row is the name, size, type, and
            # crc in that order
            print(i)

This might give some insights into the output from Kermit.

[Liam Hays]

Like I said, HPex isn't tested with the Meta Kernel calculators. Also, my Git skills aren't too sharp, so I just push to the main branch...don't clone the repo to run HPex, since it's probably broken, get the release from the GitHub sidebar.

As for the Python versioning, I was trying to make sure that some of the newer features I'm using (typehints and dataclasses were the two features in particular).

[ramon_ea1gth]

Dear Liam, I would like to remark this again: thank you for your software and your time to write it. For me, it's great to have again a connection tool available for Linux, after struggling with the intermittent glitches of hptalx. Great!

(05-17-2022 05:55 PM)Liam Hays Wrote:  Like I said, HPex isn't tested with the Meta Kernel calculators.

Yes, I read it. But I wanted to go straight to the risky tests ;-) However, the HP 50g test looks promising, as apparently Kermit starts the communication, so maybe there's a silly difference in an isolated packet that breaks the connection. Anyway, before going further, I will play with my hp 48gx to confirm that my Python setup works as expected, as I'm slightly out of some of the recommended requirements of HPex.

(05-16-2022 11:24 PM)Thomas Klemm Wrote:  In order to analyse the problem you could insert a print statement in line 858 of /home/ramros/hpex/hpex-1.0.0/hpex_gui.py:

Code:

            # each element of each row is the name, size, type, and
            # crc in that order
            print(i)

This might give some insights into the output from Kermit.

Thanks for the hint Thomas and your comments! I will try it to gather some more information and report.

[ramon_ea1gth]

I believe I have some clues about what happens with the connection with a MetaKernel calculator. If I choose in the calculator the decimal separator to be a comma, this is the error reported in the computer terminal when starting a Kermit connection:

Code:

  File "/home/ramros/hpex/hpex-1.0.0/helpers.py", line 92, in checksum_to_hexstr
    return '#' + str(hex(int(checksum))).replace('0x', '').upper() + 'h'
ValueError: invalid literal for int() with base 10: '13036,'

However, choosing a dot as the decimal separator in the calculator, the error report turns into:

Code:

  File "/home/ramros/hpex/hpex-1.0.0/helpers.py", line 92, in checksum_to_hexstr
    return '#' + str(hex(int(checksum))).replace('0x', '').upper() + 'h'
ValueError: invalid literal for int() with base 10: '13036.'

Notice the change from '13036,' to '13036.'
As the MetaKernel calculators have real numbers (type 0) and integers (type 28), MetaKernel machines differentiate them by appending the decimal separator (dot or comma) in the real number. So some numbers/results (type 0) that were output by the HP48 series with no decimal separator now are delivered with it in a MetaKernel machine. Thus, I guess that HPex has to remove this separator in a MetaKernel calculator.

[Edit]:
I forgot to include the results from Thomas Klemm's suggestion, so here they are:

(05-16-2022 11:24 PM)Thomas Klemm Wrote:  In order to analyse the problem you could insert a print statement in line 858 of /home/ramros/hpex/hpex-1.0.0/hpex_gui.py:

Code:

            # each element of each row is the name, size, type, and
            # crc in that order
            print(i)

This might give some insights into the output from Kermit.

So here it is the output of the print command:

Code:


['DibSmith', '6149.5', 'Directory', '13036.']

Good suggestion to support these ideas.