HP 75 and the PIL-Box

[Michael Fehlhammer]

Today I copied a lot of files between an HP 75 and a PC, connected through a PIL box. It was an incredibly tedious task: a single file of 1000 bytes took about 20 seconds in average ( both directions ) ! I remember the HP 71b to accomplish that one order of magnitude faster!
I tried two PCs ( Windows Vista and 7 ), two PIL- Boxes and two HP 75 ( C and D), always the same result. My PIL-Boxes are jumpered to support 115 kbs baud rate.
Any ideas why the HP 75 / PIL-box combo is so slow? Or did anyone experience a significantly higher speed ?

[Sylvain Cote]

Hello Michael,
I just did the following quick tests ...
1) HP-75C connected to a HP-9114B with a LIF formatted 720K floppy
-> it took around 2 seconds to write a 1.5KB file
2) HP-75C connected to a PIL-Box (115K) and to ILPer v1.43
-> it took around 33 seconds to write a 1.5KB file
3) HP-71B connected to a HP-9114B with a LIF formatted 720K floppy
-> it took around 2 seconds to write a 1.4KB file
4) HP-71B connected to a PIL-Box (115K) and to ILPer v1.43
-> it took around 30 seconds to write a 1.4KB file
So the delay is coming either from the PIL-Box or from the ILPer simulator or both.
Best regards,
Sylvain

[Michael Fehlhammer]

Thank you for testing and reporting the results, Sylvain!

I think we should forward these figures to J.F. Garnier and Christoph Gießelink.
Maybe they are reading?

Next interesting comparison could be: Jeff's older version of ILPER (without TCP connection) versus Christoph's version.
I can't recall that anyone has ever mentioned that a PIL-Box connection (to a modern hard disk) is more than 10 times (!!) slower than a good old 9114 disk drive.

[Sylvain Cote]

Further tests ...

Setup ...
71 test file: KEYBOARD,LEX,1.4KB
75 test file: MYFILE,BAS,1.5KB
HP-75C: [VER$ -> aaaaaa]
HP-71B: [VER$ -> HP71:1BBBB HPIL:1B]
Emu71: v1.04 for Windows [VER$ -> HP71:2CDCC HPIL:1B]
Emu71: v2.41 for DOS [VER$ -> HP71:2CDCC HPIL:1B]
HP-9114B: Build date -> Y:1985 W:41
ILPer for Windows: v1.43
PIL-Box: v1.4

Tests ... [ write-to: -> ] [ read-from: <- ]
------------------------------------------------------
HP75C -> HP-9114B = ~2 sec.
HP75C <- HP-9114B = ~2 sec.
------------------------------------------------------
HP71B -> HP-9114B = ~2 sec.
HP71B <- HP-9114B = ~2 sec.
------------------------------------------------------
Emu71 Win -> ILPer = ~2 sec.
Emu71 Win <- ILPer = ~2 sec.
------------------------------------------------------
Emu71 DOS -> HDRIVE1 = ~1 sec.
Emu71 DOS <- HDRIVE1 = ~1 sec.
------------------------------------------------------
HP75C -> PIL-Box (115Kbps) -> ILPer = ~33 sec.
HP75C <- PIL-Box (115Kbps) <- ILPer = ~29 sec.
------------------------------------------------------
HP71B -> PIL-Box (115Kbps) -> ILPer = ~30 sec.
HP71B <- PIL-Box (115Kbps) <- ILPer = ~27 sec.
------------------------------------------------------
Emu71 Win -> PIL-Box (115Kbps) -> HP-9114B = ~30 sec.
Emu71 Win <- PIL-Box (115Kbps) <- HP-9114B = ~27 sec.
------------------------------------------------------
Emu71 DOS -> PIL-Box (9.6Kbps) -> HP-9114B = ~90 sec.
Emu71 DOS <- PIL-Box (9.6Kbps) <- HP-9114B = ~110 sec.
------------------------------------------------------

The above tests tend to demonstrate that the delay is induced by the PIL-Box.

Best regards,

Sylvain

edit1: add write-to and read-from comment
edit2: add Emu71 DOS values

[hp41cx]

Thanks J.F. Garnier and Christoph Gießelink the excellent interface.
It was a great evolution in the communication of my loves.

[Sylvain Cote]

After more investigation ...

I was able to cut the transfer time by three by configuring the FTDI Windows virtual com port.

1) Show Device Manager dialog.
[attachment=106]
2) Select the COM port and show Property dialog.
[attachment=109]
3) Select Port Setting tab and press the advanced button.
[attachment=110]
4) Change the Latency Timer (msec) from 16 to 1 then press all the OK buttons
[attachment=111]

Best regards,

Sylvain

[Ángel Martin]

Thanks for the pointer Sylvain, I got to try that on my XP machine - lately I've noticed unusually slow Pil-Box response times as well, maybe this helps.

Incidentally, if anybody could bend J-F's or Christoph ear to make the ILPer window sizable... I know you can scroll the text window down but it's a shame it can't take advantage of today's larger monitors.

Cheers,
ÁM

[Michael Fehlhammer]

Now I have tried several combinations of Windows operating systems (Win 7 ultimate and Win 7 home premium, both 64 bit), FTDI serial driver versions (2.8.30 and 2.8.28 ) and driver setups ( like the modification proposed by Sylvain ), but the transfer never is faster than 33 seconds for a 1550 Bytes HP 75 lex file.
Sylvain, which operating system version do you use? You are better by a factor of three, how is that possible?

The transfer rate even breaks down further to about 87 seconds for the same file when the medium ( the LIF file on the pc ) isn't almost empty, but contains several files ( in my case: 39 files, about 50 kBytes ).
I guess the algorithm to find the next free records on the medium creates a lot of communication, resulting in almost three times as many HP IL frames being sent through the loop. I end up with an effective transfer rate of less than 20 Bytes per second. Obviously there's a lot of handshaking, welcoming, New Year's greeting and Happy Birthday wishing between the box and every single byte ... :-)
Did anyone contact J.F. Garnier on that? I'm going to write him an email.

[Sylvain Cote]

Hello Michael,

My setup is the following ...
- MacBookPro v6.1 8GB RAM
- OSX Mavericks 64 bits v10.9.1
- Parallels Desktop 9 for Mac v9.0.24172
- VM setup: 1 core with 2 GB RAM
- Windows 7 Pro 32 bits v6.1+SP1+LastUpdates
- FTDI Driver 2.8.28.0 (2013-01-18)

In the light of your experiences, just in case, I will redo the latency tests.
I will also do some tests with bigger files and with several files on the volume.
I should be able to report back tomorrow evening (UTC-5h) with the results.

Best regards,

Sylvain

[J-F Garnier]

The main limitation comes from the FTDI driver, so it is very important to reduce the lantency timer to a minimum (1ms) as pointed out by Sylvain.

The second limitation is the serial speed between USB and the PIL-Box uC, using the 115kps setting makes it negligible.

With these settings, transfer rate will be in the range of 200-400 bytes/s. Still lower than the real HP hardware and more than enough for the HP-41, but the limited transfer rate is noticeable with the HP71B/HP75.

My today's tests with 1 ms timer latency and 115kps PIL-Box settings, using a HP-71B and a 1722 byte test file, show that the transfer rate also depends on the host system:

My old Win2K desktop - FTDI 2.08.02 : 9.8s (175 bytes/s)

My latest Win8-64bit notebook - FTDI 2.08.30 : 3.8s (450 bytes/s)

J-F

(to measure transfer rate, I use something like: T=TIME @ COPY xxx TO :2 @ DISP TIME-T)

Added: with scope trace disabled in ILPer...

[Christoph Giesselink]

(01-02-2014 12:36 AM)Michael Fehlhammer Wrote:  ...
I guess the algorithm to find the next free records on the medium creates a lot of communication, resulting in almost three times as many HP IL frames being sent through the loop. I end up with an effective transfer rate of less than 20 Bytes per second. Obviously there's a lot of handshaking, welcoming, New Year's greeting and Happy Birthday wishing between the box and every single byte ... :-)
Did anyone contact J.F. Garnier on that? I'm going to write him an email.

Hello Michael,

like Sylvain demonstrated, first we have to check who's responsible for the slow transfer on your system: PIL-Box or ILPer.

Therefore a test with Emu71/Win v1.05 with ILPer v1.43 over Virtual IL will show if ILPer is working properly.

If ILPer in this configuration is slow, I think about two possibilities. 1st, I assume that you switched off the output in the Scope window by unchecking the Scope checkbox. A parallel data output of the transferred data bytes in the scope window will massively slow down the execution speed. 2nd, perhaps a virus scanner is watching the DAT/LIF file of your virtual disk. From ILPer software design, the DAT/LIF file is normally closed to allow parallel access to this file from Emu41 or Emu71/DOS. On each sector number read or write (256 bytes) the file will be opened and after accessing the sector the file will be closed again. This may trigger a virus scanner and the scanner block the file until it's checked. Or is your DAT/LIF file on an external device like a NAS? Ethernet / WLAN access produce a lot of communication overhead.

If the problem is the PIL-Box side, only J-F Ganier may help. I don't know the internals of the PIL-Box, my PIL-Box communication protocol references had been the VB version of ILPer 1.35 (for device mode) and ILCtrl 1.01 (for controller mode).

To answer an earlier question in this thread: There's no speed difference between ILPer 1.35.3 (C++ version without TCP) and ILPer 1.43 (C++ version with TCP). Both versions use more or less the same PIL-Box access code. But I don't have any information about the speed issue related to the latest VB version 1.35.

[Michael Fehlhammer]

Thank you Jean-Francois, thank you Christoph for joining our discussion!

Fortunately I finally succeeded in gaining a high transfer rate of almost 400 Bytes per second - and got closer to the root of all evil:

I did the following tests:
Just Emu71Win and ILPer v.1.43: Copying of a 1.7 kByte file took less than one second!
Then my favorite setup:
My HP 75c linked to the PIL-Box, ILPILBox (!) running on my Win7-64 machine, (FTDI driver 2.8.30, latency timer 1 ms, 115 k baud rate, IL scope disabled )
linked to ILPer by TCP/IP.
I like the flexibility and expandability of that concept. Result: More than two minutes for copying of a 7k file. :-(

Finally I omitted the ILPILBox and connected ILPer directly to the PIL-Box.
Result: Less than 18 seconds for the same file!

Christoph, how can the additional TCP-IP traffic have such a desastrous impact on the overall performance? Since the pure combination Emu71Win - ILPer, also linked by TCP/IP, is extremely fast, it seems to be a problem with ILPILBox ( Version 1.1 ) ?

[Christoph Giesselink]

(01-03-2014 12:15 AM)Michael Fehlhammer Wrote:  ...
Christoph, how can the additional TCP-IP traffic have such a desastrous impact on the overall performance? Since the pure combination Emu71Win - ILPer, also linked by TCP/IP, is extremely fast, it seems to be a problem with ILPILBox ( Version 1.1 ) ?

TNX for your test. I agree that the problem is caused by ILPilbox.exe. The serial implementation for the PIL-Box hardware inside ILPilbox.exe is different from the implementation inside ILPer.exe because with ILPilbox the PIL-Box hardware itself can operate as controller or as device, whereas with ILPer only as device.

Edit 01/06/14: I will fix the bug soon. Latest tests more or less show the same transfer time like the PIL-Box directly connected to ILPer. The transfer speed with PIL-Box as controller (Emu71/Win as controller operation) is a little bit slower comparing to the PIL-Box as device/listener, but no speed changes to ILPilbox version v1.2. This is a software design issue of the actual ILPilbox program.

[danicaB]

Maybe your PC memory is low. I also experience that so I remove some files.

[Sylvain Cote]

The following shows the time it take to write 50 files to a media.

Setup ...

Code:


========================================================
FTDI  1ms |-> HP-75C -> PIL-Box   -> ILPer (:P1,:M1) ->|
FTDI 10ms |-> HP-75C -> PIL-Box   -> ILPer (:P1,:M1) ->| 
9114B     |-> HP-75C -> HP-82162A -> HP-9114B        ->|
82161A    |-> HP-75C -> HP-82162A -> HP-82161A       ->|
========================================================

Values ...

Code:


--------------------------------------------------------
File:1500B       FTDI=10ms  FTDI=1ms     9114B    82161A
--------------------------------------------------------
Steps              Seconds   Seconds   Seconds   Seconds
--------------------------------------------------------
Initialize           9.805     4.534    76.187   322.643
Copy HP75TD00       18.993     8.027     2.363    22.912
Copy HP75TD01       19.179     8.100     3.516    26.005
Copy HP75TD02       20.430     8.577     2.395    26.765
Copy HP75TD03       21.070     8.966     2.534    27.518
Copy HP75TD04       22.000     9.475     2.569    28.277
Copy HP75TD05       22.943     9.907     2.649    29.019
Copy HP75TD06       23.873    10.331     2.784    30.539
Copy HP75TD07       24.835    10.736     3.078    35.073
Copy HP75TD08       25.775    11.173     3.013    33.013
Copy HP75TD09       26.715    12.561     3.054    33.738
Copy HP75TD10       27.661    11.990     3.232    34.475
Copy HP75TD11       28.588    12.402     3.172    35.229
Copy HP75TD12       29.524    12.797     3.306    35.938
Copy HP75TD13       30.464    13.292     3.403    36.621
Copy HP75TD14       31.389    13.688     3.544    38.113
Copy HP75TD15       32.348    14.115     4.016    42.611
Copy HP75TD16       33.286    14.498     3.856    40.542
Copy HP75TD17       34.225    14.910     3.993    39.600
Copy HP75TD18       35.177    15.285     4.186    41.845
Copy HP75TD19       36.120    15.753     4.223    42.535
Copy HP75TD20       37.059    15.564     4.163    43.228
Copy HP75TD21       38.007    16.613     4.435    43.933
Copy HP75TD22       37.928    15.970     4.476    45.406
Copy HP75TD23       39.884    17.393     4.973    49.903
Copy HP75TD24       40.823    17.882     4.807    47.781
Copy HP75TD25       41.766    18.160     4.848    48.473
Copy HP75TD26       42.707    18.512     5.119    49.155
Copy HP75TD27       43.647    18.808     5.061    49.922
Copy HP75TD28       44.585    19.345     5.196    49.642
Copy HP75TD29       45.764    19.835     5.291    50.259
Copy HP75TD30       46.448    20.135     5.426    52.792
Copy HP75TD31       47.408    20.690     7.108    57.206
Copy HP75TD32       48.352    20.751     5.840    55.069
Copy HP75TD33       49.293    21.653     5.782    55.547
Copy HP75TD34       50.232    21.873     5.975    56.314
Copy HP75TD35       51.171    22.292     6.009    57.041
Copy HP75TD36       52.113    22.575     6.052    57.406
Copy HP75TD37       52.049    23.191     5.225    58.032
Copy HP75TD38       53.974    23.397     6.265    59.503
Copy HP75TD39       54.920    23.946     6.862    90.004
Copy HP75TD40       55.870    24.416     6.695    41.158
Copy HP75TD41       56.810    24.926     6.830    42.051
Copy HP75TD42       57.753    25.219     6.909    42.832
Copy HP75TD43       59.684    25.811     6.843    43.613
Copy HP75TD44       59.644    26.201     6.985    44.388
Copy HP75TD45       60.572    26.490     7.080    45.165
Copy HP75TD46       61.496    26.898     7.214    46.899
Copy HP75TD47       62.457    27.464     7.694    51.445
Copy HP75TD48       63.391    27.667     8.733    49.421
Copy HP75TD49       64.547    28.109     7.669    50.156
--------------------------------------------------------

HP-75C & I/O-ROM code ...

Code:


10 DISP 'HP-75 Mass Memory File Copy Test' @ WAIT 2
20 OPTION BASE 0 @ STANDBY ON ! needed for 82161A at line 170
30 M1$=':M1' ! mass storage name
40 F1$='HP75TDTA' ! source filename
50 F2$='HP75TD' ! base dest filename
60 N1=50 ! number of files
70 DIM T1(50) ! time for each copy, must match var N1
80 T0=0 ! initialize time
90 M$='Initialize ' @ DISP M$&'...'
100 T=TIME @ INITIALIZE M1$ @ T0=TIME-T
110 PRINT M$&'Time:'&STR$(T0)
120 FOR I=0 TO N1-1
130 IF I<10 THEN F$=F2$&'0'&STR$(I) ELSE F$=F2$&STR$(I)
140 M$='File:'&F$&' ' @ DISP M$&'copy ...'
150 T=TIME @ COPY F1$ TO F$&M1$ @ T1(I)=TIME-T
160 PRINT M$&'Time:'&STR$(T1(I))
170 SEND MTA UNL LISTEN 2 DDL 7 UNL ! Rewind tape
180 NEXT I
190 M$='Test done' @ PRINT M$ @ DISP M$

Best regards,

Sylvain

edit: typos

[J-F Garnier]

Thanks, Sylvain, for these tests.

Globally, the PIL-Box/ILPer is about 3-4 times slower than the 9114, for write operations of a small file.

Main limitation is the USB latency, since the HP-IL protocol requires each frame to come back to the sender before the next one can be sent. That is no USB bulk transfer while following the HP-IL protocol.

If anyone has a suggession to improve this, I'm interested!

J-F

[Oulan]

Yes the latency is the main problem.

For HP-IL, the latency of the medium kill speed. I started to make some experiment on go71b for hp-il over wifi (with patched virtual il programs on a pc), but as the message should travel the whole loop before sending the next, I got a throughput of 5 bytes/sec !!

Actually I got some nice usb-serial adapter from FTDI and even usb-usb null modem adapter (it is seen as a serial device at each end with very hight speed).

One day I will try some experiments on go71b to look further in implementing HP-IL protocol but only on usb-usb or usb-serial medium as they support android with complete drivers.

Perhaps an idea to increase the throughput of the sender is to 'fake' the reply from the loop. If you know exactly what devices are on the loop, you can 'forge' the message after its travel on the loop as soon as you send it (if you make the assumption that all devices are ok and work well). But if you pull off the device ....

It can be done with go71b (you can change the behavior of the hp-il stack) or from emu71. This can also be done if the controller is connected directly to a usb/serial virtual il device (you can forge the reply). For PIL-Box, this is quite complicated as you need to rewrite a large part of the driver.

For pil-box, you can use this idea: make a special usb device which isolate the 'virtual loop' from the pil-box. This device forge immediately the reply message and then manage the virtual loop itself. No change are needed on the PIL-Box, but it can be quite hard to do for the device code.

This can be done for 'virtual loops' but for 'real loops' I don't have any ideas, sorry.

Olivier

[Christoph Giesselink]

The entire problem with slow access from a "real HP-75 - PIL-Box - ILPilbox v1.1 - ILPer virtual disc drive" has been fixed in ILPilbox v1.3.

The new version is available at http://hp.giesselink.com/hpil.htm and will increase the interface speed with PIL-Box as listener (with a HP-71B as controller) from ~50 bytes/s to ~375 bytes/s on my computers. With PIL-Box as controller (with Emu71/Win as controller) I got a transfer speed of ~265 bytes/s.

For those who already downloaded the beta version of ILPilbox, replace the your file with the one from the package please, the new one has a minor modification.

[hp41cx]

(01-07-2014 07:14 PM)Christoph Giesselink Wrote:  The entire problem with slow access from a "real HP-75 - PIL-Box - ILPilbox v1.1 - ILPer virtual disc drive" has been fixed in ILPilbox v1.3.

The new version is available at http://hp.giesselink.com/hpil.htm and will increase the interface speed with PIL-Box as listener (with a HP-71B as controller) from ~50 bytes/s to ~375 bytes/s on my computers. With PIL-Box as controller (with Emu71/Win as controller) I got a transfer speed of ~265 bytes/s.

For those who already downloaded the beta version of ILPilbox, replace the your file with the one from the package please, the new one has a minor modification.

Antivirus software detected a virus. Your downloaded file may have a virus, as a result the file you attempted to download was removed by the Windows Attachment Manager.

[Raymond Del Tondo]

(01-07-2014 10:15 PM)hp41cx Wrote:  Antivirus software detected a virus. Your downloaded file may have a virus, as a result the file you attempted to download was removed by the Windows Attachment Manager.

Which file or files are you referring to?
I just downloaded all archives from the hpil page, and my virus scanner did not find a problem with the files.

Did you check your WAM settings?