PIL-Box throughput

[Egan Ford]

Hi, I've seen other threads on this, but didn't find any with version 1.5 of the firmware. What 71B/PIL-Box rates are you all seeing with large files (~10K) and firmware 1.5?

Thanks.

[Dave Frederickson]

32K Write to ILPer using ROMCOPY took 1:08 or ~480 bytes/sec
32K Read from ILPer using ROMCOPY took 34 seconds or ~960 bytes/sec

Perhaps the Write also did a Verify.

Dave

[Christoph Giesselink]

(07-13-2015 03:12 AM)Dave Frederickson Wrote:  Perhaps the Write also did a Verify.

Definitely not. :)

The reason is the LIF format itself. The LIF format does not have a File Allocation Table (FAT) like other disc formats, so the free sectors must be extracted from the directory structure by reading all directory entries. This is the additional time spend at writing.

You can make a try, just measure the time writing a file to a quite full LIF image with many directory entries and make another try writing the same file to a newly created empty LIF image. You will get a significant difference, especially on small files.

When optimizing the ILPilbox.exe bridge software JFG gave me the tip for the throughput write measurements to write the data from the HP71 to a non-existing device. For the loop HP71-Pilbox-ILPer:

A=TIME @ COPY file TO :3 @ TIME-A

will do the job. Further, the performance of the ILPer host has also a big influence on the file throughput. I made tests with an old P4/3.2GHz and an 8 core Z600 workstation (2.66GHz nominal, 3.06GHz Turbo mode). The Z600 workstation was over 60% faster.

[Dave Frederickson]

(07-13-2015 11:01 PM)Christoph Giesselink Wrote:  

(07-13-2015 03:12 AM)Dave Frederickson Wrote:  Perhaps the Write also did a Verify.

Definitely not.

The reason is the LIF format itself. The LIF format does not have a File Allocation Table (FAT) like other disc formats, so the free sectors must be extracted from the directory structure by reading all directory entries. This is the additional time spend at writing.

You can make a try, just measure the time writing a file to a quite full LIF image with many directory entries and make another try writing the same file to a newly created empty LIF image. You will get a significant difference, especially on small files.

When optimizing the ILPilbox.exe bridge software JFG gave me the tip for the throughput write measurements to write the data from and to the HP71 itself. For the loop HP71-Pilbox-ILPer:

A=TIME @ COPY file TO :3 @ TIME-A

will do the job. Further, the performance of the ILPer host has also a big influence on the file throughput. I made tests with an old P4/3.2GHz and an 8 core Z600 workstation (2.66GHz nominal, 3.06GHz Turbo mode). The Z600 workstation was over 60% faster.

Hi Christoph,

I'm aware of the directory entries issue and that's why I INITIALIZED the image before executing ROMCOPY. Secondly, I COPYied a 13K file to ILPer and I got around 860 bytes/sec. Thirdly, if I omit the ;ROMSIZE=32768 then ROMCOPY fails after 34 seconds with "VERIFY FAILED". This seems to support my suspicion that ROMCOPY does a Verify after Writing. Did you notice that the Write time is exactly twice the Read time?

What rates do you measure?

Dave

[J-F Garnier]

(07-13-2015 11:01 PM)Christoph Giesselink Wrote:  A=TIME @ COPY file TO :3 @ TIME-A

Note that the <<:3>> address is not a typo; COPYing to a non-existing device makes the results more representative of the PIL-Box firmware and ILPer frame processing performance, independently of disc/file access, catalog search time, etc...
This is the correct way to estimate the PIL-Box/ILPer (or PIL-Box/Arduino) throughput.

(07-13-2015 11:47 PM)Dave Frederickson Wrote:  What rates do you measure?

The best I got is about 1kB/s. The main limitation is the USB latency (that's why the Arduino-SDdisc throughput is higher).
I'm still exploring new ways to improve the transfer rate, but I don't expect huge improvements.
1kB/s is acceptable for the HP-71B, but is still a bit slow for the HP Portable Plus.

J-F

Additional note: to evaluate the PIL-Box firmware performance only, stop ILPer. Frames will no more be transmitted over serial/USB and the throughput is then much higher :-)

[Dave Frederickson]

Thanks for that informantion, J-F. It looks like the highest performance is going to come from the embedded PIL-Box efforts and Tobie has posted some impressive numbers.

For what I do I'm a big fan of the virtual peripherals. The worst-case example I've experienced would be loading a SYSROM image into FRAM71 as it takes something like 40 minutes. I'd love to speed that up.

Dave

[Christoph Giesselink]

Hi Dave,

Just checked the behavior of ROMCOPY by enabling the Scope in ILPer and executing the command:

ROMCOPY :PORT(1) TO RCPYLEX:HP ;ROMSIZE=32768

Result:

• the data bytes are transferred only once
• the last commands before the data bytes are LAD 02 UNT
  

-> the disc simulation is listener at data transmission, so no verify read

Quote:What rates do you measure?

7533 bytes/sec. ;-)

To be honest, this is the transfer rate of Emu71/Win v1.07beta7 (in Authentic Speed mode) to ILPer v1.51 over IPv6.

I don't have ROM modules or the FRAM71 for the HP71, so it would take some time, which I don't have, to prepare a 32kB RAM module as ROM to make the measurements on a real machine. And as I stated before, the tests should be repeated with several ILPer hosts, checking the influence of the host performance on the transfer results...

[Dave Frederickson]

(07-14-2015 05:09 PM)Christoph Giesselink Wrote:  I don't have ROM modules or the FRAM71 for the HP71, so it would take some time, which I don't have, to prepare a 32kB RAM module as ROM to make the measurements on a real machine. And as I stated before, the tests should be repeated with several ILPer hosts, checking the influence of the host performance on the transfer results...

I ROMCOPY'd an IRAM to ILPer and back, so neither ROM modules nor FRAM71 were used to conduct the test.

Dave

[rprosperi]

(07-14-2015 05:09 PM)Christoph Giesselink Wrote:  Just checked the behavior of ROMCOPY by enabling the Scope in ILPer and executing the command...

Interesting thread guys, thanks for these insights.

Christoph - I thought running ILPer with Scope mode enabled drastically slowed performance, and I have also seen this for some activities, e.g. CATalogs, etc. I don't recall timing Scope on vs. off for a large file transfer. Did I misunderstand and/or is this true only for some types of transactions/transfers?

HP-71B PIL-Box users will like this: (OT Warning)
"Remote computing HP-71 style". I measured the distance earlier today and if I can get another set of 5M HP-IL cables, I can connect my 71B I use in my recliner in the den (I'm recovering from some procedures to my back) to my PIL_box on my desktop PC in my office. I use a ThinkPad to run a remote terminal session from the desktop. Sure, Bluetooth has advantages (and is probably cheaper!) but doing it old-school has a certain satisfaction.

Until then, I use Emu/71 and ILPILBox/IL-Per all virtual, but I like the real devices.

[Christoph Giesselink]

(07-14-2015 11:53 PM)rprosperi Wrote:  Christoph - I thought running ILPer with Scope mode enabled drastically slowed performance, and I have also seen this for some activities, e.g. CATalogs, etc. I don't recall timing Scope on vs. off for a large file transfer. Did I misunderstand and/or is this true only for some types of transactions/transfers?

Yes, you're right. To make it clear, at first step I enabled the Scope and made the transfer with ROMCOPY to have a look at the transfer data and at second step I disabled the Scope and made the transfer with ROMCOPY to measure the transfer speed. The text output in ILPer on Windows is very slow comparing to the data transfer rate, so you're right to disable the Scope.

I'm not a typical user of ILPer or of HP-IL in general. I'm not interesting in using the software, I like developing the software and understand the mechanisms behind. In this special case into the HP-IL data protocol, so the Scope in ILPer is most time switched on for my using.

[Dave Frederickson]

(07-14-2015 05:09 PM)Christoph Giesselink Wrote:  Just checked the behavior of ROMCOPY by enabling the Scope in ILPer and executing the command:

ROMCOPY :PORT(1) TO RCPYLEX:HP ;ROMSIZE=32768

Result:

• the data bytes are transferred only once
• the last commands before the data bytes are LAD 02 UNT
  

-> the disc simulation is listener at data transmission, so no verify read

I'm still puzzled by this behavior. I turned on the Scope prior to executing the ROMCOPY command and was surprised that it was silent for around half a minute, then started spitting out data bytes.

I suspect what ROMCOPY is doing during Writes is calculating the checksums first, hence the quiet HP-IL bus, and that accounts for the difference between reading and writing.

Dave

[Christoph Giesselink]

Here we are.

My configuration:

• HP-71B rev. 1BBBB
• HP-82401A rev. 1B
• CMT 32KB RAM Module in Slot 1
• Pilbox Firmware v1.5
• HP Z600 2*XEON X5550, Win7 x64
• FTDI driver v2.10.0.0
• ILPer v1.51 (compiled with VS2005) (DEVID$(":2") = HP9114B; 80 tracks, 2 sides, 16 sectors)
  

Environment:

• ROMCOPY rev. E
• Transferred data: image of the MATHROM 32KB
• Source LIF file located on a hard disc holding the image (RCPYLEX)
• Target LIF file located on a hard disc just initialized with "INITIALIZED :HP"
  

ROMCOPY RCPYLEX:HP TO :PORT(1);ROMSIZE=32768

-> 35s -> 32768bytes / 35s = 936 bytes/s

ROMCOPY :PORT(1) TO RCPYLEX:HP;ROMSIZE=32768

-> 37s -> 32768bytes / 37s = 886 bytes/s


Just to be sure, but now with a HP-82401A rev. 1A HP-IL module:

ROMCOPY RCPYLEX:HP TO :PORT(1);ROMSIZE=32768

-> 34s -> 32768bytes / 34s = 946 bytes/s

ROMCOPY :PORT(1) TO RCPYLEX:HP;ROMSIZE=32768

-> 36s -> 32768bytes / 36s = 910 bytes/s

Conclusion: no significant difference between read & write and HP-82401A rev. 1A and 1B

Christoph