libhpcalcs: portable (Windows / MacOS X / Linux) connectivity kit library

[Egan Ford]

(12-26-2013 08:17 AM)debrouxl Wrote:  Which caused you to add -std=c99 yourself
Since pretty much everyone using autoreconf/configure/make directly hits the problem, I'm adding AC_PROG_CC_C99 to configure.ac, and AC_MSG_ERROR(...) if ac_cv_prog_cc_c99 is "no", as described in the autoconf manual. Will commit after more testing, if successful.

C99 is only required because of a few lines of code in typesprime.c, e.g.:

Code:


    uint32_t i;
    for (i = 0; i < sizeof(PRIME_CONST)/sizeof(PRIME_CONST[0]) - 1; i++) {
    //for (uint32_t i = 0; i < sizeof(PRIME_CONST)/sizeof(PRIME_CONST[0]) - 1; i++) {

The commented out code is the orig, and the two lines above my changes. IOW, for standard ANSI C the variable declaration needs to be outside of the for block. I'd recommend changing the code and avoid the requirement for C99.

BTW, I read elsewhere that some where having issues with GCC 4.2 included with OS/X CLI tools. I am not having any issues.

[debrouxl]

Quote:I'd recommend changing the code and avoid the requirement for C99.

Besides the soft requirement created by the for loops that you mentioned (and which could easily be changed), C99 is a hard requirement for libhpcalcs, due to stdint.h+inttypes.h.
For compatibility with MSVC < 2010 versions, the members of the libti* family use their own stdint[1234].h, either pointing to the system one, or embedding typedefs suitable for MSVC. However, for a code base started 14 years after the advent of C99, I chose to get rid of this historical artifact
In fact, at first, the libhpcalcs code even required C11 (because I relied on typedefs from uchar.h), but despite the standard being two years old, it soon appeared that it's too early to make C11 a hard requirement without inconveniencing users (especially MacOS X users, it worked on Linux).

Quote:BTW, I read elsewhere that some where having issues with GCC 4.2 included with OS/X CLI tools. I am not having any issues.

Indeed, Apple's GCC 4.2.1, based on the last GPLv2 version of GCC, mostly works, but it's severely outdated compared to GCC 4.8 (4.9 should be released next spring) or Clang 3.4 (currently in the final release candidate phase).


I'll push the commits later, but in my local tree, among other changes, configure.ac now contains

Code:

AC_PROG_CC_C99

if test "x$ac_cv_prog_cc_c99" = "xno"; then
  AC_MSG_ERROR([libhpcalcs requires a C99 compiler])
fi

Probing code is mostly complete, needs some testing.

[debrouxl]

I have pushed 6 commits, 3 on the stable "master" branch and 3 additional ones on the testing "master2" branch.
Besides fixes for the aforementioned bugs (and more), there are two new major features (both on the master2 branch, as the code is not final, and I don't have a Prime to test):
* probing support, i.e. the ability to detect which cable and calculator types are available;
* initial support for the Prime emulator (buggy on exit, I need help with the Win32 API, in the area of named pipes and asynchronous I/O), and to a lesser extent, the 39gII emulator (last tested before I implemented probing support)

Windows users will find the latter capability moderately useful, Linux & MacOS X users could find it more useful - maybe the most direct use for that feature in the short term shall be my own testing for changes to the upper layers from Linux, through Wine.
In fact, the "send chat" and "recv chat" operations were implemented thanks to an earlier version of the emulator support, before I experienced significant trouble making large packet support work (on the reading side).
Due to my inability to close / flush the connection correctly, for now, the emulator needs to be restarted after test_hpcalcs exits.

I have updated the Windows binaries (see http://www.hpmuseum.org/forum/thread-52-...tml#pid176 for download links) with the probing and emulator support.
Beware that I might have broken support for real Prime calculators in the process, so if you want a dependable program, stick to the master branch and an earlier version of the Windows binaries

As usual, bug reports and code contributions from Windows developers are more than welcome.
Thanks in advance

[LarsF]

I found another problem, if I try to receive a file that is bigger then I got this error,

hpcalcs DEBUG: Dumping IN packet with size 64
hpcalcs DEBUG: 00 74 00 6F 00 20 00 69 00 74 00 73 00 20 00 73
hpcalcs DEBUG: 00 75 00 69 00 74 00 20 00 6F 00 6E 00 20 00 61
hpcalcs DEBUG: 00 22 00 2C 00 0A 00 20 00 20 00 22 00 64 00 65
hpcalcs DEBUG: 00 73 00 74 00 69 00 6E 00 61 00 74 00 69 00 6F
hpcables INFO: cable_prime_hid_recv: read 0bytes
hpcalcs INFO: prime_recv_data: breaking due to short packet (1)
hpcalcs WARN: prime_recv_data: expected 18672 bytes but only got 1575 bytes, output corrupted
hpcalcs DEBUG: read_vtl_pkt: command matches returned data
hpcalcs INFO: calc_prime_r_recv_file: embedded=687E computed=86A6
hpcalcs ERROR: calc_prime_r_recv_file: CRC mismatch
hpcalcs_calc_recv_file finished
Receive file success

That seems to happen on all files that are bigger than maybe 2KB.
I have used the latest code from git.
Is there anybody else that got this error on bigger files? I run on a Mac

[debrouxl]

* is the code built from the master branch, or from the master2 branch ?
* do screenshots (especially type 8, the largest one, which creates ~18-20 KB of data when the calculator is on the Apps screen) work ?
* could you use "tee" to write terminal output to both standard output and a file, then post / e-mail me the result ? I'm most interested in the dump of the first bytes when receiving a file, F7 01 xx xx xx xx <and then some more>.

Thanks in advance

[LarsF]

It's build from master,
This is when try to get a screenshot type 8

hpcalcs INFO: prime_send_data: q:0 r:3
hpcalcs DEBUG: Dumping OUT packet with size 4
hpcalcs DEBUG: 00 00 FC 08
hpcables INFO: cable_prime_hid_send: wrote 0 bytes
hpcalcs INFO: prime_send: send succeeded
hpcalcs INFO: prime_send_data: send remaining succeeded
hpcables INFO: cable_prime_hid_recv: read 64bytes
hpcalcs DEBUG: Dumping IN packet with size 64
hpcalcs DEBUG: FF 00 00 00 F4 22 73 31 20 00 0E 00 03 00 00 00
hpcalcs DEBUG: 00 00 02 00 00 00 00 00 60 00 27 31 00 00 00 00
hpcalcs DEBUG: 00 00 00 00 00 00 00 00 20 00 00 00 00 00 00 00
hpcalcs DEBUG: 00 00 00 00 00 00 00 00 78 DC 69 31 44 13 6A 31
hpcalcs ERROR: prime_recv_data: skipping packet starting with 0xFF
hpcables INFO: cable_prime_hid_recv: read 0bytes
hpcalcs INFO: prime_recv_data: breaking due to short packet (1)
hpcalcs INFO: prime_recv_data: shortening packet from 0 to 0
hpcalcs INFO: read_vtl_pkt: empty packet
hpcalcs INFO: calc_prime_r_recv_screen: packet is too short: 0bytes
hpcalcs ERROR: calc_prime_recv_screen: r_recv_screen failed
hpcalcs ERROR: hpcalcs_calc_recv_screen: recv_screen failed
hpcalcs_calc_recv_screen failed
hpcalcs INFO: Unhandled packet format

[debrouxl]

Ouch, that does not even remotely look like the reply which the calculator should return upon receiving FC 08... Perhaps something left from a previous transfer ?

The two recv_file traces which you posted / sent by e-mail (thanks !) contain:

Code:

hpcalcs WARN: prime_recv_data: expected 18672 bytes but only got 1575 bytes, output corrupted
hpcalcs WARN: prime_recv_data: expected 18672 bytes but only got 2961 bytes, output corrupted

libhpcalcs receives only a small subset of the data in the first place, and therefore, it cannot make any sense of it

I've already witnessed severe packet loss ratio on libhpcalcs once. That was on Windows, when outputting all debugging output to the creaky slow Windows terminal. I had to redirect the output to a file for the packet loss ratio to become essentially zero.
Let's try the same for MacOS X
In tests/test_hpcalcs.c, function output_log_callback()
replace

Code:

#ifndef _WIN32

by

Code:

#if 0

Thanks in advance

[LarsF]

I'm sorry, but that did not help, got the same error. It seem to be a little random when it stops

hpcalcs DEBUG: Dumping IN packet with size 64
hpcalcs DEBUG: 00 00 55 00 53 00 45 00 29 00 3D 00 22 00 7B 00
hpcalcs DEBUG: 7B 00 7D 00 2C 00 7B 00 7D 00 7D 00 22 00 3B 00
hpcalcs DEBUG: 0A 00 45 00 4E 00 44 00 3B 00 0A 00 0A 00 57 00
hpcalcs DEBUG: 61 00 69 00 74 00 46 00 6F 00 72 00 54 00 61 00
hpcables INFO: cable_prime_hid_recv: read 0bytes
hpcalcs INFO: prime_recv_data: breaking due to short packet (1)
hpcalcs WARN: prime_recv_data: expected 18672 bytes but only got 882 bytes, output corrupted
hpcalcs DEBUG: read_vtl_pkt: command matches returned data
hpcalcs INFO: calc_prime_r_recv_file: embedded=687E computed=A5E3
hpcalcs ERROR: calc_prime_r_recv_file: CRC mismatch

[debrouxl]

Yeah, the packet loss ratio (before the packets even reach libhpcalcs) seems random, but remains very high...

A quick search, using several queries, doesn't turn up USB packet capture software running on MacOS X. On Linux, there's Wireshark, and on Windows, there's a wealth of tools, both proprietary and free software - but nothing obvious for MacOS X.

[Egan Ford]

(12-28-2013 11:13 AM)LarsF Wrote:  I'm sorry, but that did not help, got the same error. It seem to be a little random when it stops

hpcalcs DEBUG: Dumping IN packet with size 64
hpcalcs DEBUG: 00 00 55 00 53 00 45 00 29 00 3D 00 22 00 7B 00
hpcalcs DEBUG: 7B 00 7D 00 2C 00 7B 00 7D 00 7D 00 22 00 3B 00
hpcalcs DEBUG: 0A 00 45 00 4E 00 44 00 3B 00 0A 00 0A 00 57 00
hpcalcs DEBUG: 61 00 69 00 74 00 46 00 6F 00 72 00 54 00 61 00
hpcables INFO: cable_prime_hid_recv: read 0bytes
hpcalcs INFO: prime_recv_data: breaking due to short packet (1)
hpcalcs WARN: prime_recv_data: expected 18672 bytes but only got 882 bytes, output corrupted
hpcalcs DEBUG: read_vtl_pkt: command matches returned data
hpcalcs INFO: calc_prime_r_recv_file: embedded=687E computed=A5E3
hpcalcs ERROR: calc_prime_r_recv_file: CRC mismatch

Hi, I am unable to reproduce this error with Prime Build 5447 and OS/X 10.8.5 (HP Cable -> USB extension cable -> ThinkGeek USB 2.0 hub with red switches per port -> USB 2.0 hub (2010 27" Apple Cinema Display) -> USB 3.0 2013 MacBook Air). I've tried several times and I always get the screen shot (colors are off in 16-bit mode and anti aliasing looks like garbage, but its consistent and repeatable (you must disable the clock to get the md5sums to match :-)).

I used the App screen for testing (18K files).

Build notes (12/27/13):

Code:


sudo port install fox

git clone https://github.com/signal11/hidapi.git hidapi
cd hidapi
./bootstrap
./configure --prefix=/usr/local
make
sudo make install
cd ..

git clone https://github.com/debrouxl/hplp
cd hplp/libhpcalcs
autoreconf -i -f
HIDAPI_LIBS="-L/usr/local/lib -lhidapi" HIDAPI_CFLAGS="-I/usr/local/include/hidapi" ./configure --prefix=/usr/local
make

My binaries (http://sense.net/~egan/prime/prime_osx_tools.tgz) if you want to test:

Code:


mkdir primetools
cd primetools
tar zxvf ~/Downloads/prime_osx_tools.tgz
./test_hpcalcs.sh

[Egan Ford]

(12-28-2013 03:06 PM)debrouxl Wrote:  Yeah, the packet loss ratio (before the packets even reach libhpcalcs) seems random, but remains very high...

Just a guess, but is Parallels/VMware running and also attached to the calculator?

[LarsF]

There is wireshark for OSX to but I don't know if it capture from USB, I run OSX 10.9.1 and connect the Prime direct to the computer, I have a Macbook Pro. Egan, I'll try your binaries tomorrow, I did not build the hidapi myself, used brew to install it.

[Egan Ford]

(12-28-2013 06:10 PM)LarsF Wrote:  connect the Prime direct to the computer

I directly connected to my Air USB 3.0 port without a hub and did 10 screen dumps. The xfer is much faster, and I do get all the data, and it is consistent and repeatable. However I cannot compare to the USB 2.0 screenshots because the screen image is different with the USB 3.0 high amp ports (battery is charging). I guess with plain 'ol low amp USB you cannot charge the Prime.

BTW on any USB port I always get the following error:

Code:


hpcalcs INFO: prime_recv_data: breaking because the expected size was reached (2)
hpcalcs INFO: prime_recv_data: shortening packet from 18774 to 18728
hpcalcs DEBUG: read_vtl_pkt: command matches returned data
hpcalcs INFO: calc_prime_r_recv_screen: embedded=1A computed=AAA
hpcalcs ERROR: calc_prime_r_recv_screen: CRC mismatch
hpcalcs INFO: hpcalcs_calc_recv_screen: recv_screen succeeded

But I always get the data, and it's clean. The number of CRC ERRORs is always just the one at the end.

Try my binaries. I'll upgrade to 10.9.1 around the 31st and retest.

[Egan Ford]

Feature request.

test_hpcalcs send key to support a comma delimited array of keys.

Thanks.

[debrouxl]

Thanks for the reminder. I had already thought of that feature request but forgot to add an item to the TODO list

I think that there are two steps in fulfilling that feature request:
Step 1: base functionality:
1) adding support for a "send_keys" operation, hpcalcs_calc_send_keys() and backends;
2) in test_hpcalcs, using strtok (once for initialization, then in a loop until NULL is returned), then for each token, strtoul with base 0 on each token (breaking upon error and upon invalid key code) to incrementally build an array of key codes, before passing that array to hpcalcs_calc_send_keys();

Level 2: refinements taking more time and left for future work:
1) adding a header file containing 4 * 51 definitions for the key codes (will take a bit of effort);
2) add an array of structs containing at least the key ID and the key name, stringified from the definition.
3) add a sorting functions by string name, so that binary search can be used;
4) add a key searching function, which parses the input string using strtoul with base 0, then if that fails, searches by string name into the sorted array, and returns a key ID or error;
5) use the key searching function in test_hpcalcs.

[LarsF]

I have some info about this, I tried to compile it all like Egan suggested but with the same result, then a connected the calculator to my monitor instead of direct to the computer, then I was able to receive files without errors :-). But screenshot give me errors still.

[debrouxl]

Quote:then a connected the calculator to my monitor instead of direct to the computer, then I was able to receive files without errors :-).

Good
So your issue is definitely outside libhpcalcs' control, at a deeper layer.

Quote:But screenshot give me errors still.

Just CRC errors, or more severe errors which prevent successful screenshot receive and therefore prevent the creation of the output file ?

[debrouxl]

6 commits pushed to the master branch this morning, after more testing I couldn't finish yesterday evening.
* 4 cleanups / improvements split aside from the big emulator support patch;
* support for sending multiple keypresses in a single operation. "122 123,124 125 126,127 128 " is a valid input for the new entry in test_hpcalcs, it posts seven consecutive letters to the calculator;
* logging improvements in test_hpcalcs. I dusted off some older unfinished work I had temporarily set aside.

The master2 branch, with the aforementioned probe and incomplete emulator support, was rebased on master.

[LarsF]

Today when testing then receive file working perfect when connecting direct to the computer! And the screenshot function almosts works :-)
It seems to get all data, but I get a CRC error

hpcalcs DEBUG: Dumping IN packet with size 64
hpcalcs DEBUG: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
hpcalcs DEBUG: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
hpcalcs DEBUG: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
hpcalcs DEBUG: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
hpcalcs INFO: prime_recv_data: breaking because the expected size was reached (2)
hpcalcs INFO: prime_recv_data: shortening packet from 199395 to 199395
hpcalcs DEBUG: read_vtl_pkt: command matches returned data
hpcalcs INFO: calc_prime_r_recv_screen: embedded=8972 computed=913B
hpcalcs ERROR: calc_prime_r_recv_screen: CRC mismatch
hpcalcs WARN: calc_prime_r_recv_screen: unknown marker at beginning of image
hpcalcs ERROR: calc_prime_recv_screen: r_recv_screen failed
hpcalcs ERROR: hpcalcs_calc_recv_screen: recv_screen failed
hpcalcs_calc_recv_screen failed
hpcalcs INFO: Unhandled packet format

388 Unhandled packet format

[LarsF]

I did get all your changes first from github and recompile it before testing.