newRPL: [UPDATED April 27-2017] Firmware for testing available for download

[matthiaspaul]

Trigged by a discussion in another thread

http://www.hpmuseum.org/forum/thread-703...l#pid62643

I'd like to make two little suggestions for newRPL.

1. In that other thread we discussed a bug in the implementation of long filenames in the original HP 49g+/50g firmware, which unfortunately was never fixed and therefore exists even in the latest firmware 2.15/2.16: Under certain conditions the calculator will create SFNs containing lowercase letters, which are invalid characters in the FAT filesystem. While this does not cause any problems for the calculator itself, most other operating systems won't be able to access such files (at least not via the SFN entry). Since file exchange between calculators using the original HP firmware and newRPL is a very common use case, I think, it would be worth the effort to implement a special case into newRPL to cope with such buggy SFNs:

(10-16-2016 04:57 PM)matthiaspaul Wrote:  The underlying technical reason why most operating systems won't be able to access FAT files stored with lowercase SFNs is because they will upcase the filename provided by the user and compare it with the filename stored on disk while assuming that it is stored in upper case already - consequently, SFNs stored in lower case will never match and thus the files are never found.

In many systems the upcase vector for codepoints 00h..7Fh is hardwired assuming an ASCII character set, so characters 61h..7Ah are just translated into 41h..5Ah. However, for codepoints 80h..FFh the upcase translation can differ significantly depending on the active codepage, country settings, and operating system. [...]

Knowing that the 49g+/50g firmware can or could create such invalid SFNs, perhaps newRPL should be enhanced by adding a little special case which would upcase 61h..7Ah in on-disk SFNs before filename comparisons in order to improve compatibility at least for direct file exchange. This would consume only a couple of bytes and not be conflictive with normal file access as these characters cannot normally occur in SFNs.

2. In the other thread we also discussed why it sometimes makes sense to create a LFN entry even if the filename would be short enough to fit into the 8.3 scheme and the only characters above > 127 are uppercase letters:

(10-17-2016 04:47 PM)Claudio L. Wrote:  

(10-16-2016 07:26 PM)matthiaspaul Wrote:  :3:"SPECIÄL" STO no reason to create a LFN entry but some system may do

There's one reason: The character >127. Creating an LFN is the only way to store it as Unicode. I modified newRPL to always create LFN's in cases like this, just to eliminate the ambiguity of using CP translation. Technically not needed if you are willing to put up with old CP mess, though.

I would like to suggest to extend this idea to the four special cases utilizing the two "case"-bits in the SFN discussed previously:

(07-28-2016 02:52 AM)Claudio L. Wrote:  

(07-27-2016 08:56 PM)matthiaspaul Wrote:  Yes, I know, there are also a number of other special cases:
[...]
- If a filename fits into the 8.3 scheme and either contains only lowercase letters or combines a lowercase filename and an uppercase extension or vice versa, the creation of an LFN can be suppressed as well. In this case only an SFN is created and the case information is stored in bits 4 and 3 at offset 0x0C in directory entries, so that the LFN can be recreated from the SFN later on.
[...]

All cases above are implemented exactly as described in newRPL.

IIRC (without looking at the sources again), newRPL maintains these four special cases on reads and writes. That's perfectly fine, but I would like to suggest to still create LFNs on writes if lower-case or special characters are used (or make this configurable on some "special-feature" mount flag). (IIRC this is the default under Linux.)

The reason for this is that many VFAT LFN implementations simply do not support these special bits (probably because this was not part of the original specification, was badly documented and is only supported in the NT line, not by Windows 9x). Without LFNs, this can have the effect that the case of shorter filenames is not always preserved by some platforms which might be used for file exchange. While only a cosmetical issue for many use cases under Windows and DOS, it can cause problems in conjunction with environments distinguishing between names only differing in their case. Now, the FAT filesystem is case-insensitive and newRPL works around such name conflicts using the "semicolon trick", sometimes not preserving the case could still break things, if the names are further processed.

As the code is already there, this looks like an easy change only consuming a few bytes to me. If you really want to be fancy, you could make this a two-stage effort: If (under the described conditions) creating the LFN fails (because there are no free directory entries any more), the system could fall back to only use the "case"-bit method. This way, it would consume additional directory entries only when they are still available.

Greetings,

Matthias