HP 50g & SD Cards: Performance, Format, Notes

[matthiaspaul]

(07-23-2016 02:05 AM)Claudio L. Wrote:  On SSD disks and other flash media, you can deallocate (and erase) blocks and leave them "ready for other use" (TRIM, etc.), but there's no such thing on SD cards.

FWIW, I just learned that a similar provision exists for MMC/SD cards as well:

https://en.wikipedia.org/wiki/Trim_(computing)#SD/MMC

I haven't used this function myself yet.

Greetings,

Matthias

[JDW]

(07-23-2016 07:00 PM)matthiaspaul Wrote:  FWIW, I just learned that a similar provision exists for MMC/SD cards as well:

https://en.wikipedia.org/wiki/Trim_(computing)#SD/MMC

Such would require the 50g to support TRIM. I highly doubt such is the case.

[matthiaspaul]

(07-23-2016 10:51 PM)JDW Wrote:  

(07-23-2016 07:00 PM)matthiaspaul Wrote:  FWIW, I just learned that a similar provision exists for MMC/SD cards as well:

https://en.wikipedia.org/wiki/Trim_(computing)#SD/MMC

Such would require the 50g to support TRIM. I highly doubt such is the case.

Well, partitioning would be carried out with the card connected to a PC. It would require the partitioning tool to support the MMC/SD ERASE command - I'm not aware of any such partitioning tool, but it could be written.

It would also require an SD card supporting this feature, of course - from what I have read in the net, most cards don't seem to support it, and some cards claiming support implement it as a dummy function only...

Anyway, while this might be interesting food for thought for some, it does not help you with your original questions at all.

Greetings,

Matthias

[JDW]

(07-24-2016 01:38 AM)matthiaspaul Wrote:  partitioning would be carried out with the card connected to a PC. It would require the partitioning tool to support the MMC/SD ERASE command - I'm not aware of any such partitioning tool, but it could be written.

Claudio, how do you partition your SD cards for use in the 50g? Is it possible on a Mac?

[Sylvain Cote]

(07-24-2016 06:27 AM)JDW Wrote:  

(07-24-2016 01:38 AM)matthiaspaul Wrote:  partitioning would be carried out with the card connected to a PC. It would require the partitioning tool to support the MMC/SD ERASE command - I'm not aware of any such partitioning tool, but it could be written.

Claudio, how do you partition your SD cards for use in the 50g? Is it possible on a Mac?

If you look at the beginning of this post (page 1), I gave you the procedure to format your SD card on the Mac.
Because FAT16 is needed, you have to do it from the command line.

Sylvain

[JDW]

(07-24-2016 10:54 AM)Sylvain Cote Wrote:  Because FAT16 is needed, you have to do it from the command line.

Formatting (i.e., using only 1 partition) in FAT16 on the Mac is easy and I've already done that. I was speaking about multiple partitions in light of what Matthiaspaul wrote, which seemed to indicate that some sort of special partitioning was required. Perhaps I merely misread what he was saying.

[Claudio L.]

(07-24-2016 12:05 PM)JDW Wrote:  Formatting (i.e., using only 1 partition) in FAT16 on the Mac is easy and I've already done that. I was speaking about multiple partitions in light of what Matthiaspaul wrote, which seemed to indicate that some sort of special partitioning was required. Perhaps I merely misread what he was saying.

SD cards with multiple partitions will simply need an MBR (Master Boot Record), like any hard disk. I believe the instructions from Sylvain already create an MBR on the Sd card:

Quote:diskutil partitiondisk /dev/disk3 1 MBRFormat "MS-DOS FAT16" "HP50G01" 507M [ENTER]

So all you have to do is use a smaller size on the first partition, then you can add more partitions on the free space.

[JDW]

Thank you for the additional clarification on SD partitioning, Claudio!

[Claudio L.]

One small note to illustrate the effect of the size and number of clusters on startup time:
I just tested newRPL's new SDHC card support with a Transcend SDHC 16 GB, Class 6, which is supposed to sustain 6 MB/s write on a PC.

A single 16 GB partition formatted in FAT32 (with 8kb clusters, total 1958886 clusters, almost 2M) took newRPL some 7 seconds to mount and calculate the free space.
Compare that with my old 128 MB card with a single 32 MB partition with FAT16 (2kb clusters, total 16322 clusters) at 0.5 seconds.
I believe there's an issue with the clock settings that's not letting me run full speed, I'm getting only 750 kb/sec from the SDHC card and 600 kb/sec from the 128 MB card, but that doesn't invalidate the measurements, as they are both comparable to each other.

Once I confirm/fix the clock speeds I'll run a raw speed test to see if "speed doesn't matter".

[JDW]

Today, I tested two old SD cards in my 50g. The first card is 256MB and was purchased in 2003 or 2004 (very old SD card tech, but almost never used throughout the years). When the SD is removed, pressing the ON button displays screen contents almost instantly (no perceivable delay). With the 256MB SD inserted (formatted FAT16, nothing on the card, only the one 256MB partition), it takes about 0.5s for the screen contents to display. Not instantaneous, but perhaps not too bothersome.

I then found another old microSD card that was purchased around 8 years ago, I think. It's a no-name brand, not new nor especially speedy SD tech. This card is 2GB. I used a full size SD card adapter, then formatted it in my iMac to exFAT (which is newer than FAT32) to see what would happen. I inserted the card in the 50g and pressed ON. It takes almost 1 full second for the screen contents to display, in the exFAT format. I then Pressed ON-F4 to bring up the special menu that allows you to test or format SD cards. I choose TEST. The screen displayed testing for a long time (5 minutes, I think), and then in the end it said TEST FAIL" and "PRESS ENTER TO EXIT."

After exiting, I choose Format this time to format the 2GB SD card to FAT16 format inside the 50g. The screen displayed "WAITING……" for about 10 minutes. But it then told me "INITIALIZATION ERROR, FORMAT FAIL, PRESS ENTER TO EXIT." I then turned off the 50g, removed the card, inserted it into my iMac, then confirmed in Disk Utility that it was still formatted exFAT. I clicked Info on the parent item in Disk Utility and noted that the BSD Device Node was disk3. I then opened the Terminal and typed the following to format it FAT32 (naming it "TEST"):

sudo diskutil eraseDisk FAT32 TEST MBRFormat /dev/disk3

It finished formatting a few seconds later and I then put the 2GB SD into the 50g (with the calc off, of course), and then pressed ON. It took a full 3s before the screen would display. (By the way, you cannot substitute "FAT16" for "FAT32" in the above to format as FAT16. You will get an error. See below for Terminal commands needed for FAT16.)

I then pressed ON & F4 and choose to Test the card. It said "TEST OK" after only a half second. So I then choose FORMAT again, to see if the 50g would format this 2GB card as FAT16. But a half second later it said:

WRITE PROTECTED
FORMAT FAIL

Of course, the SD was not write protected at all. So either the 50g doesn't like this card (despite the fact the 50g said the card pass its TEST), or the 50g simply cannot format 2GB cards as FAT16.

I then put the 2GB SD back in the iMac, launched the Terminal and used the following commands to format it FAT16 with name "NONAME":

diskutil partitiondisk /dev/disk3 1 MBR "MS-DOS FAT16" "NONAME" 0B

( More Info: http://pbxbook.com/other/dosfat16.html )

After that, I returned the 2GB SD back to the 50g, and pressed ON. It took just about 1s for the display contents to appear.

My last test on the 2GB card was a partition test. I used the following command set in the Terminal to create a 1st partition of 32MB and a 2nd partition with the remaining space:

diskutil partitionDisk /dev/disk3 MBR "MS-DOS FAT16" "SD32MB" 32M "MS-DOS FAT16" "SD1968MB" 0B

With the card formatted that way in the 50g, the screen contents take the same 0.5s as my 256MB card. Running the TEST program showed no problems.

Whether the screen content display time would take less time with a fast 2GB card, formatted FAT16 and dual-partition is unknown because I don't have a fast 2GB card with modern SD tech to test. But it does show that for older SD tech cards, a smaller card (in terms of MB's) is preferred if you want a fast ON time on the 50g.

[Joe Horn]

(08-08-2016 01:49 AM)JDW Wrote:  Today, I tested two old SD cards in my 50g. ...

Thanks, JDW! Excellent analysis!

[Claudio L.]

(08-08-2016 01:49 AM)JDW Wrote:  With the 256MB SD inserted (formatted FAT16, nothing on the card, only the one 256MB partition), it takes about 0.5s for the screen contents to display.

My last test on the 2GB card was a partition test.
With the card formatted that way in the 50g, the screen contents take the same 0.5s as my 256MB card.

Nice testing. There's one variable you didn't play with, though, and that's cluster size. The default cluster size will try to be as small as possible, which keeps the FAT table as large as possible. If the diskutil follows the MS recommendation in the link, means that:

Your 32 MB partition used 512 byte clusters --> approx 64k clusters in the FAT.
Your 256 MB partition used 4k byte clusters --> approx 64k clusters in the FAT.

This makes sense with your results: in both cases it needs about 0.5 sec to scan the whole FAT, as both FAT tables are almost identical sizes.

It would make sense for you to force for example 2k byte clusters on the 32 MB partition, which would make the FAT 4 times smaller. That is, if you want to reduce those 500 ms to an even smaller value. If you use a 256 MB partition on the 2GB card, it will probably still boot in the same amount of time because the FAT will be the same size too.

[JDW]

(08-08-2016 12:50 PM)Claudio L. Wrote:  It would make sense for you to force for example 2k byte clusters on the 32 MB partition, which would make the FAT 4 times smaller. That is, if you want to reduce those 500 ms to an even smaller value.

I of course want to reduce that 500ms to 0, if possible. And that begs the question. In your newRPL, isn't there a way to avoid checking the SD card at all immediately after pressing ON? Then once the calc is ON, you can then check the SD card "in the background"? Or is this kind of multitasking impossible on the 50g?

The thinking here is that if the calc would simply stop trying to check the card immediately after ON is pressed, card size, speed and even cluster size shouldn't matter (at least, not until you eventually try to access the card).

I don't have my 50g in front of me now, but later I will reformat the 32MB partition on the 2GB SD card and increase the cluster size to 2k, as you suggest, and then see how that impacts startup time. Interestingly, Microsoft says that drives from 16MB to 127MB in size SHOULD have a cluster size of 2k.

Thanks.

[JDW]

Claudio,

I'm sitting here at my iMac with my partitioned 2GB SD card and 50g. In the Terminal, I type the following command to get details on the 32MB SD card partition I created on my 2GB card yesterday:

diskutil info /Volumes/SD32MB

The Terminal then lists the following information:

File System Personality: MS-DOS FAT16
Type (Bundle): msdos
Name (User Visible): MS-DOS (FAT16)

Partition Type: DOS_FAT_16
OS Can Be Installed: No

Total Size: 32.0 MB (32002048 Bytes) (exactly 62504 512-Byte-Units)
Volume Free Space: 31.9 MB (31909888 Bytes) (exactly 62324 512-Byte-Units)
Device Block Size: 512 Bytes
Allocation Block Size: 2048 Bytes

You have been speaking about "clusters" but I think we need to define our terms. From what I understand, "Device Block Size" is something hardware related that I cannot alter, while "Allocation Block Size" is, I think, the "cluster size" you are speaking about. If so, as you can see, it already is at 2k.

Would this mean that it is unlikely we will get an ON-time faster than 500ms? Or will your newRPL eventually yield faster ON times?

Thank you.

[Claudio L.]

(08-09-2016 12:43 PM)JDW Wrote:  If so, as you can see, it already is at 2k.

Would this mean that it is unlikely we will get an ON-time faster than 500ms? Or will your newRPL eventually yield faster ON times?

Thank you.

Yes, it seems the utilities for MAC follow the much older recommendations you linked (funny how the same information from the same source varied over the years). For embedded systems, I like the old specs better, as less clusters means less overhead.
Did you check the cluster size on the other card? If you formatted with the same utility, then it should be at 4kbytes. If that's true, your 256 MB card has a FAT 4 times bigger than the other one, yet still starts in the same amount of time? (head scratching...)

Regarding newRPL, it doesn't suffer from this until you use the card. ON is (and will forever remain) instantaneous, it's a revolutionary design concept once used in devices of the 20th century, that apparently was abandoned by millennial developers :-).
If you've been following the newRPL thread, we already have a compromise solution that will eliminate the mount delay even on large SDHC cards. I'm testing it now, will be out soon.

[JDW]

(08-09-2016 06:37 PM)Claudio L. Wrote:  Did you check the cluster size on the other card? If you formatted with the same utility, then it should be at 4kbytes. If that's true, your 256 MB card has a FAT 4 times bigger than the other one, yet still starts in the same amount of time? (head scratching...)

My other card was the 256MB SD (1 partition of 256MB). Here's the data on that card as per the Terminal:

File System Personality: MS-DOS FAT16
Type (Bundle): msdos
Name (User Visible): MS-DOS (FAT16)

Partition Type: DOS_FAT_16

Total Size: 255.0 MB (255014400 Bytes) (exactly 498075 512-Byte-Units)
Volume Free Space: 254.8 MB (254754816 Bytes) (exactly 497568 512-Byte-Units)
Device Block Size: 512 Bytes
Allocation Block Size: 16384 Bytes

Thoughts?

[Claudio L.]

(08-10-2016 01:46 PM)JDW Wrote:  

(08-09-2016 06:37 PM)Claudio L. Wrote:  Did you check the cluster size on the other card? If you formatted with the same utility, then it should be at 4kbytes. If that's true, your 256 MB card has a FAT 4 times bigger than the other one, yet still starts in the same amount of time? (head scratching...)

My other card was the 256MB SD (1 partition of 256MB). Here's the data on that card as per the Terminal:

File System Personality: MS-DOS FAT16
Type (Bundle): msdos
Name (User Visible): MS-DOS (FAT16)

Partition Type: DOS_FAT_16

Total Size: 255.0 MB (255014400 Bytes) (exactly 498075 512-Byte-Units)
Volume Free Space: 254.8 MB (254754816 Bytes) (exactly 497568 512-Byte-Units)
Device Block Size: 512 Bytes
Allocation Block Size: 16384 Bytes

Thoughts?

32 MB divided by 2k/clusters = 16k clusters
256 MB divided by 16k/clusters = 16k clusters

so my assumption that both took similar time because both FAT tables were about the same size was correct, now the universe makes sense again.

[JDW]

Although we had discussed my two SD cards some time ago, today was the first time I tried to put both to practical use on my 50g. I've run up against problems.

When I use my partitioned card, I see it in File Manager as Port 3, but only the 32MB partition. I don't see the second larger partition. Why not?

When I subsequently select the SD 32MB partition in File Manager and then press the OK/F6 button, I see the following files:

TRASHE~1 (DIR)
FSEVEN~1 (DIR)
SD32MB (STRING)
~1.TRA (STRING)
and so on.

Those are hidden files put on there by MacOS X.

When I down-arrow to select ~1.TRA, then push the NXT button on the calc, and then push the PURGE/F1 key on the calc, I get the "Are You Sure?" confirmation. I press F1 for YES. But the 50g beeps and tells me the following:

Error:
Disk
Protected

But the SD is NOT Locked or otherwise protected. So why can't I delete that file?

I then tried to delete it another way by typing this:

:3:~1.TRA

And I then pressed the TOOL button on the calc, and then I pressed F5/PURGE. The ":3:~.TRA" vanished from the stack with no beep or error, but when I go back into the File Manager, the silly file is still there. But why? I've tried it without quoting the file name, using single quotes, and using double quotes, but to no avail.

Frustrated, I turned off my 50g, removed the partitioned SD card, then put in my 256MB SD card, which has only the one 256MB partition. Same problem on that card too. I cannot delete hidden files. Even worse, when I press F3/FORMA to format the 256MB card, I get the "WARNING" dialog and choose YES. The calc tells me "Please Wait..." for a second, but then beeps with the error:

Error:
Disk
Protected

But again, this SD is not locked or otherwise protected!

So I then put an equation on the stack. I then typed the following:

:3:X

I then pressed the STO button. But the calc beeps at me and says:

STO Error:
Disk
Protected

How do I resolve these problems?

I am able to put content on the SD card using my iMac, and then I am able to read that content off the SD and store that in Port 0 just fine. But the calc is not allowing me to delete files off the SD or format it or STO any file from the calc onto the SD. I don't understand why.

Thank you.

[JDW]

Regarding my previous post (please read it if you haven't already), I tried deleting the hidden files using my 5K iMac today, using the following command in the Terminal, followed by quickly ejecting the SD card:

dot_clean /Volumes/256MB_FAT16

Sadly, it doesn't work. I guess this Mac is so fast it restores the hidden files right after I delete them.

So I then tracked down a Windows8 PC with SD card slot and formatted it FAT16, with a 16k Allocation Block Size. I then put the card back in the 50g and used File Manager to view its contents. I still see a single directory on it named "SYSTEM~1". When I try to "VIEW" that directory on the 50g, I get the following Error:

Error:
Undefined
Name

I cannot PURGE that directory either.

And when I try to "FORMA" the card in File Manager, I get the same error again"

Error:
Disk
Protected

All of these tests I speak of in this post pertain to my 256MB SD Card that has only the single partition (i.e., it's not partitioned).

How do I solve this problem?

[rprosperi]

(08-29-2016 12:44 AM)JDW Wrote:  How do I solve this problem?

You need to remove all the existing partitions and create a new, clean one.

There are simple to follow instructions (for a Windows PC) here.

FYI - "Elevated command prompt" means to right-click on the cmd prompt icon, and select "Run as Administrator". It's equivalent to "su do" on a Mac.

Take care to be certain you are selecting the correct drives, as if you're careless, you could trash your system drive.

Once this is done you can probably format the drive in the 50g.