HP 50g & SD Cards: Performance, Format, Notes

[matthiaspaul]

(07-18-2016 04:46 AM)Claudio L. Wrote:  For compatibility reasons, you should buy a modern 2 GB card, just create a 256 MB partition to keep the calculator boot times reasonable (or better off, try different sizes and combinations of partition size/cluster size until you find the best for you). The rest of the card is not going to be unused, as SD cards have internal level-wearing. This means you will use the whole 2 GB of flash, just that your card will last much longer than a smaller card.

This sounds like a great idea at first - but it won't work as excepted, unfortunately, at least not to the extent suggested. It would work as described if it would be possible to completely "deallocate" the unused sectors from external access, but this is impossible except for by reprogramming the internal configuration of the card. [*]

A 2 GB card will "expose" sectors worth 2 GB to the user. Even if the card holds only a 256 MB partition, the unused and possibly uninitialized sectors outside of this occupied area still hold some contents. Even if these sectors contain nothing but random trash (most probably FFh or 00h on a flash card), reading the same sector again somewhen in the future will return exactly the same contents (unless there was a write access in between). The card's firmware can detect that these sectors won't be used as frequently as those inside the 256 MB partition, but it cannot know that nobody would care if the firmware would return fake data instead of the actual contents. Hence, the firmware must treat it as occupied space, which cannot be used as backup area for wear-leveling.
The space used for wear-leveling is extra space (beyond those 2 GB) inside the card, which isn't accessible from the outside.

Nevertheless, the fact that the backup space was dimensioned for a 2 GB card of which now only 256 MB are actually used should still cause the card to last longer than a genuine 256 MB card before all internal backup storage has been used up.

Greetings,

Matthias

PS. [*] Well, there is one exception: For as long as there never was a write-access to a card's sector, the card's firmware could still treat that sector as "unoccupied" and return fake data on reads without consequences. However, even a single write access to a sector would invalidate this forever.
As flash cards come preformatted to their nominal capacity, at least one write will usually happen as part of the production process. Even if the initial formating would skip initializing the free space areas in the filesystem, filling the card once or repartitioning/formatting it would force the card's firmware to treat sectors as occupied afterwards, and therefore no longer available as backup.

[Claudio L.]

(07-18-2016 07:44 PM)matthiaspaul Wrote:  This sounds like a great idea at first - but it won't work as excepted, unfortunately, at least not to the extent suggested. It would work as described if it would be possible to completely "deallocate" the unused sectors from external access, but this is impossible except for by reprogramming the internal configuration of the card.

Matthias

You're right, didn't think that before posting.
Claudio

[JDW]

I'm a bit lost by the last 2 posts.

I think I'll just stick with my ancient 256MB card. Small clusters. Few clusters. No partitioning required. Wear leveling shouldn't matter much seeing I am not going to be write and erasing to the card daily. I was mainly just curious how raw card SPEED affects boot times and overall performance. But again, it would seem my choice of a smaller card is perhaps better than using a modern (and faster) 1GB or 2GB card.

Thanks.

[Joe Horn]

(07-18-2016 04:33 AM)JDW Wrote:  Joe,

If your Pretec SD card marked "60X" performs a backup of 200kb in 1.5 seconds, then please use a slower card (say an old 128MB or so that runs at 2MB/s) to perform the same backup and then tell us how fast it copies. If a 2MB/s card backs up 200kb in 1.5 seconds just like a 15MB/s card, then such confirms there is truly no meaning to use a faster card. And such will also tell us that it would be prudent to use something smaller than 1GB for the sake of smaller and fewer clusters, to further boost card performance.

Good idea... so here it is. The following table supports my claim that card speed DOES matter, HUGELY in fact... but so does the FAT format. FAT32 and/or a slow card will dramatically slow down power-up and storing especially of large objects like backups. As you can see, the CAPACITY of the card doesn't seem to matter much, if at all.

Sorry about the format ambiguity; all I know for sure is that Win7 says that these cards are formatted as "FAT". I'm guessing that means FAT16?

Testing the differences between a nearly-empty card and a nearly-full card, and testing the 50g's speed of reading different cards, is left as an exercise for the reader.

EDIT: My earlier claim of 200K backup in 1.5 seconds was mistaken. It was a 40K backup, not a 200K backup. It's the second card in the table below.

[JDW]

Joe Horn,

Thank you for the detailed comparison! Fascinating to see that CLUSTER SIZE and CLUSTER QUANTITY doesn't matter one little bit. (So much conflicting info in this thread! Ack!)

But of course, we all want to know the single most important detail that you did not specify:

What's speed rating of that INX 1GB card that won the race? :-)

[matthiaspaul]

(07-20-2016 04:53 PM)Joe Horn Wrote:  Sorry about the format ambiguity; all I know for sure is that Win7 says that these cards are formatted as "FAT". I'm guessing that means FAT16?

Basically yes. It actually means: FAT12, FAT16, FAT16B or FAT16X depending on the volume's size and several other conditions. Since FAT12 and the original FAT16 format are only used on very small media (typically smaller 32 MB), it means FAT16B or FAT16X in practise today. FAT16X is only used if LBA-access is mandantory. As this isn't necessary for media sizes of 2 GB, the calculator (and Windows) will use FAT16B normally.

From the file system's perspective, there is no difference between FAT16, FAT16B and FAT16X, they all use 16-bit clusters. FAT16 uses an old-style BPB with 16-bit sector entries and on partitioned media it uses partition type 4, whereas FAT16B uses a "new"-style BPB with 32-bit sector entries (and partition type 6). FAT16X is almost like FAT16B except for that some of the BPB entries are not used and that it uses partition type Eh to enfore LBA-access. Except for when it is important to distinguish between these types, all these variants are called "FAT16" in practise.

When Windows mentions "FAT32", it is either FAT32 or FAT32X. (Windows does not support FAT32+ and FAT32B.) The difference between FAT32 and FAT32X is that some of the BPB entries are not used on FAT32X, and on partitioned media FAT32 uses partition type Bh, whereas FAT32X uses Ch to enfore LBA-access. Except for when it is important to distinguish between FAT32 and FAT32X, they are both called "FAT32" in practise.

Greetings,

Matthias

[Claudio L.]

(07-21-2016 12:07 AM)JDW Wrote:  Joe Horn,

Thank you for the detailed comparison! Fascinating to see that CLUSTER SIZE and CLUSTER QUANTITY doesn't matter one little bit. (So much conflicting info in this thread! Ack!)

But of course, we all want to know the single most important detail that you did not specify:

What's speed rating of that INX 1GB card that won the race? :-)

It's not so conflicting, let's clarify a bit.
What becomes slower with cluster quantity is ON time, the time between pressing the ON key and getting the stack ready to work. This is because during ON, the card free space is calculated, and it needs to read the entire FAT table for that. More clusters means larger table. Also FAT32 tables are twice the size of FAT16 tables with the same number of clusters.
Joe didn't measure ON time unfortunately. Cluster size does affect speed, but the test above only wrote 2 relatively large files, so it doesn't matter. Now if you run a test using many STO commands for small objects, it's one cluster per object. So a 32k cluster is writing 8 times more data than a 4k cluster, therefore it's 8 times faster. In Joe's test, the 40k file would write 64k or 40k, only 60% more data. The 200k file would only be 12% more data. That's why you don't see a big performance hit.

Speed rating of the card doesn't matter once the card is rated above 2MB/s. Only very old cards wouldn't reach that.
Egan did a good SD card performance analysis way back then:

http://sense.net/~egan/hpgcc/

As you can see, "modern" (Egan wrote that probably around 2008/2009, he can correct me if I'm wrong, hence the quotes on modern) 1GB and 2GB cards perform well, same as shown on Joe's tests. Only older cards are bad and also had compatibility problems on many of them.
Bear in mind Egan's tests were done using libfsystem from hpgcc, which is way faster than the 50g OS, so the cards are showing more of their "true" performance. libfsystem clocks the cards faster and uses the high speed mode, so it can theoretically transfer up to 10 MB/s. In practice you'll see cards around the 2MB/s mark (with file system + protocol overheads, copying buffers, etc. you won't get the max speed even if the card can do it).

[Joe Horn]

(07-21-2016 12:07 AM)JDW Wrote:  What's speed rating of that INX 1GB card that won the race? :-)

If I can borrow that card again, I'll measure it.

EDIT: Thanks JDW for finding its speed rating and posting it below.

(07-21-2016 12:57 PM)Claudio L. Wrote:  ... What becomes slower with cluster quantity is ON time, the time between pressing the ON key and getting the stack ready to work. ... Joe didn't measure ON time unfortunately. ...

All the cards take less than a second to turn on, except the 2GB card when formatted FAT32, which takes roughly 2 seconds to turn on. That's just a guesstimate (how can turn-on time be measured accurately?) but that suffices for me to know to use FAT16 and avoid FAT32.

[matthiaspaul]

(07-21-2016 05:07 PM)Joe Horn Wrote:  except the 2GB card when formatted FAT32, which takes roughly 2 seconds to turn on.

This delay is most probably caused by an incomplete FAT32 implementation in the HP 50g.

Many FAT32 implementations for embedded systems ignore, misinterpret, or don't properly maintain the contents of the "FAT32 filesystem info sector", which contains a "free data clusters" entry at offset +1E8h and a "most recently allocated cluster" entry at offset +1ECh. These entries specifically exist to speed up seeks on FAT32 volumes. Misinterpretion (ignoring these values when they are valid or trusting them when they are invalid) can happen if the implementation does not properly handle the "media mount flag" (bit 0 at offset +36h in a FAT32 EBPB) or the "volume mount flag" (bit 27 in the FAT32 cluster 1 entry) during startup (mount), shutdown (unmount), or media change (unmount/mount).

Unfortunately, if that would turn out to be the case with the 50g, we could do nothing to "fix" it...

Greetings,

Matthias

[JDW]

(07-21-2016 12:57 PM)Claudio L. Wrote:  What becomes slower with cluster quantity is ON time, the time between pressing the ON key and getting the stack ready to work. This is because during ON, the card free space is calculated, and it needs to read the entire FAT table for that. More clusters means larger table. Also FAT32 tables are twice the size of FAT16 tables with the same number of clusters. Joe didn't measure ON time unfortunately...

Joe subsequently replied it takes "less than a second" (900ms?) to turn ON with all cards but FAT32. Let's ignore FAT32 and focus on all those other FAT16 cards that take 900ms for the Calc to turn on. I would assume that more than one of those cards has >2MB/s speed ratings. If so, how then does one explain the speed differences Joe posted?

That is my main area of interest. I fully realize the calc is limited to 2MB/s. But if we assume that more than one of Joe's cards has a 10MB/s or even greater rating, how then do we explain the speed differences in Joe's test?

Furthermore, if Joe gets a calc turn ON time of 900ms or so, how much faster is the turn ON time for those of you who use smaller <256MB cards?


Joe, I found out that the fastest card in your test is rated at 10MB/s (p/n: INX-SD1GCN):

http://img1.kakaku.k-img.com/images/prod...010437.jpg

And the 2nd fastest SD card in your test, the Pretec 60x, has a speed rating of 9MB/s.

The 3rd fastest card in your test was the PNY 2GB formatted FAT16, which is rated at 4MB/s:

http://www.bhphotovideo.com/c/product/71...gital.html

The 4th fastest is noticeably slower than the top 3, and is the PNY 512MB. I Googled and Googled but cannot find a speed rating. Check the card's model number and Google it to find the rating. If that card's rating is 2MB/s or higher, then I would like to know why it would be noticeably slower than the first 3, seeing the calc is limited to 2MB/s.

Thanks.

[Claudio L.]

(07-22-2016 12:06 AM)JDW Wrote:  Joe subsequently replied it takes "less than a second" (900ms?) to turn ON with all cards but FAT32. Let's ignore FAT32 and focus on all those other FAT16 cards that take 900ms for the Calc to turn on. I would assume that more than one of those cards has >2MB/s speed ratings. If so, how then does one explain the speed differences Joe posted?

That is my main area of interest. I fully realize the calc is limited to 2MB/s. But if we assume that more than one of Joe's cards has a 10MB/s or even greater rating, how then do we explain the speed differences in Joe's test?

Simple explanation: because of marketing.
When they say 10 MB/s you don't know if it's peak read performance, sustained read performance, sustained write or peak write. Only good manufacturers give you both numbers, and often they cheat too. The PNY card is a class 4, SD Speed class 4 guarantees minimum sustained write speed of 4MB/s (for camcorders), it probably reads around 15 to 20 MB/s. I'd say that's the fastest card in the group.

Joe's test is a write test, so it may not match what you see on the sticker.
If you look at Egan's results, you can see a Lexar card (I think that's a good brand) that reads at 2.5MB/s but writes <300Kb/s. I'm sure they wouldn't put the write speed on the sticker. Sandisk cards are more balanced, so I'd go for their brand first.
I've seen cards rated for 10 MB/s sustained write, but they can only sustain it for 3 or 4 seconds, after that they gradually slow down to 4 MB/s. If you use it on a digital camera, you get 10 MB/s on every picture you take, but if you use it on a camcorder, no joy (learned the hard way). Looking for the SD Speed class symbol is probably your best bet.

On another subject, 900 ms boot is a lot if you use the calc often. I have mine with about half a second, and sometimes I think it's too slow. It has a 64 MB Kingston card.

[JDW]

(07-22-2016 02:32 AM)Claudio L. Wrote:  The PNY card is a class 4, SD Speed class 4 guarantees minimum sustained write speed of 4MB/s (for camcorders), it probably reads around 15 to 20 MB/s. I'd say that's the fastest card in the group.

But that PNY card was only the 3rd fastest in Joe's test.

Based on what you wrote, it seems to make sense to use a much smaller card for faster boot times. However, the smaller the card, the older it likely is, and therefore the slower it likely is. So to get the fastest speed, you'd really want a more modern, faster card with guaranteed minimum speeds. And yet if you buy that, it's impossible now to find a truly modern SD card in sizes less than 1GB. Therein lies the dilemma. What's the perfect balance?

[Claudio L.]

(07-22-2016 02:43 AM)JDW Wrote:  What's the perfect balance?

1GB or 2GB cards cost only $3 or $4 US dollars more than a smaller card. I think the optimum combination is one of those with a 128 MB partition (or try various sizes until you find your own personal "perfect balance"). Think of it like buying a better brand 128 MB card.

[Claudio L.]

(07-22-2016 02:43 AM)JDW Wrote:  But that PNY card was only the 3rd fastest in Joe's test.

True, but the write speed of the 50g OS has many other factors. If the card can write 2MB/s, it should do 40k in 0.02s (did I do the numbers right?), you could triple that if you consider writing to the FAT and updating the directory entry with the new modified date, so let's say 0.06s. You also need to read the entire directory to find conflicting names, add 0.20s for that if you want. The rest of those 1.8sec is the 50g OS overhead.
I wish the same test could be done using SDLIB:

http://www.hpcalc.org/details.php?id=6524

To show how long it takes to write a 40k and 200k file on each card.

[Guenter Schink]

(07-22-2016 02:32 AM)Claudio L. Wrote:  On another subject, 900 ms boot is a lot if you use the calc often. I have mine with about half a second, and sometimes I think it's too slow. It has a 64 MB Kingston card.

I just compared two cards, 512MB from Sigma and 1GB from Toshiba. On the 50G both need about half a second to boot. On newRPL they are there instantly.

Günter

[JDW]

(07-22-2016 06:27 PM)Claudio L. Wrote:  I think the optimum combination is one of those with a 128 MB partition...

But didn't MatthiasPaul argue in his post here that partitioning wouldn't yield what one would expect? He seems to argue against partitioning.

[Claudio L.]

(07-23-2016 12:34 AM)JDW Wrote:  

(07-22-2016 06:27 PM)Claudio L. Wrote:  I think the optimum combination is one of those with a 128 MB partition...

But didn't MatthiasPaul argue in his post here that partitioning wouldn't yield what one would expect? He seems to argue against partitioning.

What he correctly pointed out was that my assumption that because you don't use the rest of the card it will be used for wear leveling was incorrect. On SSD disks and other flash media, you can deallocate (and erase) blocks and leave them "ready for other use" (TRIM, etc.), but there's no such thing on SD cards. A sector will always be assumed to contain data and won't be reused.
This doesn't mean a smaller partition won't make your calculator boot faster, it means my claim that the card was going to last longer was wrong. It will last roughly the same (perhaps slightly more for the reasons Matthias explained).

[Claudio L.]

(07-22-2016 10:11 PM)Guenter Schink Wrote:  

(07-22-2016 02:32 AM)Claudio L. Wrote:  On another subject, 900 ms boot is a lot if you use the calc often. I have mine with about half a second, and sometimes I think it's too slow. It has a 64 MB Kingston card.

I just compared two cards, 512MB from Sigma and 1GB from Toshiba. On the 50G both need about half a second to boot. On newRPL they are there instantly.

Günter

Well, because that delay always annoyed me, I specifically avoid mounting the card during ON. The drawback is that the card will be mounted on first use, which will make the first operation a few milliseconds slower (doesn't take half a second either, mounting it's much faster on newRPL).

[JDW]

What is "newRPL"?

With regards to partitioning, do you partition on the calc or on your PC? Is there a special way we must partition it?

[Sylvain Cote]

(07-23-2016 02:20 AM)JDW Wrote:  What is "newRPL"?

The NewRPL thread is located HERE

Sylvain