VFAT: slow fs corruption? [long]

VFAT: slow fs corruption? [long]

[Albrecht Dreß]

[-- Attachment #1: Type: text/plain, Size: 2180 bytes --]

Hi all,

first, please excuse me if this is a very dump question...

I use Linux (inter alia) on an ARM9 system which is attached to a  
measurement device.  The device produces a new data set of ~10 kByte about  
every 20 seconds, and the ARM system stores the data on a 1GB SD card  
attached to it.  The kernel is unfortunately rather old - 2.6.11,  
including several vendor (Digi) supplied drivers.  I didn't have time yet  
to port them all to a recent kernel.

My first approach was to use the SD card with a VFAT file system on it, as  
such cards usually come pre-formatted with it and as the VFAT cluster size  
of 16k matches the erase block size of the card, which should help its  
built-in wear leveller to do its job.

After some time, though, I noticed that several files with measurement  
data were empty, although the directory indicated that they were not.  The  
system was not switched off, and I always call sync() after producing a  
new file.  Running dosfsck on the file system, I got several errors like

   File size is 13978 bytes, cluster chain length is 0 bytes.
   Truncating file to 0 bytes.

or

Cluster 0 out of range (4849729 > 247450). Setting to EOF.
Orphaned long file name part "5d"

My first idea was that the (Digi supplied) MMC driver is somewhat broken.   
Just to check that, I re-formatted the SD card with ext2, and ran the test  
again - and all files looked fine!  Called "e2fsck -f" - no problems with  
the file system.  That seems to indicate that the MMC layer /does/  
actually work.

Questions:
- Is the conclusion correct that the problem seems to be in the VFAT part,  
not in the MMC layer?
- Are there known issues with VFAT in 2.6.11 which might lead to the  
observed problems?  Were they fixed?
- Is it possible to change the block size in ext2 to 16k (to match the SD  
card's erase block size)?

Thanks in advance for any help,
Cheers,
Albrecht.


--
  Albrecht Dreß  -  Johanna-Kirchner-Straße 13  -  D-53123 Bonn (Germany)
        Phone (+49) 228 6199571  -  mailto:albrecht.dress@arcor.de
   GnuPG public key:  http://www.mynetcologne.de/~nc-dreszal/pubkey.asc

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: VFAT: slow fs corruption? [long]

[Benjamin LaHaise]

On Wed, Apr 18, 2007 at 07:58:40PM +0200, Albrecht Dreß wrote:
> - Are there known issues with VFAT in 2.6.11 which might lead to the  
> observed problems?  Were they fixed?
> - Is it possible to change the block size in ext2 to 16k (to match the SD  
> card's erase block size)?

Flash cards tend to be rather flaky given that they are low cost consumer 
grade commodities these days.  I would recommend getting a new card first 
and seeing if you can still replicate the problem.  Out of the box, I've 
had to replace 2 of 8 flash cards in the last 6 months when they showed 
similiarly eerie data corruption when files disappeared.  Doing an md5sum 
on the device 2 times in a row and getting back different results is Not 
Good.

		-ben
-- 
"Time is of no importance, Mr. President, only life is important."
Don't Email: <zyntrop@kvack.org>.
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VFAT: slow fs corruption? [long]

[Albrecht Dreß]

[-- Attachment #1: Type: text/plain, Size: 1870 bytes --]

Hi Ben!

Thanks a lot for your comments, and sorry for the late reply, I did more tests  
in the meantime...

Am 19.04.07 01:00 schrieb(en) Benjamin LaHaise:
> On Wed, Apr 18, 2007 at 07:58:40PM +0200, Albrecht Dreß wrote:
> > - Are there known issues with VFAT in 2.6.11 which might lead to the >  
> observed problems?  Were they fixed?
> > - Is it possible to change the block size in ext2 to 16k (to match the  
> SD > card's erase block size)?
> 
> Flash cards tend to be rather flaky given that they are low cost  
> consumergrade commodities these days.  I would recommend getting a new card  
> firstand seeing if you can still replicate the problem.

I am afraid my explanation was not completely clear: I reformatted the *same*  
card which gave a lot of VFAT errors with ext2 and ran the test again.  Up to  
now, I filled the card several times and erased it, without any problem.  Only  
after killing (switch power off) the system in mid-air, I had a non-clean fs  
and one broken file (which is not surprising, afaik), giving input/output  
errors when I tried to read it.

So my first observation, that VFAT corrupts "somehow", but ext2 doesn't, still  
seems to be valid!

> Out of the box, I've had to replace 2 of 8 flash cards in the last 6 months  
> when they showed similiarly eerie data corruption when files disappeared.

Yes, I know... Unfortunately, it's not possible to get reliable data about the  
achievable life time.

> Doing an md5sum on the device 2 times in a row and getting back different  
> results is Not Good.

Good idea - will be the next check!

Thanks, Albrecht.


--
  Albrecht Dreß  -  Johanna-Kirchner-Straße 13  -  D-53123 Bonn (Germany)
        Phone (+49) 228 6199571  -  mailto:albrecht.dress@arcor.de
   GnuPG public key:  http://www.mynetcologne.de/~nc-dreszal/pubkey.asc

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]