squashfs and NAND flash

squashfs and NAND flash

[Anil Gupta]

Hi,
We have an embedded system with a squashfs rootfs in NAND flash. The
rootfs is mounted using the RO mtdblock device. The rootfs is
occasionally upgraded, writing the entire squashfs
image into flash using nandwrite.

We found that if the NAND flash has a bad sector, the bad sector is
correctly skipped when the image is written but fs access errors occur
when the fs is mounted. Some debugging shows that the filesystem is
off by 1 when reading flash sectors beyond the bad sector.

We are using squashfs-2.1, kernel 2.4.31, mtd snapshot 20051003.

Should mtd handle bad blocks when using squashfs?

Thanks

-Anil

_________________________________________________________________
Is your PC infected? Get a FREE online computer virus scan from McAfee® 
Security. http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963

RE: squashfs and NAND flash

[Andy Hawkins]

Hi,

> Should mtd handle bad blocks when using squashfs?

I suspect you will have to write a simple 'translation layer' that
automatically skips bad blocks. This is something we had to do in our
device.

Basically, if (for example) block 4 is bad, then any request for block 4
should actually return block 5. Also, any request for block 5 should return
block 6 etc. etc. etc.

Hope this helps.

Andy

Re: squashfs and NAND flash

[Vitaly Wool]

Hi,

Anil Gupta wrote:
> Hi,
> We have an embedded system with a squashfs rootfs in NAND flash. The
> rootfs is mounted using the RO mtdblock device. The rootfs is
> occasionally upgraded, writing the entire squashfs
> image into flash using nandwrite.
>
> We found that if the NAND flash has a bad sector, the bad sector is
> correctly skipped when the image is written but fs access errors occur
> when the fs is mounted. Some debugging shows that the filesystem is
> off by 1 when reading flash sectors beyond the bad sector.
>
> We are using squashfs-2.1, kernel 2.4.31, mtd snapshot 20051003.
>
> Should mtd handle bad blocks when using squashfs?
Please see 
http://lists.infradead.org/pipermail/linux-mtd/2004-May/009695.html.
It's outdated, but I don't think it's hard to make it up-to-date.
if it's of interest to many people, I can even do that :)

Vitaly

Re: squashfs and NAND flash

[Russ Dill]

On 4/25/06, Andy Hawkins <a.hawkins@cabletime.com> wrote:
> Hi,
>
> > Should mtd handle bad blocks when using squashfs?
>
> I suspect you will have to write a simple 'translation layer' that
> automatically skips bad blocks. This is something we had to do in our
> device.
>
> Basically, if (for example) block 4 is bad, then any request for block 4
> should actually return block 5. Also, any request for block 5 should return
> block 6 etc. etc. etc.
>

Even worse, NAND flash bits occasionally flip for no reason, ie, even
when the sector isn't bad. If you read a sector, and get an ecc
correction, you should rewrite the sector to a free sector, and then
mark the current sector as free.

So in other words, even with a read only filesystem, you need a good
FTL, unless you really don't care if units stop booting after a year
or two.

Re: squashfs and NAND flash

[Phillip Lougher]

Russ Dill wrote:
> On 4/25/06, Andy Hawkins <a.hawkins@cabletime.com> wrote:
> 
>>Hi,
>>
>>
>>>Should mtd handle bad blocks when using squashfs?
>>
>>I suspect you will have to write a simple 'translation layer' that
>>automatically skips bad blocks. This is something we had to do in our
>>device.
>>
>>Basically, if (for example) block 4 is bad, then any request for block 4
>>should actually return block 5. Also, any request for block 5 should return
>>block 6 etc. etc. etc.
>>
> 
> 
> Even worse, NAND flash bits occasionally flip for no reason, ie, even
> when the sector isn't bad. If you read a sector, and get an ecc
> correction, you should rewrite the sector to a free sector, and then
> mark the current sector as free.
> 
> So in other words, even with a read only filesystem, you need a good
> FTL, unless you really don't care if units stop booting after a year
> or two.

A couple of years ago I did some work making Squashfs work with NAND. 
Because of the inherent unreliability of NAND, and the constant MTD code 
base changes, I decided not to make this work publically available.

Providing native NAND support in Squashfs represents too much work for 
absolutely no gain.  It is unlikely this is going to change in the near 
future.

Phillip Lougher

Re: squashfs and NAND flash

[Josh Boyer]

On 4/27/06, Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> A couple of years ago I did some work making Squashfs work with NAND.
> Because of the inherent unreliability of NAND, and the constant MTD code
> base changes, I decided not to make this work publically available.

Just out of curiosity, did you do this in the filesystem itself or via
a FTL like what has been discussed so far?

> Providing native NAND support in Squashfs represents too much work for
> absolutely no gain.  It is unlikely this is going to change in the near
> future.

There is gain in providing a generic mechansim that block type
filesystems can use to do this though.  Simple economics are going to
drive more devices to have NAND parts than NOR, so something needs to
eventually fill this gap.

josh

Re: squashfs and NAND flash

[Phillip Lougher]

Josh Boyer wrote:
> On 4/27/06, Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> 
>> A couple of years ago I did some work making Squashfs work with NAND.
>> Because of the inherent unreliability of NAND, and the constant MTD code
>> base changes, I decided not to make this work publically available.
> 
> 
> Just out of curiosity, did you do this in the filesystem itself or via
> a FTL like what has been discussed so far?

It was done in the filesystem itself, in a way similar to that which has 
been discussed.  Bad blocks were skipped, and on good blocks the block 
number of the stored block was written to the OOB data area.  Blocks 
were accessed by going to the expected block, and (if this wasn't the 
expected block due to bad block skipping) then scanning forwards until 
the expected block was found.

> 
>> Providing native NAND support in Squashfs represents too much work for
>> absolutely no gain.  It is unlikely this is going to change in the near
>> future.
> 
> 
> There is gain in providing a generic mechansim that block type
> filesystems can use to do this though.  Simple economics are going to
> drive more devices to have NAND parts than NOR, so something needs to
> eventually fill this gap.
> 

I think I was maybe a little unclear as to what I meant.  I think a 
generic mechanism that block filesystems could use would be very useful. 
  What I meant was providing NAND support in Squashfs would generate a 
lot more support requests and require more maintenance.  Unfortunately, 
I don't have the resources to do this.

Phillip

Re: squashfs and NAND flash

[Josh Boyer]

On 4/28/06, Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> Josh Boyer wrote:
> > There is gain in providing a generic mechansim that block type
> > filesystems can use to do this though.  Simple economics are going to
> > drive more devices to have NAND parts than NOR, so something needs to
> > eventually fill this gap.
> >
>
> I think I was maybe a little unclear as to what I meant.  I think a
> generic mechanism that block filesystems could use would be very useful.
>   What I meant was providing NAND support in Squashfs would generate a
> lot more support requests and require more maintenance.  Unfortunately,
> I don't have the resources to do this.

Oh, sure.  That's understandable.  You should be busy trying to get
squashfs into mainline ;)

josh