ubifs_scan: corrupt empty space

ubifs_scan: corrupt empty space

[Brian Hill]

One of our systems using Atmel ARM A5, Micron MT29F4G08ABADAWP -w- kernel 3.17 produced this error, which proved unrecoverable:

UBIFS error (pid 452): ubifs_scan: corrupt empty space at LEB 2670:125358
UBIFS error (pid 452): ubifs_scanned_corruption: corruption at LEB 2670:125358
UBIFS error (pid 452): ubifs_scanned_corruption: first 1618 bytes from LEB 2670:125358
00000000: fffffffd ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff

There have been various threads on this type of error over the years, such as:
http://lists.infradead.org/pipermail/linux-mtd/2012-January/039254.html

The most notable comment being:

> ubifs expect empty space to be protected by ecc. But if empty page ecc is not
> ff, this need hack in the nand driver.

I find that no, the atmel driver, unlike some others, doesn't attempt to "correct" erased pages which contain bit flips.
Is the driver still the preferred place to address this issue?  What if the erased space has more bit flips than the ECC strength of the device?  It seems like that block should simply be retired - it has no actual user data.  Will UBIFS handle this particular condition?

I thought I might try and answer "what would happen?" myself with the nandsim kernel module, but I find it can't provide bitflips in erased data - they are always simulated as "perfect".

Thanks for any input,
-Brian Hill

Re: ubifs_scan: corrupt empty space

[Sheng Yong]

On 6/26/2015 10:31 PM, Brian Hill wrote:
> 
> One of our systems using Atmel ARM A5, Micron MT29F4G08ABADAWP -w- kernel 3.17 produced this error, which proved unrecoverable:
> 
> UBIFS error (pid 452): ubifs_scan: corrupt empty space at LEB 2670:125358
> UBIFS error (pid 452): ubifs_scanned_corruption: corruption at LEB 2670:125358
> UBIFS error (pid 452): ubifs_scanned_corruption: first 1618 bytes from LEB 2670:125358
> 00000000: fffffffd ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
> 
> There have been various threads on this type of error over the years, such as:
> http://lists.infradead.org/pipermail/linux-mtd/2012-January/039254.html
> 
> The most notable comment being:
> 
>> ubifs expect empty space to be protected by ecc. But if empty page ecc is not
>> ff, this need hack in the nand driver.
> 
> I find that no, the atmel driver, unlike some others, doesn't attempt to "correct" erased pages which contain bit flips.
> Is the driver still the preferred place to address this issue?  What if the erased space has more bit flips than the ECC strength of the device?  It seems like that block should simply be retired - it has no actual user data.  Will UBIFS handle this particular condition?
> 
> I thought I might try and answer "what would happen?" myself with the nandsim kernel module, but I find it can't provide bitflips in erased data - they are always simulated as "perfect".
Nandsim can generate bitflips by setting bitflips=n. Then read will set a random
bitflip in the read buffer. But this does not always happen. You can read do_bit_flips()
in drivers/mtd/nand/nandsim.c and you will see that only if (bitflips && prandom_u32() < (1<<22)),
(but "prandom_u32() < (1 << 22)" seems unlikely happen) bitflip is generated. So if you
would like to make bitflip likely happen, you could change the condition.

thanks,
Sheng
> 
> Thanks for any input,
> -Brian Hill
> 
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 
> 

Re: ubifs_scan: corrupt empty space

[Brian Hill]

On 6/27/2015 3:28 AM, Sheng Yong wrote:
> On 6/26/2015 10:31 PM, Brian Hill wrote:
>> I thought I might try and answer "what would happen?" myself with the nandsim kernel module, but I find it can't provide bitflips in erased data - they are always simulated as "perfect".
> Nandsim can generate bitflips by setting bitflips=n. Then read will set a random
> bitflip in the read buffer. But this does not always happen. You can read do_bit_flips()
> in drivers/mtd/nand/nandsim.c and you will see that only if (bitflips && prandom_u32() < (1<<22)),
> (but "prandom_u32() < (1 << 22)" seems unlikely happen) bitflip is generated. So if you
> would like to make bitflip likely happen, you could change the condition.
>
> thanks,
> Sheng

Sheng,
Bit flips are never simulated on erased pages, they are always 0xFFs.

static void read_page(struct nandsim *ns, int num)
{
	union ns_mem *mypage;

	if (ns->cfile) {
		if (!test_bit(ns->regs.row, ns->pages_written)) {
			NS_DBG("read_page: page %d not written\n", ns->regs.row);
			memset(ns->buf.byte, 0xFF, num);           <----- perfect 0xFFs
		} else {
			loff_t pos;
			ssize_t tx;

			NS_DBG("read_page: page %d written, reading from %d\n",
				ns->regs.row, ns->regs.column + ns->regs.off);
			if (do_read_error(ns, num))
				return;
			pos = (loff_t)NS_RAW_OFFSET(ns) + ns->regs.off;
			tx = read_file(ns, ns->cfile, ns->buf.byte, num, pos);
			if (tx != num) {
				NS_ERR("read_page: read error for page %d ret %ld\n", ns->regs.row, (long)tx);
				return;
			}
			do_bit_flips(ns, num);        <----- not reached for erased pages
		}
		return;
	}


Thanks,
-Brian Hill

Re: ubifs_scan: corrupt empty space

[Richard Weinberger]

On Fri, Jun 26, 2015 at 4:31 PM, Brian Hill <brian@houston-radar.com> wrote:
>
> One of our systems using Atmel ARM A5, Micron MT29F4G08ABADAWP -w- kernel 3.17 produced this error, which proved unrecoverable:
>
> UBIFS error (pid 452): ubifs_scan: corrupt empty space at LEB 2670:125358
> UBIFS error (pid 452): ubifs_scanned_corruption: corruption at LEB 2670:125358
> UBIFS error (pid 452): ubifs_scanned_corruption: first 1618 bytes from LEB 2670:125358
> 00000000: fffffffd ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
>
> There have been various threads on this type of error over the years, such as:
> http://lists.infradead.org/pipermail/linux-mtd/2012-January/039254.html
>
> The most notable comment being:
>
>> ubifs expect empty space to be protected by ecc. But if empty page ecc is not
>> ff, this need hack in the nand driver.
>
> I find that no, the atmel driver, unlike some others, doesn't attempt to "correct" erased pages which contain bit flips.
> Is the driver still the preferred place to address this issue?  What if the erased space has more bit flips than the ECC strength of the device?  It seems like that block should simply be retired - it has no actual user data.  Will UBIFS handle this particular condition?

UBIFS assumes that an empty page contains only 0xFFs.
On modern NAND chips, especially MLC, this seems no longer to be the case.

So far no generic solution was found.
Please see:
http://lists.infradead.org/pipermail/linux-mtd/2014-March/052521.html
and
http://lists.infradead.org/pipermail/linux-mtd/2014-March/052507.html

-- 
Thanks,
//richard

Re: ubifs_scan: corrupt empty space

[nick]

On 2015-06-27 04:11 PM, Richard Weinberger wrote:
> On Fri, Jun 26, 2015 at 4:31 PM, Brian Hill <brian@houston-radar.com> wrote:
>>
>> One of our systems using Atmel ARM A5, Micron MT29F4G08ABADAWP -w- kernel 3.17 produced this error, which proved unrecoverable:
>>
>> UBIFS error (pid 452): ubifs_scan: corrupt empty space at LEB 2670:125358
>> UBIFS error (pid 452): ubifs_scanned_corruption: corruption at LEB 2670:125358
>> UBIFS error (pid 452): ubifs_scanned_corruption: first 1618 bytes from LEB 2670:125358
>> 00000000: fffffffd ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
>>
>> There have been various threads on this type of error over the years, such as:
>> http://lists.infradead.org/pipermail/linux-mtd/2012-January/039254.html
>>
>> The most notable comment being:
>>
>>> ubifs expect empty space to be protected by ecc. But if empty page ecc is not
>>> ff, this need hack in the nand driver.
>>
>> I find that no, the atmel driver, unlike some others, doesn't attempt to "correct" erased pages which contain bit flips.
>> Is the driver still the preferred place to address this issue?  What if the erased space has more bit flips than the ECC strength of the device?  It seems like that block should simply be retired - it has no actual user data.  Will UBIFS handle this particular condition?
> 
> UBIFS assumes that an empty page contains only 0xFFs.
> On modern NAND chips, especially MLC, this seems no longer to be the case.
> 
> So far no generic solution was found.
> Please see:
> http://lists.infradead.org/pipermail/linux-mtd/2014-March/052521.html
> and
> http://lists.infradead.org/pipermail/linux-mtd/2014-March/052507.html
> 
Richard,
There seem to be a lot of messages related to unfixed/issues with ubifs and current 
flash hardware. I was wondering if we can either write better documentation on the
mtd wiki for this or even better in the kernel Documentation directory to help save
you some time with answering these common questions now. :) Unfortunately I am not
aware of all the issues so letting someone else to write the Docs is better. 
Nick