UBI bitrot checking and scrubbing

UBI bitrot checking and scrubbing

[Stelling Carsten]

Hi,

I found Richard’s patches http://lists.infradead.org/pipermail/linux-mtd/2015-April/058677.html or
http://lists.infradead.org/pipermail/linux-mtd/2015-November/063122.html, but can't identify any
of the code in newer kernel versions (e.g. 4.9 or 4.11).
I wonder, if ubihealthd http://lists.infradead.org/pipermail/linux-mtd/2015-November/063128.html
was never released with mtd-utils (actually 1.5.2). Possibly, I have missed some information on
this topic.
Is there actually a comparable mechanism in UBI, I haven’t found yet?
Like wear levelling, reading and scrubbing (if necessary) *all* PEB’s from time to time is an
essential task to ensure durability of information. Especially, for PEB’s which are not accessed
frequently. With Fastmap enabled, one can't rely on cyclic reboots, because not all UBI
VID/EC headers are read at boot time.
Regarding bitrot checking, a lot of effort was made by several contributors, but I can't find
their work in mainline kernel driver code. Have I overlooked something?
Any hints or suggestions are greatly appreciated.

Thank you very much.

Regards,
Carsten

______________________________________
 Carsten Stelling
F&E Hardware / R&D Hardware

GÖRLITZ Aktiengesellschaft | August-Thyssen-Straße 32 | D-56070 Koblenz
T: +49-261-9285-336 | M:  | F: +49-261-9285-190
Mail to: Carsten.Stelling@goerlitz.com | www.goerlitz.com

Vorstand | Executive Board: Jörg Figge
Vorsitzender des Aufsichtsrates | Chairman of the supervisory board: Norbert Wagner
Registergericht | Court of registration: Amtsgericht Koblenz HRB 5346
Sitz der Gesellschaft | Registered office: Koblenz
Member of the IDS-Group

Immer up-to-date mit unserem Newsletter! Hier geht's zur Anmeldung: www.goerlitz.com/newsletter

Re: UBI bitrot checking and scrubbing

[Richard Weinberger]

Carsten,

On Tue, Jun 27, 2017 at 12:59 PM, Stelling Carsten
<Carsten.Stelling@goerlitz.com> wrote:
> Hi,
>
> I found Richard’s patches http://lists.infradead.org/pipermail/linux-mtd/2015-April/058677.html or
> http://lists.infradead.org/pipermail/linux-mtd/2015-November/063122.html, but can't identify any
> of the code in newer kernel versions (e.g. 4.9 or 4.11).
> I wonder, if ubihealthd http://lists.infradead.org/pipermail/linux-mtd/2015-November/063128.html
> was never released with mtd-utils (actually 1.5.2). Possibly, I have missed some information on
> this topic.
> Is there actually a comparable mechanism in UBI, I haven’t found yet?
> Like wear levelling, reading and scrubbing (if necessary) *all* PEB’s from time to time is an
> essential task to ensure durability of information. Especially, for PEB’s which are not accessed
> frequently. With Fastmap enabled, one can't rely on cyclic reboots, because not all UBI
> VID/EC headers are read at boot time.
> Regarding bitrot checking, a lot of effort was made by several contributors, but I can't find
> their work in mainline kernel driver code. Have I overlooked something?
> Any hints or suggestions are greatly appreciated.

Since the MLC topic kind of faded away I had no chance to work on the
bitrot stuff.
But you are right we should revive ubihealthd and the needed kernel
interface because for Fastmap
this is a topic.

Would you volunteer to forward port the patches?

-- 
Thanks,
//richard

AW: UBI bitrot checking and scrubbing

[Stelling Carsten]

Hi Richard,

> But you are right we should revive ubihealthd and the needed kernel
> interface because for Fastmap
> this is a topic.
I think it's a topic for all devices with or without Fastmap enabled, but rarely
powered off and on, or rebooted. For SLC NAND, data retention is specified
with 5 to 10 years, dependent on type and manufacturer. But data retention
time degrades with temperature and the number of P/E cycles. Read and write
disturbance is another influencing variable, which must be taken into account.
Therefore, it is only a matter of time, when accumulated bitflips within a page
cannot be corrected by the ECC. Most often, these accumulation of bitflips is
not detected while the device is operating, because executable code was read
from NAND, copied to RAM and executed there. But during reboot, e.g. caused
by power interruption, the executable code or other rootfs content cannot be
loaded properly, causing the system to fail.

> Would you volunteer to forward port the patches?
I'm stuck with a 3.14 kernel. If I've been given a timeslot to update the kernel
and toolchain, I would like to do the forward port. But actually, this seems not
possible for the next few month.

Thank you.
Carsten



______________________________________
 Carsten Stelling
F&E Hardware / R&D Hardware

GÖRLITZ Aktiengesellschaft | August-Thyssen-Straße 32 | D-56070 Koblenz
T: +49-261-9285-336 | M:  | F: +49-261-9285-190
Mail to: Carsten.Stelling@goerlitz.com | www.goerlitz.com

Vorstand | Executive Board: Jörg Figge
Vorsitzender des Aufsichtsrates | Chairman of the supervisory board: Norbert Wagner
Registergericht | Court of registration: Amtsgericht Koblenz HRB 5346
Sitz der Gesellschaft | Registered office: Koblenz
Member of the IDS-Group

Immer up-to-date mit unserem Newsletter! Hier geht's zur Anmeldung: www.goerlitz.com/newsletter

Re: UBI bitrot checking and scrubbing

[Richard Weinberger]

Carsten,

Am 27.06.2017 um 15:36 schrieb Stelling Carsten:
>> Would you volunteer to forward port the patches?
> I'm stuck with a 3.14 kernel. If I've been given a timeslot to update the kernel
> and toolchain, I would like to do the forward port. But actually, this seems not
> possible for the next few month.

So, it is still up to me? ;-)

Thanks,
//richard

AW: UBI bitrot checking and scrubbing

[Stelling Carsten]

Hi Richard,

>>> Would you volunteer to forward port the patches?
>> I'm stuck with a 3.14 kernel. If I've been given a timeslot to update the kernel
>> and toolchain, I would like to do the forward port. But actually, this seems not
>> possible for the next few month.

> So, it is still up to me? ;-)
Actually, and for newer kernel 4.x, yes :-(

Regards,
Carsten

______________________________________
 Carsten Stelling
F&E Hardware / R&D Hardware

GÖRLITZ Aktiengesellschaft | August-Thyssen-Straße 32 | D-56070 Koblenz
T: +49-261-9285-336 | M:  | F: +49-261-9285-190
Mail to: Carsten.Stelling@goerlitz.com | www.goerlitz.com

Vorstand | Executive Board: Jörg Figge
Vorsitzender des Aufsichtsrates | Chairman of the supervisory board: Norbert Wagner
Registergericht | Court of registration: Amtsgericht Koblenz HRB 5346
Sitz der Gesellschaft | Registered office: Koblenz
Member of the IDS-Group

Immer up-to-date mit unserem Newsletter! Hier geht's zur Anmeldung: www.goerlitz.com/newsletter