UBIFS corruption and UBI + unstable bit questions

UBIFS corruption and UBI + unstable bit questions

[Morand, Guy]

Hello UBI developers,

I was wondering what happens to the UBI volume if an UBIFS partition gets 
corrupted:
* Is only the partition corrupted?
* Is the whole volume corrupted?

Moreover, about this "Unstable bit issue" ... 
* Does it still make sense to use UBI/UBIFS as it doesn't seem to be fully 
  power cut failsafe?
* Have you planned to fix this issue?

Best regards,

Guy

Re: UBIFS corruption and UBI + unstable bit questions

[Richard Weinberger]

On Thu, Mar 26, 2015 at 5:00 PM, Morand, Guy <Guy.Morand@comet.ch> wrote:
> Hello UBI developers,
>
> I was wondering what happens to the UBI volume if an UBIFS partition gets
> corrupted:
> * Is only the partition corrupted?
> * Is the whole volume corrupted?

Can you please be more specific.
What exactly do you mean by "only the partition"?
Also what do you mean by "corrupted"?

If a filesystem is corrupted there is nothing you can do...
(By definition of corrupted)

> Moreover, about this "Unstable bit issue" ...
> * Does it still make sense to use UBI/UBIFS as it doesn't seem to be fully
>   power cut failsafe?

It all depends on how you define "fully power cut failsafe".
UBI/FS is not a magic bullet. People who *really* care about power cut safety
attack the problem also with their hardware.
e.g., by not using the cheapest NAND chip they can find. ;-)

> * Have you planned to fix this issue?

TBH nobody cared enough so far to either fix the issue on its own or
fund one of us UBI developers.

Are you able to trigger this problem?

-- 
Thanks,
//richard

AW: UBIFS corruption and UBI + unstable bit questions

[Morand, Guy]

> Can you please be more specific.
> What exactly do you mean by "only the partition"?
> Also what do you mean by "corrupted"?

Sorry, nomenclature problem ... I mean by a volume the layer on top of the MTD 
device (UBI):
ubiformat /dev/mtd1
ubiattach /dev/ubi_ctrl -m 1

By partition, I mean actually a volume on top of UBI (UBIFS):
ubimkvol /dev/ubi0 -N partition1 -s 64MiB
ubimkvol /dev/ubi0 -N partition2 -m

Then if there is a "unstable bit" on partition1, is also the partition2 
corrupted? This means there is nothing I can do to restore it.

> UBI/FS is not a magic bullet. People who *really* care about power cut safety
> attack the problem also with their hardware.
Yieah right, I understand... I've seen some hardware with super caps that gives 
some time to Linux to shutdown correctly, I really love it!

> Are you able to trigger this problem?
No, I'm just thinking about a nice partitioning strategy on my platform to 
avoid as much corruptions as possible with the hardware I have in hands!

Kind regards,

Guy

-----Ursprüngliche Nachricht-----
Von: Richard Weinberger [mailto:richard.weinberger@gmail.com] 
Gesendet: Donnerstag, 26. März 2015 17:45
An: Morand, Guy
Cc: linux-mtd@lists.infradead.org
Betreff: Re: UBIFS corruption and UBI + unstable bit questions

On Thu, Mar 26, 2015 at 5:00 PM, Morand, Guy <Guy.Morand@comet.ch> wrote:
> Hello UBI developers,
>
> I was wondering what happens to the UBI volume if an UBIFS partition gets
> corrupted:
> * Is only the partition corrupted?
> * Is the whole volume corrupted?

Can you please be more specific.
What exactly do you mean by "only the partition"?
Also what do you mean by "corrupted"?

If a filesystem is corrupted there is nothing you can do...
(By definition of corrupted)

> Moreover, about this "Unstable bit issue" ...
> * Does it still make sense to use UBI/UBIFS as it doesn't seem to be fully
>   power cut failsafe?

It all depends on how you define "fully power cut failsafe".
UBI/FS is not a magic bullet. People who *really* care about power cut safety
attack the problem also with their hardware.
e.g., by not using the cheapest NAND chip they can find. ;-)

> * Have you planned to fix this issue?

TBH nobody cared enough so far to either fix the issue on its own or
fund one of us UBI developers.

Are you able to trigger this problem?

-- 
Thanks,
//richard

Re: UBIFS corruption and UBI + unstable bit questions

[Richard Weinberger]

Hi!

Am 26.03.2015 um 18:18 schrieb Morand, Guy:
>> Can you please be more specific.
>> What exactly do you mean by "only the partition"?
>> Also what do you mean by "corrupted"?
> 
> Sorry, nomenclature problem ... I mean by a volume the layer on top of the MTD 
> device (UBI):
> ubiformat /dev/mtd1
> ubiattach /dev/ubi_ctrl -m 1
> 
> By partition, I mean actually a volume on top of UBI (UBIFS):
> ubimkvol /dev/ubi0 -N partition1 -s 64MiB
> ubimkvol /dev/ubi0 -N partition2 -m

So, an UBI volume. :-)

> Then if there is a "unstable bit" on partition1, is also the partition2 
> corrupted? This means there is nothing I can do to restore it.

No, UBI tries hard to preserve data.
But it depends on the type of the corruption.

>> UBI/FS is not a magic bullet. People who *really* care about power cut safety
>> attack the problem also with their hardware.
> Yieah right, I understand... I've seen some hardware with super caps that gives 
> some time to Linux to shutdown correctly, I really love it!

It is not really Linux which needs time to shutdown, mostly other hardware components.
People who care know exactly which components need some time, the NAND chip is only one of them.
Also they don't only care about a plain power cut. You also need to take care of voltage variations
during a power cut.

>> Are you able to trigger this problem?
> No, I'm just thinking about a nice partitioning strategy on my platform to 
> avoid as much corruptions as possible with the hardware I have in hands!

Okay. That's what I thought. I've never seen the issue on serious hardware.

Thanks,
//richard

UBIFS corruption and UBI + unstable bit questions

[Morand, Guy]

Hi,

Thanks for clarifying!

> But it depends on the type of the corruption.
What do you mean by "type of the corruption". Is it about a PEB containing the 
volume table corrupting every volumes on the MTD device? I can't imagine a PEB
containing "regular data" corrupting the whole MTD device or correct me if I'm 
wrong! I guess this should be extremely rare as we don't update the volume table 
every day ...

> Okay. That's what I thought. I've never seen the issue on serious hardware.
Serious hardware, does that mean with seriously select components?

Kind regards,

Guy

-----Ursprüngliche Nachricht-----
Von: Richard Weinberger [mailto:richard@nod.at] 
Gesendet: Donnerstag, 26. März 2015 19:18
An: Morand, Guy
Cc: linux-mtd@lists.infradead.org
Betreff: Re: UBIFS corruption and UBI + unstable bit questions

Hi!

Am 26.03.2015 um 18:18 schrieb Morand, Guy:
>> Can you please be more specific.
>> What exactly do you mean by "only the partition"?
>> Also what do you mean by "corrupted"?
> 
> Sorry, nomenclature problem ... I mean by a volume the layer on top of the MTD 
> device (UBI):
> ubiformat /dev/mtd1
> ubiattach /dev/ubi_ctrl -m 1
> 
> By partition, I mean actually a volume on top of UBI (UBIFS):
> ubimkvol /dev/ubi0 -N partition1 -s 64MiB
> ubimkvol /dev/ubi0 -N partition2 -m

So, an UBI volume. :-)

> Then if there is a "unstable bit" on partition1, is also the partition2 
> corrupted? This means there is nothing I can do to restore it.

No, UBI tries hard to preserve data.
But it depends on the type of the corruption.

>> UBI/FS is not a magic bullet. People who *really* care about power cut safety
>> attack the problem also with their hardware.
> Yieah right, I understand... I've seen some hardware with super caps that gives 
> some time to Linux to shutdown correctly, I really love it!

It is not really Linux which needs time to shutdown, mostly other hardware components.
People who care know exactly which components need some time, the NAND chip is only one of them.
Also they don't only care about a plain power cut. You also need to take care of voltage variations
during a power cut.

>> Are you able to trigger this problem?
> No, I'm just thinking about a nice partitioning strategy on my platform to 
> avoid as much corruptions as possible with the hardware I have in hands!

Okay. That's what I thought. I've never seen the issue on serious hardware.

Thanks,
//richard

Re: UBIFS corruption and UBI + unstable bit questions

[Richard Weinberger]

Am 27.03.2015 um 14:30 schrieb Morand, Guy:
> Hi,
> 
> Thanks for clarifying!
> 
>> But it depends on the type of the corruption.
> What do you mean by "type of the corruption". Is it about a PEB containing the 
> volume table corrupting every volumes on the MTD device? I can't imagine a PEB
> containing "regular data" corrupting the whole MTD device or correct me if I'm 
> wrong! I guess this should be extremely rare as we don't update the volume table 
> every day ...

We save the volume table twice. But you can't expect that UBI or UBIFS will
survive perfectly fine if random PEBs tun out of the sudden bad.

>> Okay. That's what I thought. I've never seen the issue on serious hardware.
> Serious hardware, does that mean with seriously select components?

Yes.

Thanks,
//richard