Re: Filesystem corruption on RAID1

[Adam Goryachev <mailinglists@websitemanagers.com.au>]

On 20/8/17 23:07, Wols Lists wrote:
> On 20/08/17 11:43, Mikael Abrahamsson wrote:
>> On Sun, 20 Aug 2017, Gionatan Danti wrote:
>>
>>> It can be even worse: if fsck reads from the disks with corrupted data
>>> and tries to repair based on these corrupted information, it can blow
>>> up the filesystem completely.
>> Indeed, but as far as I know there is nothing md can do about this. What
>> md could do about it is at least present a consistent view of data to
>> fsck (which for raid1 would be read all stripes and issue "repair" if
>> they don't match). Yes, this might indeed cause corruption but at least
>> it would be consistent and visible.
>>
> Which is exactly what my "force integrity check on read" proposal would
> have achieved, but that generated so much heat and argument IN FAVOUR of
> returning possibly corrupt data that I'll probably get flamed to high
> heaven if I bring it back up again. Yes, the performance hit is probably
> awful, yes it can only fix things if it's got raid-6 or a 3-disk-or-more
> raid-1 array, but the idea was that if you knew or suspected something
> was wrong, this would force a read error somewhere in the stack if the
> raid wasn't consistent.
>
> Switching it on then running your fsck might trash chunks of the
> filesystem, but at least (a) it would be known to be consistent
> afterwards, and (b) you'd know what had been trashed!
In the case where you know there are "probably" some inconsistencies, 
you have a few choices:
1) If you know which disk is faulty, then fail it, then clean the 
superblock and add it. It will be re-written from the known good drive
2) If you don't know which drive is faulty, or both drives accrued 
random write errors, then all you can do is make sure that both drives 
have the same data (even where it is wrong). So just do a check/repair 
which will ensure both drives are consistent, then you can safely do the 
fsck. (Assuming you fixed the problem causing random write errors first).

Your proposed option to read from all (or at least 2) data sources to 
ensure data consistency is an online version of the above process in 
(2), not a bad tool to have available, but not required in this scenario 
(IMHO). It is more useful when you think all drives are OK, and you want 
to be *sure* that they are OK on a continuous basis, not just after you 
think there might be a problem.

While I suspect patches would be accepted, without someone capable of 
actually writing the code being interested, then it probably won't 
happen (until one of those people needs it).

Regards,
Adam