Re: Issue with Raid 10 super block failing

[Drew Reusser <dreusser@gmail.com>]

On Mon, Nov 19, 2012 at 8:39 AM, Phil Turmel <philip@turmel.org> wrote:
> Hi Drew,
>
> On 11/18/2012 02:10 PM, Drew Reusser wrote:
>
> [trim /]
>
>> Sorry - did not know the rules about top posting.  Is there something
>> specific in the dmesg you are looking for?  I tried to mount it again
>> and copied everything in the buffer.
>
> Here's what I wanted to see:
>
>> [270303.640240] EXT4-fs (md0): VFS: Can't find ext4 filesystem
>
> This suggests that the ext4 superblock isn't near the beginning like
> it's supposed to be.  One of the ways that happens with MD raid is if
> someone does "mdadm --create" and destroys their old raid superblocks.
>
> I went back and looked at:
>
>>   Creation Time : Thu Nov 15 16:08:02 2012
>
> and:
>
>>     Data Offset : 262144 sectors
>
> So you've re-created the MD array.  That's bad.  Chunk size and Data
> offset size and alignment defaults have changed in the past couple
> years, so re-creating an array with a different mdadm version can cause
> these problems.  You can also lose the original order of devices, with
> similar consequences.
>
> (Side note:  there's various pieces of advice floating around the
> internet on recovering a broken array that start with re-creating the
> array.  It's horribly wrong, and only a last resort, and only after
> recording all the details about the original array.)
>
> Unless you kept a copy of "mdadm --examine /dev/sd[abde]1" for the
> original array, this will be difficult to debug further.  Your best
> chance is to go back to the version of mdadm available when you first
> built the system and recreate with that, trying the various device order
> combinations.
>
> Don't attempt to mount to check for success.  First, use "fsck -n" to
> non-destructively check the FS.  If that gives few errors, then you can
> mount the FS.
>
> Phil

Looking at this from all angles, is there a way to look at the
individual disks (like sdb and sde) and build a raid 0 from them and
see if that works?   Is there a way to see which if any are bad from a
file system point of view and exclude it and try to rebuild it?  I am
just grasping at straws trying to figure out which way to go.