Question about commit 02e7c5b75cd4ad5176441add156389c71dab6e3a - avoid including wayward devices

Question about commit 02e7c5b75cd4ad5176441add156389c71dab6e3a - avoid including wayward devices

[Alexander Lyakas]

Hi Neil,
can you pls give some details on that commit.

As far as I understand, this change attempts to protect from
split-brain, most typical to RAID1 (but also, e.g., to 4-drive RAID6)
, where part of a mirrored set was assembled independently. The code
first selects "most_recent" based on event count (as usual). Then it
applies the map check to all those devices that are not "most_recent",
and might kick them out, if it detects split-brain.
However, when there is such split-brain, and parts of mirrored sets
are assembled independently, the highest event count does not really
tell us which part of the mirrored set is "more up-to-date". This is
because event count is not tied to any hard clock or something like
that. So there is really no way to tell what part of the mirrored set
will be picked up here (WRT to user activity on the separate mirrored
sets).

What I am trying to say, I guess: don't you think that in such case,
it would be better to warn the user and abort, and not pick (more or
less) arbitrary part of the set? Or, in other words:) might you
reconsider looking at some ideas for split-brain protection I pitched
some time ago?:))

Thanks!
Alex.

Re: Question about commit 02e7c5b75cd4ad5176441add156389c71dab6e3a - avoid including wayward devices

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2065 bytes --]

On Tue, 22 May 2012 15:59:50 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
wrote:

> Hi Neil,
> can you pls give some details on that commit.
> 
> As far as I understand, this change attempts to protect from
> split-brain, most typical to RAID1 (but also, e.g., to 4-drive RAID6)
> , where part of a mirrored set was assembled independently. The code
> first selects "most_recent" based on event count (as usual). Then it
> applies the map check to all those devices that are not "most_recent",
> and might kick them out, if it detects split-brain.
> However, when there is such split-brain, and parts of mirrored sets
> are assembled independently, the highest event count does not really
> tell us which part of the mirrored set is "more up-to-date". This is
> because event count is not tied to any hard clock or something like
> that. So there is really no way to tell what part of the mirrored set
> will be picked up here (WRT to user activity on the separate mirrored
> sets).

In a split brain situation *neither* side is "more up-to-date".  They are
both simply "differently up-to-date".  A wall-clock based event count would
not change this fact.

> 
> What I am trying to say, I guess: don't you think that in such case,
> it would be better to warn the user and abort, and not pick (more or
> less) arbitrary part of the set? Or, in other words:) might you
> reconsider looking at some ideas for split-brain protection I pitched
> some time ago?:))

This is a policy question and so I am happy for an extension to the new
"policy" mechanism in mdadm to allow finer control for managing it.
I'm fairy sure that I think the default should be the current behaviour.
If you are assembling the arrays with "-I" it not really possible to reject
the first half of the brain that is found, so I don't think we should when
assembling with "-A".

I'm afraid I don't particularly remember the ideas you pitched before.  Feel
free to pitch them again -- and repeat every few weeks until you get an
answer :-)

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]