Re: domino-style OSD crash

[Tommi Virtanen <tv@inktank.com>]

On Tue, Jul 10, 2012 at 10:36 AM, Yann Dupont
<Yann.Dupont@univ-nantes.fr> wrote:
>> Fundamentally, it comes down to this: the two clusters will still have
>> the same fsid, and you won't be isolated from configuration errors or
> (CEPH-PROD is the old btrfs volume ). /CEPH is new xfs volume, completely
> redone & reformatted with mkcephfs. The volumes are totally independant :

Ahh you re-created the monitors too. That changes things, then you
have a new random fsid. I understood you only re-mkfsed the osd.

Doing it like that, your real worry is just the remembered state of
monmaps, osdmaps etc. If the daemons accidentally talk to the wrong
cluster, the fsid *should* protect you from damage; they should get
rejected. Similarly, if you use cephx authentication, the keys won't
match either.

>> Naturally, this practically means giving up on high availability.
> The idea is not just having 2 mon. I'll still use 3 differents machines for
> mon, but with 2 mon instance on each. One for the current ceph, the other
> for the old ceph.
> 2x3 Mon.

That should be perfectly doable.