Re: Prevent rbd mapping/mounting on multiple hosts workaround

[Gregory Farnum <gfarnum@redhat.com>]

On Fri, Feb 5, 2016 at 6:24 AM, Mauricio Garavaglia
<mauricio@medallia.com> wrote:
> Hello,
>
> In the January Tech Talk (PostgreSQL on Ceph under Mesos/Aurora with
> Docker [https://youtu.be/OqlC7S3cUKs]) we presented a challenge we are
> facing at Medallia when running databases on ceph under
> mesos/aurora/docker; which is related to prevent mapping/mounting the
> same rbd image in two hosts at the same time during network
> partitions.
>
> As a workaround it was mentioned that we are wrapping rbd in a shell
> script that executes extra logic around certain operations:
>
> - On Map; rbd lock add <image>
>         - If no success; then
>                 - "rbd status <image>": check for Watchers, 3 times each 15 secs
>                         - If found, ABORT the mapping. The image is
> still in use in a host that is healthy
>                 - "ceph osd blacklist add <previous lock holder>".
> Image locked without a watcher
>                 - steal the lock in <image>
>         - map the image
>
> - On Unmap;
>         - rbd lock remove
>
> - On reboot of server;
>         - "ceph osd blacklist rm <self>"
>
> I was wondering if this mechanism could be incorporated as part of the
> rbd CLI, of course controlled by an option during map. We'll be happy
> to work on it, but want to check the feasibility of having the patch
> accepted.

I actually thought we had a disable-by-default config option in later
releases that grab the locks before allowing a mount, but now I can't
find it. Jason?
-Greg