Re: mon: forwarding user commands

[Mykola Golub <mgolub@mirantis.com>]

On Mon, Jan 19, 2015 at 07:17:58AM -0800, Sage Weil wrote:
> On Mon, 19 Jan 2015, Gregory Farnum wrote:
> > On Sun, Jan 18, 2015 at 11:02 PM, Mykola Golub <mgolub@mirantis.com> wrote:
> > > On Sun, Jan 18, 2015 at 10:33:05AM -0800, Sage Weil wrote:
> > >> On Sun, 18 Jan 2015, Mykola Golub wrote:
> > >> > Hi Ceph,
> > >> >
> > >> > Right now, for not a monitor leader, if a received command is not
> > >> > supported locally, but is supported by the leader, it is forwarded to
> > >> > the leader.
> > >> >
> > >> > For the recently added "ceph tell mon.x version" it gives a confusing
> > >> > behavior: if the mon.x is not a leader and does not support "version"
> > >> > command yet, but the leader does, the user will receive the version of
> > >> > the leader, and can't be actually sure about a non leader version.
> > >> >
> > >> > I have a patch that fixes this by checking if the received "version"
> > >> > command is forwarded and returning the error in this case:
> > >> >
> > >> > https://github.com/trociny/ceph/commit/98f835357e378b1c5f05b32ba90a8b8537ba1ad8
> > >> >
> > >> > But may be we need a more general solution? We might face a similar
> > >> > issue in future, when adding a new command, which is not expected to
> > >> > be forwarded to the leader (like injectargs).
> > >>
> > >> Yeah.. that works for 'version' but not any other 'tell mon ...' command.
> > >>
> > >> I think the way to do it properly going forward is to add a flags field to
> > >> MMonCommand, with a flag NOFORWARD.
> > >>
> > >> That won't help in the version case, so we can combine it with your
> > >> patch...
> > >
> > > Do you mean
> > >
> > > 1) statically adding the NOFORWARD flag to commands definitions in
> > > MonCommands.h (and to the struct MonCommand in Monitor.h), and then
> > > checking for the flag in Monitor::handle_command(), before forwarding
> > > to the leader, or
> 
> Hmm, this is a cleaner way to approach your change that specifically 
> applies to the new 'version' command.
> 
> > > 2) extending librados API, RadosClient::mon_command(), so a user could
> > > set the flag for any sent command she don't want to be forwarded, and
> > > using this flag for 'ceph tell mon.x ...' commands?
> 
> ..but I think we also want this, so that *any* 'ceph tell mon.whatever 
> ...' command can be flagged to not get forwarded.

I have created a pull request that implements both:

https://github.com/ceph/ceph/pull/3496

The (2) looks not very useful for now -- I suppose currently the only
command that suffers from the describe problem is "version" and it has
"noforward" hard coded. This might change in future though.

-- 
Mykola Golub