Re: vhost-user protocol feature negotiation

["Michael S. Tsirkin" <mst@redhat.com>]

On Thu, Aug 06, 2020 at 08:59:09AM +0000, Alyssa Ross wrote:
> "Michael S. Tsirkin" <mst@redhat.com> writes:
> 
> > On Wed, Aug 05, 2020 at 03:13:06PM +0000, Alyssa Ross wrote:
> >> Quoting from the definition of VHOST_USER_SET_PROTOCOL_FEATURES in
> >> vhost-user.rst:
> >> 
> >> >   Only legal if feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` is present in
> >> >   ``VHOST_USER_GET_FEATURES``.
> >> > 
> >> > .. Note::
> >> >    Slave that reported ``VHOST_USER_F_PROTOCOL_FEATURES`` must support
> >> >    this message even before ``VHOST_USER_SET_FEATURES`` was called.
> >> 
> >> To me, this could mean either of two things:
> >> 
> >> (1) If VHOST_USER_F_PROTOCOL_FEATURES hasn't been set, upon receiving
> >>     VHOST_USER_SET_PROTOCOL_FEATURES, a backend should enable the
> >>     protocol features immediately.
> >> 
> >> (2) If VHOST_USER_F_PROTOCOL_FEATURES hasn't been set, upon receiving
> >>     VHOST_USER_SET_PROTOCOL_FEATURES, a backend should store those
> >>     feature bits, but not actually consider them to be enabled until
> >>     after VHOST_USER_SET_FEATURES has been received (presumably
> >>     containing VHOST_USER_F_PROTOCOL_FEATURES).
> >> 
> >> The reason I bring this up is that QEMU appears to interpret it as (1),
> >> while the vhost-user-net backend in Intel's cloud-hypervisor[1]
> >> interprets it as (2).  So I'm looking for a clarification.
> >> 
> >> [1]: https://github.com/cloud-hypervisor/cloud-hypervisor
> >> 
> >> Thanks in advance.
> >
> >
> > IMHO the intent was this: VHOST_USER_F_PROTOCOL_FEATURES bit in
> > VHOST_USER_GET_FEATURES means that qemu can send
> > VHOST_USER_GET_PROTOCOL_FEATURES and VHOST_USER_SET_PROTOCOL_FEATURES.
> >
> > With most feature bits in VHOST_USER_GET_FEATURES, the
> > specific functionality needs to only be enabled after
> > VHOST_USER_SET_FEATURES.
> >
> > However, this is for functionality dealing with guest activity.
> > VHOST_USER_SET_PROTOCOL_FEATURES has nothing to do with guest directly,
> > it's about negotiation between qemu and backend: it is only in
> > VHOST_USER_GET_FEATURES for the reason that this is the only message
> > (very) old backends reported.  Thus, the backend should not check
> > whether VHOST_USER_SET_FEATURES sets VHOST_USER_F_PROTOCOL_FEATURES,
> > instead it should simply always be ready to receive
> > VHOST_USER_GET_PROTOCOL_FEATURES and VHOST_USER_SET_PROTOCOL_FEATURES.
> >
> > Backend that isn't always ready to handle
> > VHOST_USER_GET_PROTOCOL_FEATURES and VHOST_USER_SET_PROTOCOL_FEATURES
> > should not set VHOST_USER_F_PROTOCOL_FEATURES in
> > VHOST_USER_GET_FEATURES.
> 
> Thanks for the explanation.  That matches what I had in mind with (1).
> 
> > This appears to be closer to (1), but if qemu can't distinguish
> > then we don't care, right? For example, VHOST_USER_PROTOCOL_F_REPLY_ACK
> > enables acks on arbitrary messages. Does the backend in question
> > ignore the affected bit until SET_FEATURES? If yes won't this
> > make qemu hang?
> 
> Yes.  That was my motivation for asking what the correct behaviour was,
> so that I could fix the incorrect one. :)  I suspect that up to this point,
> the cloud-hypervisor vhost-user-net backend has only been used with
> cloud-hypervisor, and so this incompatibilty with QEMU was not noticed.
> 
> > How would you suggest clarifying the wording?
> 
> Do you think this communicates everything required?
> 
> ---
> diff --git i/docs/interop/vhost-user.rst w/docs/interop/vhost-user.rst
> index 10e3e3475e..72724d292a 100644
> --- i/docs/interop/vhost-user.rst
> +++ w/docs/interop/vhost-user.rst
> @@ -854,9 +854,8 @@ Master message types
>    ``VHOST_USER_GET_FEATURES``.
>  
>  .. Note::
> -   Slave that reported ``VHOST_USER_F_PROTOCOL_FEATURES`` must
> -   support this message even before ``VHOST_USER_SET_FEATURES`` was
> -   called.
> +   ``VHOST_USER_F_PROTOCOL_FEATURES`` does not need to be acknowledged
> +   with ``VHOST_USER_SET_FEATURES``.
>  
>  ``VHOST_USER_SET_PROTOCOL_FEATURES``
>    :id: 16

Hmm I find this confusing. I think it's a good policy to ask qemu to
acknowledge it. It's just that the client should not wait for
VHOST_USER_SET_FEATURES before handling VHOST_USER_SET_PROTOCOL_FEATURES
or VHOST_USER_GET_PROTOCOL_FEATURES.


> @@ -869,8 +868,8 @@ Master message types
>    ``VHOST_USER_GET_FEATURES``.
>  
>  .. Note::
> -   Slave that reported ``VHOST_USER_F_PROTOCOL_FEATURES`` must support
> -   this message even before ``VHOST_USER_SET_FEATURES`` was called.
> +   ``VHOST_USER_F_PROTOCOL_FEATURES`` does not need to be acknowledged
> +   with ``VHOST_USER_SET_FEATURES``.
>  
>  ``VHOST_USER_SET_OWNER``
>    :id: 3