RE: [PATCH v8] introduce vfio-user protocol specification

From: Thanos Makatos <thanos.makatos@nutanix.com>
To: John Levon <john.levon@nutanix.com>,
	Alex Williamson <alex.williamson@redhat.com>
Cc: "benjamin.walker@intel.com" <benjamin.walker@intel.com>,
	John G Johnson <john.g.johnson@oracle.com>,
	Swapnil Ingle <swapnil.ingle@nutanix.com>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	John Levon <levon@movementarian.org>,
	Kirti Wankhede <kwankhede@nvidia.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	"tina.zhang@intel.com" <tina.zhang@intel.com>,
	"jag.raman@oracle.com" <jag.raman@oracle.com>,
	"james.r.harris@intel.com" <james.r.harris@intel.com>,
	Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	"Kanth.Ghatraju@oracle.com" <Kanth.Ghatraju@oracle.com>,
	Felipe Franciosi <felipe@nutanix.com>,
	"marcandre.lureau@redhat.com" <marcandre.lureau@redhat.com>,
	Yan Zhao <yan.y.zhao@intel.com>,
	"konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"yuvalkashtan@gmail.com" <yuvalkashtan@gmail.com>,
	"dgilbert@redhat.com" <dgilbert@redhat.com>,
	"ismael@linux.com" <ismael@linux.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	"changpeng.liu@intel.com" <changpeng.liu@intel.com>,
	"tomassetti.andrea@gmail.com" <tomassetti.andrea@gmail.com>,
	"mpiszczek@ddn.com" <mpiszczek@ddn.com>,
	Cornelia Huck <cohuck@redhat.com>,
	Christophe de Dinechin <cdupontd@redhat.com>,
	Raphael Norwitz <raphael.norwitz@nutanix.com>,
	"xiuchun.lu@intel.com" <xiuchun.lu@intel.com>,
	Paolo Bonzini <pbonzini@redhat.com>
Subject: RE: [PATCH v8] introduce vfio-user protocol specification
Date: Mon, 14 Jun 2021 09:47:12 +0000	[thread overview]
Message-ID: <CH0PR02MB7898A0837EC681D880E15B558B319@CH0PR02MB7898.namprd02.prod.outlook.com> (raw)
In-Reply-To: <20210519223844.GA1036405@sent>

> > > +VFIO_USER_DMA_UNMAP
> > > +-------------------
> > > +
> > > +This command message is sent by the client to the server to inform
> > > +it that a DMA region, previously made available via a
> > > +VFIO_USER_DMA_MAP command message, is no longer available for
> DMA.
> > > +It typically occurs when memory is subtracted from the client or if
> > > +the client uses a vIOMMU. If the client does not expect the server
> > > +to perform DMA then it does not need to send to the server
> > > +VFIO_USER_DMA_UNMAP commands. If the server does not need to
> > > +perform DMA then it can ignore such commands but it must still
> > > +reply to them. The table is an
> >
> > I'm confused why expectation of DMA plays a factor here.  For example,
> > if QEMU unplugs a DIMM and the server has an mmap of the file
> > descriptor related to that DIMM, does it get to retain the mmap if it
> > doesn't currently have any DMA queued targeting that address range?
> > Can QEMU skip sending an unmap if the PCI bus master bit is disabled
> > on the device preventing further DMA?  How can the associated file
> > descriptor get released?  This doesn't feel strongly specified.
> 
> I thought we'd removed those sentences actually, as they're just confusing.
> In reality, everything is going to both send and handle map/unmap
> messages.
> 
> > Are there any assumptions about address and size of the unmap command
> > relative to the original map command or is the client freely allowed
> > to bisect, overlap, or overextend previous mappings?
> 
> Good question. Filed https://github.com/nutanix/libvfio-user/issues/504 to
> track this.
> 
> I actually don't know what clients would like to be able to do in this respect.

It's probably not worth supporting such behavior at this point, especially since
there's no valid use case yet. Let's drop it. Should we ever want to introduce
such behavior in the future, can should be able to do so by a capability. Also,
the protocol format won't have to change since additional DMA regions can
simply be appended to the message payload.