Re: [PATCH v4 0/4] Implement vdpasim stop operation

From: "Michael S. Tsirkin" <mst@redhat.com>
To: Parav Pandit <parav@nvidia.com>
Cc: "Eugenio Pérez" <eperezma@redhat.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"virtualization@lists.linux-foundation.org"
	<virtualization@lists.linux-foundation.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"Jason Wang" <jasowang@redhat.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"martinh@xilinx.com" <martinh@xilinx.com>,
	"Stefano Garzarella" <sgarzare@redhat.com>,
	"martinpo@xilinx.com" <martinpo@xilinx.com>,
	"lvivier@redhat.com" <lvivier@redhat.com>,
	"pabloc@xilinx.com" <pabloc@xilinx.com>,
	"Eli Cohen" <elic@nvidia.com>,
	"Dan Carpenter" <dan.carpenter@oracle.com>,
	"Xie Yongji" <xieyongji@bytedance.com>,
	"Christophe JAILLET" <christophe.jaillet@wanadoo.fr>,
	"Zhang Min" <zhang.min9@zte.com.cn>,
	"Wu Zongyong" <wuzongyong@linux.alibaba.com>,
	"lulu@redhat.com" <lulu@redhat.com>,
	"Zhu Lingshan" <lingshan.zhu@intel.com>,
	"Piotr.Uminski@intel.com" <Piotr.Uminski@intel.com>,
	"Si-Wei Liu" <si-wei.liu@oracle.com>,
	"ecree.xilinx@gmail.com" <ecree.xilinx@gmail.com>,
	"gautam.dawar@amd.com" <gautam.dawar@amd.com>,
	"habetsm.xilinx@gmail.com" <habetsm.xilinx@gmail.com>,
	"tanuj.kamde@amd.com" <tanuj.kamde@amd.com>,
	"hanand@xilinx.com" <hanand@xilinx.com>,
	"dinang@xilinx.com" <dinang@xilinx.com>,
	Longpeng <longpeng2@huawei.com>
Subject: Re: [PATCH v4 0/4] Implement vdpasim stop operation
Date: Fri, 27 May 2022 06:55:44 -0400	[thread overview]
Message-ID: <20220527065442-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <PH0PR12MB54819C6C6DAF6572AEADC1AEDCD99@PH0PR12MB5481.namprd12.prod.outlook.com>

On Thu, May 26, 2022 at 12:54:32PM +0000, Parav Pandit wrote:
> 
> 
> > From: Eugenio Pérez <eperezma@redhat.com>
> > Sent: Thursday, May 26, 2022 8:44 AM
> 
> > Implement stop operation for vdpa_sim devices, so vhost-vdpa will offer
> > 
> > that backend feature and userspace can effectively stop the device.
> > 
> > 
> > 
> > This is a must before get virtqueue indexes (base) for live migration,
> > 
> > since the device could modify them after userland gets them. There are
> > 
> > individual ways to perform that action for some devices
> > 
> > (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there
> > was no
> > 
> > way to perform it for any vhost device (and, in particular, vhost-vdpa).
> > 
> > 
> > 
> > After the return of ioctl with stop != 0, the device MUST finish any
> > 
> > pending operations like in flight requests. It must also preserve all
> > 
> > the necessary state (the virtqueue vring base plus the possible device
> > 
> > specific states) that is required for restoring in the future. The
> > 
> > device must not change its configuration after that point.
> > 
> > 
> > 
> > After the return of ioctl with stop == 0, the device can continue
> > 
> > processing buffers as long as typical conditions are met (vq is enabled,
> > 
> > DRIVER_OK status bit is enabled, etc).
> 
> Just to be clear, we are adding vdpa level new ioctl() that doesn’t map to any mechanism in the virtio spec.
> 
> Why can't we use this ioctl() to indicate driver to start/stop the device instead of driving it through the driver_ok?
> This is in the context of other discussion we had in the LM series.

If there's something in the spec that does this then let's use that.
Unfortunately the LM series seems to be stuck on moving
bits around with the admin virtqueue ...

-- 
MST