RE: [PATCH v6 00/20] Add VFIO mediated device support and DEV-MSI support for the idxd driver

From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>, "Jiang, Dave" <dave.jiang@intel.com>
Cc: "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
	"kwankhede@nvidia.com" <kwankhede@nvidia.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"vkoul@kernel.org" <vkoul@kernel.org>,
	"Raj, Ashok" <ashok.raj@intel.com>,
	"Dey, Megha" <megha.dey@intel.com>,
	"Pan, Jacob jun" <jacob.jun.pan@intel.com>,
	"Liu, Yi L" <yi.l.liu@intel.com>,
	"Lu, Baolu" <baolu.lu@intel.com>,
	"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
	"Luck, Tony" <tony.luck@intel.com>,
	"Williams, Dan J" <dan.j.williams@intel.com>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>,
	"pbonzini@redhat.com" <pbonzini@redhat.com>,
	"dmaengine@vger.kernel.org" <dmaengine@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>
Subject: RE: [PATCH v6 00/20] Add VFIO mediated device support and DEV-MSI support for the idxd driver
Date: Thu, 3 Jun 2021 01:11:37 +0000	[thread overview]
Message-ID: <MWHPR11MB188664D9E7CA60782B75AC718C3C9@MWHPR11MB1886.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20210602231747.GK1002214@nvidia.com>

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Thursday, June 3, 2021 7:18 AM
> 
> On Wed, Jun 02, 2021 at 08:40:51AM -0700, Dave Jiang wrote:
> >
> > On 5/23/2021 4:22 PM, Jason Gunthorpe wrote:
> > > On Fri, May 21, 2021 at 05:19:05PM -0700, Dave Jiang wrote:
> > > > Introducing mdev types “1dwq-v1” type. This mdev type allows
> > > > allocation of a single dedicated wq from available dedicated wqs. After
> > > > a workqueue (wq) is enabled, the user will generate an uuid. On mdev
> > > > creation, the mdev driver code will find a dwq depending on the mdev
> > > > type. When the create operation is successful, the user generated uuid
> > > > can be passed to qemu. When the guest boots up, it should discover a
> > > > DSA device when doing PCI discovery.
> > > >
> > > > For example of “1dwq-v1” type:
> > > > 1. Enable wq with “mdev” wq type
> > > > 2. A user generated uuid.
> > > > 3. The uuid is written to the mdev class sysfs path:
> > > > echo $UUID >
> /sys/class/mdev_bus/0000\:00\:0a.0/mdev_supported_types/idxd-1dwq-
> v1/create
> > > > 4. Pass the following parameter to qemu:
> > > > "-device vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:0a.0/$UUID"
> > > So the idxd core driver knows to create a "vfio" wq with its own much
> > > machinery but you still want to involve the horrible mdev guid stuff?
> > >
> > > Why??
> >
> > Are you referring to calling mdev_device_create() directly in the mdev
> > idxd_driver probe?
> 
> No, just call vfio_register_group_dev and forget about mdev.
> 
> > I think this would work with our dedicated wq where a single mdev
> > can be assigned to a wq.
> 
> Ok, sounds great
> 
> > However, later on when we need to support shared wq where we can
> > create multiple mdev per wq, we'll need an entry point to do so. In
> > the name of making things consistent from user perspective, going
> > through sysfs seems the way to do it.
> 
> Why not use your already very complicated idxd sysfs to do this?
> 

Jason, can you clarify your attitude on mdev guid stuff? Are you 
completely against it or case-by-case? If the former, this is a big
decision thus it's better to have consensus with Alex/Kirti. If the
latter, would like to hear your criteria for when it can be used
and when not...

Thanks
Kevin