Re: [PATCH RFC v1 0/3] Introduce vfio-pci-core subsystem

From: Jason Gunthorpe <jgg@nvidia.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: Max Gurtovoy <mgurtovoy@nvidia.com>,
	Cornelia Huck <cohuck@redhat.com>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <liranl@nvidia.com>,
	<oren@nvidia.com>, <tzahio@nvidia.com>, <leonro@nvidia.com>,
	<yarong@nvidia.com>, <aviadye@nvidia.com>, <shahafs@nvidia.com>,
	<artemp@nvidia.com>, <kwankhede@nvidia.com>, <ACurrid@nvidia.com>,
	<gmataev@nvidia.com>, <cjia@nvidia.com>,
	Matthew Rosato <mjrosato@linux.ibm.com>
Subject: Re: [PATCH RFC v1 0/3] Introduce vfio-pci-core subsystem
Date: Mon, 1 Feb 2021 13:17:47 -0400	[thread overview]
Message-ID: <20210201171747.GL4247@nvidia.com> (raw)
In-Reply-To: <20210131213228.0e0573f4@x1.home.shazbot.org>

On Sun, Jan 31, 2021 at 09:32:28PM -0700, Alex Williamson wrote:

> > I think we can leave common arch specific stuff, such as s390 (IIUC) in 
> > the core driver. And only create vfio_pci drivers for 
> > vendor/device/subvendor specific stuff.
> 
> So on one hand you're telling us that the design principles here can be
> applied to various other device/platform specific support, but on the
> other you're saying, but don't do that...

The kernel code for the other three can be reworked to follow this
scheme, but because of the uABI we have established binding vfio_pci
is going to have to continue to automatically use the other drivers
for a long time.

For instance this can be accomplished by structuring the three drivers
in the new way and diverting the driver ops from vfio_pci to the other
drivers in a hardcoded way.

If this is worth doing from a maintability perspective vs just
continuing to hardwire the PCI IDs in vfio_pci, is something to look
at.

The point was it could work.

> > We can add a code to libvirt as mentioned above.
> 
> That's rather the question here, what is that algorithm by which a
> userspace tool such as libvirt would determine the optimal driver for a
> device?

Well, the PCI drivers do specify their PCI ids:

+static const struct pci_device_id mlx5_vfio_pci_table[] = {
+	{ PCI_VDEVICE(MELLANOX, 0x6001) }, /* NVMe SNAP controllers */
+	{ PCI_DEVICE_SUB(PCI_VENDOR_ID_REDHAT_QUMRANET, 0x1042,
+			 PCI_VENDOR_ID_MELLANOX, PCI_ANY_ID) }, /* Virtio SNAP controllers */
+	{ 0, }
+};

The problem in VFIO is that it doesn't have any way to create a VFIO
from a '/sys/device/blah/'. It expects userspace to know the module
name as part of the uAPI.

Generally speaking in the kernel we'd want to see some uAPI that was
'create a VFIO from /sys/device/blah' that does automatic module
loading and automatic driver selection.

For instance, by forming the '/sys/device/blah' to a normal modalias,
and then searching for the most specific VFIO modalias to handle the
device.

When implemented in netlink the whole thing is fully automatic, users
never have to know or care about any modules or their names. This is
the gold standard.

Jason