Re: [PATCH 09/12] PCI: Add a PCI_ID_F_VFIO_DRIVER_OVERRIDE flag to struct pci_device_id

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Jul 27, 2021 at 05:02:02PM -0600, Alex Williamson wrote:

> In general I think my confusion is lack of documentation and examples.
> There's good information here and in the cover letter, but reviewing
> the patch itself I'm not sure if vfio_pci: is meant to indicate the
> vfio_pci driver or the vfio_pci device api or as I've finally decided,
> just prepending "vfio_" to the modalias for a device to indicate the
> class of stuff, ie. no automatic binding but discoverable by userspace
> as a "vfio" driver suitable for this device.

Yes, the "vfio_" prefix is ment to be a generic prefix that any bus
type could use to signify the modalias entry is for the vfio flavour
of driver_override devices.

The userspace algorihtm is pretty simple.

1) Identify the sysfs path to the device:
  /sys/bus/pci/devices/0000:01:00.0/modalias

2) Get the modalias string from the kernel:
 $ cat /sys/bus/pci/devices/0000:01:00.0/modalias
pci:v000015B3d00001017sv000015B3sd00000001bc02sc00i00

3) Prefix it with vfio_:
vfio_pci:v000015B3d00001017sv000015B3sd00000001bc02sc00i00

4) Search modules.alias for the above string, select the entry that
   has the fewest *'s. See Max's sample script.

5) modprobe the matched module name

6) cat the matched modules.alias module name to
   /sys/bus/pci/devices/0000\:01\:00.0/driver_override

Further patches can make this work universally for all the current and
future vfio bus types, eg platform, fsl, etc.

Something like driverctl or libvirt can implement this algorithm and
remove all the hardwired behavior of load vfio_fsl for this or
vfio_pci for that.

I'll add the above sequence to the commit message of this patch, since
I think it makes it really clear.

> I think we need libvirt folks onboard and maybe a clearer idea what
> userspace helpers might be available.  For example would driverctl have
> an option to choose a vfio class driver for a device?

Max wrote a demo script that shows how this can work, it is linked in
the cover letter.

At the end of the day there are only two ideas that survived scrutiny:

1) This patch which makes everything dynamic and driven by
   modules.alias,

2) We continue to hardwire the driver and module names into
   libvirt/etc and just add mlx, hns, etc.

> I can also imagine that if the flag only covered the
> matching/driver_override aspect and pci_device_id further included an
> optional modalias prefix, we could do this without littering pci-core
> with vfio eccentricities.  I'll be interest to see Bjorn's thoughts on
> this.  Thanks,

This is more elegant, but we didn't do this because the pci match
struct is widely used in the kernel and bloating it further doesn't
seem to make a lot of sense at this point. Due to the macros it would
be easy to change to this scheme if was appropriate.

Jason