Re: [PATCH net-next 00/19] Mellanox, mlx5 sub function support

[Jason Gunthorpe <jgg@ziepe.ca>]

On Fri, Nov 08, 2019 at 02:52:10PM -0700, Alex Williamson wrote:
> > > 
> > > Unless there's some opposition, I'm intended to queue this for v5.5:
> > > 
> > > https://www.spinics.net/lists/kvm/msg199613.html
> > > 
> > > mdev has started out as tied to vfio, but at it's core, it's just a
> > > device life cycle infrastructure with callbacks between bus drivers
> > > and vendor devices.  If virtio is on the wrong path with the above
> > > series, please speak up.  Thanks,  
> > 
> > Well, I think Greg just objected pretty strongly.
> > 
> > IMHO it is wrong to turn mdev into some API multiplexor. That is what
> > the driver core already does and AFAIK your bus type is supposed to
> > represent your API contract to your drivers.
> > 
> > Since the bus type is ABI, 'mdev' is really all about vfio I guess?
> > 
> > Maybe mdev should grow by factoring the special GUID life cycle stuff
> > into a helper library that can make it simpler to build proper API
> > specific bus's using that lifecycle model? ie the virtio I saw
> > proposed should probably be a mdev-virtio bus type providing this new
> > virtio API contract using a 'struct mdev_virtio'?
> 
> I see, the bus:API contract is more clear when we're talking about
> physical buses and physical devices following a hardware
> specification.

Well, I don't think it matters, this is a software contract inside the
kernel between the 'struct foo_device' (as provided by the foo_bus)
and the 'struct foo_driver'

This contract is certainly easier to define when a HW specification
dictates basically how it works.

> But if we take PCI for example, each PCI device has it's own internal
> API that operates on the bus API.  PCI bus drivers match devices based
> on vendor and device ID, which defines that internal API, not the bus
> API.  

Yes, this matching is part of the API contract between the bus and
device driver.

But all of the pci_* functions that accept a 'struct pci_device *' are
also part of this API contract toward the driver.

> The bus API is pretty thin when we're talking virtual devices and
> virtual buses though.  The bus "API" is essentially that lifecycle
> management, so I'm having a bit of a hard time differentiating this

But Parav just pointed out to a virtio SW API that had something like
20 API entry points.

> instead?"  Essentially for virtual devices, we're dictating a bus per
> device type, whereas it seemed like a reasonable idea at the time to

Well, what does a driver binding to a virtual device need to know?

The virtual device API should provide all of that information.

I think things like vfio and virtio APIs are very reasonable bus
types. virtio in particular has a published 'hw-like' specification
with some good layers that can build a bus API.

Not so sure about the very HW specific things like the Intel driver and
these SFs. These will really only ever bind to one driver and seem to
have no commonalities.

For those we either create a bus per driver-specific proprietary API
(feels kind of wrong) or we have a generic bus essentially for managed
multi-function hardware that uses a simple 'void *hw_data' as the
driver API and some matching logic to support that.

> create a common virtual device bus, but maybe it went into the weeds
> when trying to figure out how device drivers match to devices on that
> bus and actually interact with them.

I think it is important to focus on the the SW API the 'struct
foo_device' is supposed to provide toward the driver that binds to it.

It should be a sensible API covering some well defined area..

Jason