Re: [PATCH 8/9] vfio/pci: export nvlink2 support into vendor vfio_pci drivers

From: Max Gurtovoy <mgurtovoy@nvidia.com>
To: Alexey Kardashevskiy <aik@ozlabs.ru>, Jason Gunthorpe <jgg@nvidia.com>
Cc: <alex.williamson@redhat.com>, <cohuck@redhat.com>,
	<kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<liranl@nvidia.com>, <oren@nvidia.com>, <tzahio@nvidia.com>,
	<leonro@nvidia.com>, <yarong@nvidia.com>, <aviadye@nvidia.com>,
	<shahafs@nvidia.com>, <artemp@nvidia.com>, <kwankhede@nvidia.com>,
	<ACurrid@nvidia.com>, <cjia@nvidia.com>, <yishaih@nvidia.com>,
	<mjrosato@linux.ibm.com>, <hch@lst.de>
Subject: Re: [PATCH 8/9] vfio/pci: export nvlink2 support into vendor vfio_pci drivers
Date: Thu, 11 Mar 2021 11:44:38 +0200	[thread overview]
Message-ID: <d2c32828-3417-1872-6451-2450e6fa763d@nvidia.com> (raw)
In-Reply-To: <73c99da0-6624-7aa2-2857-ef68092c0d07@ozlabs.ru>

On 3/11/2021 9:54 AM, Alexey Kardashevskiy wrote:
>
>
> On 11/03/2021 13:00, Jason Gunthorpe wrote:
>> On Thu, Mar 11, 2021 at 12:42:56PM +1100, Alexey Kardashevskiy wrote:
>>>>> btw can the id list have only vendor ids and not have device ids?
>>>>
>>>> The PCI matcher is quite flexable, see the other patch from Max for
>>>> the igd
>>>   ah cool, do this for NVIDIA GPUs then please, I just discovered 
>>> another P9
>>> system sold with NVIDIA T4s which is not in your list.
>>
>> I think it will make things easier down the road if you maintain an
>> exact list <shrug>
>
>
> Then why do not you do the exact list for Intel IGD? The commit log 
> does not explain this detail.

I expect Intel team to review this series and give a more precise list.

I did the best I could in finding a proper configuration for igd.

>
>
>>>> But best practice is to be as narrow as possible as I hope this will
>>>> eventually impact module autoloading and other details.
>>>
>>> The amount of device specific knowledge is too little to tie it up 
>>> to device
>>> ids, it is a generic PCI driver with quirks. We do not have a separate
>>> drivers for the hardware which requires quirks.
>>
>> It provides its own capability structure exposed to userspace, that is
>> absolutely not a "quirk"
>>
>>> And how do you hope this should impact autoloading?
>>
>> I would like to autoload the most specific vfio driver for the target
>> hardware.
>
>
> Is there an idea how it is going to work? For example, the Intel IGD 
> driver and vfio-pci-igd - how should the system pick one? If there is 
> no MODULE_DEVICE_TABLE in vfio-pci-xxx, is the user supposed to try 
> binding all vfio-pci-xxx drivers until some binds?

For example, in my local setup I did a POC patch that convert some 
drivers to be "manual binding only drivers".

So the IGD driver will have the priority, user will unbind the device 
from it, load igd-vfio-pci, bind the device to it, ends with probing.

For now we separated the driver core stuff until we all agree that this 
series is the right way to go + we also make sure it's backward compatible.

>
>
>> If you someday need to support new GPU HW that needs a different VFIO
>> driver then you are really stuck because things become indeterminate
>> if there are two devices claiming the ID. We don't have the concept of
>> "best match", driver core works on exact match.
>
>
>