[PATCH] VFIO: PCI: eliminate unnecessary overhead in vfio_pci_reflck_find

[PATCH] VFIO: PCI: eliminate unnecessary overhead in vfio_pci_reflck_find

[linmiaohe]

From: Miaohe Lin <linmiaohe@huawei.com>

The driver of the pci device may not equal to vfio_pci_driver,
but we fetch vfio_device from pci_dev unconditionally in func
vfio_pci_reflck_find. This overhead, such as the competition
of vfio.group_lock, can be eliminated by check pci_dev_driver
with vfio_pci_driver first.

Signed-off-by: Miaohe Lin <linmiaohe@huawei.com>
---
 drivers/vfio/pci/vfio_pci.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 379a02c36e37..1e21970543a6 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -1466,15 +1466,14 @@ static int vfio_pci_reflck_find(struct pci_dev *pdev, void *data)
 	struct vfio_device *device;
 	struct vfio_pci_device *vdev;
 
-	device = vfio_device_get_from_dev(&pdev->dev);
-	if (!device)
-		return 0;
-
 	if (pci_dev_driver(pdev) != &vfio_pci_driver) {
-		vfio_device_put(device);
 		return 0;
 	}
 
+	device = vfio_device_get_from_dev(&pdev->dev);
+	if (!device)
+		return 0;
+
 	vdev = vfio_device_data(device);
 
 	if (vdev->reflck) {
-- 
2.19.1

Re: [PATCH] VFIO: PCI: eliminate unnecessary overhead in vfio_pci_reflck_find

[Alex Williamson]

On Wed, 30 Oct 2019 18:57:10 +0800
linmiaohe <linmiaohe@huawei.com> wrote:

> From: Miaohe Lin <linmiaohe@huawei.com>
> 
> The driver of the pci device may not equal to vfio_pci_driver,
> but we fetch vfio_device from pci_dev unconditionally in func
> vfio_pci_reflck_find. This overhead, such as the competition
> of vfio.group_lock, can be eliminated by check pci_dev_driver
> with vfio_pci_driver first.
> 
> Signed-off-by: Miaohe Lin <linmiaohe@huawei.com>
> ---
>  drivers/vfio/pci/vfio_pci.c | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
> index 379a02c36e37..1e21970543a6 100644
> --- a/drivers/vfio/pci/vfio_pci.c
> +++ b/drivers/vfio/pci/vfio_pci.c
> @@ -1466,15 +1466,14 @@ static int vfio_pci_reflck_find(struct pci_dev *pdev, void *data)
>  	struct vfio_device *device;
>  	struct vfio_pci_device *vdev;
>  
> -	device = vfio_device_get_from_dev(&pdev->dev);
> -	if (!device)
> -		return 0;
> -
>  	if (pci_dev_driver(pdev) != &vfio_pci_driver) {
> -		vfio_device_put(device);
>  		return 0;
>  	}
>  
> +	device = vfio_device_get_from_dev(&pdev->dev);
> +	if (!device)
> +		return 0;
> +
>  	vdev = vfio_device_data(device);
>  
>  	if (vdev->reflck) {

I believe this introduces a race.  When we hold a reference to the vfio
device, an unbind from a vfio bus driver will be blocked in
vfio_del_group_dev().  Therefore if we test the driver is vfio-pci
while holding this reference, we know that it cannot be released and
the device_data is a valid vfio_pci_device.  Testing the driver prior
to acquiring a vfio device reference is meaningless as we have no
guarantee that the driver has not changed by the time we acquire a
reference.  Are you actually seeing contention here or was this a code
inspection optimization?  Thanks,

Alex

Re: [PATCH] VFIO: PCI: eliminate unnecessary overhead in vfio_pci_reflck_find

[linmiaohe]

On Wed, 6 Nov 2019 03:25
Alex Williamson wrote
>> From: Miaohe Lin <linmiaohe@huawei.com>
>> 
>> The driver of the pci device may not equal to vfio_pci_driver, but we 
>> fetch vfio_device from pci_dev unconditionally in func 
>> vfio_pci_reflck_find. This overhead, such as the competition of 
>> vfio.group_lock, can be eliminated by check pci_dev_driver with 
>> vfio_pci_driver first.
>> 
>
>I believe this introduces a race.  When we hold a reference to the vfio device, an unbind from a vfio bus driver will be blocked in vfio_del_group_dev().  Therefore if we test the driver is vfio-pci while holding this reference, we know that it cannot be released and the device_data is a valid vfio_pci_device. Testing the driver prior to acquiring a vfio device reference is meaningless as we have no guarantee that the driver has not changed by the time we acquire a reference.  Are you actually seeing contention here or was this a code inspection optimization?  Thanks,
>
>Alex

Thanks for your reply and patient explanation.  It was a code inspection optimization, and obviously, I missed something. I'm really sorry about that.

Best wishes.
Have a nice day.