Re: [PATCH v2 08/14] vfio/pci: Re-order vfio_pci_probe()

From: Auger Eric <eric.auger@redhat.com>
To: Jason Gunthorpe <jgg@nvidia.com>, kvm@vger.kernel.org
Cc: Alex Williamson <alex.williamson@redhat.com>,
	"Raj, Ashok" <ashok.raj@intel.com>,
	Christian Ehrhardt <christian.ehrhardt@canonical.com>,
	Cornelia Huck <cohuck@redhat.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Daniel Vetter <daniel@ffwll.ch>, Christoph Hellwig <hch@lst.de>,
	Kevin Tian <kevin.tian@intel.com>,
	Leon Romanovsky <leonro@nvidia.com>,
	Max Gurtovoy <mgurtovoy@nvidia.com>,
	Tarun Gupta <targupta@nvidia.com>
Subject: Re: [PATCH v2 08/14] vfio/pci: Re-order vfio_pci_probe()
Date: Thu, 18 Mar 2021 17:50:26 +0100	[thread overview]
Message-ID: <c68db5c7-873f-595d-19f9-19d6b2bee4a5@redhat.com> (raw)
In-Reply-To: <8-v2-20d933792272+4ff-vfio1_jgg@nvidia.com>

Hi Jason,

On 3/13/21 1:56 AM, Jason Gunthorpe wrote:
> vfio_add_group_dev() must be called only after all of the private data in
> vdev is fully setup and ready, otherwise there could be races with user
> space instantiating a device file descriptor and starting to call ops.
> 
> For instance vfio_pci_reflck_attach() sets vdev->reflck and
> vfio_pci_open(), called by fops open, unconditionally derefs it, which
> will crash if things get out of order.>
> Fixes: cc20d7999000 ("vfio/pci: Introduce VF token")
> Fixes: e309df5b0c9e ("vfio/pci: Parallelize device open and release")
> Fixes: 6eb7018705de ("vfio-pci: Move idle devices to D3hot power state")
> Fixes: ecaa1f6a0154 ("vfio-pci: Add VGA arbiter client")
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  drivers/vfio/pci/vfio_pci.c | 17 +++++++++--------
>  1 file changed, 9 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
> index f95b58376156a0..0e7682e7a0b478 100644
> --- a/drivers/vfio/pci/vfio_pci.c
> +++ b/drivers/vfio/pci/vfio_pci.c
> @@ -2030,13 +2030,9 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>  	INIT_LIST_HEAD(&vdev->vma_list);
>  	init_rwsem(&vdev->memory_lock);
>  
> -	ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev);
> -	if (ret)
> -		goto out_free;
> -
>  	ret = vfio_pci_reflck_attach(vdev);
>  	if (ret)
> -		goto out_del_group_dev;
> +		goto out_free;
>  	ret = vfio_pci_vf_init(vdev);
>  	if (ret)
>  		goto out_reflck;
> @@ -2060,15 +2056,20 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>  		vfio_pci_set_power_state(vdev, PCI_D3hot);
>  	}
>  
> -	return ret;
> +	ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev);
> +	if (ret)
> +		goto out_power;
> +	return 0;
>  
> +out_power:
> +	if (!disable_idle_d3)
> +		vfio_pci_set_power_state(vdev, PCI_D0);
>  out_vf:
>  	vfio_pci_vf_uninit(vdev);
>  out_reflck:
>  	vfio_pci_reflck_put(vdev->reflck);
> -out_del_group_dev:
> -	vfio_del_group_dev(&pdev->dev);
>  out_free:
> +	kfree(vdev->pm_save);
>  	kfree(vdev);
>  out_group_put:
>  	vfio_iommu_group_put(group, &pdev->dev);
> 