Re: [PATCH v6 13/20] vfio/mdev: idxd: add mdev driver registration and helper functions

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, May 21, 2021 at 05:20:26PM -0700, Dave Jiang wrote:

> +static int idxd_vdcm_probe(struct mdev_device *mdev)
> +{
> +	struct vdcm_idxd *vidxd;
> +	struct vdcm_idxd_type *type;
> +	struct device *dev, *parent;
> +	struct idxd_device *idxd;
> +	bool ims_map[VIDXD_MAX_MSIX_VECS];
> +	int rc;
> +
> +	parent = mdev_parent_dev(mdev);
> +	idxd = dev_get_drvdata(parent);
> +	dev = &mdev->dev;
> +	mdev_set_iommu_device(mdev, parent);
> +	type = idxd_vdcm_get_type(mdev);

This makes my head hurt. There is a kref guarding
mdev_unregister_device() but probe reaches into the parent idxd
device's drvdata? I'm skeptical any of this is locked right

> +static void idxd_vdcm_remove(struct mdev_device *mdev)
> +{
> +	struct vdcm_idxd *vidxd = dev_get_drvdata(&mdev->dev);
> +	struct idxd_wq *wq = vidxd->wq;
> +
> +	vfio_unregister_group_dev(&vidxd->vdev);
> +	mdev_irqs_free(mdev);
> +	mutex_lock(&wq->wq_lock);
> +	idxd_wq_put(wq);
> +	mutex_unlock(&wq->wq_lock);

It is also really weird to see something called put that requires the
caller to hold a mutex... Don't use refcount language for something
tha tis not acting like any sort of refcount.

> +static int idxd_vdcm_open(struct vfio_device *vdev)
> +{
> +	return 0;
> +}
> +
> +static void idxd_vdcm_close(struct vfio_device *vdev)
> +{
> +	struct vdcm_idxd *vidxd = vdev_to_vidxd(vdev);
> +
> +	mutex_lock(&vidxd->dev_lock);
> +	idxd_vdcm_set_irqs(vidxd, VFIO_IRQ_SET_DATA_NONE | VFIO_IRQ_SET_ACTION_TRIGGER,
> +			   VFIO_PCI_MSIX_IRQ_INDEX, 0, 0, NULL);
> +
> +	/* Re-initialize the VIDXD to a pristine state for re-use */
> +	vidxd_init(vidxd);
> +	mutex_unlock(&vidxd->dev_lock);

This is split up weird. open should be doing basic init stuff and
close should just be doing the reset stuff..

Jason