Re: [PATCH v5 05/14] vfio/mdev: idxd: add basic mdev registration and helper functions

[Dan Williams <dan.j.williams@intel.com>]

On Tue, Feb 16, 2021 at 11:05 AM Dave Jiang <dave.jiang@intel.com> wrote:
>
>
> On 2/10/2021 4:59 PM, Jason Gunthorpe wrote:
> > On Fri, Feb 05, 2021 at 01:53:24PM -0700, Dave Jiang wrote:
> >
> >> +static int check_vma(struct idxd_wq *wq, struct vm_area_struct *vma)
> >>   {
> >> -    /* FIXME: Fill in later */
> >> +    if (vma->vm_end < vma->vm_start)
> >> +            return -EINVAL;
> > These checks are redundant
>
> Thanks. Will remove.
>
> >
> >> -static int idxd_mdev_host_release(struct idxd_device *idxd)
> >> +static int idxd_vdcm_mmap(struct mdev_device *mdev, struct vm_area_struct *vma)
> >> +{
> >> +    unsigned int wq_idx, rc;
> >> +    unsigned long req_size, pgoff = 0, offset;
> >> +    pgprot_t pg_prot;
> >> +    struct vdcm_idxd *vidxd = mdev_get_drvdata(mdev);
> >> +    struct idxd_wq *wq = vidxd->wq;
> >> +    struct idxd_device *idxd = vidxd->idxd;
> >> +    enum idxd_portal_prot virt_portal, phys_portal;
> >> +    phys_addr_t base = pci_resource_start(idxd->pdev, IDXD_WQ_BAR);
> >> +    struct device *dev = mdev_dev(mdev);
> >> +
> >> +    rc = check_vma(wq, vma);
> >> +    if (rc)
> >> +            return rc;
> >> +
> >> +    pg_prot = vma->vm_page_prot;
> >> +    req_size = vma->vm_end - vma->vm_start;
> >> +    vma->vm_flags |= VM_DONTCOPY;
> >> +
> >> +    offset = (vma->vm_pgoff << PAGE_SHIFT) &
> >> +             ((1ULL << VFIO_PCI_OFFSET_SHIFT) - 1);
> >> +
> >> +    wq_idx = offset >> (PAGE_SHIFT + 2);
> >> +    if (wq_idx >= 1) {
> >> +            dev_err(dev, "mapping invalid wq %d off %lx\n",
> >> +                    wq_idx, offset);
> >> +            return -EINVAL;
> >> +    }
> >> +
> >> +    /*
> >> +     * Check and see if the guest wants to map to the limited or unlimited portal.
> >> +     * The driver will allow mapping to unlimited portal only if the the wq is a
> >> +     * dedicated wq. Otherwise, it goes to limited.
> >> +     */
> >> +    virt_portal = ((offset >> PAGE_SHIFT) & 0x3) == 1;
> >> +    phys_portal = IDXD_PORTAL_LIMITED;
> >> +    if (virt_portal == IDXD_PORTAL_UNLIMITED && wq_dedicated(wq))
> >> +            phys_portal = IDXD_PORTAL_UNLIMITED;
> >> +
> >> +    /* We always map IMS portals to the guest */
> >> +    pgoff = (base + idxd_get_wq_portal_full_offset(wq->id, phys_portal,
> >> +                                                   IDXD_IRQ_IMS)) >> PAGE_SHIFT;
> >> +    dev_dbg(dev, "mmap %lx %lx %lx %lx\n", vma->vm_start, pgoff, req_size,
> >> +            pgprot_val(pg_prot));
> >> +    vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
> >> +    vma->vm_private_data = mdev;
> > What ensures the mdev pointer is valid strictly longer than the VMA?
> > This needs refcounting.
>
> Going to take a kref at open and then put_device at close. Does that
> sound reasonable or should I be calling get_device() in mmap() and then
> register a notifier for when vma is released?

Where does this enabling ever look at vm_private_data again? It seems
to me it should be reasonable for the mdev to die out from underneath
a vma, just need some tracking to block future uses of the
vma->vm_private_data from being attempted.