Re: [PATCH v7 12/17] vfio-user: IOMMU support for remote device

[Peter Xu <peterx@redhat.com>]

On Wed, Mar 30, 2022 at 11:04:24AM +0100, Stefan Hajnoczi wrote:
> This makes me wonder whether there is a deeper issue with the
> pci_setup_iommu() API: the lack of per-device cleanup callbacks.
> Per-device IOMMU resources should be freed when a device is hot
> unplugged.
> 
> From what I can tell this is not the case today:
> 
> - hw/i386/intel_iommu.c:vtd_find_add_as() allocates and adds device
>   address spaces but I can't find where they are removed and freed.
>   VTDAddressSpace instances pointed to from vtd_bus->dev_as[] are leaked.
> 
> - hw/i386/amd_iommu.c has similar leaks.

AFAICT it's because there's no device-specific data cached in the
per-device IOMMU address space, at least so far.  IOW, all the data
structures allocated here can be re-used when a new device is plugged in
after the old device unplugged.

It's definitely not ideal since after unplug (and before a new device
plugged in) the resource is not needed at all so it's kind of wasted, but
it should work functionally.  If to achieve that, some iommu_unplug() or
iommu_cleanup() hook sounds reasonable.

One thing I'm not sure is these iommu ops are per-bus not per-device.  So
I'm not sure whether that's what we wanted here because remote device
cleanup seems to be per-device only.

Thanks,

-- 
Peter Xu