Re: [PATCH PROTOTYPE 3/6] vfio: Implement support for sparse RAM memory regions

[Peter Xu <peterx@redhat.com>]

On Wed, Nov 18, 2020 at 05:14:22PM +0100, David Hildenbrand wrote:
> That did the trick! Thanks!!!

Great!  At the meantime, I've a few questions majorly about memory unplugging
below, which could be naive - I know little on that, please bare with me.. :)

> 
> virtio-mem + vfio + iommu seems to work. More testing to be done.
> 
> However, malicious guests can play nasty tricks like
> 
> a) Unplugging plugged virtio-mem blocks while they are mapped via an
>    IOMMU
> 
> 1. Guest: map memory location X located on a virtio-mem device inside a
>    plugged block into the IOMMU
>    -> QEMU IOMMU notifier: create vfio DMA mapping
>    -> VFIO pins memory of unplugged blocks (populating memory)
> 2. Guest: Request to unplug memory location X via virtio-mem device
>    -> QEMU virtio-mem: discards the memory.
>    -> VFIO still has the memory pinned

When unplug some memory, does the user need to first do something to notify the
guest kernel that "this memory is going to be unplugged soon" (assuming echo
"offline" to some dev file)?  Then the kernel should be responsible to prepare
for that before it really happens, e.g., migrate anonymous pages out from this
memory block.  I don't know what would happen if some pages on the memblock
were used for DMA like this and we want to unplug it.  Ideally I thought it
should fail the "echo offline" operation with something like EBUSY if it can't
notify the device driver about this, or it's hard to.

IMHO this question not really related to vIOMMU, but a general question for
unplugging. Say, what would happen if we unplug some memory with DMA buffers
without vIOMMU at all?  The buffer will be invalid right after unplugging, so
the guest kernel should either fail the operation trying to unplug, or at least
tell the device drivers about this somehow?

> 
> We consume more memory than intended. In case virtio-memory would get
> replugged and used, we would have an inconsistency. IOMMU device resets/ fix
> it (whereby all VFIO mappings are removed via the IOMMU notifier).
> 
> 
> b) Mapping unplugged virtio-mem blocks via an IOMMU
> 
> 1. Guest: map memory location X located on a virtio-mem device inside an
>    unplugged block
>    -> QEMU IOMMU notifier: create vfio DMA mapping
>    -> VFIO pins memory of unplugged blocks (populating memory)

For this case, I would expect vfio_get_xlat_addr() to fail directly if the
guest driver force to map some IOVA onto an invalid range of the virtio-mem
device.  Even before that, since the guest should know that this region of
virtio-mem is not valid since unplugged, so shouldn't the guest kernel directly
fail the dma_map() upon such a region even before the mapping message reaching
QEMU?

Thanks,

> 
> Memory that's supposed to be discarded now consumes memory. This is similar
> to a malicious guest simply writing to unplugged memory blocks (to be
> tackled with "protection of unplugged memory" in the future) - however
> memory will also get pinned.
> 
> 
> To prohibit b) from happening, we would have to disallow creating the VFIO
> mapping (fairly easy).
> 
> To prohibit a), there would have to be some notification to IOMMU
> implementations to unmap/refresh whenever an IOMMU entry still points at
> memory that is getting discarded (and the VM is doing something it's not
> supposed to do).

-- 
Peter Xu