Re: [RFC PATCH 00/45] KVM: Arm SMMUv3 driver for pKVM

[Jean-Philippe Brucker <jean-philippe@linaro.org>]

On Thu, Feb 02, 2023 at 07:07:55AM +0000, Tian, Kevin wrote:
> > From: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > Sent: Wednesday, February 1, 2023 8:53 PM
> > 
> > 3. Private I/O page tables
> > 
> > A flexible alternative uses private page tables in the SMMU, entirely
> > disconnected from the CPU page tables. With this the SMMU can implement
> > a
> > reduced set of features, even shed a stage of translation. This also
> > provides a virtual I/O address space to the host, which allows more
> > efficient memory allocation for large buffers, and for devices with
> > limited addressing abilities.
> > 
> > This is the solution implemented in this series. The host creates
> > IOVA->HPA mappings with two hypercalls map_pages() and unmap_pages(),
> > and
> > the hypervisor populates the page tables. Page tables are abstracted into
> > IOMMU domains, which allow multiple devices to share the same address
> > space. Another four hypercalls, alloc_domain(), attach_dev(), detach_dev()
> > and free_domain(), manage the domains.
> > 
> 
> Out of curiosity. Does virtio-iommu fit in this usage?

I don't think so, because you still need a driver for the physical IOMMU
in the hypervisor. virtio-iommu would only replace the hypercall interface
with queues, and I don't think that buys us anything.

Maybe virtio on the guest side could be advantageous, because that
interface has to be stable and virtio comes with stable APIs for several
classes of devices. But implementing virtio in pkvm means a lot of extra
code so it needs to be considered carefully.

> If yes then there is
> no need to add specific enlightenment in existing iommu drivers. If no 
> probably because as mentioned in the start a full-fledged iommu driver 
> doesn't fit nVHE so lots of smmu driver logic has to be kept in the host?

To minimize the attack surface of the hypervisor, we don't want to load
any superfluous code, so the hypervisor part of the SMMUv3 driver only
contains code to populate tables and send commands (which is still too
much for my taste but seems unavoidable to isolate host DMA). Left in the
host are things like ACPI/DT parser, interrupts, possibly the event queue
(which informs of DMA errors), extra features and complex optimizations.
The host also has to implement IOMMU ops to liaise between the DMA API and
the hypervisor.

> anyway just want to check your thoughts on the possibility.
> 
> btw some of my colleagues are porting pKVM to Intel platform. I believe
> they will post their work shortly and there might require some common
> framework in pKVM hypervisor like iommu domain, hypercalls, etc. like 
> what we have in the host iommu subsystem. CC them in case of any early
> thought they want to throw in. 😊

Cool! The hypervisor part contains iommu/iommu.c which deals with
hypercalls and domains and doesn't contain anything specific to Arm (it's
only in arch/arm64 because that's where pkvm currently sits). It does rely
on io-pgtable at the moment which is not used by VT-d but that can be
abstracted as well. It's possible however that on Intel an entirely
different set of hypercalls will be needed, if a simpler solution such as
sharing page tables fits better because VT-d implementations are more
homogeneous.

Thanks,
Jean