Re: Discussion about virtual iommu support for Xen guest

From: "Jan Beulich" <JBeulich@suse.com>
To: Kevin Tian <kevin.tian@intel.com>
Cc: "yang.zhang.wz@gmail.com" <yang.zhang.wz@gmail.com>,
	Tianyu Lan <tianyu.lan@intel.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Eddie Dong <eddie.dong@intel.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	"ian.jackson@eu.citrix.com" <ian.jackson@eu.citrix.com>,
	"xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>,
	Jun Nakajima <jun.nakajima@intel.com>,
	"anthony.perard@citrix.com" <anthony.perard@citrix.com>,
	Roger Pau Monne <roger.pau@citrix.com>
Subject: Re: Discussion about virtual iommu support for Xen guest
Date: Tue, 07 Jun 2016 01:26:54 -0600	[thread overview]
Message-ID: <5756935E02000078000F26F5@prv-mh.provo.novell.com> (raw)
In-Reply-To: <AADFC41AFE54684AB9EE6CBC0274A5D15F89160A@SHSMSX101.ccr.corp.intel.com>

>>> On 07.06.16 at 07:14, <kevin.tian@intel.com> wrote:
> After some internal discussion with Tianyu/Eddie, I realized my earlier
> description is incomplete which takes only passthrough device into
> consideration (as you saw it's mainly around interaction between vIOMMU
> and pIOMMU). However from guest p.o.v, all the devices should be covered
> by vIOMMU to match today's physical platform, including:
> 
> 1) DMA-capable virtual device in Qemu, in Dom0 user space
> 2) PV devices, in Dom0 kernel space
> 3) Passthrough devices, in Xen hypervisor
> 
> A natural implementation is to have vIOMMU together with where the
> DMA is emulated, which ends up to a possible way with multiple vIOMMUs
> in multiple layers:
> 
> 1) vIOMMU in Dom0 user
> 2) vIOMMU in Dom0 kernel
> 3) vIOMMU in Xen hypervisor
> 
> Of course we may come up an option to still keep all vIOMMUs in Xen
> hypervisor, which however means every vDMA operations in Qemu or
> BE driver need to issue Xen hypercall to get vIOMMU's approval. I haven't
> thought thoroughly how big/complex this issue is, but it does be a
> limitation from a quick thought.
> 
> So, likely we'll have to consider presence of multiple vIOMMUs, each in 
> different layers, regardless of root-complex in Qemu or Xen. There
> needs to be some interface abstractions to allow vIOMMU/root-complex
> communicating with each other. Well, not an easy task...

Right - for DMA-capable devices emulated in qemu, it would seem
natural to have them go through a vIOMMU in qemu. Whether
that vIOMMU implementation would have to consult the hypervisor
(or perhaps even just be a wrapper around various hypercalls, i.e.
backed by an implementation in the hypervisor) would be an
independent aspect.

Otoh, having vIOMMU in only qemu, and requiring round trips
through qemu for any of the hypervisor's internal purposes doesn't
seem like a good idea to me.

And finally I don't see the relevance of PV devices here: Their
nature makes it that they could easily be left completely independent
of an vIOMMU (as long as there's no plan to bypass a virtualization
level in the nested case, i.e. a PV frontend in L2 with a backend
living in L0).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel