Re: Discussion about virtual iommu support for Xen guest

From: George Dunlap <george.dunlap@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Lan Tianyu <tianyu.lan@intel.com>,
	yang.zhang.wz@gmail.com, "Tian, Kevin" <kevin.tian@intel.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Nakajima, Jun" <jun.nakajima@intel.com>,
	"Dong, Eddie" <eddie.dong@intel.com>,
	Ian Jackson <ian.jackson@eu.citrix.com>,
	"xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>,
	Jan Beulich <jbeulich@suse.com>,
	Anthony Perard <anthony.perard@citrix.com>
Subject: Re: Discussion about virtual iommu support for Xen guest
Date: Tue, 31 May 2016 10:43:18 +0100	[thread overview]
Message-ID: <CAFLBxZZoFkMJjkMzH8_CLC3aDatMfip7fTb3R0iSBNhJWcQrBA@mail.gmail.com> (raw)
In-Reply-To: <5746DF93.8090803@citrix.com>

On Thu, May 26, 2016 at 12:35 PM, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> On 26/05/16 09:29, Lan Tianyu wrote:
>> Hi All:
>> We try pushing virtual iommu support for Xen guest and there are some
>> features blocked by it.
>>
>> Motivation:
>> -----------------------
>> 1) Add SVM(Shared Virtual Memory) support for Xen guest
>> To support iGFX pass-through for SVM enabled devices, it requires
>> virtual iommu support to emulate related registers and intercept/handle
>> guest SVM configure in the VMM.
>>
>> 2) Increase max vcpu support for one VM.
>>
>> So far, max vcpu for Xen hvm guest is 128. For HPC(High Performance
>> Computing) cloud computing, it requires more vcpus support in a single
>> VM. The usage model is to create just one VM on a machine with the
>> same number vcpus as logical cpus on the host and pin vcpu on each
>> logical cpu in order to get good compute performance.
>>
>> Intel Xeon phi KNL(Knights Landing) is dedicated to HPC market and
>> supports 288 logical cpus. So we hope VM can support 288 vcpu
>> to meet HPC requirement.
>>
>> Current Linux kernel requires IR(interrupt remapping) when MAX APIC
>> ID is > 255 because interrupt only can be delivered among 0~255 cpus
>> without IR. IR in VM relies on the virtual iommu support.
>>
>> KVM Virtual iommu support status
>> ------------------------
>> Current, Qemu has a basic virtual iommu to do address translation for
>> virtual device and it only works for the Q35 machine type. KVM reuses it
>> and Redhat is adding IR to support more than 255 vcpus.
>>
>> How to add virtual iommu for Xen?
>> -------------------------
>> First idea came to my mind is to reuse Qemu virtual iommu but Xen didn't
>> support Q35 so far. Enabling Q35 for Xen seems not a short term task.
>> Anthony did some related jobs before.
>>
>> I'd like to see your comments about how to implement virtual iommu for Xen.
>>
>> 1) Reuse Qemu virtual iommu or write a separate one for Xen?
>> 2) Enable Q35 for Xen to reuse Qemu virtual iommu?
>>
>> Your comments are very appreciated. Thanks a lot.
>
> To be viable going forwards, any solution must work with PVH/HVMLite as
> much as HVM.  This alone negates qemu as a viable option.

There's a big difference between "suboptimal" and "not viable".
Obviously it would be nice to be able to have HVMLite do graphics
pass-through, but if this functionality ends up being HVM-only, is
that really such a huge issue?

If as Paul seems to indicate, the extra work to get the functionality
in Xen isn't very large, then it's worth pursuing; but I don't think
we should take other options off the table.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel