From: Jike Song <jike.song@intel.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
"igvt-g@ml01.01.org" <igvt-g@ml01.01.org>,
"intel-gfx@lists.freedesktop.org"
<intel-gfx@lists.freedesktop.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"White, Michael L" <michael.l.white@intel.com>,
"Dong, Eddie" <eddie.dong@intel.com>,
"Li, Susie" <susie.li@intel.com>,
"Cowperthwaite, David J" <david.j.cowperthwaite@intel.com>,
"Reddy, Raghuveer" <raghuveer.reddy@intel.com>,
"Zhu, Libo" <libo.zhu@intel.com>,
"Zhou, Chao" <chao.zhou@intel.com>,
"Wang, Hongbo" <hongbo.wang@intel.com>,
"Lv, Zhiyuan" <zhiyuan.lv@intel.com>,
qemu-devel <qemu-devel@nongnu.org>,
Paolo Bonzini <pbonzini@redhat.com>,
Gerd Hoffmann <kraxel@redhat.com>
Subject: Re: [Intel-gfx] [Announcement] 2015-Q3 release of XenGT - a Mediated Graphics Passthrough Solution from Intel
Date: Mon, 23 Nov 2015 13:05:47 +0800 [thread overview]
Message-ID: <56529EAB.4030007@intel.com> (raw)
In-Reply-To: <1448040302.4697.300.camel@redhat.com>
On 11/21/2015 01:25 AM, Alex Williamson wrote:
> On Fri, 2015-11-20 at 08:10 +0000, Tian, Kevin wrote:
>>
>> Here is a more concrete example:
>>
>> KVMGT doesn't require IOMMU. All DMA targets are already replaced with
>> HPA thru shadow GTT. So DMA requests from GPU all contain HPAs.
>>
>> When IOMMU is enabled, one simple approach is to have vGPU IOMMU
>> driver configure system IOMMU with identity mapping (HPA->HPA). We
>> can't use (GPA->HPA) since GPAs from multiple VMs are conflicting.
>>
>> However, we still have host gfx driver running. When IOMMU is enabled,
>> dma_alloc_*** will return IOVA (drvers/iommu/iova.c) in host gfx driver,
>> which will have IOVA->HPA programmed to system IOMMU.
>>
>> One IOMMU device entry can only translate one address space, so here
>> comes a conflict (HPA->HPA vs. IOVA->HPA). To solve this, vGPU IOMMU
>> driver needs to allocate IOVA from iova.c for each VM w/ vGPU assigned,
>> and then KVMGT will program IOVA in shadow GTT accordingly. It adds
>> one additional mapping layer (GPA->IOVA->HPA). In this way two
>> requirements can be unified together since only IOVA->HPA mapping
>> needs to be built.
>>
>> So unlike existing type1 IOMMU driver which controls IOMMU alone, vGPU
>> IOMMU driver needs to cooperate with other agent (iova.c here) to
>> co-manage system IOMMU. This may not impact existing VFIO framework.
>> Just want to highlight additional work here when implementing the vGPU
>> IOMMU driver.
>
> Right, so the existing i915 driver needs to use the DMA API and calls
> like dma_map_page() to enable translations through the IOMMU. With
> dma_map_page(), the caller provides a page address (~HPA) and is
> returned an IOVA. So unfortunately you don't get to take the shortcut
> of having an identity mapping through the IOMMU unless you want to
> convert i915 entirely to using the IOMMU API, because we also can't have
> the conflict that an HPA could overlap an IOVA for a previously mapped
> page.
>
> The double translation, once through the GPU MMU and once through the
> system IOMMU is going to happen regardless of whether we can identity
> map through the IOMMU. The only solution to this would be for the GPU
> to participate in ATS and provide pre-translated transactions from the
> GPU. All of this is internal to the i915 driver (or vfio extension of
> that driver) and needs to be done regardless of what sort of interface
> we're using to expose the vGPU to QEMU. It just seems like VFIO
> provides a convenient way of doing this since you'll have ready access
> to the HVA-GPA mappings for the user.
>
> I think the key points though are:
>
> * the VFIO type1 IOMMU stores GPA to HVA translations
> * get_user_pages() on the HVA will pin the page and give you a
> page
> * dma_map_page() receives that page, programs the system IOMMU and
> provides an IOVA
> * the GPU MMU can then be programmed with the GPA to IOVA
> translations
Thanks for such a nice example! I'll do my home work and get back to you
shortly :)
>
> Thanks,
> Alex
>
--
Thanks,
Jike
next prev parent reply other threads:[~2015-11-23 5:06 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-07-25 8:31 [Intel-gfx] [Announcement] Updates to XenGT - a Mediated Graphics Passthrough Solution from Intel Jike Song
2014-07-29 10:09 ` [Xen-devel] " Dario Faggioli
2014-07-30 9:39 ` Jike Song
2014-12-04 2:45 ` [Intel-gfx] [Announcement] 2014-Q3 release of " Jike Song
2014-12-04 10:20 ` [Xen-devel] " Fabio Fantoni
2015-01-09 8:51 ` [Intel-gfx] [Announcement] 2015-Q1 " Jike Song
2015-01-12 3:04 ` [Intel-gfx] [Announcement] 2014-Q4 " Jike Song
2015-04-10 13:23 ` [Intel-gfx] [Announcement] 2015-Q1 " Jike Song
2015-07-07 2:49 ` [Intel-gfx] [Announcement] 2015-Q2 " Jike Song
2015-10-27 9:25 ` [Intel-gfx] [Announcement] 2015-Q3 " Jike Song
2015-11-18 18:12 ` Alex Williamson
2015-11-19 4:06 ` Tian, Kevin
2015-11-19 7:22 ` Jike Song
2015-11-19 15:32 ` Stefano Stabellini
2015-11-19 15:49 ` Paolo Bonzini
2015-11-19 16:12 ` Stefano Stabellini
2015-11-19 15:52 ` Alex Williamson
2015-11-20 2:58 ` Jike Song
2015-11-20 4:22 ` Alex Williamson
2015-11-20 5:51 ` Jike Song
2015-11-20 6:01 ` Tian, Kevin
2015-11-20 16:40 ` Alex Williamson
2015-11-23 4:52 ` [Qemu-devel] " Jike Song
2015-11-19 8:40 ` Gerd Hoffmann
2015-11-19 11:09 ` Paolo Bonzini
2015-11-20 2:46 ` Jike Song
2015-11-20 6:12 ` Tian, Kevin
2015-11-20 8:26 ` Gerd Hoffmann
2015-11-20 8:36 ` Tian, Kevin
2015-11-20 8:46 ` Zhiyuan Lv
2015-12-03 6:57 ` Tian, Kevin
2015-12-04 10:13 ` Gerd Hoffmann
2015-11-19 20:02 ` Alex Williamson
2015-11-20 7:09 ` Tian, Kevin
2015-11-20 17:03 ` Alex Williamson
2015-11-20 8:10 ` Tian, Kevin
2015-11-20 17:25 ` Alex Williamson
2015-11-23 5:05 ` Jike Song [this message]
2015-11-24 11:19 ` Daniel Vetter
2015-11-24 11:49 ` Chris Wilson
2015-11-24 12:38 ` Gerd Hoffmann
2015-11-24 13:31 ` Daniel Vetter
2015-11-24 14:12 ` Gerd Hoffmann
2015-11-24 14:19 ` Daniel Vetter
2016-01-27 6:21 ` [Intel-gfx] [Announcement] 2015-Q4 " Jike Song
2016-04-28 5:29 ` [Intel-gfx] [Announcement] 2016-Q1 " Jike Song
2016-07-22 5:42 ` [Intel-gfx] [Announcement] 2016-Q2 " Jike Song
2016-11-06 14:59 ` [Intel-gfx] [Announcement] 2016-Q3 " Jike Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56529EAB.4030007@intel.com \
--to=jike.song@intel.com \
--cc=alex.williamson@redhat.com \
--cc=chao.zhou@intel.com \
--cc=david.j.cowperthwaite@intel.com \
--cc=eddie.dong@intel.com \
--cc=hongbo.wang@intel.com \
--cc=igvt-g@ml01.01.org \
--cc=intel-gfx@lists.freedesktop.org \
--cc=kevin.tian@intel.com \
--cc=kraxel@redhat.com \
--cc=libo.zhu@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=michael.l.white@intel.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=raghuveer.reddy@intel.com \
--cc=susie.li@intel.com \
--cc=xen-devel@lists.xen.org \
--cc=zhiyuan.lv@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).