* [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest @ 2017-04-05 2:08 Xiong Zhang 2017-04-05 2:19 ` ✓ Fi.CI.BAT: success for drm/i915: Enhanced disable access to stolen memory as a guest (rev4) Patchwork ` (3 more replies) 0 siblings, 4 replies; 20+ messages in thread From: Xiong Zhang @ 2017-04-05 2:08 UTC (permalink / raw) To: daniel; +Cc: intel-gfx, intel-gvt-dev, stable, Xiong Zhang Stolen memory is in RMRR and has identity mapping in iommu, so gt could access stolen memory in host OS. While RMRR isn't supported by kvm, both EPT and guest iommu domain lack of maaping for stolen memory, so both vcpu and gt couldn't access stolen memory in guest. commit "04a68a3 drm/i915/gvt: Disable access to stolen memory as a guest" isn't enough in IGD passthrough environment where vgt code doesn't run. This patch detects i915 running as a guest through the existence of emulated ISA bridge. v2:GVT-g may run in non qemu (Zhenyu) v3:Make commit message clear (Daniel) v4:Fix typo References: c875d2c iommu/vt-d: Exclude devices using RMRRs from IOMMU API domains Link: https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> Cc: stable@vger.kernel.org --- drivers/gpu/drm/i915/i915_drv.c | 1 + drivers/gpu/drm/i915/i915_drv.h | 1 + drivers/gpu/drm/i915/i915_gem_stolen.c | 4 ++-- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 03d9e45..8b807a9 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -223,6 +223,7 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) PCI_SUBVENDOR_ID_REDHAT_QUMRANET && pch->subsystem_device == PCI_SUBDEVICE_ID_QEMU)) { + dev_priv->run_on_qemu = true; dev_priv->pch_type = intel_virt_detect_pch(dev_priv); } else diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index a5947a4..ad95c87 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -2145,6 +2145,7 @@ struct drm_i915_private { struct intel_uncore uncore; struct i915_virtual_gpu vgpu; + bool run_on_qemu; struct intel_gvt *gvt; diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c index f3abdc2..6a011b0 100644 --- a/drivers/gpu/drm/i915/i915_gem_stolen.c +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c @@ -409,8 +409,8 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv) mutex_init(&dev_priv->mm.stolen_lock); - if (intel_vgpu_active(dev_priv)) { - DRM_INFO("iGVT-g active, disabling use of stolen memory\n"); + if (dev_priv->run_on_qemu || intel_vgpu_active(dev_priv)) { + DRM_INFO("Running in guest, disabling use of stolen memory\n"); return 0; } -- 1.9.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* ✓ Fi.CI.BAT: success for drm/i915: Enhanced disable access to stolen memory as a guest (rev4) 2017-04-05 2:08 [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest Xiong Zhang @ 2017-04-05 2:19 ` Patchwork 2017-04-05 6:50 ` Daniel Vetter ` (2 subsequent siblings) 3 siblings, 0 replies; 20+ messages in thread From: Patchwork @ 2017-04-05 2:19 UTC (permalink / raw) To: Zhang, Xiong Y; +Cc: intel-gfx == Series Details == Series: drm/i915: Enhanced disable access to stolen memory as a guest (rev4) URL : https://patchwork.freedesktop.org/series/21818/ State : success == Summary == Series 21818v4 drm/i915: Enhanced disable access to stolen memory as a guest https://patchwork.freedesktop.org/api/1.0/series/21818/revisions/4/mbox/ Test gem_exec_flush: Subgroup basic-batch-kernel-default-uc: pass -> FAIL (fi-snb-2600) fdo#100007 Test gem_exec_suspend: Subgroup basic-s4-devices: dmesg-warn -> PASS (fi-kbl-7560u) fdo#100125 fdo#100007 https://bugs.freedesktop.org/show_bug.cgi?id=100007 fdo#100125 https://bugs.freedesktop.org/show_bug.cgi?id=100125 fi-bdw-5557u total:278 pass:267 dwarn:0 dfail:0 fail:0 skip:11 time: 429s fi-bdw-gvtdvm total:278 pass:256 dwarn:8 dfail:0 fail:0 skip:14 time: 423s fi-bsw-n3050 total:278 pass:239 dwarn:0 dfail:0 fail:0 skip:39 time: 576s fi-bxt-j4205 total:278 pass:259 dwarn:0 dfail:0 fail:0 skip:19 time: 509s fi-bxt-t5700 total:278 pass:258 dwarn:0 dfail:0 fail:0 skip:20 time: 529s fi-byt-j1900 total:278 pass:251 dwarn:0 dfail:0 fail:0 skip:27 time: 480s fi-byt-n2820 total:278 pass:247 dwarn:0 dfail:0 fail:0 skip:31 time: 481s fi-hsw-4770 total:278 pass:262 dwarn:0 dfail:0 fail:0 skip:16 time: 412s fi-hsw-4770r total:278 pass:262 dwarn:0 dfail:0 fail:0 skip:16 time: 410s fi-ilk-650 total:278 pass:228 dwarn:0 dfail:0 fail:0 skip:50 time: 423s fi-ivb-3520m total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time: 490s fi-ivb-3770 total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time: 469s fi-kbl-7500u total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time: 458s fi-kbl-7560u total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time: 568s fi-skl-6260u total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time: 451s fi-skl-6700hq total:278 pass:261 dwarn:0 dfail:0 fail:0 skip:17 time: 573s fi-skl-6700k total:278 pass:256 dwarn:4 dfail:0 fail:0 skip:18 time: 459s fi-skl-6770hq total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time: 492s fi-skl-gvtdvm total:278 pass:265 dwarn:0 dfail:0 fail:0 skip:13 time: 433s fi-snb-2520m total:278 pass:250 dwarn:0 dfail:0 fail:0 skip:28 time: 529s fi-snb-2600 total:278 pass:248 dwarn:0 dfail:0 fail:1 skip:29 time: 399s bf30bc2a70b83a77ba63436023f3550083715c56 drm-tip: 2017y-04m-04d-20h-00m-56s UTC integration manifest ab00c6af drm/i915: Enhanced disable access to stolen memory as a guest == Logs == For more details see: https://intel-gfx-ci.01.org/CI/Patchwork_4402/ _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest 2017-04-05 2:08 [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest Xiong Zhang @ 2017-04-05 6:50 ` Daniel Vetter 2017-04-05 6:50 ` Daniel Vetter ` (2 subsequent siblings) 3 siblings, 0 replies; 20+ messages in thread From: Daniel Vetter @ 2017-04-05 6:50 UTC (permalink / raw) To: Xiong Zhang; +Cc: daniel, intel-gfx, intel-gvt-dev, stable On Wed, Apr 05, 2017 at 10:08:26AM +0800, Xiong Zhang wrote: > Stolen memory is in RMRR and has identity mapping in iommu, so > gt could access stolen memory in host OS. While RMRR isn't supported > by kvm, both EPT and guest iommu domain lack of maaping for stolen memory, > so both vcpu and gt couldn't access stolen memory in guest. > > commit "04a68a3 drm/i915/gvt: Disable access to stolen memory as a guest" > isn't enough in IGD passthrough environment where vgt code doesn't run. This > patch detects i915 running as a guest through the existence of emulated ISA > bridge. > > v2:GVT-g may run in non qemu (Zhenyu) > v3:Make commit message clear (Daniel) > v4:Fix typo > > References: c875d2c iommu/vt-d: Exclude devices using RMRRs from IOMMU API domains > Link: https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 > > Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> > Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> > Cc: stable@vger.kernel.org Yeah, commit message is now much improved. Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > --- > drivers/gpu/drm/i915/i915_drv.c | 1 + > drivers/gpu/drm/i915/i915_drv.h | 1 + > drivers/gpu/drm/i915/i915_gem_stolen.c | 4 ++-- > 3 files changed, 4 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c > index 03d9e45..8b807a9 100644 > --- a/drivers/gpu/drm/i915/i915_drv.c > +++ b/drivers/gpu/drm/i915/i915_drv.c > @@ -223,6 +223,7 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) > PCI_SUBVENDOR_ID_REDHAT_QUMRANET && > pch->subsystem_device == > PCI_SUBDEVICE_ID_QEMU)) { > + dev_priv->run_on_qemu = true; > dev_priv->pch_type = > intel_virt_detect_pch(dev_priv); > } else > diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h > index a5947a4..ad95c87 100644 > --- a/drivers/gpu/drm/i915/i915_drv.h > +++ b/drivers/gpu/drm/i915/i915_drv.h > @@ -2145,6 +2145,7 @@ struct drm_i915_private { > struct intel_uncore uncore; > > struct i915_virtual_gpu vgpu; > + bool run_on_qemu; > > struct intel_gvt *gvt; > > diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c > index f3abdc2..6a011b0 100644 > --- a/drivers/gpu/drm/i915/i915_gem_stolen.c > +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c > @@ -409,8 +409,8 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv) > > mutex_init(&dev_priv->mm.stolen_lock); > > - if (intel_vgpu_active(dev_priv)) { > - DRM_INFO("iGVT-g active, disabling use of stolen memory\n"); > + if (dev_priv->run_on_qemu || intel_vgpu_active(dev_priv)) { > + DRM_INFO("Running in guest, disabling use of stolen memory\n"); > return 0; > } > > -- > 1.9.1 > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest @ 2017-04-05 6:50 ` Daniel Vetter 0 siblings, 0 replies; 20+ messages in thread From: Daniel Vetter @ 2017-04-05 6:50 UTC (permalink / raw) To: Xiong Zhang; +Cc: intel-gfx, intel-gvt-dev, stable On Wed, Apr 05, 2017 at 10:08:26AM +0800, Xiong Zhang wrote: > Stolen memory is in RMRR and has identity mapping in iommu, so > gt could access stolen memory in host OS. While RMRR isn't supported > by kvm, both EPT and guest iommu domain lack of maaping for stolen memory, > so both vcpu and gt couldn't access stolen memory in guest. > > commit "04a68a3 drm/i915/gvt: Disable access to stolen memory as a guest" > isn't enough in IGD passthrough environment where vgt code doesn't run. This > patch detects i915 running as a guest through the existence of emulated ISA > bridge. > > v2:GVT-g may run in non qemu (Zhenyu) > v3:Make commit message clear (Daniel) > v4:Fix typo > > References: c875d2c iommu/vt-d: Exclude devices using RMRRs from IOMMU API domains > Link: https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 > > Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> > Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> > Cc: stable@vger.kernel.org Yeah, commit message is now much improved. Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > --- > drivers/gpu/drm/i915/i915_drv.c | 1 + > drivers/gpu/drm/i915/i915_drv.h | 1 + > drivers/gpu/drm/i915/i915_gem_stolen.c | 4 ++-- > 3 files changed, 4 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c > index 03d9e45..8b807a9 100644 > --- a/drivers/gpu/drm/i915/i915_drv.c > +++ b/drivers/gpu/drm/i915/i915_drv.c > @@ -223,6 +223,7 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) > PCI_SUBVENDOR_ID_REDHAT_QUMRANET && > pch->subsystem_device == > PCI_SUBDEVICE_ID_QEMU)) { > + dev_priv->run_on_qemu = true; > dev_priv->pch_type = > intel_virt_detect_pch(dev_priv); > } else > diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h > index a5947a4..ad95c87 100644 > --- a/drivers/gpu/drm/i915/i915_drv.h > +++ b/drivers/gpu/drm/i915/i915_drv.h > @@ -2145,6 +2145,7 @@ struct drm_i915_private { > struct intel_uncore uncore; > > struct i915_virtual_gpu vgpu; > + bool run_on_qemu; > > struct intel_gvt *gvt; > > diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c > index f3abdc2..6a011b0 100644 > --- a/drivers/gpu/drm/i915/i915_gem_stolen.c > +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c > @@ -409,8 +409,8 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv) > > mutex_init(&dev_priv->mm.stolen_lock); > > - if (intel_vgpu_active(dev_priv)) { > - DRM_INFO("iGVT-g active, disabling use of stolen memory\n"); > + if (dev_priv->run_on_qemu || intel_vgpu_active(dev_priv)) { > + DRM_INFO("Running in guest, disabling use of stolen memory\n"); > return 0; > } > > -- > 1.9.1 > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Intel-gfx] [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest 2017-04-05 6:50 ` Daniel Vetter (?) @ 2017-04-05 7:44 ` Jani Nikula -1 siblings, 0 replies; 20+ messages in thread From: Jani Nikula @ 2017-04-05 7:44 UTC (permalink / raw) To: Daniel Vetter, Xiong Zhang; +Cc: intel-gfx, intel-gvt-dev, stable On Wed, 05 Apr 2017, Daniel Vetter <daniel@ffwll.ch> wrote: > On Wed, Apr 05, 2017 at 10:08:26AM +0800, Xiong Zhang wrote: >> Stolen memory is in RMRR and has identity mapping in iommu, so >> gt could access stolen memory in host OS. While RMRR isn't supported >> by kvm, both EPT and guest iommu domain lack of maaping for stolen memory, >> so both vcpu and gt couldn't access stolen memory in guest. >> >> commit "04a68a3 drm/i915/gvt: Disable access to stolen memory as a guest" >> isn't enough in IGD passthrough environment where vgt code doesn't run. This >> patch detects i915 running as a guest through the existence of emulated ISA >> bridge. >> >> v2:GVT-g may run in non qemu (Zhenyu) >> v3:Make commit message clear (Daniel) >> v4:Fix typo >> >> References: c875d2c iommu/vt-d: Exclude devices using RMRRs from IOMMU API domains What does it mean to References: a commit? The message text doesn't say anything about this commit. Does this patch fix commit 04a68a3 or c875d2c? If so, there should be a Fixes: tag. The canonical format for commit references is 04a68a35ce6d ("drm/i915/gvt: Disable access to stolen memory as a guest") c875d2c1b808 ("iommu/vt-d: Exclude devices using RMRRs from IOMMU API domains") In particular, note that 12 digits is preferred for sha1 references to avoid ambiguity in the kernel git repo. >> Link: https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf Please use References: for stuff like this. Link: is reserved for the backlink to the patch and discussion. >> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 >> >> Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> >> Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> >> Cc: stable@vger.kernel.org With a Fixes: tag it would be possible to decide which stable kernels to backport to. > Yeah, commit message is now much improved. I'm sorry, but from the fixes backports POV it still lacks clarity. I wouldn't know if and where this should be backported. BR, Jani. > > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > >> --- >> drivers/gpu/drm/i915/i915_drv.c | 1 + >> drivers/gpu/drm/i915/i915_drv.h | 1 + >> drivers/gpu/drm/i915/i915_gem_stolen.c | 4 ++-- >> 3 files changed, 4 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c >> index 03d9e45..8b807a9 100644 >> --- a/drivers/gpu/drm/i915/i915_drv.c >> +++ b/drivers/gpu/drm/i915/i915_drv.c >> @@ -223,6 +223,7 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) >> PCI_SUBVENDOR_ID_REDHAT_QUMRANET && >> pch->subsystem_device == >> PCI_SUBDEVICE_ID_QEMU)) { >> + dev_priv->run_on_qemu = true; >> dev_priv->pch_type = >> intel_virt_detect_pch(dev_priv); >> } else >> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h >> index a5947a4..ad95c87 100644 >> --- a/drivers/gpu/drm/i915/i915_drv.h >> +++ b/drivers/gpu/drm/i915/i915_drv.h >> @@ -2145,6 +2145,7 @@ struct drm_i915_private { >> struct intel_uncore uncore; >> >> struct i915_virtual_gpu vgpu; >> + bool run_on_qemu; >> >> struct intel_gvt *gvt; >> >> diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c >> index f3abdc2..6a011b0 100644 >> --- a/drivers/gpu/drm/i915/i915_gem_stolen.c >> +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c >> @@ -409,8 +409,8 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv) >> >> mutex_init(&dev_priv->mm.stolen_lock); >> >> - if (intel_vgpu_active(dev_priv)) { >> - DRM_INFO("iGVT-g active, disabling use of stolen memory\n"); >> + if (dev_priv->run_on_qemu || intel_vgpu_active(dev_priv)) { >> + DRM_INFO("Running in guest, disabling use of stolen memory\n"); >> return 0; >> } >> >> -- >> 1.9.1 >> -- Jani Nikula, Intel Open Source Technology Center ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-05 2:08 [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest Xiong Zhang @ 2017-04-12 12:20 ` Xiong Zhang 2017-04-05 6:50 ` Daniel Vetter ` (2 subsequent siblings) 3 siblings, 0 replies; 20+ messages in thread From: Xiong Zhang @ 2017-04-12 12:20 UTC (permalink / raw) To: joonas.lahtinen, daniel, zhenyuw, jani.nikula Cc: intel-gfx, intel-gvt-dev, stable, Xiong Zhang Stolen memory isn't a standard pci resource and exists in RMRR which has identity mapping in iommu table, IGD could access stolen memory in host OS. While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices using RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then both EPT and guest iommu domain table lack of maaping for stolen memory in kvm IGD passthrough environment. If IGD access stolen memory in such environment, many iommu exceptions exist in host dmesg and gpu hang exists also. DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 [fault reason 05] PTE Write access is not set DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 [fault reason 06] PTE Read access is not set So stolen memory should be disabled in KVM IGD passthrough environment, this patch detects such environment through the existence of qemu emulated isa bridge. When the real ISA bridge is also passed through to guest, guest will have two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. funcnum of emulated isa bridge is always less than the real one. Then emulated isa bridge is always detected first by pci_get_class(ISA). So stolen memory will be disabled in this case also. Stolen memory exists in kernel for a long time, but this patch depends on INTEL_PCH_QEMU_DEVICE_ID_TYPE which was introduced in v4.5 kernel, so this patch should be backported into v4.5 kernel and above. v2:GVT-g may run in non qemu (Zhenyu) v3:Make commit message clear (Daniel) v4:Fix typo v5:Exclude P2X as it is used for VMware (Joonas) Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: stable@vger.kernel.org --- drivers/gpu/drm/i915/i915_drv.c | 5 +++++ drivers/gpu/drm/i915/i915_drv.h | 1 + drivers/gpu/drm/i915/i915_gem_stolen.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 6d9944a..0d3c395 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -223,6 +223,11 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) PCI_SUBVENDOR_ID_REDHAT_QUMRANET && pch->subsystem_device == PCI_SUBDEVICE_ID_QEMU)) { + /* + * P2X is used for VMware, exclude it + */ + if (id != INTEL_PCH_P2X_DEVICE_ID_TYPE) + dev_priv->run_on_qemu = true; dev_priv->pch_type = intel_virt_detect_pch(dev_priv); } else diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index 2911c49..c87150e 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -2152,6 +2152,7 @@ struct drm_i915_private { struct intel_uncore uncore; struct i915_virtual_gpu vgpu; + bool run_on_qemu; struct intel_gvt *gvt; diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c index f3abdc2..6a011b0 100644 --- a/drivers/gpu/drm/i915/i915_gem_stolen.c +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c @@ -409,8 +409,8 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv) mutex_init(&dev_priv->mm.stolen_lock); - if (intel_vgpu_active(dev_priv)) { - DRM_INFO("iGVT-g active, disabling use of stolen memory\n"); + if (dev_priv->run_on_qemu || intel_vgpu_active(dev_priv)) { + DRM_INFO("Running in guest, disabling use of stolen memory\n"); return 0; } -- 2.9.3 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu @ 2017-04-12 12:20 ` Xiong Zhang 0 siblings, 0 replies; 20+ messages in thread From: Xiong Zhang @ 2017-04-12 12:20 UTC (permalink / raw) To: joonas.lahtinen, daniel, zhenyuw, jani.nikula Cc: intel-gfx, intel-gvt-dev, stable Stolen memory isn't a standard pci resource and exists in RMRR which has identity mapping in iommu table, IGD could access stolen memory in host OS. While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices using RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then both EPT and guest iommu domain table lack of maaping for stolen memory in kvm IGD passthrough environment. If IGD access stolen memory in such environment, many iommu exceptions exist in host dmesg and gpu hang exists also. DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 [fault reason 05] PTE Write access is not set DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 [fault reason 06] PTE Read access is not set So stolen memory should be disabled in KVM IGD passthrough environment, this patch detects such environment through the existence of qemu emulated isa bridge. When the real ISA bridge is also passed through to guest, guest will have two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. funcnum of emulated isa bridge is always less than the real one. Then emulated isa bridge is always detected first by pci_get_class(ISA). So stolen memory will be disabled in this case also. Stolen memory exists in kernel for a long time, but this patch depends on INTEL_PCH_QEMU_DEVICE_ID_TYPE which was introduced in v4.5 kernel, so this patch should be backported into v4.5 kernel and above. v2:GVT-g may run in non qemu (Zhenyu) v3:Make commit message clear (Daniel) v4:Fix typo v5:Exclude P2X as it is used for VMware (Joonas) Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: stable@vger.kernel.org --- drivers/gpu/drm/i915/i915_drv.c | 5 +++++ drivers/gpu/drm/i915/i915_drv.h | 1 + drivers/gpu/drm/i915/i915_gem_stolen.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 6d9944a..0d3c395 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -223,6 +223,11 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) PCI_SUBVENDOR_ID_REDHAT_QUMRANET && pch->subsystem_device == PCI_SUBDEVICE_ID_QEMU)) { + /* + * P2X is used for VMware, exclude it + */ + if (id != INTEL_PCH_P2X_DEVICE_ID_TYPE) + dev_priv->run_on_qemu = true; dev_priv->pch_type = intel_virt_detect_pch(dev_priv); } else diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index 2911c49..c87150e 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -2152,6 +2152,7 @@ struct drm_i915_private { struct intel_uncore uncore; struct i915_virtual_gpu vgpu; + bool run_on_qemu; struct intel_gvt *gvt; diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c index f3abdc2..6a011b0 100644 --- a/drivers/gpu/drm/i915/i915_gem_stolen.c +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c @@ -409,8 +409,8 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv) mutex_init(&dev_priv->mm.stolen_lock); - if (intel_vgpu_active(dev_priv)) { - DRM_INFO("iGVT-g active, disabling use of stolen memory\n"); + if (dev_priv->run_on_qemu || intel_vgpu_active(dev_priv)) { + DRM_INFO("Running in guest, disabling use of stolen memory\n"); return 0; } -- 2.9.3 _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-12 12:20 ` Xiong Zhang (?) @ 2017-04-12 13:21 ` Joonas Lahtinen 2017-04-13 4:15 ` Zhang, Xiong Y 2017-04-13 7:23 ` Tian, Kevin -1 siblings, 2 replies; 20+ messages in thread From: Joonas Lahtinen @ 2017-04-12 13:21 UTC (permalink / raw) To: Xiong Zhang, daniel, zhenyuw, jani.nikula, Tian, Kevin, David Woodhouse Cc: intel-gfx, intel-gvt-dev, stable, iommu + Kevin and David On ke, 2017-04-12 at 20:20 +0800, Xiong Zhang wrote: > Stolen memory isn't a standard pci resource and exists in RMRR which has > identity mapping in iommu table, IGD could access stolen memory in host OS. > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices using > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then both EPT > and guest iommu domain table lack of maaping for stolen memory in kvm IGD > passthrough environment. If IGD access stolen memory in such environment, > many iommu exceptions exist in host dmesg and gpu hang exists also. > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > [fault reason 05] PTE Write access is not set > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > [fault reason 06] PTE Read access is not set > > So stolen memory should be disabled in KVM IGD passthrough environment, > this patch detects such environment through the existence of qemu emulated > isa bridge. > > When the real ISA bridge is also passed through to guest, guest will have > two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. > funcnum of emulated isa bridge is always less than the real one. Then > emulated isa bridge is always detected first by pci_get_class(ISA). So > stolen memory will be disabled in this case also. > > Stolen memory exists in kernel for a long time, but this patch depends > on INTEL_PCH_QEMU_DEVICE_ID_TYPE which was introduced in v4.5 kernel, > so this patch should be backported into v4.5 kernel and above. > > v2:GVT-g may run in non qemu (Zhenyu) > v3:Make commit message clear (Daniel) > v4:Fix typo > v5:Exclude P2X as it is used for VMware (Joonas) > > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 > > Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> > Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > Cc: stable@vger.kernel.org The commit message still fails to address the fact that the Bugzilla entry has a completely bogus bisect, the fact that there is a later commit that allows RMRRs on graphics devices; commit 18436afdc11a00ac881990b454cfb2eae81d6003 Author: David Woodhouse <David.Woodhouse@intel.com> Date: Wed Mar 25 15:05:47 2015 +0000 iommu/vt-d: Allow RMRR on graphics devices too And the fact that GuC status is still not answered even I explicitly asked for it. By my limited understanding of VT-d details: The stolen memory is never directly accessed by i915 driver (because CPU access doesn't work even in DOM0). It is only used through the aperture, which just requires for the GT device to have access to the RMRR. Further, the GT device needs to have access to stolen memory, because that's what GuC uses for backing storage for for WOPCM. And even if after all of the above is addressed, shouldn't we rather try to detect the lack of RMRR, than presence of QEMU ISA? What comes to my mind is exporting function like device_has_rmrr() from intel-iommu.com and consuming that, if we end up doing this. That way, if somebody, some day, goes and write RMRR pass-through code currently missing, it'll start working, just like it should. Regards, Joonas -- Joonas Lahtinen Open Source Technology Center Intel Corporation ^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-12 13:21 ` Joonas Lahtinen @ 2017-04-13 4:15 ` Zhang, Xiong Y 2017-04-13 7:23 ` Tian, Kevin 1 sibling, 0 replies; 20+ messages in thread From: Zhang, Xiong Y @ 2017-04-13 4:15 UTC (permalink / raw) To: Joonas Lahtinen, daniel, zhenyuw, jani.nikula, Tian, Kevin, David Woodhouse Cc: intel-gfx, intel-gvt-dev, stable, iommu, Zhang, Xiong Y > + Kevin and David > > On ke, 2017-04-12 at 20:20 +0800, Xiong Zhang wrote: > > Stolen memory isn't a standard pci resource and exists in RMRR which has > > identity mapping in iommu table, IGD could access stolen memory in host > OS. > > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices > using > > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then > both EPT > > and guest iommu domain table lack of maaping for stolen memory in kvm > IGD > > passthrough environment. If IGD access stolen memory in such environment, > > many iommu exceptions exist in host dmesg and gpu hang exists also. > > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > > [fault reason 05] PTE Write access is not set > > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > > [fault reason 06] PTE Read access is not set > > > > So stolen memory should be disabled in KVM IGD passthrough environment, > > this patch detects such environment through the existence of qemu > emulated > > isa bridge. > > > > When the real ISA bridge is also passed through to guest, guest will have > > two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. > > funcnum of emulated isa bridge is always less than the real one. Then > > emulated isa bridge is always detected first by pci_get_class(ISA). So > > stolen memory will be disabled in this case also. > > > > Stolen memory exists in kernel for a long time, but this patch depends > > on INTEL_PCH_QEMU_DEVICE_ID_TYPE which was introduced in v4.5 kernel, > > so this patch should be backported into v4.5 kernel and above. > > > > v2:GVT-g may run in non qemu (Zhenyu) > > v3:Make commit message clear (Daniel) > > v4:Fix typo > > v5:Exclude P2X as it is used for VMware (Joonas) > > > > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 > > > > Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> > > Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> > > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > > Cc: stable@vger.kernel.org > > The commit message still fails to address the fact that the Bugzilla > entry has a completely bogus bisect, the fact that there is a later > commit that allows RMRRs on graphics devices; [Zhang, Xiong Y] Indeed when I boot kernel 3.18, gpu hang don't happen during boot process, but IOMMU DMA R/W to stolen memory exception still exist in host dmesg. When I boot kernel 3.19 and above, I see DMA R/W exception in host dmesg and gpu hang. I'm lack of the knowledge to analyze the gpu hang error. And I have updated the error into bugzilla, could you help check whether this hang is caused by GT accessing to stolen memory or not? https://bugs.freedesktop.org/show_bug.cgi?id=99028 and https://bugs.freedesktop.org/show_bug.cgi?id=99025 are the same issue which could be fixed by disable stolen memory. But bisect result are different, so I think the first bad commit of git bisect isn't accurate. > > commit 18436afdc11a00ac881990b454cfb2eae81d6003 > Author: David Woodhouse <David.Woodhouse@intel.com> > Date: Wed Mar 25 15:05:47 2015 +0000 > > iommu/vt-d: Allow RMRR on graphics devices too > [Zhang, Xiong Y] 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices Using RMRRs from IOMMU API domains")', this commit prevent devices associated with RMRR from passing through to guest. 'commit 18436afdc11a ("iommu/vt-d: Allow RMRR on graphics devices too")', this commit add an exception for graphics device to above commit. So that IGD could be assigned (pass through) to guest. Hi, David: The following message exists in your 18436afdc11a commit message: "Add an exclusion for graphics devices too, so that 'iommu=pt' works there. We should be able to successfully assign graphics devices to guests too, as long as the initial handling of stolen memory is reconfigured appropriately. This has certainly worked in the past." What's the mean of "initial handling of stolen memory is reconfigured appropriately" ? we meet guest IGD accessing stolen memory issue. > And the fact that GuC status is still not answered even I explicitly > asked for it. [Zhang, Xiong Y] GuC accessing to stolen memory bypass VT-d, Kevin has confirmed this with VPG. I add i915.enable_guc_loading=1 and i915.enable_guc_submission=1 option, and the dmesg demonstrate guc works when stolen memory is disabled. [ 5.265653] [drm:intel_uc_prepare_fw [i915]] fetch uC fw from i915/skl_guc_ver6_1.bin succeeded, fw ffff9167bc93c1e0 [ 5.265668] [drm:intel_uc_prepare_fw [i915]] firmware version 6.1 OK (minimum 6.1) [ 5.265709] [drm:intel_uc_prepare_fw [i915]] uC fw fetch status SUCCESS, obj ffff91675ce88600 [ 5.267493] [drm:intel_guc_init_hw [i915]] GuC fw status: path i915/skl_guc_ver6_1.bin, fetch SUCCESS, load NONE [ 5.267508] [drm:intel_guc_init_hw [i915]] GuC fw status: fetch SUCCESS, load PENDING [ 5.271571] [drm:guc_ucode_xfer_dma [i915]] DMA status 0x10, GuC status 0x8002f0ec [ 5.271586] [drm:guc_ucode_xfer_dma [i915]] returning 0 [ 5.271587] [drm] GuC submission enabled (firmware i915/skl_guc_ver6_1.bin [version 6.1]) [ 5.272232] [drm:i915_guc_submission_enable [i915]] reserved cacheline 0x0, next 0x40, linesize 64 [ 5.272248] [drm:i915_guc_submission_enable [i915]] Host engines 0x17 => GuC engines used 0xf [ 5.272263] [drm:__reserve_doorbell [i915]] client 0 (high prio=no) reserved doorbell: 0 [ 5.274352] [drm:i915_guc_submission_enable [i915]] new priority 2 client ffff91675cc56d80 for engine(s) 0x17: stage_id 0 [ 5.274389] [drm:i915_guc_submission_enable [i915]] doorbell id 0, cacheline offset 0x0> > By my limited understanding of VT-d details: The stolen memory is never > directly accessed by i915 driver (because CPU access doesn't work even > in DOM0). It is only used through the aperture, which just requires for > the GT device to have access to the RMRR. Further, the GT device needs > to have access to stolen memory, because that's what GuC uses for > backing storage for for WOPCM. > > And even if after all of the above is addressed, shouldn't we rather > try to detect the lack of RMRR, than presence of QEMU ISA? [Zhang, Xiong Y] Good idea. Devices know I need RMRR, but on a native machine, RMRR need bios support which allocate and reserve memory range for RMRR. So RMRR need hypervisor's help in emulated environment. Only hypervisor knows whether it support RMRR or not. In order to detect the lack of RMRR in guest, i915 driver need to detect hypervisor. So in my last mail, I try to use cupid(40000001) to detect hypervisor. Zhenyu think it is unacceptable to use cupid(40000001) in a UPT(universal pass through) driver. > > What comes to my mind is exporting function like device_has_rmrr() from > intel-iommu.com and consuming that, if we end up doing this. That way, > if somebody, some day, goes and write RMRR pass-through code currently > missing, it'll start working, just like it should. [Zhang, Xiong Y] I also want to implement RMRR pass-through code at first. But this solution is denied in my team's meeting. As kvm/qemu community discussed this before and came to a solution: https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf > > Regards, Joonas > -- > Joonas Lahtinen > Open Source Technology Center > Intel Corporation ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu @ 2017-04-13 4:15 ` Zhang, Xiong Y 0 siblings, 0 replies; 20+ messages in thread From: Zhang, Xiong Y @ 2017-04-13 4:15 UTC (permalink / raw) To: Joonas Lahtinen, daniel, zhenyuw, jani.nikula, Tian, Kevin, David Woodhouse Cc: intel-gfx, intel-gvt-dev, iommu, stable > + Kevin and David > > On ke, 2017-04-12 at 20:20 +0800, Xiong Zhang wrote: > > Stolen memory isn't a standard pci resource and exists in RMRR which has > > identity mapping in iommu table, IGD could access stolen memory in host > OS. > > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices > using > > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then > both EPT > > and guest iommu domain table lack of maaping for stolen memory in kvm > IGD > > passthrough environment. If IGD access stolen memory in such environment, > > many iommu exceptions exist in host dmesg and gpu hang exists also. > > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > > [fault reason 05] PTE Write access is not set > > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > > [fault reason 06] PTE Read access is not set > > > > So stolen memory should be disabled in KVM IGD passthrough environment, > > this patch detects such environment through the existence of qemu > emulated > > isa bridge. > > > > When the real ISA bridge is also passed through to guest, guest will have > > two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. > > funcnum of emulated isa bridge is always less than the real one. Then > > emulated isa bridge is always detected first by pci_get_class(ISA). So > > stolen memory will be disabled in this case also. > > > > Stolen memory exists in kernel for a long time, but this patch depends > > on INTEL_PCH_QEMU_DEVICE_ID_TYPE which was introduced in v4.5 kernel, > > so this patch should be backported into v4.5 kernel and above. > > > > v2:GVT-g may run in non qemu (Zhenyu) > > v3:Make commit message clear (Daniel) > > v4:Fix typo > > v5:Exclude P2X as it is used for VMware (Joonas) > > > > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 > > > > Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> > > Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> > > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > > Cc: stable@vger.kernel.org > > The commit message still fails to address the fact that the Bugzilla > entry has a completely bogus bisect, the fact that there is a later > commit that allows RMRRs on graphics devices; [Zhang, Xiong Y] Indeed when I boot kernel 3.18, gpu hang don't happen during boot process, but IOMMU DMA R/W to stolen memory exception still exist in host dmesg. When I boot kernel 3.19 and above, I see DMA R/W exception in host dmesg and gpu hang. I'm lack of the knowledge to analyze the gpu hang error. And I have updated the error into bugzilla, could you help check whether this hang is caused by GT accessing to stolen memory or not? https://bugs.freedesktop.org/show_bug.cgi?id=99028 and https://bugs.freedesktop.org/show_bug.cgi?id=99025 are the same issue which could be fixed by disable stolen memory. But bisect result are different, so I think the first bad commit of git bisect isn't accurate. > > commit 18436afdc11a00ac881990b454cfb2eae81d6003 > Author: David Woodhouse <David.Woodhouse@intel.com> > Date: Wed Mar 25 15:05:47 2015 +0000 > > iommu/vt-d: Allow RMRR on graphics devices too > [Zhang, Xiong Y] 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices Using RMRRs from IOMMU API domains")', this commit prevent devices associated with RMRR from passing through to guest. 'commit 18436afdc11a ("iommu/vt-d: Allow RMRR on graphics devices too")', this commit add an exception for graphics device to above commit. So that IGD could be assigned (pass through) to guest. Hi, David: The following message exists in your 18436afdc11a commit message: "Add an exclusion for graphics devices too, so that 'iommu=pt' works there. We should be able to successfully assign graphics devices to guests too, as long as the initial handling of stolen memory is reconfigured appropriately. This has certainly worked in the past." What's the mean of "initial handling of stolen memory is reconfigured appropriately" ? we meet guest IGD accessing stolen memory issue. > And the fact that GuC status is still not answered even I explicitly > asked for it. [Zhang, Xiong Y] GuC accessing to stolen memory bypass VT-d, Kevin has confirmed this with VPG. I add i915.enable_guc_loading=1 and i915.enable_guc_submission=1 option, and the dmesg demonstrate guc works when stolen memory is disabled. [ 5.265653] [drm:intel_uc_prepare_fw [i915]] fetch uC fw from i915/skl_guc_ver6_1.bin succeeded, fw ffff9167bc93c1e0 [ 5.265668] [drm:intel_uc_prepare_fw [i915]] firmware version 6.1 OK (minimum 6.1) [ 5.265709] [drm:intel_uc_prepare_fw [i915]] uC fw fetch status SUCCESS, obj ffff91675ce88600 [ 5.267493] [drm:intel_guc_init_hw [i915]] GuC fw status: path i915/skl_guc_ver6_1.bin, fetch SUCCESS, load NONE [ 5.267508] [drm:intel_guc_init_hw [i915]] GuC fw status: fetch SUCCESS, load PENDING [ 5.271571] [drm:guc_ucode_xfer_dma [i915]] DMA status 0x10, GuC status 0x8002f0ec [ 5.271586] [drm:guc_ucode_xfer_dma [i915]] returning 0 [ 5.271587] [drm] GuC submission enabled (firmware i915/skl_guc_ver6_1.bin [version 6.1]) [ 5.272232] [drm:i915_guc_submission_enable [i915]] reserved cacheline 0x0, next 0x40, linesize 64 [ 5.272248] [drm:i915_guc_submission_enable [i915]] Host engines 0x17 => GuC engines used 0xf [ 5.272263] [drm:__reserve_doorbell [i915]] client 0 (high prio=no) reserved doorbell: 0 [ 5.274352] [drm:i915_guc_submission_enable [i915]] new priority 2 client ffff91675cc56d80 for engine(s) 0x17: stage_id 0 [ 5.274389] [drm:i915_guc_submission_enable [i915]] doorbell id 0, cacheline offset 0x0> > By my limited understanding of VT-d details: The stolen memory is never > directly accessed by i915 driver (because CPU access doesn't work even > in DOM0). It is only used through the aperture, which just requires for > the GT device to have access to the RMRR. Further, the GT device needs > to have access to stolen memory, because that's what GuC uses for > backing storage for for WOPCM. > > And even if after all of the above is addressed, shouldn't we rather > try to detect the lack of RMRR, than presence of QEMU ISA? [Zhang, Xiong Y] Good idea. Devices know I need RMRR, but on a native machine, RMRR need bios support which allocate and reserve memory range for RMRR. So RMRR need hypervisor's help in emulated environment. Only hypervisor knows whether it support RMRR or not. In order to detect the lack of RMRR in guest, i915 driver need to detect hypervisor. So in my last mail, I try to use cupid(40000001) to detect hypervisor. Zhenyu think it is unacceptable to use cupid(40000001) in a UPT(universal pass through) driver. > > What comes to my mind is exporting function like device_has_rmrr() from > intel-iommu.com and consuming that, if we end up doing this. That way, > if somebody, some day, goes and write RMRR pass-through code currently > missing, it'll start working, just like it should. [Zhang, Xiong Y] I also want to implement RMRR pass-through code at first. But this solution is denied in my team's meeting. As kvm/qemu community discussed this before and came to a solution: https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf > > Regards, Joonas > -- > Joonas Lahtinen > Open Source Technology Center > Intel Corporation _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-12 13:21 ` Joonas Lahtinen @ 2017-04-13 7:23 ` Tian, Kevin 2017-04-13 7:23 ` Tian, Kevin 1 sibling, 0 replies; 20+ messages in thread From: Tian, Kevin @ 2017-04-13 7:23 UTC (permalink / raw) To: Joonas Lahtinen, Zhang, Xiong Y, daniel, zhenyuw, jani.nikula, David Woodhouse Cc: intel-gfx, intel-gvt-dev, stable, iommu > From: Joonas Lahtinen [mailto:joonas.lahtinen@linux.intel.com] > Sent: Wednesday, April 12, 2017 9:22 PM > [...] > By my limited understanding of VT-d details: The stolen memory is never > directly accessed by i915 driver (because CPU access doesn't work even > in DOM0). It is only used through the aperture, which just requires for > the GT device to have access to the RMRR. Further, the GT device needs > to have access to stolen memory, because that's what GuC uses for > backing storage for for WOPCM. > > And even if after all of the above is addressed, shouldn't we rather > try to detect the lack of RMRR, than presence of QEMU ISA? > > What comes to my mind is exporting function like device_has_rmrr() from > intel-iommu.com and consuming that, if we end up doing this. That way, > if somebody, some day, goes and write RMRR pass-through code currently > missing, it'll start working, just like it should. > I like what you proposed in the long run, e.g. in a nested virtualization environment L0-VMM assigns the device to L1-VMM which further wants to assign device to L2-VM. In such case RMRR information must be propagated through the path to L1-VMM. However I can see one limitation here on your proposal. There is no RMRR if VT-d is disabled in BIOS. Then you cannot use stolen memory even on bare metal in such configuration, which is possibly not desired. Also the long term direction is to move away from RMRR for Intel integrated devices. People realized its limitation (especially the objection from KVM community. I don't think RMRR passthrough would be an option there). So I'd be with Xiong's simple workaround here. :-) Thanks Kevin ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu @ 2017-04-13 7:23 ` Tian, Kevin 0 siblings, 0 replies; 20+ messages in thread From: Tian, Kevin @ 2017-04-13 7:23 UTC (permalink / raw) To: Joonas Lahtinen, Zhang, Xiong Y, daniel, zhenyuw, jani.nikula, David Woodhouse Cc: intel-gfx, intel-gvt-dev, iommu, stable > From: Joonas Lahtinen [mailto:joonas.lahtinen@linux.intel.com] > Sent: Wednesday, April 12, 2017 9:22 PM > [...] > By my limited understanding of VT-d details: The stolen memory is never > directly accessed by i915 driver (because CPU access doesn't work even > in DOM0). It is only used through the aperture, which just requires for > the GT device to have access to the RMRR. Further, the GT device needs > to have access to stolen memory, because that's what GuC uses for > backing storage for for WOPCM. > > And even if after all of the above is addressed, shouldn't we rather > try to detect the lack of RMRR, than presence of QEMU ISA? > > What comes to my mind is exporting function like device_has_rmrr() from > intel-iommu.com and consuming that, if we end up doing this. That way, > if somebody, some day, goes and write RMRR pass-through code currently > missing, it'll start working, just like it should. > I like what you proposed in the long run, e.g. in a nested virtualization environment L0-VMM assigns the device to L1-VMM which further wants to assign device to L2-VM. In such case RMRR information must be propagated through the path to L1-VMM. However I can see one limitation here on your proposal. There is no RMRR if VT-d is disabled in BIOS. Then you cannot use stolen memory even on bare metal in such configuration, which is possibly not desired. Also the long term direction is to move away from RMRR for Intel integrated devices. People realized its limitation (especially the objection from KVM community. I don't think RMRR passthrough would be an option there). So I'd be with Xiong's simple workaround here. :-) Thanks Kevin _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Intel-gfx] [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-12 12:20 ` Xiong Zhang @ 2017-04-12 18:01 ` Alex Williamson -1 siblings, 0 replies; 20+ messages in thread From: Alex Williamson @ 2017-04-12 18:01 UTC (permalink / raw) To: Xiong Zhang Cc: joonas.lahtinen, daniel, zhenyuw, jani.nikula, intel-gfx, intel-gvt-dev, stable On Wed, 12 Apr 2017 20:20:00 +0800 Xiong Zhang <xiong.y.zhang@intel.com> wrote: > Stolen memory isn't a standard pci resource and exists in RMRR which has > identity mapping in iommu table, IGD could access stolen memory in host OS. > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices using > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then both EPT > and guest iommu domain table lack of maaping for stolen memory in kvm IGD > passthrough environment. If IGD access stolen memory in such environment, > many iommu exceptions exist in host dmesg and gpu hang exists also. > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > [fault reason 05] PTE Write access is not set > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > [fault reason 06] PTE Read access is not set > > So stolen memory should be disabled in KVM IGD passthrough environment, > this patch detects such environment through the existence of qemu emulated > isa bridge. > > When the real ISA bridge is also passed through to guest, guest will have > two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. > funcnum of emulated isa bridge is always less than the real one. Then > emulated isa bridge is always detected first by pci_get_class(ISA). So > stolen memory will be disabled in this case also. Where does QEMU make this guarantee or any sort of guarantee wrt the ISA bridge? Thanks, Alex > Stolen memory exists in kernel for a long time, but this patch depends > on INTEL_PCH_QEMU_DEVICE_ID_TYPE which was introduced in v4.5 kernel, > so this patch should be backported into v4.5 kernel and above. > > v2:GVT-g may run in non qemu (Zhenyu) > v3:Make commit message clear (Daniel) > v4:Fix typo > v5:Exclude P2X as it is used for VMware (Joonas) > > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 > > Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> > Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > Cc: stable@vger.kernel.org > --- > drivers/gpu/drm/i915/i915_drv.c | 5 +++++ > drivers/gpu/drm/i915/i915_drv.h | 1 + > drivers/gpu/drm/i915/i915_gem_stolen.c | 4 ++-- > 3 files changed, 8 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c > index 6d9944a..0d3c395 100644 > --- a/drivers/gpu/drm/i915/i915_drv.c > +++ b/drivers/gpu/drm/i915/i915_drv.c > @@ -223,6 +223,11 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) > PCI_SUBVENDOR_ID_REDHAT_QUMRANET && > pch->subsystem_device == > PCI_SUBDEVICE_ID_QEMU)) { > + /* > + * P2X is used for VMware, exclude it > + */ > + if (id != INTEL_PCH_P2X_DEVICE_ID_TYPE) > + dev_priv->run_on_qemu = true; > dev_priv->pch_type = > intel_virt_detect_pch(dev_priv); > } else > diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h > index 2911c49..c87150e 100644 > --- a/drivers/gpu/drm/i915/i915_drv.h > +++ b/drivers/gpu/drm/i915/i915_drv.h > @@ -2152,6 +2152,7 @@ struct drm_i915_private { > struct intel_uncore uncore; > > struct i915_virtual_gpu vgpu; > + bool run_on_qemu; > > struct intel_gvt *gvt; > > diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c > index f3abdc2..6a011b0 100644 > --- a/drivers/gpu/drm/i915/i915_gem_stolen.c > +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c > @@ -409,8 +409,8 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv) > > mutex_init(&dev_priv->mm.stolen_lock); > > - if (intel_vgpu_active(dev_priv)) { > - DRM_INFO("iGVT-g active, disabling use of stolen memory\n"); > + if (dev_priv->run_on_qemu || intel_vgpu_active(dev_priv)) { > + DRM_INFO("Running in guest, disabling use of stolen memory\n"); > return 0; > } > ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu @ 2017-04-12 18:01 ` Alex Williamson 0 siblings, 0 replies; 20+ messages in thread From: Alex Williamson @ 2017-04-12 18:01 UTC (permalink / raw) To: Xiong Zhang; +Cc: intel-gfx, stable, intel-gvt-dev On Wed, 12 Apr 2017 20:20:00 +0800 Xiong Zhang <xiong.y.zhang@intel.com> wrote: > Stolen memory isn't a standard pci resource and exists in RMRR which has > identity mapping in iommu table, IGD could access stolen memory in host OS. > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices using > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then both EPT > and guest iommu domain table lack of maaping for stolen memory in kvm IGD > passthrough environment. If IGD access stolen memory in such environment, > many iommu exceptions exist in host dmesg and gpu hang exists also. > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > [fault reason 05] PTE Write access is not set > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > [fault reason 06] PTE Read access is not set > > So stolen memory should be disabled in KVM IGD passthrough environment, > this patch detects such environment through the existence of qemu emulated > isa bridge. > > When the real ISA bridge is also passed through to guest, guest will have > two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. > funcnum of emulated isa bridge is always less than the real one. Then > emulated isa bridge is always detected first by pci_get_class(ISA). So > stolen memory will be disabled in this case also. Where does QEMU make this guarantee or any sort of guarantee wrt the ISA bridge? Thanks, Alex > Stolen memory exists in kernel for a long time, but this patch depends > on INTEL_PCH_QEMU_DEVICE_ID_TYPE which was introduced in v4.5 kernel, > so this patch should be backported into v4.5 kernel and above. > > v2:GVT-g may run in non qemu (Zhenyu) > v3:Make commit message clear (Daniel) > v4:Fix typo > v5:Exclude P2X as it is used for VMware (Joonas) > > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99028 > > Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com> > Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com> > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > Cc: stable@vger.kernel.org > --- > drivers/gpu/drm/i915/i915_drv.c | 5 +++++ > drivers/gpu/drm/i915/i915_drv.h | 1 + > drivers/gpu/drm/i915/i915_gem_stolen.c | 4 ++-- > 3 files changed, 8 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c > index 6d9944a..0d3c395 100644 > --- a/drivers/gpu/drm/i915/i915_drv.c > +++ b/drivers/gpu/drm/i915/i915_drv.c > @@ -223,6 +223,11 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) > PCI_SUBVENDOR_ID_REDHAT_QUMRANET && > pch->subsystem_device == > PCI_SUBDEVICE_ID_QEMU)) { > + /* > + * P2X is used for VMware, exclude it > + */ > + if (id != INTEL_PCH_P2X_DEVICE_ID_TYPE) > + dev_priv->run_on_qemu = true; > dev_priv->pch_type = > intel_virt_detect_pch(dev_priv); > } else > diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h > index 2911c49..c87150e 100644 > --- a/drivers/gpu/drm/i915/i915_drv.h > +++ b/drivers/gpu/drm/i915/i915_drv.h > @@ -2152,6 +2152,7 @@ struct drm_i915_private { > struct intel_uncore uncore; > > struct i915_virtual_gpu vgpu; > + bool run_on_qemu; > > struct intel_gvt *gvt; > > diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c > index f3abdc2..6a011b0 100644 > --- a/drivers/gpu/drm/i915/i915_gem_stolen.c > +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c > @@ -409,8 +409,8 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv) > > mutex_init(&dev_priv->mm.stolen_lock); > > - if (intel_vgpu_active(dev_priv)) { > - DRM_INFO("iGVT-g active, disabling use of stolen memory\n"); > + if (dev_priv->run_on_qemu || intel_vgpu_active(dev_priv)) { > + DRM_INFO("Running in guest, disabling use of stolen memory\n"); > return 0; > } > _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: [Intel-gfx] [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-12 18:01 ` Alex Williamson (?) @ 2017-04-13 5:44 ` Zhang, Xiong Y 2017-04-13 14:53 ` Alex Williamson -1 siblings, 1 reply; 20+ messages in thread From: Zhang, Xiong Y @ 2017-04-13 5:44 UTC (permalink / raw) To: Alex Williamson Cc: joonas.lahtinen, daniel, zhenyuw, jani.nikula, intel-gfx, intel-gvt-dev, stable, Zhang, Xiong Y > On Wed, 12 Apr 2017 20:20:00 +0800 > Xiong Zhang <xiong.y.zhang@intel.com> wrote: > > > Stolen memory isn't a standard pci resource and exists in RMRR which has > > identity mapping in iommu table, IGD could access stolen memory in host > OS. > > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices > using > > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then > both EPT > > and guest iommu domain table lack of maaping for stolen memory in kvm > IGD > > passthrough environment. If IGD access stolen memory in such environment, > > many iommu exceptions exist in host dmesg and gpu hang exists also. > > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > > [fault reason 05] PTE Write access is not set > > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > > [fault reason 06] PTE Read access is not set > > > > So stolen memory should be disabled in KVM IGD passthrough environment, > > this patch detects such environment through the existence of qemu > emulated > > isa bridge. > > > > When the real ISA bridge is also passed through to guest, guest will have > > two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. > > funcnum of emulated isa bridge is always less than the real one. Then > > emulated isa bridge is always detected first by pci_get_class(ISA). So > > stolen memory will be disabled in this case also. > > Where does QEMU make this guarantee or any sort of guarantee wrt the > ISA bridge? Thanks, > > Alex > [Zhang, Xiong Y] In my guest environment I always see emulated devices are at head of pci device list, the passed through devices are at tail. Even if I want to assign the passed IGD to 00:02.0, the qemu tell me 00:02.0 has already occupied by emulated graphic card. If I pass through real ISA bridge to guest, the emulated ISA bridge is at 00:01.0, While real ISA bridge is at 00:04.0. Then I checked the code: emulated devices are created in pc_init1() function, it creates host_bridge firstly, create isa_bridge secondly, create all other devices following. So I think Qemu could guarantee. Now I'm suspect it, and need your coach. thanks ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Intel-gfx] [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-13 5:44 ` [Intel-gfx] " Zhang, Xiong Y @ 2017-04-13 14:53 ` Alex Williamson 0 siblings, 0 replies; 20+ messages in thread From: Alex Williamson @ 2017-04-13 14:53 UTC (permalink / raw) To: Zhang, Xiong Y Cc: joonas.lahtinen, daniel, zhenyuw, jani.nikula, intel-gfx, intel-gvt-dev, stable On Thu, 13 Apr 2017 05:44:18 +0000 "Zhang, Xiong Y" <xiong.y.zhang@intel.com> wrote: > > On Wed, 12 Apr 2017 20:20:00 +0800 > > Xiong Zhang <xiong.y.zhang@intel.com> wrote: > > > > > Stolen memory isn't a standard pci resource and exists in RMRR which has > > > identity mapping in iommu table, IGD could access stolen memory in host > > OS. > > > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices > > using > > > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then > > both EPT > > > and guest iommu domain table lack of maaping for stolen memory in kvm > > IGD > > > passthrough environment. If IGD access stolen memory in such environment, > > > many iommu exceptions exist in host dmesg and gpu hang exists also. > > > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > > > [fault reason 05] PTE Write access is not set > > > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > > > [fault reason 06] PTE Read access is not set > > > > > > So stolen memory should be disabled in KVM IGD passthrough environment, > > > this patch detects such environment through the existence of qemu > > emulated > > > isa bridge. > > > > > > When the real ISA bridge is also passed through to guest, guest will have > > > two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. > > > funcnum of emulated isa bridge is always less than the real one. Then > > > emulated isa bridge is always detected first by pci_get_class(ISA). So > > > stolen memory will be disabled in this case also. > > > > Where does QEMU make this guarantee or any sort of guarantee wrt the > > ISA bridge? Thanks, > > > > Alex > > > [Zhang, Xiong Y] In my guest environment I always see emulated devices > are at head of pci device list, the passed through devices are at tail. Even if > I want to assign the passed IGD to 00:02.0, the qemu tell me 00:02.0 has already > occupied by emulated graphic card. > If I pass through real ISA bridge to guest, the emulated ISA bridge is at 00:01.0, > While real ISA bridge is at 00:04.0. > Then I checked the code: emulated devices are created in pc_init1() function, it > creates host_bridge firstly, create isa_bridge secondly, create all other devices following. > So I think Qemu could guarantee. Now I'm suspect it, and need your coach. So you're calling the current default behavior a guarantee. That's not valid, it ignores that we might have future chipsets that do things differently and it ignores that the user can override some of those defaults and specify addresses for devices that may not match your expectations. There is no agreement with the QEMU community to make this a stable feature of the VM. What about using smbios information or detecting kvm (or any hypervisor) via the hypervisor MSRs? You could maybe use the VM PCI host bridge and figure out that this version of IGD never shipped on 440fx or q35, but then you'll have the maintenance headache of updating the code for any new chipset QEMU decides to implement. Thanks, Alex ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu @ 2017-04-13 14:53 ` Alex Williamson 0 siblings, 0 replies; 20+ messages in thread From: Alex Williamson @ 2017-04-13 14:53 UTC (permalink / raw) To: Zhang, Xiong Y; +Cc: intel-gfx, stable, intel-gvt-dev On Thu, 13 Apr 2017 05:44:18 +0000 "Zhang, Xiong Y" <xiong.y.zhang@intel.com> wrote: > > On Wed, 12 Apr 2017 20:20:00 +0800 > > Xiong Zhang <xiong.y.zhang@intel.com> wrote: > > > > > Stolen memory isn't a standard pci resource and exists in RMRR which has > > > identity mapping in iommu table, IGD could access stolen memory in host > > OS. > > > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude devices > > using > > > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then > > both EPT > > > and guest iommu domain table lack of maaping for stolen memory in kvm > > IGD > > > passthrough environment. If IGD access stolen memory in such environment, > > > many iommu exceptions exist in host dmesg and gpu hang exists also. > > > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > > > [fault reason 05] PTE Write access is not set > > > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > > > [fault reason 06] PTE Read access is not set > > > > > > So stolen memory should be disabled in KVM IGD passthrough environment, > > > this patch detects such environment through the existence of qemu > > emulated > > > isa bridge. > > > > > > When the real ISA bridge is also passed through to guest, guest will have > > > two isa bridges: emulated and real. Qemu guarantees the busnum:devnum. > > > funcnum of emulated isa bridge is always less than the real one. Then > > > emulated isa bridge is always detected first by pci_get_class(ISA). So > > > stolen memory will be disabled in this case also. > > > > Where does QEMU make this guarantee or any sort of guarantee wrt the > > ISA bridge? Thanks, > > > > Alex > > > [Zhang, Xiong Y] In my guest environment I always see emulated devices > are at head of pci device list, the passed through devices are at tail. Even if > I want to assign the passed IGD to 00:02.0, the qemu tell me 00:02.0 has already > occupied by emulated graphic card. > If I pass through real ISA bridge to guest, the emulated ISA bridge is at 00:01.0, > While real ISA bridge is at 00:04.0. > Then I checked the code: emulated devices are created in pc_init1() function, it > creates host_bridge firstly, create isa_bridge secondly, create all other devices following. > So I think Qemu could guarantee. Now I'm suspect it, and need your coach. So you're calling the current default behavior a guarantee. That's not valid, it ignores that we might have future chipsets that do things differently and it ignores that the user can override some of those defaults and specify addresses for devices that may not match your expectations. There is no agreement with the QEMU community to make this a stable feature of the VM. What about using smbios information or detecting kvm (or any hypervisor) via the hypervisor MSRs? You could maybe use the VM PCI host bridge and figure out that this version of IGD never shipped on 440fx or q35, but then you'll have the maintenance headache of updating the code for any new chipset QEMU decides to implement. Thanks, Alex _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: [Intel-gfx] [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-13 14:53 ` Alex Williamson (?) @ 2017-04-14 6:39 ` Zhang, Xiong Y 2017-04-18 11:26 ` Gerd Hoffmann -1 siblings, 1 reply; 20+ messages in thread From: Zhang, Xiong Y @ 2017-04-14 6:39 UTC (permalink / raw) To: Alex Williamson Cc: joonas.lahtinen, daniel, zhenyuw, jani.nikula, intel-gfx, intel-gvt-dev, stable, Zhang, Xiong Y > On Thu, 13 Apr 2017 05:44:18 +0000 > "Zhang, Xiong Y" <xiong.y.zhang@intel.com> wrote: > > > > On Wed, 12 Apr 2017 20:20:00 +0800 > > > Xiong Zhang <xiong.y.zhang@intel.com> wrote: > > > > > > > Stolen memory isn't a standard pci resource and exists in RMRR which > has > > > > identity mapping in iommu table, IGD could access stolen memory in > host > > > OS. > > > > While according to 'commit c875d2c1b808 ("iommu/vt-d: Exclude > devices > > > using > > > > RMRRs from IOMMU API domains")',RMRR isn't supported by kvm, then > > > both EPT > > > > and guest iommu domain table lack of maaping for stolen memory in > kvm > > > IGD > > > > passthrough environment. If IGD access stolen memory in such > environment, > > > > many iommu exceptions exist in host dmesg and gpu hang exists also. > > > > DMAR: [DMA Read] Request device [00:02.0] fault addr da012000 > > > > [fault reason 05] PTE Write access is not set > > > > DMAR: [DMA Read] Request device [00:02.0] fault addr da2df000 > > > > [fault reason 06] PTE Read access is not set > > > > > > > > So stolen memory should be disabled in KVM IGD passthrough > environment, > > > > this patch detects such environment through the existence of qemu > > > emulated > > > > isa bridge. > > > > > > > > When the real ISA bridge is also passed through to guest, guest will have > > > > two isa bridges: emulated and real. Qemu guarantees the > busnum:devnum. > > > > funcnum of emulated isa bridge is always less than the real one. Then > > > > emulated isa bridge is always detected first by pci_get_class(ISA). So > > > > stolen memory will be disabled in this case also. > > > > > > Where does QEMU make this guarantee or any sort of guarantee wrt the > > > ISA bridge? Thanks, > > > > > > Alex > > > > > [Zhang, Xiong Y] In my guest environment I always see emulated devices > > are at head of pci device list, the passed through devices are at tail. Even if > > I want to assign the passed IGD to 00:02.0, the qemu tell me 00:02.0 has > already > > occupied by emulated graphic card. > > If I pass through real ISA bridge to guest, the emulated ISA bridge is at > 00:01.0, > > While real ISA bridge is at 00:04.0. > > Then I checked the code: emulated devices are created in pc_init1() function, > it > > creates host_bridge firstly, create isa_bridge secondly, create all other > devices following. > > So I think Qemu could guarantee. Now I'm suspect it, and need your coach. > > So you're calling the current default behavior a guarantee. That's not > valid, it ignores that we might have future chipsets that do things > differently and it ignores that the user can override some of those > defaults and specify addresses for devices that may not match your > expectations. There is no agreement with the QEMU community to make > this a stable feature of the VM. What about using smbios information > or detecting kvm (or any hypervisor) via the hypervisor MSRs? You could > maybe use the VM PCI host bridge and figure out that this version of > IGD never shipped on 440fx or q35, but then you'll have the maintenance > headache of updating the code for any new chipset QEMU decides to > implement. Thanks, > [Zhang, Xiong Y] Thanks for your teach and propose. For smbios, could you teach me which type and field could be used ? For hypervisor MSRs, from https://www.kernel.org/doc/Documentation/virtual/kvm/msr.txt, We should use cupid(0x40000001) first, then use rdmsr(), in this case we could use cupid(0x40000000) directly to detect kvm. But I don't know whether community could accept it or not ? > Alex ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Intel-gfx] [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu 2017-04-14 6:39 ` [Intel-gfx] " Zhang, Xiong Y @ 2017-04-18 11:26 ` Gerd Hoffmann 0 siblings, 0 replies; 20+ messages in thread From: Gerd Hoffmann @ 2017-04-18 11:26 UTC (permalink / raw) To: Zhang, Xiong Y Cc: Alex Williamson, intel-gfx, joonas.lahtinen, jani.nikula, zhenyuw, daniel, stable, intel-gvt-dev Hi, > [Zhang, Xiong Y] Thanks for your teach and propose. > For smbios, could you teach me which type and field could be used ? qemu adds a specific subsystem id to all virtual devices, so you can use that to figure you are running on qemu. One good candidate to check is the host bridge (easy to find due to fixed pci address), another one is the isa bridge aka lpc (igd already searches for that one for other reasons). In fact there already is a check for qemu in intel_detect_pch() ... cheers, Gerd ^ permalink raw reply [flat|nested] 20+ messages in thread
* ✓ Fi.CI.BAT: success for drm/i915: Enhanced disable access to stolen memory as a guest (rev5) 2017-04-05 2:08 [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest Xiong Zhang ` (2 preceding siblings ...) 2017-04-12 12:20 ` Xiong Zhang @ 2017-04-12 12:32 ` Patchwork 3 siblings, 0 replies; 20+ messages in thread From: Patchwork @ 2017-04-12 12:32 UTC (permalink / raw) To: Zhang, Xiong Y; +Cc: intel-gfx == Series Details == Series: drm/i915: Enhanced disable access to stolen memory as a guest (rev5) URL : https://patchwork.freedesktop.org/series/21818/ State : success == Summary == Series 21818v5 drm/i915: Enhanced disable access to stolen memory as a guest https://patchwork.freedesktop.org/api/1.0/series/21818/revisions/5/mbox/ fi-bdw-5557u total:278 pass:267 dwarn:0 dfail:0 fail:0 skip:11 time:435s fi-bdw-gvtdvm total:278 pass:256 dwarn:8 dfail:0 fail:0 skip:14 time:426s fi-bsw-n3050 total:278 pass:242 dwarn:0 dfail:0 fail:0 skip:36 time:574s fi-bxt-j4205 total:278 pass:259 dwarn:0 dfail:0 fail:0 skip:19 time:510s fi-byt-j1900 total:278 pass:254 dwarn:0 dfail:0 fail:0 skip:24 time:501s fi-byt-n2820 total:278 pass:250 dwarn:0 dfail:0 fail:0 skip:28 time:481s fi-hsw-4770 total:278 pass:262 dwarn:0 dfail:0 fail:0 skip:16 time:406s fi-hsw-4770r total:278 pass:262 dwarn:0 dfail:0 fail:0 skip:16 time:401s fi-ilk-650 total:278 pass:228 dwarn:0 dfail:0 fail:0 skip:50 time:413s fi-ivb-3520m total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time:493s fi-ivb-3770 total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time:468s fi-kbl-7500u total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time:457s fi-kbl-7560u total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time:573s fi-skl-6260u total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time:451s fi-skl-6700hq total:278 pass:261 dwarn:0 dfail:0 fail:0 skip:17 time:572s fi-skl-6700k total:278 pass:256 dwarn:4 dfail:0 fail:0 skip:18 time:454s fi-skl-6770hq total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time:486s fi-skl-gvtdvm total:278 pass:265 dwarn:0 dfail:0 fail:0 skip:13 time:432s fi-snb-2520m total:278 pass:250 dwarn:0 dfail:0 fail:0 skip:28 time:530s fi-snb-2600 total:278 pass:249 dwarn:0 dfail:0 fail:0 skip:29 time:400s ca184421c5d2f16386609bc1ea72038ad1a32843 drm-tip: 2017y-04m-12d-11h-42m-01s UTC integration manifest c58cd15 drm/i915: Disable stolen memory when i915 runs on qemu == Logs == For more details see: https://intel-gfx-ci.01.org/CI/Patchwork_4491/ _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2017-04-18 11:26 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-04-05 2:08 [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest Xiong Zhang 2017-04-05 2:19 ` ✓ Fi.CI.BAT: success for drm/i915: Enhanced disable access to stolen memory as a guest (rev4) Patchwork 2017-04-05 6:50 ` [PATCH V4] drm/i915: Enhanced disable access to stolen memory as a guest Daniel Vetter 2017-04-05 6:50 ` Daniel Vetter 2017-04-05 7:44 ` [Intel-gfx] " Jani Nikula 2017-04-12 12:20 ` [PATCH V5] drm/i915: Disable stolen memory when i915 runs on qemu Xiong Zhang 2017-04-12 12:20 ` Xiong Zhang 2017-04-12 13:21 ` Joonas Lahtinen 2017-04-13 4:15 ` Zhang, Xiong Y 2017-04-13 4:15 ` Zhang, Xiong Y 2017-04-13 7:23 ` Tian, Kevin 2017-04-13 7:23 ` Tian, Kevin 2017-04-12 18:01 ` [Intel-gfx] " Alex Williamson 2017-04-12 18:01 ` Alex Williamson 2017-04-13 5:44 ` [Intel-gfx] " Zhang, Xiong Y 2017-04-13 14:53 ` Alex Williamson 2017-04-13 14:53 ` Alex Williamson 2017-04-14 6:39 ` [Intel-gfx] " Zhang, Xiong Y 2017-04-18 11:26 ` Gerd Hoffmann 2017-04-12 12:32 ` ✓ Fi.CI.BAT: success for drm/i915: Enhanced disable access to stolen memory as a guest (rev5) Patchwork
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.